Deep Learning Based Topology Reconstruction Method for Digitalization of Image Format Piping and Instrumentation Diagram

Abstract

The present invention relates to a method for reconstructing the topology of an object for the digitization of a deep learning-based image-format pipe instrumentation diagram, comprising: an object connection relationship identification step for identifying connection relationships between a symbol, text, and line, which are recognized objects; a topological structure graph representation step for representing objects as nodes and displaying them on graph data, and adding edges connecting nodes when a connection relationship exists between nodes; an association relationship graph generation step consisting of an object recognition data preprocessing step and a first and second edge addition step; and a graph refinement step for performing additional processing on the graph.

Description

Technology Field

[0001] The present invention relates to a digitization technology for piping instrumentation drawings that automatically forms a digital drawing by recognizing symbols, lines, and text included in an original drawing in an image format through deep learning and image processing. More specifically, the invention relates to a method for reconstructing the topology of an object for the digitization of a deep learning-based image-format piping instrumentation drawing, comprising: an object connection relationship identification step that identifies the connection relationships between the recognized objects, which are symbols, text, and lines; a topological structure graph representation step that represents the object as a node and displays it on graph data, and then adds an edge connecting the node when a connection relationship exists between the nodes; an association relationship graph generation step consisting of an object recognition data preprocessing step and a first and second edge addition step; and a graph refinement step that performs additional processing on the graph. Background Technology

[0002] A Piping and Instrumentation Diagram (P&ID) represents the interconnections between process equipment, instruments used for process control, fluids, and control signal flows for the basic design of a plant in the process plant industry.

[0003] Piping instrumentation diagrams are digitized so that all objects in the diagram can be structured and processed by a computer, and are used as master data for detailed design, purchasing and procurement, construction and commissioning of plants together with 3D CAD (Computer Aided Design) models.

[0004] The main components of a digital piping instrumentation diagram are symbols, which are broadly classified into pipes, fittings, equipment, electric and measuring instruments, and off-page connectors (OPCs), and include lines connecting the symbols of the digital piping instrumentation diagram.

[0005] Lines are classified into piping lines and signal lines, and in addition to symbols and lines, outer borders, title boxes, characters, and tables are included in digital piping instrumentation diagrams.

[0006] While most EPC (Engineering, Procurement and Construction) companies responsible for the design, procurement, and construction of plants use digital piping instrumentation diagrams, piping instrumentation diagrams generated by FEED (front end engineering and design) companies or provided by equipment manufacturers for newly constructed plants are mainly in image format due to contractual relationships or intellectual property security issues.

[0007] Piping instrumentation diagrams are used for various purposes throughout plant construction projects, including design changes, inspections, the generation of reports related to equipment and piping, and the provision of information on plant operation and maintenance. However, unlike digitized piping instrumentation diagrams, image-based piping instrumentation diagrams do not contain attribute and specification information, which has a problem of significantly limited usability.

[0008] In particular, for plants that have been in operation for a long time at plant operators, a large volume of piping instrumentation diagrams is stored in image format. Furthermore, as image-formatted piping instrumentation diagrams are continuously received from external partners for the improvement and expansion of the plant during operation, the digitization of these image-formatted piping instrumentation diagrams is required.

[0009] Previously, the process of digitizing image-based piping instrumentation diagrams involved remodeling them through manual work by operators. This resulted in significant time consumption for generating image-based diagrams and had the disadvantage of varying quality, such as errors occurring depending on the operator's skill level.

[0010] Accordingly, Korean Registered Patent Publication No. 10-2361118 (registered on February 7, 2022) proposes a deep learning-based intelligent engineering drawing generation method and apparatus as a method for automatically generating and digitizing an intelligent engineering drawing from an image-formatted piping instrumentation drawing.

[0011] The above registered patent automatically digitizes a pipe instrumentation diagram in image format by using deep learning and image processing technologies to independently recognize symbols, text, and lines, which are objects expressed in the pipe instrumentation diagram, by including them in bounding boxes, reconstructing the topology by connecting the objects based on the mutual positions of the recognized objects, and adding attributes by inputting attribute and specification information to each object of the reconstructed topology.

[0012] Examples of errors that occurred during the reconstruction of topological structures in a deep learning-based intelligent engineering drawing generation method are as follows.

[0013] In Fig. 1a, the distance between symbols is short, so a connection relationship is added between symbols; in Fig. 1b, the text is shifted to the left and connected to the symbol instead of the pipe line; and in Fig. 1c, the symbol and the line are separated from each other, so the arrow of the line that should be connected to the symbol on the right is not connected to the symbol.

[0014] As described above, errors occurring during the phase structure reconstruction process significantly reduce the precision and reproducibility of the size or positional relationships between objects included in the digitized piping instrumentation diagram. Therefore, in the digitization of image-based piping instrumentation diagrams based on deep learning, it is very important to reduce errors that occur during the phase reconstruction of objects, and it is necessary to provide a phase reconstruction process of objects with improved precision and reproducibility. Prior art literature

[0015] Korean Registered Patent Publication No. 10-2361118 The problem to be solved

[0016] The present invention aims to provide a method for reconstructing the phase of an object to improve the precision and reproducibility of a digitized pipe instrumentation diagram by minimizing errors that occur during the reconstruction of the phase structure when performing digitization of an image-format pipe instrumentation diagram based on deep learning. means of solving the problem

[0017] A method for reconstructing the topology of an object for the digitization of a deep learning-based image-format Piping and Instrumentation Diagram (P&ID) according to the present invention comprises: a process of recognizing an object including a symbol, a line, and text from an image-format Piping and Instrumentation Diagram; a process of reconstructing a topology structure by connecting the objects based on their mutual positions; and an attribute addition process of inputting attribute and specification information to each object, wherein the process of reconstructing the topology structure comprises: an object connection relationship identification step of identifying the connection relationships between the recognized objects, namely the symbol, text, and line, using object detection data generated during the object recognition process; and a topology structure graph representation step of representing all symbols, text, and lines as nodes and displaying them on graph data, and, if a connection relationship exists between the nodes displayed on the graph data, adding an edge connecting each node between the corresponding nodes. It consists of an object recognition data preprocessing step for preprocessing object recognition data, a first edge addition step for finding associations between text and symbols or between text and lines and adding edges, and a second edge addition step for finding connection relationships between symbols and lines and adding edges; and a graph refinement step for performing additional processing on the graph.

[0018] According to an embodiment of the present invention, in the object connection relationship identification step, the connection relationship of the symbol is identified using type information classified as equipment, piping component, or instrument.

[0019] According to an embodiment of the present invention, when identifying the connection relationship of a symbol, the shape of the symbol is not considered, and only the position, size, and rotation direction information of the bounding box are used.

[0020] According to an embodiment of the present invention, the preprocessing of the object recognition data preprocessing step (S231) comprises: merging duplicate lines and symbols into one; splitting lines based on the intersection point between lines and removing short lines when lines intersect each other; removing the part of the line included in the symbol when one end of the line is included in the symbol excluding text; and splitting lines based on the symbol when the line penetrates the symbol.

[0021] According to an embodiment of the present invention, in the first edge addition step, if the text is contained in only one symbol and the ratio of the overlapping portion between the text bounding box and the symbol bounding box is formed to be greater than or equal to the ratio set relative to the text bounding box area, or if the text is contained in one line and the line penetrates the text bounding box and the direction between the text and the line is the same and the text width is contained within the line width, or if the text is not contained in only one symbol or the text is not contained in one line, and the text overlaps with one symbol by 50% or more of the bounding box area and the width or height of the text is completely contained within the width or height of the symbol, then it is treated as complete inclusion.

[0022] According to an embodiment of the present invention, if the relationship between text and a symbol or between text and a line is unclear in the first edge addition step, the text is connected to the nearest placed symbol or line.

[0023] According to an embodiment of the present invention, in the second edge addition step, if the distance between the endpoints of two lines is formed within a set range, an edge is added between the lines; if the distance between the endpoint of a line and the symbol bounding box is formed within a set range, an edge is added between the line and the symbol; or if the distance between the bounding boxes of two symbols is formed within a set range, an edge is added between the symbols.

[0024] According to an embodiment of the present invention, the graph refinement step comprises a cycle removal step for removing cycles formed by three or more closely spaced symbols, a piping element addition step for finding parts forming branches and adding piping elements, a line merging step for merging multiple interconnected lines into one, a nozzle addition step for adding nozzles between lines connected by edges and equipment symbols, a line class determination step for determining the class of the lines, and a line alignment step for aligning lines in connected order while considering the connection relationships between lines.

[0025] According to an embodiment of the present invention, in the cycle removal step, cycles are searched for through a depth-first search, and the process is performed by removing the edge between the two nodes that are furthest apart among the nodes forming the cycle.

[0026] According to an embodiment of the present invention, in the step of adding a piping element, a trunk connection between lines forming a branch, a new addition of a piping element having a branching function, and a trunk connection between all lines of the added piping element and the branch are performed.

[0027] According to an embodiment of the present invention, in the line class determination step, a line connecting piping elements to each other and a line connected to equipment or a nozzle is recognized as a piping line, and a line connected to an instrument of the operator class, a line connecting instruments to each other, and a line connecting an instrument to a piping element is recognized as a signal line. Effects of the invention

[0028] According to the present invention, when performing digitization of an image-format piping instrumentation diagram based on deep learning, the connection relationship between objects such as symbols, text, and lines is identified and expressed as a topological structure graph, then the object recognition data is preprocessed and edges are added to generate an association graph, and then the graph is refined to reconstruct the topological structure, thereby minimizing errors that occur during the reconstruction of the topological structure and improving the precision and reproducibility of the digitized piping instrumentation diagram. Brief explanation of the drawing

[0029] Figure 1 is a drawing that compares a graph formed by an error that occurred during the reconstruction of a topological structure when generating a deep learning-based intelligent engineering drawing with the original piping instrumentation drawing. Figure 2 is a diagram showing the digitization process of a deep learning-based image-format piping instrumentation diagram. Figure 3 is a diagram comparing a graph generated by phase structure reconstruction with a final generated digital piping instrumentation diagram. Figure 4 is a diagram showing the types of information included in object recognition data when identifying the relationship between objects in the object relationship identification step. Figure 5 is a drawing showing an example of a piping instrumentation diagram represented by a bounding box. Figure 6 is a drawing showing a graph representation of the piping instrumentation diagram according to Figure 5. Figure 7 is a diagram showing an example of a processing method according to the object recognition data preprocessing step. Figure 8 is a diagram showing an example of an association formed when finding an association between text and a symbol, or between text and a line, and adding an edge in the first edge addition step. FIG. 9 is a diagram illustrating an example of a method for connecting text to the nearest symbol or line by irradiating a ray along the coordinate axes from the center of each corner of a text bounding box when the relationship between text and a symbol or text and a line is unclear. Figure 10 is a diagram showing an example of graph refinement according to a cycle removal step, a piping element addition step, or a line merging step in the graph refinement step. Figure 11 is a diagram showing an image-format pipe instrumentation diagram and a graph generated as a result of reconstructing the topological structure of an object recognized from the pipe instrumentation diagram. FIGS. 12 to 16 show the original drawings of test examples 1 to 5 of image-format piping instrumentation diagrams for verifying the precision and reproducibility of the phase structure reconstruction process. FIG. 17 is a table showing the number of symbols, text, and lines included in the original drawings of test examples 1 to 5 of the image-format piping instrumentation drawings of FIG. 12 to 16. FIGS. 18 to 22 are drawings showing a portion of the image-format piping instrumentation diagram of Test Examples 1 to 5, a graph before the completion of the phase structure reconstruction of the said portion, and a graph after the completion of the phase structure reconstruction, respectively. FIG. 23 is a table showing the results of phase structure reconstruction, measured precision, and reproducibility of test examples 1 to 5. Specific details for implementing the invention

[0030] Hereinafter, embodiments of the present invention will be described with reference to the attached drawings.

[0031] The parts necessary to understand the operation and function according to the present invention will be explained in detail, focusing on the parts necessary for understanding.

[0032] In describing the embodiments of the present invention, explanations of technical details that are well known in the technical field to which the present invention belongs and are not directly related to the present invention are omitted.

[0033] This is intended to convey the essence of the invention more clearly without obscuring it by omitting unnecessary explanations.

[0034] In addition, when describing the components of the present invention, different reference numerals may be assigned to components with the same name depending on the drawing, and the same reference numeral may be assigned even to different drawings.

[0035] However, even in such cases, it does not mean that the component has different functions depending on the embodiment or has the same function in different embodiments, and the function of each component should be determined based on the description of each component in the respective embodiment.

[0036] Furthermore, technical terms used in this specification shall be interpreted in the sense generally understood by those skilled in the art to which the present invention pertains, unless specifically defined otherwise in this specification, and shall not be interpreted in an overly broad or overly narrow sense.

[0037] Additionally, singular expressions used in this specification include plural expressions unless the context otherwise indicates.

[0038] In this application, terms such as "composed" or "comprising" should not be interpreted as necessarily including all of the various components or steps described in the specification, and should be interpreted as potentially excluding some of the components or steps, or including additional components or steps.

[0040] Deep learning-based digitalization of image-format piping and instrumentation diagrams is a technology that automatically recognizes various objects included in a piping and instrumentation diagram (P&ID), such as symbols, text, and lines, from an image-format piping and instrumentation diagram (P&ID) and digitizes the piping and instrumentation diagram.

[0041] The digitization of the piping instrumentation diagram consists of an object recognition process (S100), a phase structure reconstruction process (S200), and an attribute addition process (S300), as illustrated in FIG. 2.

[0042] In the object recognition process (S100), objects including symbols, lines, and text are recognized from a pipe instrumentation diagram in image format, and image processing is performed on the input pipe instrumentation diagram image data to improve object detection performance.

[0043] At this time, if there is interference between objects in the pipe instrumentation image data, partial deformation of the object shape, or noise such as spots, the success rate of object recognition may decrease. Therefore, in the object recognition process (S100), a method to extract features of objects from given data is learned through deep learning using a Convolutional Neural Network (CNN) architecture, thereby further improving the object detection performance.

[0044] Since the objects, such as symbols, text, or lines, that constitute the object recognition data detected from the image-formatted pipe instrumentation diagram through the object recognition process (S100) do not contain mutual connection relationships, the topology reconstruction process (S200) forms an object connection graph by connecting the objects to each other based on the mutual positions between each object.

[0045] The attribute addition process (S300) defines the data model of the digital piping instrumentation diagram by inputting attribute and specification information into each object, such as geometric representation of symbols required to represent the digital piping instrumentation diagram, specification of line styles, font style of text, outer border and title box, catalog definition and reference, general attributes of symbols and lines, identification of external reference data, and representation of connection information between symbols and lines.

[0046] In the attribute addition process (S300), the pipe instrumentation diagram generator takes the object connection graph formed in the phase structure reconstruction process (S200), generates symbols, text, and lines, and places them according to the positions of the symbols and lines defined in the 2D catalog, and adjusts the position, size, and rotation direction to fit the bounding box of the symbol or line according to the attribute and specification information entered into each object.

[0047] Additionally, an ID must be assigned to each object of the digital piping instrumentation diagram to identify and manage plant items (symbols or lines). In the object connection graph of the phase structure reconstruction process (S200), symbols or lines are integrated into the text, but since the ID of the plant item is not determined, an ID is assigned to the text integrated with the symbols or lines in the attribute addition process (S300).

[0048] Subsequently, a digital piping instrumentation diagram is generated by saving the piping instrumentation diagram in DEXPI (Data Exchange in the Process Industry) format using the assigned ID.

[0049] The method for reconstructing the topology of an object for digitizing a deep learning-based image-format pipe instrumentation diagram according to the present invention relates to the configuration of a topology structure reconstruction process (S200) that identifies the connection relationship between symbols, lines, and text using object recognition data recognized in an object recognition process (S100) during the digitization process of an image-format pipe instrumentation diagram configured as above.

[0050] The difference between the data generated in the phase structure reconstruction process (S200) and the digital piping instrumentation diagram data generated after the completion of the attribute addition process (S300) is as follows, and Figure 3 shows an example of the reconstructed phase structure and the digital piping instrumentation diagram.

[0052] 1. Reconstructed topological structure:

[0053] - Consists of bounding boxes and type data of objects consisting of symbols, text, or lines.

[0054] - Includes integrated information and some attribute data

[0055] - Includes connection data between objects consisting of connection information and connection points

[0056] 2. Digital Piping Instrumentation:

[0057] - Consists of bounding boxes, types, and 2D shape data of objects consisting of symbols, text, or lines.

[0058] - Includes integrated information, object IDs such as tags or lines, and all attribute data.

[0059] - Includes connection information and connection data between objects consisting of connection nodes and connection order.

[0061] In addition, the topology structure reconstruction process (S200) consists of an object connection identification step (S210), a topology structure graph representation step (S220), an association graph generation step (S230), and a graph refinement step (S240).

[0062] In the object connection relationship identification step (S210), the connection relationships between the recognized objects, such as symbols, text, and lines, are identified using the object detection data generated in the object recognition process (S100).

[0063] When identifying the connection relationship between symbols, lines, and text using the object recognition data recognized through the object recognition process (S100) in the object connection relationship identification step (S210), it is assumed that the object recognition data includes the table of FIG. 4 or the information below.

[0064] 1. Symbol

[0065] - Symbol types are classified into three types: equipment, piping components, and instruments.

[0066] - Symbol classes are classified into 76 types.

[0067] - Symbol bounding box position, size, and rotation direction

[0068] 2. Text

[0069] - Content of the recognized text

[0070] - Text bounding box position, size, and rotation direction

[0071] 3. Line

[0072] - Line types distinguished by solid lines, dashed lines, etc.

[0073] - Location of the line's start and end points

[0075] In the object connection relationship identification step (S210), the connection relationships between objects are identified using only the information given in the table of Fig. 4.

[0076] For symbols, instead of class information subdivided into 76 categories, three types of information classified into equipment, piping elements, and instruments are used, and the connection relationships between symbols are identified using only the position, size, and rotation direction information of the bounding box without considering the shape of the symbols.

[0077] In addition, symbols representing branch connections of piping lines, such as tee fittings or cross fittings, are not expressed in the piping instrumentation diagram, so they must be newly recognized in the object connection relationship identification step (S210).

[0078] In the case of lines, a line type is provided, but since the line type merely represents graphic attributes and cannot distinguish whether the line corresponds to a piping line or a signal line, the connection relationship between lines is identified using only the coordinates of the start and end points.

[0079] And since lines are recognized as segmented, a bent line is recognized as two lines.

[0080] In the topology structure graph representation step (S220), all symbols, text, and lines recognized in the object connection identification step (S210) are represented as nodes and displayed on the graph data, and if a connection exists between the nodes displayed in the graph data, an edge connecting each node is added between the corresponding nodes.

[0081] A graph refers to a data structure composed of a set of nodes and a set of edges connecting the nodes. Graphs are primarily used to represent network structures, and when a diagram is represented as a graph, symbols are used to denote nodes, while the lines connecting the symbols are used to denote edges.

[0082] However, since the relationships between objects are not fully identified prior to topological reconstruction, it is not clear to represent lines as edges; furthermore, symbols are often connected to each other without lines, or text is directly connected to symbols or lines, making the representation of edges even more unclear.

[0083] Therefore, in the topological structure graph representation step (S220), symbols, text, and lines are all represented as nodes, and if a connection relationship exists between these nodes, an edge is added between the two nodes.

[0084] Figure 5 is an example of a piping instrumentation diagram represented by a bounding box, and Figure 6 shows a graph representation of the piping instrumentation diagram of Figure 5, wherein in Figure 6, a circle made of solid lines represents a node, a line represents an edge between two nodes, and a circle made of dotted lines represents a newly added node.

[0085] Before topological reconstruction according to the topological structure graph representation step (S220), only nodes representing objects of object recognition data exist on the graph, and edges are added between nodes during the topological reconstruction of the topological structure graph representation step (S220).

[0086] Since the object recognition data does not include associations between objects, the association graph generation step (S230) generates an association graph using only the shape information of the bounding box and the line.

[0087] The association graph generation step (S230) proceeds in three steps: an object recognition data preprocessing step (S231), a first edge addition step (S232), and a second edge addition step (S233).

[0088] Since the symbol and line information of the object recognition data recognized in the object recognition process (S100) does not exactly match the drawing of the original piping instrumentation diagram and contains a certain level of error and error, the object recognition data preprocessing step (S231) performs preprocessing of the object recognition data to increase the accuracy of identifying the relationships between each object.

[0089] The preprocessing of the object recognition data preprocessing step (S231) is performed using the following several processing methods.

[0090] 1. Processing to merge duplicate lines and symbols into a single one

[0091] 2. When lines intersect, split the lines based on the intersection point and remove the shorter line.

[0092] 3. If part of a line is included in a symbol excluding text, process to remove the portion of the line contained within the symbol.

[0093] 4. Processing to split the line based on the symbol when the line penetrates the symbol

[0095] FIG. 7a shows an example of a process for dividing a line based on the intersection point of two lines where an intersection occurs and removing a short line, FIG. 7b shows an example of a process for removing one end of a line included in a symbol excluding text, and FIG. 7c shows an example of a process for dividing a line penetrating a symbol based on the symbol.

[0096] When the object recognition data preprocessing step (S231) is completed, the first edge addition step (S232) is performed, and in the first edge addition step (S232), an association between text and symbols, or between text and lines, is found and an edge is added.

[0097] When associating text with a symbol or line in the first edge addition step (S232), the one with a clear association is prioritized, and in the case of the example below, it is determined that the association is clear.

[0098] 1. If text is contained within only one symbol and the ratio of the overlap between the text bounding box and the symbol bounding box is greater than or equal to the set ratio relative to the text bounding box area, it is treated as full inclusion.

[0099] 2. If text is contained within a single line, and the line penetrates the text bounding box, the text and line share the same orientation, and the text width is included within the line width, it is treated as full inclusion.

[0100] 3. Cases not corresponding to the two examples above, where the text overlaps a symbol by 50% or more of the bounding box area and the width or height of the text is completely contained within the symbol's width or height, are treated as full containment.

[0102] FIG. 8a shows a case where text is contained in only one symbol, FIG. 8b shows a case where text is contained in one line, and FIG. 8c shows a case where text and symbols overlap, which does not correspond to the examples of FIG. 8a or FIG. 8b.

[0103] In this case, the present invention is treated as complete inclusion when the text is contained in only one symbol and the ratio of the overlapping portion between the text bounding box and the symbol bounding box is 99% or more of the text bounding box area.

[0104] And if the relationship between text and a symbol or between text and a line is unclear, the text is connected to the nearest symbol or line, and as shown in FIG. 9, when a ray is projected along the coordinate axes from the center of each corner of the text bounding box, the symbol or line that the ray reaches first is selected as the symbol or line closest to the text.

[0105] The above method of selecting the closest placed symbol or line takes into account that the text is generally placed parallel to the line or is placed somewhat close to the center of the symbol, and a symbol located diagonally to the text 'T', such as the symbol 'P' in Fig. 9, may not be selected because there is no opportunity to be connected to the text.

[0106] Additionally, since text may be displayed across multiple lines, a preliminary step may be taken to locate texts positioned close to a specific text and merge them into a single text before finding the object closest to the text.

[0107] In the second edge addition step (S233), the connection relationship between symbols and lines is found and an edge is added, and the edge addition in the second edge addition step (S233) is performed as follows.

[0108] 1. Add an edge between lines if the distance between the endpoints of two lines is within a set range.

[0109] 2. If the distance between the line endpoint and the symbol bounding box is within the set range, add an edge between the line and the symbol.

[0110] 3. If the distance between the bounding boxes of two symbols is within the set range, add an edge between the symbols

[0112] When the relationship graph generation step (S230) is completed, a graph refinement step (S240) is performed to find the connection relationships between symbols, lines, and text to primarily construct the graph and to perform additional processing on the primarily constructed graph. The graph refinement step (S240) consists of a cycle removal step (S241), a piping element addition step (S242), a line merging step (S243), a nozzle addition step (S244), a line class determination step (S245), and a line alignment step (S246).

[0113] In the cycle removal step (S241), when three or more symbols are placed close to each other and form a cycle as shown in FIG. 10a, the cycle formed between the symbols is removed by removing the edge between the two nodes that are farthest apart among the nodes forming the cycle, and the search for the cycle to be removed is performed through a depth-first search.

[0114] In the step of adding piping elements (S242), a part forming a branch is identified as shown in FIGS. 10b and FIGS. 10c, and piping elements such as tee fittings or cross fittings are added.

[0115] In the step for adding piping elements (S242), an edge connection is made between lines forming a branch, a new piping element with a branching function is added, and an edge connection is made between all lines of the added piping element and the branch.

[0116] Lines forming branches in the graph are connected to each other by edges, and the number of edges connecting one line node to another line node has a value of (branch number - 1). After finding line branches, a pipe element with branching capabilities is added, and all lines of the branch are newly connected to the edges of the pipe element.

[0117] Accordingly, in the case of tee fitting, as shown in Fig. 10c, each line node is connected to two other nodes through the edges, and in the case of cross fitting, as shown in Fig. 10d, each line node is connected to three other nodes through the edges.

[0118] In the line merging step (S243), when multiple lines are connected to each other, multiple lines connected in series are merged into one line, excluding dashed lines as shown in FIG. 10e.

[0119] Since the nozzle does not appear in the piping instrumentation diagram or is not well recognized during the object recognition process, the nozzle is added between the line connected by an edge and the equipment symbol through the nozzle addition step (S244), and the line class is determined in the line class determination step (S245).

[0120] In the line class determination step (S245), it is determined whether the line is a pipe line or a signal line.

[0121] 1. Piping line:

[0122] A piping line is recognized as a line connecting piping elements to one another, or a line connected to equipment or nozzles.

[0123] 2. Signal line:

[0124] Lines connected to instruments of the operator class, lines connecting instruments to each other, and lines connecting instruments to piping elements are recognized as signal lines.

[0125] 3. If it does not correspond to the classification of recognition as a pipe line or signal line, set it to undefined so that the user can decide directly.

[0127] In the line alignment step (S246), lines are aligned in the connected order considering the connection relationships between lines, and starting from an arbitrary line, if the direction of the surrounding lines does not match the direction of the starting line, the direction of the adjacent lines is reversed.

[0128] Figures 11a and 11b show the results of reconstructing the phase structure of an object recognized from an image-format pipe instrumentation diagram in an object recognition process (S100), the left diagram shows the original pipe instrumentation diagram, and the center and right diagrams show the graphs generated as a result of reconstructing the phase structure.

[0129] In the graphs of Figures 11a and 11b with the phase structure reconstruction completed, the blue arrows represent the visualization of the edges between each object, the red marked parts indicate that the symbols and text in the marked parts are connected to each other, and the purple and orange lines represent the signal line and the pipe line, respectively.

[0130] In the right drawing of Fig. 11a, it can be seen that nozzles and tees, which are not present in the left drawing of Fig. 11a, the original piping instrumentation drawing, have been added. In the original piping instrumentation drawing, lines were distinguished only by solid lines and dashed lines, but through phase reconstruction, they were recognized according to the function of the lines and distinguished into signal lines and piping lines through color differentiation.

[0131] The graph finally generated in the phase structure reconstruction process (S200) is stored in an object linkage graph in XML format along with object recognition data and transmitted to the attribute addition process (S300) for generating a digital piping instrumentation diagram.

[0132] The phase structure reconstruction process (S200) according to the present invention was implemented in the following system environment.

[0133] - Operating System: MS Windows 10 Pro 64bit

[0134] - CPU: Intel Core i9

[0135] - RAM: 64GB

[0136] - Language:C#

[0137] - GUI: Windows Presentation Foundation (WPF), DevExpress

[0139] FIGS. 12 to 16 are drawings showing test examples 1 to 5 of image-format pipe instrumentation drawings, which are original drawings for verifying the precision and reproducibility of the phase structure reconstruction process (S200) by generating a digital pipe instrumentation drawing, and the number of symbols, text, and lines included in each test example is as shown in FIG. 17.

[0140] 1. Test 1:

[0141] Consists of 154 symbols, 216 texts, and 222 lines

[0142] 2. Test 2:

[0143] Consists of 82 symbols, 235 texts, and 122 lines

[0144] 3. Test 3:

[0145] It consists of 78 symbols, 227 texts, and 141 lines.

[0146] 4. Test 4:

[0147] Consists of 62 symbols, 201 texts, and 112 lines

[0148] 5. Test 5:

[0149] It consists of 86 symbols, 325 texts, and 124 lines.

[0151] FIGS. 18a to 22a show a portion of the image-format piping instrumentation diagram, which is the original drawing of Test Examples 1 to 5, FIGS. 18b to 22b show a graph before the completion of the phase structure reconstruction of Test Examples 1 to 5, and FIGS. 18c to 22c show a graph after the completion of the phase structure reconstruction of Test Examples 1 to 5, respectively.

[0152] As a result of the phase structure reconstruction according to the phase structure reconstruction process (S200), the precision and reproducibility measurement results of test examples 1 to 5 are as shown in the table of FIG. 23, and the average precision and reproducibility of test examples 1 to 5 were formed as 99.56% and 96.07%, respectively.

[0154] Although embodiments of the present invention have been described with reference to the foregoing, those skilled in the art to which the present invention pertains will understand that the present invention may be implemented in other specific forms without altering its technical concept or essential features.

[0155] Therefore, the embodiments described above should be understood as exemplary in all respects and not limiting, and the scope of the invention as described in the detailed description above is defined by the claims set forth below, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the invention.

Claims

Claim 1 Digitization of a Piping and Instrumentation Diagram (P&ID) is performed by a computer with an MS Windows operating system installed, based on deep learning, through a process (S100) of recognizing objects including symbols, lines, and text from an image-format Piping and Instrumentation Diagram (P&ID), a process (S200) of reconstructing a topological structure by connecting objects based on their mutual positions, and an attribute addition process (S300) of inputting attribute and specification information to each object, wherein the process of reconstructing the topological structure (S200) comprises: an object connection relationship identification step (S210) that identifies the connection relationships between symbols, text, and lines, which are the recognized objects, using object detection data generated in the object recognition process (S100); and a topology structure graph representation in which symbols, text, and lines are all represented as nodes and displayed on graph data, and if a connection relationship exists between the nodes displayed on the graph data, an edge connecting each node is added between the corresponding nodes. Step (S220); an object recognition data preprocessing step (S231) for preprocessing object recognition data; a first edge addition step (S232) for finding association relationships between text and symbols or text and lines and adding edges; and a second edge addition step (S233) for finding connection relationships between symbols and lines and adding edges; a graph refinement step (S240) for performing additional processing on the graph;A method for reconstructing the topology of an object for the digitization of a deep learning-based image-format piping instrumentation diagram, characterized by being composed of the following: if the relationship between text and a symbol or between text and a line is unclear in the first edge addition step (S232), a ray is irradiated along the coordinate axis from the center of each corner of the text bounding box, and the symbol or line that the ray reaches first is selected and connected to the symbol or line placed closest to the text; and the attribute addition process (S300) inputs the line style of the line, the font style of the text, the outer boundary, and the title box as attribute and specification information of the object, and adjusts the position, size, and rotation direction of the symbol and text according to the attribute and specification information. Claim 2 A method for reconstructing the topology of an object for digitizing a deep learning-based image-format piping instrumentation diagram, characterized in that, in the object connection relationship identification step (S210), the identification of the connection relationship of the symbol utilizes type information classified as equipment, piping component, or instrument. Claim 3 A method for reconstructing the topology of an object for the digitization of a deep learning-based image-format piping instrumentation diagram, characterized in that, in identifying the connection relationships of symbols, only the position, size, and rotation direction information of a bounding box is used without considering the shape of the symbol. Claim 4 A method for reconstructing the topology of an object for digitizing a deep learning-based image-format pipe instrumentation diagram, wherein, in claim 1, the preprocessing of the object recognition data preprocessing step (S231) comprises: merging duplicate lines and symbols into one; dividing lines based on the intersection point between lines and removing short lines when lines intersect each other; removing the line portion included in the symbol when one end of a line is included in a symbol excluding text; and dividing lines based on the symbol when a line penetrates the symbol. Claim 5 A method for reconstructing the topology of an object for the digitization of a deep learning-based image-format piping instrumentation diagram, wherein in the first edge addition step (S232), the text is treated as fully included if the ratio of the overlapping portion between the text bounding box and the symbol bounding box is formed to be greater than or equal to the ratio set relative to the text bounding box area, or the text is treated as fully included if the text is included in a single line, but the line penetrates the text bounding box, the direction between the text and the line is the same, and the width of the text is included in the line width, or the text is not included in a single symbol, or the text is not included in a single line, and the text overlaps the bounding box area with a single symbol by 50% or more, and the width or height of the text is fully included in the width or height of the symbol. Claim 6 delete Claim 7 A method for reconstructing the topology of an object for digitizing a deep learning-based image format piping instrumentation diagram, characterized in that, in the second edge addition step (S233), if the distance between the endpoints of two lines is formed within a set range, an edge is added between the lines; if the distance between the endpoints of a line and a symbol bounding box is formed within a set range, an edge is added between the lines and a symbol; or if the distance between the bounding boxes of two symbols is formed within a set range, an edge is added between the symbols. Claim 8 A method for reconstructing the topology of an object for the digitization of a deep learning-based image-format piping instrumentation diagram, wherein the graph refinement step (S240) comprises: a cycle removal step (S241) for removing a cycle formed by three or more closely spaced symbols; a piping element addition step (S242) for finding a part that forms a branch and adding a piping element; a line merging step (S243) for merging a plurality of interconnected lines into one; a nozzle addition step (S244) for adding a nozzle between a line connected by an edge and an equipment symbol; a line class determination step (S245) for determining the class of the line; and a line alignment step (S246) for aligning the lines in the connected order considering the connection relationship between the lines. Claim 9 A method for reconstructing the topology of an object for the digitization of a deep learning-based image-format pipe instrumentation diagram, characterized in that, in the cycle removal step (S241) according to claim 8, the cycle is searched for through a depth-first search and the edge between the two nodes that are furthest apart among the nodes forming the cycle is removed. Claim 10 A method for reconstructing the topology of an object for the digitization of a deep learning-based image-format piping instrumentation diagram, characterized in that, in the step of adding a piping element (S242), an edge connection between lines forming a branch is made, a new piping element having a branch function is added, and an edge connection between all lines of the added piping element and the branch is made. Claim 11 A method for reconstructing the phase of an object for the digitization of a deep learning-based image-format piping instrumentation diagram, characterized in that, in the line class determination step (S245), a line connecting piping elements and piping elements and a line connecting to equipment or a nozzle are recognized as piping lines, and a line connecting to an instrument of an operator class, a line connecting to an instrument and an instrument, and a line connecting to an instrument and a piping element are recognized as signal lines.

Images