A method and device for automatically creating a fault knowledge graph of a fault tree image

Abstract

The present invention provides a method and device for automatically creating a fault knowledge graph of a fault tree image, the method comprising: performing knowledge modeling on a knowledge graph of a complex equipment fault, defining entity types, relationship types and attributes; performing grayscale, binarization and morphological processing on the fault tree image; segmenting the image into a number of non-intersecting detection areas, identifying the coordinates of the center points of the detection areas; identifying the content of the detection areas, matching the fault events with the event codes one by one, and completing entity extraction; completing the relationship extraction between entities based on the identified detection area content and the coordinates of the center points of the detection areas; converting entities and relationships into structured triples, establishing a fault knowledge graph, and storing it in a graph database; converging and fusing the fault knowledge graph; and performing root cause analysis and fault impact analysis using the knowledge graph. The present invention can effectively solve the problem of difficulty in manually acquiring large-scale knowledge, and realize the convergence and fusion of multi-source heterogeneous knowledge.

Description

Technical Field

[0001] The present invention belongs to the field of complex equipment health management, and in particular relates to a method and device for automatically creating a fault knowledge graph of a fault tree image. Background Art

[0002] A fault tree diagram is a logical cause-and-effect diagram, composed of events and logic gates: events are used to describe the state of system and element / component failures; logic gates link events together to represent the logical relationship between events. Fault diagnosis and root cause analysis of equipment can be well performed based on the fault tree, but since the image is an unstructured data, the knowledge in a small number of fault tree images can be obtained manually, but it cannot be applied to large-scale fault tree images, resulting in it not being fully utilized in the field of health management. Especially in the field of complex equipment health management, there is little research on fault knowledge graphs, and it is necessary to fill the gap in this field. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method and device for automatically creating a fault knowledge graph of a fault tree image, which automatically converts the acquired knowledge into a fault knowledge graph, thereby realizing the aggregation of fault knowledge in large-scale fault tree images; at the same time, it can also utilize the high performance advantages of the graph database to better realize fault diagnosis, root cause analysis and fault impact analysis, and intuitively display them in a visual form.

[0004] In order to achieve the purpose of the present invention, the following scheme is proposed:

[0005] A method for automatically creating a fault knowledge graph of a fault tree image comprises the following steps:

[0006] S1: Conduct knowledge modeling on the complex equipment fault knowledge graph and define entity types, relationship types and attributes;

[0007] S2: Perform image preprocessing on the read fault tree image, including grayscale, binarization and morphological processing, to highlight the detection target;

[0008] S3: Segment the image into several non-intersecting detection areas and identify the center point coordinates of the detection areas;

[0009] S4: Use optical character recognition technology to identify the content of the detection area, and match the fault events and event codes one by one to complete entity extraction;

[0010] S5: Based on the identified detection area content and the coordinates of the center point of the detection area, complete the relationship extraction between entities;

[0011] S6: Convert entities and relations into structured triples, build a fault knowledge graph, and store it in a graph database;

[0012] S7: Aggregate and fuse the fault knowledge graphs automatically created from multiple fault tree images;

[0013] S8: Visualize the integrated fault knowledge graph and use the knowledge graph to perform root cause analysis and fault impact analysis.

[0014] Further, S2: performing image preprocessing on the fault tree image, including grayscale, binarization and morphological processing, to highlight the detection target, including the following steps:

[0015] S21: read the fault tree image;

[0016] S22: Grayscale the fault tree image to convert each pixel value in the fault tree image into Gray ( i , j );

[0017] S23: performing binarization processing on the grayscale processed fault tree image, and setting the grayscale value of the pixel points on the image to 0 or 255;

[0018] S24: performing morphological processing on the binary fault tree image, specifically, performing an opening operation and a closing operation on the binary image, wherein the opening operation is first erosion and then expansion, and the closing operation is first expansion and then erosion.

[0019] Further, S3: dividing the image into a plurality of non-intersecting detection areas and identifying the coordinates of the center points of the detection areas, including the following steps:

[0020] S31: Extract the contour and attributes of the target detection area based on the contour extraction algorithm in image processing;

[0021] S32: traverse all target detection area contours, calculate the area of ​​each contour, and filter out areas whose area is smaller than a predetermined area threshold;

[0022] S33: Calculate the contour length of the target detection area, and obtain the approximate contour of the target detection area using a polygonal approximation method;

[0023] S34: Use the minimum rectangle area method to obtain the rectangles of the approximate contour, the number is k 1. And determine the coordinates of the four corners and the center point of the rectangle;

[0024] S35: Filter out target detection areas whose shape narrowness is less than a predetermined narrowness threshold according to the ratio of the length and width of the rectangle. The number of remaining target detection areas is k 2, k 2≤ k 1;

[0025] S36: Using a graphics drawing method, the target detection area is divided into rectangular frames and saved in order of index, where the index value is [0, k 2-1].

[0026] Further, S4: using optical character recognition technology to identify the content of the detection area, and matching the fault events and event codes one by one to complete entity extraction, including the following steps:

[0027] S41: Loop recognition of the text in the rectangular box in S36 based on pytesseract, and save the recognized text content in a dictionary form according to the sequential index;

[0028] S42: Put the elements with the same key value into a list, and merge the elements in the list;

[0029] S43: merging the rectangular boxes whose ordinate difference is less than a predetermined approximate threshold, filtering out the rectangular boxes containing the word "door", and defining the merged content in a dictionary form of {key: value}, where key is the fault input event code and value is the fault input event;

[0030] S44: When the identified content distance is less than a predetermined spacing threshold, multiple fault input events are merged into the same rectangular frame, different fault input events are distinguished according to regular expressions, and the remaining fault input event codes are determined according to the vertical coordinate of the fault input event code corresponding to the key in S43, and the fault input events are matched one by one with the fault input event codes to complete entity extraction.

[0031] Further, S5: based on the identified detection area content and the coordinates of the center point of the detection area, the relationship between entities is extracted, including the following steps:

[0032] S51: Count the number of logic gates in the target detection area n ;

[0033] S52: Find the fault output event whose vertical coordinate difference with the corresponding logic gate is less than the predetermined proximity threshold, and store the logic gate and the fault output event in a list L i (1≤ i ≤ n )middle;

[0034] S53: Determine whether the fault events are simultaneously input into a logic gate according to the relationship between the horizontal coordinates of the target detection area, and store the fault events input into the same logic gate in the same list L j (1≤ j ≤( k 2- n)*1 / 2), where j Represents the number of initial fault input events;

[0035] S54: According to S52 L i The horizontal axis of the fault output event determines the fault input event corresponding to each logic gate and the fault output event.

[0036] Further, S6: converting entities and relations into structured triples, establishing a fault knowledge graph, and storing it in a graph database, includes the following steps:

[0037] S61: Connect to Neo4j graph database;

[0038] S62: creating fault modes, logic gates, and nodes affected by the faults respectively according to regular matching;

[0039] S63: According to the list in S54, create the relationship between the fault mode and the logic gate, and the relationship between the logic gate and the fault impact to establish a fault knowledge graph and store it in the graph database.

[0040] Further, S7: aggregating and fusing the fault knowledge graphs automatically created from the multiple fault tree images, including the following steps:

[0041] S71: Repeat S1 to S6 for multiple fault tree images to obtain fault knowledge graphs of different fault tree images, and store them in a graph database;

[0042] S72: Use the Levenshtein algorithm to calculate the similarity between entities, merge the entities whose similarity exceeds the threshold, and obtain a corrected fault knowledge graph.

[0043] Further, in S8, the root cause analysis includes: according to the user input of the desired fault mode, in response to the analysis request, constructing a query statement according to the request, sending a conditional query request to the graph database, obtaining the fault cause of the desired fault mode and handing it over to the front-end rendering, and displaying the search results in the form of a graph visualization. The fault impact analysis includes: according to the user input of at least one fault mode and logic gate to be searched, in response to the analysis request, constructing a query statement according to the request, sending a conditional query request to the graph database, obtaining the fault impact of the desired fault mode and handing it over to the front-end rendering, and displaying the search results in the form of a graph visualization.

[0044] A device for automatically creating a fault knowledge graph of a fault tree image, comprising:

[0045] Modeling definition module, used to perform knowledge modeling on complex equipment fault knowledge graphs and define entity types, relationship types, and attributes;

[0046] The preprocessing module is used to perform image preprocessing on the read fault tree image, including grayscale, binarization and morphological processing, so as to highlight the detection target;

[0047] A segmentation and recognition module is used to segment the image into a number of non-intersecting detection areas and identify the center point coordinates of the detection areas;

[0048] The identification and extraction module is used to use optical character recognition technology to identify the content of the detection area, and to match the fault events and event codes one by one to complete entity extraction;

[0049] The relationship extraction module is used to extract the relationship between entities based on the identified detection area content and the coordinates of the center point of the detection area;

[0050] The conversion and establishment module is used to convert entities and relationships into structured triples, establish a fault knowledge graph, and store it in the graph database;

[0051] The convergence and fusion module is used to converge and fuse the fault knowledge graphs automatically created from multiple fault tree images;

[0052] The display and analysis module is used to visualize the integrated fault knowledge graph and use the knowledge graph to perform root cause analysis and fault impact analysis.

[0053] A device for automatically creating a fault knowledge graph of a fault tree image, comprising: at least one processor and a memory; wherein the memory stores computer-executable instructions; the computer-executable instructions stored in the memory are executed by the at least one processor, so that the at least one processor executes the method for automatically creating a fault knowledge graph of a fault tree image.

[0054] A computer-readable storage medium stores a computer program, which, when executed by a processor, controls a device where the storage medium is located to execute the method for automatically creating a fault knowledge graph of a fault tree image.

[0055] The beneficial effects of the present invention are:

[0056] 1. A method / device for automatically creating a fault knowledge graph based on a fault tree image is proposed, which can effectively solve the problem of difficulty in manually acquiring large-scale knowledge, automatically transform the acquired knowledge into a fault knowledge graph, realize the aggregation of fault knowledge in large-scale fault tree images, and realize the aggregation and fusion of multi-source heterogeneous knowledge;

[0057] 2. Based on the high performance advantage of graph database, compared with traditional fault diagnosis expert systems, it can realize real-time fault diagnosis, root cause analysis and fault impact analysis, and display them intuitively in a visual form, so as to better ensure the operation of equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The drawings described herein are for illustrative purposes only of selected embodiments and not all possible implementations, and are not intended to limit the scope of the present invention.

[0059] Figure 1 This is a flow chart of a method for automatically creating a fault knowledge graph according to an embodiment of the present application.

[0060] Figure 2 This is the knowledge modeling model of the embodiment of the present application.

[0061] Figure 3 This is the original fault tree picture of the embodiment of the present application.

[0062] FIG. 4( a ) is a schematic diagram of a segmented fault tree image according to an embodiment of the present application.

[0063] FIG4( b ) is a segmented fault tree image of an embodiment of the present application.

[0064] Figure 5 This is the fault knowledge graph of the embodiment of the present application.

[0065] Figure 6 A device structure diagram is automatically created for the fault knowledge graph of the embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings. The described implementation modes are only for example and are not intended to limit the present invention.

[0067] One aspect of the embodiments of the present application provides a method for automatically creating a fault knowledge graph of a fault tree image, such as Figure 1 As shown, the following steps are included:

[0068] S1: Conduct knowledge modeling on the complex equipment fault knowledge graph and define entity types, relationship types and attributes.

[0069] Specifically, this example adopts the top-down and bottom-up knowledge modeling methods, based on GJB / Z 1391-2006, to establish Figure 2 The data model shown in the figure includes three entity types, two relationship types and three attributes. It is represented by a triple as <fault mode—isInputTo—logic gate>, <logic gate—output—fault impact>, where the three attributes are name, id and the attribute num unique to the logic gate, where the is gate and the or gate are 1, and the and gate are n , n Represents all fault events of the input AND gate, m / n Select the gate as ( n - m ).

[0070] S2: Perform image preprocessing on the read fault tree image, including grayscale, binarization and morphological processing, to highlight the detection target. The specific steps include the following:

[0071] S21: Read the fault tree image, such as Figure 3 shown.

[0072] S22: Grayscale the fault tree image. A pixel of the image is a tuple consisting of (R, G, B), and the grayscale values ​​of the pixel are values ​​between [0, 255]. Grayscale is the process of making the R, G, and B components of the color equal. This example uses the average method to grayscale the fault tree image. The calculation is as follows, where i , j are the rows and columns of the pixel matrix respectively:

[0073] Gray ( i , j )=[ R ( i , j )+ G ( i , j )+ B ( i , j )] / 3.

[0074] S23: Binarize the grayscale processed fault tree image. After grayscale processing, the value of each pixel in the fault tree image becomes Gray ( i , j ). Binarization is the process of setting the grayscale value of the pixel on the image to 0 or 255, that is, making the entire image appear in obvious black and white. Binarization of the image greatly reduces the amount of data in the image, thereby highlighting the outline of the target. This example uses the adaptive threshold determination method of the maximum inter-class difference method to obtain the fault tree image after binarization.

[0075] S24: Perform morphological processing on the binary fault tree image. Morphological processing mainly refers to erosion, dilation, and opening and closing operations on the binary image. Both opening and closing operations are composed of erosion and dilation. The opening operation is erosion first and then dilation, while the closing operation is dilation first and then erosion. The opening operation can make the image contour smooth, disconnect narrow connections and eliminate fine burrs. The closing operation can also make the contour smooth, but unlike the opening operation, it can usually bridge narrow gaps and fill small holes. The kernel functions of erosion and dilation are set to (38, 9) and (24, 6) respectively. Erosion is performed first and then dilation. The number of iterations of erosion is set to 1 and the number of iterations of dilation is set to 6.

[0076] S3: Segment the image into several non-intersecting detection areas and identify the coordinates of the center points of the detection areas. Specifically, the steps are as follows:

[0077] S31: Based on the contour extraction algorithm in image processing, the contour and attributes of the target detection area are extracted.

[0078] S32: traverse all target detection area contours, calculate the area of ​​each contour, set the predetermined area threshold to 4000, and filter out areas with an area less than 4000.

[0079] S33: Calculate the contour length of the target detection area, and use a polygonal approximation method to obtain an approximate contour of the target detection area.

[0080] S34: Using the minimum rectangle area method, a rectangle approximating the target detection area is obtained, the number of which is 15, and the coordinates of the four corners and the center point of the rectangle can be determined.

[0081] S35: Some target detection areas with narrow shapes are screened out according to the ratio of the length and width of the rectangle, and the number of remaining target detection areas is 14.

[0082] S36: Use the graphics drawing method to divide the target detection area into rectangular frames and save them in order. There are 14 pictures in total, with index values ​​of [0, 13]. The schematic diagram of the division is shown in Figure 4(a). The divided pictures are shown in Figure 4(b). From left to right and from top to bottom, the picture content "X5" to the picture content "high-speed rotor speed abnormality" represent indexes 0 to 13 respectively. The center point coordinates of the pictures are [1980.5, 1599.5], [1550.0, 1593.5], [1994.5, 1502.0], [1766.9, 1368.5], [2458.5, 1142.0], [995.5, 1087.5], [595.5,1082.0], [176.0, 1073.0], [1732.0, 1055.0], [2427.5, 887.0], [1745.5, 866.5], [608.8, 863.1], [1368.5, 298.0], [1449.0, 140.7].

[0083] S4: Use optical character recognition technology to identify the content of the detection area, and match the fault events and event codes one by one to complete entity extraction. The specific steps include:

[0084] S41: Based on pytesseract, the text in the rectangular box in S36 is recognized in a loop, and the recognized text content is saved in a dictionary form according to the sequential index, in the form of {index: text content}, where index is 0 to 13.

[0085] S42: Put the elements with the same key value in a list, and merge the elements in the list. Specifically, since optical character recognition is performed by line, it will cause multiple lines of text in a rectangular box to be segmented, but their indexes are the same, so the text in the same rectangular box can be merged according to the same index.

[0086] S43: merge the rectangular boxes with similar ordinates, filter out the rectangular boxes containing the word "door", and define the merged content in the form of a dictionary of {key: value}, where key is the code of the fault input event and value is the fault input event, thereby completing entity extraction. Specifically, according to the center point coordinates of each segmented image obtained in S36, the absolute value of the y-axis difference between the two images is compared with a predetermined approximate threshold. For two images that are less than the predetermined approximate threshold, it can be considered that the fault input event and the code are one-to-one corresponding, where the predetermined approximate threshold is set to 100.

[0087] S44: Since the identified contents are very close, multiple fault input events will be merged into the same rectangular frame, as shown in indexes 3 and 11 in Figure 4(b). At this time, different fault events can be distinguished according to regular expressions, the index corresponding to "fault" or "abnormality" can be located, and the identified contents can be sliced ​​and merged. After slicing and merging, taking the image corresponding to index 11 in Figure 4(b) as an example, the fault input event code corresponding to image 11 is X2, and the vertical coordinate corresponding to X2 is 1082.0. By setting a predetermined spacing threshold of 20, the vertical coordinates corresponding to the fault input events X1 and X3 adjacent to X2 can be found to be 1073.0 and 1087.5, respectively. The fault input events are matched one by one with the fault input event codes according to the indexes after slicing and merging, and entity extraction is completed.

[0088] S5: Based on the identified detection area content and the coordinates of the center point of the detection area, complete the extraction of the relationship between entities. Specifically, it includes the following steps:

[0089] S51: Count the number n of logic gates in the target detection area, which is 2;

[0090] S52: Find the fault output event that is close to the vertical coordinate of the corresponding logic gate, and store the logic gate and the fault output event in a list L i (1≤ i ≤ n )middle, L 1 =[“OR gate”: “E1: Hall power supply circuit fault”], then L 2 is [“OR gate”: “High-speed rotor speed is abnormal”];

[0091] S53: Determine whether the fault events are simultaneously input into a logic gate according to the relationship between the horizontal coordinates of the target detection area, and store the fault input events input into the same logic gate in the same list L j (1≤ j ≤6), where j Represents the number of fault input events. Specifically, it can be known from S36 that the ordinates of the fault input events X1, X2, X3 and X6 are 1142.0, 1087.5, 1082.0 and 1073.0 respectively, and the ordinates of X4 and X5 are 1599.5 and 1593.5, so the fault input events X1, X2, X3 and X6 are a group, and the fault input events X4 and X5 are a group.

[0092] S54: According to S52 L iThe horizontal coordinate of the fault output event determines the fault input event corresponding to each logic gate and the fault output event. Specifically, it can be known from S52 and S36 that L 1 The horizontal coordinate of the intermediate fault event "E1: Hall power supply circuit fault" is 1745.5. According to the horizontal coordinate relationship, it can be determined that the corresponding input event is the segmented image 3, with a horizontal coordinate of 1766.9. Therefore, it can be determined L 1 The corresponding input fault events are X4 and X5. According to S51, the number of logic gates is 2, so it can be determined that L 2 The input fault events corresponding to the fault output events are X1, X2, X3, X6 and E1.

[0093] S6: Convert entities and relationships into structured triples, build a fault knowledge graph, and store it in a graph database. This includes the following steps:

[0094] S61: Connect to a local Neo4j graph database or a remote graph database.

[0095] S62: Create fault mode, logic gate and fault-affected nodes according to regular matching and indexing. Specifically, according to S54, 6 fault mode nodes can be created, namely {"labels":"fault mode","properties":{"name":"X1: High-speed rotor shaft fault","id":"fault mode_1"}}, {"labels":"fault mode","properties":{"name":"X2: Motor winding circuit fault","id":"fault mode_2"}}, {"labels":"fault mode","properties":{"name":"X3: Hall output circuit fault","id":"fault mode_3"}}, {"labels":"fault mode","properties":{"name":"X4: Connector and cable fault","id":"fault mode_4"}}, {"labels":"fault mode","properties":{"name":"X5: Abnormal increase in conductive ring impedance","id":"fault mode_6"}} mode_5"}} and {"labels":"fault mode","properties":{"name":"X6: control line fault","id":"fault mode_6"}}, 2 logic gate nodes, respectively {"labels":"logic gate","properties":{"name":"OR gate","id":"logic gate_1","num":1}} and {"labels":"logic gate","properties":{"name":"OR gate","id":"logic gate_2","num":1}}, 2 fault impact nodes, respectively {"labels":"fault impact","properties":{"name":"E1: Hall power supply circuit fault","id":"fault impact_1"}} and {"labels":"fault impact","properties":{"name":"High-speed rotor speed abnormality","id":"fault impact_2"}}.

[0096] S63: According to the list in S54 L i , create the relationship between fault modes and logic gates, and between logic gates and fault effects, thereby building a fault knowledge graph and storing it in a graph database, such as Figure 5 shown.

[0097] S7: Aggregate and fuse the fault knowledge graphs automatically created from multiple fault tree images. This includes the following steps:

[0098] S71: Repeat steps S1-S6 for multiple fault tree images to obtain fault knowledge graphs of different fault tree images and store them in a graph database.

[0099] S72: Use the Levenshtein algorithm to calculate the similarity between entities, merge entities whose similarity exceeds the threshold, and obtain the corrected fault knowledge graph. Levenshtein can calculate the edit distance between two strings, that is, the minimum number of edit operations required to convert two entities from one to another. The smaller the edit distance, the greater the similarity between the two entities. For two entities with a threshold exceeding 0.8, the user manually confirms whether they can be merged.

[0100] S8: Visualize the integrated fault knowledge graph and use the knowledge graph to perform root cause analysis and fault impact analysis.

[0101] Root cause analysis: the user enters the fault mode they want to search for on the front end, and then clicks the "Root Cause Analysis" button to send a request; the Django framework then builds a query statement based on the request, sends a conditional query request to the Neo4j graph data, and obtains the fault cause of the fault mode they want to search for, which is then rendered on the front end, displaying the search results in the form of a visual graph.

[0102] Fault impact analysis: the user enters one or more fault modes and logic gates that they want to search for on the front end, and then clicks the "Fault Impact Analysis" button to send a request; the Django framework then builds a query statement based on the request, sends a conditional query request to the Neo4j graph data, and obtains the fault impact of the fault mode that they want to search for, which is then rendered by the front end, displaying the search results in the form of a visual graph.

[0103] Another aspect of the present application embodiment provides a device for automatically creating a fault knowledge graph of a fault tree image, which is used to implement the automatic creation method described in the above embodiment, such as Figure 6 As shown, the automatic creation device of the fault knowledge graph includes: a modeling definition module, a preprocessing module, a segmentation and recognition module, an identification and extraction module, a relationship extraction module, a transformation and establishment module, an aggregation and fusion module, and a display and analysis module.

[0104] Specifically, the modeling definition module is used to perform knowledge modeling on the complex equipment fault knowledge graph, and define entity types, relationship types and attributes; the preprocessing module is used to perform image preprocessing on the read fault tree image, including grayscale, binarization and morphological processing, so as to highlight the detection target; the segmentation and recognition module is used to segment the image into several non-overlapping detection areas, and identify the center point coordinates of the detection area; the recognition and extraction module is used to use optical character recognition technology to identify the content of the detection area, and correspond the fault event and the event code one by one to complete the entity extraction; the relationship extraction module is used to complete the relationship extraction between entities based on the identified detection area content and the center point coordinates of the detection area; the transformation and establishment module is used to transform entities and relationships into structured triples, establish a fault knowledge graph, and store it in a graph database; the convergence and fusion module is used to converge and fuse the fault knowledge graph automatically created from multiple fault tree images; the display and analysis module is used to visualize the fused fault knowledge graph, and use the knowledge graph to perform root cause analysis and fault impact analysis.

[0105] According to another aspect of an embodiment of the present application, a device for automatically creating a fault knowledge graph of a fault tree image is provided, comprising: at least one processor and a memory; wherein the memory stores computer-executable instructions; the computer-executable instructions stored in the memory are executed by the at least one processor, so that the at least one processor executes the method for automatically creating a fault knowledge graph of a fault tree image described in the aforementioned embodiment.

[0106] In another aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the device where the storage medium is located is controlled to execute the method for automatically creating a fault knowledge graph of a fault tree image as described in the above embodiments.

[0107] The above are only preferred embodiments of the present invention, and are not intended to be the only or limiting embodiments of the present invention. Those skilled in the art should understand that various changes or equivalent substitutions made to the present invention without departing from the scope of the present invention are within the scope of protection of the present invention.

Claims

1. A method for automatically creating a fault knowledge graph of a fault tree image, characterized in that: Includes steps: S1: Conduct knowledge modeling on the complex equipment fault knowledge graph and define entity types, relationship types and attributes; S2: Perform image preprocessing on the read fault tree image, including grayscale, binarization and morphological processing, to highlight the detection target; S3: Segment the image into several non-intersecting detection areas and identify the center point coordinates of the detection areas; S4: Use optical character recognition technology to identify the content of the detection area, and match the fault events and event codes one by one to complete entity extraction; S5: Based on the identified detection area content and the coordinates of the center point of the detection area, complete the relationship extraction between entities; S6: Convert entities and relations into structured triples, build a fault knowledge graph, and store it in a graph database; S7: Aggregate and fuse the fault knowledge graphs automatically created from multiple fault tree images; S8: Visualize the integrated fault knowledge graph and use the knowledge graph to perform root cause analysis and fault impact analysis; S3 includes the following steps: S31: Extract the contour and attributes of the target detection area based on the contour extraction algorithm in image processing; S32: traverse all target detection area contours, calculate the area of ​​each contour, and filter out areas whose area is smaller than a predetermined area threshold; S33: Calculate the contour length of the target detection area, and obtain the approximate contour of the target detection area using a polygonal approximation method; S34: Use the minimum rectangle area method to obtain the rectangles of the approximate contour, the number is k 1. And determine the coordinates of the four corners and the center point of the rectangle; S35: Filter out target detection areas whose shape narrowness is less than a predetermined narrowness threshold according to the ratio of the length and width of the rectangle. The number of remaining target detection areas is k 2, k 2≤ k 1; S36: Using a graphics drawing method, the target detection area is divided into rectangular frames and saved in order of index, where the index value is [0, k 2-1]; S4 includes the following steps: S41: Loop recognition of the text in the rectangular box in S36 based on pytesseract, and save the recognized text content in a dictionary form according to the sequential index; S42: Put the elements with the same key value into a list, and merge the elements in the list; S43: merge the rectangular boxes whose ordinate difference is less than a predetermined approximate threshold, filter out the rectangular boxes containing the word "door", and define the merged content as a dictionary form of {key: value}, where key is the fault input event code and value is the fault input event; S44: when the identified content distance is less than a predetermined spacing threshold, multiple fault input events are merged into the same rectangular frame, different fault input events are distinguished according to the regular expression, and the remaining fault input event codes are determined according to the vertical coordinates of the fault input event codes corresponding to the key in S43, and the fault input events are matched with the fault input event codes one by one to complete the entity extraction; S5 includes the following steps: S51: Count the number of logic gates in the target detection area n ; S52: Find the fault output event whose vertical coordinate difference with the corresponding logic gate is less than the predetermined threshold value, and store the logic gate and the fault output event in a list L i (1≤ i ≤ n )middle; S53: Determine whether the fault events are simultaneously input into a logic gate according to the relationship between the horizontal coordinates of the target detection area, and store the fault events input into the same logic gate in the same list L j (1≤ j ≤( k 2- n ) / 2), where j Represents the number of initial fault input events; S54: According to S52 L i The horizontal axis of the fault output event determines the fault input event corresponding to each logic gate and the fault output event.

2. The method for automatically creating a fault knowledge graph of a fault tree image according to claim 1, characterized in that: S2: Perform image preprocessing on the fault tree image, including grayscale, binarization and morphological processing, to highlight the detection target, including the following steps: S21: read the fault tree image; S22: Grayscale the fault tree image to convert each pixel value in the fault tree image into Gray ( i , j ); S23: performing binarization processing on the grayscale processed fault tree image, and setting the grayscale value of the pixel points on the image to 0 or 255; S24: performing morphological processing on the binary fault tree image, specifically, performing an opening operation and a closing operation on the binary image, wherein the opening operation is first erosion and then expansion, and the closing operation is first expansion and then erosion.

3. The method for automatically creating a fault knowledge graph of a fault tree image according to claim 1, characterized in that: S6: Convert entities and relationships into structured triples, build a fault knowledge graph, and store it in a graph database, including the following steps: S61: Connect to Neo4j graph database; S62: creating fault modes, logic gates, and nodes affected by the faults respectively according to regular matching; S63: According to the list in S54, create the relationship between the fault mode and the logic gate, and the relationship between the logic gate and the fault impact to establish a fault knowledge graph and store it in the graph database.

4. The method for automatically creating a fault knowledge graph of a fault tree image according to claim 1, characterized in that: S7: Aggregate and fuse the fault knowledge graphs automatically created from multiple fault tree images, including the following steps: S71: Repeat S1 to S6 for multiple fault tree images to obtain fault knowledge graphs of different fault tree images, and store them in a graph database; S72: Use the Levenshtein algorithm to calculate the similarity between entities, merge the entities whose similarity exceeds the threshold, and obtain a corrected fault knowledge graph.

5. A device for automatically creating a fault knowledge graph of a fault tree image, characterized in that: include: Modeling definition module, used to perform knowledge modeling on complex equipment fault knowledge graphs and define entity types, relationship types, and attributes; The preprocessing module is used to perform image preprocessing on the read fault tree image, including grayscale, binarization and morphological processing, so as to highlight the detection target; A segmentation and recognition module is used to segment the image into a number of non-intersecting detection areas and identify the coordinates of the center points of the detection areas, including extracting the contours and attributes of the target detection areas based on the contour extraction algorithm in image processing; Traverse all target detection area contours, calculate the area of ​​each contour, and filter out areas whose area is smaller than a predetermined area threshold; Calculate the contour length of the target detection area, and use the polygon approximation method to obtain the approximate contour of the target detection area; use the minimum rectangle area method to obtain the rectangle of the approximate contour, the number is k 1, and determine the coordinates of the four corners and the center point of the rectangle; according to the ratio of the length and width of the rectangle, filter out the target detection areas whose shape narrowness is less than the predetermined narrowness threshold, and the number of remaining target detection areas is k 2, k 2≤ k 1; Use the graphics drawing method to divide the target detection area into rectangular frames and save them in order of index, with the index value being [0, k 2-1]; The recognition and extraction module is used to use optical character recognition technology to identify the content of the detection area, and to match the fault event and the event code one by one to complete the entity extraction, including loop recognition of the text in the rectangular box based on pytesseract, and saving the recognized text content in the form of a dictionary according to the sequential index; putting the elements with the same key value in a list, and merging the elements in the list; merging the rectangular boxes whose ordinate difference is less than a predetermined approximate threshold, and filtering out the rectangular boxes containing the word "door", and defining the merged content as a dictionary form of {key: value}, where the key is the fault input event code and the value is the fault input event; When the identified content distance is less than a predetermined spacing threshold, multiple fault input events are merged into the same rectangular frame, different fault input events are distinguished according to the regular expression, and the remaining fault input event codes are determined according to the vertical coordinates of the fault input event codes corresponding to the key, and the fault input events are matched with the fault input event codes one by one to complete entity extraction; The relationship extraction module is used to extract the relationship between entities based on the recognized detection area content and the coordinates of the center point of the detection area, including counting the number of logic gates in the target detection area. n ; Find the fault output event whose vertical coordinate difference with the corresponding logic gate is less than the predetermined proximity threshold, and store the logic gate and fault output event in the list L i (1≤ i ≤ n )middle; According to the relationship between the horizontal coordinates of the target detection area, determine whether the fault events are input into a logic gate at the same time, and store the fault events input into the same logic gate in the same list L j (1≤ j ≤( k 2- n ) / 2), where j Represents the number of initial fault input events; according to L i The horizontal coordinate of the fault output event determines the fault input event corresponding to each logic gate and the fault output event; The conversion and establishment module is used to convert entities and relationships into structured triples, establish a fault knowledge graph, and store it in the graph database; The convergence and fusion module is used to converge and fuse the fault knowledge graphs automatically created from multiple fault tree images; The display and analysis module is used to visualize the integrated fault knowledge graph and use the knowledge graph to perform root cause analysis and fault impact analysis.

6. A device for automatically creating a fault knowledge graph of a fault tree image, comprising: At least one processor and a memory; wherein the memory stores computer-executable instructions; characterized in that the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method for automatically creating a fault knowledge graph of a fault tree image as described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the device where the storage medium is located is controlled to execute the method for automatically creating a fault knowledge graph of a fault tree image as described in any one of claims 1 to 4.

Images