An oil and gas pipeline disaster scene knowledge graph construction method and system based on a large language model

Abstract

The application discloses a kind of oil and gas pipeline disaster scene knowledge graph construction method based on large language model, comprising: generating semantic coherent text data;The text data is input into large language model, user input prompt word and generate system prompt word guide the large language model sequentially executes entity recognition, relationship extraction, triple construction and entity standardization process, to generate knowledge triple;Before the knowledge graph construction step, the generated knowledge triple is visualized and stored to graph database. Thus, the unstructured data of multiple source heterogeneous oil and gas pipeline disasters is processed by using large language model, to realize the automatic processing of key links such as entity recognition, relationship extraction, and then generate standard triple data and be used for the construction and expansion of knowledge graph. Reasonable output result can be accurately and efficiently obtained;In the process, unstructured data processing is adopted, and the obtained knowledge graph is converted into a visual webpage.

Description

Technical Field

[0001] This invention relates to the field of large language model and knowledge graph technology, and in particular to a method and system for constructing a knowledge graph for oil and gas pipeline disaster scenarios based on a large language model. Background Technology

[0002] In some regions, mudslides, caused by a combination of rainfall and geological factors, have resulted in powerful scouring, scraping, and impact forces that have thinned pipeline burial depth, exposed pipelines, stripped anti-corrosion layers, and even caused leaks, threatening the safe operation of pipelines and the ecological environment. Integrating multi-source disaster data to assist in the management of oil and gas pipeline disasters has become a key research focus.

[0003] The core objective of knowledge graphs is to describe various entities and concepts in the real world and the complex semantic relationships between them in a structured manner. By organizing information into "node-edge-node" triples, it not only provides machines with a world model that can be understood and computed, but also lays the data foundation for higher-level applications such as semantic search, intelligent reasoning, and content recommendation. With the development of technology, knowledge graphs have been widely used in disaster prevention and control. For example, landslide disaster knowledge graphs are constructed using landslide data to assess and analyze landslide susceptibility; typhoon disaster chain knowledge graphs are constructed to study the evolution process and comprehensive risk analysis of disasters; multi-source heterogeneous data from the oil and gas pipeline industry are integrated to deeply mine the semantic relationships between pipeline entities and various types of data, promoting the intelligent management of oil and gas pipeline risks; and flood disaster knowledge graphs are constructed based on multimodal data and applied to disaster scenarios, such as crop loss assessment and flood disaster emergency response.

[0004] While knowledge graphs have shown broad application prospects in disaster prevention and control, their construction and maintenance still face many challenges. Specifically, the construction process of knowledge graphs relies heavily on manual annotation and is time-consuming, becoming a bottleneck for further utilization. Traditional construction methods often depend on manual compilation and rule definition by domain experts, which is inefficient and difficult to adapt to dynamically changing data environments. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for constructing a knowledge graph of oil and gas pipeline disaster scenarios based on a large language model.

[0006] The objective of this invention is achieved through the following technical solution: Firstly, this application discloses a method for constructing a knowledge graph of oil and gas pipeline disaster scenarios based on a large language model, comprising: performing consistency processing on raw oil and gas pipeline disaster data containing non-text data to generate semantically coherent text data; inputting the text data into a large language model, with user input prompts and system prompts guiding the large language model to sequentially perform entity recognition, relation extraction, triple construction, and entity standardization processes to generate knowledge triples; wherein, the system prompts are used to set the global identity and task rules of the large language model, and the user prompts are used to issue immediate instructions to the large language model containing specific input, tasks, and output formats; before the knowledge graph construction step, if the length of the text data exceeds a preset threshold, the text data is segmented according to preset rules, and the segmented text data is input into the large language model for processing respectively; the generated knowledge triples are visualized and stored in a graph database.

[0007] Furthermore, the standardization process includes: extracting textual information from the original non-text file using code to obtain preliminary text; inputting the preliminary text into the large language model for semantic optimization and logical connection to generate the semantically coherent text data.

[0008] Furthermore, the system prompt is used to define the global identity of the large language model as an expert in the field of oil and gas pipeline disaster technology, and to specify the output format; the user prompt is used to specify the specific text content to be processed, the types of entities and relations to be extracted, and the number and format of triples to be output.

[0009] Furthermore, the preset rule for text segmentation is: to segment the text according to a fixed character length or sentence boundary, and to set overlapping content between adjacent text blocks.

[0010] Furthermore, the formulas for expressing the system prompts and user prompts are as follows: ; in This indicates a system prompt word, which specifies the model's identity. ),rule ) etc.; among them This represents the user prompt word, which sets the input of the model ( ),Task( ), output ( ).

[0011] Furthermore, the triple G is: ; In this context, G represents a triple; S represents the subject; P represents the predicate; and O represents the object.

[0012] Secondly, this application discloses an oil and gas pipeline disaster scenario processing system, including a storage device and a processor; the storage device stores a computer program for implementing the aforementioned method for constructing an oil and gas pipeline disaster scenario knowledge graph based on a large language model, and the processor performs read and write operations on the storage device. The beneficial effects of this invention are: The automated knowledge graph construction method for oil and gas pipeline disasters in this application utilizes a large language model to process unstructured data of multi-source heterogeneous oil and gas pipeline disasters, automating key processes such as entity recognition and relation extraction. This generates standard-compliant triplet data for knowledge graph construction and expansion. It can accurately and efficiently obtain reasonable output results. During the process, unstructured data processing is employed, and the final knowledge graph is transformed into a visual webpage. Attached Figure Description

[0013] Figure 1 A simplified flowchart of a method for constructing a knowledge graph of oil and gas pipeline disaster scenarios based on a large language model, according to some embodiments of this application; Figure 2 This is a flowchart illustrating the prompt-guided process according to some embodiments of this application; Figure 3 This is a flowchart illustrating document processing according to some embodiments of this application; Figure 4 This is a flowchart illustrating the knowledge extraction process according to some embodiments of this application; Figure 5 Visualization of knowledge graphs according to some embodiments of this application. Detailed Implementation

[0014] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] In recent years, with the rapid development of artificial intelligence technology, especially the rise of large language models, a new technical path has been provided for the automated construction of knowledge graphs. Large language models possess powerful semantic representation and logical reasoning capabilities, enabling them to automatically identify entities, attributes, and relationships in text and express them in a structured manner. This allows them to read disaster report texts, extract disaster entities and relationships, and even identify relevant entities in multi-source disaster data related to oil and gas pipelines, accelerating the construction of disaster knowledge graphs. Based on this, this application discloses a method and system for constructing a knowledge graph for oil and gas pipeline disaster scenarios based on large language models.

[0016] refer to Figure 1 The method for constructing a knowledge graph of oil and gas pipeline disaster scenarios based on a large language model according to the embodiments of this application includes the following steps: starting from the original text processing, the text is uniformly processed into the same format, and then handed over to the large language model for text block processing. Then, the preliminary text blocks are subjected to triple extraction to extract entities and relations in the text, and then stored in the form of "subject-verb-object" triples. Next, the extracted triples are placed in a visualization tool for visualization operation, and at the same time, a graph database tool is connected to save the extracted knowledge graph.

[0017] First, prompt words are used to provide the necessary context and instruction guidance for the large language model to perform specific tasks.

[0018] Specifically, such as Figure 2 As shown, the prompts in this embodiment include system prompts and user prompts, expressed in the following formula: ; in This indicates a system prompt word, which specifies the model's identity. ),rule ) etc.; among them This represents the user prompt word, which sets the input of the model ( ),Task( ), output ( )wait.

[0019] System prompts are set before the dialogue begins, providing the model with a global guiding framework and basic rules.

[0020] For example, prompts can define the model's role in the interaction (e.g., "oil and gas pipeline disaster technical expert"), specifying the format, tone, or knowledge scope to follow when responding. User prompts, on the other hand, originate from the user's specific input and are used to express the immediate task or question of the current round. For instance, a user can use prompts to ask the model to "extract triples for oil and gas pipeline disasters." User prompts are dynamic and reflect the user's specific intentions during the dialogue, while the model must respond flexibly and accurately within the framework set by the system prompts. The synergy between system prompts and user prompts allows the large language model to maintain behavioral consistency under the constraints of global rules while also making targeted outputs based on the user's specific needs, thereby achieving efficient and controllable human-computer interaction.

[0021] Furthermore, knowledge graph construction includes entity recognition, relation extraction, triple construction, and entity standardization.

[0022] The entity recognition stage includes setting the model task and input through prompt words. For example, the text to be extracted is set in the prompt word part and the task of "extracting oil and gas pipeline disaster entities" is passed to the model. When the model reads the text, it will extract and save disaster entities such as "disaster name", "disaster time", "disaster location" etc. Then, the model will extract the complete triples based on the extracted disaster entities.

[0023] The relation extraction stage includes defining the model's global identity and basic rules through prompts, such as defining its identity as an "expert in knowledge extraction of the oil and gas pipeline domain" upon input. Then, during the relation extraction stage, the prompts specify that the model should extract only structured triples from the given oil and gas pipeline disaster text, and specify the number of output triples to avoid generating redundant or out-of-scope content. Furthermore, user prompts further refine the task requirements, such as specifying the content of the input text, limiting entity types, indicating relation categories, or requiring results to be returned in a specific format.

[0024] The entity standardization stage prompts are still primarily role-based. For example, if the model's role is set as "an expert responsible for the normalization of oil and gas pipeline disaster entities," the large language model will use the same noun to represent the same variant within oil and gas pipeline disasters. The user prompts in this part will specifically specify details such as standardization rules, the target knowledge base for alignment, and output format requirements, ensuring that the model completes entity alignment and semantic normalization under unified standards. Through this layered and phased prompt design, this embodiment of the application achieves fine-grained control over the large language model generation process, ensuring both the systematic and standardized nature of the knowledge graph construction process and taking into account the specificity and flexibility of each task.

[0025] Based on the aforementioned prompt word processing, a large language model is used to process various types of text content, and a text consistency strategy is employed to achieve unified processing of heterogeneous data; specifically, such as Figure 3 As shown, this embodiment follows a standardization process, converting input files of different formats into a unified text format for subsequent normalization by a large language model. This process effectively supports the automated construction of knowledge graphs, especially for non-text files. The automated construction of the knowledge graph includes: First, the original file is initially extracted using code and converted into plain text format. Then, a large language model is called to perform semantic optimization and content restructuring on the extracted text. Finally, the restructured file is passed to the large language model for further processing. For example, the data processing method can be found in Table 1.

[0026]

[0027] Furthermore, while code can effectively extract text content from non-text files, its output often suffers from problems such as messy formatting, broken sentences, or semantic incoherence. Directly using it for knowledge extraction may affect the accuracy and completeness of triple extraction. Therefore, this application further introduces a large language model to perform semantic repair and logical completion on the initially extracted text, thereby effectively avoiding the limitations of large language models in processing non-text content such as images and tables. This also allows the model to focus on understanding and structured extraction of text semantics, thereby improving the efficiency and reliability of knowledge graph construction.

[0028] Specifically, the method is based on the tabular data compiled by Disaster Excel. The large language model will perform semantic connections. For example, if the table contains positional relationships and the code extracts "oil and gas pipeline disaster" and "mountainous area", the large language model will compile it into "the oil and gas pipeline disaster occurred in the mountainous area". The large language model will also reduce the text content, removing non-disaster data and simplifying excessive descriptions of disaster data.

[0029] After processing by the large language model, the oil and gas pipeline disaster data is output as raw data in three types of non-text files, which is saved and processed directly. Before the text is fed into the large language model, to address the context window limitations and local system memory constraints of the large language model, this embodiment incorporates a text chunking mechanism. The system segments long texts into predetermined lengths according to settings, preserving a certain amount of overlap between chunks to maintain semantic coherence at sentence boundaries. Each text chunk, along with preset system and user prompts for each processing part, is input into the large language model, which then performs a triplet extraction task.

[0030] In detail, triples are the basic semantic units for constructing knowledge graphs, and in existing technologies, they are usually represented in the form of "attribute-relation-attribute". However, due to the difficulties that large language models face in processing highly abstract or implicit semantic relations, and considering the characteristics of oil and gas pipeline disaster data, this application refines and adapts the composition of triples, explicitly expressing it as a "subject-verb-object" structure. This expression is closer to the syntactic habits of natural language, which helps large language models to identify and extract relations based on grammatical structure. Accordingly, when extracting triples, the content extraction strictly follows the "subject-verb-object" grammatical logic.

[0031] In the relation extraction stage, this application embodiment uses a large language model to directly read the input text, extracts semantically clear and structurally well-defined subject-verb-object combinations, and organizes them into triples in the following form; ; In this context, G represents a triple; S represents the subject, which is the main entity in the relation; P represents the predicate, which is the relationship between the subject and the object; and O represents the object, which is the objective entity in the relation that is affected by the predicate.

[0032] As mentioned earlier, knowledge extraction mainly consists of two parts: entity extraction and relation extraction. In this embodiment, when using a large language model for knowledge extraction, both parts are performed simultaneously, extracting disaster entities and their related relations at the same time. Synonym disambiguation is also an important part of knowledge extraction; it is a crucial step in the standardized representation of knowledge graphs. Figure 4 As shown.

[0033] For example Figure 4 LLM-driven disaster knowledge extraction is mainly divided into three stages: extraction of explicit relations and entities, extraction of implicit relations, and synonym disambiguation.

[0034] This application's embodiments initially utilize a large language model to directly extract explicit triples, initially obtaining entities and explicit relationships. The large language model then infers implicit relationships based on disaster documents and these entity pairs and relationships, identifying implicit relationships not explicitly expressed in the original text and generating new triples. The generated triples together construct a disaster knowledge graph. When the large language model extracts triples, disaster texts may contain different expressions for the same entity. However, the large language model does not merge these expressions during identification but extracts them directly. Since the same entity may appear in disaster texts with different names or abbreviations, directly storing it as an independent node in the knowledge graph without processing would lead to node fragmentation or duplicate node creation, affecting the graph's structural quality and query efficiency. Therefore, this application's embodiments include an "entity standardization" stage, identifying and clustering multiple expressions pointing to the same disaster, geographical, or other entities in different disaster text fragments. The large language model then merges these clusters to unify node relationships.

[0035] refer to Figure 5Visualizing knowledge graphs is one of the most direct and intuitive ways to represent triples. This application uses PyVis to visualize a knowledge graph of oil and gas pipeline disasters. PyVis is a lightweight network graph visualization library based on Python, which relies on the powerful JavaScript visualization library Vis.js. This tool has several advantages, including support for interactive operation, rich customization options, and most importantly, the ability to output results as web pages. Therefore, it can save the generated knowledge graph in HTML format, facilitating further direct display and use. Compared to some knowledge graph visualization tools, PyVis focuses on the visualization itself, without involving specific query languages ​​or data persistence mechanisms, making it more concise and flexible in certain scenarios. It is particularly noteworthy that the visualization results generated by PyVis are saved as HTML files, making it easy to embed into various web systems or application interfaces, exhibiting good integration capabilities. In contrast, while graph databases are powerful, they often have certain technical barriers in web page integration and front-end display, hindering rapid deployment and system integration of knowledge graph results.

[0036] However, while PyVis performs well in knowledge graph visualization and interactive display, it lacks the ability to store knowledge graphs. Each time the tool runs, it needs to rebuild the graph view based on the newly input triplet data, generating a new HTML file that overwrites the previous results. This means it cannot continuously display or revisit historically constructed graphs, limiting its application in scenarios requiring long-term maintenance and reuse of knowledge graphs.

[0037] To address this limitation, this application further introduces the graph database Neo4j as an auxiliary tool. As a mature and widely used graph database, Neo4j possesses robust data persistence capabilities, stably storing triplet data. As long as it is not actively deleted, the graph data can be preserved long-term and supports repeated queries and uses. Therefore, Neo4j effectively compensates for PyVis's shortcomings in data storage, becoming an important supplement to the knowledge graph construction and visualization system in this application. However, Neo4j's native interactive experience is relatively simple, and its front-end display capabilities are not as rich and flexible as PyVis. Combining the two can create a good complementarity in terms of storage and display.

[0038] Furthermore, PyVis allows users to personalize the display style, node size, and layout of the knowledge graph, thereby improving the user-friendliness and experience of the visualization interface. In the knowledge graph visualization interface of this embodiment, some customizable knowledge graph display buttons are added, allowing users to set corresponding keywords to find nodes or relationships. For the display of the knowledge graph, this embodiment also sets some corresponding coefficients, mainly controlling the direct display effect of nodes and relationships, such as the length of relationship lines and the dynamic effect of connecting relationships when nodes are dragged. These settings make the display of the knowledge graph more diverse. In the display of relationships, this embodiment distinguishes between inferred relationships and directly extracted relationships using solid and dashed lines. At the top of the display interface, this embodiment displays the total number of nodes and relationships constructed in this knowledge graph, as well as other relevant information.

[0039] An oil and gas pipeline disaster scenario processing system according to an embodiment of this application includes a storage device and a processor; the storage device stores a computer program for implementing the above-mentioned method for constructing an oil and gas pipeline disaster scenario knowledge graph based on a large language model, and the processor performs read and write operations on the storage device.

[0040] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A method for constructing a knowledge graph of oil and gas pipeline disaster data based on a large language model, characterized in that, include: The raw oil and gas pipeline disaster data containing non-textual data is processed to achieve consistency and generate semantically coherent textual data. The text data is input into a large language model. The user inputs prompt words and generates system prompt words to guide the large language model to sequentially perform entity recognition, relation extraction, triple construction, and entity standardization processes to generate knowledge triples. The system prompt words are used to set the global identity and task rules of the large language model, and the user prompt words are used to issue real-time instructions to the large language model that include specific input, task, and output formats. Before the knowledge graph construction step, if the length of the text data exceeds a preset threshold, the text data is divided into blocks according to preset rules, and the divided text data is input into the large language model for processing. The generated knowledge triples are visualized and stored in a graph database.

2. The method for constructing a knowledge graph of oil and gas pipeline disaster scenarios based on a large language model according to claim 1, characterized in that, The standardization process includes: The code is used to extract text information from a raw non-text file to obtain preliminary text. The initial text is input into the large language model for semantic optimization and logical connection to generate semantically coherent text data.

3. The method for constructing a knowledge graph of oil and gas pipeline disaster scenarios based on a large language model according to claim 1, characterized in that, The system prompt is used to define the global identity of the large language model as an expert in the field of oil and gas pipeline disaster technology and to specify the output format; the user prompt is used to specify the specific text content to be processed, the types of entities and relations to be extracted, and the number and format of triples to be output.

4. The method for constructing a knowledge graph of oil and gas pipeline disaster scenarios based on a large language model according to claim 1, characterized in that, The preset rule for text segmentation is: to divide the text according to a fixed character length or sentence boundary, and to set overlapping content between adjacent text blocks.

5. The method for constructing a knowledge graph of oil and gas pipeline disaster scenarios based on a large language model according to claim 1, characterized in that, The formulas for expressing the system prompts and user prompts are as follows: ； in This indicates a system prompt word, which specifies the model's identity. ),rule ) etc.; among them This represents the user prompt word, which sets the input of the model ( ),Task( ), output ( ).

6. The method for constructing a knowledge graph of oil and gas pipeline disaster scenarios based on a large language model according to claim 1, characterized in that, The triple G is: ； In this context, G represents a triple; S represents the subject; P represents the predicate; and O represents the object.

7. A disaster response system for oil and gas pipelines, characterized in that, It includes a storage device and a processor; the storage device stores a computer program for implementing the method for constructing a knowledge graph of oil and gas pipeline disaster scenarios based on a large language model as described in any one of claims 1-6, and the processor performs read and write operations on the storage device.

Images