End-to-end oil business knowledge graph construction method based on large language model

Abstract

The application relates to an end-to-end petroleum business knowledge graph construction method based on a large language model, which comprises the following steps: acquiring multi-source data in a petroleum business field to form rich corpus information; constructing a knowledge extraction module, designing a petroleum business field special prompt template, extracting and generating input data under instruction prompts by using a large language model, and outputting entity relationship triple knowledge structure; constructing a knowledge consistency verification module, introducing a thinking chain reasoning technology and a self-reflection mechanism, and performing correction and optimization; constructing a knowledge storage and management module, automatically importing triple knowledge data into a graph data system, establishing a multi-dimensional index structure, and completing preliminary establishment of a knowledge graph; and constructing a knowledge updating and feedback learning module, realizing dynamic evolution and self-adaptive updating of the knowledge graph. The application can realize rapid construction and dynamic updating of a petroleum business field knowledge graph at a low cost.

Description

Technical Field

[0001] This invention relates to the field of petroleum business knowledge management technology, specifically to an end-to-end petroleum business knowledge graph construction method based on a large language model. Background Technology

[0002] In the oil and gas industry, the ever-changing processes of exploration, development, and production management generate massive amounts of information, such as geological reports, equipment records, project plans, and log information. Processed data is generally categorized into structured, semi-structured, and unstructured information. While tabular data such as seismic data tables and drilling logs can be directly stored in relational databases, semi-structured data like exploration reports and equipment records, and unstructured information such as project reports and technical papers, are more diverse, complex, and scattered, hindering data sharing, querying, updating, and maintenance. Efficiently extracting valuable knowledge from massive amounts of heterogeneous data and constructing high-quality oil and gas business knowledge graphs remains a pressing challenge for the industry.

[0003] Knowledge graphs (KGs) use a triplet structure, with entities as nodes and relationships between entities stored as directed edges. Traditional knowledge graphs still fall short in addressing complex semantics, implicit logical relationships, and the difficulty of knowledge updates. With the rapid development of Large Language Models (LLMs), they possess powerful generalization capabilities in semantic understanding and reasoning, content generation, and more. Based on LLMs, this paper proposes an innovative approach to applying LLMs to knowledge graph modeling in the petroleum business scenario. This approach combines appropriate prompt templates with thought chain reasoning techniques and self-reflection mechanisms, utilizing LoRA fine-tuning.

[0004] However, existing knowledge graph construction in petroleum business scenarios still suffers from problems such as a lack of domain knowledge, data heterogeneity, semantic conflicts, and difficulty in dynamic updates. Therefore, there is an urgent need to automate the entire process from multi-source data collection, knowledge extraction, verification to knowledge storage and updates. To address this, we propose an end-to-end petroleum business knowledge graph construction method based on a large language model to support the digital transformation and intelligent decision-making of the petroleum industry. Summary of the Invention

[0005] The purpose of this invention is to provide an end-to-end petroleum business knowledge graph construction method based on a large language model. This end-to-end petroleum business knowledge graph construction method based on a large language model is used to solve the problems of existing technologies that rely heavily on expert detection, lack structured norms, have poor semantic consistency, and are slow to update.

[0006] The technical solution adopted by this invention to solve its technical problem is as follows: This end-to-end petroleum business knowledge graph construction method based on a large language model includes the following steps:

[0007] Step 1: Acquire multi-source data from the petroleum business field, preprocess and unify the data format, use a vectorized embedding model to convert the text into vector representation, use a retrieval enhancement generation mechanism to retrieve semantically similar fragments, and then reorder and splice them with the original information to form rich corpus information;

[0008] Step 2: Construct a knowledge extraction module, design a prompt template specific to the petroleum business domain, and use a large language model to extract and generate input data under the prompt of instructions, and output the entity relation triple knowledge structure;

[0009] Step 3: Construct a knowledge consistency verification module, introduce thinking chain reasoning technology and self-reflection mechanism, decompose complex problems into sub-problems and solve them step by step, and at the same time retrieve duplicate entities and semantically conflicting relationships, and correct and optimize them.

[0010] Step 4: Construct a knowledge storage and management module. Automatically import the structured triplet knowledge data that has passed the consistency verification in Step 3 into the graph data system, assign a unique ID to each entity, relation, and attribute, establish a multi-dimensional index structure, and complete the initial establishment of the knowledge graph.

[0011] Step 5: Construct a knowledge update and feedback learning module. Based on new input data and changed data, perform LoRA fine-tuning on the large language model, automatically update existing entity nodes and relational structures, and combine human-computer feedback mechanisms to achieve dynamic evolution and adaptive updating of the knowledge graph.

[0012] Step one of the above scheme is as follows:

[0013] Text data was collected, including exploration reports, equipment operation records, daily production reports, reservoir analysis documents, and safety specifications. Data cleaning, deduplication, and standardization were used to remove noise from the text data, correct errors, and unify the data format. A retrieval enhancement generation mechanism was adopted to retrieve semantically similar fragments, and these fragments were reordered and spliced ​​with the original information to provide contextual enhancement information, forming a rich corpus of information.

[0014] Step two in the above scheme is specifically as follows:

[0015] Step 2.1: Construct instruction prompt templates to guide the large language model and automatically identify key entities in petroleum operations. Instruction prompt templates include task descriptions, format constraints, and semantic guidance.

[0016] The task description clearly states the task objectives, including: extracting key entities and relationships from the following petroleum-related texts;

[0017] Format constraints specify the output format, including: the output format is JSON;

[0018] Semantic guidance provides vocabulary or examples for oil business, including: entity types such as wells, formations, equipment, and operation types;

[0019] Step 2.2: After segmenting and dividing the text data into words, input it into the large language model according to the instruction prompt template for context encoding; generate logical relationships between entities based on semantic dependency analysis and context attention mechanism, and complete the identification and extraction of semantic association relationships between different entities;

[0020] Step 2.3: Output the extracted structure as a triplet knowledge structure and store it.

[0021] Step three in the above plan specifically refers to:

[0022] Step 3.1: Use batch pre-screening to quickly eliminate obviously erroneous and low-quality triplet knowledge structures;

[0023] Step 3.2: Perform conflict detection. Utilize semantic similarity and ontology constraint rules to detect entity duplication, attribute conflicts, relational contradictions, and temporal consistency.

[0024] Step 3.3: Based on the thinking chain reasoning technique, by showing examples to the large language model, a complex problem is decomposed into sub-problems step by step and solved sequentially, thereby improving the performance of the large language model;

[0025] The thought chain prompt consists of three parts: instructions, logical basis, and examples. Instructions describe the problem and inform the output format of the large language model. Logical basis refers to the intermediate reasoning process, including the solution to the problem, intermediate reasoning steps, and any external knowledge related to the problem. Examples provide the basic format of input-output pairs for the large language model. Each example includes: the problem, the reasoning process, and the answer.

[0026] Step 3.4: Introduce a self-reflection mechanism. Based on the multi-round reasoning ability of the large language model, perform logical review and factual verification on the triplet knowledge structure. Utilize instruction information that includes logical consistency checks, numerical correctness verification, factual accuracy checks, and multi-angle cross-verification to guide the large language model to re-analyze the generated results from a self-evaluation perspective. Execute multi-round chain reasoning and reflection mechanisms to regenerate rationalized results and compare the revised results with the original results. If ambiguity still exists, perform manual review.

[0027] The knowledge storage and management module in the above solution includes knowledge entry and semantic indexing and retrieval optimization.

[0028] Step five in the above scheme is specifically as follows:

[0029] Step 5.1: Periodically scan the newly input data source, retrieve the changed nodes and newly input entity nodes, and activate the enhanced update task;

[0030] Step 5.2: Fine-tune the large language model using LoRA based on the newly processed data to optimize its semantic extraction capabilities in petroleum operations;

[0031] LoRA fine-tuning adjusts the parameters of the low-rank matrix while maintaining the original parameter matrix. Unchanged, among which Let represent a low-rank space of dimension d×d;

[0032] Two smaller matrices are introduced alongside the original pre-trained language model. sum matrix ,in Describe the low-rank space of matrix B. Let r represent the low-rank space of matrix A, r represent the rank of LoRA, and d represent the original linear layer input dimension;

[0033] The product of matrix B and matrix A, ΔW = B × A, is used as an additional adjustment term for W0;

[0034] The overall weight is: W0 + ΔW;

[0035] During training, only the parameters of B and A are updated, while W0 remains frozen.

[0036] During the LoRA inference phase, LoRA modules are superimposed during computation: when the input x passes through the linear layer, W0 and weights with LoRA adjustments are used: (W0+ΔW)x;

[0037] Where x represents the input tensor of the current layer, and ΔW=B×A is the supplementary weight from the LoRA module;

[0038] The LoRA module can affect the final output without interfering with the main branch.

[0039] Step 5.3: Organize expert annotators to review and correct the extraction results of the large language model, and input the correction information back into the large language model to achieve adaptive optimization based on human-machine collaboration mechanism.

[0040] Beneficial effects:

[0041] 1. Leveraging data and information from the petroleum sector, this approach mines entity relationships and automatically updates them, enabling rapid and efficient construction of a knowledge graph from raw data, thus reducing the cost of manual annotation. Through the understanding and generation capabilities of a large language model, along with self-reflection and consistency verification mechanisms, the integrity and reliability of petroleum terminology are ensured. Incremental updates and feedback learning are employed to achieve dynamic evolution and domain-specific adaptability. Compared to existing methods, this approach enables the rapid construction and dynamic updating of a knowledge graph for the petroleum business domain at a lower cost.

[0042] 2. This invention significantly reduces the cost of manual annotation and rule design, improves the accuracy and timeliness of knowledge extraction, shortens the generation cycle of knowledge graphs, and provides efficient support for intelligent decision-making and knowledge services in the petroleum industry.

[0043] 3. This invention achieves the construction and dynamic updating of a knowledge graph for petroleum operations through the collaborative work of a data acquisition and preprocessing module, a knowledge extraction module, a knowledge consistency verification module, a knowledge storage and management module, and a knowledge update and feedback learning module. This invention solves the problems of high labor costs, low efficiency, and complex processes that exist in previous domain knowledge graph construction tasks, and provides a new, scientific, and simple method for the construction of knowledge graphs for petroleum operations. Attached Figure Description

[0044] Figure 1 This is an overall flowchart of the end-to-end petroleum business knowledge graph construction method based on a large language model according to an embodiment of the present invention;

[0045] Figure 2 This is an overall framework diagram of an embodiment of the present invention;

[0046] Figure 3 This is a flowchart illustrating the data collection and preprocessing module according to an embodiment of the present invention;

[0047] Figure 4 This is a flowchart illustrating the knowledge extraction module according to an embodiment of the present invention;

[0048] Figure 5 This is a flowchart illustrating the knowledge consistency verification module according to an embodiment of the present invention;

[0049] Figure 6 This is a model structure diagram of the thought chain reasoning technology prompt according to an embodiment of the present invention;

[0050] Figure 7 This is an overall schematic diagram of the self-reflection mechanism according to an embodiment of the present invention;

[0051] Figure 8 This is a schematic diagram of the knowledge incremental update and feedback learning process according to an embodiment of the present invention;

[0052] Figure 9 This is a schematic diagram of the LoRA fine-tuning structure according to an embodiment of the present invention;

[0053] Figure 10 This is a schematic diagram of a local knowledge graph constructed using "Well A" and "Well B" as examples. Detailed Implementation

[0054] The present invention will be further described below with reference to the accompanying drawings:

[0055] This end-to-end petroleum business knowledge graph construction method based on a large language model consists of the following: A data acquisition and preprocessing module is constructed to obtain multi-source data from the petroleum business domain. Data format is unified through data cleaning, deduplication, and standardization. A retrieval-enhanced generation mechanism is used to form rich corpus information. A knowledge extraction module is constructed, designing proprietary prompt templates and leveraging the understanding and generation capabilities of the large language model to extract knowledge from the input data, generating triplet knowledge structures of (entity, attribute, entity) and (entity, relation, entity). A knowledge consistency verification module is constructed, designing a thought chain reasoning technique to decompose complex problems into sub-problems and solve them step by step. The solution guides the large language model to display the reasoning process, designs a self-reflection mechanism, conducts multiple rounds of logical review and fact verification, analyzes the generated results, and corrects and optimizes duplicate entities and semantically conflicting relationships; it constructs a knowledge storage and management module to write accurate structured knowledge into a graph database and perform persistent storage and management; it designs a knowledge update and feedback learning module to fine-tune the large language model based on new input data and changed data, optimizes the model's semantic extraction capability while ensuring efficiency, automatically updates existing entity nodes and relational structures, and combines a human-computer feedback mechanism to achieve dynamic evolution and adaptive updating of the knowledge graph.

[0056] Specifically as follows:

[0057] Step 1: Construct a data acquisition and preprocessing module to collect and process multi-source data from the petroleum industry. Through data cleaning, deduplication, and standardization, remove noisy data, correct errors, and unify data formats. Utilize a vectorized embedding model to convert text into vector representations. Employ a retrieval-enhanced generation mechanism to retrieve semantically similar fragments, then reorder and merge them with the original information to form a rich corpus.

[0058] Step 2: Construct a knowledge extraction module. To reduce ambiguity and randomness in the generation of large language models and achieve high-quality extraction with a small sample size, a petroleum business domain-specific prompt template is designed. The task content is clearly defined, and knowledge is extracted from the input data to generate triple knowledge structures of (entity, attribute, entity) and (entity, relation, entity). Specifically, this includes:

[0059] Step 2.1: Construct instruction prompt templates to guide the large language model and automatically identify key entities in the petroleum business.

[0060] Design the instruction template structure based on the task description, format constraints, and semantic guidance. Adhere to the principles of scalability, interpretability, and contextual relevance, accurately distinguishing entity types, relationship categories, and logical structures. The task description clearly states the task objective, including "Extract key entities and relationships from the following petroleum industry text"; format constraints specify the output format, including "Output format is JSON"; semantic guidance provides petroleum business vocabulary or examples, including "Entity types include oil wells, formations, equipment, operation types, etc."

[0061] Step 2.2: After processing the text by word segmentation and block segmentation, input it into the large language model according to the instruction template for context encoding; generate logical relationships between entities based on semantic dependency analysis and context attention mechanism, and complete the identification and extraction of semantic association relationships between different entities;

[0062] Step 2.3: Output the extracted structure as a parsable triple structure and save it in a specific format file for later input into the knowledge graph.

[0063] Step 3: Construct a knowledge consistency verification module, design Chain-of-Thought (CoT) technology and Self-Reflection mechanism to detect duplicate entities and semantic conflict relationships in the extraction results.

[0064] Step 3.1: Use batch pre-checking to quickly eliminate obvious errors and low-quality knowledge organization structures, reducing the load on subsequent chain reasoning.

[0065] Step 3.2: Perform conflict detection. Using semantic similarity and ontology constraint rules, detect issues such as entity duplication, attribute conflict, relationship contradiction, and temporal consistency.

[0066] Step 3.3: Design the mind chain reasoning technique. By showing a small number of examples to the large language model, a complex problem is decomposed into a series of sub-problems and solved sequentially, which significantly improves the performance of the large language model.

[0067] A complete thought chain prompt consists of three parts: instructions, logical basis, and examples. Instructions describe the problem and inform the large language model of the output format; the logical basis refers to the intermediate reasoning process, which can include the solution to the problem, intermediate reasoning steps, and any external knowledge related to the problem; examples provide the large language model with the basic format of input-output pairs in a few-sample manner. Each example includes: the problem, the reasoning process, and the answer. Explaining the reasoning process in the examples encourages the large language model to display the reasoning process during validation, thereby guiding more accurate results.

[0068] Step 3.4: Introduce a self-reflection mechanism to logically review and verify the facts of the knowledge triples based on the multi-round reasoning ability of the large language model.

[0069] By using instruction information that includes logical consistency checks, numerical correctness verification, factual accuracy checks, and multi-angle cross-validation, the model is guided to re-analyze the generated results from a self-evaluation perspective.

[0070] Execute a multi-round chain reasoning and reflection mechanism to regenerate a rationalized result, and compare the revised result with the original result. If there is still ambiguity, manual review is recommended.

[0071] Step 4: Construct a knowledge storage and management module. Automatically import the structured triplet knowledge data that has passed consistency verification into the graph data system, and assign a unique ID to each entity, relation, and attribute. A multi-dimensional index structure is recommended to initially establish a complete knowledge base.

[0072] Step 5: Construct a knowledge increment update and feedback learning module, and perform LoRA fine-tuning on the large language model based on new input data and changed data. Specifically, this includes:

[0073] Step 5.1: Periodically scan the newly input data source, retrieve the changed nodes and newly input entity nodes, and activate the enhanced update task;

[0074] Step 5.2: Fine-tune the large language model using LoRA based on the newly processed data. Leveraging the flexibility and reusability of LoRA, efficiency can be effectively guaranteed even with a small amount of new data, thus optimizing the model's semantic extraction capabilities in the petroleum business.

[0075] In large language models, the weight matrix is ​​typically very large, making direct fine-tuning extremely resource-intensive. LoRA fine-tuning addresses this by:

[0076] LoRA fine-tuning adjusts the parameters of the low-rank matrix while maintaining the original parameter matrix. Unchanged, among which Let represent a low-rank space of dimension d×d;

[0077] Two smaller matrices are introduced alongside the original pre-trained language model. sum matrix ,in Describe the low-rank space of matrix B. Let r represent the low-rank space of matrix A, r represent the rank of LoRA, and d represent the input dimension of the original linear layer.

[0078] The product of matrix B and matrix A, ΔW = B × A, is used as an additional adjustment term for W0;

[0079] The overall weight is: W0 + ΔW;

[0080] During training, only the parameters of B and A are updated, while W0 remains frozen, meaning that a complex transformation is approximated using fewer parameters.

[0081] The LoRA inference phase overlays LoRA modules during computation.

[0082] When the input x passes through the linear layer, W0 and weights with LoRA adjustment terms are used: (W0+ΔW)x;

[0083] Where x represents the input tensor of the current layer, and ΔW=B×A is the supplementary weight from the LoRA module;

[0084] The LoRA module can affect the final output without interfering with the main branch.

[0085] Step 5.3: Organize expert annotators to review and correct the model extraction results, and input the correction information back into the model to achieve adaptive optimization based on human-machine collaboration mechanism.

[0086] Example:

[0087] like Figure 1 and Figure 2 As shown, this end-to-end petroleum business knowledge graph construction method based on a large language model includes:

[0088] Step 1: Collect and process multi-source data in the petroleum sector, such as... Figure 3 As shown, it specifically includes:

[0089] Step 1.1: Collect text data such as exploration reports, equipment operation records, daily production reports, reservoir analysis documents, and safety specifications;

[0090] Step 1.2: Use data cleaning, deduplication, and standardization techniques to remove noisy data, correct errors, and standardize data formats to ensure data quality and consistency.

[0091] Step 1.3: Using a retrieval enhancement generation mechanism, semantically similar fragments are retrieved, and reranking is used to splice and fuse them with the original information to provide contextual enhancement information and form rich corpus information.

[0092] The data acquisition and preprocessing module can comprehensively aggregate, standardize, and structure multi-source data from petroleum operations, unifying them into high-quality and traceable input corpora. This ensures the accuracy and stability of the subsequent modeling process, laying a data foundation for the entire knowledge graph construction.

[0093] Step 2: Construct a knowledge extraction module. To reduce ambiguity and randomness in the generation of large language models and achieve high-quality extraction with a small sample size, design a prompt template specific to the petroleum business domain, clearly define the task content, extract knowledge from the input data, and generate triplet knowledge structures of (entity, attribute, entity) and (entity, relation, entity). For example... Figure 4 As shown, the specific steps include:

[0094] Step 2.1: Construct instruction prompt templates to guide the large language model and automatically identify key entities in the petroleum business.

[0095] The template structure should be designed based on task descriptions, format constraints, and semantic guidance. Instruction prompt templates should adhere to the principles of scalability, interpretability, and contextual relevance, accurately distinguishing entity types, relationship categories, and logical structures.

[0096] The task description should clearly state the task objective, including "extract key entities and relationships from the following petroleum-related texts"; format constraints should specify the output format, including "output format is JSON"; semantic guidance should provide petroleum business vocabulary or examples, including "entity types include oil wells, formations, equipment, operation types, etc.", to help the large language model better understand domain terminology.

[0097] The instruction prompt template is as follows:

[0098] Task: Extract key entities and relationships from the following petroleum-related text.

[0099] Requirements: 1. Output format should be JSON;

[0100] 2. Entity types to be identified include: oil wells, formations, equipment, operation types, and production parameters;

[0101] 3. Relationship types include: Located_in, Uses, Produced_from, and Affected_by;

[0102] 4. For terms containing polysemous words, determine the unique entity by combining the semantic context.

[0103] Example: Input: Well A is located in the southern Tarim Basin. In 2020, fracturing operations were performed, with a daily oil production of 50 tons.

[0104] Output:

[0105] { "Entity": [

[0106] {"Name": "Well A", "Type": "Oil Well"},

[0107] {"Name": "Southern Tarim Basin", "Type": "Geographical Region"},

[0108] {"Name": "Fracturing Operation", "Type": "Operation Type"},

[0109] {"Name": "50 tons", "Type": "Production"}

[0110] ],

[0111] "relation": [

[0112] {"Subject": "Well A", "Relation": "Located in", "Object": "Southern Tarim Basin"},

[0113] {"Subject": "Well A", "Relationship": "Use", "Object": "Fracturing Operation"},

[0114] {"Subject": "Well A", "Relation": "Daily Oil Production", "Object": "50 tons"} ]

[0116] }

[0117] Step 2.2: After segmenting the processed text into words, sentences, and blocks, input it into the large language model and perform context encoding according to the instruction template. To better parse the text semantics, a deep Transformer structure is adopted to automatically identify key entities such as well groups, formations, drilling, fracturing, and flow rates. Based on semantic dependency analysis and context attention mechanisms, logical relationships between entities are generated, such as:

[0118] "Well A employs fracturing operation" → Relationship: Oil well — employs → fracturing;

[0119] "The oil well is produced from a certain oil field" → Relationship: oil well — produced from → oil field.

[0120] Step 2.3: Output the extracted structure as a parsable triple structure and store it in a JSON file for subsequent input into the knowledge graph. To standardize the output results, regularization is used to standardize the output format to a triple form of (entity, relation, entity).

[0121] Step 3: Construct a knowledge consistency verification module. To automatically detect semantic conflicts and reduce the cost of manual review, this module corrects errors during self-checking, improving credibility. It introduces thought chain reasoning technology and a self-reflection mechanism to decompose complex problems into sub-problems and solve them step by step. Simultaneously, it performs multiple rounds of logical review and factual verification, retrieving duplicate entities and semantically conflicting relationships, and then correcting and optimizing them. For example... Figure 5 As shown, it specifically includes:

[0122] Step 3.1: Use batch pre-checking to quickly eliminate obvious errors and low-quality knowledge organization structures, reducing the load on subsequent chain reasoning.

[0123] Batch pre-checking includes removing erroneous entities, standardizing attribute formats, and converting relevant units. For example, the unclear identifier "#" in "well #" is clearly not a specific well number and needs to be removed; similarly, numerical attributes such as "depth 3km", "depth 3000 meters", and "depth = 3km" are uniformly converted to standard units. Through batch pre-checking, duplicate entities and incomplete entity information can be removed, reducing the load on subsequent verification.

[0124] Step 3.2: Perform conflict detection. Using semantic similarity and ontology constraint rules, detect issues such as entity duplication, attribute conflict, relationship contradiction, and temporal consistency.

[0125] Since "page A well" and "well page A" are synonymous entities, they need to be mapped to a unified identifier.

[0126] The "casing depth of Well B" is 3000m and 2980m respectively. When the same entity and the same attribute have different values, it is necessary to mark whether they are correct.

[0127] "Fracturing" and "cement sealing" are mutually exclusive and contradictory. Based on the knowledge ontology of the petroleum field, logical conflicts are detected.

[0128] In the business documents of geological reports, "equipment replacement" appears before "equipment scrapping". This is a sequential relationship. When the sequence is unreasonable, there should be a consistency check.

[0129] Step 3.3: Based on the thinking chain reasoning technique, by showing a small number of examples to the large language model, a complex problem is decomposed into a series of sub-problems and solved sequentially, which significantly improves the performance of the large language model.

[0130] like Figure 6 The MindChain reasoning technique breaks down a multi-step reasoning problem into many intermediate steps, allocates more computational resources to them, generates more tokens, and then concatenates these answers together to solve the problem.

[0131] The Mind Chain Reasoning technique guides the model by breaking down a complex problem into multiple steps, providing a window into why the model makes mistakes and saving the expense of retraining the model. By showing a small number of examples to a large language model and explaining the reasoning process in the examples, the large language model also displays the reasoning process when answering prompts. This explanation of reasoning leads to more accurate results.

[0132] Step 3.4: Introduce a self-reflection mechanism, such as... Figure 7 Based on the multi-round reasoning capabilities of the large language model, the knowledge triples are logically reviewed and factually verified.

[0133] The self-reflection mechanism includes instructions such as logical consistency checks, numerical correctness verification, factual accuracy checks, and multi-angle cross-validation, which guide the model to re-analyze the generated results from a self-evaluation perspective and improve the quality of the model's output.

[0134] Step 4: Construct a knowledge storage and management module. Automatically import the structured triple knowledge data that has passed consistency verification into a graph data system, such as the Neo4j database, and assign a unique ID to each entity, relation, and attribute. A multi-dimensional index structure is recommended. The initial establishment of a complete knowledge base is achieved by establishing multi-level links between knowledge nodes based on a vectorized semantic retrieval mechanism.

[0135] Step 5: Construct a knowledge update and feedback learning module. To significantly reduce memory and computational costs, achieve rapid knowledge updates, ensure knowledge timeliness, and dynamically expand the knowledge graph, LoRA fine-tuning is performed on the large language model based on new input and changed data. While maintaining efficiency, the model's semantic extraction capabilities are optimized, and existing entity nodes and relational structures are automatically updated. Combined with a human-computer feedback mechanism, this enables the dynamic evolution and adaptive updating of the knowledge graph. Figure 8 As shown, it specifically includes:

[0136] Step 5.1: Periodically scan the newly input data source, retrieve the changed nodes and newly input entity nodes, and activate the enhanced update task;

[0137] The domain fine-tuning dataset comes from newly collected corpora and samples of changed data, such as new equipment descriptions and new fracturing process documents.

[0138] The newly retrieved samples need to undergo structure transformation in steps 1 and 2 before being used for incremental training.

[0139] Step 5.2: Fine-tune the large language model using LoRA based on the newly processed data to optimize the model's semantic extraction capabilities in petroleum business scenarios;

[0140] like Figure 9 As shown, the LoRA fine-tuning strategy is a low-resource approach for fine-tuning large language models, with training parameters being only one ten-thousandth of the total parameters.

[0141] The core idea of ​​LoRA is to inject trainable low-rank decomposition matrices into each layer of the Transformer architecture after freezing the weights of the pre-trained model, thereby greatly reducing the number of trainable parameters on downstream tasks.

[0142] like Figure 9 As shown, LoRA restricts the pre-trained weight matrix. The update method transforms the incremental parameter matrix ΔW of the fully parameter-tuned matrix into a low-rank approximation of two smaller matrices, B and A:

[0143] W0 + ΔW = W0 + B × A

[0144] in, and The weight matrix for low-rank adaptation of LoRA, rank r much smaller d .

[0145] At this point, the number of parameters to be fine-tuned has changed from d×d of W0 to 2×r×d of matrices B and A. Clearly, 2×r×d... < d×d (2 r<d ).

[0146] Given input Output after adding LoRA :

[0147]

[0148] Set Δh = B × Ax for subsequent calculations. During training, the original parameters W0 are frozen; that is, although they participate in forward and backward propagation, their corresponding gradients are not calculated, and their parameters are not updated.

[0149] matrix Initialize to all zeros, b i =0;

[0150] matrix Initialize using a Gaussian function. , where N(0,σ 2 ) indicates a mean of 0 and a variance of σ. 2 Gaussian distribution;

[0151] If LoRA's bypass B×A=0 before training begins, then fine-tuning can start from the pre-trained weights W0.

[0152] LoRA fine-tuning significantly reduces the hardware performance requirements for model fine-tuning due to the reduced number of weights. During inference, the LoRA weights are directly merged into the pre-trained weights, ensuring inference speed. After fine-tuning, only the newly added weights need to be saved, requiring far less memory compared to storing the entire model's weights.

[0153] Step 5.3: Organize expert annotators to review and correct the model extraction results, and input the correction information back into the model to achieve adaptive optimization based on human-machine collaboration mechanism.

[0154] The expert review includes: misidentifying "industrial oil layer" as an entity identification error in the oil testing method; confusion between "located in" and "belongs to" relationship types; and conflicts between temporal and spatial attributes. Experts can directly modify the results and mark the modifications as "high confidence".

[0155] To continuously optimize, the prompt template is updated or the prompt weight is modified after receiving modified data, thereby automatically strengthening domain expertise.

[0156] The knowledge increment update and feedback learning mechanism can maintain the real-time and accuracy of the knowledge base, realize dynamic evolution and continuous optimization of the model, ensure that new knowledge is automatically absorbed and old knowledge is corrected in the oil business production process, and achieve self-improvement through multiple rounds of human-machine feedback.

Claims

1. A method for constructing an end-to-end petroleum business knowledge graph based on a large language model, characterized in that... Includes the following steps: Step 1: Acquire multi-source data from the petroleum business field, unify the data format through preprocessing, convert the text into vector representation using a vectorized embedding model, and use a retrieval enhancement generation mechanism to retrieve semantically similar fragments, then reorder and splice them with the original information to form rich corpus information. Step 2: Construct a knowledge extraction module, design a prompt template specific to the petroleum business domain, and use a large language model to extract and generate input data under the prompt of instructions, and output the entity relation triple knowledge structure. Step 3: Construct a knowledge consistency verification module, introduce thinking chain reasoning technology and self-reflection mechanism, decompose complex problems into sub-problems and solve them step by step, and at the same time retrieve duplicate entities and semantically conflicting relationships, and correct and optimize them. Step 3.1: Use batch pre-screening to quickly eliminate obviously erroneous and low-quality triplet knowledge structures; Step 3.2: Perform conflict detection. Utilize semantic similarity and ontology constraint rules to detect entity duplication, attribute conflicts, relational contradictions, and temporal consistency. Step 3.3: Based on the thinking chain reasoning technique, by showing examples to the large language model, a complex problem is decomposed into sub-problems step by step and solved sequentially, thereby improving the performance of the large language model; The thought chain prompt consists of three parts: instructions, logical basis, and examples. Instructions describe the problem and inform the output format of the large language model. Logical basis refers to the intermediate reasoning process, including the solution to the problem, intermediate reasoning steps, and any external knowledge related to the problem. Examples provide the basic format of input-output pairs for the large language model. Each example includes: the problem, the reasoning process, and the answer. Step 3.4: Introduce a self-reflection mechanism. Based on the multi-round reasoning ability of the large language model, conduct logical review and factual verification of the triplet knowledge structure. Utilize instruction information that includes logical consistency checks, numerical correctness verification, factual accuracy checks, and multi-angle cross-verification to guide the large language model to re-analyze the generated results from a self-evaluation perspective. A multi-round chain-like reasoning and reflection mechanism is executed to regenerate a rationalized result, and the revised result is compared with the original result. If there is still ambiguity, manual review is conducted. Step 4: Construct a knowledge storage and management module. Automatically import the structured triplet knowledge data that has passed the consistency verification in Step 3 into the graph data system, assign a unique ID to each entity, relation, and attribute, establish a multi-dimensional index structure, and complete the initial establishment of the knowledge graph. Step 5: Construct a knowledge update and feedback learning module. Based on new input data and changed data, perform LoRA fine-tuning on the large language model, automatically update existing entity nodes and relational structures, and combine human-computer feedback mechanisms to achieve dynamic evolution and adaptive updating of the knowledge graph. Step 5.1: Periodically scan the newly input data source, retrieve the changed nodes and newly input entity nodes, and activate the enhanced update task; Step 5.2: Fine-tune the large language model using LoRA based on the newly processed data to optimize its semantic extraction capabilities in petroleum operations; LoRA fine-tuning adjusts the parameters of the low-rank matrix while maintaining the original parameter matrix. Unchanged, among which Let represent a low-rank space of dimension d×d; Two smaller matrices are introduced alongside the original pre-trained language model. sum matrix ,in Representation matrix The low-rank space, Representation matrix The low-rank space of LoRA, where r represents the rank of LoRA and d represents the input dimension of the original linear layer; r represents the rank of LoRA and d represents the input dimension of the original linear layer. matrix sum matrix product As a response Additional adjustment items; The overall weight is: ; During training, only update and The parameters, Keep frozen; The LoRA inference phase overlays LoRA modules during computation: when the input x passes through a linear layer, it uses... And weights with LoRA adjustment terms: ; Where x represents the input tensor of the current layer. These are supplementary weights from the LoRA module; The LoRA module can affect the final output without interfering with the main branch. Step 5.3: Organize expert annotators to review and correct the extraction results of the large language model, and input the correction information back into the large language model to achieve adaptive optimization based on human-machine collaboration mechanism.

2. The end-to-end petroleum business knowledge graph construction method based on a large language model as described in claim 1, characterized in that: Step one specifically involves: Text data was collected, including exploration reports, equipment operation records, daily production reports, reservoir analysis documents, and safety specifications. Data cleaning, deduplication, and standardization were used to remove noise from the text data, correct errors, and unify the data format. A retrieval enhancement generation mechanism was adopted to retrieve semantically similar fragments, and these fragments were reordered and spliced ​​with the original information to provide contextual enhancement information, forming a rich corpus of information.

3. The method for constructing an end-to-end petroleum business knowledge graph based on a large language model according to claim 2, characterized in that: Step two specifically involves: Step 2.1: Construct instruction prompt templates to guide the large language model and automatically identify key entities in petroleum operations. Instruction prompt templates include task descriptions, format constraints, and semantic guidance. The task description clearly states the task objectives, including: extracting key entities and relationships from the following petroleum-related texts; Format constraints specify the output format, including: the output format is JSON; Semantic guidance provides vocabulary or examples for oil business, including: entity types such as wells, formations, equipment, and operation types; Step 2.2: After segmenting and dividing the text data into words, input it into the large language model according to the instruction prompt template for context encoding; generate logical relationships between entities based on semantic dependency analysis and context attention mechanism, and complete the identification and extraction of semantic association relationships between different entities; Step 2.3: Output the extracted structure as a triplet knowledge structure and store it.

4. The method for constructing an end-to-end petroleum business knowledge graph based on a large language model according to claim 3, characterized in that: The knowledge storage and management module includes knowledge entry and semantic indexing and retrieval optimization.

Images