Intelligent query method for structured data in field of oil-gas exploration

Abstract

The invention discloses an intelligent query method for structured data in the field of oil-gas exploration, which innovatively constructs initial seed data and a'thinking chain 'strategy expansion data set aiming at the intelligent query problem of the structured data in the field of oil-gas exploration. On the basis of a large natural language processing (NLP) model with a large amount of world knowledge, a low-rank adaptation algorithm (LoRA) is applied to carry out fine adjustment on the model, and meanwhile, by means of assistance of a vectorization knowledge base, query of data in the field of oil-gas exploration can be completed while the world knowledge is used. Compared with a traditional method, the method has remarkable advantages in the aspects of question-answering performance, accuracy and robustness, and a new and effective data question-answering scheme is provided for the field of oil-gas exploration.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, specifically relating to an intelligent query method for structured data in the field of oil and gas exploration. Background Technology

[0002] Artificial intelligence (AI) technology has undergone more than 70 years of development, and in recent years it has experienced explosive growth and been widely applied in multiple fields. However, due to the characteristics of specialized data in the oil and gas sector—such as multiple knowledge carriers, high professional thresholds, strong reliance on experience, and complex reasoning processes—the amount of data is insufficient to meet the requirements of deep learning, making it difficult to obtain training samples for machine learning. Overall, AI technology research and development is still in its early exploratory stage, with few practical applications. There is an urgent need to tackle the challenges of multi-source, multi-modal data aggregation and multi-professional business integration application scenarios to improve intelligent capabilities and enhance the accuracy, level, efficiency, and effectiveness of oil and gas operations. The recent emergence of pre-trained large models such as GPT has demonstrated powerful general knowledge and certain reasoning capabilities, attracting widespread attention. How to utilize emerging technologies such as pre-trained large models for intelligent querying of structured data in the oil and gas exploration field is a hot topic of focus in the intelligent transformation of oil and gas exploration. Summary of the Invention

[0003] The purpose of this invention is to provide an intelligent query method for structured data in the field of oil and gas exploration, thereby improving question-answering performance, accuracy, and robustness.

[0004] The objective of this invention is achieved through the following technical means: a method for intelligent querying of structured data in the field of oil and gas exploration, comprising the following steps:

[0005] S1, Question Set, which collects data query types and corresponding query questions;

[0006] S2. Construct a seed dataset. Generate an SQL query statement for each collected question. The questions and their corresponding SQL query statements form the seed dataset.

[0007] S3. Define SQL generation rules for the large model and convert them into a Prompt for the large model;

[0008] S4. Input the Prompt and seed dataset into the generative language model to generate new questions and corresponding SQL statements, and integrate the newly generated questions and statements into the seed dataset.

[0009] S5. Construct a vector knowledge base by vectorizing the questions and corresponding SQL query statements in the seed dataset and storing them in the vector database.

[0010] S6. Use the Langchain framework to integrate large models with vectorized knowledge bases to build a structured data intelligent query system, ensuring compatibility between components and smooth data flow.

[0011] S7. Based on the Langchain framework, a vectorized knowledge base is used to assist the large model. According to the new question raised by the user, several most similar answers are selected from the vector database through similarity calculation and returned to the large model.

[0012] S8. Similar answers found by the large model reference are combined with the prompt to generate an SQL query statement for the current question.

[0013] The seed dataset in S2, after being formed, also performs a quality assessment on the query statements, deleting erroneous and invalid SQL query statements and their corresponding problems.

[0014] In step S5, the specific method for constructing vector data is as follows:

[0015] Select NL2SQL question-answer pairs from the dataset constructed by S4 as the initial content of the knowledge base;

[0016] Each selected question-answer pair (Q, SQL) is vectorized to obtain its corresponding vectorized representation v = Vectorize(Q, SQL);

[0017] Store the vectorized representations of all selected question-and-answer pairs in a vectorized database.

[0018] In S6, the purpose of the Langchain framework is to provide tools and interfaces for large models to call knowledge bases.

[0019] In S7, the method for selecting the most similar answer is:

[0020] When a user asks a new question Q new When, calculate Q according to the following formula. new Similarity to each question-answer pair in the knowledge base:

[0021]

[0022] Based on the calculated similarity, the Top-K most similar question-answer pairs are selected as reference information, where K is a value specified by the user.

[0023] After obtaining several most similar question-answer pairs, the K most similar question-answer pairs and the user's new question are input into the generative language model to guide the model in generating more accurate SQL statements.

[0024] In S1, questions are classified according to data type, single-round question and answer, and multi-round question and answer.

[0025] In S1, the data query types include date data, numerical data, vector data, and text data.

[0026] The Prompt specifically refers to,

[0027] Identify key information in the problem, such as the main entity being queried and any related conditions;

[0028] Determine the specific information or target the user wants to query;

[0029] Based on the content of the question, determine any necessary screening criteria or constraints;

[0030] Identify the fields involved and possible functions, and recognize the database table fields mentioned in the question.

[0031] The Prompt also includes,

[0032] Consider grouping and sorting: Identify the requirements in the problem and decide whether the query results need to be grouped or sorted;

[0033] Assess whether the problem requires generating complex SQL statements containing nested queries or loops; if so, plan to use recursive queries or common table expressions.

[0034] The beneficial effects of this invention are as follows: Addressing the intelligent query problem of structured data in the oil and gas exploration field, it innovatively constructs initial seed data and expands the dataset using a "thinking chain" strategy. Based on a large-scale natural language processing (NLP) model possessing abundant world knowledge, it applies the low-rank adaptation algorithm (LoRA) for model fine-tuning. Simultaneously, with the assistance of a vectorized knowledge base, it can complete data queries in the oil and gas exploration field while utilizing world knowledge. Compared to traditional methods, this method demonstrates significant advantages in question-answering performance, accuracy, and robustness, providing a new and effective data question-answering solution for the oil and gas exploration field. Attached Figure Description

[0035] Figure 1 A flowchart of a structured data intelligent query method in the field of oil and gas exploration;

[0036] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Detailed Implementation

[0037]

Example 1

[0038] A method for intelligent querying of structured data in the field of oil and gas exploration includes the following steps:

[0039] S1, Question Set, which collects data query types and corresponding query questions;

[0040] In step S1, the data query types include date data, numeric data, vector data, and text data. Other data types can also be added as needed.

[0041] In S1, questions are classified according to data type, single-round question and answer, and multi-round question and answer.

[0042] S2. Construct a seed dataset. Generate an SQL query statement for each collected question. The questions and their corresponding SQL query statements form the seed dataset. Ensure that the SQL statements can accurately respond to the data requirements of the questions.

[0043] The seed dataset in S2, after being formed, also performs a quality assessment on the query statements, deleting erroneous and invalid SQL query statements and their corresponding problems.

[0044] S3. Define SQL generation rules for the large model and convert them into a Prompt for the large model;

[0045] Define predefined strategies and rules for dataset expansion, and use a thought process approach to guide the generation of problems and corresponding SQL statements for large models.

[0046] When defining SQL generation rules for large models, consider the following aspects:

[0047] 1) Analyze the relevance of the problem:

[0048] The large model compares the questions asked by users with the context and historical dialogue records to determine the topic and context of the questions.

[0049] Leverage the semantic understanding capabilities of large models to identify key information and intent within a problem.

[0050] 2) Determine the query target:

[0051] Clearly define the specific information or target of the user's query, such as a date range, a specific value, or categorized data.

[0052] Extract keywords from the question, such as "maximum value", "average value", "total number", etc., to determine the aggregation target of the query.

[0053] 3) Determine the screening criteria:

[0054] Based on the question, identify the filtering conditions or constraints that need to be applied, such as time range, values ​​of specific fields, etc.

[0055] This identifies the filtering conditions in the question and converts them into the WHERE clause in an SQL query.

[0056] 4) Identify the fields involved and possible functions:

[0057] Analyze the fields mentioned in the problem to determine the database table fields that need to be queried.

[0058] Based on the field types and query requirements, determine the SQL functions that may be used, such as SUM, AVG, COUNT, etc.

[0059] 5) Consider grouping and sorting:

[0060] Depending on the requirements of the problem, determine whether it is necessary to group (GROUP BY clause) or sort (ORDER BY clause) the query results.

[0061] Analyze whether the question contains keywords related to grouping and sorting, such as "group by year" or "sort by month".

[0062] 6) Consider whether to use nested loops:

[0063] Assess whether the problem requires generating complex SQL statements containing nested queries or loops.

[0064] If necessary, use recursive queries or common table expressions (CTEs) to construct SQL statements.

[0065] 7) Transform the above rules into a Prompt for large models:

[0066] The Prompt specifically refers to,

[0067] Identify key information in the problem, such as the main entity being queried and any related conditions;

[0068] Determine the specific information or target the user wants to query;

[0069] Based on the content of the question, determine any necessary screening criteria or constraints;

[0070] Identify the fields involved and possible functions, and recognize the database table fields mentioned in the question.

[0071] The Prompt also includes,

[0072] Consider grouping and sorting: Identify the requirements in the problem and decide whether the query results need to be grouped or sorted;

[0073] Assess whether the problem requires generating complex SQL statements containing nested queries or loops; if so, plan to use recursive queries or common table expressions.

[0074] The above rules are then translated into a series of prompts that guide the large language model in generating SQL statements, for example:

[0075] "In order to generate an accurate SQL query, I need you to follow these steps:"

[0076] 1. Analyze the relevance of the question: Carefully read the user's question to understand their actual needs and the context of the question. Identify key information in the question, such as the main entity being queried and any relevant conditions.

[0077] 2. Define the query objective: Clearly define the specific information or target the user wants to query. This may involve identifying keywords in the question, such as "maximum value," "average value," or specific data points.

[0078] 3. Define the filtering criteria: Based on the content of the question, determine any necessary filtering criteria or constraints. This may include time ranges, specific values, or other limitations.

[0079] 4. Identify the fields involved and possible functions: Identify the database table fields mentioned in the problem and determine the SQL functions that may be used, such as SUM, AVG, COUNT, etc.

[0080] 5. Consider grouping and sorting: Based on the needs of the problem, decide whether to group or sort the query results. Identify keywords in the problem, such as "group by year" or "sort by month".

[0081] 6. Consider nested loops: Assess whether the problem requires generating complex SQL statements containing nested queries or loops. If so, plan how to use recursive queries or common table expressions (CTEs).

[0082] Please use these steps as guidance to generate an SQL query to answer the following user question:

[0083] S4. Input the Prompt and seed dataset into the generative language model to generate new questions and corresponding SQL statements, and integrate the newly generated questions and statements into the seed dataset.

[0084] Generative language models such as ChatGPT4.0 input seed datasets into the large model to generate new questions and corresponding SQL statements. The generated questions and corresponding SQL statements are then corrected and optimized to ensure their accuracy. The newly generated data and seed datasets are then integrated to form a dataset for subsequent fine-tuning of the large model.

[0085] After selecting a large generative language model, it is necessary to fine-tune the large model using the LoRA method.

[0086] While keeping the parameters of the backbone model frozen, inject a trainable low-rank decomposition matrix into each layer of the model;

[0087] Freeze the original pre-training weights of the pre-trained model;

[0088] The model was trained using a seed dataset built with S4, and then retrained with minor parameter adjustments to enhance its understanding of industry data. Only the parameters of the LoRA layer were updated.

[0089] The fine-tuned model was tested to ensure its high accuracy and efficiency in data retrieval in the oil and gas exploration field.

[0090] It's important to note that maintaining the backbone model parameters is crucial during the fine-tuning of the large model in the LoRA method. This preserves the model's generalization ability across a wide range of data, while allowing the fine-tuning to focus on generating SQL statements for the oil and gas sector. This approach effectively balances the model's versatility with customization needs.

[0091] S5. Construct a vector knowledge base by vectorizing the questions and corresponding SQL query statements in the seed dataset and storing them in the vector database.

[0092] In step S5, the specific method for constructing vector data is as follows:

[0093] Select NL2SQL question-answer pairs from the dataset constructed by S4 as the initial content of the knowledge base;

[0094] Each selected question-answer pair (Q, SQL) is vectorized to obtain its corresponding vectorized representation v = Vectorize(Q, SQL);

[0095] Store the vectorized representations of all selected question-and-answer pairs in a vectorized database.

[0096] S6. Use the Langchain framework to integrate large models with vectorized knowledge bases to build a structured data intelligent query system, ensuring compatibility between components and smooth data flow.

[0097] S7. Based on the Langchain framework, a vectorized knowledge base is used to assist the large model. According to the new question raised by the user, several most similar answers are selected from the vector database through similarity calculation and returned to the large model. After obtaining several most similar question-answer pairs, the K most similar question-answer pairs and the user's new question are input into the generative language large model to guide the model to generate more accurate SQL statements.

[0098] In S6, the Langchain framework provides tools and interfaces for large models to call knowledge bases.

[0099] In S7, the method for selecting the most similar answer is:

[0100] When a user asks a new question Qnew When, calculate Q according to the following formula. new Similarity to each question-answer pair in the knowledge base:

[0101]

[0102] Based on the calculated similarity, the Top-K most similar question-answer pairs are selected as reference information, where K is a value specified by the user.

[0103] Match the user's selected question-and-answer pair with the user's new question Q. new The input is fed into the LoRA-tuned model to guide it in generating more accurate SQL statements.

[0104] S8. Similar answers found by the large model reference are combined with the prompt to generate an SQL query statement for the current question.

Claims

1. A method for intelligent querying of structured data in the field of oil and gas exploration, characterized in that: Includes the following steps, S1, Question Set, which collects data query types and corresponding query questions; S2. Construct a seed dataset. Generate an SQL query statement for each collected question. The questions and their corresponding SQL query statements form the seed dataset. S3. Define SQL generation rules for the large model and convert them into a Prompt for the large model; S4. Input the Prompt and seed dataset into the generative language model to generate new questions and corresponding SQL statements, and integrate the newly generated questions and statements into the seed dataset. S5. Construct a vector knowledge base by vectorizing the questions and corresponding SQL query statements in the seed dataset and storing them in the vector database. S6. Use the Langchain framework to integrate large models with vectorized knowledge bases to build a structured data intelligent query system, ensuring compatibility between components and smooth data flow. S7. Based on the Langchain framework, a vectorized knowledge base is used to assist the large model. According to the new question raised by the user, several most similar answers are selected from the vector database through similarity calculation and returned to the large model. S8. Similar answers found by the large model reference are combined with the prompt to generate an SQL query statement for the current question.

2. The method for intelligent querying of structured data in the field of oil and gas exploration according to claim 1, characterized in that: The seed dataset in S2, after being formed, also performs a quality assessment on the query statements, deleting erroneous and invalid SQL query statements and their corresponding problems.

3. The method for intelligent querying of structured data in the field of oil and gas exploration according to claim 1, characterized in that: In step S5, the specific method for constructing vector data is as follows: Select NL2SQL question-answer pairs from the dataset constructed by S4 as the initial content of the knowledge base; Each selected question-answer pair (Q, SQL) is vectorized to obtain its corresponding vectorized representation v = Vectorize(Q, SQL); Store the vectorized representations of all selected question-and-answer pairs in a vectorized database.

4. The method for intelligent querying of structured data in the field of oil and gas exploration according to claim 1, characterized in that: In S6, the purpose of the Langchain framework is to provide tools and interfaces for large models to call knowledge bases.

5. The method for intelligent querying of structured data in the field of oil and gas exploration according to claim 1, characterized in that: In S7, the method for selecting the most similar answer is: When a user asks a new question Q new When, calculate Q according to the following formula. new Similarity to each question-answer pair in the knowledge base: Based on the calculated similarity, the Top-K most similar question-answer pairs are selected as reference information, where K is a value specified by the user.

6. A method for intelligent querying of structured data in the field of oil and gas exploration according to claim 1 or 5, characterized in that: After obtaining several most similar question-answer pairs, the K most similar question-answer pairs and the user's new question are input into the generative language model to guide the model in generating more accurate SQL statements.

7. The method for intelligent querying of structured data in the field of oil and gas exploration according to claim 1, characterized in that: In S1, questions are classified according to data type, single-round question and answer, and multi-round question and answer.

8. The method for intelligent querying of structured data in the field of oil and gas exploration according to claim 1, characterized in that: In S1, the data query types include date data, numerical data, vector data, and text data.

9. The method for intelligent querying of structured data in the field of oil and gas exploration according to claim 1, characterized in that: The Prompt specifically refers to, Identify key information in the problem, such as the main entity being queried and any related conditions; Determine the specific information or target the user wants to query; Based on the content of the question, determine any necessary screening criteria or constraints; Identify the fields involved and possible functions, and recognize the database table fields mentioned in the question.

10. The method for intelligent querying of structured data in the field of oil and gas exploration according to claim 9, characterized in that: The Prompt also includes, Consider grouping and sorting: Identify the requirements in the problem and decide whether the query results need to be grouped or sorted; Assess whether the problem requires generating complex SQL statements containing nested queries or loops; if so, plan to use recursive queries or common table expressions.

Images