A method for constructing a large model for reservoir dynamic analysis scenarios

Abstract

The present invention relates to the technical field of reservoir dynamic analysis, and specifically discloses a method for constructing a large model for reservoir dynamic analysis scenarios, wherein petroleum engineering professional vocabulary is added to the Chinese‑Alpaca‑2.0‑13B basic model, a new model is obtained by pre-training and merging Low‑RankAdaptation patches, incremental training is performed on the word segmenter after the expanded vocabulary, and the weight parameters in the model are updated by the Low‑RankAdaptation method, and three fine-tuning models are obtained based on the specific actual needs of the oil field, and further instruction fine-tuning and subsystem coupling are performed on Chinese‑Alpaca‑2.0‑13B, and finally a large model for reservoir dynamic analysis scenarios is obtained. Relying on the industry large model to integrate the information of reservoir static data, dynamic data query and calculation results, the analysis suggestion function is realized, and the artificial intelligence large model is preliminarily applied to the field of reservoir dynamic analysis for the first time, providing a reliable technical solution for the application of the large model in reservoir dynamic analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of reservoir dynamic analysis, and in particular to a method for constructing a large model of a reservoir dynamic analysis scenario. Background Art

[0002] At present, the big model is mainly based on the Transformer architecture, integrating large parameters, big data and large computing power. The emergence of a series of big models such as ChatGPT has demonstrated its strong potential and quickly attracted widespread attention from various industries. The big model can not only conduct daily conversations, but also complete complex tasks. The big models in the oil and gas industry are developing rapidly. Various research institutions and enterprises have successively launched big models with industry characteristics. For example, the PetroAI big model developed by the China National Petroleum Exploration and Development Research Institute and the large language model with a built-in 70B (billion) parameter volume have learned more than 260,000 petroleum science and technology documents and more than 100,000 exploration and development knowledge graph triples.

[0003] However, the existing technology has not been able to use its large model in specific application scenarios of reservoir dynamic analysis, and lacks application to actual business needs. Summary of the invention

[0004] The purpose of the present invention is to provide a method for constructing a large model for reservoir dynamic analysis scenarios, aiming to solve the technical problems in the prior art that the large model cannot be used in specific application scenarios of reservoir dynamic analysis and lacks application to actual business needs.

[0005] To achieve the above object, the present invention adopts a method for constructing a large model of a reservoir dynamic analysis scenario, comprising the following steps:

[0006] Step 1: Collect data in the field of petroleum engineering and use the toolkit to process the data accurately and effectively;

[0007] Step 2: Expand the professional vocabulary of the tokenizer in the field of petroleum engineering, expand the tokenizer of petroleum engineering on the Chinese-Alpaca-2.0-13B basic model, load the existing Chinese-Alpaca-2.0-13B basic tokenizer model, and add the petroleum engineering professional vocabulary to the Chinese-Alpaca-2.0-13B basic model;

[0008] Step 3: Pre-train and merge the Low-RankAdaptation patch to obtain a new model. Perform incremental training based on the standardized data in step 1 and the word segmenter after the vocabulary is expanded in step 2. Use the Low-RankAdaptation method to update the weight parameters in the model.

[0009] Step 4: For the models obtained in the above steps, three fine-tuning models are obtained based on the specific actual needs of the oil field: a fine-tuning named entity recognition model, a fine-tuning Text-to-SQL model, and a fine-tuning tool call model based on classification tasks;

[0010] Step 5: Further fine-tune the instructions and couple the subsystems of Chinese-Alpaca-2.0-13B to finally obtain a large model of the reservoir dynamic analysis scenario.

[0011] Wherein, in said step 1:

[0012] PDFminer and PyPDF2 are used to effectively extract and separate text and image information, and the Grobid tool is used to distinguish fonts and identify formulas to ensure that professional knowledge information can be effectively screened and retained;

[0013] Use OCR technology to extract useful text data from images;

[0014] Combine Jieba, NumPy and Pandas tools in Python to further clean the data and make formatting agreements;

[0015] Sentence cutting by Jieba;

[0016] NumPy and Pandas are used for data cleaning and formatting to obtain an unlabeled plain text dataset for incremental pre-training.

[0017] Among them, in the fine-tuning named entity recognition model, the named entity recognition model is fine-tuned to achieve accurate recognition of the task. During the test, the effectiveness and accuracy of the model's named entity recognition are comprehensively evaluated from three aspects: entity recognizable rate, numerical variable entity recognition accuracy, and semantic recognition accuracy.

[0018] Among them, the fine-tuning Text-to-SQL model involves 30 common natural language problems in the field of reservoir dynamic analysis. 256 unique complex SQL query statements constructed on 7 dimensions and their corresponding natural language descriptions are used as the instruction fine-tuning label data set for Text-to-SQL in the field of reservoir dynamic analysis, which is used to enhance the model's conversion capability for SQL statements in the field of oilfield dynamic analysis.

[0019] Among them, in the fine-tuning tool calling model based on classification tasks, the tool calling function is introduced in the large model, and a tool usage fine-tuning data set is constructed for 26 key technical indicators in reservoir dynamic analysis. The fine-tuned model can accurately call tools to complete the corresponding indicator calculations, and can score the called tools based on the classifier classification task, and return the tool label with the highest score, and calculate the tool corresponding to the label based on the Callback function call, and finally return the results of the relevant indicators.

[0020] The beneficial effects of the method for constructing a large model for a reservoir dynamic analysis scenario of the present invention are as follows: using named entity recognition technology to realize subtask identification in reservoir dynamic analysis, based on Text-to-SQL and tool calling technology to meet the specific task requirements of intelligent query, calling, and calculation, relying on the industry large model to integrate reservoir static data, dynamic data query and calculation results information, to realize the analysis suggestion function, for the first time, the artificial intelligence large model is preliminarily applied to the field of reservoir dynamic analysis, and a reliable technical solution is provided for the application of the large model in reservoir dynamic analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 It is a flowchart of the steps of a method for constructing a large model of a reservoir dynamic analysis scenario of the present invention. DETAILED DESCRIPTION

[0023] See also Figure 1 The present invention provides a method for constructing a large model of a reservoir dynamic analysis scenario, comprising the following steps:

[0024] Step 1: Collect data in the field of petroleum engineering and use the toolkit to process the data accurately and effectively;

[0025] Step 2: Expand the professional vocabulary of the tokenizer in the field of petroleum engineering, expand the tokenizer of petroleum engineering on the Chinese-Alpaca-2.0-13B basic model, load the existing Chinese-Alpaca-2.0-13B basic tokenizer model, and add the petroleum engineering professional vocabulary to the Chinese-Alpaca-2.0-13B basic model;

[0026] Step 3: Pre-train and merge the Low-RankAdaptation patch to obtain a new model. Perform incremental training based on the standardized data in step 1 and the word segmenter after the vocabulary is expanded in step 2. Use the Low-RankAdaptation method to update the weight parameters in the model.

[0027] Step 4: For the models obtained in the above steps, three fine-tuning models are obtained based on the specific actual needs of the oil field: a fine-tuning named entity recognition model, a fine-tuning Text-to-SQL model, and a fine-tuning tool call model based on classification tasks;

[0028] Step 5: Further fine-tune the instructions and couple the subsystems of Chinese-Alpaca-2.0-13B to finally obtain a large model of the reservoir dynamic analysis scenario.

[0029] Furthermore, in step one:

[0030] PDFminer and PyPDF2 are used to effectively extract and separate text and image information, and the Grobid tool is used to distinguish fonts and identify formulas to ensure that professional knowledge information can be effectively screened and retained;

[0031] Use OCR technology to extract useful text data from images;

[0032] Combine Jieba, NumPy and Pandas tools in Python to further clean the data and make formatting agreements;

[0033] Sentence cutting by Jieba;

[0034] NumPy and Pandas are used for data cleaning and formatting to obtain an unlabeled plain text dataset for incremental pre-training.

[0035] In this embodiment,

[0036] The selection of the general basic model compared the performance of the existing mainstream general large models in terms of the number of model parameters, accuracy, scalability, and open source. The comparison results are shown in Table 1. Finally, Chinese-Alpaca-2.0-13B was selected as the general basic model.

[0037] Table 1 Performance comparison of common general basic models

[0038]

[0039]

[0040] Expand the professional vocabulary of the tokenizer in the field of petroleum engineering, expand the tokenizer of petroleum engineering on the Chinese-Alpaca-2.0-13B model, load the existing Chinese-Alpaca-2.0-13B basic tokenizer model, and add the professional vocabulary of petroleum engineering to the Chinese-Alpaca-2.0-13B model. The comparison of the results before and after the expansion can be seen in Table 2;

[0041] Table 2 Comparison of word segmentation results after expanding the petroleum engineering vocabulary

[0042]

[0043] Pre-train and merge the Low-RankAdaptation patch to get a new model, and perform incremental training on the word segmenter after expanding the vocabulary. The specific training process can be found in Table 3.

[0044] Table 3. Construction process of professional large model in petroleum engineering field

[0045]

[0046] Furthermore, in the fine-tuned named entity recognition model, the named entity recognition model is fine-tuned to achieve accurate recognition of the task, and during testing, the effectiveness and accuracy of the model's named entity recognition are comprehensively evaluated from three aspects: entity recognizability rate, numerical variable entity recognition accuracy, and semantic recognition accuracy.

[0047] The named entity recognition technology based on the prompt word engineering can significantly improve the accuracy and efficiency of the named entity recognition system. The named entity results can be found in Table 4. The fine-tuned model was tested, and the test results can be found in Table 5.

[0048] Table 4 Named entity recognition fine-tuning dataset

[0049]

[0050] Table 5 Named entity recognition performance test based on 1300 comprehensive problem datasets in the field of reservoir dynamic analysis

[0051]

[0052] Furthermore, in the fine-tuning Text-to-SQL model, 30 common natural language problems in the field of reservoir dynamic analysis are involved. 256 unique complex SQL query statements constructed on 7 dimensions and their corresponding natural language descriptions are used as the instruction fine-tuning label data set for Text-to-SQL in the field of reservoir dynamic analysis, which is used to enhance the model's conversion capability for SQL statements in the field of oilfield dynamic analysis. See Table 6. The fine-tuned model is tested, and the test results can be seen in Table 7.

[0053] Table 6 Text-to-SQL fine-tuning dataset input and output examples

[0054]

[0055] Table 7 Text-to-SQL performance test based on 1300 common data query problem sets in the field of reservoir dynamic analysis

[0056]

[0057] Furthermore, in the fine-tuning tool calling model based on classification tasks, a tool calling function is introduced in the large model, and a tool usage fine-tuning data set is constructed for 26 key technical indicators in reservoir dynamic analysis. The fine-tuned model can accurately call tools to complete the calculation of corresponding indicators, score the called tools based on the classifier classification task, and return the tool label with the highest score, and calculate the tool corresponding to the label based on the Callback function call, and finally return the results of the relevant indicators.

[0058] The 26 key technical indicators can be found in Table 8. The fine-tuned model was tested and the test structure can be found in Table 9.

[0059] Table 8 Calling tools to fine-tune the dataset

[0060]

[0061] Table 9 Methods for associating sample descriptions with tool call numbers and their accuracy evaluation

[0062]

[0063] Further fine-tuning of instructions and subsystem coupling were performed on Chinese-Alpaca-2.0-13B, and finally a large model of reservoir dynamic analysis scenario was obtained. The model's various answering capabilities were comprehensively tested, and the generated results were in Markdown lightweight text markup format. For details, please refer to Table 10.

[0064] Table 10 Test results of the large model of reservoir dynamic analysis scenario for specific tasks of reservoir dynamic analysis

[0065]

[0066]

[0067]

[0068] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A method for constructing a large model for reservoir dynamic analysis scenarios, characterized in that: The steps include: Step 1: Collect data in the field of petroleum engineering and use the toolkit to process the data accurately and effectively; Step 2: Expand the professional vocabulary of the tokenizer in the field of petroleum engineering, expand the tokenizer of petroleum engineering on the Chinese-Alpaca-2.0-13B basic model, load the existing Chinese-Alpaca-2.0-13B tokenizer model, and add the petroleum engineering professional vocabulary to the basic tokenizer model of Chinese-Alpaca-2.0-13B; Step 3: Pre-train and merge Low-Rank Adaptation patches to obtain a new model. Perform incremental training based on the standardized data in Step 1 and the word segmenter after the vocabulary is expanded in Step 2. Use the Low-Rank Adaptation method to update the weight parameters in the model and merge the updated model with the basic model Chinese-Alpaca-2.0-13B. Step 4: For the models obtained in the above steps, three fine-tuning models are obtained based on the specific actual needs of the oil field, namely, a fine-tuning named entity recognition model, a fine-tuning Text-to-SQL model, and a fine-tuning tool calling model based on classification tasks, wherein the fine-tuning named entity recognition model is used to achieve accurate recognition of tasks; the fine-tuning Text-to-SQL model is used to enhance the model's conversion capability for SQL statements in the field of oil field dynamic analysis; the fine-tuning tool calling model based on classification tasks is used to accurately call the tool after the fine-tuning model to complete the calculation of corresponding indicators; Step 5: Further fine-tune the instructions and couple the subsystems of Chinese-Alpaca-2.0-13B to finally obtain a large model of the reservoir dynamic analysis scenario.

2. A method for constructing a large model for reservoir dynamic analysis scenarios according to claim 1, characterized in that: In step one: PDFminer and PyPDF2 are used to effectively extract and separate text and image information, and the Grobid tool is used to distinguish fonts and identify formulas to ensure that professional knowledge information can be effectively screened and retained; Use OCR technology to extract useful text data from images; Combine Jieba, NumPy and Pandas tools in Python to further clean the data and make formatting agreements; Sentence cutting by Jieba; NumPy and Pandas are used for data cleaning and formatting to obtain an unlabeled plain text dataset for incremental pre-training.

3. The method for constructing a large model for reservoir dynamic analysis scenarios according to claim 1, characterized in that: In the fine-tuned named entity recognition model, the named entity recognition model is fine-tuned to achieve accurate recognition of the task. During the test, the effectiveness and accuracy of the model's named entity recognition are comprehensively evaluated from three aspects: entity recognizability rate, numerical variable entity recognition accuracy, and semantic recognition accuracy.

4. The method for constructing a large model for reservoir dynamic analysis scenarios according to claim 1, characterized in that: In the fine-tuning Text-to-SQL model, 30 common natural language problems in the field of reservoir dynamic analysis are involved. 256 unique complex SQL query statements constructed on 7 dimensions and their corresponding natural language descriptions are used as the instruction fine-tuning label data set for Text-to-SQL in the field of reservoir dynamic analysis, which is used to enhance the model's conversion capability for SQL statements in the field of oilfield dynamic analysis.

5. The method for constructing a large model for reservoir dynamic analysis scenarios according to claim 1, characterized in that: In the fine-tuning tool calling model based on classification tasks, the tool calling function is introduced in the large model, and a tool usage fine-tuning data set is constructed for 26 key technical indicators in reservoir dynamic analysis. The fine-tuned model can accurately call tools to complete the calculation of corresponding indicators, and can score the called tools based on the classifier classification task, and return the tool label with the highest score, and calculate the tool corresponding to the label based on the Callback function call, and finally return the results of relevant indicators.

Images