Automatic instruction corpus generation method based on natural language large model

Abstract

The invention belongs to the technical field of oil field data processing, and particularly relates to an instruction corpus automatic generation method based on a natural language large model. According to the instruction corpus automatic generation method based on the natural language large model, the corpus generation quality is improved, the data extraction difficulty is reduced, the manual intervention cost is reduced, a complete automatic process is achieved, good field adaptability and expansion generalization ability are achieved, and the method is suitable for popularization and application. Powerful support is provided for industrial application in the field of petrochemical engineering. The invention provides an instruction corpus automatic generation method based on a natural language large model. The instruction corpus automatic generation method at least comprises the following steps: step 1, carrying out preprocessing and feature extraction on an original document to obtain first edition corpus data; step 2, performing error correction on the first edition corpus data obtained in the step 1; 3, a corpus template is prepared, and initial seed tasks are generated in batches; and 4, customizing a pre-training model, and carrying out instruction assembly.

Description

Technical Field

[0001] The invention belongs to the technical field of oilfield data processing, and in particular relates to a method for automatically generating instruction corpus based on a natural language large model. Background Art

[0002] With the continuous development of natural language processing technology, automatic generation and processing of language instructions have become an important part of automated control in many technical fields. Among them, the corpus can be understood as a warehouse for storing language materials. By mining the language information (text) in it, the technical knowledge of the corresponding field can be obtained. For the petrochemical field, since the business content it faces is quite different from daily and non-professional written language, its corpus features are more professional and it is impossible to directly reuse the existing corpus, so it is necessary to build a large-scale, real corpus for the petrochemical field.

[0003] However, the inventors found in the process of further research that there are still considerable challenges in generating high-quality instruction corpora in the field of petrochemicals. For example: Corpus quality issues: The existing corpora in the oil and gas industry often have problems such as inconsistent quality, confusing formats, and insufficient accuracy. This may lead to a decline in the performance of the trained model or even misleading output. Difficulty in data extraction: Extracting structured instruction corpora from a variety of documents and materials is a complex task. Traditional text extraction methods may be inefficient and inaccurate. High cost of manual intervention: In many cases, manual data cleaning and proofreading must be relied upon, which is not only time-consuming and labor-intensive, but also costly. Lack of effective automated processes: Many existing instruction corpus generation methods lack a complete automated process. From data extraction, cleaning, assembly to training, each link may require manual intervention, which reduces efficiency. Lack of adaptability to specific fields: Many existing methods may not be suitable for instruction generation in the oil and gas industry or specific application scenarios, limiting their wide application.

[0004] In this regard, those skilled in the art have made many attempts. For example, the patent document with the title of patent: A method for detecting outstanding safety problems in oil fields, application number: CN201910305672.6, and application date: 2019-04-16 records the following technical solution. The present invention relates to a method for detecting outstanding safety problems in oil fields. The method collects a large number of cases of oil field safety problems to establish a corpus; then selects certain texts from the corpus to establish a training sample set, trains the texts in the training sample set, and establishes an oil field safety outstanding problem detection model; uses the oil field safety outstanding problem detection model to predict the outstanding safety problems in the oil field to be tested, calculates the probability values ​​of each topic corresponding to the document to be tested, and selects the topic with the largest probability value as the prediction result of the document to be tested, which is the prediction result of the outstanding safety problems in the oil field to be tested. The detection method uses known data to train the manager prediction model. When using it, it only needs to input the outstanding safety problems in the oil field to be tested into the prediction model. The operation process is simple, and more importantly, the requirements for the staff are low, and the prediction results are less affected by the operator. However, the existing technologies including the above-mentioned patent documents are still not versatile enough and are not suitable for the extensive and complex production tasks in the petrochemical industry. Therefore, it is urgent for technical personnel in this field to provide an instruction corpus generation method that can fully meet the production business needs in the petrochemical industry and has better performance and broader application prospects. Summary of the invention

[0005] The present invention provides a method for automatically generating instruction corpora based on a large natural language model. The method improves the generation quality of the corpus, reduces the difficulty of data extraction, reduces the cost of manual intervention, realizes a complete automated process, has good field adaptability and expansion generalization capabilities, and provides strong support for industrial applications in the petrochemical field.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions: A method for automatically generating instruction corpus based on a natural language large model comprises the following steps: Step 1: Preprocess the original document and extract features to obtain the first version of the corpus data; Step 2: Correct the first version of the corpus data obtained in step 1; Step 3: Prepare corpus templates and generate initial seed tasks in batches; Step 4: Customize the pre-trained model and assemble the instructions.

[0007] Preferably, the method further comprises the following steps: Step 5: Train and fine-tune the customized pre-trained model to obtain a large model based on natural language.

[0008] Preferably, the process of preprocessing the original document in step 1 can be specifically described as follows: De-noise the original document; Adjust the contrast and brightness of images in the original document; Crop and scale pages from the original document.

[0009] Preferably, the process of extracting features from the original document in step 1 can be specifically described as follows: OCR technology is used to realize text recognition and layout structure information extraction of text information in the original document.

[0010] Preferably, the step 2 can be specifically described as: Use text automatic error correction technology to correct the first edition corpus data; Manual verification is performed on the initial version of the corpus data after automatic error correction.

[0011] Preferably, the corpus template prepared in step 3 includes key elements including instruction structure, parameter range, and associated description.

[0012] Preferably, step 4 can be specifically described as: Customized pre-trained models; Adopt the bootstrap expansion method to guide the generation of instructions; Identify the task category that generates the instructions; Construct corresponding instances according to the specific content of the instruction and the task category to which it belongs; Filter and post-process build instructions and the instances they generate.

[0013] Preferably, in the process of filtering and post-processing the generated instructions and the instances generated by them: When a newly generated instruction has an overlap ratio of less than 0.7 with any instruction in the existing instruction pool in terms of ROUGE-L score, it is selected for inclusion into the task pool.

[0014] The present invention provides a method for automatically generating instruction corpus based on a natural language large model, and the method for automatically generating instruction corpus based on a natural language large model at least includes the following steps: step 1: preprocessing and feature extraction of original documents to obtain first edition corpus data; step 2: error correction of the first edition corpus data obtained in step 1; step 3: preparing corpus templates and batch generating initial seed tasks; step 4: customizing pre-training models and assembling instructions. The method for automatically generating instruction corpus based on a natural language large model with the above-mentioned step characteristics combines optical character recognition, customizing large models, automatic error correction and self-guided instruction generation and other technologies on the basis of natural language processing and machine learning algorithms, and more accurately realizes the extraction, verification, organization and training of corpus data. The method can meet various needs of oilfield enterprises in different scenarios; and under the premise of ensuring the quality and accuracy of corpus, it has relatively simple architecture features, low complexity and high algorithm efficiency, and is suitable for the automatic generation of oilfield data instruction corpus in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0016] In the following drawings: Figure 1 A schematic diagram of the flow of the method for automatically generating instruction corpus based on a large natural language model provided by the present invention; Figure 2 A schematic diagram of automatically scanning and recognizing a document to extract text information therein using OCR technology; Figure 3 This is a schematic diagram of error correction for the first edition of the corpus data. DETAILED DESCRIPTION

[0017] The present invention provides a method for automatically generating instruction corpora based on a large natural language model. The method improves the generation quality of the corpus, reduces the difficulty of data extraction, reduces the cost of manual intervention, realizes a complete automated process, has good field adaptability and expansion generalization capabilities, and provides strong support for industrial applications in the petrochemical field.

[0018] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Embodiment 1

[0019] A method for automatically generating instruction corpus based on a large natural language model, such as Figure 1 As shown, the following steps are included: Step 1: Preprocess the original document and extract features to obtain the first version of the corpus data.

[0020] Specifically, through a series of fine adjustments to the original document, the original scanned or digitized document can be made suitable for recognition, thereby laying a solid foundation for subsequent processing and corpus extraction steps.

[0021] Among them, as a more preferred implementation of the present invention, the process of preprocessing the original document in step 1 can be further specifically described as follows: De-noise the original document.

[0022] For example, Gaussian filtering can effectively eliminate Gaussian noise, and median filtering can eliminate salt and pepper noise. For documents with cluttered backgrounds, the OSTU method in the local threshold can be used to better separate the foreground text and background. In addition, it can also be combined with a deep convolutional neural network (CNN) for more accurate noise removal, automatically identifying and eliminating complex background noise, watermarks, etc.

[0023] Adjust the contrast and brightness of the image in the original document. Here, you can choose to use histogram equalization to improve the overall contrast of the image, redistributing the brightness values ​​of the image to make it more balanced. Adaptive enhancement algorithm is used to automatically adjust the contrast and brightness parameters according to the characteristics of the document, such as the color of the text, the brightness of the background, etc. In addition, you can choose to use gamma correction to adjust the overall brightness of the image, and change the gamma value of the image to achieve different degrees of brightness adjustment.

[0024] Crop and scale pages from the original document.

[0025] As an optional implementation, the Canny operator and Sobel operator are used to automatically detect the edges of the document and perform cropping. For documents with skewed pages, perspective transformation technology is used to correct them. During the scaling process, bilinear interpolation and bicubic interpolation methods are used to ensure that the image quality is maintained while changing the size.

[0026] As another preferred implementation of the present invention, the process of extracting features from the original document in step 1 can be specifically described as follows: OCR technology is used to realize text recognition and layout structure information extraction of text information in the original document.

[0027] It is worth noting that Figure 2As shown, the above-mentioned OCR technology is used to automatically scan and identify documents, extract the text information therein, thus realizing the process of extracting the initial corpus data.

[0028] Based on the results of the above text structure analysis, precise positioning and automatic extraction of the required corpus data, such as instructions, descriptions, parameters, etc. are realized.

[0029] Step 2: Correct the errors in the initial corpus data obtained in Step 1.

[0030] It should be noted that the types of errors in the recognition of oil and gas industry corpus are mostly homograph errors, such as "pumping unit" - "pumping machine". Therefore, as Figure 3 shown, Chinese error correction can be divided into the following two steps. The first step is error detection, and the second step is error correction. In the error detection part, word segmentation is first performed through Jieba Chinese word segmentation. Since there are typos in the sentence, the word segmentation results often have cases of incorrect segmentation. In this way, errors are detected from both the character granularity and the word granularity, and the suspected error results of these two granularities are integrated to form a candidate set of suspected error positions. Then error correction is carried out. Specifically, all suspected error positions are traversed, and words at the error positions are replaced with homophonic and homograph dictionaries. Then the sentence perplexity is calculated through a language model, and all candidate set results are compared and sorted to obtain the optimal corrected word.

[0031] As a more preferred embodiment of the present invention, this Step 2 can be specifically described as: using a text error correction tool to automatically correct the initial corpus data.

[0032] Perform manual verification on the initial corpus data after automatic error correction.

[0033] After completing the automatic error correction using the above text error correction tool, continue with the manual verification process. Through the joint cooperation of the above two technical means, the accuracy rate of the corpus is ensured to be close to the perfect level.

[0034] Step 3: Prepare a corpus template and batch generate initial seed tasks.

[0035] Specifically, as a more preferred embodiment of the present invention, the prepared corpus template includes key elements such as instruction structure, parameter range, and associated description. With the help of the prepared corpus template, combined with a Python script program, initial seed tasks can be batch generated, such as the work of components such as instruction input and output.

[0036] Step 4: Customize a pre-trained model and perform instruction assembly.

[0037] As a more preferred embodiment of the present invention, Step 4 can be specifically described as: Customize a pre-trained model.

[0038] A bootstrap expansion method is used to guide the generation of instructions.

[0039] Identify the task category that generated the instruction.

[0040] It should be noted that since classified tasks and non-classified tasks require different processing methods, it is necessary to determine whether the generated instruction belongs to a classified task. According to the specific content of the instruction and the task category to which it belongs, the corresponding instance is constructed.

[0041] Specifically, given specific instructions and their associated task types, we construct corresponding instances for each instruction independently. This process is significantly challenging because it requires the model to deeply understand the nature of the target task based on the given instructions, accurately determine the additional input fields required, and finally complete the task through output. Fortunately, we found that the pre-trained model can generally meet this requirement when guided by instruction-input-output context examples from other tasks. In this process, a natural solution is to adopt an input-first approach, that is, the language model first determines the input fields based on the given instructions, and then generates the corresponding output. This generation order is similar to the general response of the model to instructions and inputs, but context examples from other tasks are specifically used in this scenario. However, this approach may cause the input to be biased towards a specific label, especially when dealing with classification tasks (for example, when performing grammatical error detection, grammatically incorrect inputs are often generated).

[0042] In view of this, we additionally propose an output-first method specifically for handling classification tasks. In this method, we first generate possible task category labels, and then generate corresponding inputs for each task category label. It is worth noting that we apply this output-first method specifically to the task categories identified in the previous step, while the input-first method is used for the remaining non-classification tasks. The above embodiments cleverly ensure the pertinence and effectiveness of the automated production method of the present invention.

[0043] Filter and post-process build instructions and the instances they generate.

[0044] For example, we first excluded instructions containing certain specific keywords (such as "pictures", "charts", "audio", and "video"), because these contents are usually beyond the processing capabilities of language models. Then, in the process of generating new instances for each instruction, we carefully filtered out those instances that were exactly the same, or those with the same input but different outputs. The purpose of doing this is to ensure the diversity and quality of task instructions through a series of careful design and screening, and also lay a solid foundation for further research and application. Embodiment 2

[0045] Embodiment 2 includes all the technical features of Embodiment 1. In addition, Embodiment 2 is further limited to the following technical features: The method for automatically generating instruction corpus based on a natural language large model provided by the present invention also includes the following steps: Step 5: Train and fine-tune the customized pre-trained model to obtain a large model based on natural language. Step 5 is used to supplement the training process of the large model based entirely on natural language. Embodiment 3

[0046] Embodiment 3 includes all the technical features of Embodiment 1. In addition, Embodiment 3 is further limited to the following technical features: During the filtering and post-processing of the generated instructions and their generated instances: When a newly generated instruction has an overlap ratio of less than 0.7 with any instruction in the existing instruction pool in terms of ROUGE-L score, it is selected for inclusion into the task pool.

[0047] The purpose of this setting is to ensure that the newly generated instructions are sufficiently different from the existing instructions.

[0048] The present invention provides a method for automatically generating instruction corpus based on a natural language large model, and the method for automatically generating instruction corpus based on a natural language large model at least includes the following steps: step 1: preprocessing and feature extraction of original documents to obtain first edition corpus data; step 2: error correction of the first edition corpus data obtained in step 1; step 3: preparing corpus templates and batch generating initial seed tasks; step 4: customizing pre-training models and assembling instructions. The method for automatically generating instruction corpus based on a natural language large model with the above-mentioned step characteristics combines optical character recognition, customizing large models, automatic error correction and self-guided instruction generation and other technologies on the basis of natural language processing and machine learning algorithms, and more accurately realizes the extraction, verification, organization and training of corpus data. The method can meet various needs of oilfield enterprises in different scenarios; and under the premise of ensuring the quality and accuracy of corpus, it has relatively simple architecture features, low complexity and high algorithm efficiency, and is suitable for the automatic generation of oilfield data instruction corpus in complex scenarios.

[0049] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for automatically generating instruction corpus based on a large natural language model, characterized in that: The steps include: Step 1: Preprocess the original document and extract features to obtain the first version of the corpus data; Step 2: Correct the first version of the corpus data obtained in step 1; Step 3: Prepare corpus templates and generate initial seed tasks in batches; Step 4: Customize the pre-trained model and assemble the instructions.

2. The method for automatically generating instruction corpus based on a natural language large model according to claim 1, characterized in that: It also includes the following steps: Step 5: Train and fine-tune the customized pre-trained model to obtain a large model based on natural language.

3. The method for automatically generating instruction corpus based on a natural language large model according to claim 1, characterized in that: The process of preprocessing the original document in step 1 can be specifically described as follows: De-noise the original document; Adjust the contrast and brightness of images in the original document; Crop and scale pages from the original document.

4. The method for automatically generating instruction corpus based on a natural language large model according to claim 1, characterized in that: The process of extracting features from the original document in step 1 can be specifically described as follows: Using OCR technology, text recognition and layout structure information extraction of text information in the original document can be achieved.

5. The method for automatically generating instruction corpus based on a natural language large model according to claim 1, characterized in that: The step 2 can be specifically described as: Use text automatic error correction technology to correct the first edition corpus data; Manual verification is performed on the initial version of the corpus data after automatic error correction.

6. The method for automatically generating instruction corpus based on a natural language large model according to claim 1, characterized in that: The corpus template prepared in step 3 includes key elements including instruction structure, parameter range, and associated description.

7. The method for automatically generating instruction corpus based on a natural language large model according to claim 1, characterized in that: The step 4 can be specifically described as: Customized pre-trained models; Adopt the bootstrap expansion method to guide the generation of instructions; Identify the task category that generates the instructions; Construct corresponding instances according to the specific content of the instruction and the task category to which it belongs; Filter and post-process build instructions and the instances they generate.

8. The method for automatically generating instruction corpus based on a natural language large model according to claim 7, characterized in that: During the filtering and post-processing of the generated instructions and their generated instances: When a newly generated instruction has an overlap ratio of less than 0.7 with any instruction in the existing instruction pool in terms of ROUGE-L score, it is selected for inclusion into the task pool.

Images