Geological text generation method, system and device based on large language model

Through the geological text generation method based on large language models, the challenge of massive and multi-format data processing in geology is solved, intelligent text generation is realized, data collation and analysis efficiency is improved, and high-quality text that meets the requirements of the geological field is generated.

CN119443270AActive Publication Date: 2025-02-14CHINA RAILWAY ERYUAN ENGINEERING GROUP CO LTD +1

Patent Information

Application Number
CN202411486816.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-23
Publication Date
2025-02-14
Estimated Expiration
2044-10-23

AI Technical Summary

Technical Problem

Traditional data processing methods are difficult to deal with massive, multi-format data in geology, especially in extracting useful information from data and converting it into scientific knowledge.

Method used

Using a geological text generation method based on a large language model, a large language model in the geological field is constructed by collecting, word segmentation, deduplication, cleaning and serializing geological text data, and fine-tuning is carried out through P-Tuning v2 technology to generate engineering geological scores in line with engineering The text of the stratigraphic lithology required by the paragraph specification.

Benefits of technology

It realizes intelligent text generation, improves the efficiency of manual data collation and analysis, and has good continuity and high logic, which can accurately reflect professional knowledge in the field of geology.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119443270A_ABST
    Figure CN119443270A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of information processing, and particularly discloses a geological text generation method, system and device based on a large language model, and the method comprises the steps of text data collection, data set construction, data preprocessing, large model construction, model fine tuning and the like. And intelligent text generation of stratum lithology description based on the deep learning large language model is realized. According to the method, the ChatGLM big language model is trained, fine tuning is carried out based on the P-Tuning v2 technology, and meanwhile, the geotechnical field knowledge is fused, so that the model can understand and apply the professional knowledge of the geological lithology, the stratum lithology description text meeting the requirements of the engineering geological segmentation specification is intelligently generated according to the known lithology description information, the continuity is good, the logicality is high, and the method is suitable for being used in the field of engineering geology. The whole method is simple to operate, can efficiently improve the efficiency of manual data arrangement and analysis, and is convenient to popularize and apply.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of information processing, and in particular to a geological text generation method, system and device based on a large language model. Background Art

[0002] As a discipline that studies the earth, geology covers a wide range of fields, from the earth's surface to the deep crust, from mineral rocks to oil and gas resources, from ancient environmental changes to modern geological processes. With the advancement of science and technology, the collection of geological data has become more and more efficient, and the amount of data has exploded. These geological data include not only survey reports and scientific research papers in text form, but also images, tables, maps and other formats. Traditional data processing methods are difficult to cope with such a huge amount of data, and different types of data require different processing methods. How to extract useful information from massive data and then transform it into scientific knowledge is a major challenge facing geological research.

[0003] The application of deep learning big language models in the field of geology is gradually being carried out. Developing a geological text generation method based on a big language model with good coherence and logic is of great significance for research in the geological field. Summary of the invention

[0004] The purpose of the present invention is to provide a geological text production method, system and device based on a large language model to address the problems existing in the prior art.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] A geological text generation method based on a large language model comprises the following steps:

[0007] S1. Collect geological text data and extract the stratum lithology description;

[0008] S2, performing word segmentation processing on the stratum lithology description part to form a geotechnical description data set of stratum lithology;

[0009] S3, deduplication, cleaning, and serialization of the geotechnical description data set to obtain an optimized data set;

[0010] S4, pre-training the language model ChatGLM with the optimized data set, integrating geological and geotechnical domain knowledge, and obtaining a large language model in the geological field;

[0011] S5. Fine-tune the large language model in the geological field based on P-Tuning v2 to obtain a lightweight large language model in the geological field;

[0012] S6. Inputting the original geotechnical description data into the large language model of the lightweight geological domain to generate the expected geological text.

[0013] The present invention provides a geological text generation method based on a large language model, including the steps of text data collection, data set construction, data preprocessing, large model construction, and model fine-tuning, thereby realizing intelligent text generation of stratum lithology description based on a deep learning large language model. The present invention trains the ChatGLM large language model and fine-tunes it based on the P-Tuning v2 technology, while incorporating geotechnical field knowledge, so that the model can understand and apply the professional knowledge of geological lithology, thereby intelligently generating stratum lithology description text that meets the requirements of the engineering geological segmentation specification based on the known lithology description information, with good continuity and high logic. The entire method is simple to operate, can effectively improve the efficiency of manual data collation and analysis, and is easy to promote and apply.

[0014] Furthermore, in S1, the collected geological text data includes at least one of a project geological description report and a scientific research paper, and / or geological text data collected in a database.

[0015] The database is a geological information database, a geological space database, a geological scientific research database, a geological public database or a geological environment database.

[0016] The geological text generation method based on the large language model provided by the present invention adopts the geological text data collected in the database and the geological segmentation description written by geological experts in accordance with the specifications, so as to obtain relatively accurate text data in the geological field, including professional knowledge in the geological field, and provide accuracy assistance for text knowledge processing and intelligent text generation in the geological field.

[0017] Furthermore, S2 also includes searching and matching the database according to the geotechnical description data set of the stratum lithology; manually screening and labeling the data that fail to pass the match, and then adding them to the geotechnical description data set.

[0018] This application retrieves, matches, manually screens and annotates the geotechnical description dataset of the stratum lithology obtained in the database to obtain a geotechnical description dataset, which provides accuracy guarantee for subsequent geological text generation in data processing.

[0019] Furthermore, in S2, according to the segmentation information in the engineering geological database, the corresponding naming, numbering, core stratification description, weathering degree, depth and other related information are retrieved and matched with the stratigraphic lithology description text of the engineering geological specification of the corresponding segment. The unmatched ones are manually annotated according to geological specifications, and the matched data are cleaned and serialized to construct the final engineering geological stratigraphic lithology description data set.

[0020] Furthermore, in S2, the geotechnical description dataset includes the name and characteristics of the lithology of the formation.

[0021] Furthermore, the specific operation of deduplication in S3 is: compare all segment reports in the geotechnical description dataset, find out the segments with duplicate content due to close distance, and then make a comprehensive judgment based on distance, time and content factors, and finally retain the data in one of the segment reports and delete the remaining duplicate reports.

[0022] Preferably, the specific operation of deduplication in S3 is: compare all segment reports in the geotechnical description data set, find out the segments with repeated content due to the close distance of the forecast mileage sections, and screen them in turn according to the completeness of the content fields, the order of drilling time, and the breadth of the forecast mileage sections. Finally, retain the data in one of the segment reports with a wider mileage, more stratigraphic lithology description keyword fields, and a newer drilling time, and delete the remaining duplicate reports; for example, the forecast mileage section paradigm is such as D4K280+898.5~+946.1.

[0023] Furthermore, in S4, the specific operation process of pre-training is as follows:

[0024] S41, dividing the optimized data set into a training set and a test set in a ratio of 70%-80%:30%-20%, generating a JSON file storing the training corpus, and saving;

[0025] S42. Input the text with training corpus into the ChatGLM large language model for learning and training, and standardize the model output.

[0026] Furthermore, in S42, the learning and training process is as follows:

[0027] The preset template of the original text input data is a text with n (n is a natural number greater than or equal to 3) entries (preferably, the entry text is a word at the semantic level, a text fragment at the sentence level and a document level);

[0028] Randomly select two or more spans for masking (preferably, the length of the sampling span is determined by Poisson distribution λ=3), and the masked part is replaced by [M], where M represents the span of the mask (preferably, each span contains semantic words or sentences in the original text input and continuous text paragraphs at the document level), and the masked part is replaced by the original sentence input as Part A;

[0029] Randomly sort the selected masked span tokens, and use S to indicate the start of each span of the masked part as Part B;

[0030] The two separated parts plus the corresponding 2D position encoding are used as model input;

[0031] Obtain the word prediction result at the mask in the input vector through training prediction by the Attention mechanism;

[0032] A loss value is calculated according to the word prediction result and the true label word, and the pre-trained language model is trained according to the loss value.

[0033] Furthermore, the specific operation process of fine-tuning the large language model in the geological field based on P-Tuning v2 is as follows:

[0034] First, set the dataset path and change prompt_column and response_column to the KEYs corresponding to the input text and output text in the file;

[0035] and / or,

[0036] Increase max_source_length and max_target_length to match the maximum input and output lengths in the dataset;

[0037] and / or,

[0038] Dynamically adjust the soft prompt length pre_seq_len and learning rate learning_rate parameters;

[0039] and / or,

[0040] Dynamically adjust quantization_bit to select the quantization level of the original model.

[0041] Another object of the present invention is to provide a geological text generation system.

[0042] A geological text generation system, comprising:

[0043] The first module is used to collect geological text data and extract the stratum lithology description;

[0044] The second module performs word segmentation on the stratum lithology description to form a geotechnical description dataset of stratum lithology;

[0045] The third module is used to remove duplication, clean and serialize the geotechnical description data set to obtain an optimized data set;

[0046] The fourth module is used to pre-train the language model ChatGLM with the optimized data set, integrate geological and geotechnical domain knowledge, and obtain a large language model in the geological field;

[0047] A fifth module is used to fine-tune the large language model in the geological field based on P-Tuning v2 to obtain a lightweight large language model in the geological field;

[0048] The sixth module is used to input the original geotechnical description data into the large language model of the lightweight geological field to generate the expected geological text.

[0049] A computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the method for generating geological text based on a large language model as described above is implemented.

[0050] A computer device comprises: a processor and a memory storing a computer program, wherein the processor is configured to execute the above-mentioned method for generating geological text based on a large language model when running the computer program.

[0051] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0052] 1. The present invention provides a geological text generation method based on a large language model, including the steps of text data collection, data set construction, data preprocessing, large model construction, and model fine-tuning, thereby realizing intelligent text generation of stratigraphic lithology description based on a deep learning large language model. The present invention trains the ChatGLM large language model and fine-tunes it based on the P-Tuningv2 technology, while incorporating geotechnical field knowledge, so that the model can understand and apply the professional knowledge of geological lithology, thereby intelligently generating stratigraphic lithology description text that meets the requirements of the engineering geological segmentation specification based on the known lithology description information, with good continuity and high logic. The entire method is simple to operate, can effectively improve the efficiency of manual data collation and analysis, and is easy to promote and apply.

[0053] 2. The geological text generation method based on the large language model provided by the present invention adopts the geological text data collected in the database and the geological segmentation description written by geological experts in accordance with the specifications to obtain relatively accurate text data in the geological field, including professional knowledge in the geological field, and provides accuracy assistance for text knowledge processing and intelligent text generation in the geological field.

[0054] 3. The present invention provides a geological text generation method based on a large language model, which retrieves, matches, manually screens and annotates the acquired geotechnical description data set of stratum lithology to the database to obtain a geotechnical description data set, thereby providing accuracy guarantee for subsequent geological text generation in data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 An example flow chart of the geological text generation method based on the large language model provided by the present invention.

[0056] Figure 2 This is an example structural diagram of the geological text generation system of the present invention. DETAILED DESCRIPTION

[0057] The present invention will be described in detail below in conjunction with the accompanying drawings.

[0058] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0059] Example 1

[0060] like Figure 1 As shown, a geological text generation method based on a large language model includes the following steps:

[0061] S1. Collect geological text data and extract some contents of stratigraphic lithology description; specifically, collect geological text data and database, and extract some contents of stratigraphic lithology description therefrom.

[0062] S2. Perform word segmentation on the stratum lithology description content to form a geotechnical description dataset of stratum lithology.

[0063] The extracted content is segmented using the Word2Vec model, which breaks the text into individual words or phrases. Finally, a knowledge base of stratigraphic lithology descriptions is constructed based on deep learning large language model technology. The knowledge base can include common descriptions, names, characteristics, etc. of stratigraphic lithology.

[0064] In some embodiments, it also includes searching and matching the database according to the knowledge base of stratigraphic lithology description. The data that passes the model matching can be constructed into a geotechnical description data set, and the data that fails to pass the matching needs to be manually screened and processed. The main content is to manually select some data for annotation, and then put them into the geotechnical description data set.

[0065] S3, data preprocessing: delete duplicate data and serialize the data to obtain an optimized data set.

[0066] Furthermore, the specific implementation method of step S3 is:

[0067] Compare all segment reports to find out the segments with duplicate content due to close distance, then make a comprehensive judgment based on factors such as distance, time, and content, and finally retain the data in one of the segment reports and delete the remaining duplicate reports.

[0068] Preferably, all segment reports are compared to find out the segments with duplicate content due to the close distance between the predicted mileage sections. Then, based on the distance, the completeness of the content fields, the drilling time, the content and other factors, a comprehensive judgment is made on the order and the breadth of the predicted mileage sections is screened. Finally, the data in one of the segment reports with a wider mileage, more stratigraphic lithology description keyword fields and a newer drilling time is retained, and the remaining duplicate reports are deleted.

[0069] For the processed database, each segment description is first processed, and the maximum and minimum values ​​of the depth are selected for splicing; then the spliced ​​segment descriptions are serialized and converted into a format suitable for model input; finally, the description content in the engineering geological manual is put into the model training as a reference text.

[0070] Take a data set of fine sand as an example: Original data #<3-4> Fine sand: yellowish brown, slightly wet, non-sticky, feels sandy when rubbed, and has a loose structure. <3-4> Fine sand: yellow, non-sticky, feels sandy when rubbed, and is loose. <3-4> Fine sand: yellow, wet, non-sticky, feels sandy when rubbed, has a loose structure, and has a shaking response. * Maximum thickness #10.6 * Maximum thickness #2.0 * Average thickness #nan.

[0071] S4. Pre-train the language model ChatGLM using the optimized data set, integrate geological and geotechnical domain knowledge, and obtain a large language model in the geological field.

[0072] The collected stratigraphic lithology description data is regarded as small sample data, and the small sample data is divided into training set and test set in proportion; based on the idea of ​​autoregressive space filling, the data set is input into the ChatGLM large language model for learning and training, and the relevant knowledge of stratigraphic lithology description is integrated into the model to standardize the model output. Combining geological domain knowledge and large language model can better complete downstream text generation tasks.

[0073] That is, given a piece of original text input, the model predicts the possible words or phrases based on the trained neural network to complete the generation of the entire text; since the data volume is a small sample data set and the large language model has a large number of parameters, the present invention further fine-tunes the model based on P-Tuning v2; the language model ChatGLM is fine-tuned using the constructed data set to remove unnecessary redundant parameters and reduce the total number of parameters for model fine-tuning.

[0074] Furthermore, the specific implementation method of step S4 is:

[0075] S41. Download the dependencies required by the model and deploy the local ChatGLM-6B model through API mode and Web mode (ChatGLM-6B is an open source Chinese-English bilingual dialogue language model that improves the quality of generated answers through supervised fine-tuning, feedback self-help and human feedback reinforcement learning techniques, making it perform well in Chinese question-answering tasks. The GLM model uses a unified pre-trained model and is suitable for multiple language generation and understanding tasks. It is based on the idea of ​​autoregressive blank filling and is processed through a single Transformer encoder. It randomly deletes continuous token spans of the input text and reconstructs these spans in sequence, so that it performs well on both conditional and unconditional generation tasks).

[0076] S42, access the geological report and related databases, prepare the engineering geological stratum lithology description training set and test set, generate a JSON file storing the training corpus, and save it to a local path.

[0077] S43. Pre-training language model ChatGLM based on autoregressive space filling idea:

[0078] According to the data set, the preset prompt template and the mapping relationship between the preset label words and natural words, a given original text input is put into the pre-trained language model; at the same time, the constructed geotechnical knowledge base is used as an external knowledge base to integrate geotechnical knowledge into the large language model.

[0079] The preset template of the input data takes the text input of 6 entries as an example, an original text input: [x1, x2, x3, x4, x5, x6]. For example, the stratigraphic lithology description of soft soil: Entry 1: brown-gray, yellow-brown:. Entry 2: soft plastic, relatively uniform soil, strong viscosity. Entry 3: 1 to 6 meters thick:. Entry 4: layered distribution under the soft soil layer on the surface of the trench. Entry 5: Class II ordinary soil. Entry 6:: Class D filler.

[0080] Randomly select two spans [x3] and [x5, x6] for masking, and replace the masked part with [M], where M represents the span of the mask. The masked part is replaced by the original sentence input as Part A: [x1, x2, M, x4, M]. Randomly sort the selected masked span tokens, and use S to represent the beginning of each span of the masked part as Part B: [S, x5, x6, S, x3]. The separated two parts plus the corresponding 2D position encoding are used as the model input: [x1, x2, M, x4, M, S, x5, x6, S, x3].

[0081] The word prediction result at the mask in the input vector is obtained through training prediction by the Attention mechanism, and E is used to indicate the end of a Span prediction result: [x5, x6, E, x3, E].

[0082] The loss value is calculated based on the word prediction results and the actual label words, and the language model is trained based on the loss value. The model predicts the possible words or phrases based on the trained neural network and the context information, thereby completing the generation of the entire text.

[0083] S5. Fine-tune the large language model in the geological field based on P-Tuning v2 to obtain a lightweight large language model in the geological field;

[0084] P-Tuning v2 fine-tuning: Set the local JSON format dataset path, change prompt_column and response_column to the KEY corresponding to the input text and output text in the JSON file; increase max_source_length and max_target_length to match the maximum input and output length in the dataset; dynamically adjust parameters such as soft prompt length pre_seq_len and learning rate learning_rate; the P-Tuning-v2 method will freeze all model parameters, dynamically adjust quantization_bit to select the quantization level of the original model, and select the quantization level according to the current hardware conditions.

[0085] P-Tuning v2 optimizes deep hint adjustment, and by applying continuous hints to each layer of the pre-trained model, the generation and knowledge exploration effects are better. The principle of P-Tuning v2 is to adaptively prune the large language model, remove redundant parameters, and use a special compression method to generate a more compact and efficient lightweight model, significantly reducing the number of parameters required for model fine-tuning.

[0086] The hyperparameters trained by the present invention based on the P-Tuning v2 technology are shown in Table 1. When testing the model, the checkpoint with the smallest loss on the validation set is selected.

[0087] Table 1

[0088]

[0089] Parameter training: Train and perform inference evaluation on the model after fine-tuning parameters described in S44. Call the JSON file to fine-tune the parameters and retrain the model. After execution, generate an evaluation file with Chinese Rouge score and BLEU-4 as the evaluation indicators. In the generated TXT file, labels is the predicted output in dev.json, and predict is the result generated by ChatGLM-6B. Compare the predicted output and the generated result, evaluate the model effect in combination with expert appraisal opinions, and continuously optimize parameters and train the model.

[0090] S6. Redeploy the automatic generation model of stratigraphic lithology description text based on the ChatGLM-6B large language model after fine-tuning by P-Tuning v2.

[0091] The optimal model obtained after fine-tuning is used to generate text for the dataset and output segmented instructions in the geological field. The specific values ​​of the two indicators, BLEU-4 and Rouge, are also displayed, which can be used to evaluate the degree to which the generated text conforms to the rules of stratigraphic lithology description.

[0092] Specifically, the optimal model obtained after fine-tuning is used to generate the lithology description of the geological field: the predicted output obtained by training the large language model for the input data set of fine sand is: fine sand (Q4d+p1): yellow-brown, slightly wet, non-sticky, feels sandy when rubbed, loose structure, 2 to 10 meters thick, and locally up to 10 meters thick. Distributed in surface water trenches, it is a Class 1 common soil and Group E filler.

[0093] like Figure 2 As shown, this embodiment also provides a system 100 for generating geological text, including

[0094] The first module 101 is used to collect geological text data and extract part of the stratigraphic lithology description.

[0095] The second module 102 performs word segmentation processing on the stratum lithology description part to form a geotechnical description data set of stratum lithology.

[0096] The third module 103 is used to remove duplication, clean and serialize the geotechnical description data set to obtain an optimized data set.

[0097] The fourth module 103 is used to pre-train the language model ChatGLM with the optimized data set, integrate geological and geotechnical domain knowledge, and obtain a large language model in the geological field.

[0098] The fifth module 103 is used to fine-tune the large language model in the geological field based on P-Tuning v2 to obtain a lightweight large language model in the geological field.

[0099] The sixth module 106 is used to input the original geotechnical description data into the large language model of the lightweight geological field to generate the expected geological text.

[0100] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for generating geological text based on a large language model as described above is implemented.

[0101] This embodiment also provides a computer device, including: a processor and a memory storing a computer program, wherein the processor is configured to execute the above-mentioned method for generating geological text based on a large language model when running the computer program.

[0102] The present invention provides a geological text generation method based on a large language model, including the steps of text data collection, data set construction, data preprocessing, large model construction, and model fine-tuning, thereby realizing intelligent text generation of stratum lithology description based on a deep learning large language model. The present invention trains the ChatGLM large language model and fine-tunes it based on the P-Tuning v2 technology, while incorporating geotechnical field knowledge, so that the model can understand and apply the professional knowledge of geological lithology, thereby intelligently generating stratum lithology description text that meets the requirements of the engineering geological segmentation specification based on the known lithology description information, with good continuity and high logic. The entire method is simple to operate, can effectively improve the efficiency of manual data collation and analysis, and is easy to promote and apply.

[0103] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A geological text generation method based on a large language model, characterized in that: The following steps are involved: S1. Collect geological text data and extract the stratum lithology description; S2, performing word segmentation processing on the stratum lithology description part to form a geotechnical description data set of stratum lithology; S3, deduplication, cleaning, and serialization of the geotechnical description data set to obtain an optimized data set; S4, pre-training the language model ChatGLM with the optimized data set, integrating geological and geotechnical domain knowledge, and obtaining a large language model in the geological field; S5. Fine-tune the large language model in the geological field based on P-Tuning v2 to obtain a lightweight large language model in the geological field; S6. Inputting the original geotechnical description data into the large language model of the lightweight geological domain to generate the expected geological text.

2. The method for generating geological text based on a large language model according to claim 1, characterized in that: In S1, the collected geological text data include at least one of the project geological description report and scientific research papers, as well as the geological text data collected in the database.

3. The geological text generation method based on a large language model according to claim 1, characterized in that: S2 also includes searching and matching the database according to the geotechnical description data set of the stratum lithology; the data that fail to pass the match are manually screened and labeled, and then added to the geotechnical description data set.

4. The method for generating geological text based on a large language model according to claim 1, characterized in that: The specific operation of deduplication in S3 is: compare all segment reports in the geotechnical description dataset, find out the segments with duplicate content due to close distance, and then make a comprehensive judgment based on distance, time and content factors, and finally retain the data in one of the segment reports and delete the remaining duplicate reports.

5. The method for generating geological text based on a large language model according to claim 1, characterized in that: In S4, the specific operation process of pre-training is as follows: S41, dividing the optimized data set into a training set and a test set in a ratio of 70%-80%:30%-20%, generating a JSON file storing the training corpus, and saving; S42. Input the text with training corpus into the ChatGLM large language model for learning and training, and standardize the model output.

6. The method for generating geological text based on a large language model according to claim 5, characterized in that: In S42, the learning and training process is as follows: The preset template of the original text input data is a text of n terms, where n is a natural number greater than or equal to 3; Randomly select two spans for masking, and replace the masked part with [M], where M represents the span of the mask. The masked part is replaced by the original sentence input as Part A; Randomly sort the selected masked span tokens, and use S to indicate the start of each span of the masked part as Part B; The two separated parts plus the corresponding 2D position encoding are used as model input; The word prediction result at the mask in the input vector is obtained through training prediction by the Attention mechanism; A loss value is calculated according to the word prediction result and the true label word, and the pre-trained language model is trained according to the loss value.

7. The method for generating geological text based on a large language model according to any one of claims 1 to 6, characterized in that: The specific operation process of fine-tuning the large language model in the geological field based on P-Tuning v2: Set the dataset path, and change prompt_column and response_column to the KEY corresponding to the input text and output text in the file; and / or, Increase max_source_length and max_target_length to match the maximum input and output lengths in the dataset; and / or, Dynamically adjust the soft prompt length pre_seq_len and learning rate learning_rate parameters; and / or, Dynamically adjust quantization_bit to select the quantization level of the original model.

8. A system using the geological text generation method based on a large language model as described in any one of claims 1 to 7, characterized in that: include: The first module is used to collect geological text data and extract the stratum lithology description; The second module performs word segmentation on the stratum lithology description to form a geotechnical description dataset of stratum lithology; The third module is used to remove duplication, clean and serialize the geotechnical description data set to obtain an optimized data set; The fourth module is used to pre-train the language model ChatGLM with the optimized data set, integrate geological and geotechnical domain knowledge, and obtain a large language model in the geological field; A fifth module is used to fine-tune the large language model in the geological field based on P-Tuning v2 to obtain a lightweight large language model in the geological field; The sixth module is used to input the original geotechnical description data into the large language model of the lightweight geological field to generate the expected geological text.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for generating geological text based on a large language model as described in any one of claims 1 to 7 is implemented.

10. A computer device comprising: A processor and a memory storing a computer program, wherein the processor is configured to execute the geological text generation method based on a large language model as described in any one of claims 1 to 7 when running the computer program.

Citation Information

Patent Citations

  • Method for extracting mode wave slowness based on acoustic logging information

    CN103726836A

  • Geological ontology-based geological report text information extraction method

    CN110795932A

  • Large geological specimen three-dimensional reconstruction display method and system

    CN116091717A

  • Engineering file generation method and system based on large language model

    CN116402024A

  • Professional field document abstract generation method and system under small sample condition

    CN116662537A

Cited By

  • Large language model high-quality text data set construction method and system

    CN120336527A

  • Method and system for automatically generating geological description based on geological photos

    CN120472465A