Method for automatically extracting ecological parameters and driving factors of large language model

Abstract

The invention discloses an automatic extraction method for ecological parameters and driving factors of a large language model, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring an ecological literature PDF, constructing a text block cue word, analyzing the ecological literature by utilizing a large language model, segmenting, recombining and designing a multi-level cue word system based on an ecological rule, driving the large language model to extract ecological parameters and driving factors thereof in an ecological parameter structure, and fusing and correcting the ecological parameters based on table data. Time, space and unit standards are unified, semi-automatic extraction and manual calibration of literature picture observation data are achieved through picture interactive extraction, field alignment cue words are designed to guide a large language model to automatically standardize extraction result fields, the extraction result fields are imported into a database, and an ecological standardized database is formed.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of artificial intelligence, and relates to an ecological parameter of a large language model and an automatic extraction method of driving factors thereof. BACKGROUND

[0002] A large number of ecological literatures provide key empirical support for meta-analysis, ecological big data mining, and large-scale ecological research. However, these data are scattered in the text and figures of millions of academic research papers, which seriously hinders their effective integration. Although existing research compiles experimental data sets by manual extraction, in the face of the exponential growth of ecological literature, manual extraction is inefficient and difficult to implement on a large scale, resulting in high data reuse costs.

[0003] Existing text mining technologies based on rules, traditional natural language processing, machine learning, and deep learning have made progress in automatically labeling taxonomic texts and identifying species names, but due to the reliance on pre-defined rules or training models for specific tasks, they lack flexibility and adaptability, making it difficult to accurately understand the complex grammar and professional semantics in ecological literature and efficiently process data extraction tasks specific to the ecological field.

[0004] In recent years, large language models (LLM) have shown excellent performance in literature mining in the fields of materials and medicine due to their strong semantic understanding and logical reasoning capabilities, enabling rapid extraction of key information and significantly improving data acquisition efficiency. In the field of ecology, LLM has also been initially used for the extraction of simple information such as species names and geographic coordinates, but existing methods still lack autonomy and flexibility in the efficient extraction and standardized processing of fine-grained quantitative data such as experimental design, environmental parameters, and dynamic processes. Therefore, how to fully utilize the capabilities of LLM to achieve efficient, structured, and standardized extraction of complex ecological parameters and their driving factors in ecological literature is a pressing problem. SUMMARY

[0005] The present application provides an ecological parameter and driving factor automatic extraction method of a large language model to solve the technical problems of the prior art.

[0006] An ecological parameter and driving factor automatic extraction method of a large language model, comprising the following steps: obtaining ecological literature PDF, constructing text block prompt words, analyzing ecological literature by using a large language model, segmenting and recombining, designing a multi-level prompt word system based on ecological rules, driving the large language model to extract ecological parameters and driving factors in the ecological parameter structure, fusing and correcting the ecological parameters based on table data, unifying time, space and unit standards, realizing semi-automatic extraction and manual calibration of observation data in literature pictures through interactive extraction of pictures, designing field alignment prompt words to guide the large language model to automatically standardize the extraction results, importing the database, and forming an ecological standardized database.

[0007] The advantages of the present application are: accurate extraction and standardized processing of ecological literature data are realized, the data processing efficiency and quality are improved, and at least the following technical effects or advantages are achieved:

[0008] By designing block prompt words that fit the characteristics of ecological literature, the large language model is guided to quickly identify the core content of the literature such as "research area description, sample plot investigation method, result analysis, discussion conclusion", automatically complete segmentation and recombination, replace the traditional inefficient process of manual page-by-page reading, segment-by-segment marking and manual extraction, intelligently filter out effective ecological information and redundant content, and greatly improve the efficiency of literature analysis.

[0009] A multi-level prompt word system based on ecological rules is designed to drive the large language model to deeply understand the internal relationship between ecological parameters and driving factors in the literature, accurately extract the specific values of ecological parameters and driving factors, automatically complete time alignment, space matching and unit standardization conversion, reduce the error of manual post-processing, and lay a foundation for cross-study and cross-regional ecological data comparison.

[0010] For non-structured image data such as "line chart, column chart" in the literature, image interactive tools and automatic extraction technology are developed to realize semi-automatic extraction and manual calibration of observation data in literature pictures, taking into account data extraction efficiency and verifiability, and solving the problems of traditional manual reading, such as easy to make mistakes and time-consuming.

[0011] Field alignment prompt words are constructed to guide the large language model to standardize the extraction results by field, and automatically import the preset ecological database to form a standardized database with reasonable structure, unified fields and direct call. The database supports multi-dimensional retrieval and cross-literature comparative analysis, breaks down the barriers of "data scattered storage, different formats and difficult reuse" in the field of ecology, promotes big data mining and data sharing in the field of ecology, and provides high-quality data support for ecological theory innovation and ecological protection policy making. BRIEF DESCRIPTION OF DRAWINGS

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. As shown in the figures:

[0013] Figure 1 This is a flowchart of one aspect of the present invention.

[0014] Figure 2 This is a second flowchart of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Example 1: As Figure 1 and Figure 2 As shown, an automatic extraction method for ecological parameters and driving factors of a large language model is proposed, which relates to the application of large language models in the construction of ecological literature datasets.

[0017] Large language models are artificial intelligence models trained on large-scale text data, possessing deep semantic understanding and knowledge reasoning capabilities. They can parse instructions and execute tasks based on prompts. Addressing the low efficiency of manual extraction from ecological literature and the insufficient flexibility of large language models, this invention proposes an automatic extraction method for ecological parameters and their driving factors from large language models. By designing an intelligent workflow tailored to the ecological field, combined with a multi-level prompt system and a human interaction mechanism, this method achieves efficient and accurate extraction of ecological parameters and their driving factors from ecological literature, providing powerful tools and data support for large-scale ecological research.

[0018] An automatic extraction method for ecological parameters and driving factors of a large language model, comprising the following steps:

[0019] Step 1: Obtaining Ecological Literature PDFs. Based on the pre-set ecological research goals and parameter requirements, design search keywords and search and download relevant PDF literature from literature databases such as CNKI as the basis for subsequent analysis and extraction.

[0020] Step 2, ecological literature parsing step, using text parsing technology to convert PDF to Markdown format text with title symbol, according to the preset title symbol to block the text, and build the first prompt word input large language model, guide it to reorganize the text according to the article section title, output the title of research area profile, research method, result and analysis, discussion and other paragraphs, and obtain the corresponding content through automatic association to generate the text Markdown; Tables and pictures are extracted by recognizing keywords and regular expressions, and saved as table Markdown and picture Markdown respectively. Table information includes table title and table content, and picture information includes picture title and picture path. The reorganized literature content can help the large language model accurately understand the logic of each part and improve the targeting of data extraction.

[0021] Step 3, extract ecological parameter structure step, based on the research target and ecological parameter characteristics, design the second prompt word following the ecological law, input the text Markdown and the second prompt word into the large language model, so that it can deeply understand the text content and accurately extract the specific observation value of the target ecological parameter and its corresponding driving factor data such as vegetation factor (such as vegetation type, tree height, diameter at breast height, forest age), climate factor (such as temperature, precipitation, wind speed), soil factor (such as soil PH, soil bulk density, soil nitrogen content), topographic factor (such as elevation, slope, aspect), etc. And unify the time, space, unit, etc. To ensure consistency, the extraction results form a metadata table, a full data table, and an ecological parameter time series observation table, etc. Time and space matching Markdown table, and automatically saved as Excel file.

[0022] Step 4, ecological parameter fusion and correction step, get table title through rule screening, design third prompt word, guide large language model to output table number list containing target data, according to table Markdown text association corresponding table content, form new paragraph composed of multiple tables; Based on the text extraction results obtained in step 3 and the newly composed table information, design the fourth prompt word, drive the large language model to fuse the table information based on the text extraction results, fill in the missing data, enrich the ecological field dimension, and correct the extracted results again to get more accurate and comprehensive structured data. After verification, classify each driving factor into vegetation, climate, soil, and topography, and finally generate new metadata table, vegetation / climate / soil / topographic factor table, and ecological parameter time series observation table, etc. Time and space matching table, and automatically saved as Excel table.

[0023] Step 5, picture interactive extraction step, get picture title by rule screening, design the fifth prompt word, guide large language model to output picture number list containing target data, based on picture Markdown text association corresponding picture path, according to user demand calling interactive extraction tool, set the scale value of horizontal and vertical coordinates in the automatically popped up picture interface, click the position of the data needed to extract, and get the real value based on image scale and pixel ratio conversion; After data extraction, call storage tool, set the title of horizontal and vertical coordinates by user, save it automatically to column name and row value, form Excel table.

[0024] Step 6, import database step, after manual verification of the Excel tables obtained in steps 3, 4 and 5, create corresponding MySQL database table, input the designed sixth prompt word and Excel header information into large language model, drive it to compare the fields of the Excel to be stored with the existing fields in the database, set the synonymous and similar fields as the same standard field, add the fields that do not exist in the database, automatically associate the corresponding data and import the database, form a standardized ecological database that can be directly compared and analyzed.

[0025] Example 2: as shown in Figure 1 and Figure 2 , a method for automatically extracting ecological parameters and driving factors of a large language model, comprising the following steps:

[0026] Step 1, according to the ecological research target (such as extracting litterfall amount and its driving factors), adopt theme retrieval method to retrieve relevant literature (such as litterfall amount), perform literature retrieval on authoritative database platform such as China Knowledge Network, select literature meeting research requirements according to literature title and abstract, and uniformly download as PDF format.

[0027] Step 2, use MinerU tool to analyze PDF text, convert it to Markdown format text with "#" mark, split out "research area overview", "research method", "result and analysis", "discussion" and subordinate subheadings (such as "2.1 sample site setting") based on "#" as block basis, form header list {headers}; Design the first prompt word, drive DeepSeek-V3 large language model to output text title containing key information, automatically associate title with corresponding text, and reorganize it into text Markdown; Extract table through regular expression, save it as table Markdown (including title and cell content), identify picture title and path, and save it as picture Markdown. The following is a detailed description of the first prompt word in a specific scenario:

[0028] ["1.1 Study Area Overview", "3.2 Litter Quantity Dynamics"]

[0029] ["1.1 Study Area Overview", "3.2 Litter Quantity Dynamics"] ["1.1 Study Area Overview", "3.2 Litter Quantity Dynamics"]

[0030] ["1.1 Study Area Overview", "3.2 Litter Quantity Dynamics"] 2 ["1.1 Study Area Overview", "3.2 Litter Quantity Dynamics"] 2 ["1.1 Study Area Overview", "3.2 Litter Quantity Dynamics"] ["1.1 Study Area Overview", "3.2 Litter Quantity Dynamics"]

[0031] ["1.1 Study Area Overview", "3.2 Litter Quantity Dynamics"]

[0032] The table title in the literature is as follows: {table_titles}. Please screen out the table title containing the site soil, vegetation, terrain data and total amount of litter, and output the list of numbers that meet the conditions (such as ["Table 1", "Table 3", etc.]), excluding tables containing only litter components.

[0033] Step 4.1, after obtaining the table number related to the research topic, screen out the table information corresponding to the title (such as "Table 3 Soil Physical and Chemical Properties") from the Markdown containing table information, and input these table information into the large language model together with the all_data table generated in step 3 and the designed fourth prompt word, output the supplemented and improved table data, and automatically save it as the corresponding Excel file. The following is a detailed description of the fourth prompt word in a specific scenario:

[0034] There are two data: ① all_data table (information extracted from the text); ② table data (from tables in the literature). Please merge according to the following rules: 1. Take the all_data table as the basis, and supplement the missing values with the table data (such as the soil pH of site A missing in the text, filled with Table 3 data); 2. If the data conflicts (such as the amount of litter in the text and the table is inconsistent), take the table data as the standard;

[0035] 3. Split the driving factors into four tables: vegetation factor table (site number, vegetation type, forest age, tree height); terrain factor table (site number, elevation, slope); soil factor table (site number, soil pH, total nitrogen content); climate factor table (year, annual precipitation, annual average temperature). 4. Output the merged all_data table, metadata table and four driving factor tables (Markdown format).

[0036] Step 5, extract the picture title from the picture Markdown, form the list {img_titles}, and input it into the large language model together with the designed fifth prompt word to get the picture number that needs to be extracted data, the following is a detailed description of the fifth prompt word in a specific scenario:

[0037] The picture title is as follows: {img_titles}. Please screen all titles related to the study area, site profile, litter amount (such as annual, monthly dynamics), as well as environmental variables such as temperature, precipitation, vegetation, terrain, soil, etc. that affect the production of litter amount, and output the list of numbers that meet the conditions (such as "Figure 1", "Figure 4", etc.), excluding tables containing only litter components.

[0038] Step 5.1. According to the Markdown text of the picture information, associate the picture path corresponding to the picture number, automatically pop up the picture, and decide whether to extract and whether to extract the same picture multiple times. If extracted, pop up the interactive extraction interface, and manually set the horizontal and vertical coordinates and scale value. Click the position where the data needs to be extracted, and based on the image scale and pixel ratio conversion, get the real value. Form structured data by setting the title of horizontal and vertical coordinates, and save it as an Excel file.

[0039] Step 6. After manually verifying that all Excel tables are correct, create a target table in MySQL, build the sixth prompt word, and input it into the large language model together with the existing fields in the database and the fields of the Excel table. Drive it to compare, set synonymous and similar fields as the same field, align different sources of literature to unify and standardize (such as all "elevation" are stored in the "altitude_m" field), and for fields that do not exist in the database, add new fields after converting to appropriate names through the large language model. Finally, automatically associate fields and corresponding data and store them in the database to form a standardized database that can be continuously updated and directly compared and analyzed. The following is a detailed description of the sixth prompt word in a specific scenario:

[0040] Task: Map Excel table headers to {table_name} table database fields, rules as follows: 1. Prefer to use predefined mapping: {mapping_text}; 2. If the database already exists the same meaning field (reference existing field list), use the same field name; 3. No predefined mapping and no matching field, translate English directly, keep unit, use underscore to connect, example: "soil organic matter (0-20cm)" to "soil_organic_matter_0-20cm"; 4. Record all Excel table headers, merge the same fields to make them unified and standardized.

[0041] Excel table headers: {excel_headers}

[0042] {table_name} performance has fields: {existing_columns_text}

[0043] Output strictly as a dictionary: { "database field name": "original Excel table header"}.

[0044] The above only describes the preferred embodiments of the present application and does not limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An ecological parameter and its driving factor automatic extraction method of a large language model, characterized in that, The method comprises the following steps: obtaining ecological document PDF, constructing text block prompt words, analyzing ecological document by using large language model, segmenting and reorganizing, designing multi-level prompt word system based on ecological law, driving large language model to extract ecological parameters and driving factors in ecological parameter structure, fusing and correcting ecological parameters based on table data, unifying time, space and unit standard, realizing semi-automatic extraction and manual calibration of observation data of document pictures through interactive extraction of pictures, designing field alignment prompt words to guide large language model to automatically standardize the extraction results, importing database, and forming ecological standardized database.

2. The method of claim 1, wherein the method further comprises: determining the ecological parameters of the large language model based on the driving factors of the large language model. The step of obtaining ecological document PDF comprises the following steps: obtaining ecological document, designing retrieval keywords based on preset ecological research target and parameter demand, retrieving and downloading related PDF documents from CNKI literature database as the basis for subsequent analysis and extraction.

3. The method of claim 1, wherein the method further comprises: determining the ecological parameters of the large language model and the driving factors of the ecological parameters based on the first data and the second data. The step of analyzing ecological document comprises the following steps: converting PDF to Markdown format text with title symbol by using text analysis technology, dividing the text according to the preset title symbol, and constructing the first prompt word to input large language model to guide it to reorganize the text according to the chapter title, output the title of the paragraphs related to the research area profile, research method, results and analysis, discussion, and obtain the corresponding content through automatic association to generate the text Markdown; the table and picture are extracted by recognizing the keyword and regular expression, and saved as table Markdown and picture Markdown respectively, the table information includes table title and table content, and the picture information includes picture title and picture path.

4. The method of claim 1, wherein the method further comprises: determining the ecological parameters of the large language model and the driving factors of the ecological parameters based on the first data and the second data. The step of extracting ecological parameter structure comprises the following steps: based on the research target and the characteristics of ecological parameters, the second prompt word following the ecological law is designed, the text Markdown and the second prompt word are input into the large language model, the content of the text is deeply understood, the specific observation value of the target ecological parameter and the corresponding vegetation factor (vegetation type, tree height, diameter at breast height, forest age), climate factor (temperature, precipitation, wind speed), soil factor (soil PH, soil bulk density, soil nitrogen content), topographic factor (elevation, slope, aspect) driving factor data are accurately extracted, and the time, space and unit standard are unified to ensure consistency, the extraction results form the meta information table, the full data table and the ecological parameter time sequence observation table, and are automatically saved as Excel files.

5. The method of claim 1, wherein the method further comprises: determining the ecological parameters of the large language model and the driving factors of the ecological parameters based on the first data and the second data. The ecological parameter fusion and correction step includes the following steps: obtaining a table title through rule screening, designing a third prompt word, guiding a large language model to output a table number list containing target data, associating corresponding table content according to table Markdown text, and forming a new paragraph composed of multiple tables; based on the text extraction result obtained in the ecological parameter structure extraction step and the newly composed table information, a fourth prompt word is designed to drive the large language model to take the text extraction result as a benchmark, fuse the table information, fill in the missing data, enrich the ecological field dimension, and correct the extracted result again to obtain more accurate and comprehensive structured data. After verification, each driving factor is accurately classified and divided into vegetation, climate, soil and terrain categories, and finally a new meta-information table, a vegetation / climate / soil / terrain factor table and an ecological parameter time series observation table are generated and automatically saved as an Excel table.

6. The method of claim 1, wherein the method further comprises: determining a number of parameters of the large language model; and determining a number of driving factors of the large language model. The picture interactive extraction step includes the following steps: obtaining a picture title through rule screening, designing a fifth prompt word, guiding a large language model to output a picture number list containing target data, associating corresponding picture paths based on picture Markdown text, calling an interactive extraction tool according to user requirements, setting the scale value of the horizontal and vertical coordinates in the automatically popped-up picture interface, clicking the position where data needs to be extracted, and obtaining the real value based on the image scale and pixel ratio conversion; after data extraction, a storage tool is called, the user sets the title of the horizontal and vertical coordinates, and the Excel table is automatically converted to column name and row value when saved.

7. The method of claim 1, wherein the method further comprises: determining a number of parameters of the large language model; and determining a number of driving factors of the large language model. The import database step includes the following steps: after manually verifying that the Excel tables obtained in the extraction ecological parameter structure step, the ecological parameter fusion and correction step, and the picture interactive extraction step are correct, corresponding MySQL database tables are created, a sixth prompt word and Excel table header information are input into a large language model, the fields of the Excel to be stored are compared with the existing fields in the database, synonymous and similar fields are set as the same standard field, new fields that do not exist in the database are added, corresponding data is automatically associated and imported into the database, and a standardized ecological database that can be directly compared and analyzed is formed.

Images