High-efficiency data extraction method based on large language model
Through the automated information extraction framework based on large language models and multi-model interaction, the problem of efficiently extracting structured data from academic literature has been solved, efficient and accurate data extraction and cross-domain applications have been achieved, and the transformation of scientific research into information technology has been promoted.
Patent Information
- Application Number
- CN202510739662.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-04
- Publication Date
- 2025-09-16
AI Technical Summary
Existing technologies make it difficult to efficiently and accurately extract structured data from massive academic literature, especially in the fields of materials science, chemistry, and environmental science. Traditional tools have difficulty processing complex formats and have limited cross-domain migration capabilities.
A data extraction method based on a large language model is adopted. By building an automated information extraction framework, designing a progressive prompting strategy, multi-model interaction and dataset construction, and machine learning verification, the automation of document preprocessing, multi-round question and answering, and data storage is achieved. This supports multimodal information analysis, builds high-quality domain datasets, and verifies model reliability.
It significantly improves data extraction efficiency, reduces time costs, improves the accuracy and completeness of information extraction, supports cross-domain migration, forms a closed loop of data extraction, and promotes the transformation of scientific research informationization.
Smart Images
Figure CN120654825A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data extraction, and in particular to an efficient data extraction method based on a large language model. Background Art
[0002] Current scientific research increasingly relies on data-driven analysis, and a large amount of experimental data and research results are stored in the form of academic literature. However, traditional methods of obtaining scientific research data mainly rely on manual screening and reading of literature, which is not only inefficient but also prone to information omissions and subjective biases. Especially in fields such as materials science, chemistry, and environmental science, experimental data are usually presented in unstructured or semi-structured form, making data extraction and organization more complicated. How to efficiently and accurately extract structured data from massive literature has become a major problem in the current scientific research informatization. Therefore, the development of an efficient and automated literature data extraction method has become a key demand for the current scientific research informatization.
[0003] Among some existing automated information extraction technologies, information extraction and natural language processing in the materials field still face many challenges due to the complexity and heterogeneity of text; existing technologies rely on traditional document parsing tools such as PyPDF2 and Tabula, and cannot directly process complex formats in academic literature, such as formulas, tables, and image information, and must undergo tedious preprocessing; rule-driven or shallow machine learning models have limitations in understanding complex semantics and have difficulty parsing complex chemical terms and experimental parameters; and existing technologies are highly dependent on manually annotated training data, which limits the model's ability to migrate between different fields and makes it difficult to cope with interdisciplinary information extraction tasks. Summary of the Invention
[0004] The purpose of the present invention is to provide an efficient data extraction method based on a large language model, which solves the problems existing in the background technology.
[0005] To solve the above technical problems, the present invention adopts the following technical solutions: The present invention provides an efficient data extraction method based on a large language model, including: S1. Constructing an automated information extraction framework, using automated tools to simulate manual operations, extracting key document data, and realizing the automation of the entire process of document preprocessing, uploading, multi-round question-and-answer interaction, and data storage;
[0006] S2. Hint Engineering Optimization Strategy: Using a progressive hinting strategy, we build domain-specific hint chains. By carefully designing input hints, we can guide large language models to generate more accurate and relevant outputs.
[0007] S3. Multi-model interaction and dataset construction: We select several large multimodal language models and leverage their differences in sensitivity to different parameters to extract data and perform manual annotation comparisons. By evaluating various metrics on parameter extraction for each model, we integrate the models with the best performance across all parameters to construct a high-quality domain dataset.
[0008] S4. Machine learning validation, based on high-quality domain datasets, screens the optimal feature combination through data preprocessing and feature engineering, compares the model prediction results with real experimental data, and verifies the dataset validity and model reliability.
[0009] The beneficial effects of the present invention are: 1. In constructing an automated information extraction framework, the present invention uses the Selenium automation tool to simulate the document uploading, multiple rounds of question and answer, and data storage processes, which saves the processing time of a single document compared with manual operation, avoids the repetitive labor of traditional manual extraction, greatly reduces the time cost of scientific research data collection, and significantly improves data extraction efficiency.
[0010] 2. In multi-model interaction and dataset construction, the present invention breaks through the limitations of traditional NLP technology and gets rid of the dependence on traditional document parsing tools such as PyPDF2 and Tabula. It directly processes multimodal information such as text, images, and tables through a large language model, supports complex formats such as semantic analysis of formulas and charts, improves the integrity of information extraction, and eliminates the need for large-scale manual data annotation. Through prompt engineering and multi-model collaboration, it achieves cross-domain migration capabilities.
[0011] 3. The present invention verifies the validity of the data set through the machine learning model in the machine learning verification, and the prediction accuracy of the test set is relatively high. It can provide data-driven theoretical guidance for research, forming a complete closed loop of "literature data extraction → structured data set construction → machine learning verification → experimental application", and promoting the informatization and intelligent transformation of scientific research. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0013] Figure 1 This is a connection diagram for executing the method of the present invention. DETAILED DESCRIPTION
[0014] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0015] Reference Figure 1 As shown, the present invention provides an efficient data extraction method based on a large language model. Graphitic carbon nitride (g-C3N4), as a new non-metallic photocatalytic material, has attracted much attention in the field of photocatalytic hydrogen production due to its visible light response characteristics, excellent chemical stability and low cost advantages. The present invention takes graphitic carbon nitride photocatalytic hydrogen production as an example, including:
[0016] S1. Build an automated information extraction framework, use automated tools to simulate manual operations, extract key document data, and automate the entire process of document preprocessing, uploading, multi-round question-and-answer interaction, and data storage.
[0017] In a specific embodiment of the present invention, the simulation of manual operations with the help of automated tools is specifically to simulate the interaction between humans and large language models through the Selenium automated testing tool, so as to realize the automation of the entire process of document uploading, automatic questioning, multiple rounds of question and answer, data extraction and storage, thereby greatly improving the efficiency of document data collection.
[0018] It should be noted that the automation tool specifically involves calling Selenium through Python to drive the browser to locate the upload button, fill in the file path, trigger the upload action, sequentially input ten preset questions for multiple rounds of question-and-answer sessions, extract the HTML table data from the model's last answer, and clean it into structured text. HTML is a hypertext markup language, primarily used to describe the structure of web pages. It is defined and constructed by a series of specific HTML tags and plays an important role in data extraction and processing, facilitating the structured display and interaction of data.
[0019] In a specific embodiment of the present invention, document preprocessing includes merging the main text and supporting information of published documents, driving the browser to simulate file uploading, question input and result parsing by writing code, and cleaning the text output by the large language model and writing it into a preset table.
[0020] Automated collection of experimental parameters for photocatalytic hydrogen production from graphitic carbon nitride. Writing a Python script to call a Selenium-driven browser, locating the literature upload portal of the academic database, and batch uploading literature containing research on g-C3N4 photocatalytic hydrogen production. Preset prompt chains, asking questions in rounds through automated tools, extracting HTML table data output by a large language model, and cleaning it into structured text.
[0021] S2. Prompt engineering optimization strategy, adopting a progressive prompt strategy to build a domain-specific prompt chain. By carefully designing input prompts, it guides the large language model to generate more accurate and relevant outputs.
[0022] In a specific embodiment of the present invention, the progressive prompt strategy specifically means that each question asked by the model must be based on the previous answer; subsequent prompts are developed based on the previous answers. The specific process is: first, the model is allowed to extract a certain type of basic information in the literature through initial prompts, and the model output is used as context. On this basis, subsequent prompts are designed to further ask related information. Each round of questions depends on the content of the previous round of answers, forming an interlocking chain of prompts, guiding the model to gradually focus on complex parameter extraction, ensuring the consistency and pertinence of information extraction; through this strategy, the large model is guided to generate more accurate answers, improve the accuracy of information extraction, reduce the "hallucination" phenomenon, and ensure the reliability of the extracted data; by limiting the output format, unit conversion rules and missing value processing, the consistency and accuracy of information extraction are ensured.
[0023] In a specific embodiment of the present invention, the construction of a domain-specific prompt chain is specifically to decompose a complex task in a certain professional field into multiple rounds of prompt sequences with logical associations; clarify the core information types required for the task, such as concept definitions, attribute characteristics, association relationships, etc., and design the first prompt to focus on basic information extraction, such as allowing the model to identify and list the names of objects of a specific category from the text; subsequent prompts are based on the previous answers as input, and gradually increase the depth and details of information extraction. For example, after obtaining the object name, the specific attributes or related parameters of each object are asked, and the output form is standardized through format limitations, such as tables and lists; each round of prompts is based on the results of the previous round, forming an interlocking guidance path, so that the model gradually clarifies the task goals during the interaction process and parses complex content layer by layer; the entire process converts the information logic in the field into step-by-step instructions that the model can understand through an orderly arranged prompt sequence, ensuring that the information extracted at each step provides contextual support for subsequent prompts, and ultimately achieving structured output of complex information in a specific field.
[0024] In the experiment of photocatalytic hydrogen production from graphitic carbon nitride, a chain prompt was designed based on the complex parameters of photocatalytic hydrogen production from graphitic carbon nitride g-C3N4.
[0025] S3. Multi-model interaction and dataset construction: Several large multimodal language models are selected. Their differences in sensitivity to different parameters are used to extract data and perform manual annotation and comparison. By evaluating various indicators of each model in parameter extraction, the models with the best performance in each parameter are integrated to construct a high-quality domain dataset.
[0026] In a specific embodiment of the present invention, the selection of several multimodal large language models, such as Conch AI, Doubao AI, Baichuan AI, ChatGPT, and Kimi, specifically refers to the selection of several large language models that can process various forms of information input or output and have the ability to integrate cross-modal information, including models that support multi-format input such as text, images, and tables. It can recognize complex information such as formulas and charts in the literature, make up for the limitations of a single text model, and achieve richer and more real-life interaction and task processing by integrating information from different modalities, such as vision, language, and hearing, and extract data in multiple formats to improve the comprehensiveness of chemical literature information extraction.
[0027] In a specific embodiment of the present invention, the specific method for constructing a high-quality field dataset is as follows: for each parameter, such as calcination temperature, precursor ratio, hydrogen production rate, etc., data extraction is performed through the interaction of several large models, and the model extraction performance of various models is evaluated comprehensively in terms of the quantity distribution, accuracy, recall rate and F1 score of the same data points of each parameter. Then, the idea of integration is adopted to select the model with the highest accuracy on the parameter for extraction, and finally the model with the highest accuracy for each parameter is used to construct a high-quality dataset with the highest accuracy for each parameter, and finally a high-quality dataset containing each core parameter is formed by combining to ensure the accuracy and comprehensiveness of the data.
[0028] The quantitative distribution of the same features of the same data point can be used to evaluate the performance of the model in extracting single data from the perspective of the model as a whole. The quality of the single data point extracted by each model can be obtained through the frequency distribution diagram and overall heat map of the data extracted by several models.
[0029] The accuracy rate is specifically the proportion of truly relevant documents extracted by the system; a high accuracy rate indicates that the content extracted by the system is more accurate. By comparing the accuracy data of several large models, the ones with the highest accuracy rates are selected for later application.
[0030] The recall rate is specifically the proportion of relevant documents successfully extracted by the system from all actual relevant documents. A high recall rate means that the system can find more relevant information. By comparing the recall rate data of several large models, the ones with the highest recall rates are selected for later application.
[0031] The F1 score, specifically the harmonic mean of recall and precision, is used to comprehensively evaluate system performance. By comparing the F1 scores of several large models, the ones with the highest F1 scores are selected for later application.
[0032] In the experiment of photocatalytic hydrogen production from graphite carbon nitride, ten large language models with high comprehensive ability scores and multimodal capabilities were selected, namely Gemini, Claude, ChatGLM, Command R, Conch AI, Doubao AI, Baichuan AI, ChatGPT, Kimi, and 01AI, to extract g-C3N4 related parameters respectively; for the same parameters of the same document, the output of each model was compared with the original data, and the accuracy, recall rate and F1 score were calculated, and finally the model with the best performance for each parameter was selected; according to the experimental results, GPT-4o, Claude and Baichuan AI were selected to extract different data respectively, and the corresponding extracted data were integrated to construct a 381×19 normalized dataset.
[0033] S4. Machine learning validation, based on high-quality domain datasets, screens the optimal feature combination through data preprocessing and feature engineering, compares the model prediction results with real experimental data, and verifies the dataset validity and model reliability.
[0034] In a specific embodiment of the present invention, the data preprocessing is a process of improving data quality and availability through a series of technical means, including explicitly marking missing value parameters that are not mentioned in the literature extraction data, and supplementing high-frequency missing parameters in combination with statistical or domain knowledge; filtering outliers through domain thresholds and statistical methods to eliminate obviously unreasonable data; using one-hot encoding or label encoding to convert classification features into numerical types; performing logarithmic or Box-Cox transformation on numerical features with skewed distribution to improve data distribution; unifying dimensions through standardization or normalization to ensure that different features contribute evenly to the model; the entire process solves problems such as missing data, noise, and inconsistent formats, and provides clean and standardized input data for machine learning.
[0035] In a specific embodiment of the present invention, the feature engineering screening is specifically a process of screening key features and optimizing feature space through technical means, including eliminating irrelevant or redundant features based on statistical analysis and domain knowledge, and retaining key physical and chemical parameters; quantifying feature importance with the help of SHAP value, and further screening core features in combination with recursive feature elimination; for complex relationships, performing multiplication, square and other transformations on continuous features to construct new features that reflect synergistic effects; cross-combining classification features to capture the interactive effects of synthesis conditions; dimensionality reduction through principal component analysis to reduce dimensions while retaining high variance, and evaluating the model performance of different feature combinations, ultimately forming a feature subset with "high correlation and low redundancy" to provide efficient input for model training.
[0036] In a specific embodiment of the present invention, the verification data set validity and model reliability are specifically to use the extracted data for machine learning modeling, and to conduct data prediction research, combined with experimental verification data validity, thereby not only optimizing the information extraction method, but also providing theoretical guidance for materials science research.
[0037] In the experiment of photocatalytic hydrogen production from graphitic carbon nitride, data preprocessing was performed and the target variables were logarithmically transformed to improve data quality. After processing, the prediction accuracy of the CatBoost model was improved from 0.396 to 0.568; through feature engineering, features that have a greater impact on the photocatalytic hydrogen production rate of g-C3N4 were screened out, and new features were further constructed through feature interaction; the XGBoost algorithm and the CatBoost algorithm were used to establish a machine learning model, and the preprocessed data was input. At the same time, an ensemble learning algorithm was adopted, and the final prediction accuracy reached 0.7, which is significantly better than the traditional trial and error method.
Claims
1. An efficient data extraction method based on a large language model, characterized by: include: S1. Build an automated information extraction framework, using automated tools to simulate manual operations, extract key document data, and automate the entire process from document preprocessing and uploading to multi-round question-and-answer interactions and data storage. S2. Hint Engineering Optimization Strategy: Using a progressive hinting strategy, we build domain-specific hint chains. By carefully designing input hints, we can guide large language models to generate more accurate and relevant outputs. S3. Multi-model interaction and dataset construction: We select several large multimodal language models and leverage their differences in sensitivity to different parameters to extract data and perform manual annotation comparisons. By evaluating various metrics on parameter extraction for each model, we integrate the models with the best performance across all parameters to construct a high-quality domain dataset. S4. Machine learning validation, based on high-quality domain datasets, screens the optimal feature combination through data preprocessing and feature engineering, compares the model prediction results with real experimental data, and verifies the dataset validity and model reliability.
2. The method for efficient data extraction based on a large language model according to claim 1, characterized in that: The simulation of manual operations with the help of automated tools is specifically to simulate the interaction between humans and large language models through the Selenium automated testing tool, to realize the automation of the entire process of document uploading, automatic questioning, multiple rounds of question and answer, data extraction and storage, and greatly improve the efficiency of document data collection.
3. The method for efficient data extraction based on a large language model according to claim 1, characterized in that: The document preprocessing includes merging the main text and supporting information of the published documents, driving the browser to simulate file upload, question input and result parsing by writing code, and cleaning the text output by the large language model and writing it into a preset table.
4. The method for efficient data extraction based on a large language model according to claim 1, characterized in that: The progressive prompting strategy specifically means that each question asked by the model must be based on the previous answer; subsequent prompts are based on the previous answer. The specific process is: A1. Use initial prompts to let the model extract certain basic information from the literature, use the model output as context, and design subsequent prompts based on this context; A2. Ask for related information based on the answers. Each round of questions relies on the previous round's answers, forming a chain of prompts. A3. Guide the model to gradually focus on complex parameter extraction based on prompts and answers, ensuring the consistency and pertinence of information extraction.
5. The method for efficient data extraction based on a large language model according to claim 1, characterized in that: The construction of a domain-specific prompt chain is specifically to decompose a complex task in a certain professional field into multiple rounds of prompt sequences with logical connections; clarify the core information type required for the task, design the first prompt to focus on basic information extraction, and use the previous answer as the input basis for subsequent prompts, gradually increase the depth and details of information extraction, and standardize the output form through formatting; each round of prompts is based on the results of the previous round, forming an interlocking guidance path, so that the model can gradually clarify the task objectives during the interaction process and parse complex content layer by layer; the entire process converts the information logic in the domain into step-by-step instructions that can be understood by the model through an orderly arranged prompt sequence, ensuring that the information extracted in each step provides contextual support for subsequent prompts, and ultimately achieving structured output of complex information in a specific domain.
6. The method for efficient data extraction based on a large language model according to claim 1, characterized in that: The selection of several multimodal large language models specifically refers to the selection of several large language models that can process various forms of information input or output and have the ability to integrate cross-modal information, including models that support multi-format input such as text, images, and tables, and can recognize complex information such as formulas and charts in documents, to make up for the limitations of a single text model. By fusing information from different modalities, richer and more realistic interactions and task processing can be achieved, data in various formats can be extracted, and the comprehensiveness of chemical literature information extraction can be improved.
7. The method for efficient data extraction based on a large language model according to claim 1, characterized in that: The specific method for constructing the high-quality domain dataset is as follows: for each parameter, data extraction is performed through the interaction of several large models, and the model extraction performance of various models is evaluated in terms of the number distribution of the same features of the same data points of each parameter, accuracy, recall rate and F1 score. Then, an integrated approach is adopted to select the model with the highest accuracy on the parameter for extraction, and finally the model with the highest accuracy for each parameter is used to construct a high-quality dataset with the highest accuracy for each parameter. Finally, a high-quality dataset containing each core parameter is formed by combining the two models to ensure the accuracy and comprehensiveness of the data. The distribution of the number of identical features at the same data point can be used to evaluate the performance of the model in extracting a single data point from the perspective of the model as a whole. The quality of the single data point extracted by each model can be obtained through the frequency distribution diagram and overall heat map of the data extracted by several models. The accuracy rate is specifically the proportion of truly relevant documents extracted by the system. A high accuracy rate indicates that the content extracted by the system is more accurate. By comparing the accuracy data of several large models, the ones with the highest accuracy rates are selected for later application. The recall rate is specifically the proportion of relevant documents that the system successfully extracts from all actual relevant documents. A high recall rate indicates that the system can find more relevant information. By comparing the recall rate data of several large models, the ones with the highest recall rates can be selected for later application. The F1 score, specifically the harmonic mean of recall and precision, is used to comprehensively evaluate system performance. By comparing the F1 scores of several large models, the ones with the highest F1 scores are selected for later application.
8. The method for efficient data extraction based on a large language model according to claim 1, characterized in that: The data preprocessing is a process of improving data quality and usability through a series of technical means, including explicitly marking missing value parameters that are not mentioned in the literature extraction data, and supplementing high-frequency missing parameters with statistical or domain knowledge; filtering outliers through domain thresholds and statistical methods to eliminate obviously unreasonable data; converting categorical features into numerical types using one-hot encoding or label encoding; performing logarithmic or Box-Cox transformation on numerical features with skewed distribution to improve data distribution; unifying dimensions through standardization or normalization to ensure that different features contribute evenly to the model; the entire process solves problems such as missing data, noise, and inconsistent formats, and provides clean and standardized input data for machine learning.
9. The method for efficient data extraction based on a large language model according to claim 1, characterized in that: The feature engineering screening is specifically a process of screening key features and optimizing feature space through technical means, including eliminating irrelevant or redundant features and retaining key physical and chemical parameters based on statistical analysis and domain knowledge; quantifying feature importance with SHAP values and further screening core features in combination with recursive feature elimination; multiplying and squaring continuous features for complex relationships to construct new features that reflect synergistic effects; cross-combining classification features to capture the interactive effects of synthesis conditions; reducing dimensions through principal component analysis while retaining high variance, and evaluating the model performance of different feature combinations, ultimately forming a feature subset with "high correlation and low redundancy" to provide efficient input for model training.
10. The method for efficient data extraction based on a large language model according to claim 1, characterized in that: The verification of the validity of the data set and the reliability of the model is specifically to use the extracted data for machine learning modeling and to conduct research on data prediction, combined with experimental verification of the validity of the data, thereby not only optimizing the information extraction method, but also providing theoretical guidance for materials science research.