Solid-state battery database construction method based on large language model
By using small-sample cue words and a three-class confusion matrix evaluation method, this study addresses the shortcomings of large language models in text mining and the problem of unreasonable evaluation in the field of solid-state batteries. It achieves efficient and accurate text information extraction and database construction, and is applicable to text mining in other fields.
Patent Information
- Application Number
- CN202511345984.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-19
- Publication Date
- 2026-02-10
AI Technical Summary
Existing text mining techniques based on large language models suffer from insufficient text mining capabilities and unreasonable evaluation metrics in the field of solid-state batteries, resulting in poor text mining performance.
The method of using small sample prompt words guides the model to understand the task objective, and the optimal large language model is selected by the three-class confusion matrix evaluation method. The text mining capability is evaluated using three indicators: absolute accuracy, generalized accuracy, and missed detection rate, and a solid-state battery database is established.
It significantly improves the accuracy and efficiency of text mining, enabling the efficient extraction of key information about solid-state batteries from massive amounts of literature, and establishing a standardized database applicable to text mining in other fields.
Smart Images
Figure CN121501765A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of solid-state battery technology, and in particular relates to a method for constructing a solid-state battery database based on a large language model. Background Technology
[0002] Over the past few decades, materials science and energy storage technologies have experienced rapid development, especially the widespread attention garnered by lithium-ion batteries (LIBs) in electric vehicles (EVs) and energy storage. Traditional liquid lithium batteries are widely used in consumer electronics, grid energy storage, and automobiles, but they suffer from limitations in thermal safety, temperature adaptability, and energy density. Solid-state batteries represent one effective strategy for addressing these issues. To accelerate the commercialization of solid-state batteries, researchers worldwide have conducted extensive research on polymer solid-state electrolytes, inorganic solid-state electrolytes (such as sulfides and oxides), and composite electrolytes combining polymers and inorganic fillers. However, this research data is scattered across a vast amount of literature, lacking systematic integration. Therefore, to efficiently utilize this massive amount of literature information, it is crucial to develop a text mining method capable of accurately and efficiently extracting key parameters of solid-state electrolytes from the literature and to establish a standardized solid-state battery database.
[0003] Currently, text mining technology has evolved through three main technical approaches. Early text mining primarily employed a rule-based approach, extracting information through predefined rules. However, with the expansion of application areas and the increase in data complexity, rule-based methods required significant human intervention and expert annotation to update the rules, resulting in limited generalization capabilities. Subsequently, machine learning-based text mining techniques demonstrated significant advantages in handling complex language, automatic feature extraction, and processing large-scale data. However, this method remains limited by the ability to memorize and understand contextual information. With the development of artificial intelligence technology, thanks to the powerful semantic understanding capabilities of large language models, text mining techniques based on large language models are becoming the mainstream.
[0004] However, text mining techniques based on large language models have the following drawbacks:
[0005] 1. Large models still lack text mining capabilities in specific domains: Fine-tuning methods use large datasets of specific tasks to fine-tune large models to improve their text mining capabilities in specific domains, but require a lot of data resources and training time, and have high computational requirements; RAG methods create a vectorized database of text information and retrieve specific information from the vector database through semantic similarity retrieval, but the accuracy of this method is affected by the vectorized model and has high complexity.
[0006] 2. Inappropriate evaluation metrics for text mining: Common evaluation methods for large models, such as BLEU, METEOR, and BERTScore, are based on n-gram matching degree or semantic similarity between reference text and generated text to evaluate the performance of large models. These methods cannot accurately determine the precise matching of chemical formulas and numerical values.
[0007] The aforementioned shortcomings result in poor text mining capabilities of large models in the field of solid-state batteries and unreasonable evaluation of text mining capabilities. Summary of the Invention
[0008] The purpose of this invention is to provide a method for constructing a solid-state battery database based on a large language model. By designing a method for using small-sample cue words for text mining and evaluating text mining capabilities using a three-class confusion matrix method, the aforementioned problems can be effectively solved. The technical solution adopted is as follows:
[0009] A method for constructing a solid-state battery database based on a large language model includes the following steps:
[0010] Step 1: Obtain N PDF documents on solid-state batteries;
[0011] Step 2: Select X documents from N documents and extract the text information from X documents as the validation set, where X < N;
[0012] Step 3: Design sample prompts and add extraction examples to them;
[0013] Step 4: Select the optimal large language model based on the validation set;
[0014] Step 5: Input the text information and sample prompts from NX documents into the optimal large language model to obtain solid-state battery information;
[0015] Step 6: Establish a solid-state battery database based on the solid-state battery information obtained in Step 5.
[0016] Preferably, in step 4, the optimal large language model is selected using the three-class confusion matrix evaluation method, which specifically includes:
[0017] Step 4A: Parse X PDF documents on solid-state batteries to obtain text information;
[0018] Step 4B: Input the text information obtained in Step 4A into each large language model, and each large language model outputs the model extraction results.
[0019] Extract the documents from step 4A to obtain the manual extraction results;
[0020] Calculate the three-class confusion matrix corresponding to the large language model based on the extraction results of each model and the manual extraction results;
[0021] The value types of both the model extraction results and the manual extraction results include single value A, multiple value B, and null value C; the elements in the three-class confusion matrix are combinations of one value type from the model extraction results and one value type from the manual extraction results;
[0022] Step 4C: Calculate each evaluation index based on the three-class confusion matrix, and select the large language model with the highest comprehensive score of the evaluation index as the optimal large language model.
[0023] Preferably, the evaluation metrics include: absolute accuracy (AAR), generalized accuracy (GAR), and missed detection rate (LR).
[0024] Preferably, the calculation model for the evaluation index is as follows:
[0025]
[0026] Preferably, the sample prompts include: the type of information to be extracted, the rules for extracting material properties, the chemical formula specifications, and the output requirements.
[0027] Preferably, the types of information extracted include: solid electrolyte, ionic conductivity, discharge capacity, and cycle performance.
[0028] Compared with the prior art, the advantages of the present invention are:
[0029] 1) Small sample cue word engineering: Small sample cue words (sample cue words) can effectively guide the model to quickly understand the task objective and output format by providing a small number of specific examples in the input, significantly reducing the bias caused by semantic ambiguity in zero-shot scenarios and improving the accuracy of text mining.
[0030] Here, "small sample" refers to the number of specific examples in the prompt, which is 1-5. Therefore, a sample of less than or equal to 5 is considered a small sample.
[0031] 2) Establishment of the three-class confusion matrix evaluation method: The three-class confusion matrix is an extension of the original two-class confusion matrix, and introduces three new evaluation indicators: absolute accuracy, generalized accuracy and missed detection rate, which are used to evaluate the text mining capabilities of large models and select the best large models for the establishment of a standardized database for solid-state batteries. Attached Figure Description
[0032] Figure 1 Flowchart for designing sample prompt words;
[0033] Figure 2 Flowchart for large language model evaluation and selection;
[0034] Figure 3 Flowchart for parsing PDF documents;
[0035] Figure 4 A schematic diagram illustrating the principle of a solid-state battery database construction method based on a large language model;
[0036] Figure 5 Flowchart of a method for constructing a solid-state battery database based on a large language model;
[0037] Figure 6 Examples of sample prompt words and sample results extracted by the model;
[0038] Figure 7 A performance comparison chart of five large language models in text mining capabilities;
[0039] Figure 8 A visualization interface for ionic conductivity;
[0040] Figure 9 A visualization interface for discharge performance;
[0041] Figure 10 This is a visualization of the loop performance interface. Detailed Implementation
[0042] The following will describe in more detail the solid-state battery database construction method based on a large language model according to the present invention, with reference to the schematic diagrams, which illustrate preferred embodiments of the present invention. It should be understood that those skilled in the art can modify the present invention described herein while still achieving its advantageous effects. Therefore, the following description should be understood as being of general knowledge to those skilled in the art and is not intended to limit the present invention.
[0043] like Figures 1-10 A method for constructing a solid-state battery database based on a large language model includes the following steps:
[0044] Step 1: Obtain N PDF documents about solid-state batteries.
[0045] Step 2: Select X documents and extract the text information from X documents as the validation set, where X < N.
[0046] That is, step 2 involves parsing and segmenting the PDF document into blocks for subsequent large language model information extraction.
[0047] Among them, "extracting text information from X documents" refers to the PDF parsing process.
[0048] like Figure 3 As shown, regarding PDF parsing:
[0049] 1) Input document:
[0050] Downloaded PDF documents in batches;
[0051] 2) PDF parsing:
[0052] PDF documents are parsed in batches using a loop. For documents that are successfully parsed, the text is further segmented, and the parsed and segmented text information is output as the result and fed into the large language model.
[0053] If the parsing fails, skip that document and proceed to the parsing process for the next document.
[0054] Step 3: Design sample prompts and add extraction examples to them, such as... Figure 1 As shown.
[0055] Step 3A: Design sample prompts.
[0056] Prompt words typically contain elements such as instructions, context, input data, and output indications.
[0057] The instructions represent the specific tasks that the model is to perform.
[0058] The context mainly contains external or additional contextual information, such as some extracted examples, to guide the language model to respond better;
[0059] The input data includes content or questions entered by the user.
[0060] Output instructions define the format or type of the model's output.
[0061] In this embodiment, the sample prompts include: the type of information to be extracted, the rules for extracting material properties, the chemical formula specifications, and the output requirements.
[0062] Adding extracted examples is an existing technique, known as in-context learning in large models.
[0063] Step 3A emphasizes inserting battery-specific expressions and terminology as examples to achieve in-context learning for battery-domain knowledge alignment.
[0064] As shown in Table 1, the types of information extracted include: solid electrolyte, ionic conductivity, discharge capacity, and cycle performance.
[0065] Table 1 Types of Information Extracted
[0066]
[0067] In Table 1, "solid electrolyte" refers to the ionic conductor that exists in solid form in a solid battery.
[0068] Step 3B: Input the sample prompts and validation set into each large language model. The large language model mines the text information and outputs the model extraction results.
[0069] Step 3C: Evaluate the accuracy of the model's extraction results. If the accuracy is less than the set value, optimize the sample prompt words.
[0070] The validation set consists of manually extracted true values, and the model's extraction results are compared with the true values.
[0071] One method to optimize sample prompts is to add extracted examples to the sample prompts.
[0072] Optimized sample prompts such as Figure 6 As shown.
[0073] Figure 6 The extracted examples included in the text are: 3.22x10 -4 S / cm@25℃; 144.2mAh / g@0.5C, 200 cycles; 88.9%@200 cycles, 0.5C; Li7La3Zr2O 12 .
[0074] Sample prompts effectively guide the model to quickly understand the task objective and output format by providing a few specific examples in the large model input. The few-shot method establishes the task paradigm in a simpler way, ensuring both output accuracy and efficiency.
[0075] In this embodiment, the model's extraction accuracy and stability are significantly improved through sample prompt word optimization in step 3 (by embedding extraction examples into the prompt words). The final model extraction results and corresponding sample prompt words in step 3 are as follows: Figure 6 As shown.
[0076] Step 4: Select the optimal large language model using the three-class confusion matrix evaluation method, such as... Figure 2 As shown.
[0077] In step 4, the original binary confusion matrix is expanded into a ternary confusion matrix, and three new evaluation metrics are introduced: absolute accuracy, generalized accuracy, and missed detection rate.
[0078] Step 4 specifically includes the following steps:
[0079] The three-category confusion matrix evaluation method specifically includes:
[0080] Step 4A: Parse the PDF document on solid-state batteries to obtain text information (document text information), such as... Figure 3 As shown. This section analyzes the X documents selected in step 2.
[0081] Step 4B: Input the text information obtained in Step 4A into each large language model, and each large language model outputs the model extraction results.
[0082] Extract the documents from step 4A to obtain the manual extraction results;
[0083] Calculate the three-class confusion matrix corresponding to the large language model based on the extraction results of each model and the manual extraction results.
[0084] If M large language models are used in step 4B, then M model extraction results and M three-class confusion matrices are obtained.
[0085] The value types of both the model extraction results and the manual extraction results include single value A, multiple value B, and null value C;
[0086] As shown in Table 2, the elements in the three-class confusion matrix are a combination of one value type from the model extraction result and one value type from the manual extraction result.
[0087] Table 2 Definition of the Three-Class Confusion Matrix
[0088] Multi-class confusion matrix A B C A AA AB AC B BA BB BC C CA CB CC
[0089] 1) Definition of parameters for the three-class confusion matrix:
[0090] AA: The actual value is a single value, and the extracted value is also a single value, and it is consistent with the actual value;
[0091] AB: The true value is a single value, the extracted value is multiple values, and the true value belongs to the extracted value;
[0092] AC: The true value is a single value, and the extracted value is not equal to the true value;
[0093] BA: The actual value is multi-valued, and the extracted value is either a single value or multiple values, and they are mutually dependent.
[0094] BB: The true value is multiple values, the extracted value is multiple values, and the true value and the extracted value are equal;
[0095] BC: The true value is multiple, and the extracted value is not equal to the true value;
[0096] CA: The actual value is a null value, and the extracted value is a single value;
[0097] CB: The actual value is null, and the extracted value is multi-valued;
[0098] CC: The actual value is null, and the extracted value is null;
[0099] Regarding single values, double values, and null values:
[0100] Suppose the following information was extracted from the paper: 144.2 mAh / g @ 0.5C, 200 cycles; Li7La3Zr2O 12 .
[0101] Then Li7La3Zr2O 12 That is, a single value;
[0102] 144.2 mAh / g@0.5C, 200 cycles is a multiple value, which includes three values: 144.2, 0.5 and 200. In actual large-scale model extraction, one or more of these values may be extracted, or none may be extracted. If none of them are extracted by the large-scale model, it is a null value.
[0103] Step 4C: Calculate each evaluation index based on the three-class confusion matrix, and select the large language model with the highest comprehensive score of the evaluation index as the optimal large language model.
[0104] Evaluation metrics include: Absolute Accuracy (AAR), Generalized Accuracy (GAR), and Missed Response Rate (LR).
[0105] 1. Absolute Accuracy (AAR) refers to the strict consistency between the information extracted manually from solid-state batteries and the information extracted by the large model. It reflects the completeness of information extraction by the large model in text mining. The calculation formula is:
[0106]
[0107] 2. Generalized accuracy (GAR) refers to the correlation between manually extracted information and information extracted by the large model, reflecting the model's breadth in information extraction. The calculation formula is:
[0108]
[0109] 3. The Retention Rate (LR) is used to reveal the differences between manual extraction and large-scale model extraction, reflecting the impact of the biases in the information extraction process on evaluating the information extraction ability of large-scale models. The calculation formula is as follows:
[0110]
[0111] Calculate the scores (evaluation metrics) of AAR, GAR, and LR for each large language model, and sum the scores of the three metrics.
[0112] The large language model with the highest sum of scores is selected as the large model for text mining.
[0113] In this embodiment, step 4 selects DeepseekV3, GLM-4-plus, Llama3.3-70B-Instruct, Qwen2.5-7B-Instruct, and Internm3-8B-Instruct as mainstream large language models for text mining capability evaluation, and DeepseekV3 is selected as the optimal large language model for text mining. Performance of different models is then discussed. Figure 7 As shown.
[0114] Step 5: Input the text information and sample prompts from NX documents into the optimal large language model to obtain solid-state battery information. The specific process is as follows: Figures 4-5 As shown.
[0115] Figure 5 In the text, "text information mining based on a large language model with small sample prompt words" refers to "inputting the text information and sample prompt words of NX documents into the optimal large language model".
[0116] Figure 6 In this context, "output results" refers to the model extraction results, specifically the "solid-state battery information" in step 5.
[0117] Figure 4 In the process of parsing the input documents into PDF format, the data is transmitted to the optimal large language model. The optimal large language model extracts solid-state battery information from the documents by combining small sample prompts and cleans the information. Finally, the cleaned data is stored in the established standardized database.
[0118] Step 6: Establish a solid-state battery database based on the solid-state battery information obtained in Step 5.
[0119] Figure 5 The steps “Solid-state battery information processing” and “Storing into the solid-state battery standardization database” correspond to step 6.
[0120] This enables the extraction of specific solid-state battery information from massive amounts of solid-state battery literature using large-model text mining techniques based on small-sample cue words, and the establishment of a standardized solid-state battery database and data visualization. Figures 8-10 As shown.
[0121] Figure 8 A visualization interface for ionic conductivity;
[0122] Figure 8 The English explanation is as follows:
[0123] ionic conductivity;
[0124] Material: The chemical formula of a solid electrolyte;
[0125] Temperature: Test temperature;
[0126] Figure 8 In the figure, the horizontal axis represents the solid electrolyte composition, and the vertical axis represents the ionic conductivity and the test temperature.
[0127] Figure 9 This is a visualization interface for discharge performance.
[0128] Figure 9 The English explanation is as follows:
[0129] Discharge Capacity: Discharge capacity;
[0130] Discharge Rate: Discharge rate;
[0131] Cycle Number: Number of discharges;
[0132] Figure 9 In the figure, the horizontal axis represents the solid electrolyte composition, and the vertical axis represents the discharge capacity and discharge rate.
[0133] Figure 10 A visualization interface for loop performance;
[0134] Figure 10 The English explanation is as follows:
[0135] Cycle Performance: Cyclic performance;
[0136] Cycle Rate: Cycle rate;
[0137] Cycle Number: Number of cycles;
[0138] DOI: Digital Object Identifier, is a unique identifier for a research paper.
[0139] Figure 10 In the graph, the horizontal axis represents the solid electrolyte composition, and the vertical axis represents the cycle performance and cycle ratio.
[0140] The standardized solid-state battery data can be visualized to identify and analyze materials with high cycle stability.
[0141] In this embodiment, the specific operation is as follows:
[0142] 1. Download 1000 PDF articles on solid-state batteries from online resources;
[0143] 2. Select 100 papers for the selection of the large model, and manually extract solid-state battery information from the 100 papers for use as a validation set;
[0144] 3. Design and optimize small-sample cue words for text mining in large models.
[0145] 4. Using the newly established three-class confusion matrix evaluation method, the DeepseekV3 model was selected for subsequent text mining.
[0146] 5. Utilize DeepseekV3 to further mine solid-state battery information from the remaining 900 papers.
[0147] 6. Clean the mined data, and establish a standardized database for solid-state batteries to store and visualize the data.
[0148] In summary, the advantages of this invention are:
[0149] 1. Improved rationality of large model text mining capability assessment: By using a three-class confusion matrix, three indicators are introduced: absolute accuracy, generalized accuracy, and missed detection rate, which effectively assesses the text mining capability of large models and avoids the problem of excessively high accuracy caused by similarity evaluation methods.
[0150] 2. Improved accuracy in large-scale text mining: On a validation set of 100 documents, the text mining method based on small-sample cue words effectively improved the accuracy of large-scale text mining. DeepseekV3 achieved an average generalized accuracy of 0.895.
[0151] 3. Wide applicability: This method does not require a large training dataset to train a large model's text mining capabilities in a specific domain; reasonable small-sample cue words are sufficient. By modifying the small-sample cue words, this method can be effectively applied to text mining in other domains.
[0152] The above are merely preferred embodiments of the present invention and do not constitute any limitation on the present invention. Any equivalent substitutions or modifications made by those skilled in the art to the technical solutions and content disclosed in the present invention without departing from the scope of the present invention shall be deemed to have remained within the protection scope of the present invention.
Claims
1. A method for constructing a solid-state battery database based on a large language model, characterized in that, Includes the following steps: Step 1: Obtain N PDF documents on solid-state batteries; Step 2: Select X documents from N documents and extract the text information from X documents as the validation set, where X < N; Step 3: Design sample prompts and add extraction examples to them; Step 4: Select the optimal large language model based on the validation set; Step 5: Input the text information and sample prompts from NX documents into the optimal large language model to obtain solid-state battery information; Step 6: Establish a solid-state battery database based on the solid-state battery information obtained in Step 5.
2. The method for constructing a solid-state battery database based on a large language model according to claim 1, characterized in that, In step 4, the optimal large language model is selected using the three-class confusion matrix evaluation method. The three-class confusion matrix evaluation method specifically includes: Step 4A: Parse X PDF documents on solid-state batteries to obtain text information; Step 4B: Input the text information obtained in Step 4A into each large language model, and each large language model outputs the model extraction results. Extract the documents from step 4A to obtain the manual extraction results; Calculate the three-class confusion matrix corresponding to the large language model based on the extraction results of each model and the manual extraction results; The value types of both the model extraction results and the manual extraction results include single value A, multiple value B, and null value C; the elements in the three-class confusion matrix are combinations of one value type from the model extraction results and one value type from the manual extraction results; Step 4C: Calculate each evaluation index based on the three-class confusion matrix, and select the large language model with the highest comprehensive score of the evaluation index as the optimal large language model.
3. The method for constructing a solid-state battery database based on a large language model according to claim 2, characterized in that, The evaluation metrics include: absolute accuracy (AAR), generalized accuracy (GAR), and missed detection rate (LR).
4. The method for constructing a solid-state battery database based on a large language model according to claim 2, characterized in that, The calculation model for the evaluation indicators is as follows:
5. The method for constructing a solid-state battery database based on a large language model according to claim 1, characterized in that, The sample prompts include: the type of information to be extracted, the rules for extracting material properties, the chemical formula specifications, and the output requirements.
6. The method for constructing a solid-state battery database based on a large language model according to claim 5, characterized in that, The types of information extracted include: solid electrolyte, ionic conductivity, discharge capacity, and cycle performance.