A scientific literature information extraction method based on a general large language model

Abstract

This invention discloses a method for extracting scientific information from scientific literature based on a general large language model, comprising: Step 1, constructing a sample set and preprocessing the scientific literature to be processed by dividing it into text blocks; Step 2, constructing initial prompts based on basic prompt templates, domain expert knowledge, and a training subset of the sample set, and obtaining the optimal prompt words through iterative optimization; Step 3, solving for the optimal number of context examples based on the sample set, and further obtaining global context examples; Step 4, for each text block of the scientific literature to be extracted, the information extraction model outputs structured extraction results. This method can automatically extract structured information from scientific literature without the need for large-scale labeled data and model training, significantly reducing the cost of manual labeling and model training; by improving the quality of prompt words and context examples, it can effectively improve the accuracy and consistency of information extraction results.

Description

Technical Field

[0001] This invention relates to the field of text information extraction, and more particularly to a method for extracting scientific literature information based on a general large language model. Background Technology

[0002] With the continuous growth in the number of scientific documents, how to automatically extract high-quality structured information from massive amounts of unstructured text and provide a stable data foundation for subsequent knowledge organization, statistical analysis, and scientific discovery has become an important research problem in the field of natural language processing. Compared with general text, scientific documents are usually characterized by dense terminology, diverse expressions, long sentences, strong cross-sentence semantic dependencies, and high domain knowledge requirements. This puts traditional rule-based or supervised learning-based information extraction methods under considerable pressure in terms of annotation costs, domain transfer capabilities, and long text modeling.

[0003] Information extraction is a common task in text processing, typically used to identify and extract structured information such as entities, attributes, relationships, or events from text. Existing information extraction methods can be implemented based on rules and templates, or rely on machine learning models to learn extraction capabilities from labeled data. In recent years, the development of general-purpose large language models has enabled information extraction to be performed in a prompt-word driven manner, without the need for complex training for specific tasks.

[0004] Scientific literature information extraction techniques based on large language models typically employ a holistic process of "document formatting—prompt-driven extraction—result display and saving" to structure the document content. The basic idea of ​​this approach is to first convert PDF documents into processable structured data, and then call a large language model for recognition and organization based on pre-defined extraction targets. While this approach can reduce manual processing costs to some extent, in practice, prompt word construction often relies heavily on human experience, and the organization of contextual examples often uses fixed templates or static sample sets. Therefore, it is difficult to adaptively adjust to different document topics, expression styles, and semantic differences among test samples, thus affecting the stability and accuracy of the extraction results.

[0005] While existing technologies have explored aspects such as document formatting, cue word construction, and sample selection, they still generally suffer from problems such as empirical cue word construction, static organization of contextual examples, and insufficient granularity in example selection. They are still difficult to simultaneously take into account the semantic differences between different test samples, the quality distribution of candidate samples, and the representativeness of a small number of sample contexts. Summary of the Invention

[0006] Purpose of the invention: The technical problem to be solved by the present invention is to address the shortcomings of the existing technology by providing a method for extracting scientific literature information based on a general large language model.

[0007] To address the aforementioned technical problems, this invention discloses a scientific literature information extraction method based on a general large language model. Through the collaborative design of prompt word construction and context example construction, the method enhances task relevance in prompt content. Furthermore, the context example construction incorporates perplexity grouping, example quantity selection, and dynamic filtering based on semantic similarity within groups, thereby improving the accuracy and stability of scientific literature information extraction. Within the framework of the general large language model, for scientific literature information extraction tasks, prompt words that can integrate domain knowledge are automatically constructed. Further, by combining average perplexity grouping of context examples, adaptive determination of the example quantity, and filtering based on semantic similarity within groups, a more stable and adaptable set of context examples for test samples is automatically generated, thus improving the accuracy and consistency of scientific literature information extraction.

[0008] Step 1: Construct a sample set and preprocess the scientific literature to be processed by dividing it into text blocks;

[0009] Step 2: Construct initial prompts based on the basic prompt template, domain expert knowledge, and a training subset of the sample set, and obtain the optimal prompt words through iterative optimization;

[0010] Step 3: Calculate the optimal number of context examples based on the sample set, and further obtain the global context examples;

[0011] Step 4: For each text block of the scientific document to be extracted, the information extraction model outputs structured extraction results.

[0012] The sample set mentioned in step 1 is a set of labeled samples that are in the same or similar field as the scientific literature to be processed. The sample set includes at least a training subset, a validation subset, and a candidate sample pool.

[0013] The training subset is used for prompt optimization, the validation subset is used to evaluate prompt quality and the number of context examples, and the candidate sample pool is used to generate a set of context examples for the text block to be extracted.

[0014] The preprocessing described in step 1 specifically includes:

[0015] Step 1-1: Parse the scientific documents in PDF format and convert them into MD format files to retain the titles, chapters, paragraphs, tables and other structural information of the documents;

[0016] Step 1-2: Divide the obtained MD format scientific documents into text blocks under dual constraints, including article logic constraints and text block length constraints.

[0017] Specifically, the target number of words in each text block is preset. First, the logical blocks unique to scientific literature, such as the chapter titles of introduction, methods, results, and discussion, are identified and divided. For chapters that are too long, they are further split evenly. For chapters that are too short, they are merged with adjacent chapters so that the length of each text block is kept within 1 / 2 to 2 times the preset number of words, while preserving the original structural information of the literature as much as possible.

[0018] The basic prompt template mentioned in step 2 includes task description, output format, and basic constraints, and is unrelated to specific domain content;

[0019] Step 2 specifically includes:

[0020] Step 2-1: Based on domain expert knowledge, use the prompt optimization model to automatically fill in the basic prompt template to form an initial prompt; wherein, while keeping the general task structure unchanged, the initial prompt explicitly writes domain terms, output boundaries, field constraints and extraction requirements into the prompt content;

[0021] Specifically, before the information extraction process begins, based on the information type, field definition, field boundaries and inter-field constraints of the information entities to be extracted, the relevant domain experts determine the information type, field definition, field boundaries and inter-field constraints of the information entities to be extracted. When the initialization of the prompt optimization is performed, the prompt optimization model is input and automatically loaded into the basic prompt template.

[0022] Step 2-2: On the training subset of the sample set, call the information extraction model to perform information extraction, and input the information extraction results and the training subset annotations into the prompt optimization model. The prompt optimization model will automatically adjust the prompt words according to the extraction effect.

[0023] The prompting optimization model described in step 2-1 adopts an existing large language model, such as the DeepseekV3.2 model, while the information extraction model adopts a smaller language model, such as the Qwen2.5-14B model.

[0024] Step 2-3: Evaluate the current prompt based on the validation subset in the sample set, and determine whether the extraction results on the validation set tend to be stable or whether the preset number of iterations has been reached; if so, output the optimal prompt word; otherwise, repeat step 2-2.

[0025] The automatic adjustment of prompt words mentioned in step 2-2 specifically includes:

[0026] First, analyze the error between the extraction results and the annotation, then attribute the error to different causes. A dynamic correction strategy library is maintained for different error types. New error causes are automatically added, and existing strategies are used with a certain probability for recorded error causes; otherwise, a new correction strategy is designed to enhance the diversity of iterative optimization. The probability setting reference value is 0.7.

[0027] The optimal number of context examples mentioned in step 3 is not predetermined, but is determined by combining the input constraints of the information extraction model, the sample difficulty distribution, and the performance of the validation set, so as to form a more stable matching relationship between subsequent context examples and the text blocks to be extracted.

[0028] Specifically, step 3 includes:

[0029] Step 3-1: Based on the maximum input length of the information extraction model, the prompt word length, the maximum length of the samples in the sample set, and the maximum length of the text block to be extracted, calculate the maximum number of context examples that can be accommodated in the worst case, and determine the candidate context example set accordingly; wherein, the candidate context example set is a set of multiple integer values ​​that satisfy the input length constraint;

[0030] Step 3-2: Calculate the average perplexity of each sample based on the information extraction model, and divide the samples into multiple perplexity groups according to the perplexity level; among them, the sample with higher perplexity has a higher difficulty in the recognition of the text by the representation model, and the sample with lower perplexity has a lower difficulty in the recognition of the text by the representation model.

[0031] Step 3-3: For each number of candidate context examples, perform balanced sampling within each perplexity group to form a context example group for validation evaluation; the unsampled samples constitute the validation set.

[0032] Steps 3-4: The context example group used for verification and the samples in the verification set are vectorized using a sentence vector model. The sentence vector model is used to map text into vector representations to calculate semantic similarity; all-MiniLM-L6-v2 can be used.

[0033] Steps 3-5: For each context example group, calculate the semantic similarity between each sample in the validation set and the candidate samples. For each perplexity group, select the K samples with the highest semantic similarity to form the evaluation validation set corresponding to that context example group.

[0034] K is a preset validation set size parameter. The semantic similarity can be achieved using sentence vector cosine similarity, dual encoder similarity, or other measures that can characterize the semantic similarity of texts.

[0035] Steps 3-6: Input the selected context examples and the evaluation validation set samples into the information extraction model, perform information extraction on the evaluation validation set, record the evaluation metric performance, and determine whether the preset number of repetitions has been reached. If not, return to steps 3-3 to 3-36; if the preset number of repetitions has been reached, calculate the average value after multiple repetitions, and select the number of context examples with the best performance on the metric as the optimal number of context examples N. The evaluation metric can be the F2 metric, a commonly used comprehensive evaluation metric for information extraction. The calculation formula is F2 = 5 × (precision × recall) / (4 × precision + recall), which focuses on measuring the model's recall ability.

[0036] Steps 3-7: Within each perplexity group of the candidate sample pool, multiple candidate context example sets are generated by random sampling; based on the structural differences between the sets, diversity constraints are introduced to filter the candidate context example sets. Finally, within each perplexity group, several example sets that satisfy the diversity constraints are retained, such that the number of samples in each example set is equal to the number of optimal context examples N, and the sets are combined to form a global context example.

[0037] The introduction of diversity constraints specifically includes:

[0038] (1) The differences in the type distribution of information entity fields in the output of each sample can be obtained by calculating the JS divergence;

[0039] (2) The field coverage of the information entity field in each sample output can be obtained by calculating the Jaccard distance.

[0040] Take the arithmetic mean of each pair of results. Sum the two results with weights to obtain the overall diversity. Retain the results with high diversity.

[0041] The structural differences are characterized by comparing the type distribution and field coverage of information entity fields in the output of each sample, and redundancy removal is performed on the candidate set accordingly to reduce field overlap and improve coverage.

[0042] Step 4 specifically includes:

[0043] Step 4-1: For each scientific document text block to be extracted, calculate its set-level semantic similarity with the global context examples constructed in Step 3-7; wherein, the set-level semantic similarity is defined as the average of the semantic similarities between the text block and each sample in the set. Subsequently, within each perplexity group of the candidate sample pool, sort according to the set-level semantic similarity, select the context example set with the highest score as the optimal example set for that group; and merge the optimal example sets corresponding to each group to form the context example set for information extraction.

[0044] Step 4-2: Input the optimal prompt words, the set of context examples, and the scientific literature text blocks to be extracted into the information extraction model, which will then output structured extraction results. This process can be performed in parallel on all text blocks.

[0045] Step 4-3: Summarize and organize the information extraction results corresponding to each text block, and output them as an Excel spreadsheet.

[0046] Beneficial effects:

[0047] From a technical perspective, the technical solution of this invention (1) introduces domain expert knowledge into the prompt initialization process through a two-stage prompt optimization mechanism, and iteratively corrects the prompts by combining the error analysis results on the validation set, thereby enhancing the adaptability of prompt words to professional terms, extraction boundaries and output format constraints in scientific literature, and improving the accuracy and stability of information extraction; (2) by grouping candidate examples based on average perplexity, and dynamically selecting the most relevant samples according to semantic similarity within the group after determining the total number of examples, the context examples can simultaneously cover text expressions of different complexities and maintain high semantic consistency with the test samples, thereby reducing performance fluctuations caused by random sampling and fixed example configuration; (3) without relying on model parameter fine-tuning, a stable prompt construction and context example selection mechanism suitable for scientific literature information extraction tasks is constructed, improving the universality, transferability and adaptability of the method to different scientific subdomains.

[0048] From an application perspective, the technical solution of this invention (1) enables the automatic extraction of structured information from scientific literature without the need for large-scale labeled data and model training, significantly reducing the cost of manual labeling and model training; (2) by improving the quality of prompt words and context examples, it can effectively improve the accuracy and consistency of information extraction results, and reduce the impact of extraction errors on subsequent knowledge organization, statistical analysis and scientific research conclusions; (3) the constructed method has strong reusability and scalability, and can be easily transferred to different disciplines or different extraction tasks, providing technical support for the automated processing and knowledge discovery of scientific literature. Attached Figure Description

[0049] Figure 1 This is the flowchart for this method. Detailed Implementation

[0050] This embodiment focuses on information extraction from scientific literature in the field of foraminifera fossils, extracting fossil names based on given geological periods. The sample set consists of 54 formal English academic papers in this field, with an average length of approximately 9145 words, ranging from 4976 to 19124 words per paper. The original PDFs were first converted into structured text using a document parsing tool, and then divided into balanced blocks according to a reference length of 4096 tokens. The information extraction model used was Qwen2.5-14B, and the optimization model was uniformly DeepSeek-V3.2.

[0051] Step 1: Construct a sample set and preprocess the scientific literature to be processed, dividing it into text blocks. In this embodiment, specifically: the sample set includes 1238 samples, comprising a training subset of 185, a validation subset of 437, and a candidate sample pool of 616. The training subset is used for iterative optimization of prompts, the validation subset is used to evaluate the quality of prompts and the number of context examples, and the candidate sample pool is used to generate a set of context examples for the text blocks to be extracted.

[0052] PDF scientific documents are first parsed into a unified text, and then divided into blocks according to the document structure to ensure that the semantics of the chapters are as complete as possible and to control the length of the context.

[0053] Sample example:

[0054] "input": "Figure 4. Stratigraphic range chart illustrating the distribution of key planktonic species across the boundary. Samples wereprocessed using standard micropaleontological techniques. The Oligocene-Eocene transition is characterized by a major extinction event among thehantkeninids. The disappearance of Hantkenina alabamensis andCribrohantkenina inflata provides a reliable biostratigraphic marker for thetop of the Eocene. This faunal turnover is globally recognized.\n\n< <gt>>\nEOT",

[0055] "output": "Hantkenina alabamensis\nCribrohantkenina inflata",

[0056] The corresponding text block form (but not so short): Figure 4. Stratigraphic range chartillustrating the distribution of key planktonic species across the boundary. Samples were processed using standard micropaleontological techniques. TheOligocene-Eocene transition is characterized by a major extinction eventamong the hantkeninids. This faunal turnover is globally recognized.\n\n

[0057] The domain expert knowledge mentioned in Step 1 comes from: relevant domain literature, annotation standards, and manual extraction experience. Taking the foraminifera fossil task as an example, the expert knowledge clarifies the role as an earth scientist / paleontologist, the target entity as the fossil name constrained by geological time, and provides characteristics such as common morphology of fossil names, time anchor constraints, exclusion rules, and easily confused patterns.

[0058] (Role) Geoscientist

[0059] (Field) Domain: paleontology, scientific text mining

[0060] (Task)Task: extract fossil entity names associated with a given geologic time from scientific literature

[0061] (Target Entity)Target Entity: fossil entity names constrained by geologic time

[0062] (Entity_label)Entity_label: Fossil Names

[0063] (Features)Features:

[0064] 1. Fossil names are usually genus / species forms, genus-only forms, or forms with markers such as sp., spp., cf.

[0065] 2. Extract only fossils that are explicitly linked to the specified geologic time in the same local context.

[0066] 3. If text contains a time anchor, use it as a strict filter condition.

[0067] 4. Exclude fossils mentioned only for other ages or outside the target age scope.

[0068] How Expert Would Do It:

[0069] - Comprehensive reading

[0070] - First identify the target geologic time condition

[0071] - Extract only explicit fossil mentions under that condition

[0072] Input-output format demonstration - Example:

[0073] Input:

[0074] Document contains:

[0075] "... Acarinina bullbrooki and Globigerinatheka index are common in Late Eocene assemblages... Morozovella velascoensis is reported from Paleocene..."

[0076] Given geologic time: Late Eocene

[0077] Output:

[0078] <<Extracted Fossil Names>>

[0079] Acarinina bullbrooki

[0080] Globigerinatheka index

[0081] <<End Extract>>

[0082] Ensure the prompt template contains a {gt} field in the input; if absent, insert {gt} to allow replacement with the specified geologic time.

[0083] Step 2: Construct initial prompts based on the basic prompt template, domain expert knowledge, and a training subset of the sample set, and obtain the optimal prompt word through iterative optimization. The basic prompt template only includes general content such as task role, target entity, entity features, extraction rules, output format, and input text. After domain expert knowledge is injected, the template explicitly supplements the target entity definition, time constraints, synonyms, exclusions, and extraction boundaries. Subsequently, the prompt optimization model combines the extraction error on the training subset to refine the prompts round by round. When the performance on the validation set stabilizes or the maximum number of iterations (3) is reached, the optimal prompt word is output. Basic prompt template:

[0084] # Role

[0085] You are a **{domain_expert_role}** specializing in **{research_domain}**. Your task is to extract **{target_entity_type}** from scientific literature with 100% fidelity to the original text.

[0086] # Extraction Rules

[0087] * **Literal Extraction**: Copy entities **exactly** as written (including abbreviations, modifiers, and punctuation).

[0088] * **Zero Inference**: Do NOT normalize, expand abbreviations, orinfer missing names. If it's not explicitly in the text, do not extract it.

[0089] * **Exclusions**: Strictly ignore:

[0090] * {excluded_category_1}

[0091] * {excluded_category_2}

[0092] * {excluded_category_3}

[0093] * **Features**: Entities typically exhibit: {core_features}.

[0094] # Output Format

[0095] ```text

[0096] <<Extracted {entity_label}>>

[0097] {entity_1}

[0098] {entity_2} ...

[0100] <<End Extract>>

[0101] ```

[0102] *Note: One entity per line. If no entities are found, write "null"between the tags.*

[0103] ---

[0104] # Examples

[0105] {examples}

[0106] ---

[0107] # Input

[0108] ```text

[0109] < <start>>

[0110] {doc}

[0111] < <end>>

[0112] ```

[0113] ```

[0114] Examples of domain knowledge injection prompts:

[0115] ## **Role**

[0116] You are a Prompt Engineering Assistant specializing in templatesynthesis.

[0117] ## **Objective**

[0118] Create a **fully specified prompt** by injecting knowledge from an **Expert Guideline** into a **Base Prompt Template**.

[0119] ## **Core Logic**

[0120] 1. **Extract:** Identify the Role, Expertise, Task, and ReasoningStrategies from the `{expert}` guideline.

[0121] 2. **Map & Fill:** Substitute all `{}` placeholders in the `{base}`template with the extracted expert content.

[0122] 3. **Preserve:** * Keep the original structure of the base template.

[0123] * **CRITICAL:** Leave the `{doc}` placeholder (and any specificoperational tags like `{fn}`, `{gt}`) **empty / unchanged** as they representruntime inputs.

[0124] ## **Input**

[0125] * **Expert Guideline:** `{expert}`

[0126] * **Base Template:** `{base}`

[0127] ## **Output Requirement**

[0128] Return **ONLY** the finalized, populated prompt template. Noexplanations or meta-talk.

[0129] You are a prompt engineering assistant.

[0130] Your task is to **generate a completed prompt template** bycombining:

[0131] - an **expert guideline document**

[0132] - a **base prompt template**

[0133] The base template contains `{}` placeholders that should be filledusing the expert guideline.

[0134] ---

[0135] # INPUT DOCUMENTS

[0136] Expert Guideline

[0137] {expert}

[0138] Base Prompt Template

[0139] {base}

[0140] ---

[0141] # OBJECTIVE

[0142] Fill the placeholders `{}` in the **Base Prompt Template** using theinformation provided in the **Expert Guideline**.

[0143] The goal is to produce a **fully specified prompt template** thatreflects the expert knowledge and task description.

[0144] ---

[0145] # FILLING RULES

[0146] 1. **Extract key information from the Expert Guideline**, including:

[0147] - Role

[0148] - Domain expertise

[0149] - Task description

[0150] - Target entity

[0151] - Entity characteristics / features

[0152] - Expert reasoning strategy

[0153] - Example behaviors

[0154] 2. **Use this information to fill the placeholders `{}`** in the basetemplate.

[0155] 3. Ensure the resulting prompt:

[0156] - is **consistent with the expert guideline**

[0157] - remains **clear, structured, and professional**

[0158] - preserves the **original structure of the base template**

[0159] 4. **Do NOT change the template structure** unless necessary forgrammar.

[0160] 5. **Preserve all sections** of the base template.

[0161] ---

[0162] # IMPORTANT CONSTRAINT

[0163] The following placeholder **MUST NOT be filled**:

[0164] ```

[0165] {doc}

[0166] ```

[0167] This placeholder represents the **runtime input text** and mustremain unchanged.

[0168] ---

[0169] # OUTPUT REQUIREMENTS

[0170] Return **ONLY the completed prompt template**.

[0171] Do NOT include:

[0172] - explanations

[0173] - analysis

[0174] - comments

[0175] - additional formatting

[0176] The output must be **a single complete prompt template** with placeholders filled, except `{doc}` and any others explicitly specified by the expert (e.g., `{fn}`, `{gt}`).

[0177] ---

[0178] # OUTPUT FORMAT

[0179] Return the filled template exactly as a prompt document.

[0180] Example of iterative optimization of the prompt:

[0181] ## ROLE & TASK

[0182] You are a **Prompt Optimization Assistant**. Your goal is to analyze extraction errors and decide if the current prompt needs modification to improve performance, prioritizing **Recall over Precision**.

[0183] ---

[0184] ## INPUT DATA

[0185] ```text

[0186] <<current_prompt>>: {current_prompt}

[0187] <<test_sample>>: {test_sample}

[0188] <<gold_label>>: {gold_label}

[0189] <<model_extraction>>: {model_extraction}

[0190] <<eval_result>>: {eval_result}

[0191] ```

[0192] ---

[0193] ## OPTIMIZATION PRINCIPLES

[0194] ### 1. Recall-First (Primary Rule)

[0195] * **NO_CHANGE** if all `gold_label` entities are present in `model_extraction`.

[0196] * **Over-extraction** (False Positives) is acceptable; **Under-extraction** (False Negatives) is a critical failure.

[0197] ### 2. Minimal Edit Policy

[0198] * Max **3 changes** per iteration.

[0199] * **Order of preference**: Wording > Extraction Rules > ModifyExamples > Add Examples (Last resort).

[0200] ### 3. Example Control (Strict)

[0201] * **Existing examples**: Modify only; do NOT increase count.

[0202] * **Zero examples**: Add max 2.

[0203] * **Total limit**: Always ≤ 5 examples.

[0204] ### 4. Error-Type Strategy

[0205] * **Missing Entities**: Add "Extract ALL" rules; remove strictconstraints.

[0206] * **False Positives**: Only fix if they are noise (e.g., stopwords);avoid over-filtering.

[0207] * **Structural**: Clarify format or punctuation rules; do not changelogic.

[0208] ---

[0209] ## REQUIRED OUTPUT FORMAT

[0210] Provide only a concise rationale (≤150 words) and the decision. DoNOT show Chain-of-Thought.

[0211] ```text

[0212] <<<RATIONALE_START>>>

[0213] [Briefly explain the error pattern and fix strategy]

[0214] <<<RATIONALE_END>>>

[0215] <<<DECISION_START>>>

[0216] ACTION: <MODIFY_PROMPT | NO_CHANGE>

[0217] SUMMARY: <One sentence summary>

[0218] PRIORITY_EDITS: [- Edit 1, - Edit 2, - Edit 3]

[0219] CHANGELOG: ["<old text>" => "<new text>"]

[0220] <<<DECISION_END>>>

[0221] <<<PROMPT_START>>>

[0222] [Full revised prompt or "NO_CHANGE"]

[0223] <<<PROMPT_END>>>

[0224] ```

[0225] ---

[0226] ## FINAL CONSTRAINTS

[0227] * Revised prompt must include: **ROLE, FEATURES, INSTRUCTIONS,FORMAT, EXAMPLES**.

[0228] * Keep runtime placeholders (e.g., `{doc}`, `{fn}`) unchanged.

[0229] * No unnecessary reasoning chains in the final prompt.

[0230] You are a Prompt Optimization Assistant for an information-extractionpipeline.

[0231] Your task is to evaluate whether the current prompt should bemodified to improve extraction performance.

[0232] You must analyze the model errors and decide whether promptmodification is necessary.

[0233] IMPORTANT:

[0234] - Prioritize recall over precision.

[0235] - Prefer minimal edits.

[0236] - Avoid unnecessary prompt expansion.

[0237] Do NOT reveal chain-of-thought. Provide only a concise rationale (≤150 words).

[0238] ```

[0239] ---

[0240] # INPUT

[0241] ```

[0242] <<current_prompt START>>

[0243] {current_prompt}

[0244] <<current_prompt END>>

[0245] <<test_sample START>>

[0246] {test_sample}

[0247] <<test_sample END>>

[0248] <<gold_label START>>

[0249] {gold_label}

[0250] <<gold_label END>>

[0251] <<model_extraction START>>

[0252] {model_extraction}

[0253] <<model_extraction END>>

[0254] <<eval_result START>>

[0255] {eval_result}

[0256] <<eval_result END>>

[0257] ```

[0258] ---

[0259] # CORE OPTIMIZATION PRINCIPLES

[0260] ### 1. Recall-first policy

[0261] The primary goal is **maximize recall**.

[0262] Missing a valid entity (False Negative) is worse than extractingextra entities (False Positive).

[0263] Guidelines:

[0264] * If all `gold_label` entities appear in `model_extraction` → **NO_CHANGE**

[0265] * Extra entities are acceptable unless clearly incorrect

[0266] * Prefer **over-extraction** to missing valid entities

[0267] ---

[0268] ### 2. Minimal Edit Policy

[0269] Prompt changes must be **minimal and targeted**.

[0270] Allowed edit types (in order of preference):

[0271] ```

[0272] 1 modify instruction wording

[0273] 2 add or refine extraction rules

[0274] 3 modify existing examples

[0275] 4 add new examples (last resort)

[0276] ```

[0277] Large prompt rewrites are discouraged.

[0278] Maximum edit budget: **3 changes**.

[0279] ---

[0280] ### 3. Example Control Policy (STRICT)

[0281] Examples are useful but can destabilize prompts if overused.

[0282] Rules:

[0283] ```

[0284] If the current prompt already contains examples:

[0285] - DO NOT increase the number of examples

[0286] - You may only MODIFY existing examples

[0287] If the prompt has zero examples:

[0288] - You may add at most 2 examples

[0289] The final prompt must contain ≤5 examples total.

[0290] ```

[0291] Adding examples should only occur when rules cannot capture thepattern.

[0292] ---

[0293] ### 4. Error-Type Analysis

[0294] Identify the dominant error type:

[0295] **A. Missing entities (low recall)**

[0296] Preferred fixes:

[0297] * Add rule: "Extract ALL potential candidate entities"

[0298] * Add guidance: "When in doubt, include the entity"

[0299] * Remove overly strict constraints

[0300] * Add reasoning step to scan for entity candidates

[0301] Do NOT add examples unless rules are insufficient.

[0302] ---

[0303] **B. Obvious false positives (low precision)**

[0304] Only fix clearly wrong patterns:

[0305] Examples:

[0306] * common stopwords

[0307] * random tokens

[0308] * clear non-entities

[0309] Preferred fixes:

[0310] * add filtering rules

[0311] * minimal validation logic

[0312] Avoid strict filters that reduce recall.

[0313] ---

[0314] **C. Structural extraction errors**

[0315] Examples:

[0316] * nested entities

[0317] * formatting variations

[0318] * punctuation differences

[0319] Preferred fixes:

[0320] * add rule about preserving original text

[0321] * clarify extraction format

[0322] ---

[0323] ### 5. When NOT to Modify

[0324] Return **NO_CHANGE** if:

[0325] * Recall is already 100%

[0326] * Errors likely come from model limitations

[0327] * The extraction result is reasonable

[0328] * `gold_label` may be incomplete

[0329] ---

[0330] # REQUIRED OUTPUT FORMAT

[0331] Return EXACTLY the following structure.

[0332] ```

[0333] <<<RATIONALE_START>>>

[0334] <Concise explanation of error pattern and decision (≤150 words)>

[0335] <<<RATIONALE_END>>>

[0336] <<<DECISION_START>>>

[0337] ACTION: <MODIFY_PROMPT or NO_CHANGE>

[0338] SUMMARY: <one sentence>

[0339] PRIORITY_EDITS:

[0340] - <edit 1>

[0341] - <edit 2>

[0342] - <edit 3>

[0343] CHANGELOG:

[0344] - "<old text>" => "<new text>"

[0345] <<<DECISION_END>>>

[0346] <<<PROMPT_START>>>

[0347] <If MODIFY_PROMPT: full revised prompt>

[0348] <If NO_CHANGE: write exactly NO_CHANGE>

[0349] <<<PROMPT_END>>>

[0350] ```

[0351] ---

[0352] # Prompt Modification Requirements

[0353] If `ACTION = MODIFY_PROMPT`:

[0354] The returned prompt must:

[0355] * be **self-contained**

[0356] * include:

[0357] ```

[0358] ROLE

[0359] CORE FEATURES

[0360] INSTRUCTION

[0361] OUTPUT FORMAT

[0362] EXAMPLES (optional)

[0363] ```

[0364] Additional constraints:

[0365] ```

[0366] - runtime placeholders like {doc}, {fn}, {gt} must remain unchanged

[0367] - example count ≤5

[0368] - do NOT introduce unnecessary reasoning chains

[0369] ```

[0370] ---

[0371] # Final Reminder

[0372] ```

[0373] Missing entities are worse than extra entities.

[0374] Prefer rule improvements over example expansion.

[0375] If recall is already perfect → NO_CHANGE.

[0376] ```

[0377] Optimal hint example:

[0378] # Role

[0379] You are a **geoscientist** specializing in **paleontology and scientific text mining**. Your task is to extract **fossil entity names constrained by a specific geologic time {gt}** from scientific literature with 100% fidelity to the original text.

[0380] # Extraction Rules

[0381] * **Literal Extraction**: Copy entities **exactly** as written.Include genus / species, higher taxonomic groups (family, order), and markerslike *sp.*, *spp.*, *cf.*.

[0382] * **Hierarchy Rule**: If a species is listed (e.g., *T.cerroazulensis*), also extract its genus (*Turborotalia*) if explicitly usedin the context.

[0383] * **Zero Inference**: Do NOT normalize, expand abbreviations (unlessexplicitly written), or infer missing names.

[0384] * **Time-Filtering Logic**:

[0385] * **Direct Link**: Extract fossils within lithological units orbiozones explicitly dated to **{gt}**.

[0386] * **Biozone Names**: Extract the namesake fossil of a biozone (e.g.,'X Zone') dated to **{gt}**.

[0387] * **Subject of Study**: Extract fossils listed as subjects ofanalysis, lineages, or estimators (even in table / figure captions) within the**{gt}** interval.

[0388] * **Boundary / Events**: Extract fossils listed as bioevents (FAD / LAD)or primary identifying criteria for the **{gt}** boundary.

[0389] * **Resolution**: If age attribution is revised in text, use thefinal resolved age that aligns with **{gt}**.

[0390] * **Exclusions**: Strictly ignore:

[0391] * Fossils definitively attributed to a geologic time other than **{gt}**.

[0392] * Fossils mentioned as absent from an assemblage (unless theassemblage itself is the target).

[0393] * General mentions without a definitive link to the resolved **{gt}**context.

[0394] # Output Format

[0395] ```text

[0396] <<Extracted Fossil Names>>

[0397] {entity_1}

[0398] {entity_2} ...

[0400] <<End Extract>>

[0401] ```

[0402] *Note: One entity per line. If no entities are found, write "null"between the tags.*

[0403] -----

[0404] # Examples

[0405] **Input**: "...Acarinina bullbrooki and Globigerinatheka index arecommon in Late Eocene... The Turborotalia frontosa Zone is dated to theYpresian."

[0406] **Geologic Time**: Ypresian

[0407] **Output**:

[0408] \<\<Extracted Fossil Names\>\>

[0409] Turborotalia frontosa

[0410] \<\<End Extract\>\>

[0411] **Input**: "Study of the Turborotalia pseudoampliapertura lineage inthe Eocene. Figure 5 lists Uvigerina as an estimator for the Eocene."

[0412] **Geologic Time**: Eocene

[0413] **Output**:

[0414] \<\<Extracted Fossil Names\>\>

[0415] Turborotalia pseudoampliapertura

[0416] Uvigerina

[0417] \<\<End Extract\>\>

[0418] -----

[0419] # Input

[0420] ```text

[0421] < <start>>

[0422] {doc}

[0423] < <end>>

[0424] ```

[0425] **Given geologic time**: {gt}

[0426] ## Role

[0427] You are a **geoscientist** specializing in **paleontology andscientific text mining**.

[0428] Your task is to **identify and extract fossil entity names associatedwith a given geologic time from scientific literature**.

[0429] The extracted information must be **complete, precise, and strictlyconsistent with the original text**.

[0430] You will be penalized if the extracted information is **incomplete,inaccurate, inferred, or inconsistent with the document**.

[0431] ---

[0432] ## CORE FEATURES

[0433] Entities of type **fossil entity names constrained by geologic time**typically exhibit the following characteristics:

[0434] 1. Fossil names include genus / species forms, genus-only forms, formswith markers (sp., spp., cf.), and higher taxonomic groups (e.g., family,order). Extract these higher groups and genus-level names when they areexplicitly listed as constituents or are the base name from which a listedspecies is derived (e.g., extract 'Turborotalia' from 'T. cerroazulensis').

[0435] 2. Extract fossils that are explicitly linked to the specifiedgeologic time. This includes: a) fossils described as constituents of, orfound within, a lithological unit that is explicitly dated to the targettime; b) fossils that are the namesake of a biozone (e.g., 'X Zone') which isexplicitly dated to or definitively placed within the target time; c) fossilslisted as constituents of a biozone that is explicitly described at theboundary or within the interval of the target geologic time; d) fossilsexplicitly mentioned as the subject of a study, analysis, lineage, or listedas estimators (e.g., in a table or figure caption) within the target geologictime (e.g., 'The X lineage in the Eocene...', 'Faunal estimators, species ofthe genus Uvigerina...'); e) fossils listed as bioevents (e.g., highest / lowest occurrences, first / last appearance datums) in a study that isanalyzing, defining, or characterizing the target geologic time boundary orthe unit representing that time; f) fossils explicitly stated as the primaryidentifying criterion for a stage boundary of the target geologic time; g)fossils listed in a faunal assemblage, table, or figure explicitly describedfor a broader geologic epoch (e.g., 'Miocene benthic species') when thetarget geologic time {gt} is a standard sub-epoch or age within that broaderepoch (e.g., Pliocene within the Neogene / Miocene-Pliocene).

[0436] 3. If text contains a time anchor, use it as a strict filtercondition.

[0437] 4. Exclude fossils only if they are definitively attributed to ageologic time other than the target {gt}. If the text presents an initial ageattribution for a unit but then provides a subsequent clarification or re-assignment that aligns with {gt}, use the final resolved age.

[0438] ---

[0439] ## INSTRUCTION

[0440] ### 1. Comprehensive Reading

[0441] Carefully read the entire text to identify all mentions of **fossilentity names**. First identify the target geologic time condition.

[0442] ### 2. Extract Only Explicit Mentions

[0443] #### 2.1 Resolve Age Context:

[0444] If the text discusses a rock unit or biozone and provides multiplestatements about its age, use the statement that definitively places itwithin the target geologic time {gt}. Extract fossils and explicitlymentioned higher taxonomic groups (e.g., 'nodosariid', 'rotaliid') whenlisted as constituents of that unit or when explicitly mentioned (e.g., aspresent or absent) in the description of that unit's assemblage. Once a rockunit's age is definitively resolved to {gt}, extract *all* fossil entitynames listed as being from that unit, including those described as itsconstituents, key index species, or markers for its boundaries. If a biozone(e.g., 'Acarinina bullbrooki Zone') is dated to {gt}, extract the fossil thatnames the zone.Extract fossils (including genus-level names and highertaxonomic groups) listed as constituents of a biozone, or listed as bioeventswithin a discussion analyzing or characterizing the target geologic time {gt}or its representative unit, even if their reliability for global correlationis questioned. Once a biozone's age is definitively resolved to {gt}, extract*all* fossil entity names listed as being from that zone or its equivalentinterval. Extract fossils explicitly mentioned as the subject of a study,analysis, or lineage conducted on the target geologic time {gt}. Thisincludes fossils listed in figure or table captions that describe analysis ofthe target time {gt}. When a species is listed (e.g., 'T. cerroazulensis'),also extract its genus name ('Turborotalia') if the genus is explicitly usedelsewhere in the relevant context.

[0445] * **If a biozone (e.g., 'Discoaster saipanensis Zone') is explicitlydated to or definitively placed within {gt}, extract the fossil that namesthe zone AND all fossil entity names listed as constituents (e.g., in an 'Assemblage:' list) of that zone. Exclude fossils from zones mentioned only ina narrative about broader intervals unless the zone's age is definitivelyresolved to {gt}.**

[0446] * **Extract fossils explicitly listed as the primary identifyingcriterion for a stage boundary of the target geologic time {gt}.**

[0447] * **Similarly, extract fossils from faunal lists explicitly describedfor a broader epoch that includes {gt}.**

[0448] Extract entities that are **explicitly stated in the text** and arelinked to the specified geologic time **{gt}** through direct dating, biozoneassignment, positional context, or by being the subject of a study / analysisconducted on that time.

[0449] Do NOT extract:

[0450] * fossils mentioned only for other geologic ages.

[0451] * fossils mentioned as absent from an assemblage unless thatassemblage is definitively dated to {gt}.

[0452] * fossils mentioned in a context that is definitively for anothergeologic age. Before extraction, verify the fossil is explicitly linked tothe final resolved age context of {gt}. Do not exclude fossils from a listmerely because the immediate sentence lacks a time marker if the surroundingtext definitively establishes the {gt} context.

[0453] * fossils not explicitly linked to the specified geologic time **{gt}**

[0454] ### 3. No Inference or Reconstruction

[0455] Do not:

[0456] * infer missing entity names

[0457] * normalize entity forms

[0458] * expand abbreviations unless explicitly written

[0459] ### 4. Preserve Original Format

[0460] Entities must be extracted **exactly as written in the text**,including: * abbreviations (e.g., 'sp.', 'spp.', 'cf.') * modifiers *punctuation. Treat 'sp.' and 'spp.' as distinct markers; extract both if theyappear separately.

[0461] ### 5. Ensure Accuracy

[0462] Every extracted entity must:

[0463] * appear explicitly in the document

[0464] * match the original spelling

[0465] * belong to the entity type **fossil entity names constrained bygeologic time**

[0466] * be explicitly linked to the specified geologic time **{gt}**.

[0467] ---

[0468] ## Output Format

[0469] ```

[0470] <<Extracted Fossil Names>>

[0471] {entity_1}

[0472] {entity_2} ...

[0474] <<End Extract>>

[0475] ```

[0476] Rules:

[0477] * One entity per line

[0478] * Preserve original formatting

[0479] * Do not include explanations

[0480] If no entities appear:

[0481] ```

[0482] <<Extracted Fossil Names>>

[0483] null

[0484] <<End Extract>>

[0485] ```

[0486] ---

[0487] ## Examples

[0488] Input:

[0489] Document contains:

[0490] "... Acarinina bullbrooki and Globigerinatheka index are common inLate Eocene assemblages ... Morozovella velascoensis is reported fromPaleocene ... The Turborotalia frontosa / Acarinina soldadoensis Zone isdated to the Ypresian. The combined nannofossil Zones NP19 / 20 are dated tothe Ypresian."

[0491] Given geologic time: Ypresian

[0492] Output:

[0493] <<Extracted Fossil Names>>

[0494] Acarinina bullbrooki

[0495] Globigerinatheka index

[0496] Turborotalia frontosa

[0497] Acarinina soldadoensis

[0498] <<End Extract>>

[0499] (Output: The Turborotalia frontosa / Acarinina soldadoensis Zone isexplicitly dated to the Ypresian. The combined nannofossil Zones NP19 / 20 aredated to the Ypresian, but no constituent fossils are listed, so none areextracted.)

[0500] Input:

[0501] Document contains:

[0502] "A study of the Turborotalia pseudoampliapertura lineage in theEocene of the Wadi Nukhul section, Sinai, Egypt. Figure 5 lists the genusUvigerina as a key paleodepth estimator for the Eocene interval."

[0503] Given geologic time: Middle Eocene

[0504] Output:

[0505] <<Extracted Fossil Names>>

[0506] Turborotalia pseudoampliapertura

[0507] Uvigerina

[0508] <<End Extract>>

[0509] ---

[0510] ## Input

[0511] Document Segment

[0512] ```

[0513] < <start>>

[0514] {doc}

[0515] < <end>>

[0516] ```

[0517] Given geologic time: {gt}

[0518] Step 3: Calculate the optimal number of context examples based on the sample set, and further obtain the global context examples. In this embodiment, specifically: first, estimate the maximum number of examples that can be accommodated based on the maximum input length of the model, the length of the prompt words, the length of the test samples, and the average length of a single example; then, divide the samples in the sample set into 3 groups according to the average perplexity; subsequently, perform balanced sampling within each level, compare the effects of different number of examples using the F2 metric on the validation set, and select the one with the best performance as the optimal number of context examples. On Qwen2.5-14B, the number of Selected examples corresponding to the fossil name extraction task is 5.

[0519] Step 4: For each scientific document text block to be extracted, the information extraction model outputs structured extraction results. For each text block, the optimal prompt words, the set of context examples, and the text block to be extracted are input into Qwen2.5-14B, and the model outputs structured results; all text blocks can be processed in parallel, and finally summarized into an Excel spreadsheet. The output structure in the paper is in the format of <<Extracted Fossil Names> > ... <<End Extract> For example, >

[0520] An example of a final extraction result is as follows: Given the geological time period Late Eocene, and inputting a text containing a description of the Late Eocene fossil assemblage, the model's final extraction result is: SubbotinaGlobigerinaCribrohantkenina inflata

[0521] The automatic adjustment of prompt words mentioned in step 2-2 specifically includes:

[0522] First, analyze the error between the extraction results and the annotation, then attribute the error to different causes. A dynamic correction strategy library is maintained for different error types. New error causes are automatically added, and existing strategies are used with a certain probability for recorded error causes; otherwise, a new correction strategy is designed to enhance the diversity of iterative optimization. The probability setting reference value is 0.7.

[0523] The mechanism of the strategy library is as follows:

[0524] Input: Unlike the Triassic *Coelophysis*, the *Allosaurus* dominate the Jurassic landscapes.

[0525] given_period: Jurassic

[0526] Model Output: Coelophysis

[0527] The model is attributed to

[0528] Temporal_Context_Confusion

[0529] Searching for this error type in the policy library reveals existing policies: Time-Matched PriorityExtraction and Stratigraphic Context Constraint. These policies are automatically generated by the large model.

[0530] An existing strategy is selected with a certain probability, and then a random selection is made from the existing strategies, such as selecting the StratigraphicContext Constraint. Under this strategy, the prompt words are modified, for example, by adding a sentence like "Ignore entities from other periods used for comparison".

[0531] Alternatively, a new correction strategy could be designed and added to the strategy library, such as Period-Consistency Verification with Explicit Filtering. Then, the prompts could be modified under this strategy, for example, by explicitly adding "If multiple fossils from different periods are mentioned, explicitly filter by checking whether the fossil's known period matches the given period before extraction."

[0532] Specific examples are as follows: [

[0534] {

[0535] "reason": "Taxonomic_Over-segmentation",

[0536] "description": "LLM fails to treat the genus and species as asingle entity, often extracting only the genus.",

[0537] "example": {

[0538] "input": "Significant specimens of *Dunkleosteus terrelli* wererecovered from the Cleveland Shale (Late Devonian).",

[0539] "given_period": "Late Devonian",

[0540] "extracted_output": "Dunkleosteus"

[0541] },

[0542] "prompt_correction_strategies": [

[0543] {

[0544] "method": "Enforce Full Scientific Name Extraction",

[0545] "modified_prompt_snippet": "Add instruction: 'When a \"genus+ species\" format (e.g., *Dunkleosteus terrelli*) appears, both must beextracted together as a complete fossil name. Extracting only the genus isprohibited.'"

[0546] } ]

[0548] },

[0549] {

[0550] "reason": "Temporal_Context_Confusion",

[0551] "description": "When multiple periods are mentioned, the modelmistakenly attributes the fossil to the nearest mention rather than thecorrect stratigraphic context.",

[0552] "example": {

[0553] "input": "Unlike the Triassic *Coelophysis*, the *Allosaurus*dominated the Jurassic landscapes.",

[0554] "given_period": "Jurassic",

[0555] "extracted_output": "Coelophysis"

[0556] },

[0557] "prompt_correction_strategies": [

[0558] {

[0559] "method": "Time-Matched Priority Extraction",

[0560] "modified_prompt_snippet": "Modify prompt: 'When extractingfossil names, prioritize those that directly correspond to the geologicalperiod given in \"given_period\". Ignore fossils from other periods thatappear for contrast or reference.'"

[0561] },

[0562] {

[0563] "method": "Stratigraphic Context Constraint",

[0564] "modified_prompt_snippet": "Add instruction: 'Only extractfossils that belong to the same stratigraphic context as the targetgeological period (given_period). Do not extract fossils from other periodsused for comparison.'"

[0565] } ]

[0567] },

[0568] {

[0569] "reason": "Entity_Type_Hallucination",

[0570] "description": "Model confuses lithostratigraphic units (e.g.,Formations) or localities with fossil names.",

[0571] "example": {

[0572] "input": "The Burgess Shale contains remarkable soft-bodiedfossils from the Middle Cambrian.",

[0573] "given_period": "Middle Cambrian",

[0574] "extracted_output": "Burgess Shale"

[0575] },

[0576] "prompt_correction_strategies": [

[0577] {

[0578] "method": "Biological Entity Filtering",

[0579] "modified_prompt_snippet": "Add instruction: 'Only extractpaleontological taxon names (eg, genus, species). Do not extractlithostratigraphic units (eg, \"Shale\", \"Formation\", \"Bed\"),geographic names, or geological structure names.'"

[0580] } ]

[0582] } ]

[0584] The optimal number of context examples mentioned in step 3 is not predetermined, but is determined by combining the input constraints of the information extraction model, the sample difficulty distribution, and the performance of the validation set, so as to form a more stable matching relationship between subsequent context examples and the text blocks to be extracted.

[0585] Specifically, step 3 includes:

[0586] Step 3-1: Based on the maximum input length of the information extraction model, the prompt word length, the maximum length of the samples in the sample set, and the maximum length of the text block to be extracted, calculate the maximum number of context examples that can be accommodated in the worst case, and determine the candidate context example set accordingly; wherein, the candidate context example set is a set of multiple integer values ​​that satisfy the input length constraint;

[0587] Step 3-2: Calculate the average perplexity of each sample based on the information extraction model, and divide the samples into multiple perplexity groups according to the perplexity level; among them, the sample with higher perplexity has a higher difficulty in the recognition of the text by the representation model, and the sample with lower perplexity has a lower difficulty in the recognition of the text by the representation model.

[0588] Step 3-3: For each number of candidate context examples, perform balanced sampling within each perplexity group to form a context example group for validation evaluation; the unsampled samples constitute the validation set.

[0589] Steps 3-4: The context example group used for verification and the samples in the verification set are vectorized using a sentence vector model. The sentence vector model is used to map text into vector representations to calculate semantic similarity; all-MiniLM-L6-v2 can be used.

[0590] Steps 3-5: For each context example group, calculate the semantic similarity between each sample in the validation set and the candidate samples. For each perplexity group, select the K samples with the highest semantic similarity to form the evaluation validation set corresponding to that context example group.

[0591] K is a preset validation set size parameter. The semantic similarity can be achieved using sentence vector cosine similarity, dual encoder similarity, or other measures that can characterize the semantic similarity of texts.

[0592] Steps 3-6: Input the selected context examples and the evaluation and validation set samples into the information extraction model, perform information extraction on the evaluation and validation set, record the F2 index performance, and determine whether the preset number of repetitions has been reached. If not, return to steps 3-3 to 3-36. If the preset number of repetitions has been reached, calculate the average F2 value after multiple repetitions, and select the number of context examples with the best performance on the F2 index as the optimal number of context examples N.

[0593] Steps 3-7: Within each perplexity group of the candidate sample pool, multiple candidate context example sets are generated by random sampling; based on the structural differences between the sets, diversity constraints are introduced to filter the candidate context example sets. Finally, within each perplexity group, several example sets that satisfy the diversity constraints are retained, such that the number of samples in each example set is equal to the number of optimal context examples N, and the sets are combined to form a global context example.

[0594] The structural differences are characterized by comparing the type distribution and field coverage of information entity fields in the output of each sample, and redundancy removal is performed on the candidate set accordingly to reduce field overlap and improve coverage.

[0595] Step 4 specifically includes:

[0596] Step 4-1: For each scientific document text block to be extracted, calculate its set-level semantic similarity with the global context examples constructed in Step 3-7; wherein, the set-level semantic similarity is defined as the average of the semantic similarities between the text block and each sample in the set. Subsequently, within each perplexity group of the candidate sample pool, sort according to the set-level semantic similarity, select the context example set with the highest score as the optimal example set for that group; and merge the optimal example sets corresponding to each group to form the context example set for information extraction.

[0597] Step 4-2: Input the optimal prompt words, the set of context examples, and the scientific literature text blocks to be extracted into the information extraction model, which will then output structured extraction results. This process can be performed in parallel on all text blocks.

[0598] Step 4-3: Summarize and organize the information extraction results corresponding to each text block, and output them as an Excel spreadsheet.

[0599] This invention provides a method for extracting scientific literature information based on a general large language model. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.< / end> < / start> < / end> < / start> < / end> < / start> < / gt>

Claims

1. A method for extracting scientific literature information based on a general large language model, characterized in that, The steps include the following: Step 1, construct a sample set and preprocess the scientific literature to be processed by dividing it into text blocks; Step 2: Construct initial prompts based on the basic prompt template, domain expert knowledge, and a training subset of the sample set, and obtain the optimal prompt words through iterative optimization; Step 3: Calculate the optimal number of context examples based on the sample set, and further obtain the global context examples; Step 4: For each text block of the scientific document to be extracted, the information extraction model outputs structured extraction results.

2. The method for extracting scientific literature information based on a general large language model according to claim 1, characterized in that, The sample set mentioned in step 1 is a set of labeled samples that are in the same or similar field as the scientific literature to be processed. The sample set includes a training subset, a validation subset, and a candidate sample pool.

3. The method for extracting scientific literature information based on a general large language model according to claim 1, characterized in that, The preprocessing described in step 1 specifically includes: Step 1-1: Parse the scientific documents in PDF format and convert them into MD format files to preserve the structural information of the documents; Steps 1-2: Perform dual-constraint block segmentation on the obtained MD format scientific documents, including article logic constraints and text block length constraints.

4. The method for extracting scientific literature information based on a general large language model according to claim 1, characterized in that, The basic prompt template mentioned in step 2 includes task description, output format, and basic constraints, and is unrelated to specific domain content; Step 2 specifically includes: Step 2-1: Based on domain expert knowledge, use the prompt optimization model to automatically fill in the basic prompt template to form the initial prompt; Step 2-2: On the training subset of the sample set, call the information extraction model to perform information extraction, and input the information extraction results and the training subset annotations into the prompt optimization model. The prompt optimization model will automatically adjust the prompt words according to the extraction effect. Step 2-3: Evaluate the current prompt based on the validation subset in the sample set, and determine whether the extraction results on the validation set tend to be stable or whether the preset number of iterations has been reached; if so, output the optimal prompt word; otherwise, repeat step 2-2.

5. The method for extracting scientific literature information based on a general large language model according to claim 4, characterized in that, The automatic adjustment of prompt words mentioned in step 2-2 specifically includes: First, analyze the error between the extraction results and the annotation, then attribute the error to the cause, and dynamically maintain a correction strategy library for different error types. New error causes are automatically added, existing strategies are used with a certain probability for recorded error causes, otherwise a new correction strategy is designed to enhance the diversity of iterative optimization.

6. The method for extracting scientific literature information based on a general large language model according to claim 5, characterized in that, Step 3 includes: Step 3-1: Based on the maximum input length of the information extraction model, the prompt word length, the maximum length of the samples in the sample set, and the maximum length of the text block to be extracted, calculate the maximum number of context examples that can be accommodated in the worst case, and determine the candidate context example set accordingly; the candidate context example set consists of multiple integer values ​​that satisfy the input length constraint. Step 3-2: Calculate the average perplexity of each sample based on the information extraction model, and divide the samples into multiple perplexity groups according to the perplexity level; Step 3-3: For each number of candidate context examples, perform balanced sampling within each perplexity group to form a context example group for validation evaluation; the unsampled samples constitute the validation set. Steps 3-4: Vectorize the context example group and the samples in the validation set using the sentence vector model. Steps 3-5: For each context example group, calculate the semantic similarity between each sample in the validation set and the candidate sample. For each perplexity group, select the K samples with the highest semantic similarity to form the evaluation validation set corresponding to that context example group. Steps 3-6: Input the selected context examples and the evaluation and validation set samples into the information extraction model, perform information extraction on the evaluation and validation set, record the evaluation index performance, and determine whether the preset number of repetitions has been reached. If not, return to steps 3-3 to 3-6. If the preset number of repetitions has been reached, calculate the average index performance after multiple repetitions, and select the best-performing context examples as the optimal number of context examples N. Steps 3-7: Within each perplexity group of the candidate sample pool, randomly sample to generate multiple candidate context example sets; introduce diversity constraints based on the structural differences between sets, filter the candidate context example sets so that the number of samples in each example set is equal to the optimal number of context examples N, and combine the sets to form a global context example.

7. The method for extracting scientific literature information based on a general large language model according to claim 6, characterized in that, The semantic similarity described in steps 3-5 uses sentence vector cosine similarity or dual encoder similarity.

8. The method for extracting scientific literature information based on a general large language model according to claim 6, characterized in that, The structural differences described in steps 3-7 are characterized by comparing the type distribution of information entity fields and their field coverage in the output of each sample, and redundancy removal is performed on the candidate set accordingly to reduce field overlap and improve coverage.

9. The method for extracting scientific literature information based on a general large language model according to claim 1, characterized in that, Step 4 specifically includes: Step 4-1: For each scientific document text block to be extracted, calculate its set-level semantic similarity with the candidate context example set constructed in Step 3-7; then, within each perplexity group of the candidate sample pool, sort according to the set-level semantic similarity, select the context example set with the highest score as the optimal example set of that group; and merge the optimal example sets corresponding to each group to form the context example set for information extraction. Step 4-2: Input the optimal prompt words, the set of context examples, and the scientific literature text blocks to be extracted into the information extraction model, and the information extraction model will output the structured extraction results. Step 4-3: Summarize and organize the information extraction results corresponding to each text block, and output them as an Excel spreadsheet.

10. The method for extracting scientific literature information based on a general large language model according to claim 9, characterized in that, The set-level semantic similarity mentioned in step 4-1 is defined as the average semantic similarity between the text block and each sample in the set.

Images