Bio-enlightenment relation triple combined extraction method based on fine-tuning large language model

Abstract

The invention provides a biological enlightenment relation triple combined extraction method based on a fine-tuning large language model, and solves the problems of error propagation, difficulty in entity definition and insufficiency in chain dependence modeling in the prior art. The method comprises the following steps: constructing a full-chain bionic field corpus through reverse tracing, generating an initial triple label by using an advanced teacher model, and performing manual verification; and based on the selected base model, reconstructing the extraction task into a text generation task by adopting LoRA fine tuning to realize end-to-end joint extraction. According to the method, a composite entity boundary is accurately analyzed through field fine tuning, and error propagation is avoided; explicitly modeling chained dependence of'biological entity-bionic model-bionic application '; and few-sample field self-adaption is realized. Through accuracy verification, the extraction accuracy and logic consistency of the complex bionic text are remarkably improved, and technical support is provided for constructing a high-quality biological enlightenment knowledge base.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology and proposes a method for joint extraction of biologically inspired relation triples based on fine-tuning a large language model. Background Technology

[0002] In the field of information extraction, automatically extracting structured bio-inspired relation triples from massive amounts of unstructured text—that is, logical chains consisting of "biological entities," "bionic models," and "bionic applications"—is crucial for advancing bionics research and constructing related knowledge graphs. Current technical solutions mainly rely on stepwise or joint extraction methods. Stepwise methods employ a pipeline approach of first identifying named entities and then extracting entity relations. However, their inherent drawback is that errors in the entity identification stage propagate to downstream relation classification tasks, creating cascading errors, and the two-stage models struggle to achieve effective feature sharing and joint optimization. To address this issue, joint extraction methods design a unified model architecture to simultaneously complete entity identification and relation extraction. While this alleviates error propagation to some extent, it still faces significant challenges when processing complex bio-inspired text. These challenges are mainly reflected in the following aspects: the terminology in the fields of biology and engineering is complex and highly specialized, making it difficult for generalized models to accurately define the boundaries of complex noun phrases, especially when dealing with long-tail, low-frequency specialized terms, where generalization ability is severely insufficient; the implicit chain dependency relationship of "biological entity → biomimetic model → biomimetic application" in bio-inspired triples is the core to ensure the logical correctness of triples, but existing joint models are unable to effectively capture this logical dependency that spans long distances and complex syntactic structures; the model's adaptability to the terminology system, expression habits, and internal logic unique to the field of bionics is limited, restricting its application effect in real-world scenarios. In summary, existing technologies, due to problems such as error propagation, difficulty in defining complex entities, insufficient modeling of key chain dependencies, and poor domain generalization, severely restrict the efficiency and quality of constructing automated, large-scale bio-inspired knowledge bases. Summary of the Invention

[0003] To address the aforementioned technical issues, this invention provides a joint extraction method for bio-inspired relation triples based on a fine-tuned large language model. Within an end-to-end unified framework, this method fine-tunes a pre-trained large language model, making it the core processing unit. Leveraging the general knowledge learned during pre-training and the domain knowledge injected during fine-tuning, the model can perform deep semantic understanding of input text, accurately parse the complex semantics of compound noun phrases, and precisely define ambiguous entity boundaries, thus effectively avoiding the error propagation problem of step-by-step methods. To achieve efficient domain adaptation, this invention performs supervised fine-tuning of the large language model on a specific labeled dataset in the bio-inspired domain, reconstructing the triple extraction task into a conditional text generation task following a specific instruction format. This fine-tuning process enables the model to deeply learn and internalize the domain's specialized terminology, syntactic patterns, and inherent logic, effectively overcoming the generalization bottleneck caused by the long-tail distribution of domain terminology and achieving efficient knowledge transfer in low-sample scenarios. Compared with existing technologies, this invention significantly improves the accuracy and logic of extraction, enhances the domain generalization ability of the model, and provides an automated and efficient technical support for building a large-scale, high-quality biological revelation knowledge base, which has significant technical advantages and application value. Attached Figure Description

[0004] Figure 1 This is an overall flowchart of a bio-inspired relation triple joint extraction method based on fine-tuning a large language model.

[0005] Figure 2 This is a schematic diagram of the LoRA fine-tuning mechanism used in this invention.

[0006] Figure 3 This is a flowchart of the processing procedure for the corresponding model. Detailed Implementation

[0007] This invention provides a method for joint extraction of biologically inspired relation triples based on fine-tuning of a large language model. The specific implementation process may include the following steps:

[0008] Step 1: Building the Domain Corpus

[0009] To provide high-quality, comprehensive raw data for training large language models, we constructed a foundational corpus derived from authoritative academic journals in bionics. Academic journals were chosen as the data source because they are peer-reviewed, use standardized terminology, and have clear narrative structures—qualities crucial for training models to accurately identify complex bionic relationships.

[0010] To mitigate potential sample bias in existing literature data (such as over-concentration on successfully commercialized biomimetic applications while neglecting fundamental biological research), this embodiment employs a reverse-tracking data augmentation strategy: first, representative core literature is selected from the basic corpus, and then its references are analyzed to proactively identify and include papers that reveal potentially inspiring but not yet fully explored fundamental biological mechanisms. This method significantly enhances the diversity and coverage of the dataset at the level of original innovation, ensuring that the corpus comprehensively covers the entire knowledge chain from the exploration of fundamental biological mechanisms to the final biomimetic engineering applications. This provides more comprehensive learning materials for subsequent models and enhances the robustness of the evaluation results.

[0011] Step Two: Data Processing and Cleaning

[0012] Given that most of the collected documents are in non-editable PDF format and their text length often exceeds the context window limit of large language models, we utilize PDF extraction tools to extract the text content from the documents with high accuracy and convert it into a well-structured and easily processed Markdown format. However, the extraction results often contain a large amount of invalid information (such as author contribution statements, reference lists, etc.). This content usually does not contain bio-inspired relation triples, has limited retention value, and may interfere with model understanding and training. Therefore, we combine domain knowledge and document structural characteristics, and use Python regular expressions to clean and filter the aforementioned redundant information.

[0013] Step 3: Fine-tuning the dataset construction

[0014] For the cleaned literature text, this embodiment employs a semi-automated data annotation method that combines machine extraction efficiency with human verification accuracy. The specific implementation is as follows: First, a large-scale, high-performance language model is used as the "teacher model." Carefully designed prompts are input into the model to guide it in directly parsing and extracting structured "biological entity, biomimetic model, biomimetic application" relational triples from the sentences. Then, all triples automatically extracted by the teacher model are submitted for manual review and correction. Based on professional standards such as biological structural characteristics, functional mechanisms, and environmental adaptability, the machine-generated results are evaluated, corrected, and confirmed, ultimately forming a high-quality, highly consistent triplet annotation dataset to ensure the accuracy of the fine-tuned data. The constructed dataset is organized according to the mainstream instruction-based fine-tuning data format (Alpaca format): each data sample includes an instruction field (prompts guiding the teacher model in triplet extraction), an input field (literature text), and an output field (biological inspiration relational triples extracted by the teacher model and manually verified).

[0015] Step 4: Model Fine-tuning and Evaluation

[0016] This embodiment defines the task of extracting bio-inspired relation triples as a goal-oriented text generation task. To achieve this goal, a supervised fine-tuning strategy is used to optimize the base large language model to enhance its ability to extract structured information in the field of bionics. In selecting the base model, considering factors such as the task's ability to process long texts, its understanding of complex biological concepts, and hardware resource limitations, this embodiment selects the pre-trained model Qwen-2.5-14B, which has strong language understanding capabilities and a long context window, as the fine-tuning base. For fine-tuning techniques, to balance training efficiency and model performance, the LoRA (Low-Rank Adaptation) method, a parameter-efficient fine-tuning technique, is adopted. This method freezes most of the base model's parameters and trains only a small number of newly added adapter parameters, thereby significantly reducing computational resource consumption and training time, while simultaneously achieving effective model adaptation to specific tasks. To verify the effectiveness of the method, performance evaluation is performed after model fine-tuning. The evaluation metric is precision, calculated using the following formula:

[0017]

[0018] TP indicates that the extracted field value exactly matches the standard answer annotation, while FP indicates that the extracted field value does not appear in the standard answer. This metric rigorously compares the triples extracted by the model on the test set with the manually annotated standard answers, comprehensively measuring the model's extraction performance from two aspects: entity recognition accuracy and relationship judgment correctness, and scientifically evaluating the effectiveness of this method.

[0019] Table 1

[0020]

[0021] Table 1 compares the performance of different models in the task of extracting bio-inspired relation triples. The accuracy of the base model Qwen-2.5-14B in extracting biological entities, biomimetic models, and biomimetic applications is only 10%, 6%, and 3%, respectively; the large parameter model Qwen-2.5-72B-Instruct improves to 63%, 44%, and 29%, respectively; the Qwen-2.5-14B-Finetuned model, fine-tuned by the method of this invention, performs best, with the three indicators reaching 73%, 63%, and 63%, respectively, significantly outperforming the base model with the same parameters and the larger-scale 72B model, confirming the effectiveness of the fine-tuning strategy in improving the extraction of complex relations.

Claims

1. A method for joint extraction of biologically inspired relation triples based on fine-tuning a large language model, characterized in that, Includes the following steps: Step 1: Construct a corpus of bio-inspiration. Through a reverse data expansion strategy, actively identify and include research papers that reveal potential biological mechanisms from basic literature to form a high-quality corpus covering the entire chain of "basic biological mechanisms → biomimetic engineering applications". Step 2: Process and clean the corpus, extract the document text using PDF extraction tools and convert it into a structured format, and filter out invalid information based on regular expressions; Step 3: Construct a fine-tuning dataset using a semi-automated annotation method: · Using a high-performance large language model as the teacher model, the "biological entity-bionic model-bionic application" triplet is extracted through instruction prompts; · The machine-extracted results are manually corrected to form a labeled dataset, and the format is finely adjusted and organized according to instructions. Step 4: Define the triple extraction task as an instruction-driven text generation task, select a pre-trained large language model with long context processing capabilities as the base, and use LoRA parameter efficient fine-tuning technology for supervised fine-tuning to improve the model's ability to capture complex logical relationships in the field of bio-inspiration.

2. The method according to claim 1, characterized in that, The reverse tracing data augmentation strategy described in step one specifically includes: · Representative core documents were selected from the basic corpus, and their references were traced. · To identify and include research papers on underdeveloped basic biological mechanisms, thereby enhancing the diversity and coverage of data at the level of original innovation.

3. The method according to claim 1, characterized in that, The data processing and cleaning described in step two includes: · Use a PDF extraction tool to extract the document text and convert it to Markdown format; · Based on the structural characteristics of the literature and regular expressions, redundant content such as author declarations and reference lists is removed.

4. The method according to claim 1, characterized in that, The instruction fine-tuning data format described in step three is Alpaca format, which includes: · Instruction field: Prompt words for the teacher model to extract triples; · Input field: The cleaned document text; · Output field: Manually corrected normalized triples.

5. The method according to claim 1, characterized in that, The fine-tuning technique described in step four uses LoRA (Low-Rank Adaptation), which achieves efficient domain adaptation by freezing most of the parameters of the base model and training only the newly added adapter parameters.

6. The method according to claim 1, characterized in that, In step four, the Qwen series of pre-trained large language models are selected as the base, and their context window length supports the handling of long text dependencies in the field of bio-inspiration.

7. The method according to claim 1, characterized in that, Evaluate model performance using accuracy metrics: Where TP represents the number of fields extracted by the model that exactly match the manually labeled fields, and FP represents the number of fields that were incorrectly generated by the model.

8. The method according to claim 1, characterized in that, The fine-tuned model achieved accuracy rates of 73%, 63%, and 63% in biological entity, biomimetic model, and biomimetic application extraction tasks, respectively, significantly outperforming the same parameter base model and larger-scale models.

Images