A large language model information extraction method based on multi-graph instruction fine-tuning
Patent Information
- Application Number
- CN202611195189.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-07
- Publication Date
- 2026-10-09
AI Technical Summary
但该类方法存在明显缺陷：其一，此类模型高度依赖任务专用的网络结构与解码流程，不同子任务需要设计独立的候选生成、特征交互与预测模块，技术实现复杂度高，且难以直接适配大语言模型的自回归生成范式；其二，现有方案多采用单一语义图进行建模，当文本中存在重叠实体、嵌套实体、同一实体对间多种关系、同一论元承担多种角色等复杂语义结构时，单一图空间容易造成语义类型混淆，普通单边图结构也无法完整表达多重语义连接，导致抽取准确率下降
1、通过将异构信息抽取标注统一转换为类型化有向多图，并进一步文本化为规范化序列，使图结构可以直接作为大语言模型的生成目标，无需为不同子任务设计专用的分类器、解码器或搜索模块。该方案天然适配大语言模型的自回归生成能力，在统一框架下同时支持命名实体识别、关系抽取、事件检测、事件论元抽取与联合信息抽取任务，大幅降低了系统实现复杂度与多任务适配成本。
Smart Images
Figure CN122886618A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence natural language processing technology, and in particular to a method for extracting information from a large language model based on multi-graph instruction fine-tuning. Background Technology
[0002] Information extraction (IE) is one of the core tasks in natural language processing. Its goal is to automatically identify and extract structured semantic information such as named entities, entity relationships, event triggers, and event arguments from unstructured text. It has wide applications in knowledge graph construction, intelligent question answering, enhanced retrieval generation, financial risk control, and document processing. With the development of large language models and instruction fine-tuning techniques, information extraction is gradually evolving from traditional task-specific classification models to a unified text-to-structure generation paradigm.
[0003] In the prior art, the solutions closest to this invention mainly fall into two categories: The first category is graph-based joint information extraction methods, typically represented by models such as DyGIE++ and OneIE. These methods represent text fragments such as entities and event trigger words as graph nodes, and semantic associations such as entity relationships and event argument roles as graph edges. They achieve joint prediction of multiple sub-tasks through graph modeling, structured decoding, or global constraints. However, this type of method has significant drawbacks: First, these models heavily rely on task-specific network structures and decoding processes. Different sub-tasks require the design of independent candidate generation, feature interaction, and prediction modules, resulting in high technical complexity and difficulty in directly adapting to the autoregressive generation paradigm of large language models. Second, existing solutions often use a single semantic graph for modeling. When the text contains complex semantic structures such as overlapping entities, nested entities, multiple relationships between the same entity pair, or the same argument playing multiple roles, a single graph space can easily cause semantic type confusion. Ordinary single-edge graph structures cannot fully express multiple semantic connections, leading to a decrease in extraction accuracy.
[0004] The second category comprises generative information extraction methods based on large language models, typically represented by UIE, InstructUIE, and code-based output methods such as GoLLIE and KnowCoder. These methods transform information extraction into a text generation task, guiding the large language model to directly output structured results through natural language instructions or code-based schemas. However, this type of method also has shortcomings: First, ordinary natural language instructions have weak constraints on the output structure. The generation process of large language models mainly relies on their own probability distribution and lacks explicit structural inductive bias. In complex sentences containing multiple entities, relations, and events, problems such as entity omission, relation mismatch, missing arguments, or over-generation are prone to occur, resulting in insufficient structural fidelity. Second, although the code-based output scheme improves the controllability of the format, it has extremely high requirements for the correctness of the output syntax. Once there are minor problems such as missing brackets, indentation errors, or inconsistent field names, subsequent parsing will fail, requiring additional constraint decoding, format repair, or external parsers, which increases the engineering complexity. Third, different information extraction datasets have differences in annotation specifications such as entity boundary definitions, entity type systems, relation labels, and event argument roles. Existing unified generation models lack explicit distinction of annotation specifications, which can easily lead to annotation standard confusion when training multiple datasets together, limiting the generalization performance of large language models in cross-task, cross-domain, and zero-shot scenarios. Summary of the Invention
[0005] The purpose of this invention is to provide a method for extracting information from a large language model based on multi-graph instruction fine-tuning, thereby solving the above-mentioned technical problems.
[0006] To achieve the above objectives, this invention provides a method for extracting information from a large language model based on multi-graph instruction fine-tuning, comprising the following steps: S1. Multi-graph construction: Information extraction and labeling in the input sentence are uniformly converted into a typed directed multi-graph structure, where the typed directed multi-graph is composed of semantic subgraphs corresponding to multiple semantic types; S2, Multi-graph textification: Transforming typed directed multi-graphs into normalized text sequences according to deterministic rules, serving as the generation target for large language models; S3. Multi-graph instruction fine-tuning: Construct instruction samples containing task definitions and input sentences, and use the corresponding normalized multi-graph text sequences as output targets to perform supervised fine-tuning of the large language model; S4. Reasoning and parsing: Input the text to be extracted and the task instructions into the fine-tuned large language model to generate a multi-image text sequence, and obtain the structured information extraction result by parsing according to the preset syntax.
[0007] Preferably, the semantic subgraphs of a typed directed multigraph include entity semantic subgraphs, relation semantic subgraphs, and event semantic subgraphs. Figure 3The classes correspond to the semantic structures of named entity recognition, relation extraction, and event extraction tasks, respectively.
[0008] Preferably, the nodes of the entity semantic subgraph are entity mentions bound to the text span of the input sentence, each node corresponds to an entity type, and the edge set of the entity semantic subgraph is empty.
[0009] Preferably, the nodes of the relational semantic subgraph are entity references that participate in the semantic relationship, the edges are directed edges from the head entity to the tail entity, each edge corresponds to a relation type, and the edge set of the relational semantic subgraph is a multiset, allowing multiple edges of different semantic types to exist between the same pair of nodes.
[0010] Preferably, the nodes of the event semantic subgraph include event trigger words and event argument entities, the edges of the event semantic subgraph are directed edges from event trigger words to event arguments, and the edge labels are the corresponding event argument roles.
[0011] Preferably, the normalized text sequence adopts a hierarchical structure, with the top layer being a Multi-Graph structure containing several semantic subgraph units. Each semantic subgraph unit has the format SubGraph(Task)[Type]{Nodes: list of nodes, Edges: list of edges}, where Task represents the coarse-grained task category, Type represents the fine-grained semantic type, and each edge is represented in the format (source_node-[label]->target_node).
[0012] Preferably, the deterministic rules include: sorting the semantic subgraphs according to semantic category and semantic type, sorting the node list according to the order in which the nodes appear in the original sentence, and sorting the edge list within each semantic subgraph in a fixed order.
[0013] Preferably, the training objective of supervised fine-tuning is to maximize the conditional generation probability, and the corresponding loss function is... for: ; in, The total number of training samples, For large language model parameters, For the first The instruction input for each sample, For the first Normalized multi-image text output for each sample The conditional probability of generating corresponding multi-image text when given a command input.
[0014] Preferably, when inputting the instruction to construct training samples, dataset identification information is added to indicate the annotation specifications followed by the current sample in the large language model and to distinguish the annotation conventions of different datasets.
[0015] Preferably, during the reasoning stage, there is no need to design a dedicated decoder or structured search module for the task. The structured extraction results of entities, relations, event trigger words and event arguments can be obtained directly from the normalized multi-graph text sequence generated by the large language model.
[0016] Therefore, the present invention employs the above-mentioned method for extracting information from a large language model based on multi-graph instruction fine-tuning, which has the following beneficial effects: 1. By uniformly converting heterogeneous information extraction and labeling into typified directed multi-graphs, and further textualizing them into normalized sequences, the graph structure can be directly used as the generation target of large language models, eliminating the need to design dedicated classifiers, decoders, or search modules for different subtasks. This solution is naturally adapted to the autoregressive generation capabilities of large language models, simultaneously supporting named entity recognition, relation extraction, event detection, event argument extraction, and joint information extraction tasks within a unified framework, significantly reducing system implementation complexity and multi-task adaptation costs.
[0017] 2. This method employs a multi-graph structure partitioned by semantic type, organizing entities, relations, and events into independent semantic subgraph spaces, thus avoiding structural confusion between different semantic types. It also supports multiple edges with different semantic labels between the same pair of nodes, fully expressing complex semantics such as overlapping entities, nested entities, multiple relations, and multiple events with multiple arguments. Experiments show that this method significantly improves accuracy compared to existing generative methods for complex structural tasks such as relation extraction, event argument extraction, and joint information extraction.
[0018] 3. By constraining the textualization results of multiple images through deterministic ranking rules, the same annotation result corresponds to a unique target text sequence, effectively reducing order ambiguity and format noise during training and improving the stability of large language model generation. Compared with code-based output schemes, the textualization format syntax of this invention is concise and the parsing rules are clear. It does not require a complex external parser or strict code syntax verification. The output can be directly parsed into structured data through preset syntax rules, resulting in lower engineering implementation and maintenance costs.
[0019] 4. During the instruction fine-tuning stage, dataset identification information is added to the input instructions to guide the large language model to distinguish the differences in annotation specifications between different datasets. This allows the large language model to simultaneously learn shared semantic structures and dataset-specific annotation conventions, effectively alleviating the annotation standard confusion problem in multi-dataset joint training. In zero-shot, cross-domain testing scenarios, the general extraction capability of the large language model can be evaluated without omitting the dataset identification, demonstrating good task transferability.
[0020] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0021] Figure 1 This is a flowchart of a method for extracting information from a large language model based on multi-graph instruction fine-tuning according to the present invention; Figure 2 This is a schematic diagram of the multi-graph construction, textification, and instruction generation process of the present invention; Figure 3 This is a schematic diagram of the multi-graph structure textual syntax of the present invention. Detailed Implementation
[0022] To make the objectives, technical solutions, and advantages disclosed in the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present invention and are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.
[0023] It should be noted that the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as a process, method, system, product, or server that includes a series of steps or units, not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or device.
[0024] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
[0025] like Figures 1-3 As shown, this invention provides a method for information extraction from a large language model based on multi-graph instruction fine-tuning. The method comprises four core steps: multi-graph construction, multi-graph texturing, multi-graph instruction fine-tuning, and reasoning and parsing. By transforming the structured results of information extraction into typified directed multi-graphs, and then texturing them into a generateable canonical sequence, the large language model can learn structured extraction capabilities through instruction fine-tuning. The specific implementation steps are as follows: S1. Multi-graph construction: This step extracts the entities, relationships, events, etc. from the input text and represents them in a unified way as a typed directed multi-graph structure.
[0026] Suppose the input sentence is a sequence of words. for: ; in, The total number of lexical units in the sentence. For the first sentence Each word element, This is the lexical index of the sentence. The text mention span is defined as... ,in , and These represent the starting and ending lexical positions of the mention in the input sentence, respectively. This invention allows different text spans to overlap across lexical intervals, thereby supporting the representation of overlapping and nested entities. Let the entity type set be... The set of relation types is The event type set is Then the global semantic type set for: ; Based on the aforementioned set of semantic types, each information extraction instance is represented as a set of semantic subgraphs, i.e., a typed directed multigraph. As shown below: ; in, Indicates semantic type The corresponding semantic subgraph, This is the set of nodes in the semantic subgraph. Let this be the set of edges of the semantic subgraph. The definitions of different types of semantic subgraphs are as follows: Entity semantic subgraph: for entity type Node set It consists of all entity mention nodes corresponding to this type, and each node can be represented as a triple. These correspond to the start position, end position, and entity type, respectively. Since named entity recognition only needs to identify entity mentions and types, and does not involve the connections between entities, the edge set of the entity semantic subgraph... .
[0027] Relational semantic subgraph: for relation types Node set The set of edges includes all entity mentions in the relationship. It contains directed edges pointing from the head entity to the tail entity, with each edge corresponding to a relation type. The edge set in this invention is a multiset, allowing multiple edges with different semantic types to exist between the same pair of nodes, in order to express the situation where multiple relations exist between the same pair of entities.
[0028] Event semantic subgraph: for event types Node set The set of edges containing the event trigger word node and all event argument entity nodes. It contains directed edges from event trigger word nodes to each event argument node, with the edge label being the corresponding event argument role (such as agent, patient, location, time, etc.). The same event trigger word can point to multiple arguments, and the same entity node can also be the target of multiple argument roles, supporting the expression of complex event structures.
[0029] S2. Multi-Graph Textification: Since large language models generate text in an autoregressive manner, they cannot directly output graph structure data. This step converts typed directed multi-graphs into normalized text sequences according to deterministic rules, ensuring that they retain complete graph structure information while still serving as generation targets for large language models. The textification format adopts a hierarchical structure: Top-level structure: The whole is wrapped in Multi-Graph{...}, which contains several semantic subgraph units.
[0030] Semantic subgraph unit: The format of each semantic subgraph is: plaintextSubGraph(Task)[Type]{ Nodes: [node1, node2, ...], Edges: [edge1, edge2, ...] } Among them, Task is the coarse-grained task category, with values including Entity, Relation, and Event. Type is the corresponding fine-grained semantic type (such as "person", "organization", "employment relationship", "meeting event", etc.). The Nodes field lists all node texts of the current semantic subgraph, and the Edges field lists all directed edges of the current semantic subgraph. For entity semantic subgraphs, the Edges field takes the value [].
[0031] Edge format: Each edge is represented in the form of (source node - [label] -> target node), where the label is the relation type or event argument role.
[0032] To eliminate order ambiguity and reduce training noise during the generation process, this invention applies a deterministic ranking rule to the textualized results: Semantic subgraph sorting: First sort by coarse-grained task category (Entity→Relation→Event), then sort within the same task category by a fixed order of semantic type (e.g., lexicographical order); Node sorting: Nodes within each semantic subgraph are arranged according to their order of appearance in the original sentence (ascending order of starting position); Edge sorting: Edges within each semantic subgraph are sorted according to the order in which the source nodes appear. If the source nodes are the same, they are sorted according to the order in which the target nodes appear. If they are evenly matched, they are sorted according to the fixed order of the edge labels.
[0033] By using the above rules, the same information extraction and annotation result corresponds to a unique normalized text sequence, avoiding the problem of inconsistent training samples caused by different linearization orders.
[0034] S3. Multi-Image Command Fine-Tuning: This step transforms the unified information extraction task into a command-following generation task. Supervised fine-tuning is used to train a large language model, enabling it to directly generate corresponding standardized multi-image text based on input commands. Specifically, this includes: (1) Construction of training samples: For the first training sample... Use training samples to construct instruction input. With target output , forming training pairs Command input Typically, the instructions include: task definition, optional label constraints (specifying the entity / relationship / event type to be extracted), optional few-shot examples, and the original input sentence. Additionally, to differentiate the annotation standards of different datasets, dataset identification information can be added to the instructions, indicating the annotation standards followed by the large language model for the current sample; target output. This is a normalized multi-image text sequence corresponding to the input sentence.
[0035] Heterogeneous information extraction data from different sources and for different tasks are uniformly converted into a multi-graph target space, specifically including: Named entity recognition data: converted into entity semantic subgraphs corresponding to the entity types; Relational data extraction: Converted into a relational semantic subgraph of the corresponding relation type, which also includes associated entity nodes; Event extraction data: Converted into an event semantic subgraph corresponding to the event type, which includes event trigger word nodes and argument nodes; Data extraction through joint information extraction: Simultaneously constructing corresponding entity semantic subgraphs, relation semantic subgraphs, and event semantic subgraphs.
[0036] Through a unified multi-graph representation, heterogeneous tasks and heterogeneous datasets can be jointly trained in the same target space.
[0037] (2) Training objectives: Assume the training corpus contains a total of Using a training instance, supervised fine-tuning is employed to optimize the parameters of the large language model. The training objective is to maximize the conditional generation probability of the target sequence, which is equivalent to minimizing the negative log-likelihood loss, as shown below: ; in, The parameter is Large language models, given instruction input Generate target multi-image text under the condition of The probability of this. Through this training process, the large language model can learn the mapping relationship from natural language instructions to structured multi-graphs, while mastering the structural organization rules of different semantic types.
[0038] In this embodiment, the basic large language model uses Qwen2.5-7B, with parameters loaded from the pre-trained weights released by the official documentation. The fine-tuning phase employs the AdamW optimizer with a learning rate of 2×10⁻⁵, using a cosine annealing learning rate scheduling strategy. The training consists of three epochs with a batch size of 128. The training data contains approximately 2.19 million filtered information extraction samples, covering four tasks: named entity recognition, relation extraction, event extraction, and joint information extraction. All training samples are pre-converted into normalized multi-graph text sequences according to steps S1-S2. During fine-tuning, all parameters of the large language model are updated without freezing any network layers.
[0039] S4. Reasoning and Analysis: In the reasoning phase, information extraction is performed according to the following process: Instruction construction: The text to be extracted, the task definition, and the target label constraints are combined to form the inference instruction; if it is necessary to adapt to the annotation specifications of a specific dataset, the corresponding dataset identifier can be added to the instruction.
[0040] Large Language Model Generation: Input the instructions into the fine-tuned large language model, and the large language model outputs a text sequence in a Multi-Graph structure according to the learned normalized format.
[0041] Structural parsing: The system parses the generated text according to preset grammatical rules that are the reverse of the textification process in S2, extracts the node and edge information of all semantic subgraphs in sequence, and finally converts them into structured data that can be used by downstream systems, such as entity sets, relation sets, event trigger words and argument sets.
[0042] Since the output format is deterministic canonical text, there is no need to design task-specific graph decoders, complex bundle search modules, or code-level external parsers. Structured results can be obtained simply through text syntax parsing, resulting in concise engineering implementation and higher structural consistency and parsability.
[0043] Experimental verification: The present invention (Qwen2.5-7B+ multi-graph instruction fine-tuning) was quantitatively compared with several representative large language models of existing technologies (UniNER, KnowCoder, UIE, TagPrime, UTC-IE) on multiple tasks. Experimental data show that this invention covers two major scenarios: supervised learning (NER (Named Entity Recognition), RE (Relation Extraction), EE (Event Extraction) and zero-shot learning (Zero-shot Named Entity Recognition), proving that the method is not only effective but also has excellent generalization ability. The improvement effect of this invention is particularly prominent in joint event extraction (+5.2%) and zero-shot NER (+7.8%), which corresponds to the design goals of this invention in "handling complex semantic structures" and "generalization ability". In addition, even on tasks with smaller improvements (such as the average F1 score of supervised NER), the result of this invention is still the best (87.8), proving the robustness of the method. The average F1 score of this invention and various comparative large language models in multi-class extraction tasks is compared in Table 1.
[0044] Table 1: Comparative Experiment Results
[0045] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.
Claims
1. A method for extracting information from a large language model based on multi-graph instruction fine-tuning, characterized in that, Includes the following steps: S1. Multi-graph construction: Information extraction and labeling in the input sentence are uniformly converted into a typed directed multi-graph structure, where the typed directed multi-graph is composed of semantic subgraphs corresponding to multiple semantic types; S2, Multi-graph textification: Transforming typed directed multi-graphs into normalized text sequences according to deterministic rules, serving as the generation target for large language models; S3. Multi-graph instruction fine-tuning: Construct instruction samples containing task definitions and input sentences, and use the corresponding normalized multi-graph text sequences as output targets to perform supervised fine-tuning of the large language model; S4. Reasoning and parsing: Input the text to be extracted and the task instructions into the fine-tuned large language model to generate a multi-image text sequence, and obtain the structured information extraction result by parsing according to the preset syntax.
2. The method for extracting information from a large language model based on multi-graph instruction fine-tuning according to claim 1, characterized in that: The semantic subgraphs of typified directed multigraphs include three categories: entity semantic subgraphs, relation semantic subgraphs, and event semantic subgraphs, which correspond to the semantic structures of named entity recognition, relation extraction, and event extraction tasks, respectively.
3. The method for extracting information from a large language model based on multi-graph instruction fine-tuning according to claim 2, characterized in that: The nodes of the entity semantic subgraph are entity mentions bound to the text span of the input sentence. Each node corresponds to an entity type, and the edge set of the entity semantic subgraph is empty.
4. The method for extracting information from a large language model based on multi-graph instruction fine-tuning according to claim 2, characterized in that: The nodes of the relational semantic subgraph are entity references that participate in the semantic relationship, and the edges are directed edges from the head entity to the tail entity. Each edge corresponds to a relation type. The edge set of the relational semantic subgraph is a multiset, which allows multiple edges of different semantic types to exist between the same pair of nodes.
5. The method for extracting information from a large language model based on multi-graph instruction fine-tuning according to claim 2, characterized in that: The nodes of the event semantic subgraph include event trigger words and event argument entities. The edges of the event semantic subgraph are directed edges from event trigger words to event arguments, and the edge labels are the corresponding event argument roles.
6. The method for extracting information from a large language model based on multi-graph instruction fine-tuning according to claim 1, characterized in that: The normalized text sequence adopts a hierarchical structure, with the top layer being a Multi-Graph structure containing several semantic subgraph units. Each semantic subgraph unit has the format SubGraph(Task)[Type]{Nodes: list of nodes, Edges: list of edges}, where Task represents the coarse-grained task category, Type represents the fine-grained semantic type, and each edge is represented in the format (source_node-[label]->target_node).
7. The method for extracting information from a large language model based on multi-graph instruction fine-tuning according to claim 6, characterized in that, The deterministic rules include: sorting the semantic subgraphs according to semantic categories and semantic types, sorting the node list according to the order in which nodes appear in the original sentence, and sorting the edge list within each semantic subgraph in a fixed order.
8. The method for extracting information from a large language model based on multi-graph instruction fine-tuning according to claim 1, characterized in that: The training objective of supervised fine-tuning is to maximize the conditional generation probability, and the corresponding loss function is... for: ； in, The total number of training samples, For large language model parameters, For the first The instruction input for each sample, For the first Normalized multi-image text output for each sample The conditional probability of generating corresponding multi-image text when given a command input.
9. The method for extracting information from a large language model based on multi-graph instruction fine-tuning according to claim 1, characterized in that: When inputting instructions to construct training samples, dataset identification information is added to indicate the annotation specifications followed by the large language model and to distinguish the annotation conventions of different datasets.
10. The method for extracting information from a large language model based on multi-graph instruction fine-tuning according to claim 1, characterized in that: During the reasoning phase, there is no need to design a dedicated decoder or structured search module for the task. The structured extraction results of entities, relations, event trigger words and event arguments are obtained directly from the normalized multi-graph text sequence generated by the large language model.