A large language model alignment method and system for information extraction tasks

Abstract

The application provides a large language model alignment method and system for an information extraction task, the method comprising: determining an information extraction guidance adjustment data set; the information extraction guidance adjustment data set comprising diversified inputs and answer outputs; the diversified inputs aligning the large language model on the information extraction task based on preset information extraction data; the answer outputs corresponding to the format requirements of the diversified inputs; and performing supervised fine-tuning training on a preset large language model based on the information extraction guidance adjustment data set and a preset general alignment corpus to obtain a supervised fine-tuning large language model. The application determines high-quality alignment data for the information extraction task, and performs supervised fine-tuning training on the preset large language model based on the alignment data, so that the supervised fine-tuning large language model has good generalization capability on the information extraction task and does not affect the general performance of the large language model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for aligning large language models for information extraction tasks. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology, large language models (LLMs) have shown outstanding performance in text generation, and their capabilities and future potential have received widespread attention. However, large language models still have shortcomings in information extraction (IE) tasks, especially in closed-ended information extraction tasks, where they often struggle to understand and follow the complex instructions.

[0003] To improve the performance of large language models in information extraction tasks, existing techniques employ prompting engineering, code-based methods, or multi-task fine-tuning. While these methods can enhance the ability of large language models to extract information, they also suffer from limited generalization capabilities due to the lack of diverse alignment data. Furthermore, overfitting may occur during model fine-tuning, impacting the general performance of large language models. Summary of the Invention

[0004] This invention provides a method and system for aligning large language models for information extraction tasks, addressing the shortcomings of existing large language models in terms of limited generalization ability and impact on general performance. This invention identifies high-quality alignment data for information extraction tasks and performs supervised fine-tuning training on a pre-defined large language model based on this alignment data. This enables the supervised fine-tuned large language model to exhibit better generalization ability in information extraction tasks without affecting the general performance of the large language model.

[0005] This invention provides a method for aligning a large language model for an information extraction task, comprising: determining an information extraction guidance adjustment dataset; the information extraction guidance adjustment dataset including diverse inputs and answer outputs; aligning the diverse inputs with a large language model on the information extraction task based on preset information extraction data; the answer outputs corresponding to the format requirements of the diverse inputs; performing supervised fine-tuning training on a preset large language model based on the information extraction guidance adjustment dataset and a preset general alignment corpus to obtain a supervised fine-tuned large language model; the supervised fine-tuned large language model is a large language model aligned on the information extraction task; the preset general alignment corpus is a dataset guiding the adjustment of the general performance of the large model.

[0006] According to the present invention, a method for aligning large language models for information extraction tasks is provided, wherein the preset information extraction data includes closed-ended information extraction data, open-ended information extraction data, and on-demand information extraction data; the preset information extraction data adopts an example ratio mixing strategy to set a data volume threshold.

[0007] According to the present invention, a large language model alignment method for information extraction tasks is provided, wherein the diverse inputs include a guide and input text; the guide includes a task description part, a pattern description part, and an output format description part; the input text is the text of the preset large language model for the information extraction task; the task description part is used to determine the guided task description; the guided task description is a description that iteratively generates a new task description by combining a human task description and a generated task description, and the new task description needs to be verified by the task description; the generated task description is a task description automatically generated by a preset generative converter based on the human task description; the pattern description part is used to determine the guided pattern description; the guided pattern description is a description after pattern description enhancement processing; the pattern description enhancement processing is a combination of one or more processing methods among pattern shuffling, pattern sampling, integration of guidelines, and symbol replacement categories; the output format description part is used to determine the guided output format description; the guided output format description includes a triple format description, a JSON format description, and a natural language format description.

[0008] According to the present invention, a large language model alignment method for information extraction tasks is provided, wherein the instruction further includes step-by-step explanation requirements and / or sample examples; the step-by-step explanation requirements require the preset large language model to perform step-by-step explanations before outputting the answer to the input text; the sample examples include input text examples, step-by-step explanation examples, and output answer examples; the step-by-step explanation examples correspond to the step-by-step explanation requirements.

[0009] According to the present invention, a large language model alignment method for information extraction tasks is provided, wherein the answer output includes an output format corresponding to the output format description portion; the answer output also includes a step-by-step explanation output corresponding to the step-by-step explanation requirement.

[0010] According to the present invention, a method for aligning a large language model for an information extraction task, after supervising the fine-tuning training of a preset large language model based on the information extraction-guided adjustment dataset and the preset general alignment corpus to obtain a supervised fine-tuned large language model, the method further includes: determining a direct preference optimization dataset based on the information extraction-guided adjustment dataset and the supervised fine-tuned large language model; the direct preference optimization dataset includes online data preference pairs and offline data preference pairs; and performing direct preference optimization training on the supervised fine-tuned large language model based on the direct preference optimization dataset to obtain a directly preference optimized large language model.

[0011] According to the present invention, a large language model alignment method for information extraction tasks is provided, wherein the information extraction guidance adjustment dataset includes data labels for standard answers; the step of determining a direct preference optimization dataset based on the information extraction guidance adjustment dataset and the supervised fine-tuning large language model includes: setting the output sampling of the supervised fine-tuning large language model; the output sampling settings include setting the number of samplings and the sampling temperature; based on the information extraction guidance adjustment dataset, using the output answer preference pairs of the supervised fine-tuning large language model as the online data preference pairs; the output answer preference pairs include preferred answers and unpreferred answers; the preferred answer is the answer with the highest BLEU score among the output answers of the supervised fine-tuning large language model; the unpreferred answer is the answer with the lowest BLEU score among the output answers of the supervised fine-tuning large language model; and the offline data preference pairs are determined according to the output answer preference pairs; the offline data preference pairs include the standard answers and the unpreferred answers.

[0012] This invention also provides a large language model alignment system for information extraction tasks, comprising: a dataset determination module for determining an information extraction guidance adjustment dataset; the information extraction guidance adjustment dataset includes diverse inputs and answer outputs; the diverse inputs are aligned with a large language model on the information extraction task based on preset information extraction data; the answer outputs correspond to the format requirements of the diverse inputs; and a model training module for supervised fine-tuning training of a preset large language model based on the information extraction guidance adjustment dataset and a preset general alignment corpus to obtain a supervised fine-tuned large language model; the supervised fine-tuned large language model is a large language model aligned on the information extraction task; and the preset general alignment corpus is a dataset for guiding the adjustment of the general performance of the large model.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the large language model alignment method for information extraction tasks as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the large language model alignment method for information extraction tasks as described above.

[0015] This invention provides a method and system for aligning a large language model for information extraction tasks. The method includes: determining an information extraction guidance adjustment dataset; the information extraction guidance adjustment dataset including diverse inputs and answer outputs; aligning the diverse inputs with a large language model on the information extraction task based on pre-defined information extraction data; the answer outputs corresponding to the format requirements of the diverse inputs; and supervising fine-tuning training the pre-defined large language model based on the information extraction guidance adjustment dataset and a pre-defined general alignment corpus to obtain a supervised fine-tuned large language model. This invention determines high-quality alignment data for information extraction tasks and performs supervised fine-tuning training on the pre-defined large language model based on the alignment data, enabling the supervised fine-tuned large language model to have good generalization ability on information extraction tasks without affecting the general performance of the large language model. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a method for aligning large language models for information extraction tasks provided by the present invention.

[0018] Figure 2 This is a schematic diagram illustrating the principle of a large language model alignment method for information extraction tasks provided by the present invention.

[0019] Figure 3 This is a schematic diagram illustrating the format description of tasks, datasets, and their respective proportions in IEInstruct provided by the present invention;

[0020] Figure 4a This is an example diagram (in English) of the input and output in IEInstruct provided by this invention;

[0021] Figure 4b This is an example diagram (in Chinese) of the input and output in IEInstruct provided by this invention;

[0022] Figure 5 This is a schematic diagram of the structure of a large language model alignment system for information extraction tasks provided by the present invention;

[0023] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0026] Large language models refer to language models trained on massive text corpora, containing billions (or more) of parameters. Large language models excel in various Natural Language Processing (NLP) tasks, attracting widespread attention from academia and industry. ChatGPT, a new chatbot model released by OpenAI on November 30, 2022, can learn and understand human language and generate responses to engage in dialogue based on context. ChatGPT can be used to perform various NLP tasks, such as information extraction, machine translation, code generation, and sentiment recognition.

[0027] However, large language models still have shortcomings in information extraction tasks, especially in closed-ended information extraction tasks. Large language models often struggle to understand and follow the complex instructions of information extraction tasks, such as complex task architectures and specifications, indicating that existing large language models are inconsistent with human needs in information extraction tasks.

[0028] To improve the performance of large language models in information extraction tasks, existing technologies mainly focus on three directions: (1) prompting engineering, which provides input support to large language models by providing comprehensive information, such as annotation guidelines, without the need for fine-tuning model parameters. (2) code-based large language models, which leverage their ability to understand structured information to improve the performance of information extraction tasks. (3) multi-task fine-tuning, which involves fine-tuning large language models on multiple information extraction datasets to improve their cross-task generalization ability and solve information extraction tasks.

[0029] However, existing techniques do not adequately align large language models for information extraction tasks. Hint engineering methods cannot fundamentally align large language models without adjusting model parameters. Coding approaches and multi-task fine-tuning often fine-tune models on homogeneous data, such as instances with the same input / output formats, lacking diverse data for alignment.

[0030] Therefore, the fine-tuned models exhibit limited generalization ability on information extraction tasks. Furthermore, since these models are specifically trained for information extraction, their general performance, such as in natural language understanding, may show a significant decline.

[0031] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a large language model alignment method for information extraction tasks provided by the present invention.

[0032] This invention provides a method for aligning large language models for information extraction tasks, comprising:

[0033] 101: Determine the information extraction guidance dataset; the information extraction guidance dataset includes diverse inputs and answer outputs; the diverse inputs are aligned with the large language model on the information extraction task based on the pre-defined information extraction data; the answer outputs correspond to the format requirements of the diverse inputs;

[0034] 102: Supervised fine-tuning training of a pre-defined large language model is performed based on an information extraction-guided adjustment dataset and a pre-defined general alignment corpus to obtain a supervised fine-tuned large language model; the supervised fine-tuned large language model is a large language model aligned on the information extraction task; the pre-defined general alignment corpus is a dataset that guides the adjustment of the general performance of the large model.

[0035] To address the technical problems existing in current technologies, this invention provides a method for aligning large language models for information extraction tasks. This method introduces ADELIE (Aligning large language models on Information Extraction), a large language model specifically designed for information extraction tasks. First, a high-quality guidance dataset (IEInstruct includes 83,585 instances of various information extraction tasks) is constructed for information extraction tasks. This dataset includes diverse inputs and outputs, providing rich alignment data. Then, the pre-defined large language model is trained using supervised fine-tuning (SFT) on a hybrid dataset consisting of the guidance dataset and a pre-defined general alignment corpus. This achieves sufficient alignment, giving the supervised fine-tuned large language model good generalization ability for information extraction tasks without affecting the general performance of the large language model.

[0036] IEInstruct is constructed as follows: First, multiple instructions were manually written for different information extraction tasks. Then, using the Self-Instruct method, the instruction set was expanded using GPT-4 (Generative Pre-trained Transformer 4). The instructions were further expanded through various enhancement techniques, such as adding annotation guidelines. IEInstruct also includes diverse output formats, such as triples, natural language, and JSON. Furthermore, GPT-4 was used to generate thought chain explanations (step-by-step explanations) for 10% of the instances in IEInstruct.

[0037] The preset general alignment corpus can be, but is not limited to, for -V2-mix.

[0038] Preset large language models can be, but are not limited to, LLAMA 2 (Large Language Model Augmented with Meta-learned Approaches 2), GPT-3 (Generative Pre-trained Transformer 3), GPT-4 (Generative Pre-trained Transformer 3), and PaLM (Scaling Language Modeling with Pathways).

[0039] For supervised finetuning (SFT) training of the large language model, to preserve the general capabilities of the large language model during alignment, a general alignment corpus used in the TULU 2 model training was utilized. Specifically, IEInstruct (83,585 instances) was mixed with instances from a 320,000-unit general alignment corpus as the training dataset. LLAMA 2 was used as the backbone model, with two batches trained, an effective batch size of 128, a learning rate of 2e-5 using a cosine scheduler, and a 3% warm-up period. To better facilitate the learning of few-shot settings and document-level information extraction, the context length was set to 2048 tags. After training the model for 6,306 gradient steps, the supervised fine-tuned large language model ADELIE_SFT was obtained.

[0040] Information extraction refers to extracting specific event or factual information from natural language text to help automatically classify, extract, and reconstruct text content. This information typically includes entities, relations, and events. The information extraction mentioned in this invention is divided into three parts: (1) Closed-ended information extraction, which extracts the required information from the text based on the given text and pattern, including relation extraction, named entity recognition, and event extraction. (2) Open-ended information extraction, which extracts n-gram relation groups from natural language text in a pattern-independent manner, given only the text. (3) On-demand information extraction, which extracts the information needed by the user from the text, such as extracting the taste and shape of fruit from text describing fruit, and finally organizing it into a structured table.

[0041] Alignment refers to the fact that while large language models have enormous potential, they may not be able to fully understand human-inputted text. Therefore, employing alignment techniques to ensure these models can correctly understand human input becomes crucial.

[0042] The method of this invention is applicable to both Chinese and English information extraction, and no particular limitation is made here.

[0043] The methods provided in the embodiments of the present invention can be applied to information extraction tasks in general fields, as well as to information extraction tasks in specific fields, such as smart terminals, intelligent transportation, smart healthcare, smart education, autonomous driving, and smart cities.

[0044] Based on the above embodiments:

[0045] Please refer to Figure 2 , Figure 2 This is a schematic diagram illustrating the principle of a large language model alignment method for information extraction tasks provided by the present invention.

[0046] As a preferred embodiment, the preset information extraction data includes closed-ended information extraction data, open-ended information extraction data, and on-demand information extraction data; the preset information extraction data uses a mixed example ratio strategy to set the data volume threshold.

[0047] Please refer to Figure 3 , Figure 3 This is a schematic diagram illustrating the format description of tasks, datasets, and their respective proportions in IEInstruct provided by the present invention.

[0048] In this embodiment, multiple information extraction task datasets (preset information extraction data) were collected, including closed information extraction data, open information extraction data, and on-demand information extraction data, covering various fields such as general fields, financial fields, and biomedical fields.

[0049] This embodiment can also filter out 80% of NA (Not Available, no valid information to extract) data, which does not contain the information that needs to be extracted.

[0050] Of course, in order to balance different datasets, this embodiment adopts a mixed example ratio strategy and sets a data volume threshold for each dataset size (for example, the data volume of each dataset is limited to 5000).

[0051] Specifically, IEInstruct collects 13 publicly available datasets covering 9 types of information extraction tasks: (1) Named Entity Recognition (NER), which identifies entities from text and classifies them into types in a schema; (2) Relation Classification (RC), which classifies the relationship between two given entities based on text and schema; (3) Relation Extraction (RE), which identifies entity pairs in text and analyzes the relationship between them; (4) Event Detection (ED), which detects events by identifying event triggers and classifying event types; (5) Event Argument Extraction (EAE), which extracts event parameters and classifies their roles; (6) Event Extraction (EE), which combines the previous two tasks, identifying event trigger words and extracting event parameters; (7) Event Relation Extraction (ERE), which analyzes coreference, temporal, causal, and hierarchical relationships between events; and (8) On-Demand Information Extraction. Extraction (ODIE) aims to extract the required content from related text according to the instructions and present it in a structured table format; (9) Information Extraction (Open IE) is an ontology-free information extraction paradigm that can generate extraction forms (subject; relation; object).

[0052] In a preferred embodiment, the diverse inputs include a guide and input text; the guide includes a task description section, a pattern description section, and an output format description section; the input text is the text of a preset large language model for the information extraction task; the task description section is used to determine the guiding task description; the guiding task description is a description that iteratively generates a new task description by combining a human task description and a generated task description, and the new task description needs to be verified by the task description; the generated task description is a task description automatically generated by a preset generative converter based on the human task description; the pattern description section is used to determine the guiding pattern description; the guiding pattern description is a description after pattern description enhancement processing; the pattern description enhancement processing is a combination of one or more processing categories such as pattern shuffling, pattern sampling, integration of guidelines, and symbol replacement; the output format description section is used to determine the guiding output format description; the guiding output format description includes a triple format description, a JSON (JavaScript Object Notation) format description, and a natural language format description.

[0053] As a preferred embodiment, the instructions also include step-by-step explanation requirements and / or sample examples; the step-by-step explanation requirements require the pre-defined large language model to perform step-by-step explanations before outputting the answer to the input text; the sample examples include input text examples, step-by-step explanation examples, and output answer examples; the step-by-step explanation examples correspond to the step-by-step explanation requirements.

[0054] Please refer to Figure 4a , Figure 4a An example diagram (in English) of input and output in IEInstruct provided by this invention.

[0055] Please refer to Figure 4b , Figure 4b An example diagram (in Chinese) of input and output in IEInstruct provided for this invention.

[0056] To better align large language models with information extraction tasks, this embodiment constructs diverse inputs. These diverse inputs primarily consist of a guide and a piece of input text. The guide typically comprises three parts: a task description, a pattern description, and an output format description. The pattern description is used only in closed-ended information extraction tasks, as open-ended and on-demand information extraction do not include patterns. Some inputs also include several demonstrations, i.e., input-output examples, to enhance few-shot context learning capabilities. The enhancement process of the three parts of the guide and the construction of the sample examples will be described below.

[0057] In the task description section, for each information extraction task, 10 task descriptions were first manually created (human task descriptions). Then, GPT-4 was used to generate 20 similar descriptions (generated task descriptions). Specifically, to enrich the diversity of the generated descriptions, similar to Self-Instruct, an iterative generation process was used. This process used 3 manually written descriptions and 2 generated descriptions as prompts for GPT-4 to generate new descriptions. Finally, the generated descriptions were manually verified, and illusions were filtered out.

[0058] For closed-ended information extraction tasks, the pattern description section, inspired by GoLLIE (Guideline following Large Language Model for IE), enhances pattern description in three ways, namely, category information: (a) Pattern shuffling and pattern sampling: The order of categories in the pattern is randomly shuffled, and a random subset of 1 to the maximum number of categories included in the guideline is selected. This technique aims to prevent the model from overfitting patterns in the training corpus, forcing the model to learn to output only instances of classes defined in the input. This not only encourages the model to focus on the pattern definition but also minimizes the occurrence of illusions during reasoning. (b) Incorporation of guidelines: The guidelines are the definitions of the patterns that can enhance the model's ability to understand the pattern definitions, thereby improving the model's zero-shot generalization ability on unseen tasks. For example: "victim" means "the person who was injured / the person who died." Therefore, guideline information can be added to 20% of the data in the training corpus. Similar to GoLL information extraction, this embodiment also includes several examples (similar texts that appeared in the training set before, such as the name of the victim) in the guidelines for each category. The remaining data does not include guidelines to prevent the model from memorizing pattern definitions and to enhance data diversity. (c) Replace categories with symbols: Replace category names with random symbols (e.g., LABEL_1) to prevent the model from overfitting to category names and to enhance the model's context learning ability.

[0059] The output format description section addresses the challenge that large language models sometimes struggle to adhere to required output formats in information extraction tasks. To enhance the model's ability to comply with format requirements, various output format descriptions are introduced in the guide, specifying the appropriate output requirements. Specifically, for each closed and open information extraction task, there are three main types of formats: (a) Triple format, specifying output in various triple formats, such as (head entity; relation; tail entity) or (head entity; tail entity; relation) in relation extraction; (b) JSON format, requiring the model to output results in JSON format; and (c) Natural language format, with no specific format requirements, allowing the model to output in natural language. The process of constructing the output corresponding to the format requirements is detailed in the answer generation section. On-demand information extraction does not involve output format descriptions, as its output is typically in a fixed Markdown format.

[0060] A step-by-step explanation requires, for example, first explaining your idea step by step, and then providing the answer.

[0061] Sample examples are used to enhance the model's few-shot context learning capabilities by augmenting the training corpus with few-shot demo inputs. Specifically, 50% of the training data is randomly selected, and 1 to 8 randomly sampled examples are added to the original input. These examples consist of an input text and an output, where the output format conforms to the guidelines. For each instance, the demos are randomly sampled and shuffled to prevent the model from overfitting to a fixed set of demos.

[0062] In a preferred embodiment, the answer output includes an output format corresponding to the output format description section; the answer output also includes a step-by-step explanation output corresponding to the step-by-step explanation requirements.

[0063] In this embodiment, corresponding output formats are constructed based on the format requirements in the guidance document generated from diverse inputs. Specifically, for each closed-ended and open-ended information extraction task, the output includes three formats: (a) Triple format, which converts the output into a serialized triple form. For outputs containing multiple triples, the order of the triples is randomly shuffled to mitigate potential order bias. (b) JSON format, for which a set of JSON formats is designed for each task, and the answers are converted into corresponding JSON data. (c) Natural language format, for which several natural language output templates are manually written for each task, and the corresponding natural language output is constructed based on these templates. For on-demand information extraction, the original answers from its dataset are used.

[0064] To enhance the large language model's understanding of the information extraction task process, Chain of Thought (CoT) explanations (step-by-step explanations) were added to a subset (10%) of closed and open-ended information extraction. To generate high-quality CoT explanations, the input text and its actual answers were fed into GPT-4. Specifically, 1000 instances were first randomly sampled for each task. Then, using the text input and corresponding answers as input, GPT-4 was used to generate CoT explanations. 200 instances were randomly selected to evaluate the quality of the CoT explanations, and it was found that GPT-4 generally generated effective and informative step-by-step thinking for the final answer.

[0065] As a preferred embodiment, after supervising the fine-tuning training of a pre-defined large language model based on the information extraction-guided adjustment dataset and the pre-defined general alignment corpus to obtain the supervised fine-tuned large language model, the method further includes: determining a direct preference optimization dataset based on the information extraction-guided adjustment dataset and the supervised fine-tuned large language model; the direct preference optimization dataset includes online data preference pairs and offline data preference pairs; and performing direct preference optimization training on the supervised fine-tuned large language model based on the direct preference optimization dataset to obtain the direct preference optimized large language model.

[0066] As a preferred embodiment, the information extraction-guided adjustment dataset includes data labels for standard answers; determining the direct preference optimization dataset based on the information extraction-guided adjustment dataset and the supervised fine-tuning large language model includes: setting the output sampling of the supervised fine-tuning large language model; the output sampling settings include setting the number of samplings and the sampling temperature; based on the information extraction-guided adjustment dataset, using the output answer preference pairs of the supervised fine-tuning large language model as online data preference pairs; the output answer preference pairs include preferred answers and unpreferred answers; the preferred answer is the answer with the highest BLEU score among the output answers of the supervised fine-tuning large language model; the unpreferred answer is the answer with the lowest BLEU score among the output answers of the supervised fine-tuning large language model; determining offline data preference pairs based on the output answer preference pairs; the offline data preference pairs include standard answers and unpreferred answers.

[0067] Following supervised fine-tuning of the large model training, this embodiment continues training ADELIE_SFT using the DPO (Direct Preference Optimization) objective. First, a direct preference optimization dataset is constructed, mixing online and offline preference pairs to obtain more diverse data. Information extraction guides the adjustment of the dataset to include standard answer data labels, thus automatically constructing preference pairs. Using the model's own output and the original standard labels, without the need for additional manual annotation of preference pairs, sufficiently reduces manual intervention and saves manpower.

[0068] Specifically, BLEU scores are used as a metric to automatically construct preference pairs. The output of ADELIE_SFT is sampled five times, with a sampling temperature of 1.0 (ensuring that the five responses are not exactly identical). If the difference between the output answer with the highest BLEU score and the output answer with the lowest BLEU score exceeds 10%, the corresponding outputs are considered preference pairs, with the higher BLEU output being the preferred answer. The constructed preferred and unpreferred answers are called online data preference pairs. Alternatively, the lowest BLEU output can be considered the unpreferred answer, and the standard answer the preferred answer; these standard and unpreferred answers are called offline data preference pairs.

[0069] Online and offline preference pairs constitute the Direct Preference Optimization dataset (Feedback), containing 3,000 online preference pairs and 7,000 offline preference pairs. Then, using the DPO objective, similar to SFT, this embodiment trains the DPO model for three epochs. The global batch size for model training is 32. A linear learning rate scheduler is used, with a peak learning rate of 5e-7$ and a 10% warm-up phase. Finally, an additional 937 gradient steps are performed on ADELIE_SFT to obtain ADELIE_DPO (Direct Preference Optimization Large Language Model).

[0070] This invention presents extensive experiments and analyses, providing meaningful insights into the research of large language model alignment for information extraction tasks. A comprehensive evaluation of ADELIE_SFT and ADELIE_DPO was conducted on closed, open, and on-demand information extraction. Results show that the large language model of this invention achieves SoTA (State of the Art) performance compared to previous open-source models and GPT-4. ADELIE's general capabilities, such as MMLU (Measuring Massive Multitask Language Understanding) and BBH (Big-Bench Hard), show no significant degradation. Furthermore, this invention analyzes several key factors in the alignment process and provides insightful findings, such as a hybrid strategy for information extraction and general alignment data.

[0071] In closed IE tasks, the model of this invention was compared with existing models in five types of tasks. The models mainly include: (1) general open source LLMs, including the powerful base model LLAMA 2 and the instruction-tuned TULU 2 model, using version 7B of these models. (2) proprietary LLMs, including GPT-3.5 and GPT-4. (3) models optimized for IE tasks, including code-based LLMs fine-tuned for IE tasks (GoLLIE) and LLMs trained for multiple IE tasks (InstructUIE).

[0072] The experimental results in Table 1 show that: (a) the model performance of this invention is significantly better than the original LLAMA 2, and surpasses all IE LLMs and GPT-3.5, even comparable to GPT-4. This indicates that the alignment with the rich data IEInstruct of this invention is effective. (b) DPO performance is further improved. ADELIE_DPO consistently outperforms ADELIE_SFT on most datasets. This shows that for extraction tasks with true answers, further alignment using DPO can also improve model performance. (c) Incorporating contextual representation during the reconciliation process is necessary. Previous work only focused on zero-shot capability while ignoring the few-shot capability of LLMs, resulting in no significant improvement or even a decline when providing few-shot representations, such as a 4.3% drop in the F1 score for GoLLIE. In contrast, ADELIE_SFT's few-shot performance is far superior to its zero-shot performance, indicating that ADELIE_SFT has the ability to learn few-shot context for closed IE tasks. It demonstrates the effectiveness of incorporating contextual representation during the reconciliation process.

[0073] In addition to closed IE, the model of this invention also achieves the effect of the open source model SoTA on both open IE and on-demand IE.

[0074] Table 1 Model Performance Test Table

[0075]

[0076] The large language model alignment system for information extraction tasks provided by this invention will be described below. The large language model alignment system for information extraction tasks described below can be referred to in correspondence with the large language model alignment method for information extraction tasks described above.

[0077] Please refer to Figure 5 , Figure 5 This is a schematic diagram of the structure of a large language model alignment system for information extraction tasks provided by the present invention.

[0078] This invention also provides a large language model alignment system for information extraction tasks, comprising: a dataset determination module 1, used to determine an information extraction guidance adjustment dataset; the information extraction guidance adjustment dataset includes diverse inputs and answer outputs; the diverse inputs are aligned with the large language model on the information extraction task based on preset information extraction data; the answer outputs correspond to the format requirements of the diverse inputs; and a model training module 2, used to perform supervised fine-tuning training on the preset large language model based on the information extraction guidance adjustment dataset and a preset general alignment corpus to obtain a supervised fine-tuned large language model; the supervised fine-tuned large language model is a large language model aligned on the information extraction task; and the preset general alignment corpus is a dataset for guiding the adjustment of the general performance of the large model.

[0079] Figure 6 An example is a schematic diagram of the structure of an electronic device, such as... Figure 6As shown, the electronic device may include: a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604. The processor 601 can call logical instructions in the memory 603 to execute a large language model alignment method for an information extraction task. This method includes: determining an information extraction guidance adjustment dataset; the information extraction guidance adjustment dataset includes diverse inputs and answer outputs; aligning the diverse inputs with the large language model on the information extraction task based on preset information extraction data; the answer outputs corresponding to the format requirements of the diverse inputs; performing supervised fine-tuning training on the preset large language model based on the information extraction guidance adjustment dataset and a preset general alignment corpus to obtain a supervised fine-tuned large language model; the supervised fine-tuned large language model is the large language model aligned on the information extraction task; and the preset general alignment corpus is a dataset guiding the adjustment of the general performance of the large model.

[0080] Furthermore, the logical instructions in the aforementioned memory 603 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0081] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the large language model alignment method for information extraction tasks provided by the above methods. The method includes: determining an information extraction guidance adjustment dataset; the information extraction guidance adjustment dataset includes diverse inputs and answer outputs; aligning the diverse inputs with a large language model on the information extraction task based on preset information extraction data; the answer outputs corresponding to the format requirements of the diverse inputs; performing supervised fine-tuning training on the preset large language model based on the information extraction guidance adjustment dataset and a preset general alignment corpus to obtain a supervised fine-tuned large language model; the supervised fine-tuned large language model is a large language model aligned on the information extraction task; and the preset general alignment corpus is a dataset that guides the adjustment of the general performance of the large model.

[0082] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a method for aligning a large language model for an information extraction task as provided by the methods described above. This method includes: determining an information extraction guidance adjustment dataset; the information extraction guidance adjustment dataset including diverse inputs and answer outputs; aligning the diverse inputs with a large language model on the information extraction task based on preset information extraction data; the answer outputs corresponding to the format requirements of the diverse inputs; performing supervised fine-tuning training on the preset large language model based on the information extraction guidance adjustment dataset and a preset general alignment corpus to obtain a supervised fine-tuned large language model; the supervised fine-tuned large language model being a large language model aligned on the information extraction task; and the preset general alignment corpus being a dataset guiding the adjustment of the general performance of the large model.

[0083] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0084] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for aligning large language models for information extraction tasks, characterized in that, include: The information extraction guidance dataset is determined; the information extraction guidance dataset includes diverse inputs and answer outputs; The diverse inputs are aligned with a large language model on the information extraction task based on preset information extraction data; the answer output corresponds to the format requirements of the diverse inputs. The diverse inputs include instructions and input text; Based on the information extraction guidance adjustment dataset and the preset general alignment corpus, the preset large language model is trained under supervision to obtain a supervised fine-tuned large language model; the supervised fine-tuned large language model is a large language model aligned on the information extraction task. The preset general alignment corpus is a dataset used to guide the adjustment of the general performance of large models; After the supervised fine-tuning training of the preset large language model based on the information extraction-guided adjustment dataset and the preset general alignment corpus, to obtain the supervised fine-tuned large language model, the process further includes: Based on the information extraction guidance of the adjusted dataset and the supervised fine-tuning of the large language model, a direct preference optimization dataset is determined; the direct preference optimization dataset includes online data preference pairs and offline data preference pairs. The supervised fine-tuned large language model is trained using direct preference optimization based on the direct preference optimization dataset to obtain a direct preference optimized large language model. The information extraction guide adjusts the dataset to include the data labels for the standard answers; The process of determining direct preference optimization datasets based on the information extraction-guided dataset adjustment and the supervised fine-tuning of the large language model includes: The supervised fine-tuned large language model is configured with output sampling settings, including settings for the number of samplings and sampling temperature. Based on the information extraction guidance, the dataset is adjusted, and the output answer preference pairs of the supervised fine-tuning large language model are used as the online data preference pairs; the output answer preference pairs include preferred answers and unpreferred answers; the preferred answer is the answer with the highest BLEU score among the output answers of the supervised fine-tuning large language model; the unpreferred answer is the answer with the lowest BLEU score among the output answers of the supervised fine-tuning large language model. The offline data preference pair is determined based on the output answer preference pair; the offline data preference pair includes the standard answer and the non-preference answer.

2. The method for aligning large language models for information extraction tasks according to claim 1, characterized in that, The preset information extraction data includes closed-loop information extraction data, open-loop information extraction data, and on-demand information extraction data; the preset information extraction data uses a mixed example ratio strategy to set the data volume threshold.

3. The method for aligning large language models for information extraction tasks according to claim 1, characterized in that, The instruction manual includes a task description section, a pattern description section, and an output format description section; the input text is the text of the preset large language model for the information extraction task; The task description section is used to determine the guiding task description; the guiding task description is a description that combines the manual task description and the generated task description to iteratively generate a new task description, and the new task description needs to be verified by the task description; the generated task description is a task description that is automatically generated by a preset generative converter based on the manual task description. The pattern description section is used to determine the guiding pattern description; the guiding pattern description is a description after pattern description enhancement processing; the pattern description enhancement processing is a combination of one or more of the following processing: pattern shuffling, pattern sampling, integration of guidelines, and symbol replacement categories; The output format description section is used to determine the guiding output format description; the guiding output format description includes a triplet format description, a JSON format description, and a natural language format description.

4. The method for aligning large language models for information extraction tasks according to claim 3, characterized in that, The instructions also include step-by-step explanations of the requirements and / or sample examples; The step-by-step explanation requirement means that the preset large language model performs a step-by-step explanation before outputting the answer to the input text; The sample examples include input text examples, step-by-step explanation examples, and output answer examples; The step-by-step explanation example corresponds to the step-by-step explanation requirements.

5. The method for aligning large language models for information extraction tasks according to claim 4, characterized in that, The answer output includes an output format corresponding to the output format description section; The answer output also includes step-by-step explanation output corresponding to the step-by-step explanation requirement.

6. A large language model alignment system for information extraction tasks, characterized in that, include: A dataset determination module is used to determine the information extraction guidance adjustment dataset; the information extraction guidance adjustment dataset includes diverse inputs and answer outputs; The diverse inputs are aligned with a large language model on the information extraction task based on preset information extraction data; the answer output corresponds to the format requirements of the diverse inputs; the diverse inputs include instructions and input text; The model training module is used to perform supervised fine-tuning training on the preset large language model based on the information extraction guidance adjustment dataset and the preset general alignment corpus to obtain a supervised fine-tuned large language model; the supervised fine-tuned large language model is a large language model aligned on the information extraction task. The preset general alignment corpus is a dataset used to guide the adjustment of the general performance of large models; After the supervised fine-tuning training of the preset large language model based on the information extraction-guided adjustment dataset and the preset general alignment corpus, to obtain the supervised fine-tuned large language model, the process further includes: Based on the information extraction guidance of the adjusted dataset and the supervised fine-tuning of the large language model, a direct preference optimization dataset is determined; the direct preference optimization dataset includes online data preference pairs and offline data preference pairs. The supervised fine-tuned large language model is trained using direct preference optimization based on the direct preference optimization dataset to obtain a direct preference optimized large language model. The information extraction guide adjusts the dataset to include the data labels for the standard answers; The process of determining direct preference optimization datasets based on the information extraction-guided dataset adjustment and the supervised fine-tuning of the large language model includes: The supervised fine-tuned large language model is configured with output sampling settings, including settings for the number of samplings and sampling temperature. Based on the information extraction guidance, the dataset is adjusted, and the output answer preference pairs of the supervised fine-tuning large language model are used as the online data preference pairs; the output answer preference pairs include preferred answers and unpreferred answers; the preferred answer is the answer with the highest BLEU score among the output answers of the supervised fine-tuning large language model; the unpreferred answer is the answer with the lowest BLEU score among the output answers of the supervised fine-tuning large language model. The offline data preference pair is determined based on the output answer preference pair; the offline data preference pair includes the standard answer and the non-preference answer.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the large language model alignment method for information extraction tasks as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the large language model alignment method for information extraction tasks as described in any one of claims 1 to 5.

Images