Large language model editing method and device, electronic equipment and storage medium

Abstract

The invention provides a large language model editing method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence, and the method comprises the steps: carrying out the initial training of a to-be-edited large language model through a first question and answer set of to-be-updated knowledge, and obtaining a first model loss, editing model parameters of the to-be-edited large language model based on the first model loss; performing self-supervision retraining on the to-be-edited large language model by utilizing the knowledge document set of the to-be-updated knowledge to obtain second model loss, and performing fine adjustment on the model parameters based on the second model loss; performing structure enhancement training on the to-be-edited large language model by using the second question and answer set of the to-be-updated knowledge to obtain third model loss, and performing secondary fine tuning on the model parameters based on the third model loss; the knowledge document set and the second question and answer set are determined based on the context structure of the knowledge to be updated. According to the method, a knowledge-structure collaborative co-evolution mechanism is adopted, so that the flexibility and portability of accessing new knowledge of different structures by the edited large language model are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, and storage medium for editing large language models. Background Technology

[0002] Large Language Models (LLMs) have greatly promoted the widespread application of various Natural Language Processing (NLP) tasks such as intelligent assistants, automatic translation, and text generation. However, while LLMs empower NLP tasks, their massive number of parameters also brings great difficulties to parameter updates.

[0003] Currently, the main methods for updating the parameters of an LLM include Parameter Efficient Fine-Tuning (PEFT) and Knowledge Editing (KE).

[0004] However, while PEFT and knowledge editing perform well in terms of efficient knowledge injection, they still have technical problems in adaptability to new query structures. That is, when dealing with new query structures that are different from the original problem in terms of expression or query structure, the large language model cannot flexibly access the new knowledge after editing. At the same time, it is insufficient in terms of the portability index of the edited model's ability to perform aliases, composition, reasoning, and logical generalization. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and storage medium for editing large language models, which addresses the shortcomings of existing technologies in the adaptability of edited large language models to new query structures, and realizes a large language model editing scheme that enables the co-evolution of knowledge and structure of new knowledge.

[0006] This invention provides a method for editing large language models, comprising: The first question-and-answer set of knowledge to be updated is used to initially train the large language model to be edited, to obtain the first model loss, and the model parameters of the large language model to be edited are edited based on the first model loss; The large language model to be edited is self-supervised and retrained using the knowledge document set of the knowledge to be updated to obtain a second model loss, and the model parameters are fine-tuned based on the second model loss. The second question-and-answer set of the knowledge to be updated is used to perform structural enhancement training on the large language model to be edited, to obtain the third model loss, and the model parameters are fine-tuned a second time based on the third model loss; The knowledge document set and the second question-and-answer set are determined based on the contextual structure of the knowledge to be updated.

[0007] According to a method for editing a large language model provided by the present invention, before initially training the large language model to be edited using a first question-and-answer set of knowledge to be updated to obtain a first model loss, and editing the model parameters of the large language model to be edited based on the first model loss, the method further includes: The triple combination and knowledge graph subgraph of the knowledge to be updated are determined; the triple combination and the knowledge graph subgraph are input into the generative model to obtain the knowledge document set and the second question-answer set output by the generative model; the generative model is implemented based on a large language model.

[0008] According to a large language model editing method provided by the present invention, the second question-answer set is a multi-hop reasoning question-answer set, which includes question-answer pairs with at least two different knowledge structures.

[0009] According to a method for editing a large language model provided by the present invention, the step of editing the model parameters of the large language model to be edited based on the first model loss includes: Based on the low-rank decomposition parameter fine-tuning technique and the first model loss, the first target parameter and the first parameter update amount of the large language model to be edited are determined; based on the first parameter update amount, the first target parameter is updated. The fine-tuning of the model parameters based on the second model loss includes: Based on the low-rank decomposition parameter fine-tuning technique and the second model loss, the second target parameter and the second parameter update amount of the large language model to be edited are determined; based on the second parameter update amount, the second target parameter is fine-tuned. The second fine-tuning of the model parameters based on the third model loss includes: Based on the low-rank decomposition parameter fine-tuning technique and the third model loss, the third target parameter and the third parameter update amount of the large language model to be edited are determined; based on the third parameter update amount, the third target parameter is fine-tuned a second time.

[0010] According to a large language model editing method provided by the present invention, the first question-answer set includes multiple first question-answer pairs; the second question-answer set includes multiple second question-answer pairs; the first question-answer pair includes a first question and a first answer corresponding to the first question; the second question-answer pair includes a second question and a second answer corresponding to the second question; the first model loss is determined based on the cross-entropy loss between the predicted word sequence of the first question and the real word sequence corresponding to the first answer; the third model loss is determined based on the cross-entropy loss between the predicted word sequence of the second question and the real word sequence corresponding to the second answer.

[0011] According to a large language model editing method provided by the present invention, the knowledge document set includes multiple knowledge documents; the second model loss is determined based on the negative log-likelihood loss between the predicted word sequence of the knowledge document and the corresponding true word sequence of the knowledge document.

[0012] The present invention also provides a large language model editing device, comprising: The initial editing module is used to perform initial training on the large language model to be edited using the first question-and-answer set of knowledge to be updated, obtain the first model loss, and edit the model parameters of the large language model to be edited based on the first model loss; The first fine-tuning module is used to perform self-supervised retraining on the large language model to be edited using the knowledge document set of the knowledge to be updated, to obtain the second model loss, and to fine-tune the model parameters based on the second model loss; The second fine-tuning module is used to perform structural enhancement training on the large language model to be edited using the second question-answer set of the knowledge to be updated, to obtain the third model loss, and to perform secondary fine-tuning on the model parameters based on the third model loss; The knowledge document set and the second question-and-answer set are determined based on the contextual structure of the knowledge to be updated.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the large language model editing method as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the large language model editing method as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the large language model editing method as described above.

[0016] The large language model editing method, apparatus, electronic device, and storage medium provided by this invention employ a three-step knowledge-structure co-evolution mechanism—model parameter update, fine-tuning, and secondary fine-tuning—as a data-enhanced knowledge editing strategy. First, it utilizes a general question-and-answer set of new knowledge to enable the large language model to answer questions about new knowledge. Then, it uses a knowledge document set including the context structure of new knowledge to achieve self-supervised knowledge injection into the large language model. Finally, it uses a question-and-answer set including the context structure of new knowledge to enhance the absorption and application of updated knowledge by the large language model to be edited. This solves the problem of insufficient adaptability of the edited large language model to new query structures, greatly improving the flexibility and portability of the edited large language model in accessing new knowledge with different structures, and achieving a more accurate, stable, and logical large language model editing scheme. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the large language model editing method provided by the present invention.

[0019] Figure 2 This is an example diagram of the subgraph structure of the single-hop reasoning knowledge structure provided by the present invention.

[0020] Figure 3 This is an example diagram of the subgraph structure of the double-hop reasoning knowledge structure provided by the present invention.

[0021] Figure 4 This is an example diagram of the subgraph structure of the intersection reasoning knowledge structure provided by the present invention.

[0022] Figure 5 This is a schematic diagram of the structure of the large language model editing device provided by the present invention.

[0023] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] It should be noted that, in the description of this invention, the term "comprising" or any other variations thereof is intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Those skilled in the art will understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0026] The terms "first," "second," etc., used in this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more.

[0027] The following is combined Figures 1-6 This invention describes the large language model editing method, apparatus, electronic device, and storage medium provided by the present invention.

[0028] In the field of natural language processing, the introduction of deep learning has transformed traditional rule-based and statistical processing methods. Convolutional Neural Networks (CNNs) and Long Short-Term Memory Networks (LSTMs) have achieved significant breakthroughs in tasks such as speech recognition and sentiment analysis. Pre-trained Language Models (PLMs) based on the Transformer architecture, such as BERT, GPT, and T5, have also made significant progress in NLP tasks. The emergence of a series of large language models, such as ChatGPT, LLaMA, Qwen, and Baichuan, has greatly promoted the widespread application of various tasks such as intelligent assistants, automatic translation, and text generation, profoundly influencing the research direction of academia and driving technological innovation in big data processing and natural language understanding in industry.

[0029] However, while large language models empower NLP tasks, their massive number of parameters also brings great difficulties to parameter updates, leading to the emergence of technologies such as efficient parameter fine-tuning and knowledge editing.

[0030] Low-rank adaptation (LoRA) in efficient parameter fine-tuning significantly reduces the number of parameter updates by decomposing the parameter matrix into the product of two low-rank matrices, thereby reducing computational complexity. AdaLoRA introduces an adaptive selection mechanism on top of LoRA, which can automatically select the parameters that need to be updated based on importance, further improving the efficiency of parameter updates. QLoRA further compresses parameter storage space through quantization techniques, reducing memory usage and making parameter updates more efficient.

[0031] In the field of knowledge editing, there are also two main paradigms: knowledge editing methods based on external intervention and those based on internal updates.

[0032] External intervention-based methods employ additional memory to update knowledge. For example, the SERAC method uses a "range classifier" to determine if a user query falls within the scope of "re-edited" facts; if so, it references the response of a smaller proxy model to assign higher weight to the "correct answer" in memory. The IKE method constructs a domain knowledge base, retrieves the most relevant knowledge as context based on the user query, and then leverages the context learning capabilities of a larger model to generate reliable output based on knowledge hints. Other methods use additional parameters as extra memory to update knowledge, such as the T-patcher method adding neuron patches for erroneous knowledge in the last layer of the feedforward network, the CaliNet method adding similar parameter layers to the model's feedforward layer, and the GRACE method using discrete codebooks as adapters to maintain and update knowledge.

[0033] Internal update-based methods focus on updating the model parameter matrix to achieve editing, with the "locate first, then edit" approach being a classic example. Specifically, "knowledge attribution" technology locates "knowledge neurons," identifies key influencing parameters, and updates them; the ROME method uses causal mediation analysis to locate the editing region and update the MLP parameter matrix, with the improved MEMIT scheme enabling batch editing; the PMET method further incorporates an attention mechanism for optimization; meanwhile, meta-learning methods are also widely used in knowledge editing. The KnowledgeEditor method uses a bidirectional LSTM model to predict weight updates, while the MEND method proposes gradient decomposition of the knowledge editor network to improve editing efficiency and performance.

[0034] Although PEFT and knowledge editing perform well in efficient knowledge injection, the following problems still exist: (1) When dealing with new query structures that are different from the original problem in terms of expression or query structure, the large language model cannot flexibly access the new knowledge after editing; (2) The edited model can effectively recall the edited facts, but it does not perform well in dealing with multi-hop problems; (3) It is insufficient in terms of the portability index of the edited model in terms of aliasing, composition and reasoning and logical generalization.

[0035] In view of this, the present invention provides a large language model editing method, apparatus, electronic device and storage medium to solve at least one of the above-mentioned technical problems.

[0036] Figure 1 This is a flowchart illustrating the large language model editing method provided by the present invention, as shown below. Figure 1 As shown, the large language model editing method includes, but is not limited to, steps 101 to 103.

[0037] It should be noted that the execution subject of the large language model editing method provided by the present invention is the corresponding large language model editing device, which can be a server, computer equipment, such as mobile phone, tablet computer, laptop computer, handheld computer, vehicle electronic equipment, wearable device, ultra-mobile personal computer (UMPC), netbook or personal digital assistant (PDA), etc.

[0038] Step 101: Initially train the large language model to be edited using the first question-and-answer set of knowledge to be updated, obtain the first model loss, and edit the model parameters of the large language model to be edited based on the first model loss.

[0039] The first question-and-answer set is a set of general questions and answers determined using the knowledge to be updated. It includes multiple first question-and-answer pairs, and each first question-and-answer pair includes the first question and the first answer corresponding to the first question.

[0040] For example, the knowledge to be updated is input into a large language model, which then generates the first question-and-answer set.

[0041] Specifically, in the knowledge-structure co-evolution mechanism (EvoEdit) for new knowledge, the first step is to use the first question-and-answer set of the knowledge to be updated to perform initial training on the large language model to be edited, obtain the first model loss, and edit the model parameters of the large language model to be edited based on the first model loss. In this way, the model parameters of the large language model are updated by using the general question answers to update the data, giving the language model the ability to answer questions.

[0042] Step 102: Use the knowledge document set of the knowledge to be updated to perform self-supervised retraining on the large language model to be edited, obtain the second model loss, and fine-tune the model parameters based on the second model loss.

[0043] The knowledge document set is determined based on the contextual structure of the knowledge to be updated and includes multiple knowledge documents.

[0044] For example, the contextual structure of the knowledge to be updated is input into a large language model, which then generates an output knowledge document set.

[0045] Specifically, in the knowledge-structure co-evolution mechanism of new knowledge, the second step is to use the knowledge document set of the knowledge to be updated to perform self-supervised retraining on the initially trained large language model to be edited, obtain the second model loss, and fine-tune the model parameters of the large language model to be edited based on the second model loss, injecting new knowledge into the large language model to achieve self-supervised knowledge injection.

[0046] Step 103: Use the second question-and-answer set of the knowledge to be updated to perform structural enhancement training on the large language model to be edited, obtain the third model loss, and perform secondary fine-tuning of the model parameters based on the third model loss.

[0047] The second question-and-answer set is determined based on the contextual structure of the knowledge to be updated. It is a general question-and-answer set determined using the contextual structure of the knowledge to be updated, and includes multiple second question-and-answer pairs. Each second question-and-answer pair includes a second question and a second answer corresponding to the second question.

[0048] For example, the contextual structure of the knowledge to be updated is input into a large language model, which then generates and outputs a second question-and-answer set.

[0049] Specifically, in the knowledge-structure co-evolution mechanism of new knowledge, the third step is to use the second question-and-answer set of the knowledge to be updated to perform structure enhancement training on the self-supervised retrained large language model to be edited, obtain the third model loss, and then perform secondary fine-tuning of the model parameters of the large language model to be edited based on the third model loss, so as to enhance the absorption and application of updated knowledge by the large language model to be edited.

[0050] After three rounds of editing, the large language model has mastered the new knowledge and its contextual structure. This overall update process is called "knowledge coevolution." The new knowledge and its contextual structure mastered by the edited large language model "co-evolve" so that the large language model can adapt to the complex problems of new knowledge and its contextual structure.

[0051] The large language model editing method provided by this invention employs a three-step knowledge-structure co-evolution mechanism—model parameter update, fine-tuning, and secondary fine-tuning—as a data-augmented knowledge editing strategy. First, it utilizes a general question-and-answer set of new knowledge to enable the large language model to answer questions about new knowledge. Then, it uses a knowledge document set including the context structure of new knowledge to achieve self-supervised knowledge injection into the large language model. Finally, it uses a question-and-answer set including the context structure of new knowledge to enhance the absorption and application of updated knowledge by the large language model to be edited. This solves the problem of insufficient adaptability of the edited large language model to new query structures, greatly improving the flexibility and portability of the edited large language model in accessing new knowledge with different structures, and achieving a more accurate, stable, and logical large language model editing scheme.

[0052] Based on the above embodiments, as an optional embodiment, the large language model editing method further includes, before performing initial training on the large language model to be edited using the first question-and-answer set of knowledge to be updated to obtain a first model loss, and editing the model parameters of the large language model to be edited based on the first model loss: Determine the triplet combinations and knowledge graph subgraphs of the knowledge to be updated; The triple combination and the knowledge graph subgraph are input into the generative model to obtain the knowledge document set and the second question-and-answer set output by the generative model.

[0053] The generative model is implemented based on a large language model.

[0054] Specifically, before using new knowledge to edit the large language model, a knowledge document set and a second question-and-answer set representing the contextual structure of the new knowledge are first obtained. Specifically, triple combinations are extracted and expanded from the knowledge to be updated. And use triple combinations to construct a knowledge graph subgraph of the knowledge to be updated. ,in, It is the head entity in the triple. A collection of directly related entities It is a set of association relationships.

[0055] Further combine the triplet and knowledge graph subgraph The input is fed into a generative model based on a large language model (such as GPT-4), and the result is a set of knowledge documents related to the subgraph structure output by the generative model. Second Question and Answer Set .

[0056] Taking the updating of knowledge about the "capital of France" as an example, the knowledge to be updated about the "capital of France" is extracted and expanded from the triple combinations such as "France → capital → Paris" and "Paris → population → 2.14 million" and the corresponding knowledge graph subgraphs. The large language model generates a knowledge document set related to the subgraph structure, such as "Paris, as the capital of France, is an important political center in Europe with 2.14 million residents...".

[0057] In one embodiment, the triple combination and the knowledge graph subgraph are input into the generative model, and the expression for the knowledge document set output by the generative model is as follows: ; in, for; For generating models; A combination of triplets representing knowledge to be updated; This is a subgraph of the knowledge graph that needs to be updated.

[0058] After three rounds of editing, the large language model has mastered the triple combinations of new knowledge and their contextual structure. This overall update process is called "knowledge coevolution." The edited large language model "co-evolves" the triple combinations of new knowledge and their contextual structure so that the large language model can adapt to the complex problems of triple combinations of new knowledge and their contextual structures.

[0059] The large language model editing method provided by this invention obtains a knowledge document set and a question-and-answer set that include the contextual structure of the new knowledge by using the combination of triples of the knowledge to be updated and the knowledge graph subgraph. It then uses the knowledge document set and question-and-answer set that include the contextual structure of the new knowledge to realize the large language model editing of the knowledge-structure co-evolutionary training mechanism. At the same time, it considers the contextual structure of the new knowledge, thereby enhancing the understanding of the new knowledge context by the edited model, enabling the edited model to better absorb, access and adapt to the edited new knowledge and its context.

[0060] Based on the above embodiments, as an optional embodiment, in the large language model editing method, the second question-answer set is a multi-hop reasoning question-answer set, which includes question-answer pairs with at least two different knowledge structures.

[0061] The different knowledge structures include, but are not limited to, at least two of the following knowledge structures: single-hop reasoning (1-hop, 1p), intersection reasoning (2-intersection, 2i), double-hop reasoning (2-hop, 2p), and multi-hop reasoning; correspondingly, the multi-hop reasoning question-and-answer set includes, but is not limited to, at least two of the following question-and-answer pairs: single-hop reasoning question-and-answer pairs, intersection reasoning question-and-answer pairs, double-hop reasoning question-and-answer pairs, and multi-hop reasoning question-and-answer pairs.

[0062] Specifically, the second question-and-answer set obtained by inputting the triplet combination and the knowledge graph subgraph into the large language model is a set of multiple reasoning question-and-answer pairs, including but not limited to at least two of the following question-and-answer pairs: single-hop reasoning question-and-answer pairs, intersection reasoning question-and-answer pairs, double-hop reasoning question-and-answer pairs, and multi-hop reasoning question-and-answer pairs, which correspond to knowledge structures such as single-hop reasoning, intersection reasoning, double-hop reasoning, and multi-hop reasoning, respectively, and there is no limit to the number of each type of question-and-answer pair.

[0063] Figure 2 This is an example diagram of the subgraph structure of the single-hop reasoning knowledge structure provided by the present invention, such as... Figure 2 As shown, the head entity of the subgraph structure of the single-hop reasoning knowledge structure is "France", the tail entity is "Paris", and the association is "HasCapital". Therefore, the question in the single-hop reasoning question-answer pair generated by this subgraph structure is "What is the capital city of France?", and the answer is "Paris".

[0064] Figure 3 This is an example diagram of the subgraph structure of the double-hop reasoning knowledge structure provided by the present invention, such as... Figure 3 As shown, the subgraph structure of the double-jump reasoning knowledge structure includes the head entity "Pulitzer Prize for Fiction" and "The Netanyahus". "The Netanyahus" is the tail entity relative to the head entity "Pulitzer Prize for Fiction". The tail entity relative to the head entity "The Netanyahus" is "Joshua Cohen". The relationship between the head entity "Pulitzer Prize for Fiction" and the tail entity "The Netanyahus" is "Win". The relationship between the head entity "The Netanyahus" and the tail entity "Joshua Cohen" is "Joshua Cohen".

[0065] Therefore, the question in the double-hop reasoning question-and-answer pair generated by the subgraph structure of this double-hop reasoning knowledge structure is "Who is the author of the book that won the Pulitzer Prize for Fiction?", and the answer is "Joshua Cohen".

[0066] Figure 4 This is an example diagram of the subgraph structure of the intersection reasoning knowledge structure provided by the present invention, such as... Figure 4As shown, the subgraph structure of the intersection reasoning knowledge structure includes the head entity "New York Times Best Seller", the head entity "Man Booker Prize", and the tail entity "Life of Pi". The relationship between the head entity "New York Times Best Seller" and the tail entity "Life of Pi" is "win", and the relationship between the head entity "Man Booker Prize" and the tail entity "Life of Pi" is "win".

[0067] Therefore, the question in the intersection reasoning question-answer pair generated by the subgraph structure of this intersection reasoning knowledge structure is "What are the books that have been on The New York Times Best Seller list and also won the Man Booker Prize?", and the answer is "Life of Pi".

[0068] Taking the aforementioned update of the knowledge about the "capital of France" as an example, after inputting the triple combination and knowledge graph subgraph into the generative model, the generated multi-hop reasoning question-and-answer set includes single-hop reasoning question-and-answer pairs, intersection reasoning question-and-answer pairs, and double-hop reasoning question-and-answer pairs. Specifically, the question in the single-hop reasoning question-and-answer pair is "Q: What is the capital of France?", and the answer is "A: Paris."; the question in the intersection reasoning question-and-answer pair is "Q: Which cities are capitals and have over 2M population?", and the answer is "A: Paris."; the question in the double-hop reasoning question-and-answer pair is "What country's capital borders Germany?", and the answer is "A: Paris."

[0069] When existing knowledge editing models edit models for new knowledge problems, they usually only examine the same or equivalent problems.

[0070] The large language model editing method provided by this invention generates a multi-hop reasoning question-and-answer set covering different knowledge structure patterns such as single-hop reasoning, intersection reasoning, double-hop reasoning, and multi-hop reasoning. This multi-hop reasoning question-and-answer set, which includes the contextual structure of new knowledge, enhances the absorption and application of updated knowledge by the large language model to be edited. As a result, the edited large language model performs better in handling multi-hop problems, solving the problem of insufficient adaptability of the edited large language model to new multi-hop query structures. This improves the adaptability of the edited model to new knowledge and its contextual structure, as well as its overall performance.

[0071] Based on the above embodiments, as an optional embodiment, in the large language model editing method, the step of editing the model parameters of the large language model to be edited based on the first model loss includes: Based on the Low-Rank Adaptation (LoRA) technique and the first model loss, the first target parameter and the first parameter update amount of the large language model to be edited are determined. The first target parameter is updated based on the first parameter update amount; The fine-tuning of the model parameters based on the second model loss includes: Based on LoRA technology and the second model loss, the second objective parameter and the second parameter update amount of the large language model to be edited are determined; Based on the update amount of the second parameter, the second target parameter is fine-tuned; The second fine-tuning of the model parameters based on the third model loss includes: Based on LoRA technology and the third model loss, the third objective parameter and the third parameter update amount of the large language model to be edited are determined; Based on the update amount of the third parameter, the third target parameter is fine-tuned a second time.

[0072] Among them, the first target parameter and the first parameter update amount are the model parameters that need to be updated and their corresponding parameter update amounts during the initial training phase; the second target parameter and the second parameter update amount are the model parameters that need to be updated and their corresponding parameter update amounts during the self-supervised retraining phase; and the third target parameter and the third parameter update amount are the model parameters that need to be updated and their corresponding parameter update amounts during the structure enhancement training phase.

[0073] Specifically, in each of the three stages of the knowledge-structure co-evolution mechanism of new knowledge, the LoRA technique is used to decompose the original parameter matrix into the product of two low-rank matrices to reduce the number of parameters to be updated and achieve efficient parameter updates.

[0074] After obtaining the first model loss by initial training of the large language model to be edited using the first question-answer set, the first target parameter and the first parameter update amount are determined based on LoRA technology and the first model loss. The first target parameter is then updated using the first parameter update amount, thus completing the update of the model parameters of the large language model to be edited in the initial training stage and giving the large language model to be edited the ability to answer questions.

[0075] After further utilizing the knowledge document set to perform self-supervised retraining on the large language model to be edited to obtain the second model loss, the second target parameter and the second parameter update amount are determined based on LoRA technology and the second model loss. The second target parameter is then fine-tuned using the second parameter update amount, completing the parameter fine-tuning of the large language model to be edited in the self-supervised retraining stage, and realizing the self-supervised knowledge injection of the large language model to be edited.

[0076] Furthermore, after using the second question-and-answer set to perform structural enhancement training on the large language model to be edited to obtain the third model loss, the third target parameter and the third parameter update amount are determined based on LoRA technology and the third model loss. The third target parameter is then fine-tuned using the third parameter update amount, completing the second fine-tuning of the parameters of the large language model to be edited in the structural enhancement training stage, thereby enhancing the absorption and application of new knowledge by the large language model to be edited.

[0077] In one embodiment, the expression for efficient parameter updating of a large language model based on LoRA technology is as follows: ; ; in, To represent the amount of model parameter updates for the large language model to be edited; , It is a low-rank fitting matrix, and ; These are the original model parameters for the large language model to be edited; These are the updated model parameters for the large language model to be edited.

[0078] The large language model editing method provided by this invention uses LoRA technology and model loss in all three stages of the knowledge-structure co-evolution mechanism to update and fine-tune the model parameters of the large language model, which can reduce the number of parameters updated and achieve efficient parameter updates.

[0079] Based on the above embodiments, as an optional embodiment, in the large language model editing method, the first question-answer set includes multiple first question-answer pairs; the second question-answer set includes multiple second question-answer pairs. The first question-and-answer pair includes a first question and a first answer corresponding to the first question; the second question-and-answer pair includes a second question and a second answer corresponding to the second question. The first model loss is determined based on the cross-entropy loss between the predicted word sequence of the first question and the true word sequence corresponding to the first answer; The third model loss is determined based on the cross-entropy loss between the predicted word sequence of the second question and the actual word sequence corresponding to the second answer.

[0080] In one specific embodiment, the first model loss is determined based on the following expression: ; in, The loss is the first model loss; This is the first set of problems; The number of correct answers to the first question in the first question set; This is the first question and answer pair in the first question set; This represents the number of lexical units in the actual lexical sequence corresponding to the first answer in the first question-answer pair. For the large language model to be edited in the given first problem And the first answer The former Given the nth word, predict the nth word. The probability distribution of each word element; The first word in the real word sequence corresponding to the first answer Each word element; for; The cross-entropy loss is calculated between the predicted word sequence and the actual word sequence corresponding to the first answer.

[0081] In one specific embodiment, the third model loss is determined based on the following expression: ; in, The loss is for the third model; For the second set of problems; for; This is a second question-and-answer pair within the second question set; This refers to the number of lexical units in the actual lexical sequence corresponding to the second answer in the second question-and-answer pair. For the large language model to be edited in the given second problem Second answer The former Predicting the first word under the condition of lexical units. The probability distribution of each word element; The first word in the real word sequence corresponding to the second answer Each word element; The cross-entropy loss is calculated between the predicted word sequence and the actual word sequence corresponding to the second answer.

[0082] The large language model editing method provided by this invention determines the first model loss and the third model loss based on the cross-entropy loss between the predicted word sequence and the real word sequence of the question-answer pair, and updates and fine-tunes the model parameters based on the first model loss and the third model loss respectively. This can directly optimize the overall probability distribution of the word sequence and improve the accuracy of large language model editing.

[0083] Based on the above embodiments, as an optional embodiment, in the large language model editing method, the knowledge document set includes multiple knowledge documents; The second model loss is determined based on the negative log-likelihood loss between the predicted word sequence of the knowledge document and the corresponding real word sequence of the knowledge document.

[0084] Among them, the negative log-likelihood loss is used to maximize the probability that the model generates real text sequences during language model training.

[0085] In one specific embodiment, the second model loss is determined based on the following expression: ; in, The loss is for the second model; A collection of knowledge documents; For a knowledge document in a knowledge document set, there is a corresponding real word sequence; For knowledge documents The total number of lexical units in the text; For knowledge documents The corresponding real lexical sequence in the first Each word element; For before Predicting the first word under the condition of word number Each word element is The probability of; For knowledge documents Predicted word sequences and knowledge documents The negative log-likelihood loss between the corresponding real word sequences.

[0086] The large language model editing method provided by this invention determines the second model loss based on the negative log-likelihood loss between the predicted word sequence and the actual word sequence of the knowledge document, and fine-tunes the model parameters based on the second loss. This method better reflects the characteristics of long sequence dependencies in knowledge documents and can also directly optimize the overall probability distribution of word sequences, thereby improving the accuracy of large language model editing.

[0087] Existing knowledge editing models typically only examine the model on the same or equivalent problems when editing it for new knowledge problems. While portability metrics broaden the scope of examination, they still do not consider how well the edited model adapts to complex problems involving new knowledge triples and their contextual structures (e.g., subgraphs in a knowledge graph) after knowledge editing in response to external changes such as knowledge updates.

[0088] In other words, existing assessment methods are insufficient to reflect the extent to which a large language model, edited based on a knowledge-structure co-evolution mechanism, has mastered the edited knowledge.

[0089] Therefore, by introducing a logical consistency verification metric for the first time, the following provides a multi-dimensional evaluation framework based on direct accuracy and logical consistency to evaluate the large language model editing method provided by this invention.

[0090] Specifically, the expression for the multi-dimensional evaluation framework based on direct accuracy and logical consistency is as follows: ; in, A direct accuracy score, used to measure the accuracy of standard question and answer responses; Scoring logical consistency is a constraint verification based on logical implication.

[0091] The expression for direct accuracy scoring is as follows: ; in, For direct accuracy scoring; To assess the total number of problems; This is an indicator function; its value is 1 when the internal condition is true, and 0 otherwise. For a large language model to be edited in a given problem Under the given conditions, the answer with the highest probability of being generated; For a given problem, the standard correct question is...

[0092] The expression for logical consistency scoring is as follows: ; in, Scoring for logical consistency; This represents the total number of logical implication constraints. For large language models to be edited, given only the problem When answering the correct answer The probability of; To edit a large language model while simultaneously providing a problem Knowledge graph subgraphs of knowledge to be updated When given context, answer the correct answer. The probability of.

[0093] Based on the logical consistency score described in the aforementioned expression, the stability and logical consistency of the model's responses after knowledge injection can be measured.

[0094] Table 1 shows the experimental results of direct accuracy provided by this invention, and Table 2 shows the experimental results of logical consistency provided by this invention. Based on the large language model editing method provided by this invention, LoRA (r=8) was applied to the large language model to be edited on LLaMA-7B for 2 hours (single card A100) for fine-tuning and joint training. The results were evaluated using a multi-dimensional evaluation framework based on direct accuracy and logical consistency. As shown in Table 1, through knowledge subgraph enhancement, compared with MEMIT and MEND, the knowledge structure co-evolution mechanism (EvoEdit) significantly enhances the direct accuracy of knowledge editing. As shown in Table 2, when a subgraph structure is incorporated into knowledge editing, true knowledge injection is achieved. LLM enhances its understanding of injected knowledge as the amount of knowledge increases. EvoEdit contrasts sharply with the obvious shortcomings of traditional knowledge editing methods MEMIT and MEND.

[0095] Table 1. Results of Direct Accuracy Experiment

[0096] Table 2. Results of Logical Consistency Experiment

[0097] Overall, the large language model editing method provided in this invention proposes a knowledge injection framework based on the subgraph structure of a knowledge graph. It achieves the co-evolution of knowledge triples and contextual structures through a three-stage process. When designing training data, it considers the knowledge to be updated, its subgraph structure, and the rewritten knowledge simultaneously. It leverages existing mainstream closed-source large language models such as ChatGPT to fuse relevant knowledge components, generating knowledge documents and corresponding question-answer pairs for training. Training is performed on the knowledge documents and question-answer pairs to achieve updated knowledge injection. Finally, the effectiveness of knowledge injection is measured by the accuracy of answering questions on the question-answer pairs.

[0098] Figure 5 This is a schematic diagram of the structure of the large language model editing device provided by the present invention, as shown below. Figure 5As shown, the large language model editing device includes, but is not limited to, an initial editing module 501, a first fine-tuning module 502, and a second fine-tuning module 503.

[0099] The initial editing module 501 is used to perform initial training on the large language model to be edited using the first question-and-answer set of knowledge to be updated, obtain the first model loss, and edit the model parameters of the large language model to be edited based on the first model loss.

[0100] The first fine-tuning module 502 is used to perform self-supervised retraining on the large language model to be edited using the knowledge document set of the knowledge to be updated, to obtain the second model loss, and to fine-tune the model parameters based on the second model loss.

[0101] The second fine-tuning module 503 is used to perform structural enhancement training on the large language model to be edited using the second question-and-answer set of the knowledge to be updated, to obtain the third model loss, and to perform secondary fine-tuning on the model parameters based on the third model loss.

[0102] The knowledge document set and the second question-and-answer set are determined based on the contextual structure of the knowledge to be updated.

[0103] It should be noted that the large language model editing device provided by the present invention can execute the large language model editing method described in any of the above embodiments during specific operation, which will not be elaborated in this embodiment.

[0104] The large language model editing device provided by this invention employs a three-step knowledge-structure co-evolution mechanism—model parameter update, fine-tuning, and secondary fine-tuning—as a data-enhanced knowledge editing strategy. First, it utilizes a general question-and-answer set of new knowledge to enable the large language model to answer questions about new knowledge. Then, it uses a knowledge document set including the context structure of new knowledge to achieve self-supervised knowledge injection into the large language model. Finally, it uses a question-and-answer set including the context structure of new knowledge to enhance the absorption and application of updated knowledge by the large language model to be edited. This solves the problem of insufficient adaptability of the edited large language model to new query structures, greatly improving the flexibility and portability of the edited large language model in accessing new knowledge with different structures, and achieving a more accurate, stable, and logical large language model editing scheme.

[0105] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 6As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute the large language model editing method provided in any of the above embodiments. The large language model editing method includes, but is not limited to, the following steps: initial training of the large language model to be edited using a first question-and-answer set of knowledge to be updated to obtain a first model loss, and editing the model parameters of the large language model to be edited based on the first model loss; self-supervised retraining of the large language model to be edited using a knowledge document set of knowledge to be updated to obtain a second model loss, and fine-tuning the model parameters based on the second model loss; structural enhancement training of the large language model to be edited using a second question-and-answer set of knowledge to be updated to obtain a third model loss, and fine-tuning the model parameters a second time based on the third model loss; wherein the knowledge document set and the second question-and-answer set are determined based on the context structure of the knowledge to be updated.

[0106] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0107] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the large language model editing method provided in any of the above embodiments. The large language model editing method includes, but is not limited to, the following steps: initial training of the large language model to be edited using a first question-and-answer set of knowledge to be updated to obtain a first model loss, and editing the model parameters of the large language model to be edited based on the first model loss; self-supervised retraining of the large language model to be edited using a knowledge document set of the knowledge to be updated to obtain a second model loss, and fine-tuning the model parameters based on the second model loss; structural enhancement training of the large language model to be edited using a second question-and-answer set of the knowledge to be updated to obtain a third model loss, and fine-tuning the model parameters a second time based on the third model loss; wherein the knowledge document set and the second question-and-answer set are determined based on the contextual structure of the knowledge to be updated.

[0108] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the large language model editing method provided in any of the above embodiments. The large language model editing method includes, but is not limited to, the following steps: initial training of the large language model to be edited using a first question-and-answer set of knowledge to be updated to obtain a first model loss, and editing the model parameters of the large language model to be edited based on the first model loss; self-supervised retraining of the large language model to be edited using a knowledge document set of the knowledge to be updated to obtain a second model loss, and fine-tuning the model parameters based on the second model loss; structural enhancement training of the large language model to be edited using a second question-and-answer set of the knowledge to be updated to obtain a third model loss, and fine-tuning the model parameters a second time based on the third model loss; wherein the knowledge document set and the second question-and-answer set are determined based on the contextual structure of the knowledge to be updated.

[0109] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0110] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for editing large language models, characterized in that, include: The first question-and-answer set of knowledge to be updated is used to initially train the large language model to be edited, to obtain the first model loss, and the model parameters of the large language model to be edited are edited based on the first model loss; The large language model to be edited is self-supervised and retrained using the knowledge document set of the knowledge to be updated to obtain a second model loss, and the model parameters are fine-tuned based on the second model loss. The second question-and-answer set of the knowledge to be updated is used to perform structural enhancement training on the large language model to be edited, to obtain the third model loss, and the model parameters are fine-tuned a second time based on the third model loss; The knowledge document set and the second question-and-answer set are determined based on the contextual structure of the knowledge to be updated.

2. The large language model editing method according to claim 1, characterized in that, Before initially training the large language model to be edited using the first question-and-answer set of knowledge to be updated, obtaining the first model loss, and editing the model parameters of the large language model to be edited based on the first model loss, the method further includes: Determine the triplet combinations and knowledge graph subgraphs of the knowledge to be updated; The triple combination and the knowledge graph subgraph are input into the generative model to obtain the knowledge document set and the second question-answer set output by the generative model; the generative model is implemented based on a large language model.

3. The large language model editing method according to claim 2, characterized in that, include: The second question-and-answer set is a multi-hop reasoning question-and-answer set, which includes question-and-answer pairs with at least two different knowledge structures.

4. The large language model editing method according to claim 1, characterized in that, The step of editing the model parameters of the large language model to be edited based on the first model loss includes: Based on the low-rank decomposition parameter fine-tuning technique and the first model loss, the first target parameter and the first parameter update amount of the large language model to be edited are determined. The first target parameter is updated based on the first parameter update amount; The fine-tuning of the model parameters based on the second model loss includes: Based on the low-rank decomposition parameter fine-tuning technique and the second model loss, the second target parameter and the second parameter update amount of the large language model to be edited are determined. Based on the update amount of the second parameter, the second target parameter is fine-tuned; The second fine-tuning of the model parameters based on the third model loss includes: Based on the low-rank decomposition parameter fine-tuning technique and the third model loss, the third objective parameter and the third parameter update amount of the large language model to be edited are determined. Based on the update amount of the third parameter, the third target parameter is fine-tuned a second time.

5. The large language model editing method according to claim 1, characterized in that, The first question-and-answer set includes multiple first question-and-answer pairs; the second question-and-answer set includes multiple second question-and-answer pairs. The first question-and-answer pair includes a first question and a first answer corresponding to the first question; the second question-and-answer pair includes a second question and a second answer corresponding to the second question. The first model loss is determined based on the cross-entropy loss between the predicted word sequence of the first question and the actual word sequence corresponding to the first answer; The third model loss is determined based on the cross-entropy loss between the predicted word sequence of the second question and the actual word sequence corresponding to the second answer.

6. The large language model editing method according to claim 1, characterized in that, The knowledge document set includes multiple knowledge documents; The second model loss is determined based on the negative log-likelihood loss between the predicted lexical sequence of the knowledge document and the corresponding true lexical sequence of the knowledge document.

7. A large language model editing device, characterized in that, include: The initial editing module is used to perform initial training on the large language model to be edited using the first question-and-answer set of knowledge to be updated, obtain the first model loss, and edit the model parameters of the large language model to be edited based on the first model loss; The first fine-tuning module is used to perform self-supervised retraining on the large language model to be edited using the knowledge document set of the knowledge to be updated, to obtain the second model loss, and to fine-tune the model parameters based on the second model loss; The second fine-tuning module is used to perform structural enhancement training on the large language model to be edited using the second question-answer set of the knowledge to be updated, to obtain the third model loss, and to perform secondary fine-tuning on the model parameters based on the third model loss; The knowledge document set and the second question-and-answer set are determined based on the contextual structure of the knowledge to be updated.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the large language model editing method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the large language model editing method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the large language model editing method as described in any one of claims 1 to 6.

Images