Model editing method and device
By determining the enhanced representation of query statements to be edited and the knowledge database in a large-scale language model, editing with input embedding representations, and defining multiple loss functions, the problem of capability degradation in the model editing process in the existing technology is solved, and lifelong Model editing and performance retention.
Patent Information
- Application Number
- CN202411055531.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-01
AI Technical Summary
In the prior art, large-scale language models are prone to deterioration in the editing process, resulting in forgetfulness and hallucination phenomena, and cannot effectively deal with long-sequence knowledge editing and multiple-sequence sequence editing.
By determining three sets of query statements to be edited, searching the knowledge database, generating enhanced representations and prompt character representations, editing large-scale language models with input embedding representations, defining reliability, universality and generalization loss functions, and performing multiple edits Without changing the internal parameters of the model, lifelong model editing is realized.
It effectively reduces the problem of ability degradation of large-scale language models during editing, maintains the general performance and generalization capabilities of the model, and avoids the occurrence of forgetting phenomena.
Smart Images

Figure CN118916378A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and more specifically, to a model editing method and device. Background Art
[0002] With the development of science and technology, the application of intelligent question-answering systems is becoming more and more extensive. Specifically, after a user asks a question to the intelligent question-answering system, the intelligent question-answering system will give an answer to the question raised by the user. The above question-answering process can be implemented through large-scale language models LLMs, which have the ability to understand, analyze and generate text; the above LLMs have an important disadvantage, that is, due to their large number of parameters, the computational cost of pre-training is huge. When new knowledge needs to be introduced into the pre-trained LLMs, considering the overhead of computing resources and data resources, training cannot be started from scratch. Therefore, it is necessary to accurately modify specific knowledge parameters in LLMs when introducing new knowledge.
[0003] In the prior art, the serialized model editing SME method is used to edit LLM, but the above SME method is only applicable to single editing and simple knowledge editing. The above SME method can only process a small amount of data and still faces great challenges for long sequence knowledge editing; and in scenarios involving sequence editing of multiple pieces of knowledge, only multiple edits can be performed. Multiple edits will cause the general ability of LLMs to degrade during the editing process, resulting in forgetting problems, which will in turn affect other parameters in LLMs during the editing process, causing the above LLMs to produce hallucinations.
[0004] In summary, how to reduce the ability degradation of LLMs during the editing process is a problem that needs to be solved at present. Summary of the invention
[0005] In view of this, an embodiment of the present invention provides a method and apparatus for model editing, which can realize lifelong model editing and reduce the problem of capability degradation of LLMs during the editing process.
[0006] In a first aspect, an embodiment of the present invention provides a method for editing a model, the method comprising:
[0007] Determine three groups of query statements to be edited, wherein each group of query statements to be edited includes at least one query statement to be edited;
[0008] According to each group of query statements to be edited, at least one piece of knowledge data corresponding to the query statement is retrieved from the knowledge database;
[0009] Determine respectively each group of query statements to be edited and the enhanced representation of at least one piece of knowledge data corresponding thereto;
[0010] Determine, according to the enhanced representation, a prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited;
[0011] Determine an input embedding representation corresponding to each query statement to be edited in each group of query statements to be edited;
[0012] Editing the large-scale language model according to a combination of the prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited and the input embedding representation, generating an updated large-scale language model, and generating at least one semantic vector;
[0013] Determine a loss function according to the semantic vectors corresponding to the three groups of query sentences to be edited and the enhanced representation;
[0014] The updated large-scale language model is edited according to the loss function to generate a target large-scale language model.
[0015] Optionally, the retrieving at least one piece of knowledge data corresponding to each group of query statements to be edited from a knowledge database specifically includes:
[0016] In the knowledge database, at least one knowledge spectrum graph corresponding to each query statement to be edited in each group of query statements to be edited is retrieved by using a search enhancement generation method;
[0017] The at least one piece of knowledge data is generated according to the at least one knowledge graph.
[0018] Optionally, the respectively determining of each group of query statements to be edited and the enhanced representation of the at least one piece of knowledge data corresponding thereto specifically includes:
[0019] Determine respectively the semantic representation of each group of query statements to be edited and at least one piece of knowledge data corresponding thereto;
[0020] The enhanced representation is generated from the semantic representation.
[0021] Optionally, the generating the enhanced representation from the semantic representation specifically includes:
[0022] The semantic representation of each group of query statements to be edited is input into the first multi-layer perceptron layer to generate enhanced representations corresponding to the query statements to be edited.
[0023] Optionally, the generating the enhanced representation from the semantic representation specifically includes:
[0024] The semantic representation of the at least one piece of knowledge data is input into the second multi-layer perceptron layer to generate an enhanced representation corresponding to the at least one piece of knowledge data.
[0025] Optionally, determining, according to the enhanced representation, a prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited specifically includes:
[0026] Inputting the enhanced representation corresponding to the at least one piece of knowledge data into the third multi-layer perceptron layer, and then generating a prompt character representation corresponding to the at least one piece of knowledge data through an activation function;
[0027] Determine a maximum similarity index corresponding to each query statement to be edited in each group of query statements to be edited, wherein the maximum similarity index represents an index of knowledge data having the greatest similarity to each query statement to be edited;
[0028] The prompt character representation corresponding to each query sentence to be edited is determined according to the maximum similarity index.
[0029] Optionally, the loss function includes an editing loss function and a prompt learning loss function, and the editing loss function includes a reliable performance loss function, a general performance loss function and a generalization performance loss function.
[0030] In a second aspect, an embodiment of the present invention provides a model editing device, the device comprising:
[0031] A determining unit, configured to determine three groups of query statements to be edited, wherein each group of query statements to be edited includes at least one query statement to be edited;
[0032] A retrieval unit, configured to retrieve at least one piece of knowledge data corresponding to each group of query statements to be edited from a knowledge database;
[0033] The determining unit is further used to respectively determine the enhanced representation of each group of query statements to be edited and the at least one piece of knowledge data corresponding thereto;
[0034] The determining unit is further configured to determine, according to the enhanced representation, a prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited;
[0035] The determining unit is further configured to determine an input embedding representation corresponding to each query statement to be edited in each group of query statements to be edited;
[0036] a generating unit, configured to edit the large-scale language model according to a combination of the prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited and the input embedding representation, generate an updated large-scale language model, and generate at least one semantic vector;
[0037] The determining unit is further used to determine a loss function according to the semantic vectors respectively corresponding to the three groups of query statements to be edited and the enhanced representation;
[0038] The generating unit is further configured to edit the updated large-scale language model according to the loss function to generate a target large-scale language model.
[0039] Optionally, the retrieval unit is specifically used for:
[0040] In the knowledge database, at least one knowledge spectrum graph corresponding to each query statement to be edited in each group of query statements to be edited is retrieved by using a search enhancement generation method;
[0041] The at least one piece of knowledge data is generated according to the at least one knowledge graph.
[0042] Optionally, the determining unit is specifically configured to:
[0043] Determine respectively the semantic representation of each group of query statements to be edited and at least one piece of knowledge data corresponding thereto;
[0044] The enhanced representation is generated from the semantic representation.
[0045] Optionally, the determining unit is specifically configured to:
[0046] The semantic representation of each group of query statements to be edited is input into the first multi-layer perceptron layer to generate enhanced representations corresponding to the query statements to be edited.
[0047] Optionally, the determining unit is specifically configured to:
[0048] The semantic representation of the at least one piece of knowledge data is input into the second multi-layer perceptron layer to generate an enhanced representation corresponding to the at least one piece of knowledge data.
[0049] Optionally, the determining unit is further configured to:
[0050] Inputting the enhanced representation corresponding to the at least one piece of knowledge data into the third multi-layer perceptron layer, and then generating a prompt character representation corresponding to the at least one piece of knowledge data through an activation function;
[0051] Determine a maximum similarity index corresponding to each query statement to be edited in each group of query statements to be edited, wherein the maximum similarity index represents an index of knowledge data having the greatest similarity to each query statement to be edited;
[0052] The prompt character representation corresponding to each query sentence to be edited is determined according to the maximum similarity index.
[0053] Optionally, the loss function includes an editing loss function and a prompt learning loss function, and the editing loss function includes a reliable performance loss function, a general performance loss function and a generalization performance loss function.
[0054] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, the memory being used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement a method as described in the first aspect or any possible embodiment of the first aspect.
[0055] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement a method as described in the first aspect or any possible embodiment of the first aspect.
[0056] In an embodiment of the present invention, three groups of query statements to be edited are determined, wherein each group of query statements to be edited includes at least one query statement to be edited; according to each group of query statements to be edited, at least one piece of knowledge data corresponding to it is retrieved from a knowledge database; enhanced representations of each group of query statements to be edited and at least one piece of knowledge data corresponding to it are respectively determined; a prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited is determined according to the enhanced representation; an input embedding representation corresponding to each query statement to be edited in each group of query statements to be edited is determined; according to a combination of the prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited and the input embedding representation, a large-scale language model is edited to generate an updated large-scale language model, and at least one semantic vector is generated; a loss function is determined according to the semantic vectors corresponding to the three groups of query statements to be edited and the enhanced representation; and according to the loss function, the updated large-scale language model is edited to generate a target large-scale language model. Through the above method, at least one piece of knowledge data corresponding to each query statement to be edited is retrieved from the knowledge database by using retrieval enhancement generation, and further compressed after enhanced representation to generate a prompt character representation. The semantic vector is determined based on the combination of the prompt character representation and the input embedding representation. Since there are three groups of query statements to be edited, three loss functions corresponding to reliability, universality, and generalization can be determined, as well as a prompt learning loss function. The large-scale language model is edited according to multiple loss functions to generate a target large-scale language model. Since the above model editing method does not change the internal parameters of the large-scale data model and only edits new knowledge, lifelong model editing can be achieved and the problem of capacity degradation of LLMs during the editing process can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:
[0058] Figure 1 It is a flow chart of a method for model editing in the prior art;
[0059] Figure 2 It is another flow chart of a method for model editing in the prior art;
[0060] Figure 3 is a flow chart of a method for model editing in an embodiment of the present invention;
[0061] Figure 4 is a flow chart of another method for model editing in an embodiment of the present invention;
[0062] Figure 5 is a flow chart of another model editing method in an embodiment of the present invention;
[0063] Figure 6 is a schematic diagram of a device for model editing in an embodiment of the present invention;
[0064] Figure 7 is a schematic diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION
[0065] The present application is described below based on embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, some specific details are described in detail. It is possible for those skilled in the art to fully understand the present application without the description of these details. In order to avoid confusing the essence of the present application, known methods, processes, flows, components and circuits are not described in detail.
[0066] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.
[0067] Unless the context clearly requires otherwise, the words "include", "comprising" and similar words throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, the meaning is "including but not limited to".
[0068] In the description of this application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" is two or more.
[0069] In the prior art, a simple Sequential Model Editing (SME) method, a MEND method, and a ROME method are used to edit large-scale language models (LLMs).
[0070] Specifically, the SME method is only applicable to single editing and simple knowledge editing. The SME method can only process small amounts of data and still faces great challenges in editing long sequences of knowledge. Moreover, in scenarios involving sequence editing of multiple pieces of knowledge, only multiple edits can be performed. Multiple edits will cause the general capabilities of LLMs to degrade during the editing process, resulting in forgetting problems, which in turn will affect other parameters in the LLMs during the editing process, causing the LLMs to have capability degradation problems during the editing process, namely forgetting problems, and thus produce hallucination phenomena.
[0071] The MEND method is a group of small auxiliary editing networks based on gradient decomposition, which uses a single expected input-output data to quickly locally edit the pre-trained model. MEND obtains the gradient through standard fine-tuning, decomposes the gradient into two low-rank tensors, and uses the network modeling interaction between the tensors to provide better editing performance. Even for models with more than 10 billion parameters, MEND can complete the training of editing parameters on a single graphics processing unit (GPU) within one day, and quickly apply new editing parameters to the pre-trained large-scale language model. However, the MEND method can only model the tensor interactions between single editing samples, and cannot capture the association between input sample knowledge well. In the sequence editing scenario, for multiple input knowledge, the MEND method can only process each piece of knowledge in sequence, and will also suffer from the problem of capacity degradation; during the editing process, the MEND method will gradually increase the internal parameters in the large-scale data model in the feedforward (FFN) layer, as shown in the following example. Figure 1 As shown, LLM0 is the initial large-scale data model, which transforms knowledge data k1, k2, ... n To edit to the LLM0, first, edit the knowledge data k1, that is, the LLM0 generates LLM1 through Edit k1, and new parameters are added inside the LLM1, that is, the large-scale data model is updated from the LLM0 to LLM1; then, edit the knowledge data k2, that is, the LLM1 generates LLM2 through Edit k2, and new parameters are added inside the LLM2, that is, the large-scale data model is updated from the LLM1 to LLM2; and so on, after multiple edits, the knowledge data k t To edit, that is, the LLMt-1 After Edit k t Generate LLM t , the LLM t New parameters are added internally, namely the large-scale data model is composed of the LLM k-1 Update to LLM k ; Wherein, the knowledge data k1 is that the president of a country is AA-BB (The president of the country is AA-BB); the knowledge data k2 is that the general manager of Amazon is Jeff Bezos-Andy Jassy (The CEO of Amazon is Jeff Bezos-Andy Jassy); the knowledge data k t The location of the largest solar power plant is California—India; the query statement to be edited includes Q1, Q2, Q3 and Q4, the content of Q1 is "The President of the country is _", the content of Q2 is "The chief executive officer of Amazon is _", the content of Q3 is "The Space Needle is located in _", and the content of Q4 is "The location of the largest solar power plant is _". According to the LLM k The answers to Q1, Q2, Q3 and Q4 are Q1: AA; Q2: Anssy; Q3: Seadia; Q4: India. The first three answers to the four questions are wrong because the LLM k The change of internal parameters between the initial LLM0 is very large, which is the main reason for the forgetting problem. As the number of large-scale data model edits increases, more and more parameters are added to the large-scale data model, resulting in the degradation of the internal parameters of the large-scale data model. The essential reason is that the internal parameter training is the overall alignment training, and the increase of parameters only suddenly changes part of the parameter structure, without modifying the knowledge editing from the perspective of the global parameter space; the knowledge data k1, k2...k n It can be any knowledge data that needs to be edited, and the query statements Q1, Q2, Q3 and Q4 to be edited can be any query statements to be edited, which is only an exemplary description here.
[0072] The ROME method is an attribution method based on knowledge editing. In the knowledge editing process, the model parameters are first located according to the input knowledge to be edited, and then the model is edited according to the located parameters. The ROME method is also based on a single piece of knowledge for location and editing, and there is a lack of interaction between the location process and the editing process of each piece of knowledge for sequence knowledge editing data. Therefore, the MEND method will also have the problem of capacity degradation; the ROME method will gradually modify the internal parameters in the large-scale data model at the FFN layer during the editing process, as shown in the following example. Figure 2 As shown, LLM0 is the initial large-scale data model, which transforms knowledge data k1, k2, ... n To edit to the LLM0, first, edit the knowledge data k1, that is, the LLM0 generates LLM1 through Edit k1, and the internal parameters of the LLM0 are modified, that is, the large-scale data model is updated from LLM0 to LLM1; then, edit the knowledge data k2, that is, the LLM1 generates LLM2 through Edit k2, and the internal parameters of the LLM1 are modified, that is, the large-scale data model is updated from LLM1 to LLM2; and so on, after multiple edits, the knowledge data k t To edit, that is, the LLM t-1 After Edit k t Generate LLM t , the LLM t-1 The internal parameters are modified, that is, the large-scale data model is composed of the LLM k-1 Update to LLM k ; Wherein, the knowledge data k1 is that the president of a country is AA—BB (The president of the country is AA—BB); the knowledge data k2 is that the CEO of Amazon is Jeff Bezos—Andy Jassy (The CEO of Amazon is Jeff Bezos—Andy Jassy); the knowledge data k tThe location of the largest solar power plant is California—India; the query statement to be edited includes Q1, Q2, Q3 and Q4, the content of Q1 is "The President of the country is _", the content of Q2 is "The chief executive officer of Amazon is _", the content of Q3 is "The Space Needle is located in _", and the content of Q4 is "The location of the largest solar power plant is _". According to the LLM k The answers to Q1, Q2, Q3 and Q4 are Q1: Bimp; Q2: Jeff Jassy; Q3: Califord; Q4: California. The answers to the four questions are all wrong because the LLM k The change of internal parameters between the initial LLM0 is very large, which is the main reason for the forgetting problem. With the increase of the number of large-scale data model editing, the parameters modified inside the large-scale data model will become more and more, resulting in the degradation of the internal parameters of the large-scale data model. The essential reason is that the internal parameter training is the overall alignment training, and the modification of parameters only suddenly changes part of the parameter structure, without modifying the knowledge editing from the perspective of the global parameter space; the knowledge data k1, k2...k n It can be any knowledge data that needs to be edited, and will not be described in detail here.
[0073] Therefore, how to achieve lifelong model editing of large-scale language models (LLMs) and reduce the ability degradation problem of the LLMs during the editing process is a problem that needs to be solved at present.
[0074] In order to solve the above problems, a model editing method is proposed in the embodiment of the present invention. Figure 3 As shown, the method includes:
[0075] Step S301: determine three groups of query statements to be edited.
[0076] Specifically, each group of query statements to be edited includes at least one query statement to be edited.
[0077] In an embodiment of the present invention, the three groups of query statements to be edited are respectively used to determine the reliability performance loss function, the general performance loss function and the generalization performance loss function included in the editing loss function, and are used to determine the prompt learning loss function.
[0078] In a possible implementation, taking a group of query statements to be edited as an example, the query statements to be edited include q1, q2 and q3, the content of q1 is "The CEO of Amazon is_", the content of q2 is "The Country'executive branch is led by President_", and the content of q3 is "The birthplace of Xiao ming is_".
[0079] Step S302: According to each group of query statements to be edited, at least one piece of corresponding knowledge data is retrieved from the knowledge database.
[0080] Specifically, in the knowledge database, retrieval-augmented
[0081] generation, RAG) method to retrieve at least one knowledge graph corresponding to each query statement to be edited in each group of query statements to be edited; and generate at least one knowledge data according to the at least one knowledge graph (KG).
[0082] In an embodiment of the present invention, the knowledge database stores multiple knowledge graphs, each of which is composed of triples. For example, according to the query statement to be edited, the knowledge graphs corresponding to each group of the query statements to be edited are retrieved in the knowledge database through a retrieval enhancement generation method as "President of a certain country BB", "Amazon General Manager Andy Jassy"... "The largest solar power plant in India"; after obtaining the knowledge graph, it is necessary to generate natural language sentences from the knowledge graph. For example, the knowledge graph "President of a certain country BB" generates knowledge data k1 as "The president of the country is BB"; the knowledge graph "Amazon General Manager Andy Jassy" generates knowledge data k2 as "The CEO of Amazon is Andy Jassy";... The knowledge graph "The largest solar power plant in India" generates knowledge data k tThe largest solar power plant is located in India.
[0083] In an embodiment of the present invention, the knowledge data is acquired in the knowledge database through enhanced retrieval according to the query statement to be edited, and the specific content acquired is determined according to the content of the query statement to be edited. This is only an exemplary description and is determined according to actual conditions.
[0084] Step S303: respectively determine the enhanced representation of each group of query statements to be edited and the at least one piece of knowledge data corresponding thereto.
[0085] Specifically, the semantic representation of each group of query statements to be edited and the at least one piece of knowledge data corresponding thereto is determined respectively; and the enhanced representation is generated from the semantic representation.
[0086] In the embodiment of the present invention, the query statements q1, q2, q3 to be edited and the knowledge data k1, k2, ..., k t Through the RoBERTa model, the corresponding semantic representations are obtained respectively. Specifically, the RoBERTa model can be used with the function f rm Indicates that the semantic representation of the query statement to be edited is f rm (q), the semantic representation of the knowledge data is f rm (k t ), according to the f rm (q) and said f rm (k t ) further generates enhanced representations, as follows:
[0087] The semantic representation of each group of query statements to be edited is input into the first multi-layer perceptron layer (MLP Q ), generating an enhanced representation corresponding to the query statement to be edited.
[0088] Specifically, the f rm (q) Input to MLP Q , generating an enhanced representation corresponding to the query statement to be edited, which is expressed as follows:
[0089]
[0090] In the embodiment of the present invention, the enhanced representations corresponding to the query statements q1, q2, and q3 to be edited are: and
[0091]
[0092] The semantic representation of the at least one piece of knowledge data is input into the second multi-layer perceptron layer (MLP K ), generating the at least one corresponding enhanced representation.
[0093] Specifically, the f rm (k t ) Input knowledge data to MLP K , generate the enhanced representation corresponding to the knowledge data, expressed as follows:
[0094]
[0095] In the embodiment of the present invention, the knowledge data k1, k2...k t The corresponding enhancement representation is In the embodiment of the present invention, the multilayer perceptron (MLP) layers of the first multilayer perceptron layer and the second multilayer perceptron layer are independently encoded.
[0096] Step S304: determining, according to the enhanced representation, a prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited.
[0097] Specifically, the enhanced representation corresponding to the at least one piece of knowledge data is input into the third multi-layer perceptron layer, and then the prompt character representation corresponding to the at least one piece of knowledge data is generated through an activation function; the maximum similarity index corresponding to each query statement to be edited in each group of query statements to be edited is determined, wherein the maximum similarity index represents the index of the knowledge data with the greatest similarity to each query statement to be edited; and the prompt character representation corresponding to each query statement to be edited is determined according to the maximum similarity index.
[0098] In the embodiment of the present invention, the third multi-layer perceptron layer is MLP P , the activation function is f resp , the prompt character representation corresponding to the knowledge data, which can also be called prompt token representation, is expressed as follows:
[0099]
[0100] The prompt character representation can control the length, and thus control the length of the large-scale data model for reasoning. In the embodiment of the present invention, a knowledge prompt information (prompt) θ is matched for each knowledge data. For example, the θ includes θ 1 ,θ 2 ,θ 3 ,θ 4 ,θ 5 ,θ 6 ...θC The knowledge prompt information is not a continuous natural language text, but a corresponding model embedding representation. If the knowledge prompt information is relatively long, the prompt characters (tokens) generated by the knowledge prompt information can be compressed. Assuming that each knowledge graph is composed of triples, the prompt character representation is determined to be a triple representation, that is, three token compression amounts can obtain better large-scale semantic model editing effects.
[0101] In the embodiment of the present invention, the prompt character corresponding to k1 is represented as The prompt character corresponding to k2 is represented as ...the k t The corresponding prompt character representation is Wherein, the l refers to the number of the token.
[0102] In a possible implementation, when determining the prompt character representation, it is necessary to pass the function f rm and the second multilayer perceptron layer (MLP K ) Generate r θ 、 θ and stated After the third multi-layer perceptron layer is MLP P and the activation function is f resp , jointly generate the prompt character representation.
[0103] In the embodiment of the present invention, it is necessary to determine the maximum similarity index corresponding to each query statement to be edited in each group of query statements to be edited. Then, the corresponding prompt character representation is determined for each query statement to be edited according to the maximum similarity index. It is represented by KS(q) score, as follows:
[0104]
[0105] Among them, the For the The transpose of It means that the knowledge data does not exist, the KS score is Null, and the retrieval process is terminated; specifically, the knowledge data does not exist in the current query library, and a None placeholder is set for retrieval enhancement generation.
[0106] In the embodiment of the present invention, it is determined that the query statement to be edited q1 and Correspondingly, the query statement to be edited q2 and correspond.
[0107] Step S305: Determine the input embedding representation corresponding to each query statement to be edited in each group of query statements to be edited.
[0108] Specifically, the query statement to be edited q i Through the function f emb Determine qi , as follows:
[0109] e qi =f emb (q i );
[0110] In the embodiment of the present invention, the input embedding representation corresponding to q1 is: The input embedding representation corresponding to q2 is The input embedding representation corresponding to q1 is
[0111] Step S306: edit the large-scale language model according to the combination of the prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited and the input embedding representation, generate an updated large-scale language model, and generate at least one semantic vector.
[0112] Specifically, the prompt character representation corresponding to each query statement to be edited and the input embedding representation are combined. The combination is not spliced at the natural language level, but spliced in the embedding semantic space, which is specifically expressed as:
[0113]
[0114] Input the combination into the updated large-scale language model Generate semantic vector a q ,Right now:
[0115]
[0116] In the embodiment of the present invention, the semantic vector a of the query statement q1 to be edited q1 :Andy Jassy;The semantic vector a of the query statement q2 to be edited q2 :BB; the semantic vector a of the query statement q3 to be edited q3 :Hawaii.
[0117] Step S307: determining a loss function according to the semantic vectors corresponding to the three groups of query statements to be edited and the enhanced representation.
[0118] In an embodiment of the present invention, the loss function includes an editing loss function and a prompt learning loss function, and the editing loss function includes a reliability performance loss function, a general performance loss function and a generalization performance loss function.
[0119] Specifically, the reliability performance loss function is as follows:
[0120]
[0121] Among them, the a ei is the semantic vector corresponding to the query statement to be edited corresponding to the reliable performance, and the q ei is the query statement to be edited corresponding to the reliable performance. The updated large-scale language model.
[0122] The general performance loss function is as follows:
[0123]
[0124] Among them, the a gi is the semantic vector corresponding to the query statement to be edited corresponding to the general performance, and the q gi It is the query statement to be edited corresponding to the general performance.
[0125] The generalization performance loss function is as follows:
[0126]
[0127] Among them, the a li is the semantic vector corresponding to the query statement to be edited corresponding to the generalization performance, and the q li is the query statement to be edited corresponding to the generalization performance, and the f llm is an initial large-scale language model, and the KL is the KL divergence (Kullback–Leibler divergence).
[0128] In the embodiment of the present invention, the prompt learning process for determining the prompt learning loss function is to perform prompt token training by a contrastive learning process, taking the currently recalled knowledge as positive samples and other samples as negative samples, and performing contrastive learning training using infoNCE loss. Specifically, the formula for generating positive sample training is as follows:
[0129]
[0130] Wherein, R is a sample set.
[0131] The formula for generating negative sample training is as follows:
[0132]
[0133] Among them, the R\k i is the set from which positive samples are removed.
[0134] The formula of the prompt learning loss function is as follows:
[0135]
[0136] In a possible implementation, the δ is expressed as follows:
[0137]
[0138] Among them, the k + is a positive sample.
[0139] Step S308: Edit the updated large-scale language model according to the loss function to generate a target large-scale language model.
[0140] Specifically, through the and Regarding the Edit and generate target LLM s .
[0141] In the embodiment of the present invention, the data flow diagram of the model editing process is as follows: Figure 4 shown.
[0142] Through the above embodiment, the LLM0 is an initial large-scale data model, which transforms the knowledge data k1, k2...k n To edit the LLM0, first, edit the knowledge data k1, that is, the LLM0 is edited by Edit k1 outside LLM0 to generate LLM1, and Edit1 is knowledge retrieval; then, edit the knowledge data k2, that is, the LLM1 is edited by Edit k2 outside LLM1 to generate LLM2; and so on, after multiple edits, the knowledge data k t To edit, that is, the LLM t-1 After Edit k t In LLM t-1 Externally perform Edit1, Edit2, ... Edit t to generate LLM t , the LLM s The internal parameters have not been changed, only the external parameters have been changed. The specific diagram is as follows Figure 5As shown, it is assumed that the knowledge data k1 is that the president of a country is AA-BB (The president of the country is AA-BB); the knowledge data k2 is that the general manager of Amazon is Jeff Bezos-Andy Jassy (The CEO of Amazon is Jeff Bezos-AndyJassy); the knowledge data k t The location of the largest solar power plant is California—India; the query statement to be edited includes Q1, Q2, Q3 and Q4, the content of Q1 is "The President of the country is _", the content of Q2 is "The chief executive officer of Amazon is _", the content of Q3 is "The Space Needle is located in _", and the content of Q4 is "The location of the largest solar power plant is _". According to the LLM k The answers to Q1, Q2, Q3 and Q4 are Q1: BB; Q2: Andy Jassy; Q3: Seattle; Q4: India. The answers to the four questions are all correct, indicating that the large-scale data model edited multiple times by the method described in the embodiment of the present invention has better general performance and will not cause forgetting problems.
[0143] In the embodiment of the present invention, the knowledge to be edited is continuously stored in the knowledge database for recall learning, that is, the knowledge data is retrieved for training using the retrieval capability, which can maintain the reusability of the knowledge and the performance of lifelong editing of the large-scale language model. In addition, during the editing process of the large-scale language model, the length of the large model input is compressed by compressing the number of tokens through the MLP layer, which improves the utilization rate of the large model for the input data, and enables the large-scale language model to perform the knowledge editing process without changing the internal parameters, thereby reducing the degradation of the general performance of the large-scale language model.
[0144] In the embodiment of the present invention, the model editing method may be referred to as RECIPE. The effects of different model editing methods on different editing times and different data sets are shown in Table 1 below. Specifically:
[0145] Table 1
[0146]
[0147]
[0148]
[0149] The average values in Table 1 have positive and negative deviations, which will not be elaborated here;
[0150] As can be seen from Table 1, RECIPE performs very well in reliability, universality and generalization when editing different numbers of times on different datasets, and the universal performance is not affected by the difference in datasets.
[0151] In the embodiment of the present invention, the model editing method may be called RECIPE. As the number of edits increases, the general capabilities change as shown in Table 2 below. Specifically:
[0152] Table 2
[0153]
[0154] In the above Table 2, N / A represents the general performance of the unedited model. It can be seen from Table 2 that the general performance has good results on CSQA, MMLU, ANLI, and SQUAD-2, and the performance of the large-scale language model in general performance will not be reduced due to the increase in the number of edits.
[0155] In an embodiment of the present invention, a model editing device is provided, such as Figure 6As shown, it specifically includes: a determination unit 601, a retrieval unit 602 and a generation unit 603; wherein the determination unit 601 is used to determine three groups of query statements to be edited, wherein each group of query statements to be edited includes at least one query statement to be edited; the retrieval unit 602 is used to retrieve at least one piece of knowledge data corresponding to each group of query statements to be edited in the knowledge database; the determination unit 601 is also used to respectively determine the enhanced representation of each group of query statements to be edited and the at least one piece of knowledge data corresponding to the query statements to be edited; the determination unit 601 is also used to determine the prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited according to the enhanced representation; the determination unit The determining unit 601 is further used to determine the input embedding representation corresponding to each query statement to be edited in each group of query statements to be edited; the generating unit 603 is used to edit the large-scale language model according to the combination of the prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited and the input embedding representation, generate an updated large-scale language model, and generate at least one semantic vector; the determining unit 601 is further used to determine the loss function according to the semantic vectors corresponding to the three groups of query statements to be edited and the enhanced representation; the generating unit 603 is further used to edit the updated large-scale language model according to the loss function to generate a target large-scale language model.
[0156] Furthermore, the retrieval unit is specifically used for:
[0157] In the knowledge database, at least one knowledge spectrum graph corresponding to each query statement to be edited in each group of query statements to be edited is retrieved by using a search enhancement generation method;
[0158] The at least one piece of knowledge data is generated according to the at least one knowledge graph.
[0159] Furthermore, the determining unit is specifically configured to:
[0160] Determine respectively the semantic representation of each group of query statements to be edited and at least one piece of knowledge data corresponding thereto;
[0161] The enhanced representation is generated from the semantic representation.
[0162] Furthermore, the determining unit is specifically configured to:
[0163] The semantic representation of each group of query statements to be edited is input into the first multi-layer perceptron layer to generate enhanced representations corresponding to the query statements to be edited.
[0164] Furthermore, the determining unit is specifically configured to:
[0165] The semantic representation of the at least one piece of knowledge data is input into the second multi-layer perceptron layer to generate an enhanced representation corresponding to the at least one piece of knowledge data.
[0166] Furthermore, the determining unit is further configured to:
[0167] Inputting the enhanced representation corresponding to the at least one piece of knowledge data into the third multi-layer perceptron layer, and then generating a prompt character representation corresponding to the at least one piece of knowledge data through an activation function;
[0168] Determine a maximum similarity index corresponding to each query statement to be edited in each group of query statements to be edited, wherein the maximum similarity index represents an index of knowledge data having the greatest similarity to each query statement to be edited;
[0169] The prompt character representation corresponding to each query sentence to be edited is determined according to the maximum similarity index.
[0170] Optionally, the loss function includes an editing loss function and a prompt learning loss function, and the editing loss function includes a reliable performance loss function, a general performance loss function and a generalization performance loss function.
[0171] Figure 7 Schematic diagram of the structure of the electronic device in the embodiment of the present invention. Figure 7 As shown, it includes a general computer hardware structure, which at least includes a processor 701 and a memory 702. The processor 701 and the memory 702 are connected via a bus 703. The memory 702 is suitable for storing instructions or programs executable by the processor 701. The processor 701 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 701 executes the instructions stored in the memory 702, thereby executing the method flow of the embodiment of the present invention as described above to realize the processing of data and the control of other devices. The bus 703 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 704 and the display device and the input / output (I / O) device 705. The input / output (I / O) device 705 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output device 705 is connected to the system through an input / output (I / O) controller 706.
[0172] The instructions stored in the memory 702 are executed by at least one processor 701 to implement: determining three groups of query statements to be edited, wherein each group of query statements to be edited includes at least one query statement to be edited; according to each group of query statements to be edited, retrieving at least one piece of knowledge data corresponding to it in a knowledge database; respectively determining enhanced representations of each group of query statements to be edited and at least one piece of knowledge data corresponding to it; determining a prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited according to the enhanced representation; determining an input embedding representation corresponding to each query statement to be edited in each group of query statements to be edited; editing a large-scale language model according to a combination of a prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited and the input embedding representation, generating an updated large-scale language model, and generating at least one semantic vector; determining a loss function according to the semantic vectors corresponding to the three groups of query statements to be edited and the enhanced representation; and editing the updated large-scale language model according to the loss function to generate a target large-scale language model.
[0173] Specifically, the electronic device includes: one or more processors 701 and a memory 702, Figure 7 Take a processor 701 as an example. The processor 701 and the memory 702 may be connected via a bus or other means. Figure 7 In the example, the bus connection is used. The memory 702 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The processor 701 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions and modules stored in the memory 702, that is, the above-mentioned method for determining model editing is implemented.
[0174] The memory 702 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store a list of options, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 702 may optionally include a memory remotely arranged relative to the processor 701, and these remote memories may be connected to an external device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0175] One or more modules are stored in the memory 702, and when executed by one or more processors 701, the model editing method in any of the above method embodiments is executed.
[0176] As will be appreciated by those skilled in the art, various aspects of the embodiments of the present invention may be implemented as a system, method or computer program product. Therefore, various aspects of the embodiments of the present invention may take the form of a complete hardware implementation, a complete software implementation (including firmware, resident software, microcode, etc.), or an implementation that combines software aspects with hardware aspects, which may generally be referred to herein as a "circuit," "module," or "system." In addition, various aspects of the embodiments of the present invention may take the form of a computer program product implemented in one or more computer-readable media having a computer-readable program code implemented thereon.
[0177] Any combination of one or more computer-readable media can be utilized. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, (but not limited to) electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or apparatuses, or any suitable combination of the foregoing. More specific examples (non-exhaustive enumeration) of computer-readable storage media will include the following: an electrical connection with one or more wires, a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of an embodiment of the present invention, a computer-readable storage medium can be any tangible medium that can contain or store a program used by an instruction execution system, device or apparatus or a program used in conjunction with an instruction execution system, device or apparatus.
[0178] A computer readable signal medium may include a propagated digital signal having a computer readable program code implemented therein, such as in baseband or as part of a carrier wave. Such propagated signals may take any of a variety of forms, including but not limited to electromagnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any of the following computer readable media: not a computer readable storage medium, and may communicate, propagate, or transmit a program used by or in conjunction with an instruction execution system, device, or apparatus.
[0179] Program code embodied on a computer readable medium may be transmitted using any appropriate medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0180] The computer program code for performing operations for various aspects of the embodiments of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, Smalltalk, C++, etc., and conventional process programming languages such as "C" programming language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer as a stand-alone software package; partially on the user's computer and partially on a remote computer; or completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., by using the Internet of an Internet service provider).
[0181] The flowchart legend and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present invention described above describe various aspects of the embodiment of the present invention.It will be understood that each block of the flowchart legend and / or block diagram and the combination of the blocks in the flowchart legend and / or block diagram can be implemented by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that the instructions (executed by the processor of the computer or other programmable data processing device) create a device for implementing the function / action specified in the flowchart and / or block diagram block or block.
[0182] These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing device, or other apparatus to operate in a particular manner, so that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions / actions specified in the flowchart and / or block diagram blocks or blocks.
[0183] The computer program instructions may also be loaded onto a computer, other programmable data processing device or other apparatus so that a series of operable steps are performed on the computer, other programmable device or other apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide a process for implementing the functions / actions specified in the flowchart and / or block diagram block or blocks.
[0184] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A method for model editing, It is characterized in that The method comprises: Determine three groups of query statements to be edited, wherein each group of query statements to be edited includes at least one query statement to be edited; According to each group of query statements to be edited, at least one piece of knowledge data corresponding to the query statement is retrieved from the knowledge database; Determine respectively each group of query statements to be edited and the enhanced representation of at least one piece of knowledge data corresponding thereto; Determine, according to the enhanced representation, a prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited; Determine an input embedding representation corresponding to each query statement to be edited in each group of query statements to be edited; Editing the large-scale language model according to a combination of the prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited and the input embedding representation, generating an updated large-scale language model, and generating at least one semantic vector; Determine a loss function according to the semantic vectors corresponding to the three groups of query sentences to be edited and the enhanced representation; The updated large-scale language model is edited according to the loss function to generate a target large-scale language model.
2. The method according to claim 1, It is characterized in that The step of retrieving at least one piece of knowledge data corresponding to each group of query statements to be edited from a knowledge database specifically includes: In the knowledge database, at least one knowledge spectrum graph corresponding to each query statement to be edited in each group of query statements to be edited is retrieved by using a search enhancement generation method; The at least one piece of knowledge data is generated according to the at least one knowledge graph.
3. The method according to claim 1, It is characterized in that The step of respectively determining each group of query statements to be edited and the enhanced representation of at least one piece of knowledge data corresponding thereto specifically includes: Determine respectively the semantic representation of each group of query statements to be edited and at least one piece of knowledge data corresponding thereto; The enhanced representation is generated from the semantic representation.
4. The method according to claim 3, It is characterized in that The generating the enhanced representation from the semantic representation specifically includes: The semantic representation of each group of query statements to be edited is input into the first multi-layer perceptron layer to generate enhanced representations corresponding to the query statements to be edited.
5. The method according to claim 3, It is characterized in that The generating the enhanced representation from the semantic representation specifically includes: The semantic representation of the at least one piece of knowledge data is input into the second multi-layer perceptron layer to generate an enhanced representation corresponding to the at least one piece of knowledge data.
6. The method according to claim 1, It is characterized in that The determining, according to the enhanced representation, a prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited specifically includes: Inputting the enhanced representation corresponding to the at least one piece of knowledge data into the third multi-layer perceptron layer, and then generating a prompt character representation corresponding to the at least one piece of knowledge data through an activation function; Determine a maximum similarity index corresponding to each query statement to be edited in each group of query statements to be edited, wherein the maximum similarity index represents an index of knowledge data having the greatest similarity to each query statement to be edited; The prompt character representation corresponding to each query sentence to be edited is determined according to the maximum similarity index.
7. The method according to claim 1, It is characterized in that The loss function includes an editing loss function and a prompt learning loss function, and the editing loss function includes a reliable performance loss function, a general performance loss function and a generalization performance loss function.
8. A device for model editing, It is characterized in that The device comprises: A determining unit, configured to determine three groups of query statements to be edited, wherein each group of query statements to be edited includes at least one query statement to be edited; A retrieval unit, configured to retrieve at least one piece of knowledge data corresponding to each group of query statements to be edited from a knowledge database; The determining unit is further used to respectively determine the enhanced representation of each group of query statements to be edited and the at least one piece of knowledge data corresponding thereto; The determining unit is further configured to determine, according to the enhanced representation, a prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited; The determining unit is further configured to determine an input embedding representation corresponding to each query statement to be edited in each group of query statements to be edited; a generating unit, configured to edit the large-scale language model according to a combination of the prompt character representation corresponding to each query statement to be edited in each group of query statements to be edited and the input embedding representation, generate an updated large-scale language model, and generate at least one semantic vector; The determining unit is further used to determine a loss function according to the semantic vectors respectively corresponding to the three groups of query statements to be edited and the enhanced representation; The generating unit is further configured to edit the updated large-scale language model according to the loss function to generate a target large-scale language model.
9. An electronic device comprising a memory and a processor, It is characterized in that The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, It is characterized in that The computer readable storage The medium stores a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Data retrieval method, device and system, electronic equipment and readable storage medium
CN118093962A
Dataset generation using large language models
US20240185001A1
