A knowledge editing method based on super network parameter generation
Patent Information
- Application Number
- CN202611037597.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-13
- Publication Date
- 2026-08-07
AI Technical Summary
[0007]本发明的目的在于提供一种基于超网络参数生成的知识编辑方法，旨在解决现有事实纠正方法中存在多跳推理链路断裂、邻域知识漂移以及灾难性遗忘的技术问题
[0060]本发明的有益效果是：本发明通过对知识图谱中实体与关系的拓扑依赖及层级结构进行显式建模，解决了现有系统在进行事实纠正时，因孤立修改文本片段而导致多跳推理链路断裂的问题，实现了在信息检索与智能问答场景中，系统能够准确理解更新事实及其关联实体的逻辑联系，显著提升了复杂事实问答和关联检索的准确率。
Smart Images
Figure CN122528834A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a knowledge editing method based on hypernetwork parameter generation, belonging to the field of knowledge editing technology. Background Technology
[0002] Intelligent information service systems, such as information retrieval, intelligent question answering, and personalized recommendations, are widely used in people's production and daily lives. In these practical applications, these systems often rely on massive amounts of factual knowledge to provide users with accurate search results, reasonable recommendations, or precise question-and-answer responses. However, the facts of the objective world are dynamically changing (e.g., the attributes of a product change), which requires the system to have efficient fact correction or knowledge updating capabilities. If an intelligent system cannot correct outdated or erroneous facts in a timely and accurate manner, it will directly lead to search bias, recommendation failure, or question-and-answer errors, seriously affecting the system's reliability and user experience.
[0003] To achieve efficient fact storage and logical reasoning in the aforementioned intelligent systems, knowledge graph (KG) technology is widely used as the underlying knowledge support. Knowledge graphs represent objective facts in a structured way through triples and often employ knowledge graph embedding (KGE) methods based on pre-trained language models (PLMs) to map entities and relations to a continuous vector space, thus providing a computational foundation for link prediction and relation reasoning in the system. However, in practical fact correction tasks, existing language model-based knowledge graph embedding methods have significant limitations. Existing models are usually static models; when facts change in the application scenario, the entire underlying model often needs to be retrained for knowledge updates. This retraining method is computationally expensive and extremely inefficient, failing to meet the real-time requirements of fact correction in information retrieval or recommendation systems.
[0004] To address the inefficiency of model updates in fact-correction tasks, knowledge editing techniques have been introduced, aiming to locally correct or inject specific facts without compromising the overall capabilities of the model. However, most existing knowledge editing methods directly adopt the parameter editing approach of general pre-trained language models (such as generating parameter update amounts through hypernetworks or directly modulating feedforward network parameters). These methods typically treat the facts to be corrected as isolated text fragments, lacking explicit modeling of the underlying knowledge graph's structural properties. In practical information retrieval or question answering, knowledge entities exhibit close topological dependencies and hierarchical relationships. Isolated editing methods struggle to maintain the stability of multi-hop inference links, easily leading to errors in associative inference.
[0005] Furthermore, entity relationships in knowledge graphs possess complex transitivity, inversion, and diverse constraint characteristics, with different relationship types exhibiting significant differences in semantic rigidity and structural density. When correcting facts, applying unconstrained or uniformly strong parameter updates to the model, while potentially forcibly correcting the target fact, can easily lead to anisotropic perturbations in the embedding space, disrupting the original semantic manifold. This can trigger severe neighborhood knowledge drift, structural collapse, and catastrophic forgetting—that is, while correcting a specific fact, it unintentionally compromises the accuracy of other non-target facts in the system. In addition, existing methods generally face bottlenecks such as large editing parameter sizes, insufficient local constraints on non-target facts, and difficulty in adaptively adjusting for different relationships.
[0006] In summary, in practical applications such as information retrieval, intelligent question answering, and recommendation systems, how to ensure the accurate and rapid correction of target facts while also taking into account the integrity of the underlying knowledge graph structure, the stability of non-target knowledge, and the efficiency of model parameter updates is a pressing technical problem that needs to be solved in the current application of knowledge graphs based on language models. Summary of the Invention
[0007] The purpose of this invention is to provide a knowledge editing method based on hypernetwork parameter generation, which aims to solve the technical problems of multi-hop reasoning link breakage, neighborhood knowledge drift and catastrophic forgetting in existing fact correction methods.
[0008] To achieve the above objectives, the technical solution of the present invention is: a knowledge editing method based on hypernetwork parameter generation, comprising the following steps:
[0009] S1: Construct a knowledge graph embedding and editing model; wherein, the knowledge graph embedding and editing model includes a pre-trained language model backbone network, an editing branch parallel to the pre-trained language model backbone network, and a super network;
[0010] S2: Convert the obtained target edit triplet into the corresponding natural language query template, and then input the natural language query template into the knowledge graph embedding editing model. Through the pre-trained language model backbone network, obtain the vector representation of the editing instruction corresponding to the target fact.
[0011] S3: Use the hypernetwork to generate low-rank editing parameters based on the vector representation of the editing instructions;
[0012] S4: Inject the low-rank editing parameters into the feedforward network layer in the editing branch and the feedforward network layer of the pre-trained language model backbone network to obtain the edited hidden representation of the target fact and the output of the backbone network;
[0013] S5: Construct a set of local constraints based on the target editing triples; wherein, the set of local constraints includes entity neighborhood samples, same relation prototype samples, and global anchor point samples;
[0014] S6: Extract the semantic representation of the target relation based on the natural language description of the target relation, construct the relation-aware editing gating coefficient based on the semantic representation, dynamically scale the edited hidden representation using the relation-aware editing gating coefficient, and fuse it with the output of the backbone network to obtain the final hidden representation;
[0015] S7: Construct a total loss to jointly optimize the knowledge graph embedding and editing model; wherein, the total loss includes the editing loss of the target fact and the local constraint loss on the set of locality constraints;
[0016] S8: Input the query data to be edited into the jointly optimized knowledge graph embedding and editing model to obtain the entity or relation prediction results of the knowledge graph, and realize knowledge editing based on the entity or relation prediction results.
[0017] Optionally, S2 specifically includes:
[0018] Define the target edit triple as ,in, For the head entity, For the relationship, To query the tail entity;
[0019] Edit the target triplet The header entity and relation are concatenated and converted into declarative text to obtain a natural language query template: the [relation] of [head entity] is [MASK], where [MASK] is a mask;
[0020] The natural language query template is input into the knowledge graph embedding editing model to obtain a vector representation of the editing instructions corresponding to the target fact.
[0021] Optionally, S3 specifically includes:
[0022] The hypernetwork is a two-layer multilayer perceptron. The input dimension is the same as the hidden layer dimension of the pre-trained language model, but the hidden layer dimension is larger than the input dimension. The hypernetwork takes the vector representation of editing instructions as input and outputs a value of [value missing]. After obtaining the vector, the low-rank editing parameters are obtained, which include the low-rank decomposition matrix. and The expression is:
[0023]
[0024] in, , ,and , It is a low-rank number. For the hidden layer dimension of the pre-trained language model, Indicates a hypernetwork. This is a vector representation of editing instructions.
[0025] Optionally, S4 specifically includes:
[0026] After generating low-rank editing parameters, the pre-trained language model backbone network keeps its weights frozen. These low-rank editing parameters are then injected into the feedforward network layer in the editing branch and the feedforward network layer of the pre-trained language model backbone network, resulting in:
[0027]
[0028]
[0029] in, For the output of the backbone network, This represents the activation function. This is the original weight matrix. This is the original offset. The input is hidden. The output of the edit branch represents the hidden representation after editing. This is the offset. The expression for the model parameter update amount is:
[0030]
[0031] in, This is the scaling factor.
[0032] Optionally, S5 specifically includes:
[0033] The expression for the set of locality constraints is:
[0034]
[0035] in, For a set of local constraints, For entity neighborhood samples, This is a prototype sample of the same relation. This is a global anchor point sample;
[0036] in:
[0037] Entity neighborhood samples It consists of knowledge graph triples that share a head entity or query tail entity with the target editing triple, but have different relationships;
[0038] Same relation prototype sample It consists of knowledge graph triples that have the same relationship as the target edit triples, and whose head entity and query tail entity are different;
[0039] Global anchor point samples It consists of knowledge graph triples that are structurally separate from the target edit triples and whose head entity, query tail entity, and relation are all different.
[0040] Optionally, S6 specifically includes:
[0041] Natural language description based on target relations extracts the semantic representation of the target relation, expressed as:
[0042] ,
[0043] in, This is the semantic representation of the target relation. For pre-trained language models, The word sequence for the target relation;
[0044] The semantic representation of the target relation Inputting the gating network yields the relation-aware editing gating coefficients, expressed as:
[0045]
[0046] in, Edit the gating coefficients for relation-aware editing. This represents the Sigmoid activation function. and These represent the weights and biases of the gating network; wherein the gating network is a lightweight multilayer perceptron.
[0047] The edited hidden representation is dynamically scaled using the relation-aware editing gating coefficients and then fused with the output of the backbone network to obtain the final hidden representation, expressed as:
[0048]
[0049] in, This is the final hidden representation.
[0050] Optionally, the locality constraint loss specifically refers to:
[0051] Excluding the original erroneous fact triples from the local constraint set, and constructing the local constraint loss by comparing the output distribution differences of the model before and after editing on the local constraint set, the local constraint loss adopts the Kullback-Leibler divergence, expressed as:
[0052]
[0053] in, This represents the loss due to local constraints. Represents the set of locality constraints The number of samples; Indicates a non-target query. This represents the tag entity corresponding to the non-target query; and These represent the vector distributions before and after editing, respectively. This indicates the model parameters before editing. This represents the parameters for embedding the knowledge graph after editing. This represents the Kullback-Leibler divergence.
[0054] Optionally, the total loss is specifically:
[0055]
[0056] in, For the total loss, For balance coefficient, For editing losses;
[0057] The editing loss is the negative log-likelihood loss of the target tail entity, expressed as:
[0058]
[0059] in, For the target tail entity, For the target query, To query a given target In this case, the model correctly predicts the target tail entity. The probability of.
[0060] The beneficial effects of this invention are as follows: By explicitly modeling the topological dependencies and hierarchical structure of entities and relationships in a knowledge graph, this invention solves the problem of broken multi-hop inference links caused by isolated modification of text fragments when performing fact correction in existing systems. It enables the system to accurately understand the logical connections between updated facts and their associated entities in information retrieval and intelligent question answering scenarios, significantly improving the accuracy of complex fact question answering and related retrieval. Attached Figure Description
[0061] Figure 1 This is a flowchart of the steps of the present invention. Detailed Implementation
[0062] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.
[0063] Example 1: As Figure 1 As shown, a knowledge editing method based on hypernetwork parameter generation includes the following steps:
[0064] S1: Construct a knowledge graph embedding and editing model; wherein, the knowledge graph embedding and editing model includes a pre-trained language model backbone network, an editing branch parallel to the pre-trained language model backbone network, and a super network;
[0065] S2: Convert the obtained target edit triplet into the corresponding natural language query template, and then input the natural language query template into the knowledge graph embedding editing model. Through the pre-trained language model backbone network, obtain the vector representation of the editing instruction corresponding to the target fact.
[0066] Optionally, S2 specifically includes:
[0067] Define the target edit triple as ,in, For the head entity, For the relationship, To query the tail entity;
[0068] Edit the target triplet The header entity and relation are concatenated and converted into declarative text to obtain a natural language query template: the [relation] of [head entity] is [MASK], where [MASK] is a mask;
[0069] The natural language query template is input into the knowledge graph embedding editing model to obtain a vector representation of the editing instructions corresponding to the target fact.
[0070] S3: Use the hypernetwork to generate low-rank editing parameters based on the vector representation of the editing instructions;
[0071] Optionally, the hypernetwork is a two-layer multilayer perceptron, with the input dimension matching the hidden layer dimension of the pre-trained language model, and the hidden layer dimension being larger than the input dimension. The hypernetwork takes the vector representation of the editing instructions as input and outputs a value of [value missing]. After obtaining the vector, the low-rank editing parameters are obtained, which include the low-rank decomposition matrix. and The expression is:
[0072]
[0073] in, , ,and , It is a low-rank number. For the hidden layer dimension of the pre-trained language model, Indicates a hypernetwork. This is a vector representation of editing instructions.
[0074] S4: Inject the low-rank editing parameters into the feedforward network layer in the editing branch and the feedforward network layer of the pre-trained language model backbone network to obtain the edited hidden representation of the target fact and the output of the backbone network;
[0075] Optionally, after generating low-rank editing parameters, the pre-trained language model backbone network keeps its weights frozen, and injects the low-rank editing parameters into the feedforward network layer in the editing branch and the feedforward network layer of the pre-trained language model backbone network, resulting in:
[0076]
[0077]
[0078] in, For the output of the backbone network, This represents the activation function. This is the original weight matrix. This is the original offset. The input is hidden. The output of the edit branch represents the hidden representation after editing. This is the offset. The expression for the model parameter update amount is:
[0079]
[0080] in, This is the scaling factor.
[0081] S5: Construct a set of local constraints based on the target editing triples; wherein, the set of local constraints includes entity neighborhood samples, same relation prototype samples, and global anchor point samples;
[0082] Optionally, the expression for the set of locality constraints is:
[0083]
[0084] in, For a set of local constraints, For entity neighborhood samples, This is a prototype sample of the same relation. This is a global anchor point sample;
[0085] in:
[0086] Entity neighborhood samples It consists of knowledge graph triples that share a head entity or query tail entity with the target editing triple, but have different relationships;
[0087] Same relation prototype sample It consists of knowledge graph triples that have the same relationship as the target edit triples, and whose head entity and query tail entity are different;
[0088] Global anchor point samples It consists of knowledge graph triples that are structurally separate from the target edit triples and whose head entity, query tail entity, and relation are all different.
[0089] S6: Extract the semantic representation of the target relation based on the natural language description of the target relation, construct the relation-aware editing gating coefficient based on the semantic representation, dynamically scale the edited hidden representation using the relation-aware editing gating coefficient, and fuse it with the output of the backbone network to obtain the final hidden representation;
[0090] Optionally, the semantic representation of the target relation is extracted based on the natural language description of the target relation, and the expression is:
[0091] ,
[0092] in, This is the semantic representation of the target relation. For pre-trained language models, The word sequence for the target relation;
[0093] The semantic representation of the target relation Inputting the gating network yields the relation-aware editing gating coefficients, expressed as:
[0094]
[0095] in, Edit the gating coefficients for relation-aware editing. This represents the Sigmoid activation function. and These represent the weights and biases of the gating network; wherein the gating network is a lightweight multilayer perceptron.
[0096] The edited hidden representation is dynamically scaled using the relation-aware editing gating coefficients and then fused with the output of the backbone network to obtain the final hidden representation, expressed as:
[0097]
[0098] in, This is the final hidden representation.
[0099] It is understandable that this embodiment introduces a relation-aware editing gating coefficient to reduce the injection strength of the editing signal (the hidden representation after editing) for semantically rigid relation types, and to increase the injection strength of the editing signal for semantically flexible relation types, so as to achieve adaptive editing under different relation types.
[0100] S7: Construct a total loss to jointly optimize the knowledge graph embedding and editing model; wherein, the total loss includes the editing loss of the target fact and the local constraint loss on the set of locality constraints;
[0101] Optionally, the locality constraint loss specifically refers to:
[0102] Excluding the original erroneous fact triples from the local constraint set, and constructing the local constraint loss by comparing the output distribution differences of the model before and after editing on the local constraint set, the local constraint loss adopts the Kullback-Leibler divergence (relative entropy), expressed as:
[0103]
[0104] in, This represents the loss due to local constraints. Represents the set of locality constraints The number of samples; Indicates a non-target query. This represents the tag entity corresponding to the non-target query; and These represent the vector distributions before and after editing, respectively. This indicates the model parameters before editing. This represents the parameters for embedding the knowledge graph after editing. This represents the Kullback-Leibler divergence.
[0105] Optionally, the total loss is specifically:
[0106]
[0107] in, For the total loss, For balance coefficient, For editing losses;
[0108] The editing loss is the negative log-likelihood loss of the target tail entity, expressed as:
[0109]
[0110] in, For the target tail entity, For the target query, To query a given target In this case, the model correctly predicts the target tail entity. The probability of;
[0111] It is understood that this embodiment jointly optimizes the knowledge graph embedding and editing model by constructing a total loss, thereby ensuring the successful editing of target facts while suppressing interference with non-target knowledge.
[0112] S8: Input the query data to be edited into the jointly optimized knowledge graph embedding and editing model to obtain the entity or relation prediction results of the knowledge graph, and realize knowledge editing based on the entity or relation prediction results.
[0113] It is understandable that intelligent question answering and information retrieval systems are prone to reasoning errors and destruction of existing knowledge when performing fact correction. This embodiment S1 solves the technical problem in existing fact correction methods that the lack of explicit modeling of the topological dependencies of the underlying knowledge graph leads to the breakage of multi-hop reasoning links when processing complex facts. Furthermore, through S2-S6, it solves the technical problem that the failure to apply adaptive editing constraints to the features of different entity relationships leads to serious neighborhood knowledge drift and catastrophic forgetting when updating target facts.
[0114] Example 2: Based on the technical solution provided in Example 1, this example verifies the effectiveness of the present invention through experiments based on a specific knowledge editing task application scenario.
[0115] Specifically, this experiment was conducted in an Ubuntu 20.04 environment, using an NVIDIA GeForce RTX4090 graphics processor. The model was implemented using the PyTorch framework and trained on a single GPU. The backbone model used was BERT-base-cased, which contains 12 Transformer layers with 768 hidden layers. An early stopping strategy was employed during training, and the Succ@1 metric on the validation set was used as the model selection criterion.
[0116] Furthermore, during the parameter search process, the learning rate is set to a range of 1e−5 to 1e−3, the batch size is set to 8, 32, 64, and 128, and the low-rank order r is selected from 1, 2, 4, 6, 8, 16, 32, 64, and 128.
[0117] Furthermore, this embodiment employs two datasets specifically designed for knowledge graph embedding and editing tasks: E-FB15k-237 and E-WN18RR. These datasets are derived from the two benchmark knowledge graphs, FB15k-237 and WN18RR, and cover the fact-correction task. Specifically, E-FB15k-237 includes 310,117 pre-trained triples, 3,087 training samples, 3,087 test samples, and 7,051 locality test samples; E-WN18RR includes 93,003 pre-trained triples, 1,491 training samples, 830 test samples, and 5,003 locality test samples.
[0118] Furthermore, the evaluation metrics used in this embodiment include knowledge editing reliability metrics, knowledge locality metrics, and editing efficiency metrics; among which, the knowledge editing reliability metrics include Success@k (Succ@k) and EditedKnowledge Rate of Change (…). Knowledge locality metrics include Retaining Knowledge@k (RK@k) and Retaining Knowledge Rate of Change (RK@k). The editing efficiency metrics include the number of trainable parameters (Params) and the time required for a single edit. These metrics comprehensively validate the effectiveness of the method described in this invention from three aspects: accuracy in editing target facts, ability to retain non-target knowledge, and parameter overhead.
[0119] Furthermore, this embodiment verifies the application effect of the method of the present invention in knowledge editing tasks. For example, in practical application scenarios such as e-commerce product recommendation or intelligent question answering, the underlying system usually relies on knowledge graphs to manage massive amounts of product attributes and related information. With the dynamic changes of real-world business information, the system must have efficient knowledge editing capabilities. Here, knowledge editing refers to the process of updating outdated, erroneous, or changed local entity attributes in the system to the current true objective state. Only by completing fact-correction in a timely manner can the system avoid recommending products with incorrect prices or outdated configurations to users.
[0120] Taking the correction of product attributes in an e-commerce knowledge graph as an example: the target erroneous fact refers to the local knowledge triple in the current graph that does not conform to objective reality, such as (product X, charging interface, old model); while the non-target fact refers to other correct attributes of the product that have not changed and need to remain stable, such as (product X, brand, brand Y). This invention, while correcting the target erroneous fact, effectively avoids catastrophic forgetting of non-target facts by introducing a specific constraint mechanism, and ensures the coherence of topological reasoning links such as related product recommendations (e.g., accessory recommendations). The method of this invention transforms the target triple into a natural language query and generates low-rank editing parameters by the hypernetwork. Under the premise of freezing the main parameters of the backbone network, controlled updates are performed only on the editing branches, thereby completing the specific knowledge correction. Simultaneously, the output offset of non-target facts is constrained through locality constraint loss. Experimental results are shown in Table 1.
[0121] Table 1. Experimental results of this invention on knowledge editing tasks on the E-FB15k-237 and E-WN18RR datasets.
[0122]
[0123] As shown in Table 1, on the E-FB15k-237 dataset, the method of this invention has 16.3M trainable parameters, a single edit time of 0.112 seconds, a Succ@1 score of 0.901, a Succ@3 score of 0.992, an ERRorc score of 0.999, a RK@3 score of 0.896, and a RKroc score of 0.588. On the E-WN18RR dataset, the method of this invention has a single edit time of 0.146 seconds, a Succ@1 score of 0.867, a Succ@3 score of 0.866, an ERRorc score of 0.995, a RK@3 score of 0.972, and a RKroc score of 0.202. These results demonstrate that the method of this invention can not only accurately correct erroneous facts but also effectively preserve the original non-target knowledge.
[0124] Furthermore, comparing the method of this invention with methods such as Knowledge Graph Embedding Fine-tuning (KGE-FT), K-Adapter, CALINET, KE, MEND, AnyEdit, MM4KE, and KGEditor, it can be seen that the method of this invention achieves both high editing reliability and knowledge locality in knowledge editing tasks. Specifically, compared to KGEditor, the method of this invention improves Succ@1 from 0.866 to 0.901 on the E-FB15k-237 dataset and from 0.833 to 0.867 on the E-WN18RR dataset, while maintaining a high RK@3, indicating that the method of this invention achieves a better balance between the accuracy of target knowledge updates and the preservation of non-target knowledge. This demonstrates that the locality constraint loss, low-rank editing transformation, and relation-aware gating collaborative mechanism proposed in this invention can effectively suppress knowledge drift in knowledge editing scenarios, avoiding the structural destruction and catastrophic forgetting problems that are prone to occur in traditional editing methods.
[0125] Example 3: Based on the technical solution provided in Example 1, this example further illustrates the present invention based on a practical knowledge editing example.
[0126] Specifically, based on the knowledge triple to be edited, the original fact is selected as: (Player A, plays for, Team B), where the head entity is Player A, the relation is "plays for", and the tail entity is Team B. Now, due to changes in real-world knowledge, this fact needs to be updated to: (Player A, plays for, Team C), that is, "Player A plays for Team B" is edited to "Player A plays for Team C".
[0127] In traditional editing methods, although the target fact may be successfully modified, it is easy to cause unexpected changes in other knowledge related to player A. For example, the relationship structure of his position, profession, etc. may be disturbed, which will lead to the structural collapse of the knowledge graph embedding space. However, the method of this invention, through low-rank editing branches and structural constraint mechanisms, can keep other knowledge relationships related to the entity basically unchanged while updating the target fact.
[0128] Specifically, in this embodiment, the target edit facts are first converted into edit instructions and input into the supernetwork. The supernetwork generates a corresponding low-rank parameter update matrix based on the target triplet. Then, the generated low-rank edit parameters are loaded into the parallel edit branch. Under the condition of freezing the main parameters of the backbone network, directional corrections are applied only to the local representations corresponding to the target facts. At the same time, combined with the locality constraint loss, constraints are applied to the related neighborhood entities and similar prototype samples of player A, so that the edited model is consistent with the output before editing in non-target facts, thereby avoiding the incorrect rewriting of related knowledge such as "on-field position" and "occupation".
[0129] After editing, the model should output team C for the query "Player A plays for [MASK]", while maintaining the original correct prediction results for non-target queries such as "Player A's position on the field is [MASK]" and "Player A's profession is [MASK]". This demonstrates that the method of this invention can achieve accurate updates to target knowledge while maintaining the consistency of the local structure and multi-relation semantics of the knowledge graph.
[0130] In summary, this invention converts the target editing triples into natural language queries and inputs them into the backbone intervention training language model; it constructs editing branches and a supernetwork corresponding to the target facts, with the supernetwork generating low-rank editing parameters based on editing instructions to perform low-rank updates on the feedforward network in the backbone model; it constructs a set of locality constraints based on target entity neighborhood samples, same-relation prototype samples, and global anchor point samples, using locality constraint loss to limit the shift in the distribution of non-target knowledge before and after editing; it extracts the semantic representation of the target relations and uses a relation-aware editing gating mechanism to adaptively modulate the output of the editing branches; and it trains the model using a combination of editing loss and locality constraint loss to obtain the edited knowledge graph embedding editing model. This invention achieves accurate correction of target facts and injection of new knowledge while effectively maintaining the stability of the original topology and multi-hop inference links of the knowledge graph, reducing the risk of catastrophic forgetting, and significantly reducing the number of trainable parameters and editing computational overhead. It is suitable for knowledge editing tasks in practical application scenarios such as e-commerce product recommendation or intelligent question answering.
[0131] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.
Claims
1. A knowledge editing method based on hypernetwork parameter generation, characterized in that, The method includes the following steps: S1: Construct a knowledge graph embedding and editing model; wherein, the knowledge graph embedding and editing model includes a pre-trained language model backbone network, an editing branch parallel to the pre-trained language model backbone network, and a super network; S2: Convert the obtained target edit triplet into the corresponding natural language query template, and then input the natural language query template into the knowledge graph embedding editing model. Through the pre-trained language model backbone network, obtain the vector representation of the editing instruction corresponding to the target fact. S3: Use the hypernetwork to generate low-rank editing parameters based on the vector representation of the editing instructions; S4: Inject the low-rank editing parameters into the feedforward network layer in the editing branch and the feedforward network layer of the pre-trained language model backbone network to obtain the edited hidden representation of the target fact and the output of the backbone network; S5: Construct a set of local constraints based on the target editing triples; wherein, the set of local constraints includes entity neighborhood samples, same relation prototype samples, and global anchor point samples; S6: Extract the semantic representation of the target relation based on the natural language description of the target relation, construct the relation-aware editing gating coefficient based on the semantic representation, dynamically scale the edited hidden representation using the relation-aware editing gating coefficient, and fuse it with the output of the backbone network to obtain the final hidden representation; S7: Construct a total loss to jointly optimize the knowledge graph embedding and editing model; wherein, the total loss includes the editing loss of the target fact and the local constraint loss on the set of locality constraints; S8: Input the query data to be edited into the jointly optimized knowledge graph embedding and editing model to obtain the entity or relation prediction results of the knowledge graph, and realize knowledge editing based on the entity or relation prediction results.
2. The knowledge editing method based on hypernetwork parameter generation according to claim 1, characterized in that, Specifically, S2 is: Define the target edit triple as ,in, For the head entity, For the relationship, To query the tail entity; Edit the target triplet The header entity and relation are concatenated and converted into declarative text to obtain a natural language query template: the [relation] of [head entity] is [MASK], where [MASK] is a mask; The natural language query template is input into the knowledge graph embedding editing model to obtain a vector representation of the editing instructions corresponding to the target fact.
3. The knowledge editing method based on hypernetwork parameter generation according to claim 1, characterized in that, Specifically, S3 is: The hypernetwork is a two-layer multilayer perceptron. The input dimension is the same as the hidden layer dimension of the pre-trained language model, but the hidden layer dimension is larger than the input dimension. The hypernetwork takes the vector representation of editing instructions as input and outputs a value of [value missing]. After obtaining the vector, the low-rank editing parameters are obtained, which include the low-rank decomposition matrix. and The expression is: ； in, , ,and , It is a low-rank number. For the hidden layer dimension of the pre-trained language model, Indicates a hypernetwork. This is a vector representation of editing instructions.
4. The knowledge editing method based on hypernetwork parameter generation according to claim 3, characterized in that, Specifically, S4 is: After generating low-rank editing parameters, the pre-trained language model backbone network keeps its weights frozen. These low-rank editing parameters are then injected into the feedforward network layer in the editing branch and the feedforward network layer of the pre-trained language model backbone network, resulting in: ； ； in, For the output of the backbone network, This represents the activation function. This is the original weight matrix. This is the original offset. The input is hidden. The output of the edit branch represents the hidden representation after editing. This is the offset. The expression for the model parameter update amount is: ； in, This is the scaling factor.
5. The knowledge editing method based on hypernetwork parameter generation according to claim 1, characterized in that, Specifically, S5 is: The expression for the set of locality constraints is: ； in, For a set of local constraints, For entity neighborhood samples, This is a prototype sample of the same relation. This is a global anchor point sample; in: Entity neighborhood samples It consists of knowledge graph triples that share a head entity or query tail entity with the target editing triple, but have different relationships; Same relation prototype sample It consists of knowledge graph triples that have the same relationship as the target edit triples, and whose head entity and query tail entity are different; Global anchor point samples It consists of knowledge graph triples that are structurally separate from the target edit triples and whose head entity, query tail entity, and relation are all different.
6. The knowledge editing method based on hypernetwork parameter generation according to claim 4, characterized in that, Specifically, S6 is: Natural language description based on target relations extracts the semantic representation of the target relation, expressed as: ； in, This is the semantic representation of the target relation. For pre-trained language models, The word sequence for the target relation; The semantic representation of the target relation Inputting the gating network yields the relation-aware editing gating coefficients, expressed as: ； in, Edit the gating coefficients for relation-aware editing. This represents the Sigmoid activation function. and These represent the weights and biases of the gating network; wherein the gating network is a lightweight multilayer perceptron. The edited hidden representation is dynamically scaled using the relation-aware editing gating coefficients and then fused with the output of the backbone network to obtain the final hidden representation, expressed as: ； in, This is the final hidden representation.
7. The knowledge editing method based on hypernetwork parameter generation according to claim 1, characterized in that, The local constraint loss is specifically as follows: Excluding the original erroneous fact triples from the local constraint set, and constructing the local constraint loss by comparing the output distribution differences of the model before and after editing on the local constraint set, the local constraint loss adopts the Kullback-Leibler divergence, expressed as: ； in, This represents the loss due to local constraints. Represents the set of locality constraints The number of samples; Indicates a non-target query. This represents the tag entity corresponding to the non-target query; and These represent the vector distributions before and after editing, respectively. This indicates the model parameters before editing. This represents the parameters for embedding the knowledge graph after editing. This represents the Kullback-Leibler divergence.
8. The knowledge editing method based on hypernetwork parameter generation according to claim 7, characterized in that, The total loss is specifically as follows: ； in, For the total loss, For balance coefficient, For editing losses; The editing loss is the negative log-likelihood loss of the target tail entity, expressed as: ； in, For the target tail entity, For the target query, To query a given target In this case, the model correctly predicts the target tail entity. The probability of.