Fine adjustment method and device for knowledge graph inference model, medium and program product
By extracting part of triple data from the original knowledge graph and combining it with the added data, a fine-tuning data set is constructed and the knowledge graph inference model is fine-tuning, which solves the problems of high fine-tuning costs and poor inference performance in the existing technology, and achieves efficient and low-cost model updates.
Patent Information
- Application Number
- CN202411973351.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-04-29
AI Technical Summary
When the knowledge graph is updated, the existing technology needs to retrain the inference model, resulting in high costs; and using only new knowledge for fine-tuning cannot guarantee the inference performance of the model.
By extracting part of triple data from the original knowledge graph and building a fine-tuning data set with the newly added triple data, fine-tuning the inference model to obtain the updated model.
While ensuring the model inference performance, the cost of model fine-tuning is reduced and the catastrophic forgetting problem of the inference model is avoided.
Smart Images
Figure CN119903905A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of model fine-tuning technology, and specifically, to a method, device, medium and program product for fine-tuning a knowledge graph reasoning model. Background Art
[0002] With the rapid development and widespread application of Internet technology, network security issues have increasingly become the focus of attention from all walks of life. In order to effectively deal with the security threats brought by cyber attacks, researchers in the field of network security have been exploring more intelligent and efficient security protection methods. In recent years, artificial intelligence-based technologies, especially machine learning and deep learning methods, have shown great potential in the field of network security detection and defense.
[0003] As a structured semantic knowledge base, knowledge graph can effectively represent entities and the relationships between them, which provides a new perspective for people to understand the complex network security environment. By building a network security knowledge graph, we can not only integrate heterogeneous security information from different sources, but also discover potential security threats and attack patterns through the knowledge graph reasoning model.
[0004] Currently, when the knowledge data of the knowledge graph is updated, it is usually necessary to retrain the inference model using the updated full data. However, this will make the cost of retraining the model too high. If only the newly added knowledge is used to fine-tune the model, the inference performance of the model cannot be guaranteed. Summary of the invention
[0005] The purpose of the embodiments of the present application is to provide a method, device, medium and program product for fine-tuning a knowledge graph reasoning model, which can reduce the cost of model fine-tuning while ensuring the model reasoning performance.
[0006] In a first aspect, an embodiment of the present application provides a method for fine-tuning a knowledge graph reasoning model, including:
[0007] In response to an inference model update instruction triggered by a knowledge graph update event, obtaining a newly added triple data set corresponding to the knowledge graph update event;
[0008] Determine the amount of triple data that needs to be extracted from the original knowledge graph, obtain an original triple data set from the original knowledge graph based on the amount of triple data, and construct a fine-tuning data set based on the original triple data set and the newly added triple data set;
[0009] The original reasoning model is fine-tuned based on the fine-tuning dataset to obtain an updated reasoning model.
[0010] In an embodiment of the present application, by extracting some original triple data from the original knowledge graph and fine-tuning the inference model based on these original triple data in combination with the newly added triple data, the cost of model fine-tuning can be reduced while ensuring the model reasoning performance.
[0011] In some possible embodiments, fine-tuning the original reasoning model based on the fine-tuning dataset to obtain an updated reasoning model includes:
[0012] Acquire a to-be-fine-tuned reasoning model based on the original reasoning model;
[0013] Inputting the fine-tuning data set into the original reasoning model and the reasoning model to be fine-tuned respectively;
[0014] Obtain a first feature extraction result set of the original reasoning model for the fine-tuning data set, and at the same time, obtain a second feature extraction result set of the to-be-fine-tuned reasoning model for the fine-tuning data set in combination with feature extraction information of the original reasoning model for the fine-tuning data set;
[0015] Determine an output difference between the original inference model and the inference model to be fine-tuned based on the first feature extraction result set and the second feature extraction result set, and determine a model fine-tuning loss function by combining the output difference with an original loss function of the original inference model;
[0016] The model parameters of the inference model to be fine-tuned are trained based on the model fine-tuning loss function to obtain an updated inference model.
[0017] In an embodiment of the present application, by combining the feature extraction information of the original reasoning model in the process of fine-tuning the new model, it is possible to learn new knowledge while retaining the experience of the old model, thereby further improving the reasoning performance of the fine-tuned model.
[0018] In some possible embodiments, combining the feature extraction information of the fine-tuning data set with the original inference model to obtain a second feature extraction result set of the to-be-fine-tuned inference model for the fine-tuning data set includes:
[0019] Merging the first feature extraction information output by each feature extraction network layer in the original reasoning model with the second feature extraction information output by each corresponding feature extraction network layer in the to-be-fine-tuned reasoning model;
[0020] The feature extraction information after merging the feature extraction network layers in the to-be-fine-tuned reasoning model is used as the input information of the next feature extraction network layer, and finally a second feature extraction result set of the to-be-fine-tuned reasoning model for the fine-tuning data set is obtained.
[0021] In an embodiment of the present application, by using the features extracted from each layer of the new model and the features extracted from the corresponding layer of the old model as the input of the next layer, the knowledge learned by the old model can be effectively utilized, thereby further improving the reasoning performance of the fine-tuned model.
[0022] In some possible embodiments, determining the output difference between the original reasoning model and the to-be-fine-tuned reasoning model based on the first feature extraction result set and the second feature extraction result set includes:
[0023] The output difference between the original inference model and the inference model to be fine-tuned is determined based on the first feature extraction result set and the second feature extraction result set by using a measurement method of the Pearson correlation coefficient.
[0024] In the embodiment of the present application, by using the Pearson correlation system to measure the difference in output between the new and old models, the efficiency of training the model and the reasoning ability of the model can be further improved.
[0025] In some possible embodiments, determining the amount of triple data currently required to be extracted from the original knowledge graph includes:
[0026] Based on the preset triple data amount optimization problem, the result obtained by optimizing using the preset simulated annealing algorithm is used as the triple data amount that currently needs to be extracted from the original knowledge graph.
[0027] In the embodiment of the present application, the efficiency of the training model can be further improved by using a simulated annealing algorithm to optimize and obtain a suitable amount of triple data.
[0028] In some possible embodiments, the result obtained by optimizing the preset triple data amount using the preset simulated annealing algorithm as the triple data amount currently required to be extracted from the original knowledge graph includes:
[0029] Initialization steps:
[0030] Randomly generate several initial triplet data volumes as the current data volume set, and initialize the initial temperature of the simulated annealing algorithm;
[0031] Taking each of the initial triplet data amounts as the local optimal solution corresponding to the current data amount set;
[0032] Calculate the model loss value of each of the initial triple data sets respectively, and take the initial triple data set with the smallest model loss value as the global optimal solution of the current data set;
[0033] Candidate solution generation steps:
[0034] Generate corresponding candidate solutions for each triplet of data in the current data set;
[0035] Data volume update steps:
[0036] Calculate the model loss value of each triple data volume and the corresponding candidate solution respectively, and update each triple data volume to the corresponding candidate solution or keep it unchanged based on the comparison of the model loss value and the preset jump probability rule;
[0037] Optimal solution update steps:
[0038] Re-determine the corresponding local optimal solution and global optimal solution based on the model loss value of each triple data set after the update;
[0039] Temperature update steps:
[0040] Updating the temperature of the simulated annealing algorithm, and updating the exit probability rule based on the updated temperature;
[0041] The candidate solution generating step, the data amount updating step, the optimal solution updating step and the temperature updating step are executed in a loop until a preset loop termination condition is reached, and the global optimal solution obtained after the loop termination is used as the triple data amount currently required to be extracted from the original knowledge graph.
[0042] In an embodiment of the present application, by using the optimal solution obtained by optimizing using an annealing algorithm as the final triple data volume, the efficiency of the training model and the model reasoning ability can be further improved.
[0043] In some possible embodiments, after fine-tuning the original reasoning model based on the fine-tuning dataset to obtain an updated reasoning model, the method further includes:
[0044] In response to the knowledge graph reasoning task, determining known triple elements and initial screening conditions based on the knowledge graph reasoning task;
[0045] Based on the known triple elements, a number of candidate triple elements that meet the initial screening conditions are obtained from the updated knowledge graph; wherein the updated knowledge graph is obtained by updating the original knowledge graph based on the newly added triple data set;
[0046] Constructing candidate triples corresponding to each candidate triple element based on the known triple elements;
[0047] Based on the updated reasoning model, the scoring results corresponding to each candidate triple are calculated respectively, and the reasoning results corresponding to the knowledge graph reasoning task are determined based on the scoring results.
[0048] In an embodiment of the present application, by preliminarily screening candidate triple elements according to the reasoning task, and then using the preliminarily screened triple elements to construct candidate triples for scoring and determining reasoning results, the efficiency of knowledge reasoning can be further improved.
[0049] In a second aspect, an embodiment of the present application provides a fine-tuning device for a knowledge graph reasoning model, comprising:
[0050] An update trigger module, used to respond to an inference model update instruction triggered by a knowledge graph update event, and obtain a newly added triple data set corresponding to the knowledge graph update event;
[0051] A data set construction module, used to determine the amount of triple data currently required to be extracted from the original knowledge graph, obtain an original triple data set from the original knowledge graph based on the triple data amount, and construct a fine-tuning data set based on the original triple data set and the newly added triple data set;
[0052] The model fine-tuning module is used to fine-tune the original reasoning model based on the fine-tuning data set to obtain an updated reasoning model.
[0053] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement the method described in any embodiment of the first aspect when executing the program.
[0054] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in any embodiment of the first aspect can be implemented.
[0055] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program, wherein when the computer program is executed by a processor, the method described in any embodiment of the first aspect can be implemented. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0057] Figure 1 A flowchart of a method for fine-tuning a knowledge graph reasoning model provided in an embodiment of the present application;
[0058] Figure 2 A system architecture diagram of the knowledge graph reasoning model provided in the embodiment of the present application;
[0059] Figure 3 A diagram of the model fine-tuning architecture provided in an embodiment of the present application;
[0060] Figure 4 A schematic diagram of the structure of a fine-tuning device for a knowledge graph reasoning model provided in an embodiment of the present application;
[0061] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0062] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.
[0063] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0064] It should be noted that by constructing a network security knowledge graph, we can not only integrate heterogeneous security information from different sources, but also discover potential security threats and attack patterns through knowledge graph reasoning.
[0065] In practical applications, it is often necessary to continuously collect information and update the knowledge graph. After updating the knowledge graph, in order to ensure the timeliness of the reasoning model based on the knowledge graph, it is necessary to fine-tune the reasoning model based on the updated knowledge graph.
[0066] At present, the existing technology either uses the updated knowledge graph to retrain the reasoning model, which consumes a lot of training time and computing resources and is costly; or only uses the newly added knowledge to fine-tune the reasoning model. Although the fine-tuned model can learn the newly added knowledge, it may cause catastrophic forgetting of the old knowledge and cannot guarantee the reasoning performance of the model.
[0067] In response to the problems existing in the above-mentioned prior art, an embodiment of the present application provides a method for fine-tuning a knowledge graph reasoning model. By utilizing the newly added knowledge and part of the original knowledge to construct a fine-tuning dataset, and then fine-tuning the reasoning model based on the fine-tuning dataset, not only the time cost and computing resources required for fine-tuning the model are reduced, but also the learning of the new knowledge and old knowledge by the reasoning model is taken into account, thereby reducing the cost of model fine-tuning while ensuring the model reasoning performance.
[0068] like Figure 1 As shown, the embodiment of the present application provides a method for fine-tuning a knowledge graph reasoning model, which may include the following steps:
[0069] S1. In response to the inference model update instruction triggered by the knowledge graph update event, obtain the newly added triple data set corresponding to the knowledge graph update event.
[0070] Specifically, the knowledge graph update event, which is the amount of new knowledge added to the knowledge graph, can be used as a basis for triggering the inference model update instruction. Exemplarily, a knowledge graph update event can be generated each time new knowledge is added to the knowledge graph to trigger the inference model update instruction; optionally, when the cumulative amount of new knowledge added to the knowledge graph reaches a preset threshold, a corresponding knowledge graph update event can be generated to trigger the inference model update instruction.
[0071] It is understandable that when an event of new knowledge being added to the knowledge graph or a certain amount of new knowledge is added, in order to ensure the timeliness of the reasoning model, it is necessary to retrain the reasoning model based on the new knowledge (triggering the reasoning model update instruction) to update the reasoning model.
[0072] The reasoning model of the embodiment of the present application refers to a model for inferring and outputting a third element for two elements in a given triple, wherein the reasoning model is trained based on a knowledge graph. Exemplarily, it is assumed that the knowledge graph G = {(h, r, t)} consists of multiple triple data (h, r, t), where h represents the head entity, r represents the relationship, and t represents the tail entity; the knowledge graph reasoning task is: given two elements in a triple, infer the third element.
[0073] For example, the inference model framework of the embodiment of the present application is as follows: Figure 2 As shown in Figure 1, the feature extraction of input data can be implemented by using an autoencoder. First, the triple data is initialized into a vector form, where the initial vector of the head entity h is represented by V h , the initial matrix of relation r is represented as V r , the initial vector of the tail entity t is represented by V t ; By using the autoencoder to learn the features of the triple data, the final feature result extracted by the autoencoder is: the feature vector corresponding to the head entity h The characteristic matrix corresponding to the relation r The feature vector V corresponding to the tail entity t t d .
[0074] Then, based on the extracted feature results, the DistMult model can be used to score the corresponding triple data, where the scoring function can be expressed as:
[0075] The optimization during model training is divided into two parts, including:
[0076] (1) Maximize the score of the correct triples in the knowledge graph triples, while minimizing the score of the wrong triples. The loss function can be defined using binary cross entropy, which is expressed as:
[0077]
[0078] Among them, n represents the number of triplet data involved in the current training, y i Indicates the type of the current triple, y i =1 means the current triple is the correct triple, y i =0 means the current triplet is an incorrect triplet; σ is the sigmoid function.
[0079] (2) Minimize the reconstruction error between the input and output of the autoencoder. The loss function is expressed as:
[0080]
[0081] To sum up the above two parts, the total loss function during model training is expressed as: loss = L1 + L2.
[0082] When the inference model update instruction is triggered, the newly added triple data set corresponding to the current knowledge graph update event is obtained, wherein the newly added triple data set refers to the triple data formed by one or more newly added knowledge.
[0083] S2. Determine the amount of triple data that needs to be extracted from the original knowledge graph, obtain the original triple data set from the original knowledge graph based on the triple data set, and construct a fine-tuning data set based on the original triple data set and the newly added triple data set.
[0084] It should be noted that the original knowledge graph refers to the original knowledge graph before the newly added knowledge. The triple data volume refers to the number of triple data that needs to be extracted from the original knowledge graph this time. Optionally, the triple data volume can be determined based on the newly added knowledge volume, or.
[0085] By determining a triple data volume, a corresponding amount of original triple data can be randomly obtained from the original knowledge graph to form an original triple data set. Then, the original triple data set and the newly added triple data set are combined to construct a fine-tuning data set for retraining or fine-tuning the inference model.
[0086] S3. Fine-tune the original inference model based on the fine-tuning dataset to obtain an updated inference model.
[0087] It should be noted that the original reasoning model refers to the reasoning model obtained by training using the original knowledge graph. Based on the original reasoning model, the original reasoning model is updated using the fine-tuning dataset constructed this time to obtain an updated reasoning model. Among them, the method of updating the reasoning model can be based on the original training method of the original reasoning model, using the fine-tuning dataset as a training sample to retrain the original reasoning model to obtain an updated reasoning model.
[0088] Based on this, by extracting some original triple data from the original knowledge graph and fine-tuning the reasoning model based on these original triple data combined with the newly added triple data, there is no need to use all the updated knowledge graph data to retrain the model, which effectively reduces the cost of model fine-tuning. In addition, since the impact of both new and old knowledge on the reasoning model is taken into account, the problem of catastrophic forgetting in the reasoning model is avoided, thereby ensuring the reasoning performance of the model.
[0089] In some possible embodiments, step S3, fine-tuning the original reasoning model based on the fine-tuning data set to obtain an updated reasoning model, may include:
[0090] S301, obtaining a to-be-fine-tuned reasoning model based on the original reasoning model;
[0091] S302, inputting the fine-tuning data set into the original reasoning model and the reasoning model to be fine-tuned respectively;
[0092] S303, obtaining a first feature extraction result set of the original reasoning model for the fine-tuning data set, and at the same time, combining the feature extraction information of the original reasoning model for the fine-tuning data set, obtaining a second feature extraction result set of the to-be-fine-tuned reasoning model for the fine-tuning data set;
[0093] S304, determining the output difference between the original inference model and the inference model to be fine-tuned based on the first feature extraction result set and the second feature extraction result set, and determining the model fine-tuning loss function by combining the output difference and the original loss function of the original inference model;
[0094] S305 . Training the model parameters of the inference model to be fine-tuned based on the model fine-tuning loss function to obtain an updated inference model.
[0095] Furthermore, in step S303, combining the feature extraction information of the original inference model on the fine-tuning data set to obtain a second feature extraction result set of the to-be-fine-tuned inference model on the fine-tuning data set may include:
[0096] S3031, merging the first feature extraction information output by each feature extraction network layer in the original reasoning model with the second feature extraction information output by each corresponding feature extraction network layer in the reasoning model to be fine-tuned;
[0097] S3032. The feature extraction information after merging the feature extraction network layers in the to-be-fine-tuned reasoning model is used as the input information of the next feature extraction network layer, and finally a second feature extraction result set of the to-be-fine-tuned reasoning model for the fine-tuning data set is obtained.
[0098] Furthermore, in step S304, determining the output difference between the original inference model and the inference model to be fine-tuned based on the first feature extraction result set and the second feature extraction result set may include:
[0099] S3041. Using the Pearson correlation coefficient measurement method, determine the output difference between the original inference model and the inference model to be fine-tuned based on the first feature extraction result set and the second feature extraction result set.
[0100] Specifically, first, the original inference model is copied to obtain an inference model to be fine-tuned.
[0101] Then, if Figure 3 As shown, in the process of fine-tuning the model, the training data is input into the original reasoning model and the reasoning model to be fine-tuned respectively, and the feature extraction information of the same layer of the original reasoning model and the reasoning model to be fine-tuned is merged as the input of the next layer of the reasoning model to be fine-tuned.
[0102] During the training process, it is necessary to fix the weights of the original inference model and continuously adjust the model parameters (such as the weights of the network layer) of the inference model to be fine-tuned according to information such as the loss function until the iteration termination condition is reached, such as the number of training times reaches the set upper limit or the loss value is lower than the set threshold, to obtain an updated inference model.
[0103] It should be noted that in the process of fine-tuning the model, the optimization target of the training model consists of two parts, one of which is the same loss function as in the original inference model training process, and the other is the difference between the predicted values of the fine-tuned inference model and the original inference model for the same training data. The predicted value here refers to the triple feature vector (feature extraction result) output by the autoencoder in the inference model.
[0104] Exemplarily, the difference can be measured using the Pearson correlation coefficient. Specifically, for the triple data (h, r, t) in the fine-tuning dataset, the feature vector output by the autoencoder in the original inference model (old model) is recorded as last, and the feature vector output by the autoencoder in the inference model to be fine-tuned (new model) is recorded as now. The difference between the predicted values of the inference model to be fine-tuned and the original inference model for the fine-tuning dataset can be expressed by the following formula:
[0105]
[0106] Among them, last i 、now i are the feature vectors output by the original inference model and the inference model to be fine-tuned for the i-th element of the fine-tuning dataset, respectively. are the average values of the feature vectors last and now respectively.
[0107] Then, combined with the original loss function of the original inference model and the output difference between the original inference model and the inference model to be fine-tuned, the loss function of the model fine-tuning process can be determined based on the preset weights: Γ = loss + δ·loss p . Among them, loss represents the original loss function of the original inference model, and δ is an adjustable weight parameter.
[0108] Based on this, by combining the feature extraction information of the original inference model to fine-tune the new model, it is possible to learn new knowledge while retaining the experience of the old model, thereby further improving the inference performance of the updated model.
[0109] In some possible embodiments, in step S2, determining the amount of triple data currently required to be extracted from the original knowledge graph may include:
[0110] S201. Based on the preset triple data amount optimization problem, the result obtained by optimizing using the preset simulated annealing algorithm is used as the triple data amount that currently needs to be extracted from the original knowledge graph.
[0111] Furthermore, in step S201, based on the preset triple data amount optimization problem, the result obtained by optimizing using the preset simulated annealing algorithm as the triple data amount currently required to be extracted from the original knowledge graph may include:
[0112] Initialization steps:
[0113] Randomly generate several initial triplet data volumes as the current data volume set, and initialize the initial temperature of the simulated annealing algorithm;
[0114] The data volume of each initial triplet is taken as the local optimal solution corresponding to the current data volume set;
[0115] Calculate the model loss value of each initial triple data set respectively, and take the initial triple data set with the smallest model loss value as the global optimal solution of the current data set;
[0116] Candidate solution generation steps:
[0117] Generate corresponding candidate solutions for each triplet of data in the current data set;
[0118] Data volume update steps:
[0119] Calculate the data volume of each triple and the model loss value of the corresponding candidate solution respectively, and update the data volume of each triple to the corresponding candidate solution or keep it unchanged based on the comparison of the model loss value and the preset jump probability rule;
[0120] Optimal solution update steps:
[0121] Re-determine the corresponding local optimal solution and global optimal solution based on the model loss value of each triple data set after the update;
[0122] Temperature update steps:
[0123] Update the temperature of the simulated annealing algorithm, and update the exit probability rule based on the updated temperature;
[0124] The candidate solution generation step, data volume update step, optimal solution update step and temperature update step are executed cyclically until the preset loop termination condition is reached, and the global optimal solution obtained after the loop termination is used as the triple data volume that needs to be extracted from the original knowledge graph.
[0125] It should be noted that the constructed fine-tuning dataset consists of two parts: one part is the newly added triple data, and the other part is some triple data extracted from the original knowledge graph.
[0126] It is understandable that in order to ensure the performance of the fine-tuned inference model while reducing the cost of model fine-tuning, it is necessary to reduce the number of original triple data extracted (triple data volume) while minimizing the loss of the model as much as possible, so it is necessary to determine an appropriate triple data volume. The embodiment of the present application uses a simulated annealing algorithm to automatically adjust the size of the triple data volume, and finally obtains an appropriate triple data volume.
[0127] Specifically, assuming that the amount of triple data is represented by K, the algorithm flow is as follows:
[0128] 1. Initialization steps:
[0129] (1) Randomly set n triples of data K = {K1, K2, ..., K n} as multiple initial solutions of the algorithm and set the initial temperature T of the algorithm.
[0130] (2) The amount of data for each triple K n ∈K is recorded as the local optimal solution corresponding to the current data volume n .
[0131] (3) Calculate the model loss corresponding to all triple data volumes K (using the above loss function loss), and record the triple data volume with the lowest model loss value as the global optimal solution global for all triple data volumes K.
[0132] 2. Steps to generate candidate solutions:
[0133] For each triple data volume K n ∈K, generate its new candidate solution K' n In this embodiment of the present application, the local optimal solution corresponding to the amount of data of each triple is local n And the global optimal solution of all triple data to generate a new candidate solution K' n The candidate solution generation formula is as follows:
[0134] K' n =K n +c1×(localn-K n )+c2×(global-K n )
[0135] Among them, c1 and c2 are configurable parameters.
[0136] 3. Data volume update steps:
[0137] Calculate when the number of triples is K n and K' n , the corresponding model loss values Γ and Γ′.
[0138] (1) If Γ′<Γ, then the triple data volume K n Update to the corresponding candidate solution K' n ;
[0139] (2) If Γ′≥Γ, a probability value θ between 0 and 1 is randomly generated, and a judgment is made based on the preset exit probability rule: If Then the triple data volume K n Update to the corresponding candidate solution K' n , otherwise maintain the triple data volume as K n .
[0140] 4. Optimal solution update steps:
[0141] (1) Update the local optimal solution: For each triple data volume K n , if the candidate solution K' n The model loss value is less than K n The model loss value is updated locally n K' n , otherwise local n constant.
[0142] (2) Update the global optimal solution: Update the global optimal solution from all local optimal solutions n The solution with the smallest model loss value is selected as the new global optimal solution global.
[0143] 5. Temperature update steps:
[0144] The embodiment of the present application adopts an adaptive temperature update strategy to adjust the temperature according to the difference between the model loss values of the new and old models. The specific update strategy is as follows:
[0145] (1) Initialization. Set the proportional factor α to control the temperature drop rate. α is a constant less than 1. Set the temperature adjustment coefficient β to adjust the temperature according to the difference in model loss. β is a positive proportional coefficient.
[0146] (2) Temperature update. If Γ′<Γ, or Γ′≥Γ, but K n If not updated, then T = T·α; if Γ′≥Γ, and K n Update to K' n , then T=T+β·|Γ-Γ′|.
[0147] 6. Repeat steps 2 to 5 of the above process until the preset loop termination condition is reached, such as the maximum number of iterations is reached or the temperature drops to a preset threshold, and output the final global optimal solution as the amount of triple data that needs to be extracted from the original knowledge graph.
[0148] In some possible embodiments, after step S3, the following steps may also be included:
[0149] S401, in response to the knowledge graph reasoning task, determining known triple elements and initial screening conditions based on the knowledge graph reasoning task;
[0150] S402, based on the known triple elements, obtaining a number of candidate triple elements that meet the initial screening conditions from the updated knowledge graph; wherein the updated knowledge graph is obtained by updating the original knowledge graph based on the newly added triple data set;
[0151] S403, constructing candidate triples corresponding to each candidate triple element based on the known triple elements;
[0152] S404. Calculate the scoring results corresponding to each candidate triple based on the updated reasoning model, and determine the reasoning results corresponding to the knowledge graph reasoning task based on the scoring results.
[0153] It should be noted that when using the updated reasoning model to perform reasoning tasks, the triple elements in the knowledge graph can be preliminarily screened based on the preset rules, and then the reasoning model is scored and the output results are output based on the results of the preliminarily screened. The triple element refers to the head entity, relationship or tail entity in a triple data.
[0154] Exemplarily, the entity or relationship types in the knowledge graph can be classified and managed. When initiating a knowledge graph reasoning task, the entity type or relationship type of the reasoning result can be specified. At this time, all three-element elements (entities or relationships) that meet the type conditions can be preliminarily screened from the knowledge graph based on the specified type.
[0155] Specifically, suppose the current knowledge graph reasoning task is: given the head entity h and relation r in a triple, reason about the triple<h,r,?> The tail entity in , where the tail entity type that needs to be inferred is specified as type x.
[0156] First, search for all possible tail entities of type x from the knowledge graph to form a candidate tail entity set candidate. Then, traverse the candidate tail entity set candidate, and for each c∈candidate, use the updated scoring function of the inference model (refer to the scoring function in step S1) to evaluate the triples.<s,r,c> Scoring is performed, and the top k (set according to requirements) triples with the highest scores are output as reasoning results.
[0157] Based on this, by preliminarily screening the candidate triple elements according to the entity or relationship type, and then using the preliminarily screened triple elements to construct candidate triples for scoring and determining the reasoning results, the efficiency of knowledge reasoning can be further improved.
[0158] Taking the threat intelligence correlation analysis scenario as an example, the following is a detailed description of the fine-tuning and reasoning process of the knowledge graph reasoning model:
[0159] 1. Build a network security knowledge graph: collect information such as assets, intelligence, vulnerabilities, etc. to build a network security knowledge graph.
[0160] 2. Network security knowledge graph reasoning model training: Build a network security time series knowledge graph reasoning model, build a training set based on the knowledge graph, and train the reasoning model until the stopping condition of the model training is reached.
[0161] 3. Fine-tuning of the network security knowledge graph reasoning model: When a new event occurs, the knowledge graph is updated based on the newly added knowledge. At the same time, a fine-tuning dataset is constructed based on the newly added knowledge and part of the knowledge of the original knowledge graph, and the reasoning model is fine-tuned based on the fine-tuning dataset until the preset stopping condition is reached.
[0162] 4. Prediction of the network security knowledge graph reasoning model: Input the reasoning task, such as the reasoning question "Which asset does organization A control?", then the triple to be reasoned is <organization A, control,?>. Search all possible candidate tail entities C in the knowledge graph based on the triple to be reasoned. Input the triple to be reasoned into the reasoning model, use the scoring function to score all triples to be reasoned <organization A, control, C>, and take the candidate tail entities corresponding to the top k triples with the highest scores as the reasoning results of the model.
[0163] Please refer to Figure 4 , Figure 4 The following is a block diagram showing the composition of a fine-tuning device for a knowledge graph reasoning model provided in some embodiments of the present application. It should be understood that the fine-tuning device for the knowledge graph reasoning model is similar to the above-mentioned Figure 1 Corresponding to the method embodiment, it is able to execute each step involved in the above method embodiment. The specific functions of the fine-tuning device of the knowledge graph reasoning model can be found in the description above. In order to avoid repetition, the detailed description is appropriately omitted here.
[0164] Figure 4 The fine-tuning device of the knowledge graph reasoning model includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the fine-tuning device of the knowledge graph reasoning model, and the fine-tuning device of the knowledge graph reasoning model includes:
[0165] An update trigger module 410 is used to respond to an inference model update instruction triggered by a knowledge graph update event and obtain a newly added triple data set corresponding to the knowledge graph update event;
[0166] The data set construction module 420 is used to determine the amount of triple data that needs to be extracted from the original knowledge graph, obtain the original triple data set from the original knowledge graph based on the triple data amount, and construct a fine-tuning data set based on the original triple data set and the newly added triple data set;
[0167] The model fine-tuning module 430 is used to fine-tune the original reasoning model based on the fine-tuning data set to obtain an updated reasoning model.
[0168] It can be understood that the above-mentioned device item embodiment corresponds to the method item embodiment of the present invention. The fine-tuning device of a knowledge graph reasoning model provided by the embodiment of the present invention can implement the fine-tuning method of the knowledge graph reasoning model provided by any method item embodiment of the present invention.
[0169] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.
[0170] like Figure 5 As shown, some embodiments of the present application provide an electronic device 500, which includes: a memory 510, a processor 520, and a computer program stored in the memory 510 and executable on the processor 520, wherein the processor 520 reads the program from the memory 510 through a bus 530 and executes the program to implement a method of any embodiment of the fine-tuning method of the knowledge graph reasoning model as described above.
[0171] Processor 520 can process digital signals and can include various computing structures, such as complex instruction set computer structure, reduced instruction set computer structure, or a structure that implements a combination of multiple instruction sets. In some examples, processor 520 can be a microprocessor.
[0172] The memory 510 may be used to store instructions executed by the processor 520 or data related to the execution of instructions. These instructions and / or data may include codes for implementing some or all functions of one or more modules described in the embodiments of the present application. The processor 520 of the disclosed embodiment may be used to execute instructions in the memory 510 to implement the method shown above. The memory 510 includes a dynamic random access memory, a static random access memory, a flash memory, an optical memory, or other memory known to those skilled in the art.
[0173] Some embodiments of the present application further provide a computer-readable storage medium having a computer program stored thereon. The computer program is executed by a processor to execute the method described in the method embodiment.
[0174] Some embodiments of the present application further provide a computer program product, which, when executed on a computer, enables the computer to execute the method described in the method embodiment.
[0175] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0176] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0177] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
[0178] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0179] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.
[0180] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0181] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
Claims
1. A fine-tuning method for a knowledge graph reasoning model, characterized in that: include: In response to an inference model update instruction triggered by a knowledge graph update event, obtaining a newly added triple data set corresponding to the knowledge graph update event; Determine the amount of triple data that needs to be extracted from the original knowledge graph, obtain an original triple data set from the original knowledge graph based on the amount of triple data, and construct a fine-tuning data set based on the original triple data set and the newly added triple data set; The original reasoning model is fine-tuned based on the fine-tuning dataset to obtain an updated reasoning model.
2. The fine-tuning method of the knowledge graph reasoning model according to claim 1 is characterized in that: The fine-tuning the original reasoning model based on the fine-tuning data set to obtain an updated reasoning model includes: Acquire a to-be-fine-tuned reasoning model based on the original reasoning model; Inputting the fine-tuning data set into the original reasoning model and the reasoning model to be fine-tuned respectively; Obtain a first feature extraction result set of the original reasoning model for the fine-tuning data set, and at the same time, obtain a second feature extraction result set of the to-be-fine-tuned reasoning model for the fine-tuning data set in combination with feature extraction information of the original reasoning model for the fine-tuning data set; Determine an output difference between the original inference model and the inference model to be fine-tuned based on the first feature extraction result set and the second feature extraction result set, and determine a model fine-tuning loss function by combining the output difference with an original loss function of the original inference model; The model parameters of the inference model to be fine-tuned are trained based on the model fine-tuning loss function to obtain an updated inference model.
3. The fine-tuning method of the knowledge graph reasoning model according to claim 2 is characterized in that: The step of extracting feature information of the fine-tuning data set by combining the original inference model with the feature extraction information of the fine-tuning data set to obtain a second feature extraction result set of the to-be-fine-tuned inference model for the fine-tuning data set includes: Merging the first feature extraction information output by each feature extraction network layer in the original reasoning model with the second feature extraction information output by each corresponding feature extraction network layer in the to-be-fine-tuned reasoning model; The feature extraction information after merging the feature extraction network layers in the to-be-fine-tuned reasoning model is used as the input information of the next feature extraction network layer, and finally a second feature extraction result set of the to-be-fine-tuned reasoning model for the fine-tuning data set is obtained.
4. The fine-tuning method of the knowledge graph reasoning model according to claim 2 is characterized in that: The determining the output difference between the original reasoning model and the to-be-fine-tuned reasoning model based on the first feature extraction result set and the second feature extraction result set includes: The output difference between the original inference model and the inference model to be fine-tuned is determined based on the first feature extraction result set and the second feature extraction result set by using a measurement method of the Pearson correlation coefficient.
5. The fine-tuning method of the knowledge graph reasoning model according to claim 1, characterized in that: The step of determining the amount of triple data that needs to be extracted from the original knowledge graph includes: Based on the preset triple data amount optimization problem, the result obtained by optimizing using the preset simulated annealing algorithm is used as the triple data amount that currently needs to be extracted from the original knowledge graph.
6. The fine-tuning method of the knowledge graph reasoning model according to claim 5, characterized in that: The result obtained by optimizing the preset triple data amount using the preset simulated annealing algorithm as the triple data amount currently required to be extracted from the original knowledge graph includes: Initialization steps: Randomly generate several initial triplet data volumes as the current data volume set, and initialize the initial temperature of the simulated annealing algorithm; Taking each of the initial triplet data amounts as the local optimal solution corresponding to the current data amount set; Calculate the model loss value of each of the initial triple data sets respectively, and take the initial triple data set with the smallest model loss value as the global optimal solution of the current data set; Candidate solution generation steps: Generate corresponding candidate solutions for each triplet of data in the current data set; Data volume update steps: Calculate the model loss value of each triple data volume and the corresponding candidate solution respectively, and update each triple data volume to the corresponding candidate solution or keep it unchanged based on the comparison of the model loss value and the preset jump probability rule; Optimal solution update steps: Re-determine the corresponding local optimal solution and global optimal solution based on the model loss value of each triple data set after the update; Temperature update steps: Updating the temperature of the simulated annealing algorithm, and updating the exit probability rule based on the updated temperature; The candidate solution generating step, the data amount updating step, the optimal solution updating step and the temperature updating step are executed in a loop until a preset loop termination condition is reached, and the global optimal solution obtained after the loop termination is used as the triple data amount currently required to be extracted from the original knowledge graph.
7. The method for fine-tuning the knowledge graph reasoning model according to claim 1, characterized in that: After fine-tuning the original reasoning model based on the fine-tuning data set to obtain an updated reasoning model, the method further includes: In response to the knowledge graph reasoning task, determining known triple elements and initial screening conditions based on the knowledge graph reasoning task; Based on the known triple elements, a number of candidate triple elements that meet the initial screening conditions are obtained from the updated knowledge graph; wherein the updated knowledge graph is obtained by updating the original knowledge graph based on the newly added triple data set; Constructing candidate triples corresponding to each candidate triple element based on the known triple elements; Based on the updated reasoning model, the scoring results corresponding to each candidate triple are calculated respectively, and the reasoning results corresponding to the knowledge graph reasoning task are determined based on the scoring results.
8. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for fine-tuning the knowledge graph reasoning model described in any one of claims 1-7 can be implemented.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the method for fine-tuning the knowledge graph reasoning model as described in any one of claims 1 to 7 is executed.
10. A computer program product, characterized in that The computer program product includes a computer program, which, when executed by a processor, implements the method for fine-tuning the knowledge graph reasoning model described in any one of claims 1-7.