A knowledge graph completion method based on large model

Abstract

The present invention relates to the technical field of knowledge graph completion tasks, and specifically refers to a knowledge graph completion method, device and readable storage medium based on a large model, including: based on the embedding features of each entity, constructing the node centrality function and node indicator function of each entity, determining the sum of the centrality and the sum of the node indicator function values ​​of all entities; based on the entity distribution probability of each triple to be predicted, constructing the focal loss function of each triple to be predicted, and combining the sum of the centrality and the sum of the node indicator function values ​​of all entities, constructing the focal balance loss function based on the node centrality of each triple to be predicted; according to the preset batch size, constructing the target loss function of the current batch. The present invention improves the convergence speed and accuracy of the large model, improves the reasoning speed and prediction accuracy of the two-stage completion framework, and improves the accuracy of the final completion result of the knowledge graph and the processing efficiency of the completion process.

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge graph completion tasks, and in particular to a knowledge graph completion method, device and readable storage medium based on a large model. Background Art

[0002] Knowledge graph is a technology used to structure and organize data. It represents entities and their relationships in the real world in the form of nodes and edges. With the development of big data and artificial intelligence, knowledge graphs have been widely used in information retrieval, natural language processing, recommendation systems and other fields. However, knowledge graphs often face the problems of incompleteness and inaccuracy in practical applications. Therefore, it is necessary to use knowledge graph completion algorithms to automatically identify and fill in the missing parts in the knowledge graph.

[0003] Existing description-based knowledge graph completion methods mainly use the text descriptions of entities and relationships in the knowledge graph as information and use large models for completion; large models usually refer to deep learning models with hundreds of millions or even trillions of parameters; large models are pre-trained on massive data to learn extensive knowledge and complex semantic structures of the language, thereby completing the knowledge graph.

[0004] The large model used in the "masked language model based on precoding" proposed in the application document with application number "202410358013.X" is the BERT model. BERT is a pre-trained language model based on Transformer, which can perform pre-training of bidirectional context to capture richer semantic information; in the pre-training stage, BERT is trained on a large amount of unlabeled text data to learn language representation; in the fine-tuning stage, BERT can be further trained on labeled data according to specific tasks; therefore, the "masked language model based on precoding" proposed in the application document solves the high memory requirements of large models when inputting long texts. The invention solves the problem of large-scale classification of entities and shortens the time of model training and inference. However, due to the long-tail problem in the process of knowledge graph completion, when the application document uses the conventional cross-entropy classification loss function to optimize the weights of the masked language model based on precoding, the cross-entropy classification loss function cannot fully optimize the few entities with low frequency in the training data set, resulting in problems such as unbalanced model training, slow model convergence speed and low convergence accuracy. Among them, the long-tail problem refers to the fact that some entities appear very few times in the training data, resulting in poor prediction performance of the model for these entity samples, which in turn leads to low accuracy of the final knowledge graph completion result and low overall operating efficiency. Summary of the invention

[0005] To this end, the technical problem to be solved by the present invention is to overcome the problem that the cross entropy classification loss function in the prior art cannot fully optimize a few entities with low frequency of occurrence in the training data set, resulting in unbalanced model training, slow model convergence speed and low convergence accuracy, which results in low accuracy of the final completion result of the knowledge graph and low overall completion efficiency.

[0006] In order to solve the above technical problems, the present invention provides a knowledge graph completion method based on a large model, comprising:

[0007] Constructing a two-stage completion framework based on a large model, wherein the two-stage completion framework based on a large model includes a first large model and a second large model;

[0008] Input each entity description sequence and each relationship description sequence in the training set into the encoder of the first large model, output each entity description feature and each relationship description feature, and obtain each entity embedding feature and each relationship embedding feature based on the pooling operation; construct the node centrality function of each entity according to each entity embedding feature, and construct the node indicator function of each entity according to the preset node degree threshold; based on the node centrality function and node indicator function of each entity, obtain the sum of the centrality values ​​of all entities in the training set and the sum of the node indicator function values;

[0009] Based on the embedding features of each entity and each relationship, construct each triple to be predicted, and combine with the position encoding to obtain the input sequence corresponding to each triple to be predicted, and input it into the encoder of the second largest model, output the vector at the last mask position corresponding to each triple to be predicted, and then pass it through the classification layer to obtain the entity distribution probability corresponding to each triple to be predicted; according to the entity distribution probability corresponding to each triple to be predicted, construct the focal loss function of each triple to be predicted;

[0010] According to the sum of the centrality of all entities in the training set, the sum of the node indicator function values, and the focal loss function of each triple to be predicted, a focal balance loss function based on the node centrality of each triple to be predicted is constructed, and according to the preset batch size, the focal balance loss function based on the node centrality of all triples to be predicted in the current batch is weighted averaged to obtain the target loss function of the current batch;

[0011] The second largest model and the classification layer are iteratively trained using the training set until the target loss function value is minimized, and the optimal values ​​of all parameters in the second largest model and the classification layer are obtained, thereby obtaining a trained two-stage completion framework based on the large model.

[0012] Preferably, the expression of the node centrality function is:

[0013]

[0014] in, represents the embedding feature of the i-th entity; C i Represents the degree center of the i-th entity.

[0015] Preferably, the expression of the node indication function is:

[0016]

[0017] Among them, I i represents the node indicator function of the i-th entity; represents the embedding feature of the i-th entity; C i represents the centrality of the i-th entity; D represents the preset node degree threshold; for any entity in the training set, if the centrality of the current entity is greater than the preset node degree threshold, the value of the node indicator function of the current entity is 0; if the centrality of the current entity is less than or equal to the preset node degree threshold, the value of the node indicator function of the current entity is 1.

[0018] Preferably, the expression of the focus loss function is:

[0019]

[0020] Among them, FL m represents the focal loss function of the mth triplet to be predicted; Represents the entity distribution probability corresponding to the mth triple to be predicted; represents the probability value of the real tail entity corresponding to the mth triple to be predicted in the entity distribution probability of the mth triple to be predicted; γ represents the adjustment factor; Represents the negative class probability corresponding to the mth triple to be predicted.

[0021] Preferably, the expression of the focal balance loss function based on node centrality is:

[0022]

[0023] Among them, CFL m represents the focal balance loss function based on node centrality of the mth triple to be predicted; U represents the total number of all entities in the training set; I[C i ≤D] represents the indicator function value of the i-th entity. If C i ≤D is true, then I[C i ≤D] takes 1, if C i ≤D is false, then I[C i ≤D] takes 0; Represents the entity distribution probability corresponding to the mth triple to be predicted; represents the probability value of the true tail entity corresponding to the mth triple to be predicted in the entity distribution probability corresponding to the mth triple to be predicted; γ represents the adjustment factor; Represents the negative class probability corresponding to the mth triple to be predicted.

[0024] Preferably, the expression of the objective loss function is:

[0025]

[0026] Among them, LF P represents the target loss function of the Pth batch; M represents the preset batch size; CFL m represents the focal balance loss function based on node centrality for the mth triple to be predicted.

[0027] Preferably, the first large model and the second large model are both BERT large models.

[0028] Preferably, the pooling operation is average pooling, and the average pooling formula is:

[0029]

[0030] Among them, Y represents the average pooling feature; L represents the total length of the preset text; a represents the text length; X represents the description text, which includes the entity description text and the relationship description text; R represents the description text set; M represents the preset batch size; and B represents the dimension.

[0031] The present invention also provides a knowledge graph completion device based on a large model, comprising:

[0032] Memory for storing computer programs;

[0033] A processor is used to implement the steps of the above-mentioned method for completing a knowledge graph based on a large model when executing the computer program.

[0034] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned large-model-based knowledge graph completion method are implemented.

[0035] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0036] The invention discloses a knowledge graph completion method based on a large model, which uses a two-stage completion framework based on a large model, wherein the framework includes two large models; the first large model is used to obtain all entity description features and relationship description features corresponding to a training set, a validation set, and a test set, and then average pool all entity description features and relationship description features to obtain all entity embedding features and relationship embedding features; based on the entity embedding features, a node centrality function is constructed, and according to a preset node degree threshold, a node indicator function is constructed; the encoder of the second large model is used to obtain the entity distribution probability corresponding to each triple to be predicted; based on the entity distribution probability corresponding to the triple to be predicted, a focal loss function is constructed; based on the node centrality function, the node indicator function, and the focal loss function, a focal balance loss function (FocalLoss) based on the node centrality is constructed, and according to a preset batch size, a target loss function of the current batch is constructed, and the second large model and the classification layer are iteratively trained using the training set. When the target loss function value is the smallest, all parameters in the second large model and the classification layer are optimal, and then the validation set is used for verification to obtain the trained knowledge graph completion method based on the large model. Two-stage completion framework for large models; Node centrality is an indicator to measure the importance of nodes in the network. For any undirected graph, the degree of a node is the number of edges connected to the node; Since the knowledge graph belongs to an undirected graph, the entity is taken as a node. According to the embedding features of each entity, the centrality of each entity can be calculated, that is, the number of relationship connections around each entity in the knowledge graph can be known, which fully considers the relationship between each entity and ensures the integrity of the data set in the subsequent loss calculation; FocalLoss is a loss function used to deal with the problem of class imbalance. This loss function reduces the impact of the class imbalance problem by reducing the weight of easy-to-classify samples. Therefore, the focal balance loss function based on node centrality can well optimize a few entities with fewer occurrences in the data set, solve the problem that the existing long-tail entities are difficult to optimize, and improve the convergence speed and convergence accuracy of the large model, thereby further improving the reasoning speed and prediction accuracy of the two-stage completion framework. It can significantly improve the operating efficiency of the large model under resource-constrained devices, thereby improving the accuracy of the final completion result of the knowledge graph and the processing efficiency of the completion process. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0038] Figure 1 is the frequency graph of entity distribution;

[0039] Figure 2 It is a flow chart of a knowledge graph completion method based on a large model provided by the present invention;

[0040] Figure 3 This is a schematic diagram of the two-stage completion framework based on the large model;

[0041] Figure 4 It is a comparison chart of loss function optimization. DETAILED DESCRIPTION

[0042] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0043] Embodiment 1

[0044] Reference Figure 1 As shown in the figure, 5% of the entities account for 30% of the occurrence frequencies of all entities, and more than 50% of the entities appear less than 3 times. At the same time, due to the long-tail problem in the knowledge graph, the conventional cross-entropy classification loss function cannot fully optimize the few entities with low occurrence frequencies in the data set. That is, during the training process, the large model cannot learn the rules of a few entities well, resulting in poor prediction performance for a few entities.

[0045] Based on the above problems, the present invention proposes a focal balance loss function based on node centrality to ensure the balance of model training, thereby optimizing the learning of a few entities; the specific focal balance loss function based on node centrality is applied to a knowledge graph completion method based on a large model provided by the present invention to obtain the optimal values ​​of all parameters of the second largest model and the classification layer in the two-stage completion framework based on the large model; since the first largest model used in the design and experimental process is a trained model, the parameter optimization process is only performed on the second largest model and the classification layer.

[0046] Reference Figure 2 As shown, Figure 2 A flowchart of a knowledge graph completion method based on a large model provided by the present invention; specifically comprising:

[0047] S21: Acquire a data set, preprocess the data set to obtain a preprocessed data set; based on the preprocessed data set, acquire a training set, a validation set, and a test set; the preprocessing includes data removal, cleaning, and truncating or padding the text length of each description text in the data set to a preset text length;

[0048] That is, filter all entity description texts and relationship description texts in the data set, remove the entity description texts and relationship description texts that do not appear in the training set, so as not to affect the training effect; clean the validation set and test set to ensure that all entity description texts and relationship description texts in the validation set and test set appear in the training set;

[0049] The default text length is 128. If the length of any description text is greater than 128, the text length of the current description text will be truncated to 128, otherwise it will be padded to 128.

[0050] S22: Construct a two-stage completion framework based on a large model, such as Figure 3 As shown; wherein the two-stage completion framework based on the large model includes a first large model and a second large model;

[0051] S23: Based on the training set, using the word segmenter, obtain each entity description sequence and each relationship description sequence, and input them into the encoder of the first large model, output each entity description feature and each relationship description feature; perform pooling operation on each entity description feature and each relationship description feature to obtain each entity embedding feature and each relationship embedding feature, and construct the embedding layer of the second large model;

[0052] The pooling operation is average pooling, and the average pooling formula is:

[0053]

[0054] Wherein, Y represents the average pooling feature; L represents the total length of the preset text; a represents the text length; X represents the description text, which includes the entity description text and the relationship description text; R represents the description text set; M represents the preset batch size; B represents the dimension; in the specific embodiment of the present invention, the dimension is 768 dimensions;

[0055] According to the embedding features of each entity, the node centrality function of each entity is constructed, and its expression is: The node centrality formula is:

[0056] in, represents the embedding feature of the i-th entity; C i Represents the centrality of the i-th entity;

[0057] Based on the node centrality function of each entity and the preset node degree threshold, the node indicator function of each entity is constructed, and its expression is:

[0058]

[0059] Among them, I i represents the node indicator function of the i-th entity; represents the embedding feature of the i-th entity; C irepresents the centrality of the ith entity; D represents the preset node degree threshold; for any entity in the training set, if the centrality of the current entity is greater than the preset node degree threshold, the value of the node indicator function of the current entity is 0; if the centrality of the current entity is less than or equal to the preset node degree threshold, the value of the node indicator function of the current entity is 1;

[0060] Based on the node centrality function and node indicator function of each entity, the sum of the centrality values ​​and the sum of the node indicator function values ​​of all entities in the training set are obtained;

[0061] S24: Based on the embedding features of each entity and each relationship, construct each triple to be predicted, and combine the position encoding to obtain the input sequence corresponding to each triple to be predicted, and input it into the encoder of the second largest model, output the vector at the last mask position corresponding to each triple to be predicted, and then pass through the classification layer to obtain the entity distribution probability corresponding to each triple to be predicted; according to the entity distribution probability corresponding to each triple to be predicted, construct the focal loss function of each triple to be predicted, and its expression is:

[0062] Among them, FL m represents the focal loss function of the mth triplet to be predicted; Represents the entity distribution probability corresponding to the mth triple to be predicted; represents the probability value of the real tail entity corresponding to the mth triple to be predicted in the entity distribution probability of the mth triple to be predicted; γ represents the adjustment factor; Represents the negative class probability corresponding to the mth triple to be predicted;

[0063] S25: Based on the sum of the centrality of all entities in the training set, the sum of the node indicator function values, and the focal loss function of each triple to be predicted, a focal balance loss function based on the node centrality of each triple to be predicted is constructed, and its expression is:

[0064]

[0065] Among them, CFL m represents the focal balance loss function based on node centrality of the mth triple to be predicted; I[C i ≤D] represents the node indicator function value of the i-th entity. If C i ≤D is true, then I[C i ≤D] takes 1, if C i ≤D is false, then I[C i ≤D] is 0; in a specific embodiment of the present invention, the value of D is 1, 2, or 3;

[0066] According to the focal balance loss function based on node centrality and the preset batch size, the focal balance loss function based on node centrality of all triples to be predicted in the current batch is weighted averaged to obtain the target loss function of the current batch; the expression of the target loss function is:

[0067]

[0068] Among them, LF P represents the target loss function of the Pth batch; M represents the preset batch size; CFL m represents the focal balance loss function based on node centrality of the mth triple to be predicted;

[0069] S26: Iteratively train the second largest model and the classification layer using the training set until the target loss function value is minimized, thereby obtaining the optimal values ​​of all parameters in the second largest model and the classification layer;

[0070] In a specific embodiment of the present invention, the iterative training process specifically includes:

[0071] If the current round is less than the preset training round, samples are extracted from the embedding layer of the second largest model according to the preset batch size to construct a sample set corresponding to the current batch; based on the entity embedding features and relationship embedding features corresponding to each sample in the sample set corresponding to the current batch, a triple to be predicted corresponding to each sample in the current batch is constructed; each triple to be predicted is positionally encoded to obtain an input sequence corresponding to each triple to be predicted in the current batch, and is input into the encoder of the second largest model, and a vector at the last mask position corresponding to each triple to be predicted is output, and then the vector is passed through the classification layer to obtain the entity distribution probability corresponding to each triple to be predicted in the current batch;

[0072] Based on the entity distribution probability corresponding to each triple to be predicted in the current batch, the entity distribution probability corresponding to each triple to be predicted in the current batch is calculated, so as to obtain the target loss function of the current batch; the minimum value of the target loss function of the current batch is solved to obtain the second largest model corresponding to the current batch and all parameter values ​​in the classification layer; if all samples in the embedding layer of the second largest model have not been extracted at this time, continue to extract samples to construct the sample set of the next batch, and continue the above operation until all samples in the embedding layer of the second largest model are extracted, and the second largest model corresponding to the current round and all parameter values ​​in the classification layer are obtained;

[0073] The evaluation function is used to evaluate the second largest model in the current round. If the performance of the second largest model in the current round is better than that of the second largest model in the previous round, the weight of the second largest model in the current round is saved and the next round of training is carried out until the preset number of training rounds is reached. The training is terminated to obtain the optimal values ​​of all parameters in the second largest model and the classification layer.

[0074] S27: Use the validation set to verify the optimal values ​​of all parameters in the second largest model and the classification layer to obtain a trained two-stage completion framework based on the large model.

[0075] Based on the above process, after obtaining the trained two-stage completion framework based on the large model, it is also necessary to use the test set to verify and evaluate the two-stage completion framework based on the large model, including:

[0076] Step 1: Based on all entity and relationship description texts in the test set, use the word segmenter to obtain each entity description sequence and each relationship description sequence; input each entity description sequence and each relationship description sequence into the encoder of the first large model respectively, and output each entity description feature and each relationship description feature; perform average pooling on each entity description feature and each relationship description feature to obtain each entity embedding feature and each relationship embedding feature, and construct the embedding layer of the second large model;

[0077] Step 2: Extract samples from the embedding layer of the second largest model according to the preset batch size to obtain a sample set; based on the embedding features of each entity and each relationship in the sample set, use a mask to replace the tail entity to be predicted, construct the triple to be predicted corresponding to each sample in the sample set, and combine the position encoding to obtain the input sequence of each triple to be predicted;

[0078] Among them, assuming that any sample is selected from the sample set, the triple to be predicted corresponding to the current sample is expressed as: [u h ,u r ,u MASK ]; the input sequence of the triple to be predicted corresponding to the current sample is expressed as: Among them, u start 、u end Respectively represent the embedding features corresponding to the start and end symbols of the input sequence corresponding to the current sample, u h 、u r Respectively represent the entity embedding features and relationship embedding features corresponding to the current sample; u MASK represents the mask vector; Indicates the position code corresponding to the g-th position of the input sequence corresponding to the current sample;

[0079] Sequentially obtain the triples to be predicted and the input sequence corresponding to all samples in the current sample set;

[0080] Step 3: Since u corresponding to different samples h 、u r Different, but different samples correspond to u MASK For the same mask feature, the encoder of the second largest model ish 、u r Attention learning, the last layer u MASK The vector at the position can be used as the distinguishing feature of the entity to be predicted, and the output corresponding to the current sample is expressed as: output MASK =TripleEncoder(input)[p z ,:]; among them, p z Indicates the position encoding corresponding to the input mask; output MASK represents the output of the encoder of the second largest model;

[0081] That is, the input sequence corresponding to each sample obtained in step 2 is input into the encoder of the second largest model, and the vector at the last mask position corresponding to each sample is output;

[0082] Step 4: Link prediction (entity, relationship,?) can be regarded as a multi-classification problem of masked vectors; the classification layer is a feedforward neural network, and the input dimension of the classification layer is u MASK The output dimension is the number of entities, and the last layer u MASK The vector at the position is input into the classification layer, and then the entity classification probability is obtained through the softmax function. The entity classification probability corresponding to the current sample is expressed as: Among them, W and b represent the weight and bias of the feedforward network respectively;

[0083] That is, the vector at the last mask position corresponding to each sample obtained in step 3 is input into the classification layer, and finally the entity classification probability corresponding to each sample is output, that is, the entity classification probability corresponding to each triple to be predicted;

[0084] Step 5: After obtaining the classification probabilities of all entities, it is necessary to filter the possible candidate entities; possible candidate entities are all tail entities with the same (head entity, relationship); entity filtering is a strategy widely used in current knowledge graph completion algorithms; for example, there are the following triples: (1) triple (cold, therapeutic drugs, Lianhua Qingwen capsules) (2) triple (cold, therapeutic drugs, ibuprofen) (3) triple (cold, therapeutic drugs, acetaminophen). During the evaluation, the prediction of triple (2) must filter the two entities "Lianhua Qingwen capsules" and "acetaminophen" to prevent equally correct entities from affecting the evaluation results;

[0085] Filter possible candidate entities according to the entity filtering strategy, and then use the ranking formula rank(n) = 1 + ∑ t′∈ε，t′≠ tJ[f(h, r, t′)>f(h, r, t)], calculates the ranking of the correct entities corresponding to each triple to be predicted; according to the calculated ranking of the correct entities, calculates the evaluation index of the two-stage completion model based on the large model; wherein, ε represents the set of all entities; f(h, r, t) represents the score calculated for the triple; h, r, t represent entity, relationship, and tail entity, respectively; t′ represents the predicted tail entity; J[·] represents the ranking indicator function, if f(h, r, t′)>f(h, r, t) is true, then J[-] takes the value of 1, if f(h, r, t′)>f(h, r, t) is false, then J[·] takes the value of 0.

[0086] Embodiment 2

[0087] In a specific embodiment of the present invention, the first large model and the second large model are both BERT models, and constitute a two-stage completion model based on the large model; in order to verify the reasoning performance of the second-order completion model based on the large model, an experiment is conducted on the second-order completion model based on the large model:

[0088] 1. Dataset

[0089] This model is experimented on the WN18RR benchmark dataset, which is a subset of the WordNet database. It describes the association features between English words and retains the symmetric, asymmetric, and combination relationships in the WordNet dataset. It is used to evaluate link prediction tasks. The scale of the WN18RR benchmark dataset is shown in Table 1:

[0090] Table 1 Dataset specifications

[0091] Dataset Number of entities Number of relationships Training set length Validation set length Test set length WN18RR 40559 11 86835 2824 2924

[0092] It should be noted that the entities have filtered out the triplets that did not appear in the training set, and the entities and relations in the validation set and training set have also appeared in the training set; the data in the validation set and test set do not participate in model optimization, and the subsequent experimental conclusions are all evaluation results on the test set.

[0093] 2. Evaluation Indicators

[0094] The missing triples (head entity, relationship, ?) of the knowledge graph may correspond to multiple tail entities. According to convention, when evaluating, all possible entities (except label entities) that appear in the training set, validation set, and test set need to be removed from the candidate triples; among the filtered entities, the softmax value output by the classification layer is used for ranking; the total number of all triples in the test set is N, and the ranking of the true answer of the nth triple in the test set among all entities is defined as follows:

[0095]

[0096] Where ε represents the set of all entities, i.e., the test set; f(h, r, t) represents the score calculated for the triple; h, r, t represent entity, relation, and tail entity, respectively; t′ represents the predicted tail entity; J[·] represents the ranking indicator function. If f(h, r, t′)>f(h, r, t) is true, then J[·] takes the value of 1; if f(h, r, t′)>f(h, r, t) is false, then J[·] takes the value of 0.

[0097] 1. Mean ranking (MR)

[0098] MR refers to the average value of all ranks (n) in the test set, and its expression is:

[0099]

[0100] This indicator reflects the ranking of the true answer among the candidate entities on average. The lower the MR, the better the reasoning performance of the model.

[0101] 2. Mean Reciprocal Ranking (MRR)

[0102] MRR refers to the average of the reciprocals of all rank(i) in the test set, and its expression is:

[0103]

[0104] Compared with MR, this indicator focuses more on whether the model can find a high ranking of the true answer among the candidate entities; the higher the MRR, the better the model reasoning performance;

[0105] 3. Average number of top K nodes (Hits@K)

[0106] Hits@K refers to the average value of all rank(n)≤K in the test set, where K is usually 10, 3, or 1. Its expression is:

[0107]

[0108] This indicator model reflects whether the model can rank the true answer to the top K, and only focuses on the high ranking performance of the model; the larger the Hits@K, the better the model reasoning performance.

[0109] 3. Experimental Environment

[0110] The experimental environment may affect the experimental results. The experimental environment is as follows for research reference:

[0111] Table 2 Experimental environment

[0112] Experimental environment Specification processor 12 vCPU Intel(R)Xeon(R)Platinum 8255C CPU@2.50GHz Graphics RTX2080 TI (11GB) Memory 40GB Development Language Python 3.10 Development System Ubuntu 22.04 Development Framework Pytorch 2.1.0+cuda12.1

[0113] 4. Parameter settings

[0114] Parameter settings may affect the experimental results. The detailed parameter settings of this experiment are as follows:

[0115] Table 3 Parameter settings

[0116]

[0117] V. Experimental Results

[0118] In order to compare the reasoning performance of the model of the present invention fairly and reasonably, the benchmark models selected are description-based baseline models, including KG-BERT, MTL-KGC, StAR, KGLM, and LP-BERT; the WN18RR dataset usually uses MRR as the main evaluation indicator. Table 4 shows the specific experimental results; except for the experimental results of this model, all other experimental results are taken from the original paper results, where "—" indicates data not described in the paper. Bold indicates the best result under this indicator.

[0119] Table 4 Experimental results

[0120]

[0121] Among them, Our-CE represents a model that uses the cross entropy loss formula to optimize parameters, and Our-FL / D=1, 2, and 3 represent a two-stage completion model based on a large model that uses the preset node degree thresholds of 1, 2, and 3 for parameter optimization, respectively; Figure 4 To compare the average ranking reciprocal optimization curve using the cross entropy loss and the focal balance loss based on the node centrality proposed by the present invention; Figure 4 It can be concluded that the focal balance loss based on node centrality proposed in the present invention effectively accelerates the convergence speed of the model.

[0122] Table 5 Performance comparison

[0123]

[0124] In order to compare the inference speed and memory requirement advantages of the two-stage completion framework based on a large model proposed in the present invention, KG-BERT and StAR models are selected for comparison with the two-stage completion model based on a large model; the inference speed of KG-BERT is related to the entity scale in the data set. The more entities there are, the longer the inference speed; Table 5 shows the performance comparison of each model. In Table 5, Our represents the two-stage completion framework based on a large model proposed in the present invention. It can be seen from Table 5 that the two-stage framework far exceeds the comparison model in terms of inference time and required memory; the inference experiment is carried out on the WN18RR test set.

[0125] The pooling method of the description text selected by the present invention is subjected to comparative experiments of multiple pooling methods; the description representation generated by the description encoder will have an impact on the subsequent prediction results; CLS pooling, average pooling and maximum pooling are often used as semantic representations of sentences; among them, CLS pooling uses the first feature of the last hidden state as the representation form; average pooling averages the last hidden state of each word; maximum pooling selects the maximum value of the last hidden state of each word; the comparison of these three pooling methods is shown in Table 6, average pooling provides superior semantic representation quality, and compared with maximum pooling, the MRR on the WN18RR dataset is improved by 6.5%.

[0126] Table 6 Performance of different pooling methods

[0127] Pooling Methods MRR Hits@1 Hits@3 Hits@10 CLS Pooling 0.531 0.490 0.565 0.609 Average Pooling 0.545 0.502 0.557 0.627 Max Pooling 0.480 0.449 0.489 0.540

[0128] Large models can improve completion accuracy, have strong learning and representation capabilities, and can better capture the relationships and attributes between entities, thereby improving the accuracy of knowledge graph completion. For diverse knowledge graphs, large models can learn more general and diverse knowledge representations through large-scale training data, and can quickly adapt to knowledge graph completion tasks in different fields and of different scales through fine-tuning. Since they are trained on a large number of samples, large models usually have better generalization capabilities and can achieve better results on smaller-scale knowledge graphs, thereby reducing the dependence of knowledge graph completion algorithms on data volume.

[0129] In the future, with the continuous development and improvement of large model technology, the knowledge graph completion algorithm based on large models will be able to be further improved in effect and performance, bringing more possibilities and development space for knowledge graph applications. At the same time, the efficiency of large models will also become an important direction for future research to better meet the needs of practical applications.

[0130] Embodiment 3

[0131] A specific embodiment of the present invention also provides a knowledge graph completion device based on a large model, including: a memory for storing a computer program; and a processor for implementing the steps of any one of the knowledge graph completion methods based on a large model when executing the computer program.

[0132] Embodiment 4

[0133] A specific embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the large-model-based knowledge graph completion methods are implemented.

[0134] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0135] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0138] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A knowledge graph completion method based on a large model, characterized in that: include: Constructing a two-stage completion framework based on a large model, wherein the two-stage completion framework based on a large model includes a first large model and a second large model; Input each entity description sequence and each relationship description sequence in the training set into the encoder of the first large model, output each entity description feature and each relationship description feature, and obtain each entity embedding feature and each relationship embedding feature based on the pooling operation; According to the embedding features of each entity, the node centrality function of each entity is constructed, and according to the preset node degree threshold, the node indicator function of each entity is constructed; based on the node centrality function and node indicator function of each entity, the sum of the centrality values ​​and the sum of the node indicator function values ​​of all entities in the training set are obtained; wherein the pooling operation is average pooling, and the average pooling formula is: Wherein, Y represents the average pooling feature; L represents the total length of the preset text; a represents the text length; X represents the description text, which includes the entity description text and the relationship description text; R represents the description text set; M represents the preset batch size; B represents the dimension; Based on the embedding features of each entity and each relationship, construct each triple to be predicted, and combine with the position encoding to obtain the input sequence corresponding to each triple to be predicted, and input it into the encoder of the second largest model, output the vector at the last mask position corresponding to each triple to be predicted, and then pass it through the classification layer to obtain the entity distribution probability corresponding to each triple to be predicted; according to the entity distribution probability corresponding to each triple to be predicted, construct the focal loss function of each triple to be predicted; According to the sum of the centrality of all entities in the training set, the sum of the node indicator function values, and the focal loss function of each triple to be predicted, a focal balance loss function based on the node centrality of each triple to be predicted is constructed, and according to the preset batch size, the focal balance loss function based on the node centrality of all triples to be predicted in the current batch is weighted averaged to obtain the target loss function of the current batch; The second largest model and the classification layer are iteratively trained using the training set until the target loss function value is minimized, and the optimal values ​​of all parameters in the second largest model and the classification layer are obtained, thereby obtaining a trained two-stage completion framework based on the large model.

2. A knowledge graph completion method based on a large model according to claim 1, characterized in that: The expression of the node centrality function is: in, represents the embedding feature of the i-th entity; C i Represents the degree center of the i-th entity.

3. A knowledge graph completion method based on a large model according to claim 1, characterized in that: The expression of the node indication function is: Among them, I i represents the node indicator function of the i-th entity; C i represents the centrality of the i-th entity; D represents the preset node degree threshold; for any entity in the training set, if the centrality of the current entity is greater than the preset node degree threshold, the value of the node indicator function of the current entity is 0; if the centrality of the current entity is less than or equal to the preset node degree threshold, the value of the node indicator function of the current entity is 1.

4. According to the large model-based knowledge graph completion method of claim 1, it is characterized in that: The expression of the focal loss function is: Among them, FL m represents the focal loss function of the mth triplet to be predicted; Represents the entity distribution probability corresponding to the mth triple to be predicted; represents the probability value of the real tail entity corresponding to the mth triple to be predicted in the entity distribution probability of the mth triple to be predicted; γ represents the adjustment factor; Represents the negative class probability corresponding to the mth triple to be predicted.

5. A knowledge graph completion method based on a large model according to claim 1, characterized in that: The expression of the focal balance loss function based on node centrality is: Among them, CFL m represents the focal balance loss function based on node centrality of the mth triple to be predicted; U represents the total number of all entities in the training set; I[C i ≤D] represents the indicator function value of the i-th entity. If C i ≤D is true, then I[C i ≤D] takes 1, if C i ≤D is false, then I[C i ≤D] takes 0; Represents the entity distribution probability corresponding to the mth triple to be predicted; represents the probability value of the true tail entity corresponding to the mth triple to be predicted in the entity distribution probability corresponding to the mth triple to be predicted; γ represents the adjustment factor; Represents the negative class probability corresponding to the mth triple to be predicted.

6. A knowledge graph completion method based on a large model according to claim 1, characterized in that: The expression of the objective loss function is: Among them, LF P represents the target loss function of the Pth batch; M represents the preset batch size; CFL m represents the focal balance loss function based on node centrality for the mth triple to be predicted.

7. A knowledge graph completion method based on a large model according to claim 1, characterized in that: The first large model and the second large model are both BERT large models.

8. A knowledge graph completion device based on a large model, characterized in that: include: Memory for storing computer programs; A processor, used to implement the steps of a large model-based knowledge graph completion method as described in any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a large-model-based knowledge graph completion method as described in any one of claims 1 to 7.

Images