Fuzzy knowledge graph comparison embedding method and device, electronic equipment and storage medium

Abstract

The invention discloses a knowledge graph embedding method, and particularly relates to a fuzzy knowledge graph comparison embedding method and device, electronic equipment and a storage medium. The method comprises the steps that a first view and a second view are constructed based on a fuzzy knowledge graph, the first view is used for representing correlation among a plurality of triads from the perspective of a head entity, and the second view is used for representing correlation among a plurality of triads from the perspective of a tail entity; encoding the triad in the first view by using a knowledge graph embedding model to obtain a first embedding, and encoding the triad in the second view by using the knowledge graph embedding model to obtain a second embedding; training the knowledge graph embedding model based on the fuzzy RDF KG embedding loss and the contrast learning loss to obtain a trained knowledge graph embedding model; and obtaining knowledge graph embedding based on the trained knowledge graph embedding model. According to the method, the generation quality of knowledge graph embedding can be improved.

Description

Technical Field

[0001] The present invention relates to a knowledge graph embedding method, and specifically to a fuzzy knowledge graph comparison embedding method, device, electronic device and storage medium. Background Art

[0002] Knowledge Graph (KG) has very important applications in different fields. For example, in the financial field, KG can deeply perceive and widely connect isolated data, conduct highly intelligent shared analysis, help customers expand the breadth and depth of existing digital resources, and support intelligent applications. For example, banks can use the important application value of knowledge graphs in the entire financial scenario to conduct risk assessment, credit rating, and fraud detection. In addition, knowledge graphs also play an important role in public security, medical care, e-commerce and other fields. In the field of public security, knowledge graphs are used to process massive data, break the problem of data silos, and mine valuable information behind the data. In the medical field, it can establish a systematic and complete knowledge base, provide efficient retrieval, promote the intelligent processing of massive data, and assist in diagnosis and treatment. In the field of e-commerce, knowledge graphs are used for product recommendations and user portrait analysis to improve personalized recommendation capabilities.

[0003] The core goal of knowledge graph embedding is to learn a low-dimensional vector representation for each entity and relationship, so as to perform various downstream tasks such as entity classification, link prediction, relationship reasoning, etc. while maintaining the original graph structure information. Therefore, whether the knowledge graph performs various complex downstream tasks in different fields, or optimizes or completes the knowledge graph itself, it is necessary to obtain knowledge graph embedding. The quality of knowledge graph embedding has a crucial impact on the performance of downstream tasks.

[0004] Resource Description Framework (RDF) is rapidly becoming the standard data model for metadata management. Due to its applicability for semantic data modeling in an open information extraction environment, as well as the openness of the environment and the decentralized maintenance of various users, RDF data inevitably contains erroneous, outdated and inconsistent information as its scale increases significantly.

[0005] The knowledge graph (KG) modeled by RDF is an extremely useful resource in artificial intelligence applications. In order to quickly build and continuously enrich the knowledge graph, an automatic construction and update mechanism is usually used, which inevitably brings fuzziness or uncertainty. Ignoring its existence will seriously affect downstream tasks. Due to the characteristics of KG graph structure, semantic links, real-time updates and cross-domain, fuzzy RDF KG has become the latest research hotspot.

[0006] Current research on fuzzy RDF KG faces challenges from three main aspects: (1) The inherent fuzziness of the real world. A large part of human knowledge is inherently imprecise. For example, in the industrial field, product design is usually based on experience and probabilistic information. It is not enough to just consider the existence of relational facts, but also the corresponding possibility should be measured, which is particularly valuable in practical applications such as biology and intelligent manufacturing. (2) Noise introduced by the construction process. In order to handle large amounts of data, most KGs are automatically extracted from web corpora using heuristic algorithms. The automated construction process usually leads to fuzzy knowledge graphs containing inaccurate facts. The validity of these facts needs to be quantified by metrics such as fuzziness and confidence. (3) The inherent properties of triples. Although fuzzy RDF KG is effective in representing structured data, the symbolic nature of its triples often makes the knowledge graph difficult to operate.

[0007] To address this problem, many studies have embedded fuzzy knowledge graphs based on negative samples generated by randomly replacing existing triple entities. However, the fuzzy knowledge graphs in the real world are very complex, that is, there are many triple elements that are not matched but related. Embedding fuzzy knowledge graphs by randomly replacing existing triple entities with negative samples is not sufficient to prototype the actual KG fuzziness, resulting in poor quality of knowledge graph embedding generation. Summary of the invention

[0008] The technical problem to be solved by the present invention is that the above-mentioned method commonly used in the prior art uses negative samples generated by randomly replacing existing triple entities to embed fuzzy knowledge graphs, which is not enough to prototype the actual KG fuzziness, resulting in poor quality of knowledge graph embedding generation. In order to solve the above problem, the present invention provides a fuzzy knowledge graph comparative embedding method, device, electronic device and storage medium.

[0009] The content of the present invention includes:

[0010] In a first aspect, an embodiment of the present invention provides a fuzzy knowledge graph comparison embedding method, including:

[0011] Constructing a first view and a second view based on a fuzzy knowledge graph, wherein the fuzzy knowledge graph includes a plurality of triples, the first view is used to represent the correlation between the plurality of triples from the perspective of a head entity, and the second view is used to represent the correlation between the plurality of triples from the perspective of a tail entity;

[0012] Encoding the triples in the first view using a knowledge graph embedding model to obtain a first embedding, and encoding the triples in the second view using the knowledge graph embedding model to obtain a second embedding;

[0013] The knowledge graph embedding model is trained based on fuzzy RDF KG embedding loss and contrastive learning loss to obtain a trained knowledge graph embedding model;

[0014] Obtain knowledge graph embedding based on the trained knowledge graph embedding model.

[0015] Optionally, constructing the first view and the second view based on the fuzzy knowledge graph includes:

[0016] Taking each triple of the plurality of triples as a node;

[0017] The first view is obtained by connecting the first node and the second node, and the second view is obtained by connecting the third node and the fourth node. The triples corresponding to the first node and the second node have the same head entity, and the triples corresponding to the third node and the fourth node have the same tail entity.

[0018] Optionally, encoding the triples in the first view using the knowledge graph embedding model to obtain a first embedding includes:

[0019] For each first triplet in the first view, obtaining a first initial embedding of the first triplet;

[0020] The first initial embedding is updated based on the attention coefficients of neighboring triples of the first triplet to obtain the first embedding.

[0021] Optionally, encoding the triples in the second view by using the knowledge graph embedding model to obtain a second embedding includes:

[0022] For each second triplet in the second view, obtaining a second initial embedding of the second triplet;

[0023] The second initial embedding is updated based on the attention coefficients of neighboring triplets of the second triplet to obtain the second embedding.

[0024] Optionally, before training the knowledge graph embedding model based on fuzzy RDF KG embedding loss and contrastive learning loss to obtain the trained knowledge graph embedding model, the method further includes:

[0025] Taking the triples in the fuzzy knowledge graph as positive example triples;

[0026] Randomly replacing the entity relations and relations of the positive triples with other entities and relations to obtain negative triples;

[0027] The fuzzy RDF KG embedding loss is calculated based on the difference score of the positive triplet and the difference score of the negative triplet.

[0028] Optionally, the contrastive learning loss is determined based on a similarity between the first embedding and the second embedding corresponding to a triple in the fuzzy knowledge graph.

[0029] Optionally, the contrastive learning loss is:

[0030]

[0031] Among them, {x1,x2,...,x n} represents the first embedding corresponding to n triples in the fuzzy knowledge graph, {z1,z2,...,z n} represents the second embedding corresponding to n triples in the fuzzy knowledge graph, (x i ,z i ) represents the first embedding corresponding to the i-th triple in the fuzzy knowledge graph and the second embedding corresponding to the i-th triple in the fuzzy knowledge graph, τ represents the temperature parameter, sim(x i ,z i ) represents x i and z i The cosine similarity of , n is an integer greater than 2, and i is an integer less than or equal to n.

[0032] In a second aspect, an embodiment of the present invention further provides a fuzzy knowledge graph comparison and embedding device, comprising:

[0033] A construction module, configured to construct a first view and a second view based on a fuzzy knowledge graph, wherein the fuzzy knowledge graph includes a plurality of triples, wherein the first view is used to represent the correlation between the plurality of triples from the perspective of a head entity, and the second view is used to represent the correlation between the plurality of triples from the perspective of a tail entity;

[0034] an encoding module, configured to encode the triples in the first view using a knowledge graph embedding model to obtain a first embedding, and to encode the triples in the second view using the knowledge graph embedding model to obtain a second embedding;

[0035] A training module, used for training the knowledge graph embedding model based on fuzzy RDF KG embedding loss and contrastive learning loss to obtain a trained knowledge graph embedding model;

[0036] An acquisition module is used to acquire knowledge graph embedding based on the trained knowledge graph embedding model.

[0037] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps in the fuzzy knowledge graph comparison and embedding method as described in the first aspect.

[0038] In a fourth aspect, an embodiment of the present invention provides a readable storage medium for storing a program, which, when executed by a processor, implements the steps in the fuzzy knowledge graph comparison and embedding method as described in the first aspect.

[0039] The beneficial effect of the present invention is that, in the embodiment of the present invention, the first view and the second view are first constructed by treating multiple triples in the fuzzy knowledge graph as nodes. The first view and the second view characterize the connection of multiple triples from different angles, so they can be used as two consistent views in contrastive learning. Then, the method trains the knowledge graph embedding model based on the fuzzy RDF KG embedding loss and the contrastive learning loss. The closeness of the representation of the same triple in different views can be used as an indicator of knowledge graph representation learning. Through contrastive learning, the quality of the learned knowledge graph embedding is higher, thereby improving the execution effect of the downstream reasoning tasks corresponding to the knowledge graph embedding. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Attached Figure 1 One of the flowcharts of the fuzzy knowledge graph comparison embedding method provided by an embodiment of the present invention;

[0041] Attached Figure 2 The second flowchart of the fuzzy knowledge graph comparison embedding method provided by the embodiment of the present invention;

[0042] Attached Figure 3a for Figure 2 A schematic diagram of constructing a first view and a second view portion;

[0043] Attached Figure 3b for Figure 3a Example diagram of fuzzy knowledge graph in ;

[0044] Attached Figure 3c for Figure 3a Example graph that treats triples as nodes;

[0045] Attached Figure 3d for Figure 3a An example diagram of a first view and a second view;

[0046] Attached Figure 4 for Figure 2 A schematic diagram of obtaining the first embedded and second embedded parts;

[0047] Attached Figure 5 for Figure 2Schematic diagram of the joint training part of the knowledge graph embedding model in ;

[0048] Attached Figure 6 A schematic diagram of the structure of a fuzzy knowledge graph comparison and embedding device provided in an embodiment of the present invention;

[0049] Attached Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In the embodiments of the present application, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In the embodiments of the present application, the term "multiple" refers to two or more, and other quantifiers are similar. The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0051] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] For ease of understanding, each term involved in this application is explained below.

[0053] See also Figure 1 , Figure 1 A flowchart of a fuzzy knowledge graph comparison embedding method provided by an embodiment of the present invention is provided. The method specifically includes the following steps:

[0054] Step 101, construct a first view and a second view based on a fuzzy knowledge graph, wherein the fuzzy knowledge graph includes multiple triples, the first view is used to represent the correlation between the multiple triples from the perspective of a head entity, and the second view is used to represent the correlation between the multiple triples from the perspective of a tail entity.

[0055] Step 102: Encode the triples in the first view using the knowledge graph embedding model to obtain a first embedding, and encode the triples in the second view using the knowledge graph embedding model to obtain a second embedding.

[0056] Step 103: Train the knowledge graph embedding model based on fuzzy RDF KG embedding loss and contrastive learning loss to obtain a trained knowledge graph embedding model.

[0057] Step 104, obtaining knowledge graph embedding based on the trained knowledge graph embedding model.

[0058] In an embodiment of the present invention, a first view and a second view are first constructed by treating multiple triples in a fuzzy knowledge graph as nodes. The first view and the second view characterize the connection of multiple triples from different angles, so they can be used as two consistent views in contrastive learning. Then, this method trains the knowledge graph embedding model based on the fuzzy RDF KG embedding loss and contrastive learning loss. The closeness of the representation of the same triple in different views can be used as an indicator of knowledge graph representation learning. Through contrastive learning, the quality of the learned knowledge graph embedding is higher, thereby improving the execution effect of the downstream reasoning tasks corresponding to the knowledge graph embedding.

[0059] See also Figure 2 and Figure 3a , it should be understood that Figure 2 The overall flow chart of this method is as follows: Figure 3a , Figure 4 and Figure 5 They are Figure 2 The enlarged schematic diagram of the three parts, further, Figure 3b-3d for Figure 3a In step 101, two triple-level views can be generated by treating the triples in the fuzzy knowledge graph as nodes, which can be used as two consistent views in contrastive learning. In some optional embodiments, step 101 includes:

[0060] Taking each triple of the plurality of triples as a node;

[0061] The first view is obtained by connecting the first node and the second node, and the second view is obtained by connecting the third node and the fourth node. The triples corresponding to the first node and the second node have the same head entity, and the triples corresponding to the third node and the fourth node have the same tail entity.

[0062] Exemplarily, for two triples of shared entities T1=(s1, p1, o1) and T2=(s2, p2, o2) in the fuzzy knowledge graph, there are two connection modes, namely, the connection mode of the shared head entity, i.e., s1=s2 or s1=o2; and the connection mode of the shared tail entity, i.e., o1=s2 or o1=o2.

[0063] Figure 3b is a specific example graph of a fuzzy knowledge graph, such as Figure 3c As shown in the figure, the triples in the fuzzy knowledge graph are regarded as nodes, and the nodes are connected according to the above two connection modes to obtain the following Figure 3d Specifically, the first view can be obtained by connecting the nodes according to the connection mode of the shared head entity, and the second view can be obtained by connecting the nodes according to the connection mode of the shared tail entity.

[0064] It should be understood that for each triple in the fuzzy knowledge graph, the triple corresponds to a node in the first view and the second view, and the fuzziness of each triple can be evaluated subsequently by the consistency between the representations learned from the first view and the second view.

[0065] In this embodiment, the first view and the second view can be regarded as consistent views of the fuzzy knowledge graph. The first view models the correlation between triples from the perspective of the head entity, and the second view represents the distribution of relation triples from the perspective of the tail entity. Since triples sharing the same entity are always semantically related, normal triples in the fuzzy knowledge graph can easily find enough associated neighboring triples in the first view to reconstruct the semantics learned from the second view.

[0066] In some optional embodiments, encoding the triples in the first view using the knowledge graph embedding model to obtain a first embedding includes:

[0067] For each first triplet in the first view, obtaining a first initial embedding of the first triplet;

[0068] The first initial embedding is updated based on the attention coefficients of neighboring triples of the first triplet to obtain the first embedding.

[0069] In some optional embodiments, encoding the triples in the second view using the knowledge graph embedding model to obtain a second embedding includes:

[0070] For each second triplet in the second view, obtaining a second initial embedding of the second triplet;

[0071] The second initial embedding is updated based on the attention coefficients of neighboring triplets of the second triplet to obtain the second embedding.

[0072] In step 102, the embedding of entities and relations in the fuzzy knowledge graph is first randomly initialized, and then a local modeling layer (i.e., a set of bidirectional long short-term memory network units (Bi-LSTM)) is constructed to learn the local relational structure within each triple. Local structural information in a triple can be captured through local modeling.

[0073] like Figure 4 As shown, taking the triple (s, p, o) as an example, the formula (1) of the local information modeling layer is as follows. The triple embedding q can be obtained by the following formula i :

[0074] q i =G local (s,p,o)=f concat (f BiLSTM (e s ,e p ,e o ))(1);

[0075] In the triple (s, p, o), s is used to represent the subject, p is used to represent the predicate, o is used to represent the object, and f is used to represent the verb. concat () is used to characterize f BiLSTM () is a function for concatenating vectors, which is usually used to merge feature vectors from multiple different sources into a longer vector. BiLSTM () is used to represent a bidirectional long short-term memory network (Bidirectional Long Short-Term Memory, BiLSTM), which is used to input vector (e s ,e p ,e o ) performs sequence modeling to capture contextual information. s ,e p ,e o ) is used to represent the embedding representation vector of the subject, predicate, and object.

[0076] Since the above triples are embedded in q iThe relational structure inside the input triple can be well captured, so in some embodiments, the triple embedding is used as the initial embedding. Specifically, for each first triple in the first view, the first initial embedding of the first triple is obtained according to the above formula (1); similarly, for each second triple in the second view, the second initial embedding of the second triple is obtained according to the above formula (1).

[0077] In some embodiments, a triple-level attention mechanism is introduced to learn the global triple representation of the first triple and the second triple. Since a high-ambiguity triple has a weaker relationship connection with its neighboring entities, the attention mechanism can adaptively assign a smaller attention weight to the high-ambiguity triple.

[0078] The following takes the first triple as an example to illustrate: Based on its neighboring triples {q1,q2,...,q m}’s attention coefficient to update its embedding representation.

[0079] The attention coefficient (also called weight) between a given triplet i and its adjacent j-th triplet is calculated as follows:

[0080] att ij =f att (Wq i ,Wq j )(2);

[0081] Among them, att ij It is used to characterize the importance of its neighboring j-th triplet to the given triplet i. is a learnable linear augmentation matrix used to project the initial triplet representation into the same vector space, f att Used to characterize the attention function:

[0082] Furthermore, in some embodiments, in order to make the attention coefficients easier to compare between different triplets, they are normalized using a softmax function:

[0083]

[0084] Among them, m is used to represent the number of neighboring triples, att ik Used to characterize the weight between a given triple i and its neighboring kth triple.

[0085] Furthermore, in some embodiments, in order to prevent the propagation of information from high-ambiguity triples, a hyperparameter is introduced As threshold:

[0086]

[0087] After the weights between the given triple i and its neighboring triples are calculated in the above manner, the first initial embedding of the given triple is updated based on the attention coefficients of the neighboring triples of the given triple to obtain the first embedding. Specifically, it can be calculated using the sigmoid function:

[0088]

[0089] Among them, σ() is used to represent the sigmoid function.

[0090] For the second triplet in the second view, the attention aggregation of its neighborhood features can be performed with reference to the above content to update its embedding to obtain its corresponding second embedding. The specific steps can be found in the above content and will not be described here.

[0091] For the second triplet, the second embedding is as follows:

[0092]

[0093] Among them, α i ' j It is used to characterize the importance of its neighboring j-th triple to the given triple i, q' j Used to characterize the jth triplet in its neighborhood.

[0094] It should be noted that for each triple in the fuzzy knowledge graph, the triple corresponds to a node in both the first view and the second view. However, since the first view and the second view are constructed according to different connection modes, the neighboring triples corresponding to the same triple in the first view may be different from the neighboring triples corresponding to the triple in the second view, so the resulting embeddings are also different. In the above manner, different embeddings corresponding to the same triple in the fuzzy knowledge graph can be obtained.

[0095] In step 103, the knowledge graph embedding model is trained based on the fuzzy RDF KG embedding loss and the contrastive learning loss. Figure 5 , the comprehensive objective function is defined as follows:

[0096] L=L con +αL emb (7);

[0097] Among them, L emb Used to characterize the fuzzy RDF KG embedding loss, L conIt is used to characterize contrastive learning loss. α is a hyperparameter. In specific implementation, the hyperparameter can be dynamically adjusted according to different data sets to obtain the best model effect. Its specific value is not limited here.

[0098] Through the above method, the fuzzy RDF KG embedding loss and contrastive learning loss are combined to train the model, so that they promote each other in each iteration and improve the effect to obtain the best embedding learning result. The fuzzy RDF KG embedding loss and contrastive learning loss are explained below.

[0099] Optionally, in some embodiments, before step 103, the method further includes:

[0100] Taking the triples in the fuzzy knowledge graph as positive example triples;

[0101] Randomly replacing the entity relations and relations of the positive triples with other entities and relations to obtain negative triples;

[0102] The fuzzy RDF KG embedding loss is calculated based on the difference score of the positive triplet and the difference score of the negative triplet.

[0103] To calculate the fuzzy RDF KG embedding loss, first construct the negative example triples:

[0104] The entity relations and relations of the positive triples are randomly replaced with other entities and relations to obtain the negative triples T′. Furthermore, to prevent noise, the negative triples in T′ are checked in T, and the pseudo negative triples are removed. The final negative triples are as follows:

[0105] T′={(s′,p,o)|s′∈L v}∪{(s,p,o′)|o′∈L v}∪{(s,p′,o)|p′∈L E}(8);

[0106] Among them, L v It is used to characterize the feature set related to the entity (subject, object), which can be understood as the attributes or features related to the entity in the triple. E A set of features related to a relationship (predicate) is used to represent the features or attributes of the relationship in a triple.

[0107] It should be noted that the knowledge graph embedding model provided in this embodiment is a translation operation performed on the low-dimensional embedding of entities at the triple level. Therefore, the more a triple conforms to the translation hypothesis, the better the translation operation is. The more credible the triple should be.

[0108] In this embodiment, the training objective uses negative sampling to formalize the margin-based scoring function. The goal of this pairwise scoring function is to make the score of the positive triple higher than the score of the negative triple. It allows the model to focus on those triples with high membership (i.e., those that are considered more important or reliable in the knowledge graph). The fuzzy RDF KG embedding loss is as follows:

[0109]

[0110] Among them, E(s,p,o) is used to characterize the difference score of the positive example triple, E(s′,p′,o′) is used to characterize the difference score of the negative example triple, γ>0 is used to characterize the hyperparameter of the marginal value, and T′ is used to characterize the set of negative example triplets.

[0111] Contrastive learning is a widely used self-supervised learning algorithm, the core idea of ​​which is to make the representation of positive sample pairs as consistent as possible, and the representation of negative sample pairs as different as possible. The negative sampling used in the fuzzy RDF KG embedding loss can guide the model to learn rich structural and semantic information within the triple. In order to further effectively learn the knowledge graph embedding model, the knowledge graph embedding model is trained by combining contrastive learning loss in this embodiment.

[0112] Contrastive learning can learn rich representations of key semantics in normal instances, while the lack of such features in abnormal triplets forces their representations to be dispersed in the latent vector space. Therefore, the consistency of the representations of triplets in different views can theoretically serve as a good learning signal. By learning and comparing between the first view and the second view, the knowledge graph embedding model can learn rich representations of key semantics in normal instances, thereby forcing abnormal triplets to stay away from the lack of such features in the latent vector space. It should be understood that in order to integrate contrastive learning into knowledge graph embedding, this embodiment uses normalized temperature-scaled cross entropy loss as a contrastive learning objective to jointly train the model.

[0113] Optionally, in some embodiments, the contrastive learning loss is determined based on the similarity between the first embedding and the second embedding corresponding to the triple in the fuzzy knowledge graph.

[0114] Specifically, we randomly sample a mini-batch of n examples from the fuzzy knowledge graph and define a training task on the augmented example pairs derived from the mini-batch, thus obtaining n first embeddings {x1,x2,...,x n} and n second embeddings {z1,z2,...,z n},(x i ,z i) is the corresponding embedding of the same triple (the i-th triple) in the fuzzy knowledge graph in the first view and the second view respectively.

[0115] Optionally, the contrastive learning loss is:

[0116]

[0117] Among them, {x1,x2,...,x n} represents the first embedding corresponding to n triples in the fuzzy knowledge graph, {z1,z2,...,z n} represents the second embedding corresponding to n triples in the fuzzy knowledge graph, (x i ,z i ) represents the first embedding corresponding to the i-th triple in the fuzzy knowledge graph and the second embedding corresponding to the i-th triple in the fuzzy knowledge graph, τ represents the temperature parameter, sim(x i ,z i ) represents x i and z i The cosine similarity of , n is an integer greater than 2, and i is an integer less than or equal to n.

[0118] In this embodiment, contrastive learning under multiple views of the knowledge graph can learn rich key semantic representations in deterministic triples, while in uncertain or abnormal triples, ambiguity leads to the loss of triple features, making their representations far apart in the latent vector space. Therefore, in this embodiment, the closeness of the representation of the same triple in different views is used as an indicator for knowledge graph representation learning, and the credibility of each triple is evaluated based on the consistency between the representations learned from multiple views, so as to perceive the uncertainty in the knowledge graph and learn better knowledge representation.

[0119] As a specific embodiment, the following describes the values ​​of each hyperparameter in the knowledge graph embedding model during specific implementation. First, a grid search is performed on the three hyperparameters of learning rate (lr) in the training process, margin (γ) in formula (9), and attention threshold (μ) in formula (4) to determine the optimal hyperparameter combination. The value range of the hyperparameters is shown in Table 1.

[0120] Table 1 Grid search results for hyperparameters

[0121] Hyper-parameters Search space Learning rate(lr) [0.001,0.01,0.1] Margin(γ) [0.1,0.3,0.5,0.7,1.0] Attention threshold(μ) [0.001,0.01,0.05,0.1,0.2]

[0122] The learning rate is a key hyperparameter that controls the update amplitude of model parameters. A smaller learning rate allows the model to make more precise parameter adjustments, which may converge to a better solution in a longer training process; while a larger learning rate makes the model parameters update faster, but it may also lead to unstable convergence or fall into a suboptimal solution too early. In order to study the impact of the learning rate on model performance, this experiment changes it from 0.001 to 0.1. The experimental results show that when lr = 0.1, the performance of the model is not ideal in different parameter combinations. This is because a larger learning rate causes the model parameters to update too quickly, which may skip the global optimal solution and make the model unable to fully learn the intrinsic structure and regularity of the data. As the learning rate decreases, the update pace of the model slows down, and the parameter adjustment is more refined, which is conducive to the model converging to a better solution in a more stable training process. When lr = 0.01, the performance of the model is optimal. However, when lr continues to drop to 0.001, the actual effect is not as ideal as when lr is higher. Therefore, a moderate learning rate can achieve a good balance between stable convergence and shorter training time, thereby achieving the best detection effect.

[0123] γ is a parameter that represents the distance between negative pairs and positive pairs in the margin control fuzzy RDF KG embedding loss. In order to study the impact of γ, this experiment changes it from 0.1 to 1.0. Except for the optimal γ of 0.3 when lr = 0.01, and the model effect is not much different from that when γ = 0.1, the remaining optimal values ​​are all 0.1. In summary, the optimal γ is 0.1.

[0124] μ is a parameter that controls the attention threshold. A smaller μ allows more irrelevant neighbors to be paid attention to when learning global triplet embedding representations, while a larger μ forces the encoder to focus only on a few but closely related neighbor triplets. In order to study the impact of μ, it was varied from 0.001 to 0.02 in this experiment. When μ = 0.02, the performance on all parameter combinations is not ideal. This is because when μ is large, only a small amount of global context features are mined, lacking sufficient context information. As μ decreases, the encoder begins to pay attention to more neighbor triplets, thereby learning richer global information, so the performance continues to improve. The best performance is achieved at μ = 0.01. However, when μ continues to decrease, the situation changes in the opposite direction, because a smaller μ allows more irrelevant and highly ambiguous neighbors to participate in the representation learning process, leading to the overfitting problem of representation learning.

[0125] Furthermore, the hyperparameter τ in formula (10) is used to control the degree of scaling of the similarity when calculating the loss function. A smaller τ will cause the similarity score to be amplified, so that the model will emphasize the difference between positive sample pairs with high similarity and negative sample pairs with low similarity, which will cause the model to more strictly distinguish between positive and negative sample pairs during the learning process. A larger τ value will cause the similarity score to be reduced in the softmax function, making the model less sensitive to the difference between positive and negative sample pairs. According to experiments, the optimal value of τ is 0.5, which is in the middle.

[0126] In summary, as a specific embodiment, the above hyperparameter combination is: Learning rate (lr) = 0.01, Margin (γ) = 0.1, Attention threshold (μ) = 0.001, τ = 0.5.

[0127] The following will take a specific experiment as an example to illustrate the beneficial effects of the fuzzy knowledge graph contrast embedding method provided by an embodiment of the present invention.

[0128] First, we introduce the detailed experimental settings, including benchmark datasets, comparative baseline methods, and evaluation metrics.

[0129] Dataset: In this experiment, the trained knowledge graph embedding model provided in this embodiment (referred to as A Joint Learning Framework for Fuzzy RDF KGContrastive Embedding (FuKCE) model in this experiment) is evaluated based on WN18RR. WN18RR was created to overcome the test data leakage problem in the original WN18 dataset. By excluding simple reverse relationships, it improves the fairness and accuracy of model evaluation. Based on WN18RR, three different fuzzy knowledge graphs WN18RR-N1, WN18RR-N2, and WN18RR-N3 were generated, with negative triples accounting for 5%, 10%, and 15% of the positive triples, respectively. On this basis, experiments were conducted on the model effects of different fuzzy triple ratios.

[0130] Baseline: URGE is the first model for computing uncertain graph embeddings, but it cannot handle multi-class relations and only generates node embeddings. Therefore, relation types are also ignored in the experiments. BEUrRE is the first model to use geometric embedding methods for UKG embedding. In this model, each entity is encoded as a Gumbel box representation whose volume represents the marginal probability. Relations are modeled by two affine transformations on the boxes of the head and tail entities. UKGsE first uses Word2Vec to pre-train word vectors, and then learns the corresponding confidence levels through LSTM. UKGEbert builds on this and embeds latent commonsense semantics through BERT. In this model, each piece of knowledge is regarded as a short sentence and input into BERT for training, and then the uncertainty distribution of knowledge confidence is learned through a recurrent neural network. UPGAT is an uncertainty-aware graph attention mechanism that is used to capture probabilistic subgraph features while alleviating the irrelevant neighbor problem.

[0131] Evaluation metrics: Ranking metrics are used in this experiment to evaluate the performance of all comparison methods. Specifically, all triples are sorted in ascending order according to their fuzzy membership in the target knowledge graph. In order to fairly evaluate the performance of knowledge graph verification, the following two evaluation metrics are used in this experiment:

[0132] Precision@K: represents the percentage of negative triples found among the top K triplets with the lowest fuzzy membership.

[0133] Recall@K: It indicates the proportion of recognized negative triples in all negative triples.

[0134] Implementation Details: All baseline methods and the proposed framework were implemented in PyTorch in this experiment. Specifically, all models were optimized using the Adam optimizer with a fixed batch size of 256. The model parameters were initialized using the default Xavier initializer with an initial learning rate of 0.01. The embedding dimension of all models was fixed to 100. For hyperparameter selection, a grid search method was used, which involves traversing a given parameter grid to find the best parameter settings. This means determining different ranges and combinations of parameter values ​​and utilizing a validation set to help evaluate model performance.

[0135] In this experiment, a comprehensive comparative experiment was carried out on WN18RR-N1, and the experimental results are shown in Table 2.

[0136] Table 2 Experimental results on WN18RR-N1

[0137]

[0138]

[0139] From the analysis of the results in Table 2, we can see that no matter what the value of K is, the performance of the FuKCE model is significantly better than other methods. Specifically, in terms of the Precision@K and Recall@K indicators, FuKCE has shown extremely strong performance at each K value, especially in Precision@K, when K=1%, FuKCE's score is as high as 0.942, far exceeding other models. When K=5%, although Precision@K drops slightly to 0.598, it is still the highest among all models. This superiority shows that the FuKCE model has strong robustness and stability in terms of accuracy, and can always maintain its leading position under different K values.

[0140] UPGAT uses an enhanced attention mechanism to learn knowledge graph representation. Although the effect is very outstanding compared to other models, it is still not as good as the FuKCE method provided in this embodiment. This is because UPGAT applies attention mechanisms from the perspectives of entities and relationships respectively, and the uncertainty in the knowledge graph usually occurs at the triple level, that is, entities exist naturally in the knowledge graph, and errors are usually caused by the mismatch between the head entity, the tail entity and the corresponding relationship. Therefore, FuKCE uses triple-level embedding as input, which can effectively filter out highly ambiguous triples.

[0141] According to the above content, the loss function of this embodiment is mainly composed of two parts: fuzzy RDF KG embedding loss and contrastive learning loss. In view of the above content, this experiment conducted an ablation study. In the two variants of Variant_Kge and Variant_Con, only the translation-based fuzzy RDF KG embedding loss (i.e., negative sampling) and contrastive learning loss were used to train the model. The experimental results are shown in Table 3:

[0142] Table 3 Experimental results of ablation study

[0143]

[0144] According to the content shown in Table 3, only the variant of fuzzy RDF KG embedding loss, Var_Kge, has a decreased performance under different k, especially Precision begins to decrease significantly after ratio = 0.005, which shows that although the knowledge graph embedding loss helps to identify abnormal samples, its effect is not as good as the model combined with contrastive learning. The variant of contrastive learning loss, Var_Con, only has results output at K = 0.1%, because Var_Con mainly relies on the relative similarity between sample pairs and does not combine the global structural information of Var_Kge. When the feature difference between abnormal samples and normal samples is small, the model cannot effectively distinguish between the two, resulting in abnormal samples being saturated in the small top_k range. The abnormal samples cannot be further distinguished from normal samples in the subsequent ratios. This shows that the model that relies only on contrastive learning loss lacks the ability to capture abnormal samples globally and is prone to instability and insufficiency in detection. Therefore, the respective limitations of these two loss functions indicate that in the anomaly detection task, a model with a single loss function is difficult to perform well in all scenarios, while the method combining contrastive learning and knowledge graph embedding loss can better balance local features and global structures, thereby improving the overall performance of the model.

[0145] In order to explore the effect of the FuKCE model provided in this embodiment under different fuzzy triplet ratios, experiments were carried out on WN18RR-N1, WN18RR-N2 and WN18RR-N3 respectively, and the experiments used the optimal parameter combination (i.e., the aforementioned Learning rate (lr) = 0.01, Margin (γ) = 0.1, Attention threshold (μ) = 0.001, τ = 0.5).

[0146] According to the experimental results, as the proportion of fuzzy triples increases, the Precision@K of different data sets shows a gradual downward trend. WN18RR-N3 shows the highest Precision at all K values, which shows that FuKCE can maintain high accuracy even when the uncertainty level is high. This is due to the fuzziness handling ability of the method. When the proportion of negative triples is high, the impact of fuzzy triples on the results is reduced by better distinguishing the embedding space. WN18RR-N1 performs the worst in Precision@K, which may be because when the uncertainty level is low, the model has not fully utilized the advantages of fuzziness, resulting in slightly lower accuracy in practical applications. The performance trends of Recall@K of different data sets are relatively consistent, and they all gradually improve with the increase of K value. Unlike Precision@K, Recall@K pays more attention to the coverage of retrieval results, so as K increases, Recall will naturally increase.

[0147] In general, the FuKCE model shows a certain robustness on KGs with different ambiguity levels. The results of Precision@K and Recall@K show that as the ambiguity ratio increases, the model shows a balance between precision and recall.

[0148] The fuzzy knowledge graph embedding method in the prior art mainly relies on synthetic negative triples to train the model, that is, the negative triples are generated by replacing the head entity or the tail entity in the triple with a random entity. In the method provided in this embodiment, contrastive learning of negative triples that do not require synthesis is introduced to train the model, and the fuzzy RDF KG embedding loss and contrastive learning loss are combined for joint training to learn the uncertainty degree of the fuzzy triples, so that the model has a better effect and further improves the quality of fuzzy knowledge graph embedding.

[0149] See also Figure 6 , an embodiment of the present invention provides a fuzzy knowledge graph comparison and embedding device 600, comprising:

[0150] A construction module 601 is used to construct a first view and a second view based on a fuzzy knowledge graph, wherein the fuzzy knowledge graph includes a plurality of triples, the first view is used to represent the correlation between the plurality of triples from the perspective of a head entity, and the second view is used to represent the correlation between the plurality of triples from the perspective of a tail entity;

[0151] An encoding module 602 is used to encode the triples in the first view using the knowledge graph embedding model to obtain a first embedding, and to encode the triples in the second view using the knowledge graph embedding model to obtain a second embedding;

[0152] A training module 603 is used to train the knowledge graph embedding model based on fuzzy RDF KG embedding loss and contrastive learning loss to obtain a trained knowledge graph embedding model;

[0153] The acquisition module 604 is used to acquire the knowledge graph embedding based on the trained knowledge graph embedding model.

[0154] Optionally, the construction module 601 includes:

[0155] a processing unit, configured to treat each triple of the plurality of triples as a node;

[0156] A connection unit is used to connect a first node and a second node to obtain the first view, and to connect a third node and a fourth node to obtain the second view, wherein the triples corresponding to the first node and the second node have the same head entity, and the triples corresponding to the third node and the fourth node have the same tail entity.

[0157] Optionally, the encoding module 602 includes:

[0158] A first acquisition unit, configured to acquire, for each first triple in the first view, a first initial embedding of the first triple;

[0159] A first updating unit is used to update the first initial embedding based on the attention coefficient of the neighboring triplet of the first triplet to obtain the first embedding.

[0160] Optionally, the encoding module 602 includes:

[0161] A second acquiring unit, configured to acquire, for each second triplet in the second view, a second initial embedding of the second triplet;

[0162] A second updating unit is used to update the second initial embedding based on the attention coefficient of the neighboring triplet of the second triplet to obtain the second embedding.

[0163] Optionally, the fuzzy knowledge graph comparison and embedding device 600 further includes:

[0164] A processing module, used for taking the triples in the fuzzy knowledge graph as positive example triples;

[0165] A replacement module, used for randomly replacing the entity relations and relations of the positive triples with other entities and relations to obtain negative triples;

[0166] A calculation module is used to calculate the fuzzy RDF KG embedding loss based on the difference score of the positive example triple and the difference score of the negative example triple.

[0167] Optionally, the contrastive learning loss is determined based on a similarity between the first embedding and the second embedding corresponding to a triple in the fuzzy knowledge graph.

[0168] Optionally, the contrastive learning loss is:

[0169]

[0170] Among them, {x1,x2,...,x n} represents the first embedding corresponding to n triples in the fuzzy knowledge graph, {z1,z2,...,z n} represents the second embedding corresponding to n triples in the fuzzy knowledge graph, (x i ,z i) represents the first embedding corresponding to the i-th triple in the fuzzy knowledge graph and the second embedding corresponding to the i-th triple in the fuzzy knowledge graph, τ represents the temperature parameter, sim(x i ,z i ) represents x i and z i The cosine similarity of , n is an integer greater than 2, and i is an integer less than or equal to n.

[0171] The fuzzy knowledge graph comparison and embedding device 600 provided in the embodiment of the present application can execute the above method embodiment, and its implementation principle and technical effects are similar, which will not be repeated in this embodiment.

[0172] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0173] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0174] like Figure 7 As shown, an embodiment of the present application provides an electronic device 700, including: a memory 702, a processor 701, and a program stored on the memory 702 and executable on the processor 701; the processor 701 is used to read the program in the memory 702 to implement the steps in the fuzzy knowledge graph comparison embedding method as described above.

[0175] An embodiment of the present application also provides a readable storage medium, on which a program is stored. When the program is executed by a processor, the various processes of the above-mentioned fuzzy knowledge graph comparison and embedding method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here. Among them, the readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.), optical storage (such as compact disk (CD), digital video disc (DVD), Blu-ray Disc (BD), high-definition versatile disc (HVD), etc.), and semiconductor memory (such as read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read only memory (EEPROM), non-volatile memory (NAND FLASH), solid-state drive (Solid State Disk or Solid State Drive, SSD)), etc.

[0176] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0177] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0178] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A fuzzy knowledge graph contrast embedding method, characterized by: include: Constructing a first view and a second view based on a fuzzy knowledge graph, wherein the fuzzy knowledge graph includes a plurality of triples, the first view is used to represent the correlation between the plurality of triples from the perspective of a head entity, and the second view is used to represent the correlation between the plurality of triples from the perspective of a tail entity; Encoding the triples in the first view using a knowledge graph embedding model to obtain a first embedding, and encoding the triples in the second view using the knowledge graph embedding model to obtain a second embedding; The knowledge graph embedding model is trained based on fuzzy RDF KG embedding loss and contrastive learning loss to obtain a trained knowledge graph embedding model; Obtain knowledge graph embedding based on the trained knowledge graph embedding model.

2. The method according to claim 1, characterized in that: The step of constructing the first view and the second view based on the fuzzy knowledge graph includes: Taking each triple of the plurality of triples as a node; The first view is obtained by connecting the first node and the second node, and the second view is obtained by connecting the third node and the fourth node. The triples corresponding to the first node and the second node have the same head entity, and the triples corresponding to the third node and the fourth node have the same tail entity.

3. The method according to claim 1, characterized in that: The step of encoding the triples in the first view by using the knowledge graph embedding model to obtain a first embedding includes: For each first triplet in the first view, obtaining a first initial embedding of the first triplet; The first initial embedding is updated based on the attention coefficients of neighboring triples of the first triplet to obtain the first embedding.

4. The method according to claim 1, characterized in that: The step of encoding the triples in the second view by using the knowledge graph embedding model to obtain a second embedding includes: For each second triplet in the second view, obtaining a second initial embedding of the second triplet; The second initial embedding is updated based on the attention coefficients of neighboring triplets of the second triplet to obtain the second embedding.

5. The method according to claim 1, characterized in that: Before training the knowledge graph embedding model based on fuzzy RDF KG embedding loss and contrastive learning loss to obtain the trained knowledge graph embedding model, the method further includes: Taking the triples in the fuzzy knowledge graph as positive example triples; Randomly replacing the entity relations and relations of the positive triples with other entities and relations to obtain negative triples; The fuzzy RDFKG embedding loss is calculated based on the difference score of the positive triplet and the difference score of the negative triplet.

6. The method according to any one of claims 1 to 5, characterized in that: The contrastive learning loss is determined based on a similarity between the first embedding and the second embedding corresponding to a triple in the fuzzy knowledge graph.

7. The method according to claim 6, characterized in that: The contrastive learning loss is: Among them, {x1,x2,...,x n } represents the first embedding corresponding to n triples in the fuzzy knowledge graph, {z1,z2,...,z n } represents the second embedding corresponding to n triples in the fuzzy knowledge graph, (x i ,z i ) represents the first embedding corresponding to the i-th triple in the fuzzy knowledge graph and the second embedding corresponding to the i-th triple in the fuzzy knowledge graph, τ represents the temperature parameter, sim(x i ,z i ) represents x i and z i The cosine similarity of , n is an integer greater than 2, and i is an integer less than or equal to n.

8. A fuzzy knowledge graph comparison embedding device, characterized in that: include: A construction module, configured to construct a first view and a second view based on a fuzzy knowledge graph, wherein the fuzzy knowledge graph includes a plurality of triples, wherein the first view is used to represent the correlation between the plurality of triples from the perspective of a head entity, and the second view is used to represent the correlation between the plurality of triples from the perspective of a tail entity; an encoding module, configured to encode the triples in the first view using a knowledge graph embedding model to obtain a first embedding, and to encode the triples in the second view using the knowledge graph embedding model to obtain a second embedding; A training module, used for training the knowledge graph embedding model based on fuzzy RDF KG embedding loss and contrastive learning loss to obtain a trained knowledge graph embedding model; An acquisition module is used to acquire knowledge graph embedding based on the trained knowledge graph embedding model.

9. An electronic device, comprising: A memory, a processor, and a program stored in the memory and executable on the processor; wherein the processor is used to read the program in the memory to implement the steps in the fuzzy knowledge graph comparison embedding method as described in any one of claims 1 to 7.

10. A readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of the fuzzy knowledge graph comparison and embedding method as described in any one of claims 1 to 7 are implemented.

Images