A method for completing Chinese dynamic knowledge graph based on carbon neutrality latent semantics

Abstract

The present invention discloses a method for completing Chinese dynamic knowledge graphs based on the latent semantics of carbon neutrality, which uses key information of carbon neutrality text to facilitate accurate search of carbon neutrality and is also beneficial to carbon neutrality. Based on contrastive learning, combined with positive sample and negative sample data, the problem of insufficient data is effectively alleviated. Taking into account time and similar soft logic, a topological structure method based on time path and a similar soft logic constraint method are proposed to integrate topological structure and other soft logic into the language model to improve the error of insufficient semantic completion. In order to avoid the model completely ignoring the relationship between negative samples and positive samples during training, soft labels and hard labels are used to learn from each other to effectively complete the knowledge graph, so that the obtained results are more reasonable and reliable. In addition to being used for carbon neutrality dynamic knowledge graph completion, the present invention can also be used for other downstream tasks that require quadruple embedding, and has a wide range of application value.

Description

Technical Field

[0001] The present invention relates to the fields of carbon neutrality-related technologies and knowledge graphs, and in particular to a method for completing Chinese dynamic knowledge graphs based on the latent semantics of carbon neutrality. Background Art

[0002] To facilitate the query and application of carbon neutrality data, it is particularly important to build a carbon neutrality knowledge graph. Knowledge completion is one of the important steps in building a knowledge graph.

[0003] Knowledge graphs (KGs) are a way to represent knowledge about a specific domain, such as medicine, finance, or sports. They are typically described as graphs or networks consisting of nodes and edges, where nodes represent entities or concepts and edges represent relationships between them. Entities and relationships form multiple triples, but these relationships are often time-bound. Triples change over time, forming a dynamic knowledge graph. Their advantage is that they can capture complex relationships between entities and facilitate reasoning and inference. Knowledge graphs are also widely used in information retrieval, question answering, recommender systems, and data integration. Due to limited resources and technology, the knowledge in most knowledge graphs is incomplete. However, these incomplete knowledge graphs cannot meet the needs of downstream applications. To address this, the task of knowledge graph completion (KGC) has emerged, aiming to discover and fill in missing information such as entities, relationships, and attributes in incomplete knowledge graphs to improve their completeness and accuracy.

[0004] The task of knowledge graph completion is often inseparable from knowledge embedding. Good knowledge embedding can improve the overall effect of KGC. For example, TransE and DistMult proposed embedding entities and relationships into a continuous vector space for computing knowledge graph completion. However, the above methods only consider a single triple and do not consider the latent semantics of entity relationships. To solve the semantic problem, related models propose to use language models for graph completion, that is, to replace the corresponding entities and relationships in the triples with text descriptions. Common language model methods only consider local triples and do not consider the topological structure and soft logic rules of the entire knowledge graph, and use the two to improve the embedding of entities and relationships. In recent years, a large number of scholars have proposed related solutions, such as using graph attention networks to jointly learn the local structural features and soft logic rules of entities, but they do not utilize the latent semantics of entity relationships. Summary of the Invention

[0005] The present invention mainly overcomes the shortcomings of the existing technology. The purpose of the present invention is to provide a method for completing Chinese dynamic knowledge graph based on carbon neutrality potential semantics.

[0006] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0007] A method for completing a Chinese dynamic knowledge graph based on carbon neutrality latent semantics, characterized in that the method comprises the following steps:

[0008] S1: Separate the carbon neutrality related quaternary into a head entity of the time relation and a separate tail entity;

[0009] S2: Replace the head entity of the temporal relationship obtained above with the relevant text description, and combine the text description into [CLS], [h text ],[SEP],[T],[r text ],[SEP] are fed into the MacBERT encoder to obtain the relevant embedding vector e hTr ;

[0010] S3: Replace the above-obtained single tail entity with the relevant text description, combine its text description into [CLS], [ttext], [SEP] and send it to another MacBERT encoder to obtain the relevant embedding vector e t ;

[0011] S4: Calculate the head entity vector e of the temporal relationship using cosine similarity hTr and tail entity vector e t Perform dot product calculations for triple completion;

[0012] S5: During this period, a comparative experiment was conducted to improve the overall model efficiency by adding negative samples for comparative learning with positive samples;

[0013] S6: The topology and soft logic rules of the knowledge graph also contain some latent semantics. The topology and soft logic rules of the knowledge graph are embedded into the language pre-training model.

[0014] S7: When calculating the original neural network loss using average pooling and the soft label module using max pooling, during the overall model testing phase, average pooling is used to obtain one set of prediction scores. Then, the text description sentence embeddings of the candidate tail entities obtained using max pooling are used to obtain another set of prediction scores. Finally, the two sets of scores are compared and analyzed to obtain the final prediction result.

[0015] The present invention provides a method for completing a Chinese dynamic knowledge graph based on carbon-neutral latent semantics. With MacBERT encoding as the core, and taking into account time and similar soft logic, a topological structure method based on a time path and a similar soft logic constraint method are proposed to integrate the topological structure and other soft logic into the language model, and then use positive and negative samples for comparative learning. In order to avoid the model completely ignoring the relationship between negative and positive samples during training, we propose to use soft labels and hard labels to learn together to effectively complete the knowledge graph. In addition to being used for link prediction, the present invention can also be used for other downstream applications of the required quadruple embedding method, and has a wide range of application value.

[0016] Beneficial effects:

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] With MacBERT encoding at its core and taking into account temporal and similarity soft logic, we propose a temporal path-based topology structure method and similarity soft logic constraint rules to integrate topology and other soft logic into the language model. We then use contrastive learning to improve efficiency. To prevent the model from completely ignoring the relationship between negative and positive samples during training, we propose to effectively complete the knowledge graph by jointly learning soft and hard labels. In addition to being used for link prediction, this invention can also be used in other downstream applications requiring quadruple embedding methods, and has broad application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Complete the overall flow chart for the dynamic carbon neutrality knowledge graph;

[0020] Figure 2 This is a partial picture of the dynamic carbon neutrality knowledge graph. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0022] Example:

[0023] A method for completing a Chinese dynamic knowledge graph based on carbon neutrality latent semantics. The specific calculation method includes the following steps:

[0024] S1: The carbon neutrality related quadruple is divided into a head entity of the time relationship and a separate tail entity and sent to MacBERT for encoding to obtain the head entity vector e of the time relationship. hTr and tail entity vector e t ;

[0025] S2: The head entity vector e of the above-mentioned time relationship hTr and tail entity vector e t Perform similarity calculation. The similarity calculation formula is:

[0026]

[0027] Where S is the similarity between two entities; e hTr is the head entity vector of the time relationship; e t is the tail entity vector.

[0028] S3: Use the similarity dot product result to calculate the prediction score. The prediction score calculation formula is:

[0029]

[0030] Where argmax is the maximum value of the predicted score; t i is the i-th candidate tail entity, e hTr is the head entity vector of the time relationship; is the tail entity vector.

[0031] S4: The above prediction scores are based on the graph re-ranking strategy, and the candidate tail entity t is ranked according to the graph in the training set. i The score increases by α≥0, and the prediction score calculation formula is updated to:

[0032]

[0033] Where argmax is the maximum value of the predicted score; t i is the i-th candidate tail entity, e hTr is the head entity vector of the time relationship; is the tail entity vector; α is the reordering threshold.

[0034] S5: Similar soft logic rule constraint problem. When similar constraints are met, the prediction score increases by β≥0, and the prediction score calculation formula is updated to:

[0035]

[0036] Where argmax is the maximum value of the predicted score; t i is the i-th candidate tail entity, e hTr is the head entity vector of the time relationship; is the tail entity vector; α is the reordering threshold; β is the similar soft logic threshold.

[0037] S6: When calculating the original neural network loss, average pooling is used. When the soft label module uses maximum pooling, then during the overall model testing phase, average pooling is used to obtain the text description sentence embeddings of the head entity and relationship, and average pooling is used to obtain the text description sentence embeddings of the candidate tail entity to obtain a set of prediction scores.

[0038] S7: Use the maximum pooling to obtain the text description sentence embedding of the candidate tail entity to obtain another set of prediction scores. Finally, compare and analyze the two sets of scores to obtain the final prediction result. The prediction score calculation formula is updated to:

[0039]

[0040] Where argmax is the maximum value of the predicted score; t i is the i-th candidate tail entity, e hTr is the head entity vector of the time relationship; is the tail entity vector; α is the reordering threshold; β is the similar soft logic threshold; It is calculated by combining the maximum pooling and average pooling results.

[0041] S8: The prediction results obtained above are the completion results of the carbon neutrality dynamic knowledge graph.

[0042] The present invention provides a method for completing a Chinese dynamic knowledge graph based on carbon-neutral latent semantics. With MacBERT encoding as the core, it uses time and similarity soft logic to propose a topological structure method based on time path and similarity soft logic constraint rules to integrate topological structure and other soft logic into the language model, which can improve the knowledge graph completion results and then use contrastive learning to improve efficiency. In order to avoid the model completely ignoring the relationship between negative samples and positive samples during training, we propose to use soft labels and hard labels to learn together to effectively complete the knowledge graph. In addition to being used for knowledge graph completion, the present invention can also be used for downstream applications of other required quadruple embedding methods, and has a wide range of application value.

[0043] The above description does not limit the present invention to the above embodiments. Any modifications and changes made by professionals within the scope of the technical solution of the present invention using the above content are within the technical scope of the present invention.

Claims

1. A method for completing Chinese dynamic knowledge graph based on carbon neutrality latent semantics, characterized in that: The following steps are involved: S1: Separate the carbon neutrality related quaternary into a head entity of the time relation and a separate tail entity; S2: Replace the head entity of the temporal relationship obtained above with the relevant text description, and combine the text description into [CLS], [h text ],[SEP],[T],[r text ],[SEP] are fed into the MacBERT encoder to obtain the relevant embedding vector e hTr ; S3: Replace the above-obtained single tail entity with the relevant text description, combine its text description into [CLS], [ttext], [SEP] and send it to another MacBERT encoder to obtain the relevant embedding vector e t ; S4: Calculate the head entity vector e of the temporal relationship using cosine similarity hTr and tail entity vector e t Perform dot product calculations for triple completion; S5: During this period, a comparative experiment was conducted to improve the overall model efficiency by adding negative samples for comparative learning with positive samples and re-ranking the graphs; S6: The topology and soft logic rules of the knowledge graph also contain some latent semantics. The topology and soft logic rules of the knowledge graph are embedded in the language pre-training model. S7: When calculating the original neural network loss, average pooling is used, and when the soft label module uses maximum pooling, then during the overall model testing phase, the text description sentence embeddings of the head entity and relationship obtained by average pooling and the text description sentence embeddings of the candidate tail entity obtained by average pooling are used to obtain a set of prediction scores. The text description sentence embeddings of the candidate tail entity obtained by maximum pooling are then used to obtain another set of prediction scores. Finally, the two sets of scores are compared and analyzed to obtain the final prediction result.

2. A method for completing a Chinese dynamic knowledge graph based on carbon neutrality latent semantics as described in claim 1, characterized in that: The head entity vector e of the time relationship obtained in steps S2 and S3 hTr and tail entity vector e t .

3. A method for completing a Chinese dynamic knowledge graph based on carbon neutrality latent semantics as described in claim 1, characterized in that: The similarity calculation formula in step S4 is: In the formula, S is the similarity between two entities; e hTr is the head entity vector of the time relationship; e t is the tail entity vector.

4. A method for completing a Chinese dynamic knowledge graph based on carbon neutrality latent semantics as described in claim 1, characterized in that: The negative samples in step S5 must be sufficient.

5. A method for completing a Chinese dynamic knowledge graph based on carbon neutrality latent semantics as described in claim 1, characterized in that: In the graph-based re-ranking strategy in step S5, the prediction score is increased by α≥0, and the prediction score calculation formula is updated to: Where argmax is the maximum value of the predicted score; t i is the i-th candidate tail entity, e hTr is the head entity vector of the time relationship; is the tail entity vector; α is the reordering threshold.

6. A method for completing a Chinese dynamic knowledge graph based on carbon neutrality latent semantics as described in claim 1, characterized in that: In the similar soft logic rule constraint problem in step S6, when similar constraint conditions are met, the prediction score is increased by β≥0, and the prediction score calculation formula is updated: Where argmax is the maximum value of the predicted score; t i is the i-th candidate tail entity, e hTr is the head entity vector of the time relationship; is the tail entity vector; α is the reordering threshold; β is the similar soft logic threshold.

7. A method for completing a Chinese dynamic knowledge graph based on carbon neutrality latent semantics as described in claim 1, characterized in that: In step S7, the two sets of scores are compared and analyzed to obtain the final prediction result, and the prediction score calculation formula is updated to: Where argmax is the maximum value of the predicted score; t i is the i-th candidate tail entity, e hTr is the head entity vector of the time relationship; is the tail entity vector; α is the re-ranking threshold; β is the similarity soft logic threshold; It is calculated by combining the maximum pooling and average pooling results.

8. The method for completing a Chinese dynamic knowledge graph based on carbon neutrality latent semantics according to claim 1, characterized in that: Two sentence embedding methods are used in step S7.

Images