Entity attribute similarity calculation method and system based on BERT

Abstract

The application provides a BERT-based entity attribute similarity calculation method and system, which filters out the relationship triplets, redundant or invalid triplets in the knowledge graphs through preprocessing two knowledge graphs, and obtains pre-aligned entity pairs; aligns the relationships of the first knowledge graph and the second knowledge graph; uses the BERT model to perform unsupervised learning on the tail entities, i.e. attribute texts, in the attribute triplets, and obtains a trained BERT model; according to the relationship alignment result, the attribute text similarity with the same relationship is calculated through the trained BERT model, and the results of the entity attribute similarity are obtained through calculation and integration; the application can calculate the entity attribute similarity between the knowledge graphs with high accuracy through the unsupervised algorithm in the case that the knowledge graph data is huge and lacks annotation, provides support for the fusion of the knowledge graphs, and can effectively solve the problems of high labor cost and low accuracy of the annotated samples in the knowledge graph fusion process.

Description

Technical Field

[0001] This invention relates to a method and system for calculating entity attribute similarity based on BERT, belonging to the field of semantic network technology. Background Technology

[0002] In recent years, with the continuous development of hardware and network technologies, artificial intelligence technologies such as deep learning and knowledge graphs are developing rapidly, propelling society towards the era of "AI+". Knowledge graphs were officially proposed by Google on May 17, 2012, with the initial aim of improving search engine capabilities and enhancing user search quality and experience.

[0003] Currently, with the continuous development of network services, knowledge graphs have been widely applied in fields such as intelligent search, intelligent question answering, and personalized recommendations. Essentially, a knowledge graph is a semantic network that represents the relationships between entities, effectively describing the relationships between different entities. In various industries such as finance, education, and healthcare, it is widely used for complex data analysis and decision support.

[0004] However, with the rapid development of network technology and big data technology, there are massive amounts of heterogeneous data in the network. The knowledge graphs generated from this data have problems such as uneven knowledge quality, duplication of knowledge from different data sources, and unclear relationships between knowledge. Therefore, it is necessary to integrate knowledge graphs to ensure their quality.

[0005] Knowledge graph fusion is a high-level organization of knowledge. It involves integrating entities from different knowledge graphs under the same standard through steps such as heterogeneous data integration, disambiguation, processing, reasoning verification, and updating, achieving the fusion of data, relationships, and content to generate higher-quality knowledge graphs. Current knowledge graph fusion methods mostly employ supervised machine learning methods when calculating entity attribute similarity, which consumes a significant amount of manpower in labeling positive examples.

[0006] The above issues are problems that should be considered and resolved in the process of calculating entity attribute similarity. Summary of the Invention

[0007] The purpose of this invention is to provide a BERT-based method and system for calculating entity attribute similarity, which addresses the problems in existing technologies where supervised machine learning methods suffer from high labor costs and low accuracy due to the large number of entities and the high manual cost of knowledge graph fusion.

[0008] The technical solution of this invention is:

[0009] A BERT-based method for calculating entity attribute similarity includes the following steps:

[0010] S1. Preprocess the first and second knowledge graphs, filtering out relation triples, redundant or invalid triples, retaining attribute triples containing entity-attribute-attribute value, and obtaining pre-aligned entity pairs.

[0011] S2. Align the relationship between the first and second knowledge graphs;

[0012] S3. Use the BERT model to perform unsupervised learning on the tail entity, i.e., the attribute text, in the attribute triple to obtain a trained BERT model.

[0013] S4. Based on the relationship alignment results, the text is converted into a high-dimensional vector using the trained BERT model. The similarity of attribute texts with the same relationship is calculated by comparing the cosine values ​​between the vectors. Finally, the attribute text similarities are summarized and averaged to obtain the similarity results of each entity attribute.

[0014] Furthermore, in step S1, pre-aligned entity pairs in the knowledge graph are obtained by using encyclopedia webpage link URLs or text-to-vector models, i.e., Word2Vec models, which are included in both the first and second knowledge graphs.

[0015] Further, in step S1, obtaining pre-aligned entity pairs specifically involves selecting entities with the same URLs on wiki and Baidu Encyclopedia from both the first and second knowledge graphs as pre-aligned entity pairs if the URLs are the same across different knowledge graphs. Otherwise, the entity pairs with the highest name similarity are selected as pre-aligned entity pairs using the Word2Vec model.

[0016] Furthermore, in step S2, the relationship between the first knowledge graph and the second knowledge graph is aligned, specifically as follows:

[0017] S21. Count all relations contained in the attribute triples in the first and second knowledge graphs, and calculate the similarity of relations in the two knowledge graphs using the Word2Vec model.

[0018] S22. When the similarity exceeds the set value, align the relationships.

[0019] Furthermore, in step S22, when the similarity exceeds a set value, the relationship is aligned. Specifically, when a relationship in one knowledge graph has multiple relationships in another knowledge graph with similarity exceeding the set value, the relationship that exists in a pair of pre-aligned entity pairs is selected as the pair of aligned relationships. Otherwise, the relationship with higher similarity is selected as the aligned relationship.

[0020] Furthermore, in step S3, the BERT model is used to perform unsupervised learning on the tail entity, i.e., the attribute text, in the attribute triples. Specifically,

[0021] S31. According to the batch size, a set number of entity pairs are randomly selected from the first knowledge graph and the second knowledge graph for a batch of training. During the random selection process, entity pairs that have been pre-aligned between knowledge graphs are not selected. The training will take the attribute text of the entity pair itself as the positive example and the attribute text of other randomly selected entity pairs in the same batch as the negative example.

[0022] S32. Encode positive and negative examples using a random discarding method to obtain the corresponding low-dimensional vectors;

[0023] S33. Then, train the BERT model using gradient descent, adjust the parameters to optimize the similarity calculation results, until the loss value is lower than the set threshold, and obtain the trained BERT model.

[0024] A system for implementing the BERT-based entity attribute similarity calculation method described above includes a preprocessing module, a relation alignment module, a model training module, and a similarity calculation module.

[0025] Preprocessing module: preprocesses the first and second knowledge graphs, filtering out relation triples, redundant or invalid triples, retaining attribute triples containing entity-attribute-attribute value, and obtaining pre-aligned entity pairs;

[0026] Relation alignment module: Aligns the relationships between the first and second knowledge graphs;

[0027] Model training module: Use the BERT model to perform unsupervised learning on the tail entity, i.e., the attribute text, in the attribute triple to obtain a trained BERT model;

[0028] Similarity calculation module: Based on the result of relation alignment, the similarity of attribute texts with the same relation is calculated using the trained BERT model. The similarity results of each entity attribute are obtained by integrating the calculations.

[0029] The beneficial effects of this invention are as follows: Compared with existing technologies, this BERT-based entity attribute similarity calculation method and system can reduce manual annotation costs and model training costs by using an unsupervised algorithm when the knowledge graph data is massive and lacks annotations. It also relies on the BERT model to ensure the quality of the final trained model and achieves high accuracy in calculating the similarity of entity attributes between knowledge graphs, thus providing support for knowledge graph fusion. This effectively solves the problems of high manual cost and low accuracy of annotating samples during knowledge graph fusion. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating the entity attribute similarity calculation method based on BERT according to an embodiment of the present invention.

[0031] Figure 2 This is a flowchart illustrating the process of fine-tuning and training the BERT model.

[0032] Figure 3 This is a schematic diagram illustrating the entity attribute similarity calculation system based on BERT in the embodiment. Detailed Implementation

[0033] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0034] Example

[0035] A BERT-based method for calculating entity attribute similarity, such as... Figure 1 This includes the following steps:

[0036] S1. Preprocess the first and second knowledge graphs, filtering out relation triples, redundant or invalid triples, retaining attribute triples containing entity-attribute-attribute value, and obtaining pre-aligned entity pairs.

[0037] In step S1, pre-aligned entity pairs are obtained from the knowledge graphs by using encyclopedia webpage link URLs or the Word2Vec model, which is included in both the first and second knowledge graphs. Specifically, if both the first and second knowledge graphs contain URLs corresponding to entities on wikis and Baidu Encyclopedias, entities with the same URLs across different knowledge graphs are selected as pre-aligned entity pairs; otherwise, the Word2Vec model selects the entity pair with the highest name similarity as the pre-aligned entity pair.

[0038] S2. Align the relationship between the first and second knowledge graphs;

[0039] S21. Count all relations contained in the attribute triples in the first and second knowledge graphs, and calculate the similarity of relations in the two knowledge graphs using the Word2Vec model.

[0040] S22. When the similarity exceeds the set value, align the relationships.

[0041] In step S22, when the similarity exceeds a set value, the relationship is aligned. Specifically, if a relationship in one knowledge graph has multiple relationships in another knowledge graph with similarity exceeding the set value, then the relationship that exists in a pair of pre-aligned entity pairs is selected as the pair of aligned relationships. Otherwise, the relationship with higher similarity is selected as the aligned relationship.

[0042] S3. Use the BERT model to perform unsupervised learning on the tail entity, i.e., the attribute text, in the attribute triple to obtain a trained BERT model.

[0043] In step S3, the BERT model is used to perform unsupervised learning on the tail entity, i.e., the attribute text, in the attribute triples. Specifically, as follows: Figure 2 :

[0044] S31. According to the batch size, a set number of entity pairs are randomly selected from the first knowledge graph and the second knowledge graph for a batch of training. During the random selection process, entity pairs that have been pre-aligned between knowledge graphs are not selected. The training will take the attribute text of the entity pair itself as the positive example and the attribute text of other randomly selected entity pairs in the same batch as the negative example.

[0045] S32. Encode positive and negative examples using a random discarding method to obtain the corresponding low-dimensional vectors;

[0046] S33. Then, train the BERT model using gradient descent, adjust the parameters to optimize the similarity calculation results, until the loss value is lower than the set threshold, and obtain the trained BERT model.

[0047] The BERT model uses a stack of Transformer encoders (self-attention encoders). Each Transformer encoder is a pair of encoders and decoders. Through training, the parameters of the neural network will be corrected, thereby improving the training results of the entire BERT model. It also includes components such as Multi-head Attention (self-attention model), dropout (random dropout layer), and residuals to optimize the neural network.

[0048] S4. Based on the relationship alignment results, the text is converted into a high-dimensional vector using the trained BERT model. The similarity of attribute texts with the same relationship is calculated by comparing the cosine values ​​between the vectors. Finally, the attribute text similarities are summarized and averaged to obtain the similarity results of each entity attribute.

[0049] Compared with existing technologies, this BERT-based entity attribute similarity calculation method and system can reduce manual annotation and model training costs by using an unsupervised algorithm, even with massive amounts of knowledge graph data and a lack of annotations. It relies on the BERT model to ensure the quality of the final trained model and achieves high accuracy in calculating entity attribute similarity between knowledge graphs. This provides support for knowledge graph fusion and effectively solves the problems of high manual cost and low accuracy in annotating samples during knowledge graph fusion.

[0050] like Figure 3 The embodiment also provides a system for implementing the BERT-based entity attribute similarity calculation method described above, including a preprocessing module, a relation alignment module, a model training module, and a similarity calculation module.

[0051] Preprocessing module: preprocesses the first and second knowledge graphs, filtering out relation triples, redundant or invalid triples, retaining attribute triples containing entity-attribute-attribute value, and obtaining pre-aligned entity pairs;

[0052] Relation alignment module: Aligns the relationships between the first and second knowledge graphs;

[0053] Model training module: Use the BERT model to perform unsupervised learning on the tail entity, i.e., the attribute text, in the attribute triple to obtain a trained BERT model;

[0054] Similarity calculation module: Based on the result of relation alignment, the text is converted into a high-dimensional vector using a trained BERT model. The similarity of attribute texts with the same relation is calculated by comparing the cosine values ​​between the vectors. Finally, the attribute text similarities are summarized and averaged to obtain the result of the similarity of each entity attribute.

[0055] This BERT-based entity attribute similarity calculation method and system can calculate the similarity of entity attributes between knowledge graphs using an unsupervised algorithm, even when the knowledge graph data is massive and lacks annotations. It has high accuracy and can effectively avoid the problem of excessive manual cost of annotating samples during the knowledge graph fusion process.

[0056] This BERT-based entity attribute similarity calculation method and system combines the BERT algorithm (a bidirectional encoder representation algorithm based on a self-attention model) with a contrast algorithm. By training the system with text itself as positive examples and other texts as negative examples, it maps the textual information of entity attributes relatively evenly into a low-dimensional, dense vector space. Then, it compares attribute similarity by calculating the distance between vectors. This enables unsupervised, low-cost, and high-precision knowledge graph fusion. It can compare entity attribute similarity even on unlabeled knowledge graphs, reducing the manual cost of knowledge graph fusion, improving the accuracy of knowledge graph fusion, and achieving highly accurate attribute similarity calculation. It solves the problem of high preprocessing costs in supervised attribute similarity calculation algorithms caused by the large number of entities in knowledge graphs.

[0057] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept. The technical solutions of this invention have not been published in any articles.

Claims

1. A method for calculating entity attribute similarity based on BERT, characterized in that: Includes the following steps, S1. Preprocess the first and second knowledge graphs, filtering out relation triples, redundant or invalid triples, retaining attribute triples containing entity-attribute-attribute value, and obtaining pre-aligned entity pairs. S2. Align the relationship between the first and second knowledge graphs; S21. Count all relations contained in the attribute triples in the first and second knowledge graphs, and calculate the similarity of relations in the two knowledge graphs using the Word2Vec model. S22. When the similarity exceeds the set value, perform alignment between relations. Specifically, if a relation in one knowledge graph has multiple relations in another knowledge graph with similarity exceeding the set value, then the relation that exists in a pair of pre-aligned entity pairs is selected as the alignment relation. Otherwise, the relation with higher similarity is selected as the alignment relation. S3. Use the BERT model to perform unsupervised learning on the tail entity, i.e., the attribute text, in the attribute triple to obtain a trained BERT model. S31. According to the batch size, a set number of entity pairs are randomly selected from the first knowledge graph and the second knowledge graph for a batch of training. During the random selection process, entity pairs that have been pre-aligned between knowledge graphs are not selected. The training will take the attribute text of the entity pair itself as the positive example and the attribute text of other randomly selected entity pairs in the same batch as the negative example. S32. Encode positive and negative examples using a random discarding method to obtain the corresponding low-dimensional vectors; S33. Then train the BERT model using gradient descent, adjust the parameters to optimize the similarity calculation results until the loss value is lower than the set threshold, and obtain the trained BERT model. S4. Based on the alignment results, the text is converted into a high-dimensional vector using the trained BERT model. The similarity of attribute texts with the same relationship is calculated by comparing the cosine values ​​between the vectors. Finally, the attribute text similarities are summarized and averaged to obtain the similarity results of each entity attribute.

2. The entity attribute similarity calculation method based on BERT as described in claim 1, characterized in that: In step S1, pre-aligned entity pairs in the knowledge graph are obtained by using encyclopedia webpage link URLs or the Word2Vec model, which are included in both the first and second knowledge graphs.

3. The entity attribute similarity calculation method based on BERT as described in claim 1, characterized in that: In step S1, pre-aligned entity pairs are obtained. Specifically, if both the first and second knowledge graphs contain the URLs of the entities on wiki and Baidu Encyclopedia, then entities with the same URLs between different knowledge graphs are selected as pre-aligned entity pairs; otherwise, the entity pairs with the highest entity name similarity are selected as pre-aligned entity pairs using the Word2Vec model.

4. A system for implementing the BERT-based entity attribute similarity calculation method according to any one of claims 1-3, characterized in that: It includes a preprocessing module, a relation alignment module, a model training module, and a similarity calculation module. Preprocessing module: preprocesses the first and second knowledge graphs, filtering out relation triples, redundant or invalid triples, retaining attribute triples containing entity-attribute-attribute value, and obtaining pre-aligned entity pairs; Relation alignment module: Aligns the relationships between the first and second knowledge graphs; Model training module: Use the BERT model to perform unsupervised learning on the tail entity, i.e., the attribute text, in the attribute triple to obtain a trained BERT model; Similarity calculation module: Based on the result of relation alignment, the text is converted into a high-dimensional vector using the trained BERT model. The similarity of attribute texts with the same relation is calculated by comparing the cosine values ​​between the vectors. Finally, the attribute text similarities are summarized and averaged to obtain the result of the similarity of each entity attribute.

Images