A knowledge fusion method of encyclopedia graph based on frame matching

Abstract

The present invention discloses an encyclopedia graph knowledge fusion method based on framework matching, which divides the encyclopedia knowledge fusion process into two stages: logic layer and data layer knowledge fusion. In the logic layer data fusion stage, through the processes of concept alignment and attribute unification, the logical description of multi-source encyclopedia data is unified and the semantics is clear; in the data layer knowledge fusion stage, based on the result of framework matching, through entity alignment and attribute fusion, the encyclopedia graph entities from different sources are automatically fused. The present invention realizes the graph logic layer data through concept consistency judgment, concept fusion, and concept attribute fusion, and outputs the fused graph concepts and attributes; on the basis of the logic layer data, the graph data layer entities and their relationship fusion are realized through entity alignment and entity attribute disambiguation, so as to realize the logical unification of multi-source data and clear semantic information, output the fused knowledge graph, and finally realize the automatic fusion of multi-source encyclopedia graph data.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph technology, and in particular to an encyclopedia graph knowledge fusion method based on framework matching. Background Art

[0002] Knowledge is not simply an accumulation of data and information; it refers to information that can be used to guide practice. It is the sum of the understanding and experience people gain through the practice of transforming the world. This includes the formalized results of understanding the objective world through practice, descriptions of facts, skills acquired through education and practice, and systematic understanding gained through various channels, which has been refined, summarized, and condensed.

[0003] Due to different knowledge storage scenarios and acquisition methods, knowledge storage forms vary. The knowledge involved in this project primarily consists of encyclopedia graphs. Encyclopedia graphs are knowledge graphs constructed using encyclopedia websites as data sources. Knowledge fusion refers to the integration of knowledge from multiple sources. It involves aligning, linking, and merging related knowledge into an organic whole, eliminating structural differences and semantic gaps between sources to form a more complete knowledge set. Framework matching, also known as graph logic-layer fusion, involves matching and fusing the knowledge description systems of graphs, including concepts, attributes, and relationships. Different knowledge graphs have different conceptual hierarchies, and a fused knowledge graph can only have one conceptual hierarchy. The key to framework matching is finding equivalent concepts. Because the conceptual hierarchy is crucial and scalable, current mainstream systems rely on manual matching to ensure fusion quality, resulting in low efficiency and accuracy. Summary of the Invention

[0004] The main purpose of this invention is to provide an encyclopedia graph knowledge fusion method based on framework matching. This method divides the encyclopedia knowledge fusion process into two stages: logic-layer and data-layer knowledge fusion. The logic-layer data fusion stage achieves unified logical descriptions and clear semantics for multi-source encyclopedia data through processes such as concept alignment and attribute unification. The data-layer knowledge fusion stage, based on the results of framework matching, automatically integrates entities from encyclopedia graphs from different sources through entity alignment and attribute fusion.

[0005] To achieve the above objectives, the present invention provides an encyclopedia graph knowledge fusion method based on framework matching, the method comprising the following steps:

[0006] S1: Obtain the original entity data from several encyclopedia databases and store them in the encyclopedia entity list to be fused;

[0007] S2: Extract the concept value and attribute type of each encyclopedia entity to be fused, generate a candidate concept-attribute list, and perform concept alignment and attribute fusion in sequence to obtain the fused concept value, attribute type and the corresponding table before fusion;

[0008] S3: According to the correspondence table, the concept values ​​and attribute types of the original entity data in the encyclopedia entity list to be fused are replaced with the fused concept values ​​and attribute types to obtain fused entity data, and entity fusion is performed based on the concept similarity and attribute similarity between the original entity data and the fused entity data to obtain the fused encyclopedia entity.

[0009] Optionally, the expression of the encyclopedia entity list to be integrated is:

[0010] {X entity}={'labels':[label1,...,label n ],'properties':{'pro1':'pro1_value',...,'pro3':'pro3_value'}}, where 'labels' represents the concept list of the entity, label i Indicates the concept value to which the entity belongs.

[0011] 'properties' represents the properties of the entity, pro1_value and pro3_value are the property values ​​of the property types pro1 and pro3 respectively.

[0012] Optionally, the step of extracting the concept value and attribute type of each encyclopedia entity to be fused and generating a candidate concept-attribute list specifically includes:

[0013] From the encyclopedia entity list to be integrated {X entity} to extract the concept label i All attribute types of the entity are stored in the candidate concept-attribute list {concepts} = [{'label i ':['pro i1 ','pro i2 ',...,'pro in ']}], where pro i For {X entity} in label i The corresponding attribute type.

[0014] Optionally, the concept alignment specifically includes:

[0015] S201: Construct the fused concept list fuse_concepts and the correspondence table between fused concepts and original concepts concept_ori2fused, both of which are initially empty;

[0016] S202: Select a concept from the candidate concept-attribute list concepts. If the fused concept list fuse_concepts is empty, add the concept to fuse_concepts. Otherwise, execute step S203.

[0017] S203: Calculate the concept value similarity and attribute type similarity between concept and any concept fuse_concept in fuse_concepts in sequence;

[0018] S204: Based on the similarity of concept values ​​and attribute types, the concept values ​​and attribute types of concept and fuse_concept are merged and duplicated, and the corresponding table between the fused concept and the original concept, concept_ori2fused, is updated.

[0019] Optionally, the attribute fusion specifically includes:

[0020] S211: Build the fused attribute class dictionary fuse_pros, which is initially empty;

[0021] S212: Select the concept values ​​in the fused concept list fuse_concepts in turn, and read all attribute types pro_labels contained in the concept;

[0022] S213: Select the attribute type label in pro_labels in turn. If fuse_pros is empty, add label as the key and the corresponding value to fuse_pros; otherwise, calculate the text cosine similarity between label and all keys in fuse_pros in turn;

[0023] S214: According to the text cosine similarity between label and all keys in fuse_pros, label is added as the corresponding key value to fuse_pros, and the fused attribute class dictionary fuse_pros is output to obtain the correspondence table pro_ori2fused between the fused attributes and the original attributes.

[0024] Optionally, step S3 specifically includes:

[0025] S301: Input the encyclopedia entity list to be integrated {X entity}, based on the correspondence table concept_ori2fused between fused concepts and original concepts, and the correspondence table pro_ori2fused between fused attributes and original attributes, replace the concept values ​​and attribute types in the original entity data with the fused values, and construct the fused entity table fused_entities, which is initially empty;

[0026] S302: Select from the encyclopedia entity list to be integrated {X entity}, if fused_entities is empty, add ori_data to fused_entities, otherwise execute step S303;

[0027] S303: Calculate the similarity between the entity ori_data and any entity fused_entity in the fused entity table in terms of concept and attribute in two dimensions: concept_sim and pros_sims; if the similarity is greater than the threshold, execute step S304; otherwise, add ori_data to fused_entities;

[0028] S304: Merge and deduplicate the concept values ​​of the two entities ori_data and fused_entity, and output the fused encyclopedia entity fused_entities.

[0029] An embodiment of the present invention proposes an encyclopedia graph knowledge fusion method based on framework matching, which divides the encyclopedia knowledge fusion process into two stages: logic layer and data layer knowledge fusion. In the logic layer data fusion stage, through processes such as concept alignment and attribute unification, the logical description of multi-source encyclopedia data is unified and the semantics are clear; in the data layer knowledge fusion stage, based on the results of framework matching, through entity alignment and attribute fusion, the encyclopedia graph entities from different sources are automatically fused. The present invention realizes graph logic layer data through concept consistency judgment, concept fusion, and concept attribute fusion, and outputs fused graph concepts and attributes; on the basis of logic layer data, through entity alignment and entity attribute disambiguation, the graph data layer entities and their relationships are fused, and the logic of multi-source data is unified and the semantic information is clear, and the fused knowledge graph is output, thereby finally realizing the automatic fusion of multi-source encyclopedia graph data, and at the same time, the fusion process and results have a certain degree of explainability. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a flowchart of the encyclopedia graph knowledge fusion method based on framework matching.

[0031] Figure 2 This is a schematic diagram of the encyclopedia graph knowledge fusion principle based on framework matching.

[0032] Figure 3 It is a diagram of Baidu Encyclopedia entity information.

[0033] Figure 4 It is a schematic diagram of entity information on Hudong Encyclopedia.

[0034] Figure 5 It is a process flow chart of the logical layer concept value fusion method.

[0035] Figure 6 It is a schematic diagram of the logical layer attribute type fusion process method.

[0036] Figure 7 It is a schematic diagram of the data layer entity fusion method.

[0037] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0038] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0039] At present, in the relevant technical field, since the conceptual system is very important and the scale is controllable, the current mainstream system mainly uses manual methods for matching to ensure the quality of fusion, and the fusion efficiency and accuracy are not high.

[0040] To address this issue, various embodiments of the framework-matching-based encyclopedia graph knowledge fusion method of the present invention are proposed. The framework-matching-based encyclopedia graph knowledge fusion method provided by the present invention implements graph logical layer data through concept consistency judgment, concept fusion, and concept attribute fusion, and outputs the fused graph concepts and attributes. Based on the logical layer data, the method integrates graph data layer entities and their relationships through entity alignment and entity attribute disambiguation, achieving logical unification and semantic clarity of multi-source data. The method then outputs the fused knowledge graph, ultimately achieving the automatic fusion of multi-source encyclopedia graph data.

[0041] The embodiment of the present invention provides an encyclopedia graph knowledge fusion method based on framework matching, referring to Figure 1 , Figure 1 This is a flow chart of an embodiment of the encyclopedia graph knowledge fusion method based on framework matching of the present invention.

[0042] In this embodiment, the encyclopedia graph knowledge fusion method based on framework matching includes the following steps:

[0043] S1: Obtain the original entity data from several encyclopedia databases and store them in the encyclopedia entity list to be fused;

[0044] S2: Extract the concept value and attribute type of each encyclopedia entity to be fused, generate a candidate concept-attribute list, and perform concept alignment and attribute fusion in sequence to obtain the fused concept value, attribute type and the corresponding table before fusion;

[0045] S3: According to the correspondence table, the concept values ​​and attribute types of the original entity data in the encyclopedia entity list to be fused are replaced with the fused concept values ​​and attribute types to obtain fused entity data, and entity fusion is performed based on the concept similarity and attribute similarity between the original entity data and the fused entity data to obtain the fused encyclopedia entity.

[0046] In order to explain this application more clearly, a specific example of an encyclopedia graph knowledge fusion method based on framework matching is proposed.

[0047] like Figure 2 As shown, taking the fusion of Baidu Encyclopedia and Hudong Encyclopedia knowledge as an example, Baidu Encyclopedia knowledge consists of a single entity, each entity is as follows Figure 3 As shown in the figure, 'labels' represents the concept list of the entity, which is stored in an array, and 'properties' represents the attributes of the entity, which are stored in a dictionary. The dictionary key is the attribute type and the value is the corresponding attribute value. Figure 4 shown.

[0048] The process flow of the knowledge fusion method based on framework matching provided by the embodiment of the present invention includes the following steps:

[0049] 1. Original concept-attribute list acquisition: Get original entity data from Baidu Encyclopedia and Hudong Encyclopedia databases {X baidu_entity}like Figure 3 As shown, {X hudong_entity}like Figure 4 As shown, the original entity list is stored in sequence:

[0050] {X entity}={'labels':[label1,...,label n ],'properties':{'pro1':'pro1_value',...,'pro3':'pro3_value'}}; among them, label i It represents the concept value to which the entity belongs, pro3 represents the attribute type of the entity, and pro3_value represents the attribute value corresponding to the attribute type pro3.

[0051] 2. Logical layer knowledge fusion: Based on methods such as synonyms and text cosine similarity, concept-attribute data are integrated in sequence to complete logical layer knowledge fusion and achieve a unified description of the encyclopedia data at the logical layer.

[0052] 2.1. Generation of candidate concept-attribute list: for {X entity}, each label1, from {X entity} to extract the concept label i All attribute types of the entity are stored in the candidate concept-attribute list {concepts} = [{'label i ':['pro i1 ','pro i2 ',...,'pro in ']}], where pro i For {X entity} in label i The corresponding attribute type.

[0053] 2.2, such as Figure 5 As shown in Figure 2, concept alignment mainly includes the following steps:

[0054] 2.2.1、Input the fused concept list fuse_concepts, the original concept and fused concept correspondence table f_concept2o_concept, both of which are initially empty. The format of fuse_concepts is consistent with that of concepts, {f_concept2o_concept}={'label_f':[label_o1,...,label_o n ]}, f_concept2o_concept is the concept value after fusion, label_on is the concept value before fusion.

[0055] 2.2.2. Select any concept from the candidate concept-attribute list concepts. If the fused concept list fuse_concepts is empty, add concept to fuse_concepts, otherwise go to the next step.

[0056] 2.2.3. Calculate the similarity between the concept value and attribute type of concept and any concept fuse_concept in fuse_concepts. If both the concept value and attribute type similarity are similar, it indicates that concept and fuse_concept refer to the same concept. In this case, proceed to step 2.2.4 and merge the concept values ​​and attribute types of concept and fuse_concept. Otherwise, delete concept from concepts and continue with step 2.2.2 until concepts is empty.

[0057] Concept value similarity calculation: Let the concept values ​​of concept and fuse_concept be Lc =[label 1c ,...,label nc ], L f =[label 1f ,...,label nf ], if L c 、L f For inclusion relationship or L c 、L f There are synonyms in L c 、L f The similarity is 1; otherwise, L c 、L f Concept value splicing, calculate the cosine similarity of the spliced ​​text. If the similarity is higher than the threshold, it means that the concept values ​​of concept and fuse_concept are similar.

[0058] Attribute type similarity calculation: Assume that the attribute types of concept and fuse_concept are pro c =[pro 1c ,...,pro nc ], L d =[label 1d ,...,label nd ], calculate the cosine similarity of the concatenated attribute type texts. If the similarity is higher than the threshold, it means that the attribute types of concept and fuse_concept are similar.

[0059] 2.2.4. Concept value fusion: Based on the similarity between individual concept values ​​and attribute types, merge and deduplicate the concept values ​​and attribute types of concept and fuse_concept, and update the corresponding table f_concept2o_concept between the original concept value and the fused concept value.

[0060] 2.3, such as Figure 6 As shown in the figure, attribute type fusion mainly includes the following steps:

[0061] 2.3.1. Input the fused concept list fuse_concepts, the correspondence table between original concepts and fused concepts f_concept2o_concept, and the fused attribute class dictionary fuse_pros, which are initially empty.

[0062] 2.3.2. Select the concept values ​​in the fused concept list fuse_concepts in turn, read all the attribute types pro_labels contained in the concept, and go to step 2.3.3.

[0063] 2.3.3. Select the attribute type label in pro_labels in turn. If fuse_pros is empty, add label as the key and the corresponding value to fuse_pros. Otherwise, calculate the text cosine similarity between label and all keys in fuse_pros in turn. If the similarity is higher than the threshold, add label as the corresponding key value.

[0064] 2.3.4 Output the fused attribute class dictionary fuse_pros.

[0065] 3. Data layer knowledge fusion: For the original entity data obtained from Baidu and Hudong Baike {X entity},like Figure 7 As shown in Figure 1, based on the results of concept alignment and attribute type fusion output in step 2, encyclopedia graph knowledge fusion is achieved through processes such as logical layer data unification, entity alignment, and entity fusion.

[0066] 3.1. Logical layer description unification: Input original entity table {X entity}, based on the output of step 2, the correspondence table between fused concepts and original concepts (concept_ori2fused), and the correspondence table between fused attributes and original attributes (pro_ori2fused), replace the concept values ​​and attribute types in the original entity data with the fused values; the input fused entity table (fused_entities) is empty.

[0067] 3.2 Encyclopedia Entity Alignment

[0068] 3.2.1, from the original entity table {X entity}, if fused_entities is empty, add ori_data to fused_entities, otherwise go to step 3.2.2.

[0069] 3.2.2. Calculate the similarity between the entity ori_data and any entity in the fused_entity table, along the concept and attribute dimensions (concept_sim and pros_sims). If any similarity is greater than the threshold, proceed to step 3.3 and fuse the ori_data and fused_entity entities. Otherwise, proceed to step 3.2.1. If the similarity between the entity ori_data and any entity in fused_entities is less than the threshold, add ori_data to fused_entities.

[0070] Concept similarity calculation: Concatenate the concepts of ori_data and fused_entity into a string, calculate the cosine similarity of the concatenated text, and use this similarity as the concept similarity of the two entities.

[0071] Attribute Similarity Calculation: For each attribute type pro in the entity to be fused, ori_data, we calculate the attribute type pro_f in fused_entity that is most similar to pro based on text similarity. We then calculate the text similarity sim_pro between the attribute value pro_value corresponding to pro in ori_data and the attribute value pro_f_value corresponding to pro_f in fused_entity. Similarly, if the attribute value texts pro_value and pro_f_value are in an inclusion relationship, their similarity is 1. The average of the similarities corresponding to each attribute type pro in the entity to be fused, ori_data, is used as the attribute similarity value for the entity to be fused, ori_data.

[0072] 3.3 Encyclopedia Entity Fusion: Merge and remove duplicate concept values ​​of the two entities, ori_data and fused_entity. Based on the process data calculated by attribute similarity, merge and remove duplicate attribute values ​​of the two entities. The final output is the fused encyclopedia entity, fused_entities.

[0073] This embodiment provides an encyclopedia graph knowledge fusion method based on framework matching. The method realizes graph logic layer data through concept consistency judgment, concept fusion, and concept attribute fusion, and outputs the fused graph concepts and attributes; based on the logical layer data, the graph data layer entities and their relationships are fused through entity alignment and entity attribute disambiguation, so as to achieve logical unification of multi-source data and clear semantic information, output the fused knowledge graph, and finally realize automatic fusion of multi-source encyclopedia graph data. At the same time, the fusion process and results have a certain degree of interpretability.

[0074] The above are only preferred embodiments of the invention and are not intended to limit the patent scope of the invention. Any equivalent structure or equivalent process transformation made using the contents of the invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the invention.

Claims

1. A method for fusion of encyclopedia graph knowledge based on frame matching, characterized in that: The method comprises the following steps: S1: Obtain original entity data from several encyclopedia databases and store them in a list of encyclopedia entities to be merged, wherein the expression of the list of encyclopedia entities to be merged is: {X entity }={'labels':[label1,...,label n ],'properties':{'pro1':'pro1_value',...,'pro3':'pro3_value'}}, where 'labels' represents the concept list of the entity, label i Indicates the concept value to which the entity belongs. 'properties' represents the properties of the entity, pro1_value and pro3_value are the property values ​​of the property types pro1 and pro3 respectively; S2: Extract the concept value and attribute type of each encyclopedia entity to be fused, generate a candidate concept-attribute list, and perform concept alignment and attribute fusion in sequence to obtain the fused concept value, attribute type and the corresponding table before fusion, wherein the concept alignment specifically includes: S201: construct the fused concept list fuse_concepts and the correspondence table between fused concepts and original concepts concept_ori2fused, both of which are initially empty; S202: Select a concept from the candidate concept-attribute list concepts. If the fused concept list fuse_concepts is empty, add concept to fuse_concepts. Otherwise, execute step S203. S203: Calculate the concept value similarity and attribute type similarity between concept and any concept fuse_concept in fuse_concepts in turn, wherein the calculation method of concept value similarity includes: assuming that the concept values ​​of concept and fuse_concept are L c =[label 1c ,...,label nc ], L f =[label 1f ,...,label nf ], if L c , L f For inclusion relations or L c , L f There are synonyms in L c , L f The similarity is 1; otherwise, L c , L f Concept value splicing, calculate the cosine similarity of the spliced ​​text. If the similarity is higher than the threshold, it means that the concept values ​​of concept and fuse_concept are similar; S204: Based on the similarity of concept values ​​and attribute types, the concept values ​​and attribute types of concept and fuse_concept are merged and deduplicated, and the corresponding table between the fused concept and the original concept, concept_ori2fused, is updated; S3: According to the correspondence table, the concept value and attribute type of the original entity data in the encyclopedia entity list to be fused are replaced with the fused concept value and attribute type to obtain fused entity data, and entity fusion is performed according to the concept similarity and attribute similarity of the original entity data and the fused entity data to obtain the fused encyclopedia entity. The step S3 specifically includes: S301: Input the encyclopedia entity list to be fused {X entity }, based on the correspondence table between fused concepts and original concepts concept_ori2fused, and the correspondence table between fused attributes and original attributes pro_ori2fused, replace the concept values ​​and attribute types in the original entity data with the fused values, and construct the fused entity table fused_entities, which is initially empty; S302: select the encyclopedia entity list to be fused {X entity }, if fused_entities is empty, add ori_data to fused_entities, otherwise execute step S303; S303: calculate the similarity concept_sim and pros_sims between entity ori_data and any entity fused_entity in the fused entity table in terms of concept and attribute; if there is a similarity greater than the threshold, execute step S304, otherwise, add ori_data to fused_entities; S304: merge and deduplicate the concept values ​​of ori_data and fused_entity, and output the fused encyclopedia entity fused_entities; wherein, the calculation method of the concept value similarity includes: concatenating the concepts of ori_data and fused_entity into a character string, calculating the cosine similarity of the concatenated text, and taking the similarity as the concept similarity of the two entities.

2. The encyclopedia graph knowledge fusion method based on frame matching according to claim 1, characterized in that: The step of extracting the concept value and attribute type of each encyclopedia entity to be fused and generating a candidate concept-attribute list specifically includes: From the encyclopedia entity list to be integrated {X entity } to extract the concept label i All attribute types of the entity are stored in the candidate concept-attribute list {concepts} = [{'label i ':['pro i1 ','pro i2 ',...,'pro in ']}], where pro i For {X entity } i The corresponding attribute type.

3. The encyclopedia graph knowledge fusion method based on frame matching according to claim 1, characterized in that: The attribute fusion specifically includes: S211: Build the fused attribute class dictionary fuse_pros, which is initially empty; S212: Select the concept values ​​in the fused concept list fuse_concepts in turn, and read all attribute types pro_labels contained in the concept; S213: select the attribute type label in pro_labels in turn, if fuse_pros is empty, add label as the key and the corresponding value to fuse_pros; otherwise, calculate the text cosine similarity between label and all keys in fuse_pros in turn; S214: According to the text cosine similarity between label and all keys in fuse_pros, label is added as the corresponding key value to fuse_pros, and the fused attribute class dictionary fuse_pros is output, and the corresponding table pro_ori2fused between fused attributes and original attributes is obtained.

Images