A fine-grained knowledge graph fusion method for two-dimensional overlapping large sample data sources

Abstract

The application discloses a kind of fine-grained knowledge graph fusion methods of two-dimensional overlapping large sample data source, comprising: S1.to the attribute triple corresponding to knowledge graph is iteratively aligned entity pair, obtain the entity pair set to it, the similarity of entity pair is multilevel classification, obtain high confidence entity pair;The high confidence entity pair obtained is as the training data set of embedded model, using relationship triple does the structured embedding of embedded model, obtains the high-dimensional space vector representation of entity and relationship, sets weight to attribute and relationship, obtains the final similarity of attribute and relationship;S2.according to the similarity obtained, entity attribute is filtered, obtains the final similarity of attribute;S3.based on the classifier model trained by machine learning, atomic expression algorithm automatically completes the knowledge feature fusion of high confidence entity pair and attribute;S4.based on knowledge feature fusion, complete bidirectional supervision interactive data fusion.

Description

Technical Field

[0001] This invention relates to the field of big data processing technology, and in particular to a fine-grained knowledge graph fusion method for two-dimensional overlapping large sample data sources. Background Technology

[0002] A knowledge graph (KG) contains a large number of examples, and in practical applications, it is usually represented by a triple.<H,R,T> Let H and T represent the head entity and tail entity, respectively, and R represent the inherent relationship between entities H and R. In the two knowledge graphs KG1 and KG2, the following definition applies: There exists a set of entity pairs Entity = {A1, A2, A3...A...} n}, where A i (i = 1, 2, 3, ..., n) is defined as a quadruple structure<ID,E1,E2,S> In this context, ID is set as a unique identifier, E1∈KG1, E2∈KG2, and S represents the similarity value between two entities, where S is located between [0, 1]. In different knowledge graphs, equivalent entities may have different identifiers due to differences in the builders or language, and they possess semantic similarity.

[0003] Existing technologies often employ the following methods for knowledge fusion: 1) Utilizing textual information about entities and relationships; these methods are mostly based on simple string matching and are easy to operate. 2) Based on data structures, using node similarity for data matching. 3) Using third-party datasets for indirect matching. 4) Using machine learning algorithms for data feature processing, including learning fusion expressions and training classification models. Alternatively, aggregation functions can be used for multi-algorithm fusion to achieve better results. However, these methods often have the following drawbacks:

[0004] 1) It ignores the fact that pre-fused knowledge graphs have data overlap in both the entity and attribute dimensions.

[0005] 2) Existing solutions often focus on entity fusion and matching a large number of entity pairs rather than improving the quality of the knowledge graph as the ultimate goal. This neglects the crucial factor of entity attribute fusion, resulting in a knowledge graph that is large but lacks precision. From the perspective of improving the quality of the knowledge graph, the cross-fusion of ontology and attributes is an indispensable dimension, and their mutual promotion is both a challenge and a key point in achieving high knowledge fusion.

[0006] 3) Ideally, large-scale data processing is best handled by computers. However, due to technological limitations, manual methods such as data block partitioning or data labeling must be used, or lightweight algorithms must be applied manually. This results in a significant waste of human and financial resources. Summary of the Invention

[0007] The present application aims at the problems of low quality of knowledge graph obtained by existing knowledge fusion method, and large resource consumption in fusion process, and the like, and combines linguistic information, fuses spatial information and machine learning algorithm, and aims to solve one or more difficulties existing in existing knowledge graph fusion to a certain extent, and provides a fine-grained knowledge graph fusion method of two-dimensional overlapping large sample data source.

[0008] In order to achieve the above object, the present application adopts the following technical scheme:

[0009] A fine-grained knowledge graph fusion method of two-dimensional overlapping large sample data source comprises:

[0010] S1. Iterative entity alignment is performed on attribute triple corresponding to the knowledge graph to obtain an entity pair set, multi-level classification of similarity of the entity pair is performed to obtain a high-confidence entity pair; the obtained high-confidence entity pair is taken as a training data set of an embedded model, structured embedding of the embedded model is performed by using a relation triple to obtain high-dimensional space vector representation of the entity and the relation, weights are set for the attribute and the relation to obtain final similarity of the attribute and the relation;

[0011] S2. The entity attribute is screened according to the obtained similarity to obtain final similarity of the attribute;

[0012] S3. Knowledge feature fusion of the high-confidence entity pair and the attribute is automatically completed by using a classifier model trained based on machine learning and an atomic expression algorithm;

[0013] S4. Based on the knowledge feature fusion, bidirectional supervised interactive data fusion is completed.

[0014] Further, the step S1 specifically comprises:

[0015] S11. Entity alignment is performed on the attribute triple based on an iterative model, entity matching operation is performed based on the attribute and the attribute corresponding attribute value to obtain an entity pair set, and attribute similarity matching operation is performed by using the entity pair set to obtain an attribute pair set, and a high-confidence entity pair is obtained;

[0016] S12. The obtained high-confidence entity pair is taken as a training data set of an embedded model, structured embedding is performed by using a relation triple, global structure of the knowledge graph to be fused is described and modeled, and finally high-dimensional space vector representation of the entity and the relation is obtained;

[0017] S13. Alignment of the attribute and the relation is fused based on different weights to obtain alignment results of two dimensions of the relation and the attribute, and linear combination is adopted to obtain total similarity of the attribute and the relation.

[0018] Further, the step S2 specifically comprises:

[0019] S21. Calculate the similarity between properties;

[0020] S22. Calculate the similarity between adjacent entities;

[0021] S23. Calculate the similarity of the property label set;

[0022] S24. Screen the upper concept path of entity properties in the knowledge graph, form the path vector, and calculate the final similarity of the properties.

[0023] Further, the step S3 specifically comprises:

[0024] S31. Use machine learning to train the classifier model, and use the binary classification method to process entity fusion;

[0025] S32. Screen the properties using atomic expressions;

[0026] S33. Combine the atomic expressions for use, complete the knowledge feature fusion of high-confidence entity pairs and properties.

[0027] Further, the step S4 specifically comprises:

[0028] S41. Embed the vectors of the triplets based on the TransE algorithm and the PtransE algorithm, complete the training of a single knowledge graph;

[0029] S42. Remap the high-dimensional space vectors of the processed entities and relationships in the low-dimensional space, and form constraints on entity and relationship vectors respectively during the mapping process, complete the bidirectional supervised interactive data fusion.

[0030] Further, the step S11 specifically comprises:

[0031] S111. Set a uniform weight for the common properties during property alignment, and calculate the similarity between entities, represented as:

[0032]

[0033] Wherein, Sim A (e1, e2) represents the similarity between entity e1 and entity e2; represents the kth property of entity e1 that is common to both entities; represents the kth property of entity e2 that is common to both entities; n represents the total number of common properties of the two entities; Sim v represents the similarity between two property values and , represented as:

[0034]

[0035] wherein, levenshteinSim represents similarity based on Levenshtein distance; lcsSim represents similarity by longest common substring of strings;

[0036] S112. Finding potential aligned attribute pairs according to aligned entity pairs, denoted as:

[0037]

[0038] wherein, represents similarity of attribute pair . represents the number of elements in the finite set of entities; represents similarity between attribute values.

[0039] Further, the step S13 obtains the total similarity of attributes and relations, denoted as:

[0040] Sim (Ei, Ej) = λ x simR (ei, ej) + (1 - λ) x simA (ei, ej)

[0041] wherein, simR represents similarity based on relation triplets; simA represents similarity obtained by attribute triplets; λ represents weight; sim (E i , E j ) represents total similarity.

[0042] Further, the step S21 calculates similarity between attributes, denoted as:

[0043] Sim property = COS (Name property1 Name property2 )

[0044] wherein Sim property represents similarity of two attributes property1 and property2 at attribute name level; Name property1 and Name property2 respectively represent high-dimensional space vector representation.

[0045] Further, the step S22 calculates similarity between adjacent entities, denoted as:

[0046] Sim entity = | entityList1∩entityList2 | / | entityList1∪entityList2 |

[0047] wherein Simentity Sim represents the similarity of two adjacent entities; entityList1 and entityList2 represent the limited set of adjacent entities of property1 and property2.

[0048] Further, the similarity of the attribute label set calculated in step S23 is represented as:

[0049] Sim label = COS (label property1 , label property2 )

[0050] Wherein, Sim label represents the limited label similarity of property;

[0051] The final similarity of the attribute calculated in step S24 is represented as:

[0052] Sim con = COS (concept property1 , concept property2 )

[0053] Wherein, Sim con represents the upper concept similarity of property1 and property2, concept property1 , concept property2 respectively represent the upper level concept path label of the attribute.

[0054] Compared with the prior art, the beneficial effects of the present application are:

[0055] 1) The ultimate purpose is to improve the quality of the knowledge graph, and the financial field knowledge graph is fused at the attribute level to meet the requirement of fine-grained fusion, a large number of entity pairs are matched based on entity fusion, and the key factor of attribute fusion in knowledge graph fusion is emphasized. The close relationship between entities and attributes is used to promote the fusion effect, and finally the quality and precision of the fused knowledge graph are improved.

[0056] 2) Reduce the manual intervention in the fusion process, realize a machine learning algorithm that only needs positive sample fusion expression, and achieve good fusion effect and reduce resource consumption. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is a fine-grained knowledge graph fusion method flowchart of a two-dimensional overlapping large sample data source provided by embodiment one. DETAILED DESCRIPTION

[0058] Following make the embodiments of the present application specific, specific examples, those skilled in the art can easily understand the advantages and effects of the present application from the disclosure. The present application can also be implemented or applied by different specific embodiments, the details in the specification can be based on different points of view and applications, various modifications or changes without departing from the spirit of the present application. It should be noted that the following examples and features in the examples can be combined with each other without conflict.

[0059] The purpose of the present application is to overcome the defects of the prior art, provide a fine-grained knowledge graph fusion method for two-dimensional overlapping large sample data sources, reduce the workload while focusing on the use of entity attributes in the knowledge graph, and use the promoting relationship between attributes and entities to ensure the accuracy and quality of the fused knowledge graph.

[0060] Embodiment one

[0061] The present embodiment provides a fine-grained knowledge graph fusion method for two-dimensional overlapping large sample data sources, as shown in Figure 1 The method comprises the following steps:

[0062] S1. Iterative entity alignment of attribute triples corresponding to the knowledge graph is performed to obtain the entity pair set, and the similarity of the entity pair is multi-level classified to obtain high-confidence entity pairs; the obtained high-confidence entity pairs are used as the training data set of the embedded model, the relationship triple is used for structured embedding of the embedded model, the high-dimensional space vector representation of the entity and the relationship is obtained, the weight of the attribute and the relationship is set, and the final similarity of the attribute and the relationship is obtained;

[0063] S2. According to the obtained similarity, the entity attribute is screened to obtain the final similarity of the attribute;

[0064] S3. The classifier model based on machine learning training, atomic expression algorithm automatically complete high-confidence entity pair and attribute knowledge feature fusion;

[0065] S4. Based on the knowledge feature fusion, the bidirectional supervised interactive data fusion is completed.

[0066] Industry knowledge graph is for a specific vertical field, and has more strict pre-data mode and more accurate accuracy requirements for data, emphasizing "depth". Financial data is a typical big data with "4V" characteristics (Volume, Variety, Value, and Velocity). Further, the financial field is the most effective way to monetize data. The financial industry is very broad, and the main categories include banks, investments, insurance, and others. Further granularity can be divided into currency, bonds, funds, trust, and other asset management plans, factor markets, credit, and loans. The application of knowledge graph in the financial field mainly includes risk control, credit investigation, auditing, anti-fraud, data analysis, and automated reporting. Different institutions, organizations, and individuals build knowledge graphs for different needs. The data in these graphs can be divided into three categories: 1) structured data: represented by e-government forms, usually anchored by the ID of a person or organization to aggregate different information such as name, occupation, and income; subsequent evolution of basic libraries, theme libraries, and special topic libraries, and a series of organizational forms. 2) unstructured data: represented by video, image, voice, and text, which usually needs to be analyzed and processed into structured data before being used. 3) spatio-temporal data: represented by geographic information, IoT, and trajectory data.

[0067] This embodiment is aimed at the current situation in the financial field, and conceives how to organize and reasonably automate the fusion of these heterogeneous knowledge graphs to improve the coverage and quality of knowledge, solve the problems of low data quality or data missing in individual knowledge graphs, and achieve better application results. The entity alignment algorithm and attribute alignment algorithm in the knowledge graph fusion field are organically combined. Finally, a fine-grained knowledge graph fusion method based on positive and negative sample fusion expression learning algorithm for financial field heterogeneous two-dimensional overlapping large sample data sources is designed and implemented.

[0068] In the first step S1, the attribute triples corresponding to the knowledge graph are iteratively aligned to obtain the entity pair set, and the similarity of the entity pairs is multi-level classified to obtain high-confidence entity pairs. The high-confidence entity pairs obtained are used as the training data set of the embedded model, the relationship triples are used for structured embedding of the embedded model, the high-dimensional space vector representation of the entity and relationship is obtained, the weights of the attributes and relationships are set, and the final similarity of the attributes and relationships is obtained.

[0069] Based on the probability model, the attribute triple is iteratively aligned. The similarity of the entity is graded in multiple levels to form a hierarchical tree structure. Based on the tree structure, different thresholds are set to obtain high-confidence entity pairs. Based on these entity pairs, we can ensure the relative high quality of the entity pairs, and use these entity pairs for embedded model training to obtain high-dimensional vector representation of low-dimensional data. After training the logistic regression model, the high-dimensional vector is combined to form a unified mapping of the similarity in the high-dimensional and low-dimensional space, and then the final similarity is obtained based on the weight setting. Specifically, it includes:

[0070] S11. Based on the alignment of attribute triples, the entity alignment of attribute triples is performed based on the iterative model. The entity matching operation is performed based on the attributes and the attribute values corresponding to the attributes, to obtain a set of entity pairs, and the attribute similarity matching operation is performed using the set of entity pairs to obtain a set of attribute pairs. The two steps are repeatedly executed until no new entity and attribute pair set can be generated, to obtain high-confidence entity pairs.

[0071] S111. A uniform weight is set for the common attributes during attribute alignment, and the similarity between entities is calculated.

[0072] Due to the low coverage of attributes, the diversity of attribute names and attribute values often leads to different attributes of the same entity. Based on the alignment of attributes, any common attribute is particularly important. The common attributes of two entities are set to a uniform weight, and the similarity between the two entities is calculated according to the following formula, which is represented as:

[0073]

[0074] wherein, Sim A (e1, e2) represents the similarity between entity e1 and entity e2. represents the kth attribute of entity e1 that is common to both entities; represents the kth attribute of entity e2 that is common to both entities; n represents the total number of common attributes of the two entities; Sim v represents the similarity between the two attribute values and , which is represented as:

[0075]

[0076] wherein, levenshteinSim represents the similarity calculated based on the Levenshtein distance; lcsSim represents the similarity calculated by the longest common substring of the string;

[0077] S112. According to the aligned entity pairs, the potential aligned attribute pairs are found.

[0078] From S111, the potential aligned attribute pairs can be found according to the aligned entity pairs, specifically, the aligned entity pair set is obtained, then the subset containing the potential aligned attribute pairs is found in the aligned entity set, and the attribute name similarity is measured according to the subset, and the formula for calculating the attribute pair in the entity pair is as follows, which is represented as:

[0079]

[0080] wherein, represents the similarity of the attribute pair . represents the number of elements in the entity finite set; represents the similarity between attribute values.

[0081] Based on the above mathematical model, the interactive alignment of attributes and entities is carried out according to the following algorithm, and the algorithm is as follows:

[0082]

[0083] S12. Based on the alignment method of the relationship embedding, the high-confidence entity pair obtained is used as the training data set of the embedding model, the relationship triple is used for structured embedding, the global structure of the knowledge graph to be fused is described and modeled, and finally the high-dimensional space vector representation of the entity and the relationship is obtained;

[0084] The structural embedding model optimizes the maximum boundary loss function to make the score of the positive sample higher than that of the negative sample, and the formula is as follows:

[0085] O SE =∑∑(f(tr)-α(tr'))

[0086] Wherein, f(tr) = ||h+r-t|| represents the score function; Tr and Tr' represent the finite set of positive sample triplets and negative sample triplets; alpha is between 0 and 1, which represents a hyperparameter, and is used to weight the positive and negative samples. Based on the embedding process, the high-dimensional vectors of the entities in the two knowledge graphs are obtained, and the similarity is obtained by cosine distance.

[0087] S13. Based on different weights, the attribute alignment and the relationship alignment are fused to obtain the alignment results of the relationship and the attribute in two dimensions, and a linear combination method is adopted to obtain the total similarity of the attribute and the relationship, which is represented as:

[0088] Sim(Ei, Ej) = λ x simR(ei, ej) + (1-λ) x simA(ei, ej)

[0089] Wherein, sim(E i , E j) represents the total similarity; simR represents the similarity based on the relationship triple; simA represents the similarity obtained by using the attribute triple; λ represents the weight, and the weight is learned by the regression model. More specifically, the importance of attributes and relationships is different in different data sets, the quality of relationships and attributes of different knowledge graphs is different, if the quality of the relationship in the knowledge graph is very high, then the alignment based on the relationship obviously has higher confidence, in the sparse knowledge graph, the result of the alignment based on the attribute has higher confidence.

[0090] In step S2, the entity attributes are screened according to the obtained similarity, and the final similarity of the attributes is obtained.

[0091] According to the similarity of the entity, the attributes with universal meanings in the knowledge base are screened, the threshold standard is determined, the attribute function is designed to automatically screen the entity attributes, and the partial information of the entity attributes is used, including the upper and lower concepts of the entity, the label, the attribute value and the like. The similarity obtained by using the information and the attribute name similarity are combined to obtain the final similarity of the attributes, the pruning operation is performed to reduce the redundancy of the entity attributes in the fused knowledge graph. Finally, they are executed interactively, and mutual promotion is realized. The input two graphs are output as the fused graph, and not only the aligned entity pairs are output. Specifically, the following steps are included.

[0092] S21. Calculate the similarity between attributes;

[0093] The attribute name similarity is particularly important for the semantic information of the attribute itself. The embodiment expects that the similarity calculation can deeply understand the specific information contained in the semantic layer of different attributes, rather than simple matching at the character level. In this embodiment, the open source AILab Chinese word vector library is used to calculate the similarity based on the following formula, which is represented as:

[0094] Sim property = COS (Name property1 Name property2 )

[0095] Wherein Sim property represents the similarity of two attributes property1 and property2 at the attribute name level; Name property1 and Name property2 respectively represent the high-dimensional space vector representation, and the finally obtained result is the cosine value of the two attribute name vectors.

[0096] S22. Calculate the similarity between adjacent entities;

[0097] Correlation entity similarity, in addition to the entity itself similarity, the embodiment of the present application also takes into account the adjacent relationship of the entity, which can improve the quality of knowledge fusion. It is assumed that if the similarity of two adjacent entities of an attribute reaches a certain threshold, the attribute pair can be considered similar. For the adjacent entities of the above attributes property1 and property2, the calculation formula is as follows:

[0098] Sim entity = |entityList1∩entityList2| / |entityList1∪entityList2|

[0099] Wherein Sim entity represents the similarity of two adjacent entities; entityList1 and entityList2 represent the finite set of adjacent entities of the attributes property1 and property2.

[0100] S23. Calculate the similarity of the attribute label set;

[0101] Correlation entity label similarity, in multiple search engines such as Wikipedia, Baidu, Sogou, etc., we can find that the label of the entity feature is summarized, such as searching for "Ren Zhengfei", there will be such labels as president, CEO, etc. Such labels often have considerable representativeness.

[0102] Based on such a situation, from the perspective of improving the quality of knowledge graph, the label vector Label property1 =(X1,X2,...X n ) of the attribute property1 and the label vector Label property2 =(y1,y2,...y n ) of the attribute property2 are constructed, so as to calculate the similarity of the finite set of attribute labels, and the formula is as follows:

[0103] Sim label =COS(label property1， label property2 )

[0104] Wherein Sim label represents the similarity of the finite label of the attribute property;

[0105] S24. Screen the upper concept path of the entity attribute in the knowledge graph, form a path vector, and calculate the final similarity of the attribute.

[0106] Similarity of the upper concept of the associated entity, there is a hierarchical concept tree in the knowledge graph, for the most common people, the root node is "people", and then differentiated into "political field", "economic field", "entertainment circle", etc., and the economic field can be divided into "real estate", "automobile industry", etc., and finally forms a concept hierarchical tree, based on this situation, extract the upper concept path of the entity attribute in the two knowledge graphs, form the path vector, and then calculate the similarity, the formula is as follows:

[0107] Sim con =COS(concept property1 , concept property2 )

[0108] Wherein, Sim con indicates the upper concept similarity of the attribute property1 and property2, concept property1 , concept property2 respectively indicate the upper hierarchical concept path label of the attribute.

[0109] In step S3, the knowledge feature fusion of the high confidence entity pair and the attribute is automatically completed based on the classifier model trained by machine learning and the atomic expression algorithm.

[0110] In view of the reality that the knowledge fusion process usually does not record negative samples, the data features are automatically extracted based on the machine learning algorithm, under the condition that the manual intervention can be reduced, the knowledge can achieve good fusion effect, including:

[0111] S31. Utilize machine learning to train a classifier model, and utilize a binary classification method to process entity fusion;

[0112] The classification function formula is as follows:

[0113]

[0114] According to the high-quality entity pair and attribute set obtained in steps S1 and S2, the attributes that meet the standard and can be calculated for similarity are determined, the atomic expression is utilized to make the F-measure of each pair of attributes reach the maximum, and the creation of the expression tree is completed based on the and operation.

[0115] S32. Utilize atomic expression to screen attributes;

[0116] The premise of the atomic expression is to determine a suitable metric function so as to be able to configure a suitable threshold value for screening the function participating in the similarity calculation. Obviously, after the work of steps S1 and S2, the next step of screening the attribute pair is still needed, which is to further improve the accuracy and quality of the fused knowledge graph. The first principle of attribute screening is the universality of the attribute, and the screening formula is as follows:

[0117]

[0118] Wherein, Cover(p) represents the attribute universality, the numerator is the number of attributes p, and the denominator is the number of all subject entities; The limited sets P1 and P2 of attributes are obtained by screening KG1 and KG2 respectively. Then, the Cartesian product operation of P1 and P2 is performed according to the self-defined function M, the function Mp1,p2 and the corresponding threshold index θ that make the F-measure value of the attribute pair (P1, P2) reach the maximum are found, and then the atomic expression set is obtained, as follows:

[0119]

[0120] S33. Combine the atomic expressions to complete the knowledge feature fusion of high-confidence entity pairs and attributes.

[0121] Considering the defect that the atomic expression only uses the local information of the attribute, the combination of the atomic expression is considered, and the formula is expressed as:

[0122]

[0123] Wherein, φ(E) represents the operator symbol; ∪, ∩, \ represent OR, AND and DIFF operators respectively.

[0124] In step S4, based on the knowledge feature fusion, the bidirectional supervised interactive data fusion is completed.

[0125] The bidirectional supervised interactive data fusion algorithm is realized, based on the assumption that the to-be-fused knowledge graphs have considerable fitting degree, interactive supervision training is performed on each other, and the quality and quantity of the knowledge graph are enhanced in the cyclic fusion process, the structural information between entities is highlighted, the similarity weight of the linguistic similarity in the fusion process is weakened, the low-dimensional character similarity is converted into high-dimensional space vector similarity, the cross-domain structural fusion is realized, which specifically includes:

[0126] S41. Based on the TransE algorithm and the PtransE algorithm, the vector of the triple is embedded to complete the training of a single knowledge graph;

[0127] Based on the TransE and PTransE models, the vector embedding of the RDF triple is realized, and the loss function of TransE is defined as follows:

[0128] L(h,r,t) = [y + E[h,r,t] - E(h',r',t')] +

[0129] Wherein, L(h,r,t) represents a loss function; y represents an interval value; E(h,r,t) represents a vector embedding, and (h',r',t') is an error triple.

[0130] The algorithm flow of TransE is as follows:

[0131]

[0132] TransE is different from other training methods which set too many parameters of triplets, thereby causing the model to be complex and difficult to understand, and is suitable for processing a knowledge graph with a large data set but simple data content. Considering that the processing process will inevitably involve a large-scale and complex data set, the PtransE model is considered to be used for processing, and the specific algorithm flow is as follows:

[0133]

[0134] S42. The high-dimensional space vectors of the processed entities and relations are remapped in the low-dimensional space, and constraints on the entity and relation vectors are formed during the mapping process, and the bidirectional supervised interactive data fusion is completed.

[0135] In step S41, the training of a single knowledge graph is mainly completed, and the essence of fusion is to remap the high-dimensional space vectors of the processed entities and relations in the low-dimensional space, therefore, this subsection completes the bidirectional supervised training of two knowledge graphs based on the pre-fusion entity pair information, and constraints on the respective vectors are formed during the process, and the specific algorithm pseudo code is as follows:

[0136]

[0137] S43. Knowledge representation learning and supervised learning are a reciprocating iterative process, for the entity pair e1 and e2 in the two networks, a threshold value θ is determined, if E(e1,e2)<θ, they are considered to be similar entities, and (e1,e2) is called a standard entity pair, and the standard entity pair can find more entity pairs through bidirectional supervision in the iteration process, and the iteration process is as follows:

[0138]

[0139] Compared with the prior art, the embodiment has the beneficial effects that:

[0140] 1) In order to improve the quality of the knowledge graph, the financial field knowledge graph is adopted to achieve the requirement of fine-grained fusion in the attribute level fusion method. Based on the matching of a large number of entity pairs, the key factor of attribute fusion in the knowledge graph fusion is focused on. The close relationship between entities and attributes is used to promote the fusion effect, and finally the quality and accuracy of the fused knowledge graph are improved.

[0141] 2) Reduce the manual intervention in the fusion process, realize a machine learning algorithm that only needs positive sample fusion expression, and achieve good fusion effect and reduce resource consumption.

[0142] The specific embodiments described herein are merely illustrative of the spirit of the present application. Those skilled in the art of the present application can make various modifications or supplements to the described specific embodiments or replace them with similar ways, but will not deviate from the spirit of the present application or exceed the scope defined by the appended claims.

Claims

1. A fine-grained knowledge graph fusion method for two-dimensional overlapping large sample data sources, characterized in that, include: S1. Iteratively align the corresponding attribute triples in the financial knowledge graph to obtain a set of entity pairs. Then, perform multi-level similarity classification on the entity pairs to obtain entity pairs with high confidence. The obtained high-confidence financial entity pairs are used as the training dataset for the embedded model. The structured embedding of the embedded model is performed using relation triples to obtain a high-dimensional vector representation of financial entities and relations. Weights are set for attributes and relations to obtain the final similarity of attributes and relations in finance. S2. Based on the obtained similarity, the attributes of financial entities are filtered to obtain the final similarity of financial attributes; S3. Based on the classifier model trained by machine learning and the atomic expression algorithm, knowledge feature fusion of high-confidence financial entity pairs and attributes is automatically completed; S4. Based on knowledge feature fusion, complete the interactive data fusion for two-way supervision in the financial field; Step S1 specifically includes: S11. Perform entity alignment on attribute triples based on an iterative model, perform entity matching operation based on attributes and their corresponding attribute values ​​to obtain a set of entity pairs, and use the set of entity pairs to perform attribute similarity matching operation to obtain a set of attribute pairs, thus obtaining entity pairs with high confidence. S12. The obtained high-confidence entity pairs are used as the training dataset for the embedding model. Relation triples are used for structured embedding to characterize and model the global structure of the knowledge graph to be fused, and finally a high-dimensional vector representation of entities and relations is obtained. S13. Based on different weights, attribute alignment and relation alignment are fused to obtain the alignment results of the two dimensions of relation and attribute, and the overall similarity of attribute and relation is obtained by linear combination. Step S2 specifically includes: S21. Calculate the similarity between attributes; S22. Calculate the similarity between adjacent entities; S23. Calculate the similarity of attribute label sets; S24. Filter the upper-level concept paths of entity attributes in the knowledge graph, form path vectors, and calculate the final similarity of attributes; Step S3 specifically includes: S31. Use machine learning to train a classifier model and use a binary classification method to handle entity fusion; S32. Filtering attributes using atomic expressions; S33. Combine atomic expressions to complete the knowledge feature fusion of high-confidence entity pairs and attributes; Step S4 specifically includes: S41. Embed the vectors of triples based on the TransE algorithm and the PtransE algorithm to complete the training of a single knowledge graph; The data in the graph is divided into the following three categories: 1) Structured data, including e-government table data, which aggregates different information using the ID of a person or organization as the anchor; 2) Unstructured data, including video, images, voice and text, which need to be analyzed and processed into structured data before they can be used; 3) Spatiotemporal data, including geographic information, IoT, and trajectory data. S42. Remap the processed entity and relation high-dimensional space vectors into a low-dimensional space, and form constraints on entity and relation vectors respectively during the mapping process to complete bidirectional supervised interactive data fusion.

2. The fine-grained knowledge graph fusion method for a two-dimensional overlapping large sample data source according to claim 1, characterized in that, Step S11 specifically includes: S111. Set uniform weights for common attributes during attribute alignment and calculate the similarity between entities, expressed as: ; Among them, Sim A (e1, e2) represents the similarity between entity e1 and entity e2; This represents the k-th attribute that entity e1 shares with both entities; This represents the k-th attribute shared by entity e2 in both entities; n represents the total number of shared attributes between the two entities; Sim v This represents two attribute values. and The similarity between them is expressed as: ； Where levenshteinSim represents the similarity calculated based on the Levenshtein distance; lcsSim represents the similarity calculated based on the longest common substring of the strings. S112. Find potential aligned attribute pairs based on aligned entity pairs, represented as: ； in, Represents attribute pairs , Similarity; Z=EP Indicates the number of elements in a finite set of entities; This indicates the similarity between attribute values.

3. The fine-grained knowledge graph fusion method for a two-dimensional overlapping large sample data source according to claim 2, characterized in that, The total similarity of attributes and relationships obtained in step S13 is expressed as: ； Where simR represents the similarity obtained based on relation triples; simA represents the similarity obtained using attribute triples; λ represents the weight; sim(E i E j () indicates the overall similarity.

4. The fine-grained knowledge graph fusion method for a two-dimensional overlapping large sample data source according to claim 3, characterized in that, The similarity between attributes calculated in step S21 is expressed as follows: ； Sim property This indicates the similarity between two properties, property1 and property2, at the property name level; Name property1 With Name property2 These represent vector representations in high-dimensional space.

5. The fine-grained knowledge graph fusion method for a two-dimensional overlapping large sample data source according to claim 4, characterized in that, In step S22, the similarity between adjacent entities is calculated and expressed as follows: ； Sim entity This indicates the similarity between two adjacent entities; entityList1 and entityList2 represent finite sets of entities adjacent to property1 and property2.

6. The fine-grained knowledge graph fusion method for a two-dimensional overlapping large sample data source according to claim 5, characterized in that, The similarity of the attribute tag sets calculated in step S23 is expressed as follows: ； Among them, Sim label This represents the finite label similarity of the property attribute; The final similarity of the attributes calculated in step S24 is expressed as: ； Among them, Sim con This indicates the similarity between the parent concepts of property1 and property2. property1 concept property2 These represent the path tags of the upper-level concepts of the attributes.

Images