A multi-domain knowledge conflict detection method for knowledge fusion

Abstract

The application discloses a multi-domain knowledge conflict detection method for knowledge fusion, which comprises the following steps: dividing a multi-source knowledge set composed of triples from multiple sources into blocks to obtain a first multi-source knowledge set and a second multi-source knowledge set; wherein the first multi-source knowledge set comprises triples with attribute redundancy or attribute conflict; the second multi-source knowledge set comprises triples with relation redundancy or relation conflict; performing attribute conflict detection on the first multi-source knowledge set and relation conflict detection on the second multi-source knowledge set. The application improves the accuracy of conflict detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to a multi-domain knowledge conflict detection method for knowledge fusion. BACKGROUND

[0002] In the information age, information sources such as social networks, sensors, and crowdsourcing platforms are constantly generating data. For the same thing, the superposition of information from different sources makes the existing knowledge present the characteristics of multi-source. Under the trend of knowledge interconnection, knowledge graph has become the preferred way to organize multi-source data. It organizes entities, entity attributes, and entity relationships into an ordered structured form using a graph structure, providing support for different industry applications such as intelligent question answering and recommendation systems. However, due to the immaturity of automatic extraction technology, the non-uniformity and inaccuracy of data source descriptions, and other reasons, there are problems such as semantic heterogeneity, knowledge redundancy, and knowledge conflict between graphs. A large amount of redundant and conflicting knowledge not only reduces the quality of knowledge graphs and affects users' decision-making, but also increases the burden on storage systems. In view of the above problems, it is urgent to develop an effective conflict detection and resolution scheme to automatically eliminate the uncertainty of knowledge and form a correct, consistent, and reliable knowledge graph.

[0003] The patent application "Text authentication method based on text conflict detection" (Patent application number: 202211324359.5) of the Tenth Institute of China Electronics Technology Group, the method has the following shortcomings: when detecting event element conflicts, the relationship between event elements may be complex, and there is no accurate modeling and reasoning algorithm for event element extraction and representation, which affects the accuracy of conflict detection.

[0004] The patent application "Standard document conflict detection method and system based on standard knowledge graph" (Patent application number: 202110732648.8) of South China University of Technology, the method has the following shortcomings: only the matching of head entities and relationships is considered, and other information such as entity attributes is not considered. This method is based on similarity matching, so there may be matching errors, especially when the text description similarity is low, which may lead to inaccurate matching.

[0005] The patent application "Cross-service target fusion conflict detection method and device" (Patent application number: 202110530055.3) of Wuhan University, the method has the following shortcomings: labeling the target to be fused in the main field target model requires subjective evaluation and classification of the target, so there may be inconsistencies in the evaluation standards between labelers, leading to uncertainty and subjectivity in the labeling results. SUMMARY

[0006] To solve the above technical problems, the application provides a multi-domain knowledge conflict detection method for knowledge fusion.

[0007] The application discloses a multi-domain knowledge conflict detection method for knowledge fusion, which comprises the following steps:

[0008] The multi-source knowledge set composed of triples from multiple sources is divided into blocks to obtain a first multi-source knowledge set and a second multi-source knowledge set; the first multi-source knowledge set comprises triples with attribute redundancy or attribute conflict; and the second multi-source knowledge set comprises triples with relation redundancy or relation conflict.

[0009] Attribute conflict detection is performed on the first multi-source knowledge set, and relation conflict detection is performed on the second multi-source knowledge set.

[0010] Further, from the attributes and relations at the instance level, the redundancy and conflict between different triples in the multi-source knowledge set are considered, and four types of knowledge conflict, i.e., attribute redundancy, attribute conflict, relation redundancy and relation conflict, are defined in combination with attribute and relation mapping.

[0011] The common point of attribute redundancy and attribute conflict is that both have the same attribute mode, i.e., the entity and attribute in the knowledge triple are the same, and only the attribute value is different; and the common point of relation redundancy and relation conflict is that both have the same relation mode, i.e., the head entity and tail entity in the knowledge triple are the same, and only the relation value is different.

[0012] Further, the multi-source knowledge set composed of triples from multiple sources is divided into blocks to obtain a first multi-source knowledge set and a second multi-source knowledge set, which comprises the following steps:

[0013] When the attribute mode defined on the multi-source knowledge set is a triple (E k ,X,t kX ), the triples with attribute redundancy and attribute conflict are obtained from the multi-source knowledge set according to the attribute mode to constitute the first multi-source knowledge set; wherein k is a data source, E k is an entity described by the kth source; X is a feature attribute set composed of attributes in {A1,...,A M}; t kX is a value combination on X; for each attribute A m ∈X, t km is a numerical constant or character constant of A m in Z.

[0014] When the relation mode defined on the multi-source knowledge set is a triple (H k ,R,T k), according to the relational model, triples with relational redundancy and relational conflict are obtained from the multi-source knowledge set to form the second multi-source knowledge set; wherein k is the data source, H k and T k are the head entity and tail entity provided by the k-th source respectively, and R is the character constant of the relationship between the entity pairs.

[0015] Furthermore, performing attribute conflict detection on the first multi-source knowledge set includes:

[0016] If the attribute value of the triple in the first multi-source knowledge set is a character attribute value, then the similarity Sim between the attributes in the attribute value is calculated. If the similarity Sim is not less than a preset similarity threshold σ, then a random attribute value is retained; otherwise, the triple in the first multi-source knowledge set is added to the conflict resolution candidate set Φ;

[0017] If the attribute value of the triple in the first multi-source knowledge set is a numerical attribute value, that is, it is redundant, then a random attribute value is retained; otherwise, the triple in the first multi-source knowledge set is added to the conflict resolution candidate set Φ.

[0018] Furthermore, string similarity measurement methods include Hamming distance, substring similarity and edit distance; the similarity between any two character attribute values ​​is in the interval [0,1].

[0019] Furthermore, performing relationship conflict detection on the second multi-source knowledge set includes:

[0020] Constructing a relationship hierarchy system for the second multi-source knowledge set to detect whether there is a hierarchical relationship between relationship values;

[0021] Calculate the similarity of the relationships between different triples in the second multi-source knowledge set after constructing the relationship hierarchy system.

[0022] Furthermore, building a relational hierarchy system mainly includes two parts: relation classification and relation tree generation; relation classification is achieved through relation classifier;

[0023] The training process of the relation classifier is:

[0024] First, a certain number of word pairs with hyponymy and hyponymy are selected as positive samples, and an equal number of word pairs without hyponymy and hyponymy are selected as negative samples. Then, these positive and negative sample words are input into a search engine to obtain a large amount of relevant relationship pair corpus. Based on this relationship pair corpus, a word vector representation model of the relationship words is trained. Finally, a classifier is trained using the samples with hyponymy and hyponymy tags to obtain a relationship classification model, denoted as Classifier.

[0025] The generation process of the relation tree is as follows:

[0026] The relation tree is constructed by node insertion. First, the frequency of each word in the relation word set R is calculated, and then the frequency is used as the weight of the relation node. R is sorted from high to low according to the frequency. Finally, the root node is created, and the relation words in R are inserted into the tree in combination with the classifier Classifier and the frequency.

[0027] Further, the similarity between the relations in different triples in the second multi-source knowledge set after the relation hierarchy is constructed is calculated, including:

[0028] The similarity between the relations in different triples is calculated. If the similarity is 1, one triple is randomly retained. If the similarity is not less than a preset similarity threshold τ and the relations in different triples have a hierarchical relationship, the lower word is retained. Otherwise, one triple is randomly retained.

[0029] If the similarity is not 1 and is greater than or equal to the preset similarity threshold τ, the triples in the second multi-source knowledge set are added to the conflict resolution candidate set Φ.

[0030] Further, the similarity between the relations in different triples is calculated, including:

[0031] The vector representation of the relation word is learned based on the word vector representation model, and the cosine similarity between the relations in different triples is calculated.

[0032] Further, the vector representation of the relation word is learned based on the word vector representation model, and the cosine similarity between the relations in different triples is calculated, specifically:

[0033] Let the word vector representation of the relation R i be The word vector representation of the relation R j is The cosine similarity calculation method based on the word vector is:

[0034]

[0035] When the similarity Sim ED based on the edit distance and the cosine similarity Sim cos between the relation R i and R j are both greater than the similarity threshold, it is determined that the relation R i is similar to R j .

[0036] By adopting the technical scheme, the application has the following advantages: the application optimizes the division mode of the conflict type, so that the knowledge with mapping association is covered as much as possible, the problem that the mapping association of the knowledge element is not fully considered in the conflict classification is solved, different detection schemes are designed for the conflict type to improve the detection accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art based on these drawings.

[0038] Figure 1 The figure is a multi-source knowledge conflict classification and detection framework of the embodiment of the application.

[0039] Figure 2 The figure is a conflict detection result comparison diagram of the embodiment of the application.

[0040] Figure 3 The figure is a conflict detection P, R value comparison diagram based on different similarity thresholds of the embodiment of the application. DETAILED DESCRIPTION

[0041] The application will be further described in conjunction with the drawings and embodiments. Obviously, the described embodiments are only some of the embodiments of the application, but not all the embodiments. All other embodiments obtained by those skilled in the art should belong to the protection scope of the embodiments of the application.

[0042] The application can define knowledge conflict classification of triples from multiple sources, specifically:

[0043] In the knowledge graph fusion process, due to different information sources, different extractors and different ontology definitions, a large amount of redundant or contradictory knowledge will appear when merging multiple different graphs. These conflicting knowledge not only causes burden to the storage system, but also affects the retrieval result of the user and thus interferes with the application decision. The application defines four types of knowledge conflict types from the mapping association of the instance attribute and the relationship, including attribute redundancy, relationship redundancy, attribute conflict and relationship conflict, and the specific performance of each type of knowledge conflict is shown in Table 1.

[0044] Table 1: Classification of knowledge conflicts

[0045]

[0046] (1) Attribute redundancy

[0047] The table shows information about books collected from three online bookstores {X1, X2, X3}. Each bookstore provides a set of statements, each describing a book using the three attributes author, publication, and date. Author represents the author list, publication represents the publisher, and date represents the release date. All correct values ​​are bolded and italicized, and missing values ​​are indicated by a "-".

[0048] In Table 2, different bookstores describe different attribute information for the same book. This indicates that attribute duplication and contradiction are common in multi-source knowledge collections. Considering the diverse nature of attribute value data types, this paper provides a definition of attribute-related conflict classification.

[0049] Suppose there are K data sources, each of which provides multiple statements describing a set of entities. The entity attributes observed by different data sources may be different. Suppose each entity has M attributes {A1,...,A M}, the i-th statement is represented by c i ={c i1 ,...,c iM}, C k Represents the set of statements provided by the k-th data source, and a multi-source data set with attribute-level conflicts is called Z = ∑ k∈K C k It consists of multiple records provided by K data sources.

[0050] Table 2 Real examples of conflicting book information

[0051]

[0052] Definition 1 (Attribute Pattern) The attribute pattern defined on Z is a triple (E k ,X,t kX ), where k is the data source, E k is the entity described by the kth source; X is a set of characteristic attributes, consisting of {A1,...,A M} in the attribute composition; t kX is a combination of values ​​on X. For each attribute A m ∈X,t km It's A m A numeric or character constant in Z.

[0053] As the above definition indicates, each attribute pattern corresponds to a different instance; that is, the entity sets of different attribute patterns are disjoint. For example, in the table, (ISBN, Author) is an attribute pattern. For each attribute pattern, when the number of declaration sets n in the pattern exceeds 1, attribute conflicts should be discussed in different cases.

[0054] Given character attribute values x and y, and a similarity computation function Sim and a threshold σ. Attribute values x and y are similar if and only if Sim(x, y) ≥ σ. The invention regards similar attributes as a kind of semantic equivalence. The definition of equivalent attributes is as follows.

[0055] Definition 2 (Equivalent Attributes) Given an attribute schema, triples (E i ,X m ,t im ) and (E j ,X m ,t jm ) are two concrete statements provided by sources X i and X j in the attribute schema set. Two are equivalent attribute values if and only if t im is equal or similar to t jm , denoted as t im :t jm .

[0056] Equivalent attribute values have various forms according to the data type of attribute values. The equivalent attribute value forms of different data types are shown in Table 3.

[0057] Table 3 Attribute Value Equivalent Forms

[0058] Data type Explanation Equivalence form Numeric type t im ,t jm ∈N]]> t iim = t jm ]]> Interval type t im = [a, b], t jm ∈ [c, d]] a = c and b = d Boolean type t im ,t jm ∈{0,1}]]> t iim = t jm <!-- 5 -->]]> Character type t im ,t jm ∈S, S represents a single or multiple characters Sim(t im ,t jm )≥σ]]>

[0059] Definition 3 (Intersected Attributes) Given triples (E i ,X m ,t im ) and (E j ,X m ,t jm ) are two concrete statements provided by sources X i and X j in the given attribute schema set. Two are intersected attribute values if and only if t im ∩t jm = {t | t ∈ t im and t ∈ t jm}, denoted as t im ∩t jm . Particularly, when t im ∩t jm = t im or t im ∩t jm = t jm , attribute t jm (t im ) contains t im (t jm ).

[0060] Definition 4 (Attribute Redundancy) Given an attribute schema, triples (E i ,X m ,t im ) and (E j ,X m ,t jm ) are two specific statements provided by sources X i and X j . We say that attribute redundancy occurs when t im and t jm are equivalent attributes or intersecting attributes.

[0061] According to the definition, when attribute redundancy occurs, not only can the knowledge base not be enriched, but also it will cause storage burden and affect the retrieval results. Therefore, knowledge fusion needs to detect and eliminate redundant attribute values.

[0062] (2) Attribute Conflict

[0063] Contradiction is another polar expression opposite to redundancy. Contradictory attributes cannot exist in a knowledge graph at the same time. For example, a person's age is a uniquely determined value, and a person's hometown is a unique string. Attributes such as age, hometown, and birth date, which have a unique true value, cannot have multiple different values.

[0064] Definition 5 (Opposite Attributes) Given triples (E i ,X m ,t im ) and (E j ,X m ,t jm ) are two specific statements provided by sources X i and X j under a given attribute schema. We say that t m and t im are opposite attribute values when X jm is a single-valued attribute and t im ≠ t jm .

[0065] According to the data type of attribute values, there are various forms of opposite attribute values. Table 4 gives the specific forms of opposite attribute values under different data types.

[0066] Table 4 Opposite Attribute Value Forms

[0067] Data type Explanation Opposite form Numeric type t im ,t jm ∈N]]> [CAT im ≠t jm ]]> Interval type t im = [a, b], t jm ∈ [c, d]] a > d or b < c Boolean type t im ,t jm ∈{0,1}]]> t im ≠t jm ]]> Character type t im ,t jm ∈S, S represents a single or multiple characters]]> Sim(t im ,t jm )<σ]]>

[0068] Definition 6 (Attribute Conflict) Given an attribute schema, triples (E i ,X m ,t im ) and (E j ,X m ,tjm ) is from source X i and X j Two specific statements are provided. When t im and t jm are opposite attributes, it is said that the two attributes conflict.

[0069] By definition, conflicting attributes cannot exist simultaneously and will reduce the quality of the knowledge graph. Therefore, knowledge fusion needs to detect and eliminate conflicting attribute values.

[0070] (3) Relationship redundancy

[0071] Part of the entity relationship information collected from three sources {X1, X2, X3} is shown in Table 5. Each source provides a set of statements, each describing the relationship between a pair of entities. The missing value is represented by "-".

[0072] Table 5 Real example of entity relationship conflict

[0073]

[0074] In Table 5, different sources describe the relationship information of the same entity pair with the same head entity and tail entity. This indicates that there is a phenomenon of relationship repetition and contradiction in multi-source knowledge collection. Considering the rich semantics of relationship words, the invention gives the relevant definitions of relationship conflict classification.

[0075] Let there be K data sources, each providing a set of statements describing the relationship between entities. The entity relationship observed by different data sources may be different. Assume that the correct relationship value between each pair of entities is R, and the relationship value declared by the kth source is R k . Y k represents the set of statements provided by the kth data source. A multi-source data with relationship level conflict is called Q = ∪ k∈K Y k .

[0076] Definition 7 (relationship pattern) The relationship pattern defined on Q is a triple (H k , R, T k ), where k is the data source, H k and T k are the head entity and tail entity provided by the kth source, and R is the relationship character constant between the entity pair.

[0077] As can be seen from the above definition, each relational schema has different pairs of instances. For each relational schema, when the number of sets n declared in the schema is greater than 1, relational conflicts need to be discussed in different cases. Given character-type relation values ​​p and q, a similarity calculation function Sim, and a threshold τ, relation values ​​p and q are said to be similar if and only if Sim(p,q) ≥ τ. The definition of an equivalence relation is as follows.

[0078] Definition 8 (Equivalence Relation) Under a given relational schema, a triple (H i ,R i ,T ii ) and (H j ,R j ,T j ) is source X i and X j Two specific statements are provided. If and only if R i and R j When they are equal or similar, they are called equivalent relation values, denoted by R i :R j .

[0079] In addition, the relationships provided by multiple sources for the same pair of entities may also have part-whole relationships or hyponymy relationships. i With the relative word R j Semantically satisfying “R i It is R j Category, R j It is R i When the instance of R i With R j There is a hypernym relationship. A word with a more general meaning and a broader scope is called a hypernym, while a word with a more specific meaning and a more specific scope is called a hyponym. The definition of the hypernym relationship is as follows.

[0080] Definition 9 (Hypernym Relationship): Given a triple (H i ,R i ,T i ) and (H j ,R j ,T j ) is the source X in the given relational schema set i and X j Two specific statements are provided. When R i (R j ) is R j (R i )·, the two are said to have a hyponymous relationship, which is recorded as or

[0081] As can be seen from Definition 9, the hierarchical relationship is usually hierarchical.

[0082] Definition 10 (Relation Redundancy) Given a relation schema, triples (H i ,R i ,T ii ) and (H j ,R j ,T j ) are two specific statements provided by sources X i and X j . When R i and R j are equivalent relations or subsumption relations, it is said that the two have relation redundancy.

[0083] Relation redundancy is not conducive to the improvement of the quality of the knowledge graph, and should be detected and eliminated in the knowledge fusion process.

[0084] (4) Relation Conflict

[0085] Similar to attribute conflict, relation conflict refers to the phenomenon that the associations between entities provided by multiple sources cannot exist simultaneously. The present invention gives the relevant definition of relation conflict.

[0086] Definition 11 (Opposite Relation) Given a relation schema, triples (H i ,R i ,T ii ) and (H j ,R j ,T j ) are two specific statements provided by sources X i and X j . When the semantics of R i and R j are opposite or do not conform to logic, it is said that the two have opposite relations, denoted as R i = ~R j .

[0087] Definition 12 (Relation Conflict) Given a relation schema, triples (H i ,R i ,T ii ) and (H j ,R j ,T j ) are two specific statements provided by sources X i and X j . When R i and R j are opposite relations, it is said that the two have relation conflict.

[0088] As can be seen from the definition, conflict relations will reduce the quality of the knowledge graph, and the present invention needs to detect such conflicts and eliminate them.

[0089] The present application considers the redundancy and conflict between multi-source knowledge sets from the attributes and relations of the instance layer, and defines four knowledge conflict types of attribute redundancy, attribute conflict, relation redundancy and relation conflict in combination with the attribute / relation mapping of equivalence, opposition and superordinate and subordinate meaning. The common performance of attribute redundancy and relation redundancy is that the attributes (relations) appear repetition or similar phenomenon, the attribute conflict is divided into two cases of numerical value type conflict and character type conflict according to the data type of the value, and the relation conflict mainly refers to the opposition relation conflict. Through starting from different dimensions, the present application gives detailed knowledge conflict classification and definition to solve the problem that different forms of conflict knowledge are neglected in the fusion process, and prepares for knowledge conflict classification detection.

[0090] Referring to Figure 1 The present application provides a multi-domain knowledge conflict detection method for knowledge fusion, which comprises the following steps:

[0091] S101: The multi-source knowledge sets composed of triples from multiple sources are divided into blocks to obtain a first multi-source knowledge set and a second multi-source knowledge set; wherein the first multi-source knowledge set comprises triples with attribute redundancy or attribute conflict; and the second multi-source knowledge set comprises triples with relation redundancy or relation conflict.

[0092] In this embodiment, the multi-source knowledge sets composed of triples from multiple sources are divided into blocks to obtain a first multi-source knowledge set and a second multi-source knowledge set; wherein the first multi-source knowledge set comprises triples with attribute redundancy or attribute conflict; and the second multi-source knowledge set comprises triples with relation redundancy or relation conflict.

[0093] In this embodiment, the redundancy and conflict between different triples in the multi-source knowledge sets are considered from the attributes and relations of the instance layer, and the attribute redundancy, attribute conflict, relation redundancy and relation conflict are defined in combination with the attribute and relation mapping.

[0094] In this embodiment, the common point of attribute redundancy and attribute conflict is that both have the same attribute mode, that is, the entity and attribute in the knowledge triple are the same, and only the attribute value is different; and the common point of relation redundancy and relation conflict is that both have the same relation mode, that is, the head entity and tail entity in the knowledge triple are the same, and only the relation value is different.

[0095] In this embodiment, the multi-source knowledge sets composed of triples from multiple sources are divided into blocks to obtain a first multi-source knowledge set and a second multi-source knowledge set, which comprises:

[0096] When the attribute mode defined on the multi-source knowledge set is a triple (E k ,X,t kX), according to the attribute pattern, triples with attribute redundancy and attribute conflict are obtained from the multi-source knowledge set to form the first multi-source knowledge set; wherein k is the data source, E k is the entity described by the kth source; X is a set of characteristic attributes, consisting of {A1,...,A M} in the attribute composition; t kX is a combination of values ​​on X; for each attribute A m ∈X,t km It's A m a numeric or character constant in Z;

[0097] When the relational model defined on the multi-source knowledge set is a triple (H k ,R,T k ), according to the relational model, triples with relational redundancy and relational conflict are obtained from the multi-source knowledge set to form the second multi-source knowledge set; wherein k is the data source, H k and T k are the head entity and tail entity provided by the k-th source respectively, and R is the character constant of the relationship between the entity pairs.

[0098] S102: performing attribute conflict detection on the first multi-source knowledge set.

[0099] The common point between attribute redundancy and attribute conflict is that they have the same attribute pattern (E k ,X,t kX ), that is, the entities and attributes in the knowledge triples are the same, but only the attribute values ​​are different. To detect redundant and conflicting attribute values, the present invention designs a detection process as shown in Algorithm 1.

[0100]

[0101]

[0102] In Algorithm 1, the first multi-source knowledge set is input, and different conflict detection methods are performed based on the data type of the attribute values. When attribute redundancy occurs, the present invention adopts a strategy of randomly retaining a triplet; for conflicting attribute values, the present invention directly adds them to the conflict resolution candidate set Φ.

[0103] As shown in Algorithm 1, attribute conflict detection relies on the calculation of attribute similarity. The similarity between any two character attribute values ​​is a value in the interval [0, 1]. Common string similarity metrics include Hamming distance, substring similarity, and edit distance.

[0104] Given any two character attribute values ​​s and t, their Hamming distance similarity is calculated as shown in formula (1).

[0105]

[0106] Given any two character attribute values s and t, let x be their maximum common substring, then the substring similarity is calculated as shown in equation (2).

[0107]

[0108] The minimum edit distance is the minimum number of operations required to transform one string into another, including the three operations of addition, deletion and replacement. For example, transforming "abc" into "acd" requires two replacement operations, b→c and c→d, and the minimum edit distance is 2. Let ED st denote the minimum edit distance of s and t, then the similarity is calculated as shown in equation (3).

[0109]

[0110] S103: Relationship conflict detection is performed on the second multi-source knowledge set.

[0111] The common point of relationship redundancy and relationship conflict is that they have the same relationship pattern (H k ,R,T k ), that is, the head entity and the tail entity in the knowledge triple are the same, and only the relationship value (relation) is different. Therefore, before performing specific attribute conflict detection, it is necessary to find triples that share the same relationship pattern. The relationship conflict detection process designed by the present application is shown in Algorithm 2.

[0112]

[0113] In Algorithm 2, the second multi-source knowledge set is input. First, data is blocked according to the relationship pattern (H k ,R,T k ), that is, triples with the same head entity and tail entity are divided into the same data block, and then a relationship hierarchy is constructed to detect whether there is a hierarchical relationship between the relationship values. If there is only one triple in the data block, no conflict detection is needed; otherwise, the similarity

[0114] between the relationships is calculated. For relationships with a hierarchical relationship, the present application chooses to retain the more specific lower word with semantics; for opposite relationships, the present application directly adds the corresponding triples to the conflict resolution candidate set Φ.

[0115] As can be seen from Algorithm 2, the key steps of relationship conflict detection mainly include relationship similarity calculation and relationship hierarchy construction.

[0116] Most of the relationships between entities are in the form of words or phrases. When calculating the similarity of relationship words, the present invention comprehensively considers the literal features and semantic features of the character string and proposes a hybrid similarity calculation method based on edit distance and word vector.

[0117] Calculating the similarity of related words based on edit distance has the advantages of simplicity and efficiency, but this method only considers the literal features of the character string. In order to improve the accuracy of the calculation, the present invention learns the vector representation of the related words based on Word2Vec and calculates the cosine similarity of the relationship.

[0118] Note the relation R i The word vector is represented as Relation R j The word vector is represented as The cosine similarity calculation method based on word vector is:

[0119]

[0120] When the relationship R i With R j Similarity Sim based on edit distance ED and cosine similarity Sim cos When both are greater than the similarity threshold, the relationship R is determined. i With R j resemblance.

[0121] Constructing a relationship hierarchy system assists in determining whether there is a semantically subordinate relationship between two relational terms. Constructing a relationship hierarchy system mainly includes two parts: relationship classification and relationship tree generation.

[0122] To train a relational classifier, the present invention first selects a certain number of word pairs with hyponymy as positive samples, and selects the same number of word pairs without hyponymy as negative samples; then these positive and negative sample words are input into a search engine to obtain a large amount of relevant relational pair corpus, and the Word2Vec representation of the relational words is obtained based on the training of the corpus; finally, the classifier is trained using samples with hyponymy and hyponymy tags to obtain a relational classification model, which is denoted as Classifier.

[0123] The present invention constructs a relationship tree by inserting nodes. First, the frequency of occurrence of each word in the relationship word set R is calculated, and then used as the weight of the relationship node. R is sorted from high to low according to frequency. Finally, a root node is created, and the relationship words in R are inserted into the tree by combining the classifier Classifier and the frequency.

[0124] It should be noted that S102 and S103 can be performed simultaneously or in sequence.

[0125] While the conflict forms in the multi-domain knowledge graph fusion process are various, judging whether the knowledge from multiple sources produces conflict is essentially a binary classification process. The invention uses the precision (P), recall (R) and harmonic mean F1 of the classification task to measure the rationality and effectiveness of the conflict types and detection scheme defined by the invention.

[0126] The invention has carried out experiments on artificially annotated data sets, namely book-author data set, question-answer data set and common sense data set. To verify the effectiveness of the knowledge conflict detection model, the invention uses two methods based on similarity and retrieval as baseline models, and compares them with the CCD detection model proposed by the invention. Among them, the conflict detection method based on similarity judges whether the knowledge conflicts by calculating the similarity of triples; the conflict detection method based on retrieval uses a search engine to obtain the associated information of the current knowledge, and retrieves the information that may exist conflict in the knowledge base according to the information. Table 6 shows the conflict detection results of different models on three real data sets.

[0127] Table 6 Comparison results of knowledge conflict detection

[0128]

[0129] Table 6 and Figure 2 The data in Table 6 show that the F1 value of the CCD model proposed by the invention on the three data sets is higher than that of the other two baseline methods. The F1 value of the conflict detection method based on similarity on the three data sets is low, because the vector representation learning of triples is greatly affected by the model training parameters, and the similarity calculation under the condition of incomplete accurate representation learning will reduce the detection accuracy. The retrieval method based on search engine depends on the retrieval results of the search engine, and when the relevance between the retrieval list and the content of the knowledge base is low, it will have a negative impact on the conflict detection. Compared with the above two single detection methods, the CCD model proposed by the invention performs differentiated detection on different conflict types based on classification, and achieves good conflict detection results in the multi-domain knowledge graph fusion scene, which shows that the classification design of the detection algorithm is effective in the scene with complex and various conflict forms.

[0130] To further verify the feasibility of the classification model proposed by the invention, the invention calculates the P, R and F1 value results of the CCD model in the four conflict types of attribute redundancy, attribute conflict, relationship redundancy and relationship conflict. Since the book-author data set only contains attribute triples, the invention artificially synthesizes relationship triples to test the performance of the model.

[0131] Table 7 Knowledge conflict classification detection results

[0132]

[0133] As shown in Table 7, the detection accuracy of the scheme proposed by the application is different for different conflict types. Among them, the F1 value of the detection of attribute redundancy and attribute conflict on the book-author dataset is the highest, which is 88.7% and 88.4% respectively; the F1 value of the detection of relation redundancy and relation conflict on the common sense dataset is the highest, which reaches 83.6%. In addition, the accuracy and recall rate of the CCD model on the book-author dataset are higher than those on the other two datasets, and the accuracy and recall rate on the question-answer dataset are relatively low. On the one hand, it may be because the conflicts in the book-author dataset are mostly attribute conflicts, and the diversity of the user's professional knowledge in the question-answer dataset leads to the diversification of the conflict types, and the relation conflict detection method defined by the application cannot cover and accurately detect the conflicts in the question-answer dataset, thereby resulting in a lower accuracy. On the other hand, it may be because the similarity threshold value set by the application is too high or too low, which has a negative impact on the detection results of attribute redundancy and relation redundancy.

[0134] To verify the influence of the size of the similarity threshold value on the conflict detection results, the application sets a group of control experiments. The value range of the similarity threshold value is set as [0.5, 1], and the value interval is 0.1. Table 8 shows the calculation results of P, R and F 11 values of the application method on the three datasets when the similarity threshold value takes different values.

[0135] Table 8 Conflict classification detection results under different similarity threshold values

[0136]

[0137] As shown in Table 8 and Figure 3 , when the threshold value is set to be less than 0.7, the accuracy and recall rate both show a significant downward trend; when the threshold value is set to be about 0.8, the accuracy and recall rate both obtain higher values; and when the threshold value is set to be more than 0.9, the accuracy and recall rate gradually decrease. The reason is that when the similarity threshold value is set to be too low, the mutually contradictory knowledge triples may be considered as highly similar attributes or relations and be directly fused, and will not be added to the conflict resolution candidate set, that is, FN increases, resulting in a decrease in the recall rate. When the similarity threshold value is set to be too high, the similar triples that should be directly fused are divided into contradictory knowledge, and are added to the conflict resolution candidate set, that is, FP increases, resulting in a decrease in the accuracy. From the comparison experiment results, it can be known that the size of the similarity threshold value has a great influence on the conflict detection results, and it is reasonable to set the threshold value to be 0.8.

[0138] The application can replace the relationship classification and relationship tree generation method for constructing the relationship hierarchical system in the technical scheme with the analytic hierarchy process, which is a method for quantitatively analyzing the relative importance between multiple factors and decomposing complex decision-making problems into multiple levels.

[0139] The present application can replace Word2Vec used in knowledge conflict classification detection in technical solutions with other word vector representation models, such as GloVe or BERT models.

[0140] Multi-domain knowledge conflict detection for knowledge fusion can be applied in various fields. The following are some practical application scenarios:

[0141] Cross-domain data integration: In cross-domain data integration, data from different domains often needs to be fused to achieve global data analysis and decision support. For example, in the medical field, patient medical data comes from multiple hospitals and doctors, and these data need to be fused and integrated to help doctors make more accurate diagnoses and treatment plans. In this process, there may be conflicts in the medical records of the same patient from different hospitals, which need to be detected and resolved.

[0142] Information extraction and integration: In the process of information extraction and integration, information from different sources needs to be fused and integrated for subsequent analysis and application. For example, in the e-commerce field, product information comes from different sellers and websites, and these information needs to be integrated and standardized to facilitate consumers to compare and purchase. In this process, there may be conflicts in the same product information between different sellers, which need to be detected and resolved.

[0143] Cross-language information retrieval: In cross-language information retrieval, users need to use keywords in different languages to retrieve information, and the system needs to fuse and integrate information in different languages to facilitate users to find and use. For example, in machine translation, users need to translate Chinese text into English text, and the translation system needs to fuse and integrate knowledge in Chinese and English languages to generate accurate translation results. In this process, there may be conflicts between different languages, which need to be detected and resolved.

[0144] Enterprise knowledge management: Enterprises often use multiple systems and applications to manage business processes and data. These systems and applications may come from different vendors or be developed internally, and there may be a lot of data conflicts between them. For example, within an enterprise, multiple departments or teams maintain their own customer information, and a customer may have records in multiple departments, but the detailed information in each record may be different, such as address, contact information, etc., which may cause data conflicts. In this case, using multi-domain knowledge conflict detection methods for knowledge fusion can help enterprises quickly detect data conflicts and solve problems, improve data quality and business efficiency.

[0145] Information retrieval and intelligent question answering: In information retrieval and intelligent question answering, the system needs to obtain information from different knowledge sources and then integrate these information to answer user's questions. Through multi-domain knowledge conflict detection oriented to knowledge fusion, the system can detect the differences and conflicts in different knowledge sources and handle them accordingly to improve the accuracy of system answers and user satisfaction.

[0146] Social media analysis: In social media analysis, a large amount of user data needs to be analyzed and mined to discover potential information and trends. Through multi-domain knowledge conflict detection oriented to knowledge fusion, user data from different social media platforms can be integrated to discover conflicts and repetitions and handle them accordingly to improve the accuracy and credibility of the analysis results.

[0147] In summary, multi-domain conflict detection oriented to knowledge fusion can play an important role in scenarios involving multiple domain knowledge, helping to improve the accuracy and efficiency of decision-making.

[0148] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application and not to limit it, although the present application has been described in detail with reference to the above examples, those skilled in the art should understand that: the specific embodiments of the present application can still be modified or replaced by the equivalent, without departing from the spirit and scope of the present application. Any modification or equivalent replacement, which should be covered within the protection scope of the claims of the present application.

Claims

1. A multi-domain knowledge conflict detection method for knowledge fusion, characterized by: include: A multi-source knowledge set composed of triples from multiple sources is divided into blocks to obtain a first multi-source knowledge set and a second multi-source knowledge set; wherein the first multi-source knowledge set includes triples with attribute redundancy or attribute conflict; and the second multi-source knowledge set includes triples with relationship redundancy or relationship conflict; Performing attribute conflict detection on the first multi-source knowledge set and performing relationship conflict detection on the second multi-source knowledge set; Starting from the attributes and relationships at the instance level, the redundancy and conflict between different triples in the multi-source knowledge set are considered, and four types of knowledge conflict, namely attribute redundancy, attribute conflict, relationship redundancy and relationship conflict, are defined by combining attribute and relationship mapping. The common point between attribute redundancy and attribute conflict is that they have the same attribute pattern, that is, the entities and attributes in the knowledge triples are the same, only the attribute values ​​are different; the common point between relationship redundancy and relationship conflict is that they have the same relationship pattern, that is, the head entity and tail entity in the knowledge triples are the same, only the relationship values ​​are different; The multi-source knowledge set composed of triples from multiple sources is divided into blocks to obtain a first multi-source knowledge set and a second multi-source knowledge set, including: When the attribute pattern defined on the multi-source knowledge set is a triple (E k ,X,t kX ), according to the attribute pattern, triples with attribute redundancy and attribute conflict are obtained from the multi-source knowledge set to form the first multi-source knowledge set; wherein k is the data source, E k is the entity described by the kth source; X is a set of characteristic attributes, consisting of {A1,...,A M } in the attribute composition; t kX is a combination of values ​​on X; for each attribute A m ∈X,t km It's A m a numeric or character constant in Z; When the relational model defined on the multi-source knowledge set is a triple (H k ,R,T k ), according to the relational model, triples with relational redundancy and relational conflict are obtained from the multi-source knowledge set to form the second multi-source knowledge set; wherein k is the data source, H k and T k are the head entity and tail entity provided by the k-th source respectively, and R is the character constant of the relationship between the entity pairs; The performing attribute conflict detection on the first multi-source knowledge set includes: If the attribute value of the triple in the first multi-source knowledge set is a character attribute value, then the similarity Sim between the attributes in the attribute value is calculated. If the similarity Sim is not less than a preset similarity threshold σ, then a random attribute value is retained; otherwise, the triple in the first multi-source knowledge set is added to the conflict resolution candidate set Φ; If the attribute value of the triple in the first multi-source knowledge set is a numerical attribute value, that is, it is redundant, then randomly retain an attribute value; otherwise, add the triple in the first multi-source knowledge set to the conflict resolution candidate set Φ; String similarity measurement methods include Hamming distance, substring similarity and edit distance; the similarity between any two character attribute values ​​is in the interval [0,1]; The performing relationship conflict detection on the second multi-source knowledge set includes: Constructing a relationship hierarchy system for the second multi-source knowledge set to detect whether there is a hierarchical relationship between relationship values; Calculate the similarity of the relationships between different triples in the second multi-source knowledge set after constructing the relationship hierarchy system.

2. The method according to claim 1, characterized in that Building a relationship hierarchy system mainly includes two parts: relationship classification and relationship tree generation; relationship classification is achieved through relationship classifier; The training process of the relation classifier is: First, a certain number of word pairs with hyponymy and hyponymy relationships are selected as positive samples, and the same number of word pairs without hyponymy and hyponymy relationships are selected as negative samples. Then, these positive and negative sample words are input into a search engine to obtain a large amount of relevant relationship pair corpus. Based on this relationship pair corpus, a word vector representation model of the relationship words is trained. Finally, a classifier is trained using the samples with hyponymy and hyponymy tags to obtain a relationship classification model, denoted as Classifier. The process of generating the relationship tree is as follows: The relationship tree is constructed by inserting nodes. First, the frequency of occurrence of each word in the relationship word set R is calculated, and then it is used as the weight of the relationship node. R is sorted from high to low according to frequency. Finally, a root node is created, and the relationship words in R are inserted into the tree by combining the classifier Classifier and frequency.

3. The method according to claim 1, characterized in that The calculating of the similarity of the relationships between different triples in the second multi-source knowledge set after the relationship hierarchy system is constructed includes: Calculate the similarity between the relations in different triples. If the similarity is 1, randomly retain a triple. If the similarity is not less than the preset similarity threshold τ and the relations in different triples have a hyponym relationship, retain the hyponym. Otherwise, randomly retain a triple. If the similarity is not 1 and is greater than or equal to a preset similarity threshold τ, the triples in the second multi-source knowledge set are added to the conflict resolution candidate set Φ.

4. The method according to claim 3, characterized in that The calculating of the similarity between relations in different triples includes: The vector representation of relational words is learned based on the word vector representation model, and the cosine similarity of the relations between different triplets is calculated.

5. The method according to claim 4, characterized in that The word vector representation model is based on the learning of the vector representation of the relationship word, and the cosine similarity of the relationship between different triples is calculated, specifically: Note the relation R i The word vector is represented as Relation R j The word vector is represented as The cosine similarity calculation method based on word vector is: When the relationship R i With R j Similarity Sim based on edit distance ED and cosine similarity Sim cos When both are greater than the similarity threshold, the relationship R is determined. i With R j resemblance.

Images