Method for verifying industrial knowledge graph based on semantic primitive prediction

Abstract

The application discloses an industrial knowledge graph verification method based on semantic primitive prediction and belongs to the technical field of knowledge graphs. It comprises the following steps: S1, preprocessing data, building a semantic primitive prediction model based on the preprocessed data, and simultaneously performing rule mining on the preprocessed data; S2, training the semantic primitive prediction model of step S1 to obtain a training model; S3, based on the rule mining in step S1 and the training model in step S2, verifying the knowledge graph to obtain a verified knowledge graph. The application solves the problem of difficult verification of a vertical field knowledge graph, uses an expert-annotated Hownet semantic primitive library to train the semantic primitive prediction model, and adds the principle that a semantic primitive is the most basic and smallest semantic unit that is not suitable for further segmentation. The semantic primitive is combined with the knowledge graph verification, and the accuracy and verification efficiency of the vertical field knowledge graph are greatly improved.

Description

Technical Field

[0001] This invention relates to a method for verifying industrial knowledge graphs based on semantic primitive prediction, and belongs to the field of knowledge graph technology. Background Technology

[0002] HowNet is a large-scale language knowledge base annotated by Mr. Dong Zhendong and Mr. Dong Qiang over several decades, primarily targeting Chinese vocabulary and concepts. HowNet adheres to the reductionist approach, believing that words and meanings can be described using smaller semantic units. These semantic units are called semantic primitives, which, as the name suggests, are atomic semantics—the smallest semantic units that are the most basic and indivisible. Since their inception, semantic primitives have received widespread attention in the field of natural language processing, with scholars both domestically and internationally exploring their important applications in areas such as lexical semantic disambiguation, similarity calculation, and text classification. Because new words and phrases emerge daily, and the semantics of existing concepts constantly change, annotating new words and phrases is time-consuming and laborious for human experts. Therefore, in the area of ​​automatic semantic primitive annotation, Xie Ruobing et al. proposed the semantic primitive prediction task. This task selects a set of semantic atoms suitable for constructing definition descriptions of new concepts from the HowNet semantic primitive set. The semantic primitive prediction task effectively addresses the challenge of semantic primitive annotation for those without HowNet background knowledge or specialized training.

[0003] A knowledge graph is a knowledge database that integrates data using a graph structure or topological model. It consists of a series of interconnected entities (objects, events, scenarios, etc.) and their attributes. It is a semi-structured or structured data model designed to capture and organize vast amounts of knowledge. Knowledge graphs allow humans or computers to process and use data in a network in an efficient and unambiguous way. Knowledge graphs can be organized and stored in various forms, the most common being the Resource Description Frame (RDF) proposed by the W3C. This framework expresses relationships between entities in the form of triples <subject-relationship-object>, which greatly facilitates the construction and storage of knowledge graphs. Currently, knowledge graphs can be applied to various fields to provide knowledge services, knowledge search, big data analysis, and other services in different domains.

[0004] Currently, knowledge graphs are widely used in the industrial sector. However, industrial data often contains noise, outliers, and incomplete data. This can be caused by sensor malfunctions, equipment failures, human error, or other interference factors. Therefore, knowledge graphs built on industrial data contain many errors, usually introduced during data entry, integration, or processing. Common error categories include entity errors, relation errors, attribute errors, and logical errors. Failure to detect errors in knowledge graphs not only affects their quality but also the quality of downstream tasks such as question-answering systems, information retrieval, recommendation systems, entity linking, and semantic search. Therefore, a fast and reliable method is urgently needed to detect errors in industrial knowledge graphs, ensuring their quality. Summary of the Invention

[0005] The present invention addresses the technical problems mentioned in the background section by employing the following technical solution:

[0006] The industrial knowledge graph verification method based on semantic primitive prediction includes the following steps:

[0007] S1. Preprocessing data: establishing a semantic primitive prediction model and performing rule mining on the preprocessed data;

[0008] S2. The training model is obtained by training the semantic primitive prediction model described in step S1.

[0009] S3. Based on the rule mining in step S1 and the training model in step S2, verify the knowledge graph to obtain the verified knowledge graph.

[0010] As a preferred example, the steps for preprocessing the data are as follows: it is necessary to obtain the training dataset of the semantic primitive training model and to annotate the structured unlabeled text dataset.

[0011] Furthermore, the training dataset for the semantic primitive training model is obtained using the following method:

[0012] Step 1: Obtain unstructured text and large corpus data of industrial data;

[0013] Step 2: Perform data cleaning, missing value imputation, and sentence and word segmentation preprocessing on the unstructured text mentioned in Step 1 to obtain a structured unlabeled text dataset.

[0014] Step 3: Label the unlabeled structured text dataset to obtain labeled data;

[0015] Step 4: The training dataset is obtained by integrating the labeled data from Step 3 with the data from the large corpus.

[0016] Furthermore, in step 2, the text dataset is denoted as Y = {y1, y2, ..., y...} i ,...,y N}, where N represents the number of sentences in a batch; y i Let i represent the i-th sentence, and Where M represents the number of words in the sentence; This represents the i-th sentence y. i The j-th word.

[0017] The structured unlabeled text dataset from step 3 is annotated using the following method, and the text annotation is denoted as Y:

[0018] D1. Manually label the industrial entities present in the sentence to obtain the entity types;

[0019] D2. Manually label the relationships between entities in the sentence to obtain the relationship types;

[0020] D3. Manually annotate the entities in the sentence with semantic primitives to obtain the semantic primitive types;

[0021] D4. Manually annotate the semantic primitives that point from the head entity to the tail entity in the relation pair to obtain the semantic primitive set.

[0022] Furthermore, in step D1, the entity type is denoted as: E = {e1, e2, ..., e} k ,...,e n}, where n represents the total number of entity types; where e k This represents the k-th entity type; in step D2, the relation type is denoted as R = {r1, r2, ..., r...} l ,...,r m}, where m represents the total number of relation types between entities; r l Let l represent the l-th relation type, and let i represent the triples in the i-th sentence. ,in This represents the i-th sentence y. i The k-th triplet, This represents the k-th head entity. This represents the k-th tail entity. Indicates the relation type of the triple. This indicates the entity type of the k-th header entity. This represents the entity type of the k-th tail entity; in step D3, the primitive type is denoted as S = {s1, s2, ..., s...} z ,...s v}, where v represents the total number of semantic primitive types existing in the entity; s zThis represents the z-th type of semantic primitive; in step D4, the set of semantic primitives is denoted as... This indicates that the semantic primitive of the first head entity points to the semantic primitive of the first tail entity;

[0023] As a preferred example, in step S1, the semantic primitive prediction model is established in the following manner:

[0024] Step 1: Establish a pre-trained character-enhanced word vector model;

[0025] Step 2: Based on the pre-trained character-enhanced word vector model in Step 1, build a character-enhanced model for the semantic primitive prediction model;

[0026] Step 3: Based on the encoded word vectors constructed in Step 2, build the position score model of the semantic primitive prediction model.

[0027] Step 4: Build a semantic primitive prediction model based on the character enhancement model and the positional score model.

[0028] Preferably, in step one, the character embedding, type embedding, position embedding, and segmentation embedding are added together as input to establish a pre-trained character-enhanced word vector model.

[0029] Furthermore, in step one, the method for establishing the pre-trained character-enhanced word vector model is as follows: a masked Transformer Encoder is used as the pre-trained character-enhanced word vector model. This model is based on a multi-head self-attention mechanism, and its masked self-attention training process formula is as follows:

[0030]

[0031] Where X∈R n×d X is a representation of the input sequence. The output sequence is represented by T, where T is the transpose symbol, n represents the number of tokens in the sequence, and d and d' ... k For each dimension of the token, Softmax() is the activation function. Its W... Q W K , These are learnable parameters, where M is the self-attention matrix and M∈R. n×n ;

[0032] In step two, the character enhancement model of the semantic primitive prediction model is built. First, its encoder is constructed, and the encoder encodes word-related information into vectors. The characters in each word are first concatenated using word embeddings, and the formula is as follows:

[0033]

[0034] Its c iW represents the i-th character in the word. U Let u be the projection matrix. i To obtain character-enhanced word embeddings through pre-training, The concatenated word vectors are then fed into the Bi-LSTM encoding. The connection between the last hidden state in each direction is denoted as h and fed into the multi-label classifier, with the following formula:

[0035]

[0036] Where W∈R |S|×2l ,b∈R|S|, w is the pre-trained word vector of a large corpus, Bi-LSTM is a bidirectional long short-term memory network, W is the projection matrix, b is the bias, l represents the dimension of the hidden state in a single direction, and its S G The original meaning of the word in question is scored;

[0037] In step three, the positional scoring model of the semantic primitive prediction model is built. First, the word-to-semantic primitive matrix M is constructed. ij M ij ∈{0,1} represents the word w i The corresponding semantic source s j Label;

[0038] Next, the word is decomposed into characters according to their positions, and the word w is defined as w = c1c2...c |w| Its decomposition formula is as follows:

[0039]

[0040] Where B, M, and E represent the positions Begin, Middle, and End, respectively; π represents the character set decomposed by position, and c represents the character decomposed.

[0041] Then, the character positional semantic primitive score function is constructed, and its formula is as follows:

[0042]

[0043] Where c and p represent the given character and its position, respectively, s j The primitive representing the score being calculated, π p Representing π B ,π M ,π E One of them, P p (s j |c) is the character positional semantic score;

[0044] Finally, the word position sense primitive scoring function is constructed, and its formula is as follows:

[0045]

[0046] Where P(s) j |w) represents the original score for word position meaning;

[0047] In step four, a semantic primitive prediction model is built based on the character enhancement model and the positional score model. First, the semantic primitive score of the character enhancement model is obtained, denoted as S. G And obtain the character positional semantic primitive score of the positional scoring model, denoted as S. L Then S G and S L The weighted sum of the two scores, combined with the final primitive score, is calculated using the following formula:

[0048] S E =αS G +(1-α)S L (7)

[0049] Wherein: S E This represents the final score of a word after the semantic primitive prediction model predicts its semantic primitives. α represents the weighting coefficient and is a constant that is fixed after the model is trained.

[0050] As a preferred example, in step S1, rule mining is performed on the preprocessed data according to the following method:

[0051] I. Based on y in step D2 i Triple i Head entity Tail-end entity Obtain the entity set, or, based on y in step D2 i Triple i Head entity Relationship type The entity type of the head entity Obtain the head entity relationship set; or, based on y in step D2 i Triple i tail entity Relationship type entity type of tail entity Obtain the tail entity relation set;

[0052] II. Traverse the entity set described in step I, and count the entity type and corresponding semantic primitive of each entity to obtain the semantic primitive set corresponding to the entity type, i.e., mine the semantic primitive rules corresponding to the entity class; or, traverse the head entity relationship pairs in the head entity relationship set described in step I, and count the relationship type to which the head entity belongs, the entity type of the head entity, and the corresponding semantic primitive in the head entity relationship pair to mine the semantic primitive rules corresponding to the relationship head entity; or, traverse the tail entity relationship pairs in the tail entity relationship set described in step I, and count the relationship type to which the tail entity belongs, the entity type of the tail entity, and the corresponding semantic primitive in the tail entity relationship pair to mine the semantic primitive rules corresponding to the relationship tail entity; or, traverse the y i Triple i Furthermore, it counts the set of semantic primitives that the head entity points to the tail entity, which allows it to mine the rules that the relationship points to the corresponding semantic primitives.

[0053] Furthermore, in step I, the entity set is represented as: Entity = {entity1, entity2, ..., entity} k ,...,entity n}, where: entity k Let n represent the k-th entity, and n represent the total number of entities.

[0054] The representation of the head entity relation set is as follows: in: This represents a head entity relation pair consisting of the k-th head entity, the l-th relation type, and the entity type of the k-th head entity, where n and m represent the number of head entities and the number of relation types, respectively.

[0055] Tail entity relation set, denoted as in This represents a head entity relation pair consisting of the k-th tail entity, the l-th relation type, and the entity type of the k-th tail entity, where n and m represent the number of head entities and the number of relation types, respectively.

[0056] In step II, the semantic primitive rules corresponding to entity classes can be mined, expressed as follows:

[0057] Where: entity represents a specific entity, e and E represent the type of entity and the collection of entity types, respectively; Sememe() and Type() retrieve the semantic primitive and entity type of a given entity, respectively; G e This represents the set of semantic primitives corresponding to this entity type;

[0058] The rules for mining relational header entities correspond to semantic primitives, expressed as follows:

[0059] Where r(head, tail) represents a relation. R and R represent the type of r and the set of relation types, respectively, and head and tail are the head entities of r. and H r These represent the semantic primitive set corresponding to the header entity of the relation type and the entity type of the header entity of the relation type, respectively.

[0060] The rules for mining relation tail entities correspond to semantic primitives, expressed as follows:

[0061]

[0062] and These represent the semantic primitive set corresponding to the tail entity of the relation type and the entity type of the tail entity of the relation type, respectively.

[0063] The mineable relation points to the corresponding semantic primitive rule, expressed as follows:

[0064]

[0065] Where: Sememe_d represents the primitive pointing to the semantic primitive. This indicates that the relation points to the corresponding semantic primitive set.

[0066] As a preferred example, in S2, the trained model is obtained in the following manner:

[0067] i. Take the data from the corpus and the labeled data as input, where the corpus data is used as pre-trained word vectors and the labeled data is used to train the semantic primitive prediction model in step S1.

[0068] ii. The pre-trained character-enhanced word vector model described in step one, the semantic primitive prediction model in step i, is the trained model, which is used to predict the semantic primitives of words.

[0069] Furthermore, in step i, the corpus data is used as pre-trained word vectors, the labeled data is used to train the semantic primitive prediction model, 15% of the input sequence is randomly masked, of which 80% are replaced with its mask token, 10% are replaced with random tokens of the same token type, and 10% remain unchanged, and formula (1) is used as the basic architecture of the pre-trained word vectors.

[0070] In step ii, the batch size of the pre-trained model is set to 1024, the word embedding dimension of the semantic primitive prediction model is set to 200, the hidden state dimension of the Bi-Lstm is set to 512*2, and the batch size is 128. The Adam optimizer is used to train the pre-trained model and the semantic primitive prediction model with a learning rate of 0.0001. The loss function L is calculated, and the error is backpropagated to the model to update the model parameters. Training stops when the loss function L no longer decreases.

[0071] As a preferred example, in step S3, the knowledge graph is verified in the following manner to obtain the verified knowledge graph:

[0072] s1, Based on the Triple in step D2 i The format is used to obtain the triples of the knowledge graph to be verified, and there are n sets of triples.

[0073] s2. Build the semantic primitive prediction module;

[0074] s3, Traverse Triple kg Get triples Head entity H k With tail entity T k The semantic primitive prediction module is used to obtain the semantic primitive set corresponding to the entity. The entity class is verified by rule (8) whether it corresponds to the semantic primitive rule. If the rule is met, the entity type of the entity is correct; otherwise, the entity type of the entity is incorrect.

[0075] s4, Get Triples Head entity relationship The semantic primitive prediction module is used to obtain the semantic primitive set corresponding to the head entity. The head entity is verified by rule (9) whether it corresponds to the semantic primitive rule. If the rule is satisfied, the entity type of the head entity in the relation is correct; otherwise, it is incorrect.

[0076] s5, Get Triples Tail entity relationship The semantic primitive prediction module is used to obtain the semantic primitive set corresponding to the tail entity. The rule (10) is used to verify whether the tail entity corresponds to the semantic primitive rule. If the rule is satisfied, the entity type of the tail entity in the relation is correct; otherwise, it is incorrect.

[0077] s6, Get Triples Then, the semantic primitive prediction module is used to obtain the semantic primitive set of the head entity pointing to the tail entity of the relation. The relation pointing to rule (11) is used to verify whether it corresponds to the semantic primitive rule. If the rule is satisfied, the triple is correct; otherwise, it is incorrect.

[0078] The beneficial effects of this invention are: by labeling data, this invention uses the labeled data to train the semantic primitive prediction model, obtains the optimal semantic primitive prediction model, and uses the training set to mine verification rules. The semantic primitive prediction module is built using the semantic primitive prediction model and instance classes in the Hownet library, and the semantic primitive prediction module is used to verify its knowledge graph. Triples in the graph that do not meet the verification rules are deleted, thereby completing the verification of errors in the knowledge graph.

[0079] This invention solves the problem of difficult verification of vertical domain knowledge graphs. It uses the expert-annotated Hownet semantic primitive library to train the semantic primitive prediction model and incorporates the principle that semantic primitives are the smallest semantic units that are the most basic and cannot be further divided. By combining semantic primitives with knowledge graph verification, it significantly improves the accuracy and efficiency of verifying vertical domain knowledge graphs. Attached Figure Description

[0080] Figure 1 This is a flowchart illustrating the implementation of the present invention;

[0081] Figure 2 This is a diagram of the semantic primitive prediction module in this invention;

[0082] Figure 3 This is a flowchart of the entity class correspondence primitive rule verification process used in this invention;

[0083] Figure 4 This is a flowchart illustrating the verification process of the relational header entity correspondence primitive rule in this invention;

[0084] Figure 5 This is a flowchart of the verification process for the relation tail entity corresponding to the semantic primitive rule in this invention;

[0085] Figure 6 This is a flowchart of the verification process for relational pointers to corresponding semantic primitives in this invention. Detailed Implementation

[0086] To facilitate a clear understanding of the technical means, creative features, objectives, and effects of this invention, the invention will be further described below in conjunction with specific illustrations and embodiments.

[0087] Example

[0088] like Figure 1-6 As shown, the industrial knowledge graph verification method based on semantic primitive prediction includes the following steps:

[0089] S1. Preprocessing data: A semantic primitive prediction model is built based on the preprocessed data, and rule mining is performed on the preprocessed data.

[0090] S2. The training model is obtained by training the primitive prediction model of step S1.

[0091] S3. Based on the rule mining in step S1 and the training model in step S2, the knowledge graph is verified to obtain the verified knowledge graph.

[0092] The preprocessed data includes preprocessing unstructured text from industrial data and data from large corpora, and text annotation of structured unlabeled text datasets to obtain the training dataset for the semantic primitive training model.

[0093] Obtain the training dataset for the semantic primitive training model using the following method:

[0094] Step 1: Obtain unstructured text and large corpus data of industrial data;

[0095] Step 2: Perform data cleaning, missing value imputation, and sentence and word segmentation preprocessing on the unstructured text mentioned in Step 1 to obtain a structured unlabeled text dataset.

[0096] Step 3: Label the unlabeled structured text dataset to obtain labeled data;

[0097] Step 4: The training dataset is obtained by integrating the labeled data from Step 3 with the data from the large corpus.

[0098] Wherein: In step 2, the text dataset is denoted as Y = {y1, y2, ..., y...} i ,...,y N}, where N represents the number of sentences in a batch; y i Let i represent the i-th sentence, and Where M represents the number of words in the sentence; This represents the i-th sentence y. i The j-th word.

[0099] The structured unlabeled text dataset from step 3 is annotated using the following method, and the text annotation is denoted as Y:

[0100] D1. Manually label the industrial entities present in the sentence to obtain the entity types;

[0101] D2. Manually label the relationships between entities in the sentence to obtain the relationship types;

[0102] D3. Manually annotate the entities in the sentence with semantic primitives to obtain the semantic primitive types;

[0103] D4. Manually annotate the semantic primitives that point from the head entity to the tail entity in the relation pair to obtain the semantic primitive set.

[0104] In step D1, the entity type is denoted as: E = {e1, e2, ..., e} k ,...,en}, where n represents the total number of entity types; where e k This represents the k-th entity type; in step D2, the relation type is denoted as R = {r1, r2, ..., r...} l ,...,r m}, where m represents the total number of relation types between entities; r l Let l represent the l-th relation type, and let i represent the triples in the i-th sentence. in This represents the i-th sentence y. i The k-th triplet, This represents the k-th head entity. This represents the k-th tail entity. Indicates the relation type of the triple. This indicates the entity type of the k-th header entity. This represents the entity type of the k-th tail entity; in step D3, the primitive type is denoted as S = {s1, s2, ..., s...} z ,...s v}, where v represents the total number of semantic primitive types existing in the entity; s z This represents the z-th type of semantic primitive; in step D4, the set of semantic primitives is denoted as... This indicates that the semantic primitive of the first head entity points to the semantic primitive of the first tail entity.

[0105] In step S1, the semantic primitive prediction model is established as follows:

[0106] Step 1: Combine character embeddings, type embeddings, position embeddings, and segmentation embeddings as input to build a pre-trained character-enhanced word vector model;

[0107] Step 2: Based on the pre-trained character-enhanced word vector model in Step 1, build a character-enhanced model for the semantic primitive prediction model;

[0108] Step 3: Based on the enhanced model in Step 2, build the position score model of the Yiyuan prediction model.

[0109] Step 4: Build a semantic primitive prediction model based on the character enhancement model and the positional score model.

[0110] In step one, the method for establishing the pre-trained character-enhanced word vector model is as follows: A masked TransformerEncoder is used as the pre-trained character-enhanced word vector model. This model is based on a multi-head self-attention mechanism, and its masked self-attention training process formula is as follows:

[0111]

[0112] Where X∈R n×d X is a representation of the input sequence. The output sequence is represented by T, where T is the transpose symbol, n represents the number of tokens in the sequence, and d and d' ... k For each dimension of the token, Softmax() is the activation function, and its W... Q W K , These are learnable parameters, where M is the self-attention matrix and M∈R. n×n ;

[0113] In step two, the character enhancement model of the semantic primitive prediction model is built. First, its encoder is constructed, and the encoder encodes word-related information into vectors. The characters in each word are first concatenated using word embeddings, and the formula is as follows:

[0114]

[0115] Its c i W represents the i-th character in the word. U Let u be the projection matrix. i To obtain character-enhanced word embeddings through pre-training, The concatenated word vectors are then fed into the Bi-LSTM encoding. The connection between the last hidden state in each direction is denoted as h and fed into the multi-label classifier, with the following formula:

[0116]

[0117] Where W∈R |S|×2l ,b∈R|S|, w is the pre-trained word vector of a large corpus, Bi-LSTM is a bidirectional long short-term memory network, W is the projection matrix, b is the bias, l represents the dimension of the hidden state in a single direction, and its S G The original meaning of the word in question is scored;

[0118] For training, the loss function formula is as follows:

[0119]

[0120] Where σ is the sigmoid function, y j ∈{0,1} indicates whether the j-th semantic primitive is in the semantic primitive set of word w;

[0121] In step three, the positional scoring model of the semantic primitive prediction model is built. Specifically, the word-to-semantic primitive matrix M is constructed first. ij M ij ∈{0,1} represents the word w i The corresponding semantic source s j Annotation; where M ij=1 represents the semantic primitive S j Belongs to the word w i The original collection of annotations now that M ij =0 indicates S j Not a word w i The original collection of annotations now that

[0122] Next, the word is decomposed into characters according to their positions, and the word w is defined as w = c1c2...c |w| Its decomposition formula is as follows:

[0123]

[0124] Where B, M, and E represent the positions Begin, Middle, and End, respectively; π represents the character set decomposed by position, and c represents the character decomposed.

[0125] Then, the character positional semantic primitive score function is constructed, and its formula is as follows:

[0126]

[0127] Where c and p represent the given character and its position, respectively, s j The primitive representing the score being calculated, π p Representing π B ,π M ,π E One of them, P p (s j |c) is the character positional semantic score;

[0128] Finally, the word position sense primitive scoring function is constructed, and its formula is as follows:

[0129]

[0130] Where P(s) j |w) represents the original score for word position meaning;

[0131] In step four, a semantic primitive prediction model is built based on the character enhancement model and the positional score model. First, the semantic primitive score of the character enhancement model is obtained, denoted as S. G And obtain the character positional semantic primitive score of the positional scoring model, denoted as S. L Then S G and S L The weighted sum of the two scores, combined with the final primitive score, is calculated using the following formula:

[0132] S E =αS G +(1-α)SL (7),

[0133] Where S E This represents the final score of a word after the semantic primitive prediction model predicts its semantic primitives. α represents the weighting coefficient and is a constant that is fixed after the model is trained.

[0134] In step S1, rule mining is performed on the preprocessed data according to the following method:

[0135] I. Based on y in step D2 i Triple i Head entity Tail-end entity Obtain the entity set, or, based on y in step D2 i Triple i Head entity and relation types Obtain the head entity relationship set; or, based on y in step D2 i Triple i tail entity and relation types Obtain the tail entity relation set;

[0136] II. Traverse the entity set in step I and count the entity type and corresponding semantic primitive of each entity to obtain the semantic primitive set corresponding to the entity type, i.e., mine the semantic primitive rules corresponding to the entity class; or, traverse the head entity relationship pairs in the head entity relationship set in step I and count the relationship type to which the head entity belongs, the entity type of the head entity, and the corresponding semantic primitive of each head entity relationship pair to mine the semantic primitive rules corresponding to the head entity; or, traverse the tail entity relationship pairs in the tail entity relationship set in step I and count the relationship type to which the tail entity belongs, the entity type of the tail entity, and the corresponding semantic primitive of each tail entity relationship pair to mine the semantic primitive rules corresponding to the tail entity; or, traverse the y in step I... i Triple i Furthermore, it counts the set of semantic primitives that the head entity points to the tail entity, which allows it to mine the rules that the relationship points to the corresponding semantic primitives.

[0137] In step I, the entity set is represented as: Entity = {entity1, entity2, ..., entity} k ,...,entity n}, where: entity k Let n represent the k-th entity, and n represent the total number of entities.

[0138] The representation of the head entity relation set is as follows: in: This represents a head entity relation pair consisting of the k-th head entity, the l-th relation type, and the entity type of the k-th head entity, where n and m represent the number of head entities and the number of relation types, respectively.

[0139] Tail entity relation set, denoted as in This represents a head entity relation pair consisting of the k-th tail entity, the l-th relation type, and the entity type of the k-th tail entity, where n and m represent the number of head entities and the number of relation types, respectively.

[0140] In step II, the semantic primitive rules corresponding to entity classes can be mined, expressed as follows:

[0141] Where: entity represents a specific entity, e and E represent the type of entity and the collection of entity types, respectively; Sememe() and Type() retrieve the semantic primitive and entity type of a given entity, respectively; G e This represents the set of semantic primitives corresponding to this entity type;

[0142] The rules for mining relational header entities correspond to semantic primitives, expressed as follows:

[0143] Where r(head, tail) represents a relation. R and R represent the type of r and the set of relation types, respectively, and head and tail are the head entities of r. and H r These represent the semantic primitive set corresponding to the header entity of the relation type and the entity type of the header entity of the relation type, respectively.

[0144] The rules for mining relation tail entities correspond to semantic primitives, expressed as follows:

[0145]

[0146] and These represent the semantic primitive set corresponding to the tail entity of the relation type and the entity type of the tail entity of the relation type, respectively.

[0147] The mineable relation points to the corresponding semantic primitive rule, expressed as follows:

[0148]

[0149] Where: Sememe_d represents the primitive pointing to the semantic primitive. This indicates that the relation points to the corresponding semantic primitive set.

[0150] In S2, the trained model is obtained as follows:

[0151] i. Take the data from the corpus and the labeled data as input, where the corpus data is used as pre-trained word vectors and the labeled data is used to train the semantic primitive prediction model in step S1.

[0152] ii. The pre-trained character-enhanced word vector model in step one and the semantic primitive prediction model in step i are trained to obtain the trained model, which is used to predict the semantic primitives of words.

[0153] In step i, corpus data is used as pre-trained word vectors, labeled data is used to train the semantic primitive prediction model, 15% of the input sequence is randomly masked, of which 80% are replaced with their mask tokens, 10% are replaced with random tokens of the same token type, and 10% remain unchanged, and formula (1) is used as the basic architecture of the pre-trained word vectors.

[0154] In step ii, the batch size of the pre-trained model is set to 1024, the word embedding dimension of the semantic primitive prediction model is set to 200, the hidden state dimension of the Bi-Lstm is set to 512*2, and the batch size is 128. The Adam optimizer is used to train the pre-trained model and the semantic primitive prediction model with a learning rate of 0.0001. The loss function L is calculated, and the error is backpropagated to the model to update the model parameters. Training stops when the loss function L no longer decreases.

[0155] In step S3, the knowledge graph is verified in the following manner to obtain the verified knowledge graph:

[0156] s1, Based on Triple in D2 i The format is used to obtain the triples of the knowledge graph to be verified, and there are n sets of triples, specifically the triples in (D2). i The format for obtaining the triples of the knowledge graph to be verified is denoted as follows:

[0157] s2. Build the semantic primitive prediction module, specifically referring to... Figure 2 First, obtain the word whose semantic primitives are to be predicted. Then, search for a certain entity semantic primitive in the Hownet library. If it is found, obtain the semantic primitive lookup function get_related_sememes() by instantiating the OpenHownet.HowNetDict() class, set the parameters relation = 'hypernym' and return_triples = True, and recursively search for its semantic primitives.

[0158] If the search in the library fails, the trained semantic primitive prediction model in (4) is used to perform semantic primitive prediction.

[0159] s3, Traverse Triple kg Get triples Head entity H k With tail entity T k The semantic primitive prediction module is used to obtain the semantic primitive set corresponding to the entity. The entity class is verified by using the mined rule (8) to see if it corresponds to the semantic primitive rule. If the rule is satisfied, the entity type of the entity is correct; otherwise, the entity type of the entity is incorrect.

[0160] s4, Get Triples Head entity relationship The semantic primitive prediction module is used to obtain the semantic primitive set corresponding to the head entity. The head entity is verified by rule (9) whether it corresponds to the semantic primitive rule. If the rule is satisfied, the entity type of the head entity in the relation is correct; otherwise, it is incorrect.

[0161] s5, Get Triples Tail entity relationship The semantic primitive prediction module is used to obtain the semantic primitive set corresponding to the tail entity. The rule (10) is used to verify whether the tail entity corresponds to the semantic primitive rule. If the rule is satisfied, the entity type of the tail entity in the relation is correct; otherwise, it is incorrect.

[0162] s6, Get Triples Then, the semantic primitive prediction module is used to obtain the semantic primitive set of the head entity pointing to the tail entity of the relation. The relation pointing to rule (11) is used to verify whether it corresponds to the semantic primitive rule. If the rule is satisfied, the triple is correct; otherwise, it is incorrect.

[0163] The basic principles, main features, and advantages of the present invention have been shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention, all of which fall within the scope of the claims. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for verifying industrial knowledge graphs based on semantic primitive prediction, characterized in that, Includes the following steps: S1. Preprocessing data: A semantic primitive prediction model is built based on the preprocessed data, and rule mining is performed on the preprocessed data. The preprocessed data includes: preprocessing unstructured text of industrial data and data from a large corpus, and text annotation of structured unlabeled text datasets to obtain the training dataset for the semantic primitive training model. S2. The training model obtained from the semantic primitive prediction model described in step S1 is the trained model, i.e. Step 1: Obtain unstructured text and large corpus data of industrial data; Step 2: Perform data cleaning, missing value imputation, and sentence and word segmentation preprocessing on the unstructured text mentioned in Step 1 to obtain a structured unlabeled text dataset. Step 3: Label the unlabeled structured text dataset to obtain labeled data; Step 4: Integrate the labeled data from Step 3 with the data from the large corpus to obtain the training dataset; Additionally, the structured unlabeled text dataset from step 3 is annotated using the following method; D1. Manually label the industrial entities present in the sentence to obtain the entity types; D2. Manually label the relationships between entities in the sentence to obtain the relationship types; D3. Manually annotate the entities in the sentence with semantic primitives to obtain the semantic primitive types; D4. Manually annotate the semantic primitives that point from the head entity to the tail entity in the relation pair to obtain the semantic primitive set; Rule mining was performed on the preprocessed data using the following methods: I. Based on the i-th sentence in step D2 triples Obtain the entity set from the head and tail entities, or, based on the i-th sentence in step D2. Obtain the head entity relation set by the head entity, relation type, and entity type of the head entity in the triple; or, based on the i-th sentence in step D2. Given the tail entity, relation type, and entity type of the triple, obtain the tail entity relation set; II. Traverse the entity set described in step I, and count the entity type and corresponding semantic primitive of the entity to obtain the semantic primitive set corresponding to the entity type, that is, mine the semantic primitive rules corresponding to the entity class. Alternatively, by traversing the head entity relation pairs in the head entity relation set described in step I, and counting the type of the head entity, the entity type of the head entity, and the corresponding semantic primitive in the head entity relation pair, the semantic primitive rules corresponding to the relation head entity can be mined. Alternatively, by traversing the tail entity relation pairs in the tail entity relation set described in step I, and counting the relation type to which the tail entity belongs, the entity type of the tail entity, and the corresponding semantic primitive in the tail entity relation pairs, the semantic primitive rules corresponding to the relation tail entity can be mined. Alternatively, iterate through the i-th sentence described in step I. triples And count the semantic primitives that the head entity points to the tail entity. This allows us to uncover relationships that point to corresponding semantic primitive rules; S3. Based on the rule mining in step S1 and the training model in step S2, verify the knowledge graph to obtain the verified knowledge graph. Additionally, verify the knowledge graph in the following manner to obtain the verified knowledge graph: S1. Based on the rule mining in step D2... The format is used to obtain the triples of the knowledge graph to be verified, and there are n sets of triples. s2. Build the semantic primitive prediction module; s3. Traverse the triples of the knowledge graph to be verified Get triples Head entity With tail entity The semantic primitive prediction module is used to obtain the semantic primitive set corresponding to the entity. The entity class is then verified by checking whether it corresponds to the semantic primitive rule. If the rule is met, the entity type is correct; otherwise, the entity type is incorrect. s4, Get Triples Head entity relationship The semantic primitive prediction module is used to obtain the semantic primitive set corresponding to the head entity. The semantic primitive rule is used to verify whether the head entity of the mineable relation corresponds to the semantic primitive. If the rule is satisfied, the entity type of the head entity in the relation is correct; otherwise, it is incorrect. s5, Get Triples Tail entity relationship The semantic primitive prediction module is used to obtain the semantic primitive set corresponding to the tail entity. The semantic primitive rule is used to verify whether the tail entity of the mineable relation corresponds to the semantic primitive. If the rule is satisfied, the entity type of the tail entity in the relation is correct; otherwise, it is incorrect. s6, Get Triples The semantic primitive prediction module is used to obtain the set of semantic primitives for the head entity pointing to the tail entity of the relation. The corresponding semantic primitive rule is verified by using the mineable relation pointer. If the rule is satisfied, the triple is correct; otherwise, it is incorrect.

2. The verification method according to claim 1, characterized in that: The text annotation in step 3 is marked as Y: Entity type is denoted as: , where n represents the total number of entity types; Indicates the k-th entity type; The relation type is denoted as , where m represents the total number of relationship types between entities; Let l represent the l-th relation type, and let i represent the triples in the i-th sentence. ,in This represents the i-th sentence. The k-th triplet, This represents the k-th head entity. This represents the k-th tail entity. Indicates the relation type of the triple. , This indicates the entity type of the k-th header entity. Indicates the entity type of the k-th tail entity; The semantic primitive type is denoted as , where v represents the total number of semantic primitives present in the entity; This represents the z-th semantic primitive type; Yoshihara Collection, recorded as , This indicates that the semantic primitive of the first head entity points to the semantic primitive of the first tail entity.

3. The verification method according to claim 1, characterized in that, In step S1, the semantic primitive prediction model is established as follows: Step 1: Establish a pre-trained character-enhanced word vector model; Step 2: Based on the pre-trained character-enhanced word vector model in Step 1, build a character-enhanced model for the semantic primitive prediction model; Step 3: Based on the enhanced model in Step 2, build the position score model of the Yiyuan prediction model; Step 4: Build a semantic primitive prediction model based on the character enhancement model and the positional score model.

4. The verification method according to claim 3, characterized in that: In step one, character embedding, type embedding, position embedding, and segmentation embedding are added together as input to build a pre-trained character-enhanced word vector model.

5. The verification method according to claim 1, characterized in that: In step D2 triples The head entity is denoted as The entity with the tail is denoted as Or, based on step D2 triples The head entity is denoted as The relation type is denoted as The entity type of the header entity is denoted as ; Or, based on step D2 triples The tail entity is denoted as The relation type is denoted as The entity type of the tail entity is denoted as ; Mining entity class corresponding semantic primitive rules, denoted as ,in: Represents a certain entity, and They represent A collection of types and entity types, and These represent the primitive and entity type of a given entity, respectively. This represents the set of semantic primitives corresponding to this entity type; The rules for mining relational header entity correspondence primitives are denoted as: ; in: To indicate a certain relationship, and They represent A collection of types and relational types, and for The head entity, and These represent the semantic primitive set corresponding to the header entity of the relation type and the entity type of the header entity of the relation type, respectively. The key-value rules for mining relation tail entities are denoted as: ; The mineable relation points to the corresponding semantic primitive rule, denoted as ; in: This indicates the origin of the referential relationship. This indicates that the relation points to the corresponding semantic primitive set.

6. The verification method according to claim 1, characterized in that: In step S2, the trained model is obtained as follows: i. Take the data from the corpus and the labeled data as input, where the corpus data is used as pre-trained word vectors and the labeled data is used to train the semantic primitive prediction model in step S1. ii. The pre-trained character-enhanced word vector model described in step one and the semantic primitive prediction model in step i are trained to obtain the trained semantic primitive prediction model, which is used to predict the semantic primitives of words.

Images