A method for constructing knowledge graph based on automatic extraction and alignment of government affairs triples
Through the automatic extraction and alignment of government affairs triples, the problems of low efficiency and poor accuracy of government affairs knowledge graph construction are solved, and efficient, accurate update and rapid iteration of government affairs knowledge graphs are achieved.
Patent Information
- Application Number
- CN202210166232.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-23
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-02-23
AI Technical Summary
Traditional relational databases are difficult to support intelligent government affairs Q&A in terms of efficiency and knowledge reasoning. The construction of government affairs knowledge graphs is large and inefficient, and the knowledge sources are diverse and the update time is not uniform, resulting in low knowledge accuracy and update efficiency.
Using the method of automatic extraction and alignment based on government affairs triples, the government affairs knowledge data set, term data set and relational data set are constructed, and entity annotation and feature extraction are used using the BERT-BiLSTM-CRF model, and the entity similarity is calculated in combination with the rules of government affairs ontology. to realize the automated construction and rapid update of government affairs knowledge graph.
It improves the accuracy and update efficiency of the government knowledge graph, supports rapid iteration and efficient construction, and ensures the timeliness and accuracy of knowledge.
Smart Images

Figure CN114580639B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a method for constructing a knowledge graph based on automatic extraction and alignment of government affairs triples. Background Art
[0002] Traditional relational databases cannot meet the requirements of supporting intelligent government question answering in terms of efficiency and knowledge reasoning. At the same time, the government affairs field involves a wide range of sub-fields, such as taxation, medical care, education, industry and commerce, food and medicine, etc., which makes the manual construction of government affairs knowledge graphs too labor-intensive and inefficient. Government affairs knowledge also has problems such as many sources and inconsistent update time. In addition, the application of domain knowledge bases requires high accuracy of knowledge. Domain ontology provides a formal description of domain entity concepts and interrelationships, domain activities, and the characteristics and laws of the field. Extracting government affairs knowledge based on domain ontology can improve the accuracy of extraction. Summary of the invention
[0003] In view of the problems existing in the background technology, the present invention provides a method for constructing a knowledge graph based on automatic extraction and alignment of government affairs triples, which is characterized by comprising the following steps:
[0004] Step 1: Construct a government knowledge dataset;
[0005] Step 2: Construct the government terminology dataset, government terminology relationship dataset, and government terminology attribute dataset respectively to define the government ontology;
[0006] Step 3: For the structured data obtained in step 1, a mapping method is used to obtain the government affairs triples;
[0007] Step 4: Use the BIO system to annotate government entities in the government knowledge dataset obtained in step 1 for training the BERT model. Use the trained BERT model to convert the government knowledge dataset obtained in step 1 into word vectors. Input the word vectors into the BILSTM model, extract feature information through training to obtain feature vectors, output the feature vectors to the CRF layer, and calculate the best annotation sequence. Use the rules in the ontology constructed in step 3 to constrain the entities in the annotation sequence.
[0008] Step 5: Construct rule templates by using the method of pre-defined relationships;
[0009] Step 6: Calculate the similarity between the entities in the triples extracted in step 4 and the entities in the existing government knowledge graph, and import the calculation results into the triple set T;
[0010] Preferably, the government affairs knowledge dataset in step 1 is defined as:
[0011] K=(k1,k2,...,k n )
[0012] type i ={data i,t , data i,c}
[0013] Among them, k i is the i-th government knowledge, i∈[1,n], n represents the number of government knowledge in the government knowledge dataset, type i Represents the attribute data set of the i-th government affairs knowledge, data i,t represents the time attribute of the i-th government knowledge, data i,t By obtaining the release time of the i-th government knowledge, data i,c represents the confidence attribute of the i-th government knowledge, data i,c The confidence score of the source of the i-th government knowledge is generated by organizing government experts. Government knowledge is divided into structured data and unstructured data. Structured data is data logically expressed and implemented by a two-dimensional table structure, strictly following the data format and length specifications. Other data that does not meet the structured definition is collectively referred to as unstructured data.
[0014] Preferably, the government terminology dataset in step 2 is:
[0015] J=(j1,j2,...,j M )
[0016] Among them, J is the government terminology dataset, j i is the i-th government term, i∈[1,M], M represents the number of government terms in the government terminology set;
[0017] The government terminology relationship dataset described in step 2 is:
[0018] R=(r i,j )
[0019] i∈[1,M],j∈[1,M], and i≠j
[0020] Among them, R is the government terminology relation dataset, r i,j is the connection between the i-th government term and the j-th government term, M is the number of government terms in the government term set, if there is no connection between the i-th government term and the j-th government term, then
[0021] Step 2 describes the construction of the government terminology attribute dataset as follows:
[0022] P=(p1,p2,...,p M )
[0023] p i =(p i,1 ,p i,2 ,,...,p i,Ni )
[0024] Among them, P is the government terminology attribute dataset, p i is the attribute dataset of the i-th government term, p i,j is the jth attribute in the attribute data set of the i-th government term;
[0025] Preferably, the step 3 is specifically as follows:
[0026] Use the D2R tool to map the structured data into a triple set according to the constraints of the government ontology described in step 2. The header of the structured data is mapped to the government terms in the government ontology, and the field name of the structured data is mapped to the attribute name of the corresponding entity.
[0027] Preferably, the BERT layer in step 4 converts the text in the government knowledge dataset obtained in step 1 into a word vector, and the specific process is as follows:
[0028] The BERT (Bidirectional Encoder Representations from Transformers) layer converts text into word vectors. The sentence input to BERT is recorded as:
[0029] S=(s1,s2,...,s n )
[0030] Among them, s i is the i-th word in the sentence, i∈[1,n], and the corresponding BIO label is recorded as:
[0031] L=(l1,l2,...,l n )
[0032] Among them, l i is the label of the i-th word in the sentence, i∈[1,n]. After inputting the sentence S into the BERT model, we get the vectorized output S of S. v , recorded as:
[0033] S v =(c1,c2,...,c n )
[0034] Among them, c i is the word vector of the i-th word in the sentence.
[0035] S v Input into the BILSTM model described in step 4;
[0036] The core of LSTM is mainly the following structure: forget gate, input gate, output gate and memory cell. The forget gate and input gate work together to clear invalid information and pass valid information to the next moment;
[0037] The output of the entire structure is mainly obtained by multiplying the outputs of the memory cell and the output gate. Its structure is expressed by the following formula:
[0038] i t =σ(x t W xi +h t-1 W hi +c t-1 W ci +b i )
[0039] z t =tanh(x t W xc +h t-1 W hc +b c )
[0040] f t =σ(x t W xf +h t-1 W hf +c t-1 W cf +b f )
[0041] c t =c j-1 f+i t z t
[0042] o t =tanh(x t W xo +h t-1 W ho +c t-1 W co +b o )
[0043] h t = otanh(c t )
[0044] Among them, σ is the sigmoid function, W is the weight matrix, b is the bias vector, i t 、f t , o t are the output results of the input gate, forget gate and output gate respectively, z tis the content to be added, c t is the updated state at time t, h t is the output of the entire LSTM unit at time t.
[0045] The basic idea of BILSTM is to process the input vector S from the forward and backward directions respectively. v Encode, then merge the outputs at the same time, output the predicted label corresponding to each word, and the label sequence is recorded as:
[0046] Y=(y1,y2,...,y n )
[0047] Among them, y i is the predicted label for the i-th word in the sentence.
[0048] Input Y into the CRF layer described in step 4 and calculate the probability of the label sequence using the following formula:
[0049]
[0050] Z(x)=∑ y' exp(score(x,y'))
[0051] score(x,y)=∑ i Emit(x i ,y i )+Trans(y i-1 ,y i )
[0052] The corresponding loss function is as follows:
[0053] -logp=(y|x)=-score(x,y)+logZ(x)
[0054] The output is the predicted label sequence. The predicted entity label starts with B and continues with I. The predicted entity set is denoted by N.
[0055] The pattern constraint layer uses the rules in ontology O to constrain the entities in entity set N, and the entity set that meets the constraints is recorded as N'.
[0056] Let the time and confidence attributes of the entities in N' be equal to the time and confidence attributes of the source government knowledge, respectively, and record them as:
[0057] n′ i (p t ) = k j (p t ),n′ i (p c ) = k j (p c )
[0058] Among them, n' i , k j are the entity set N' and the corresponding government knowledge, p t 、p c They are time and confidence attributes respectively.
[0059] Preferably, the step 5 is specifically as follows:
[0060] Based on the template, the government affairs triple t is extracted from the government affairs knowledge K, which is recorded as:
[0061] t i =(n x1 ,r l ,n x2 )|t i =(n x3 ,p y ,v z )
[0062] Where n, r, p, and v represent entities, relations, attributes, and attribute values, respectively. Let the time and confidence attributes of the government affairs triplet be equal to the time and confidence attributes of the source government affairs knowledge, respectively, and record them as:
[0063] t i (p t ) = k j (p t ),t i (p c ) = k j (p c )
[0064] where t i , k j are the government affairs triples and the corresponding government affairs knowledge, p t 、p c They are time and confidence attributes respectively.
[0065] Preferably, the step 6 is specifically as follows:
[0066] The three layers of the entity similarity calculation model are the pre-trained language model BERT, the fully connected neural network (DNN) and Softmax, which builds an "entity pair" binary classification model. The model inputs an "entity pair" (n1, n2) and constructs a standard input sequence through character processing:
[0067] [CLS]Q1...Qn[SEP]R1...Rm[SEP]
[0068] [CLS] and [SEP] are two special symbol markers of BERT, which are used to distinguish different entities; [CLS] appears at the beginning of the entity pair, and [SEP] appears at the boundary between two entities and at the end of the entity pair. The model inputs the vector of the last layer of BERT into DNN (fully connected neural network) for dimensionality reduction and feature extraction, and then performs binary classification through Softmax to obtain the similarity probability distribution result. The calculation formula is as follows:
[0069]
[0070] Among them, i represents a category in k, k∈(0,2],g i Represents the value of the classification. The loss function of the model uses the cross entropy loss function, which is as follows:
[0071]
[0072] where y i Indicates the label of sample i, the positive class is 1, the negative class is 0, p i represents the probability that sample i is predicted to be positive. Finally, by setting a threshold in the prediction stage, similar entity pairs are calculated. The entities extracted in step 4 are compared with the time attributes (t i (p t )) and confidence attribute (t j (p c )) to compare. Update entities with newer time and higher confidence to the government knowledge graph.
[0073] When extracting government affairs triples, the present invention marks their credibility and update time to ensure the accuracy and timeliness of the knowledge graph. At the same time, this method can efficiently and automatically construct a government affairs knowledge graph that supports rapid update and iteration. BRIEF DESCRIPTION OF THE DRAWINGS
[0074] Figure 1 :It is a flow chart of the present invention;
[0075] Figure 2 : The model layer is a flowchart of the government ontology construction, and the data layer is the process of building the government data into the government knowledge graph;
[0076] Figure 3 : It is a partial diagram of the government ontology and instances created based on the seven-step method;
[0077] Figure 4 : An example of mapping structured data into triples;
[0078] Figure 5 : An example of a pre-training dataset for annotated government named entity recognition;
[0079] Figure 6 : BERT-BiLSTM-CRF model diagram used for entity extraction;
[0080] Figure 7 :It is the LSTM unit structure diagram;
[0081] Figure 8 : Comparison of the results of directly extracting government entities and extracting government entities based on ontology;
[0082] Fig. 9 : BERT-DNN-Softmax model diagram used for entity similarity calculation. DETAILED DESCRIPTION
[0083] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0084] The following is combined with Figure 1-9 The implementation of the present invention is further described.
[0085] As attached Figure 1 As shown, the embodiment of the present invention is a method for constructing a knowledge graph based on automatic extraction and alignment of government affairs triples, and the specific steps are as follows:
[0086] Step 1: Construct a government knowledge dataset;
[0087] Government knowledge is crawled from government websites, including but not limited to national, provincial and municipal government service websites, official websites of departments at all levels, etc. Government knowledge is including but not limited to information disclosure guidelines, government service guides and other data with a certain structure and text data such as policy documents, legal and regulatory documents.
[0088] The government affairs knowledge dataset described in step 1 is defined as:
[0089] K=(k1,k2,...,k n )
[0090] type i ={data i,t , data i,c}
[0091] Among them, k iis the i-th government knowledge, i∈[1,n], n represents the number of government knowledge in the government knowledge dataset, type i Represents the attribute data set of the i-th government affairs knowledge, data i,t represents the time attribute of the i-th government knowledge, data i,t By obtaining the release time of the i-th government knowledge, data i,c represents the confidence attribute of the i-th government knowledge, data i,c The confidence score of the source of the i-th government knowledge is generated by organizing government experts. Government knowledge is divided into structured data and unstructured data. Structured data is data logically expressed and implemented by a two-dimensional table structure, strictly following the data format and length specifications. Other data that does not meet the structured definition is collectively referred to as unstructured data.
[0092] Step 2: Build the government terminology dataset, government terminology relationship dataset, and government terminology attribute dataset respectively to define the government ontology.
[0093] The government terminology dataset described in step 2 is:
[0094] J=(j1,j2,...,j M )
[0095] Among them, J is the government terminology dataset, j i is the i-th government term, i∈[1,M], M represents the number of government terms in the government terminology set;
[0096] The government terminology relationship dataset described in step 2 is:
[0097] R=(r i,j )
[0098] i∈[1,M],j∈[1,M], and i≠j
[0099] Among them, R is the government terminology relation dataset, r i,j is the connection between the i-th government term and the j-th government term, M is the number of government terms in the government term set, if there is no connection between the i-th government term and the j-th government term, then
[0100] Step 2 describes the construction of the government terminology attribute dataset as follows:
[0101] P=(p1,p2,...,p M )
[0102] p i =(p i,1 ,p i,2 ,,...,pi,Ni )
[0103] Among them, P is the government terminology attribute dataset, p i is the attribute dataset of the i-th government term, p i,j is the jth attribute in the attribute data set of the i-th government term;
[0104] Step 3: For the structured data obtained in step 1, a mapping method is used to obtain the government affairs triplet.
[0105] Use the D2R tool to map the structured data into a triple set according to the constraints of the government ontology described in step 2. The header of the structured data is mapped to the government terms in the government ontology, and the field name of the structured data is mapped to the attribute name of the corresponding entity. Figure 4 An example of mapping structured data to triples.
[0106] Step 4: Use the BERT-BiLSTM-CRF model for entity extraction. Figure 6 The structure diagram is as follows: The BIO system is used to annotate government entities in the government knowledge dataset obtained in step 1 for training the BERT model. Figure 5 For a labeling example (where B is the beginning of a noun phrase, I is the middle of a noun phrase, and O is not a noun phrase), use the trained BERT model to convert the government knowledge dataset obtained in step 1 into a word vector; input the word vector into the BILSTM model, extract feature information through training to obtain a feature vector, output the feature vector to the CRF layer, and calculate the optimal labeling sequence; use step 3 to construct the rules in the ontology to constrain the entities in the labeling sequence.
[0107] The BERT layer in step 3 converts the text in the government knowledge dataset obtained in step 1 into word vectors. The specific process is as follows:
[0108] The BERT (Bidirectional Encoder Representations from Transformers) layer converts text into word vectors. The sentence input to BERT is recorded as:
[0109] S=(s1,s2,...,s n )
[0110] Among them, s i is the i-th word in the sentence, i∈[1,n], and the corresponding BIO label is recorded as:
[0111] L=(l1,l2,...,l n )
[0112] Among them, li is the label of the i-th word in the sentence, i∈[1,n]. After inputting the sentence S into the BERT model, we get the vectorized output S of S. v , recorded as:
[0113] S v =(c1,c2,...,c n )
[0114] Among them, c i is the word vector of the i-th word in the sentence.
[0115] S v Input the BILSTM model described in step 3, Figure 7 The core of LSTM is mainly the following structure: forget gate, input gate, output gate and memory cell. The forget gate and input gate work together to clear invalid information and pass valid information to the next moment. The output of the entire structure is mainly obtained by multiplying the output of the memory cell and the output gate. Its structure is expressed by the following formula:
[0116] i t =σ(x t W xi +h t-1 W hi +c t-1 W ci +b i )
[0117] z t =tanh(x t W xc +h t-1 W hc +b c )
[0118] f t =σ(x t W xf +h t-1 W hf +c t-1 W cf +b f )
[0119] c t =c j-1 f+i t z t
[0120] o t =tanh(x t W xo +h t-1 W ho +ct-1 W co +b o )
[0121] h t = otanh(c t )
[0122] Among them, σ is the sigmoid function, W is the weight matrix, b is the bias vector, i t 、f t , o t are the output results of the input gate, forget gate and output gate respectively, z t is the content to be added, c t is the updated state at time t, h t is the output of the entire LSTM unit at time t.
[0123] The basic idea of BILSTM is to process the input vector S from the forward and backward directions respectively. v Encode, then merge the outputs at the same time, output the predicted label corresponding to each word, and the label sequence is recorded as:
[0124] Y=(y1,y2,...,y n )
[0125] Among them, y i is the predicted label for the i-th word in the sentence.
[0126] Input Y into the CRF layer described in step 3 and calculate the probability of the label sequence using the following formula:
[0127]
[0128] Z(x)=∑ y' exp(score(x,y'))
[0129] score(x,y)=∑ i Emit(x i ,y i )+Trans(y i-1 ,y i )
[0130] The corresponding loss function is as follows:
[0131] -log p=(y|x)=-score(x,y)+log Z(x)
[0132] The output is the predicted label sequence. The predicted entity label starts with B and continues with I. The predicted entity set is denoted by N.
[0133] (4) The Schema Constraint (SC) layer uses the rules in the ontology O to constrain the entities in the entity set N. For example, Figure 8 As shown, the entity set that meets the constraints is denoted as N'.
[0134] Let the time and confidence attributes of the entities in N' be equal to the time and confidence attributes of the source government knowledge, respectively, and record them as:
[0135] n′ i (p t ) = k j (p t ),n′ i (p c ) = k j (p c )
[0136] Among them, n' i , k j are the entity set N' and the corresponding government knowledge, p t 、p c They are time and confidence attributes respectively.
[0137] Step 5: Domain relationship extraction refers to extracting semantic relationships between entities in one or more domains. The extraction modes are divided into pre-defined relationships and directly extracting result-based relationships from text. Step 2 has defined the relationship set R (attribute set P) between terms in ontology O, and step 4 also links entities and terms during entity extraction, so the method of pre-defined relationships (attributes) is used here to construct rule templates.
[0138] Based on the template, the government affairs triple t is extracted from the government affairs knowledge K, which is recorded as:
[0139] t i =(n x1 ,r l ,n x2 )|t i =(n x3 ,p y ,v z )
[0140] Where n, r, p, and v represent entities, relations, attributes, and attribute values, respectively. Let the time and confidence attributes of the government affairs triplet be equal to the time and confidence attributes of the source government affairs knowledge, respectively, and record them as:
[0141] t i (p t ) = k j (p t ),t i (p c ) = k j(p c )
[0142] where t i , k j are the government affairs triples and the corresponding government affairs knowledge, p t 、p c They are time and confidence attributes respectively.
[0143] Step 6: Calculate the similarity between the entities in the triples extracted in step 4 and the entities in the existing government knowledge graph, and import the calculation results into the triple set T.
[0144] The entity similarity calculation model is as follows Fig. 9 As shown in the figure, the three layers are the pre-trained language model BERT, the fully connected neural network (DNN) and Softmax, which build the "entity pair" binary classification model. The model inputs the "entity pair" (n1, n2) and constructs it into a standard input sequence through character processing:
[0145] [CLS]Q1...Qn[SEP]R1...Rm[SEP]
[0146] [CLS] and [SEP] are two special symbol markers of BERT, which are used to distinguish different entities; [CLS] appears at the beginning of the entity pair, and [SEP] appears at the boundary between two entities and at the end of the entity pair. The model inputs the vector of the last layer of BERT into DNN (fully connected neural network) for dimensionality reduction and feature extraction, and then performs binary classification through Softmax to obtain the similarity probability distribution result. The calculation formula is as follows:
[0147]
[0148] Among them, i represents a category in k, k∈(0,2],g i Represents the value of the classification. The loss function of the model uses the cross entropy loss function, which is as follows:
[0149]
[0150] where y i Indicates the label of sample i, the positive class is 1, the negative class is 0, p i represents the probability that sample i is predicted to be positive. Finally, by setting a threshold in the prediction stage, similar entity pairs are calculated. The entities extracted in step 4 are compared with the time attributes (t i (p t )) and confidence attribute (t j (p c )) to compare. Update entities with newer time and higher confidence to the government knowledge graph.
[0151] Finally, it should be noted that the protection scope of the present invention is not limited to the above-mentioned embodiments. Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the scope and spirit of the present invention. If these changes and modifications fall within the scope of the claims of the present invention and their equivalents, the present invention is intended to include these changes and modifications.
Claims
1. A method for constructing a knowledge graph based on automatic extraction and alignment of government affairs triples, characterized in that: The following steps are involved: Step 1: Construct a government knowledge dataset; Step 2: Construct the government terminology dataset, government terminology relationship dataset, and government terminology attribute dataset respectively to define the government ontology; Step 3: For the structured data obtained in step 1, a mapping method is used to obtain the government affairs triples; Step 4: Use the BIO system to annotate government entities in the government knowledge dataset obtained in step 1 for training the BERT model. Use the trained BERT model to convert the government knowledge dataset obtained in step 1 into word vectors. Input the word vectors into the BILSTM model, extract feature information through training to obtain feature vectors, output the feature vectors to the CRF layer, and calculate the best annotation sequence. Use the rules in the ontology constructed in step 3 to constrain the entities in the annotation sequence. Step 5: Construct rule templates by using the method of pre-defined relationships; Step 6: Calculate the similarity between the entities in the triples extracted in step 4 and the entities in the existing government knowledge graph, and import the calculation results into the triple set T; The government affairs knowledge dataset described in step 1 is defined as: K=(k1,k2,...,k n ) type i ={data i,t ,data i,c } Among them, k i is the i-th government knowledge, i∈[1,n], n represents the number of government knowledge in the government knowledge dataset, type i Represents the attribute data set of the i-th government affairs knowledge, data i,t represents the time attribute of the i-th government knowledge, data i,t By obtaining the release time of the i-th government knowledge, data i,c represents the confidence attribute of the i-th government knowledge, data i,c Government knowledge is generated by organizing government experts to score the confidence of the source of the i-th government knowledge; government knowledge is divided into structured data and unstructured data. Structured data is data that is logically expressed and implemented by a two-dimensional table structure, strictly following the data format and length specifications. Other data that does not meet the structured definition is collectively referred to as unstructured data; The government terminology dataset described in step 2 is: J=(j1,j2,...,j M ) Among them, J is the government terminology dataset, j i is the i-th government term, i∈[1,M], M represents the number of government terms in the government terminology set; The government terminology relationship dataset described in step 2 is: R=(r i,j ) i∈[1,M],j∈[1,M], and i≠j Among them, R is the government terminology relation dataset, r i,j is the connection between the i-th government term and the j-th government term, M is the number of government terms in the government term set, if there is no connection between the i-th government term and the j-th government term, then Step 2 describes the construction of the government terminology attribute dataset as follows: P=(p1,p2,...,p M ) Among them, P is the government terminology attribute dataset, p i is the attribute dataset of the i-th government term, p i,j is the jth attribute in the attribute data set of the i-th government term; The step 3 is specifically as follows: Use the D2R tool to map the structured data into a triple set according to the constraints of the government ontology described in step 2. The header of the structured data is mapped to the government terms in the government ontology, and the field name of the structured data is mapped to the attribute name of the corresponding entity. The BERT layer in step 4 converts the text in the government knowledge dataset obtained in step 1 into word vectors. The specific process is as follows: The BERT layer converts text into word vectors and records the sentence input to BERT as: S=(s1,s2,...,s n ) Among them, s i is the i-th word in the sentence, i∈[1,n], and the corresponding BIO label is recorded as: L=(l1,l2,...,l n ) Among them, l i is the label of the i-th word in the sentence, i∈[1,n]; after inputting the sentence S into the BERT model, the vectorized output S of S is obtained v , recorded as: S v =(c1,c2,...,c n ) Among them, c i is the word vector of the i-th word in the sentence; S v Input into the BILSTM model described in step 4; The core of LSTM is mainly the following structure: forget gate, input gate, output gate and memory cell. The forget gate and input gate work together to clear invalid information and pass valid information to the next moment; The output of the entire structure is mainly obtained by multiplying the outputs of the memory cell and the output gate. Its structure is expressed by the following formula: i t =σ(x t W xi +h t-1 W hi +c t-1 W ci +b i ) z t =tanh(x t W xc +h t-1 W hc +b c ) f t =σ(x t W xf +h t-1 W hf +c t-1 W cf +b f ) c t =c j-1 f+i t z t o t =tanh(x t W xo +h t-1 W ho +c t-1 W co +b o ) h t =otanh(c t ) Among them, σ is the sigmoid function, W is the weight matrix, b is the bias vector, i t 、f t , o t are the output results of the input gate, forget gate and output gate respectively, z t is the content to be added, c t is the updated state at time t, h t is the output of the entire LSTM unit at time t; The basic idea of BILSTM is to process the input vector S from the forward and backward directions respectively. v Encode, then merge the outputs at the same time, output the predicted label corresponding to each word, and the label sequence is recorded as: Y=(y1,y2,...,y n ) Among them, y i is the predicted label of the i-th word in the sentence; Input Y into the CRF layer described in step 4 and calculate the probability of the label sequence using the following formula: Z(x)=∑ y' exp(score(x,y')) score(x,y)=∑ i Emit(x i ,y i )+Trans(y i-1 ,y i ) The corresponding loss function is as follows: -logp=(y|x)=-score(x,y)+logZ(x) The output is the predicted label sequence; the label of the predicted entity starts with B and continues with I, and the predicted entity set is recorded as N; The pattern constraint layer uses the rules in ontology O to constrain the entities in entity set N. The entity set that meets the constraints is recorded as N'; Let the time and confidence attributes of the entities in N' be equal to the time and confidence attributes of the source government knowledge, respectively, and record them as: n i '(p t )=k j (p t ),n i '(p c )=k j (p c ) Among them, n' i , k j are the entity set N' and the corresponding government knowledge, p t 、p c They are time and confidence attributes respectively.
2. The method for constructing a knowledge graph based on automatic extraction and alignment of government affairs triples according to claim 1 is characterized in that: The following steps are involved: The step 5 is specifically as follows: Based on the template, the government affairs triple t is extracted from the government affairs knowledge K, which is recorded as: t i =(n x1 ,r l ,n x2 )|t i =(n x3 ,p y ,v z ) Where n, r, p, and v represent entities, relations, attributes, and attribute values, respectively; let the time and confidence attributes of the government affairs triplet be equal to the time and confidence attributes of the source government affairs knowledge, respectively, recorded as: t i (p t )=k j (p t ),t i (p c )=k j (p c ) where t i , k j are the government affairs triples and the corresponding government affairs knowledge, p t 、p c They are time and confidence attributes respectively.
3. The method for constructing a knowledge graph based on automatic extraction and alignment of government affairs triples according to claim 1 is characterized in that: The following steps are involved: The step 6 is specifically as follows: The three layers of the entity similarity calculation model are the pre-trained language model BERT, the fully connected neural network (DNN) and Softmax, which build an "entity pair" binary classification model; the model inputs the "entity pair" (n1, n2) and constructs it into a standard input sequence through character processing: [CLS]Q1...Qn[SEP]R1...Rm[SEP] [CLS] and [SEP] are two special symbol markers of BERT, which are used to distinguish different entities; [CLS] appears at the beginning of the entity pair, and [SEP] appears at the boundary between two entities and at the end of the entity pair; the model inputs the vector of the last layer of BERT into DNN (fully connected neural network) for dimensionality reduction and feature extraction, and then performs binary classification through Softmax to obtain the similarity probability distribution result. The calculation formula is as follows: Among them, i represents a category in k, k∈(0,2],g i Represents the value of the classification; the loss function of the model uses the cross entropy loss function, which is as follows: where y i Indicates the label of sample i, the positive class is 1, the negative class is 0, p i represents the probability that sample i is predicted to be positive; finally, by setting a threshold in the prediction stage, similar entity pairs are calculated; the entities extracted in step 4 are compared with the time attributes (t i (p t )) and confidence attribute (t j (p c )) for comparison; update entities with newer time and higher confidence to the government knowledge graph.
