Underground space knowledge graph construction method based on Chinese entity relationship extraction

Abstract

The invention discloses an underground space knowledge graph construction method based on Chinese entity relationship extraction, which can efficiently process and analyze a large amount of unstructured underground space field text information, extract a relationship triple and clearly reveal the association between entities so as to construct an underground space knowledge graph. An initial representation is generated for each mark in the text through a Bert pre-training model, and key context features are captured. Next, extracting a head entity, a tail entity and a relationship in a cascade mode, constructing a Bi-CasRTE condition cascade network, accurately determining a relationship triple, and meanwhile, introducing an improved prediction depolarization strategy through a reconstruction depolarization strategy to reduce deviation due to the fact that data types obtained in a cross-scene mode are unbalanced; the method not only solves the problem of obtaining the Chinese text information in the underground space field, but also improves the information processing efficiency and accuracy, provides powerful support for deep exploration and application in the underground space field, and lays a solid foundation for development and utilization of underground space information in the future.

Description

Technical Field

[0001] The present invention belongs to the technical field of knowledge graphs, and in particular, relates to a method for constructing an underground space knowledge graph based on Chinese entity relationship extraction. Background Art

[0002] Relation extraction is the core task of building a knowledge graph. It aims to identify and extract semantic relationships between entities from unstructured texts, and express them in the form of triples (head entity, relationship, tail entity) to clearly reflect the specific connections between entities. Early methods mainly relied on manually constructed grammatical and semantic rules. Although they could achieve certain results, they were time-consuming and labor-intensive, and difficult to adapt to diverse and complex data. With the development of deep learning, technologies based on deep learning have gradually become mainstream, greatly improving the level of automation of relationship extraction.

[0003] At present, relation extraction models are mainly divided into two categories: pipeline learning-based methods and joint learning methods. The pipeline learning method first identifies entities and then classifies relations between entities. Although the process is clear, the information interaction is weak. The joint learning method simultaneously performs entity recognition and relation classification, which can share underlying features and representations, and effectively alleviate the impact of data sparsity and labeling errors on the model. Zheng et al. proposed a joint labeling model, which unified the modeling of entity and relation extraction tasks through a novel labeling scheme; Peng et al. proposed a joint learning method based on graph convolutional neural network, which avoided the error propagation problem in the pipeline method by simultaneously learning the interaction between multiple subtasks; Bekoulis et al. modeled the entity recognition and relation extraction problems as multi-head selection problems and learned them in a joint manner. Compared with the traditional pipeline method of first identifying entities and then identifying relations, this method effectively improved the accuracy of relation extraction. Literature research shows that the joint learning method is superior to the pipeline method in generalization ability and robustness, but there are certain challenges in model complexity and computational cost.

[0004] However, traditional methods have great limitations in extracting useful text information from a variety of complex descriptive information in the field of underground space. The relationships between entities are complicated, which greatly increases the difficulty of obtaining text information in the field of underground space. In addition to public underground space planning reports, there is little other relevant text information in the existing literature. Underground space information plays a vital role in disaster response, urban management, national defense and security, and how to effectively extract knowledge has become a challenge. Summary of the invention

[0005] The present invention proposes a method for constructing an underground space knowledge graph based on Chinese entity relationship extraction. By constructing a Bi-CasRTE conditional cascade network, relationship extraction is performed on Chinese text information in underground space to construct an underground space knowledge graph, which solves the current shortcoming that Chinese text information in underground space is redundant and difficult to obtain.

[0006] The technical solution to realize the present invention is: a method for constructing an underground space knowledge graph based on Chinese entity relationship extraction, comprising the following steps:

[0007] Step 1: Collect structured and unstructured Chinese text data of underground space. The Chinese text data is sentences. Preprocess the Chinese text data to obtain a text data set and proceed to step 2.

[0008] Step 2: Build the Bi-CasRTE conditional cascade network:

[0009] The Bi-CasRTE conditional cascade network includes a Bert pre-training model, a Subject tagging module, an Object tagging module, and a dual affine relationship extraction module, and then proceeds to step 3.

[0010] Step 3: Input the sentences in the text dataset into the Bert pre-training model one by one to generate the initial representation vector h of the i-th entity pair token i , go to step 4.

[0011] Step 4: Generate the initial representation vector h i Input to the Subject tagging module, obtain whether each token is the start or end position of the head entity through the binary tag sequence, and go to step 5.

[0012] Step 5: Input the start and end positions of the head entity as prior conditions into the Object tagging module, extract all tail entities corresponding to the head entity, generate all Subject-Object pairs, and proceed to step 6.

[0013] Step 6: Input the Subject-Object pair into the dual affine relationship extraction module, combine the shared encoding features of the head and tail entities, obtain the preliminary relationship prediction between the two entities, and use the reconstruction debiasing strategy to introduce the improved prediction distribution. After reweighting through TL loss training, the final relationship prediction result between the head and tail entities is obtained, and then go to step 7.

[0014] Step 7: Store the final relationship prediction results between the head and tail entities in the form of triples in the graph database neo4j to construct an underground space knowledge graph.

[0015] Compared with the prior art, the present invention has the following significant advantages:

[0016] (1) The present invention extracts relationships from Chinese text information in underground space to construct a knowledge graph. By constructing a conditional cascade network, the present invention extracts relationships through a Bert pre-training model, a Subject tagging module, an Object tagging module, and a dual affine relationship extraction module, thereby obtaining the relationships between text information in underground space and constructing a knowledge graph for underground space, thereby making up for the shortcomings of underground space information being difficult to obtain and summarize.

[0017] (2) This paper proposes a reconstruction debiasing strategy for the first time and introduces an improved prediction distribution The original prediction distribution is combined with the prediction distribution of the minority class to form a new prediction distribution, which solves the problem of imbalanced class distribution.

[0018] (3) The present invention introduces the relationship extraction module into the Biaffine classifier and combines the shared coding features of the head and tail entities to obtain the preliminary relationship prediction between the two entities, which can better complete the task of relationship extraction for problems such as redundant text information in underground space.

[0019] (4) The present invention proposes a cross entropy loss function TL based on threshold T1 to solve the problem of unbalanced number of categories in triplets of different underground space information relationships. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a partial entity relationship diagram of the underground space field of the present invention.

[0021] Figure 2 This is a flow chart of the method for constructing an underground space knowledge graph based on Chinese entity relationship extraction of the present invention.

[0022] Figure 3 This is the Bi-CasRTE conditional cascade network diagram of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0025] The following will further introduce the specific implementation method, as well as the technical difficulties and inventive points of this invention in combination with this design example.

[0026] Combination Figure 2 and Figure 3 The method for constructing an underground space knowledge graph based on Chinese entity relationship extraction described in the present invention comprises the following specific steps:

[0027] Step 1: Collect structured and unstructured Chinese text data in four different underground space environments: urban subways, underground shopping malls, civil defense projects, and integrated pipe corridors. The Chinese text data is sentences. Preprocess the Chinese text data to obtain a text data set and proceed to step 2.

[0028] Step 2: Build the Bi-CasRTE conditional cascade network:

[0029] The Bi-CasRTE conditional cascade network includes a Bert pre-training model, a Subject tagging module, an Object tagging module, and a bi-affine relationship extraction module.

[0030] Among them, the dual affine relationship extraction module is as follows Figure 3 As shown, the Biaffine classifier is introduced and the process goes to step 3.

[0031] Step 3: Input the sentences in the text dataset into the Bert pre-training model one by one to generate the initial representation vector h of the i-th entity pair token i , the initial representation vector h of the i-th token of the contextual features of the head entity, tail entity, and relationship i , as follows:

[0032]

[0033]

[0034]

[0035] Among them, the training weight Bias represents the feature representation of the i-th token in the head entity, Indicates the feature representation of the i-th token in the tail entity, represents the feature representation of the i-th token in the relationship, W sub , W obj , W rel is the trainable weight matrix of the model, corresponding to the weighted mapping of the head entity, tail entity and relationship features, each dimension is d h ×d h , and dh is the dimension of the hidden layer, b sub 、b obj 、b rel They are the bias terms for the head entity, tail entity and relationship feature representation respectively.

[0036] Go to step 4.

[0037] Step 4: Generate the initial representation vector h i Input to the Subject tag module, and obtain each token through the binary tag sequence to determine whether it is the start or end position of the head entity, as follows:

[0038]

[0039]

[0040] in, represents the probability that the i-th token is the start tag of the head entity, represents the probability that the i-th token is the end tag of the head entity, σ represents the sigmoid activation function, which converts the output value into a probability between 0 and 1. is the trainable weight matrix of the model, which is used to map the head entity feature representation to the annotation dimension of "whether it is the beginning or the end". They are the offsets for the start and end of the head entity respectively.

[0041] Go to step 5.

[0042] Step 5: Input the start and end positions of the head entity as prior conditions into the Object tagging module, extract all tail entities corresponding to the head entity, and generate all Subject-Object pairs for all relations of the input Subject-Object pairs, as follows:

[0043]

[0044]

[0045] in, is the vector representation of the k-th head entity, is the vector representation of the j-th tail entity, is a probability distribution representing the relationship between the kth head entity and the jth tail entity pair, |R| is the relationship set The quantity of represents the starting and ending positions of the two input entities, LOC represents the set of word positions of the input entity pair, and |LOC| is the number of tokens where the entity pair is located, σ represents the sigmoid activation function, and W r Represented as a trainable weight matrix, represents the relational context feature representation of the i-th token, ° represents the Hadamard product, that is, the element-by-element multiplication operation, Represents the bias vector for relationship prediction.

[0046] Go to step 6.

[0047] Step 6: Input the Subject-Object pair into the dual affine relationship extraction module, combine the shared encoding features of the head and tail entities, obtain the preliminary relationship prediction between the two entities, and re-weight the final relationship prediction result between the head and tail entities by reconstructing the debiasing strategy and introducing the improved prediction distribution. The results are as follows:

[0048] The Subject-Object pair is input into the biaffine relation extraction module, and the biaffine relation extraction module is introduced into the Biaffine classifier as follows:

[0049]

[0050]

[0051]

[0052] Among them, the score matrix is ​​obtained using the Biaffine classifier l is the length of the sentence, c is the number of entities, U m is a biaffine matrix, W m is a linear transformation matrix, d represents the dimension of the feature vector, c represents the number of entity types, and h s (i) h e (i) represents the feature vectors of the Subject and Object of the i-th token respectively. The inputs are FFNN s Represents the feedforward neural network used for Subject feature extraction, FFNN e represents a feedforward neural network for object feature extraction; ⊕ represents a concatenation operation, b m is the bias vector, and then combined with the shared encoding features of the head and tail entities, the preliminary relationship prediction between the two entities is obtained, and T represents transposition.

[0053] This paper proposes for the first time to introduce an improved prediction distribution by reconstructing the debiasing strategy The problem of uneven distribution of data categories in different underground scenes is balanced in a novel way. The distribution takes into account the original prediction distribution y and the prediction distribution for the minority class. Train a classifier specifically for the minority class and adjust the loss function by increasing the weight of the minority class samples to obtain Combine the original prediction distribution with the prediction distribution of the minority class to form a new prediction distribution as follows:

[0054]

[0055] Among them, α and β are weights used to balance the original prediction distribution and the minority class prediction distribution; the optimal values ​​of the weights are determined by cross-validation or hyperparameter optimization methods.

[0056] In calculation When training, consider the class imbalance problem by oversampling the minority class or using synthetic data to increase its representation during training.

[0057] Then, after TL loss training and reweighting, the final relationship prediction result between the head and tail entities is obtained, as follows:

[0058] There are many types of underground space, including urban subway, underground shopping mall, civil defense project and integrated pipe gallery. Figure 1 As shown in the figure, the triples describing different underground space information relationships will have the problem of imbalanced number of categories. A threshold-based cross entropy loss function TL is proposed. A threshold T1 is defined, which is used to distinguish samples of imbalanced classification, as follows:

[0059]

[0060]

[0061] Where p is the probability that the sample belongs to the positive class, γ is a focusing parameter used to adjust the weight of difficult and easy samples; when γ is 0, TL degenerates into the standard cross entropy loss, and when γ increases, the model will pay more attention to difficult-to-classify samples; λ(p,T1) is a function based on the predicted probability p and the threshold T1, which is used to adjust the weight of the sample. When p>T1, λ(p,T1) should return a smaller value λ min , when p≤T1, returns a larger value λ max .

[0062] Go to step 7.

[0063] Step 7: Generate a triple structure of "entity 1-relationship-entity 2" from the final relationship prediction result between the head and tail entities, and store it in the graph database neo4j in the form of triples. Then, connect the two entities through the relationship, and mark their relationship in the graph for visualization to construct an underground space knowledge graph.

Claims

1. A method for constructing an underground space knowledge graph based on Chinese entity relationship extraction, characterized in that: The following steps are involved: Step 1: Collect structured and unstructured Chinese text data of underground space. The Chinese text data is sentences. Preprocess the Chinese text data to obtain a text data set, and then proceed to step 2. Step 2: Build the Bi-CasRTE conditional cascade network: The Bi-CasRTE conditional cascade network includes a Bert pre-training model, a Subject tagging module, an Object tagging module, and a bi-affine relationship extraction module, and then proceeds to step 3; Step 3: Input the sentences in the text dataset into the Bert pre-training model one by one to generate the initial representation vector h of the i-th entity pair token i , go to step 4; Step 4: Generate the initial representation vector h i Input to the Subject tag module, obtain each token through the binary tag sequence to see whether it is the start or end position of the head entity, and then go to step 5; Step 5: Input the start and end positions of the head entity as prior conditions into the Object tagging module, extract all tail entities corresponding to the head entity, generate all Subject-Object pairs, and proceed to step 6; Step 6: Input the Subject-Object pair into the dual affine relationship extraction module, combine the shared encoding features of the head and tail entities, obtain the preliminary relationship prediction between the two entities, and re-weight the final relationship prediction result between the head and tail entities by reconstructing the debiasing strategy and introducing the improved prediction distribution. Then, the TL loss training is performed and the weighting is performed. Then, the module goes to step 7. Step 7: Store the final relationship prediction results between the head and tail entities in the form of triples in the graph database neo4j to construct an underground space knowledge graph.

2. The method for constructing an underground space knowledge graph based on Chinese entity relationship extraction according to claim 1, characterized in that: In step 1, structured and unstructured Chinese text data of underground space are collected and cleaned, including eliminating noise, removing duplicate content, and ensuring the accuracy and consistency of the data. Then, the Chinese character sequence is converted into a token sequence using Bert's built-in word segmenter, and the cleaned data is annotated and formatted to obtain a text dataset.

3. The method for constructing an underground space knowledge graph based on Chinese entity relationship extraction according to claim 2 is characterized in that: In step 3, the sentences in the text dataset are input into the Bert pre-trained model one by one to generate the initial representation vector h of the i-th token i , the initial representation vector h of the i-th token of the contextual features of the head entity, tail entity, and relationship i , as follows: Among them, the training weight Bias Indicates the feature representation of the i-th token in the head entity, Indicates the feature representation of the i-th token in the tail entity, represents the feature representation of the i-th token in the relationship, W sub , W obj , W rel is the trainable weight matrix of the model, corresponding to the weighted mapping of the head entity, tail entity and relationship features, each dimension is d h ×d h , and d h is the dimension of the hidden layer, b sub 、b obj 、b rel They are the bias terms for the head entity, tail entity and relationship feature representation respectively.

4. The method for constructing an underground space knowledge graph based on Chinese entity relationship extraction according to claim 3 is characterized in that: In step 4, the generated initial representation vector h i Input to the Subject tagging module, take each token in the sentence as input, and obtain whether each token is the start or end position of the head entity through the binary tag sequence, as follows: in, represents the probability that the i-th token is the start tag of the head entity, represents the probability that the i-th token is the end tag of the head entity, σ represents the sigmoid activation function, which converts the output value into a probability between 0 and 1. Indicates the feature representation of the i-th token in the head entity, is the trainable weight matrix of the model, which is used to map the head entity feature representation to the annotation dimension of "whether it is the beginning or the end". They are the offsets for the start and end of the head entity respectively.

5. The method for constructing an underground space knowledge graph based on Chinese entity relationship extraction according to claim 4 is characterized in that: In step 5, the starting position of the head entity and end position As a priori condition, the Object tagging module is input to extract all tail entities corresponding to the head entity and generate all Subject-Object pairs for all relations of the input Subject-Object pair, as follows: in, is the vector representation of the k-th head entity, is the vector representation of the j-th tail entity, is a probability distribution representing the relationship between the kth head entity and the jth tail entity pair, |R| is the relationship set The quantity of represents the starting and ending positions of the two input entities, LOC represents the set of word positions of the input entity pair, and |LOC| is the number of tokens where the entity pair is located, σ represents the sigmoid activation function, and W r Represented as a trainable weight matrix, represents the relational context feature representation of the i-th token, ° represents the Hadamard product, that is, the element-by-element multiplication operation, Represents the bias vector for relationship prediction.

6. The method for constructing an underground space knowledge graph based on Chinese entity relationship extraction according to claim 5 is characterized in that: In step 6, the Subject-Object pair is input into the dual affine relationship extraction module, and the shared encoding features of the head and tail entities are combined to obtain the preliminary relationship prediction between the two entities. The improved prediction distribution is introduced through reconstruction debiasing strategy, and the TL loss training is performed for reweighting to obtain the final relationship prediction result between the head and tail entities, as follows: The Subject-Object pair is input into the biaffine relation extraction module, and the biaffine relation extraction module is introduced into the Biaffine classifier as follows: Among them, the score matrix is ​​obtained using the Biaffine classifier l is the length of the sentence, c is the number of entity types, U m is a biaffine matrix, W m is a linear transformation matrix, d represents the dimension of the eigenvector, h s (i) h e (i) represents the feature vectors of the Subject and Object of the i-th token respectively. The inputs are FFNN s Represents the feedforward neural network used for Subject feature extraction, FFNN e Represents a feedforward neural network for object feature extraction; represents the splicing operation, b m is the bias vector, and then combined with the shared encoding features of the head and tail entities, the preliminary relationship prediction between the two entities is obtained, and T represents transposition; Introducing an improved prediction distribution by reconstructing the debiasing strategy The distribution takes into account the original prediction distribution y and the prediction distribution for the minority class Train a classifier specifically for the minority class and adjust the loss function by increasing the weight of the minority class samples to obtain Combine the original prediction distribution with the prediction distribution of the minority class to form a new prediction distribution as follows: Among them, α and β are weights used to balance the original prediction distribution and the minority class prediction distribution; the optimal value of the weight is determined by cross-validation or hyperparameter optimization method; In calculation When training, consider the class imbalance problem by oversampling the minority class or using synthetic data to increase its representation during training; Then, after TL loss training and reweighting, the final relationship prediction result between the head and tail entities is obtained, as follows: There are many types of underground spaces. The triples describing the information relationships of different underground spaces will have the problem of imbalanced number of categories. A threshold-based cross entropy loss function TL is proposed. A threshold T1 is defined, which is used to distinguish samples of imbalanced classification, as follows: Where p is the probability that the sample belongs to the positive class, γ is a focusing parameter used to adjust the weight of difficult and easy samples; when γ is 0, TL degenerates into the standard cross entropy loss, and when γ increases, the model will pay more attention to difficult-to-classify samples; λ(p,T1) is a function based on the predicted probability p and the threshold T1, which is used to adjust the weight of the sample. When p>T1, λ(p,T1) should return a smaller value λ min , when p≤T1, returns a larger value λ max .

7. The method for constructing an underground space knowledge graph based on Chinese entity relationship extraction according to claim 6 is characterized in that: In step 7, the final relationship prediction result between the head and tail entities is generated into a triple structure of "entity 1-relationship-entity 2", which is stored in the graph database neo4j in the form of triples. The two entities are then connected through a relationship, and their relationship is marked in the graph for visual display to construct an underground space knowledge graph.

Images