Bridge health care knowledge graph construction method based on deep learning

Abstract

The application discloses a bridge health care knowledge graph construction method based on deep learning, which firstly marks entities and relations in the text in the bridge management and maintenance field to form a knowledge extraction corpus in the field; secondly, an information extraction model based on a pre-training language model is proposed, an embedding layer of the model shares an ALBERT pre-training language model to perform knowledge representation on field text description, a named entity recognition module adopts a Transform to extract semantic features of word embedding, a relation extraction module adopts a BiGCN to extract features, CRF and Softmax are respectively used for label prediction of named entities and relations between entities, and field annotated corpus is input for model training; then, the trained knowledge extraction model is used for named entity recognition and relation extraction in the field; and finally, a bridge health care knowledge graph is constructed according to the extracted entities and relations. The application is applied to the field knowledge graph construction process, realizes semi-automatic information extraction of bridge health care knowledge, and reduces the artificial construction cost.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing, deep learning, and big data knowledge engineering, specifically to a method for constructing a knowledge graph for bridge health and wellness based on deep learning. Background Technology

[0002] Knowledge graphs are a crucial method for organizing and representing knowledge in the era of big data. They are a key technology for achieving cognitive intelligence and play a vital role in areas such as semantic search, intelligent question answering, personalized recommendation, and intelligence analysis. Knowledge graphs use network structures to describe concepts, entities, and their relationships in the real world. Based on knowledge graphs, the continuous accumulation, integration, and expansion of professional domain knowledge can be achieved, as well as the selective extraction of key knowledge elements required for engineering projects, the integration of semantic and pragmatic information from engineering knowledge, and the provision of a foundation for the intelligent application and visualization of knowledge.

[0003] The semantic search, intelligent question answering, and personalized recommendation functions of knowledge graphs have been widely applied in fields such as medicine, social networking, film and television, humanities, and economics, and have become a research hotspot in the knowledge service field. As the application of knowledge graphs in various fields becomes increasingly widespread, many scholars have begun to introduce them into the field of civil engineering and have conducted multifaceted research. However, problems remain, such as insufficient research depth and a narrow research scope. For example, the bridge types studied are relatively limited, the completeness of the knowledge base still needs to be verified, and there are still relatively few case studies of knowledge graph research in the field of civil engineering. Knowledge graphs developed for a specific bridge type cannot be applied to other types of bridges in practical applications.

[0004] As knowledge graphs are increasingly applied across various fields, they are demonstrating their immense value. Knowledge graphs can provide a "smart core" for bridge and health care platforms, enabling truly intelligent, scientific, and accurate decision-making. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the actual problem that this invention needs to solve is: how to effectively extract valuable entities and relationships between entities from texts in the field of bridge health and wellness, so as to realize the semi-automatic construction of knowledge graphs in this field.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A method for constructing a knowledge graph for bridge-related health and wellness based on deep learning, comprising:

[0008] S1. Annotate entities and relationships in texts related to bridge maintenance to form a knowledge extraction corpus for this field;

[0009] S2. Construct a knowledge extraction model for the bridge health and wellness field, and input the domain-annotated corpus for model training;

[0010] S3. Use the trained knowledge extraction model to perform named entity recognition and relation extraction in this domain;

[0011] S4. Construct a knowledge graph of bridge health and wellness based on the extracted entities and relationships.

[0012] Preferably, the entity labeling strategy in step S1 is the BIO three-dimensional sequence labeling method, where the labels "NAM, LOS, DIM, DDE, DMS" correspond to "disease name, disease location, disease detection method, disease severity, and disease maintenance suggestions," respectively. To improve the accuracy of relationship identification, the relationship labeling strategy not only labels the relationships between entities but also labels the position information of the start and end characters of the head and tail entities in the text.

[0013] Preferably, the bridge maintenance and care domain knowledge extraction model in step S2 includes two main modules: named entity recognition and relation extraction, and shares an ALBERT embedding layer. The bridge maintenance text is then processed. Input the ALBERT pre-trained language model to obtain its embedded representation output. in For bridge maintenance text C b A sequence of string vectors.

[0014] Preferably, in step S2, the named entity recognition module uses Transformer for semantic feature extraction and CRF for label prediction of named entities. This embeds text from the bridge health and wellness field into the representation. Directly inputting the Transformer feature extractor yields the outputs y learned by the Multi-Head Attention mechanism and the FeedForward Network, respectively. m and y f :

[0015]

[0016] y f =LayerNorm(y m +FFN(y m ))

[0017] After passing through an encoder-decoder module consisting of multiple multi-head attention mechanisms and a feedforward neural network, and using linear mapping and the softmax activation function, the final semantic feature extraction result of the bridge maintenance text is obtained.

[0018]

[0019] in, It is the output of the Transformer's Encoder-Decoder encoding and decoding.

[0020] Preferably, in step S2, the relationship extraction module uses a bidirectional graph convolutional neural network (BiGCN) to extract features and uses Softmax to predict the labels of relationships between entities. The entities in the bridge and elderly care field obtained in the named entity recognition stage are used as graph nodes. Based on the entity's category label, the predefined relationships at the concept layer are used as the edge set of the graph structure. Through forward GCN and backward GCN learning, the semantic feature extraction results of the relationships between bridge health and wellness entities are obtained. The calculation method is as follows:

[0021]

[0022]

[0023] in, This represents the hidden layer output of a single GCN. This represents the hidden layer output of the forward GCN. This indicates the hidden layer output of the backward GCN. (Pr) e (v,u) represents the probability that there is a relationship e between nodes v and u. w and b represent the weight parameters and biases to be learned during model training, respectively, and ReLU is an activation function.

[0024] In summary, compared with the prior art, the present invention has the following technical advantages:

[0025] (1) This invention uses the ALBERT pre-trained language as the embedding representation for the knowledge extraction model. Based on BERT, matrix factorization and parameter sharing are performed, thereby reducing the number of model parameters and effectively solving the problem of insufficient resources in specific domains. The ALBERT model replaces the NSP task with the SOP task, mainly focusing on inter-sentence coherence, which solves the problem of inefficient next-sentence prediction loss in BERT. Therefore, the ALBERT model is more suitable for knowledge embedding representation in the specific domain of bridge health care.

[0026] (2) By sharing the ALBERT model embedding layer, the computational resources for the two subtasks of named entity recognition and relation extraction to perform knowledge representation independently are further saved.

[0027] (3) Named entity recognition is a typical sequence labeling task based on token classification. Compared with RNN neural network for feature extraction, Transformer has a stronger feature extraction effect and is more suitable for model training with few samples in specific fields.

[0028] (4) A bidirectional graph convolutional network (BiGCN) is used for relation reasoning. The relation prediction results are used to aggregate entity nodes and update deep contextual features for joint extraction. Relation information is further added to entity recognition to improve the performance of joint extraction. Attached Figure Description

[0029] Figure 1 This invention discloses a method flowchart for constructing a bridge health and wellness knowledge graph based on deep learning.

[0030] Figure 2 Example of named entity annotation in the field of bridge health and wellness;

[0031] Figure 3 This is a schematic diagram illustrating the principle of the information extraction model in an implementation of a deep learning-based method for constructing a knowledge graph for bridge health and wellness disclosed in this invention.

[0032] Figure 4 This is an example of a knowledge graph in the field of bridge health and wellness. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings.

[0034] like Figure 1 As shown, this invention discloses a method for constructing a bridge health and wellness knowledge graph based on deep learning, including:

[0035] S1. Annotate entities and relationships in texts related to bridge maintenance to form a knowledge extraction corpus for this field;

[0036] like Figure 2 As shown, in specific implementation, the entity labeling strategy in step S1 is the BIO three-dimensional sequence labeling method, and the labels "NAM, LOS, DIM, DDE, DMS" correspond to "disease name, disease location, disease detection method, disease severity, and disease maintenance suggestions" respectively; in order to improve the accuracy of relationship identification, the relationship labeling strategy not only labels the relationship between entities, but also labels the position information of the start and end characters of the head entity and tail entity in the text.

[0037] S2. Construct a knowledge extraction model for the bridge health and wellness field, and input the domain-annotated corpus for model training;

[0038] like Figure 3As shown, in specific implementation, the bridge maintenance and care domain knowledge extraction model in step S2 includes two main modules: named entity recognition and relation extraction, and shares the ALBERT embedding layer. The bridge maintenance text is then processed. Input the ALBERT pre-trained language model to obtain its embedded representation output. in For bridge maintenance text C b A sequence of string vectors.

[0039] like Figure 3 As shown, in specific implementation, the named entity recognition module in step S2 uses Transformer for semantic feature extraction and CRF for label prediction of named entities. This embeds text from the bridge health and wellness field into the representation. Directly inputting the data into the Transformer feature extractor yields the outputs y learned by the Multi-Head Attention mechanism and the Feed Forward Network, respectively. m and y f :

[0040]

[0041] y f =LayerNorm(y m +FFN(y m ))

[0042] After passing through an encoder-decoder module consisting of multiple multi-head attention mechanisms and a feedforward neural network, and using linear mapping and the softmax activation function, the final semantic feature extraction result of the bridge maintenance text is obtained.

[0043]

[0044] in, It is the output of the Transformer's Encoder-Decoder encoding and decoding.

[0045] like Figure 3 As shown, in specific implementation, the relationship extraction module in step S2 uses a bidirectional graph convolutional neural network (BiGCN) to extract features and uses Softmax to predict the labels of relationships between entities. The entities in the bridge and elderly care field obtained in the named entity recognition stage are used as graph nodes. Based on the entity's category label, the predefined relationships at the concept layer are used as the edge set of the graph structure. Through forward GCN and backward GCN learning, the semantic feature extraction results of the relationships between bridge health and wellness entities are obtained. The calculation method is as follows:

[0046]

[0047]

[0048] in, This represents the hidden layer output of a single GCN. This represents the hidden layer output of the forward GCN. This indicates the hidden layer output of the backward GCN. (Pr) e (v,u) represents the probability that there is a relationship e between nodes v and u. w and b represent the weight parameters and biases to be learned during model training, respectively, and ReLU is an activation function.

[0049] S3. Use the trained knowledge extraction model to perform named entity recognition and relation extraction in this domain;

[0050] S4. Construct a knowledge graph of bridge health and wellness based on the extracted entities and relationships.

[0051] like Figure 4 As shown, the extracted named entities in the bridge health and wellness field are used as nodes, the relationships between entities are used as edges, and Neo4j is used as the storage carrier for structured domain knowledge, thereby completing the construction of a knowledge graph for the bridge health and wellness field.

[0052] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A bridge health and wellness knowledge graph construction method based on deep learning, characterized in that, Comprise: S1, the entity and relationship in the bridge maintenance field text are marked, and the field knowledge extraction corpus is formed; S2, the bridge health care field knowledge extraction model is constructed, and the field marked corpus is input for model training; The bridge health care field knowledge extraction model in S2 includes two main modules of named entity recognition and relation extraction, and shares an ALBERT embedding layer, and the bridge health care text is input into an ALBERT pre-training language model to obtain its embedding representation output wherein is a string vector sequence of the bridge health care text C b . The named entity recognition module adopts a Transformer for semantic feature extraction, uses CRF for label prediction of named entities, and embeds text in the bridge health care field Directly input the Transformer feature extractor to obtain the output y of the multi-head attention mechanism Multi-Head Attention and the feed forward neural network Feed Forward Network after learning, respectively m And y f : y f = LayerNorm(y m + FFN(y m )) The final bridge maintenance text semantic feature extraction result is obtained through an encoding-decoding module composed of multiple multi-head attention mechanisms and feedforward neural networks, Linear linear mapping and Softmax activation function wherein, is the output of the Encoder-Decoder encoding-decoding of the Transformer; The relationship extraction module extracts features by using a bidirectional graph convolutional neural network Bi GCN, and uses Softmax to predict the labels of the relationships between entities, and the bridge health care field entities obtained in the named entity recognition stage are used as graph nodes According to the category labels of the entities, the pre-defined relationships in the concept layer are used as the edge set of the graph structure Through the learning of the forward GCN and the backward GCN, the semantic feature extraction result of the relationships between the bridge health care entities is obtained The calculation method is: wherein, represents the hidden layer output of a single GCN, represents the hidden layer output of a forward GCN, represents the hidden layer output of a backward GCN, Pr e (v, u) represents the probability that node v and node u have a relationship e, w and b represent the weight parameters and bias to be learned during model training, respectively, and Relu is an activation function. S3, the trained knowledge extraction model is used for the field named entity recognition and relationship extraction; S4, the bridge health care knowledge graph is constructed according to the extracted entity and relationship.

2. The bridge health and wellness knowledge graph construction method based on deep learning according to claim 1, characterized in that, The entity marking strategy in S1 is BIO three-bit sequence marking method, and the labels "NAM, LOS, DIM, DDE, DMS" correspond to "disease name, disease location, disease detection method, disease degree and disease maintenance suggestion" respectively; The relationship marking strategy marks the position information of the start character and end character of the head entity and tail entity in the text while marking the relationship between entities.

Images