Chemical device question and answer method and system fused with knowledge graph

Abstract

The embodiment of the invention provides a chemical device question answering method and system fused with a knowledge graph, and belongs to the technical field of semantic recognition. The method comprises the following steps: collecting knowledge information of a chemical device, and performing vector representation on words in the knowledge information; performing feature extraction on the knowledge information after vector representation based on a pre-trained entity relationship joint extraction model, and constructing a knowledge graph based on extracted features; performing entity linking by taking the constructed knowledge graph as a knowledge base; and recovering question text information of the user, performing answer matching based on the knowledge base, and feeding back a matched answer to the user side. According to the scheme, the question-answering system is constructed based on the knowledge graph, users can answer questions and answers conveniently, and the knowledge utilization rate is increased.

Description

Technical Field

[0001] The present invention relates to the field of semantic recognition technology, and in particular to a chemical plant question-answering method integrating a knowledge graph and a chemical plant question-answering system integrating a knowledge graph. Background Art

[0002] Chemical plants are one of the most important plants in the oil refining process. The complex chemical reactions and high temperature and high pressure environment of the plants are prone to accidents and safety risks. It is of great significance to establish a safety risk knowledge base for chemical plants and to effectively extract and manage knowledge.

[0003] At present, most of the safety risk knowledge bases in the existing technology are based on manual construction, which has the problems of high construction difficulty and high management and maintenance costs; at the same time, the data form in chemical plants is also relatively complex, including production time series data, text, images, videos and other unstructured data, which poses challenges to knowledge extraction and knowledge graph construction; the existing knowledge extraction methods are to separate entity recognition and relationship extraction, which can easily cause error propagation. In addition, for the search and consultation of chemical plant related issues, if it is not a professional such as a consulting expert, it will rely more on Internet search engines to find relevant knowledge, which is inefficient and inaccurate. In view of the low efficiency and inaccuracy of the existing chemical plant question and answer solutions, it is necessary to propose a new chemical plant question and answer solution. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a chemical plant question-answering method and system that integrates a knowledge graph, so as to at least solve the problems of low efficiency and inaccuracy in existing chemical plant question-answering solutions.

[0005] In order to achieve the above-mentioned objectives, the first aspect of the present invention provides a chemical plant question-answering method integrating a knowledge graph, the method comprising: collecting chemical plant knowledge information, and performing vector representation on the words in the knowledge information; based on a pre-trained entity-relationship joint extraction model, performing feature extraction on the knowledge information after vector representation, and constructing a knowledge graph based on the extracted features; using the constructed knowledge graph as a knowledge base for entity linking; recovering the user's question text information, and performing answer matching based on the knowledge base, and feeding back the matching answers to the user end.

[0006] Optionally, the chemical plant knowledge information includes: structured, semi-structured and unstructured data extracted from an open source chemical plant indication library.

[0007] Optionally, before performing vector representation on the words in the knowledge information, the scheme also includes: identifying the data type and selecting a corresponding preprocessing scheme based on the identified data type; performing preprocessing on the knowledge information based on the selected preprocessing scheme; including: performing text annotation on unstructured data; and combining the unstructured data after text annotation and the structured data into training samples.

[0008] Optionally, the performing vector representation on the words in the knowledge information includes: training the knowledge information based on a Bert model or an Ernie model to obtain words represented by vectors.

[0009] Optionally, the pre-trained entity-relationship joint extraction model performs feature extraction on the knowledge information after vector representation, including: performing BIO annotation on the knowledge information after vector representation to obtain an annotation sequence; inputting the vector of each word into the Bi-LSTM layer; wherein the Bi-LSTM layer includes two LSTM models, which are respectively for extracting forward hidden features and extracting reverse hidden features; in the Bi-LSTM layer, sentence features of the annotation sequence are extracted based on a merging process of the extracted forward hidden features and the extracted reverse hidden features; in the conditional random field layer, feature relationships between word vectors are extracted based on sentence features; in the annotation sequence, head entities and tail entities are identified, and a scoring function is executed on the head entity vector and the tail entity vector to extract attribute features; and feature extraction results are output based on sentence features, feature relationships and attribute features.

[0010] Optionally, the Bi-LSTM layer is expressed as:

[0011]

[0012]

[0013] in, To extract forward hidden features; To extract reverse hidden features; i t is the t-th word vector.

[0014] Optionally, the identifying the head entity and the tail entity in the annotation sequence includes: identifying triples in the annotation sequence, and obtaining the head entity and the tail entity by an embedding summation method;

[0015] The score function is:

[0016]

[0017] Among them, e1 is the head entity vector; e2 is the tail entity vector; r is the embedding vector generated based on the feature relationship.

[0018] Optionally, after entity linking is performed on the constructed knowledge graph as a knowledge base, the method further includes taking all entities extracted based on the knowledge information as an entity set; determining whether there is a situation in the connected knowledge base that does not contain the entity target in the entity set; if so, linking new entities based on the corresponding entity targets and adding them to the knowledge base.

[0019] Optionally, the method of recycling the user's question text information, matching answers based on the knowledge base, and feeding back the matching answers to the user end includes: extracting features from the question text information, embedding questions and entities to obtain their dense vector representations, and performing relevant matching in candidate knowledge graphs; embedding corresponding node relationships to obtain their dense vector representations, calculating the matching degree between the question and the candidate answers, and screening out the most matching answer based on the matching degree.

[0020] The second aspect of the present invention provides a chemical plant question-and-answer system that integrates a knowledge graph, the system comprising: a collection unit, used to collect chemical plant knowledge information and perform vector representation on words in the knowledge information; a graph construction unit, used to perform feature extraction on the knowledge information after vector representation based on a pre-trained entity-relationship joint extraction model, and construct a knowledge graph based on the extracted features; a linking unit, used to perform entity linking using the constructed knowledge graph as a knowledge base; and a feedback unit, used to recover the user's question text information, perform answer matching based on the knowledge base, and feed back the matching answer to the user end.

[0021] A third aspect of the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed on a computer, enables the computer to execute the above-mentioned chemical equipment question-and-answer method integrating a knowledge graph.

[0022] A fourth aspect of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned chemical equipment question-and-answer method integrating a knowledge graph when executing the computer program.

[0023] Through the above technical solution, the solution of the present invention provides a joint knowledge extraction method, which extracts entities and relationships at the same time. In the extraction process, the model uses a shared underlying feature method to represent the underlying features of entities and their contained relationships. By constructing a safety risk knowledge graph of a chemical plant, the automatic extraction, management and analysis of safety risk knowledge of chemical plants is realized; at the same time, a question-and-answer system is constructed based on the knowledge graph to facilitate users to ask questions and improve the utilization rate of knowledge.

[0024] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0026] Figure 1 It is a flowchart of the steps of a chemical plant question-answering method integrating a knowledge graph provided by an embodiment of the present invention;

[0027] Figure 2 is a schematic diagram of a knowledge information preprocessing process provided by an embodiment of the present invention;

[0028] Figure 3 It is a system structure diagram of a chemical plant question-and-answer system integrating a knowledge graph provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0029] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0030] Chemical plants are one of the most important plants in the oil refining process. The complex chemical reactions and high temperature and high pressure environment of the plants are prone to accidents and safety risks. It is of great significance to establish a safety risk knowledge base for chemical plants and to effectively extract and manage knowledge.

[0031] At present, most of the safety risk knowledge bases in the existing technology are based on manual construction, which has the problems of high construction difficulty and high management and maintenance costs. At the same time, the data form in chemical plants is also relatively complex, including production time series data, text, images, videos and other unstructured data, which poses challenges to knowledge extraction and knowledge graph construction. The existing knowledge extraction methods are to separate entity recognition and relationship extraction, which can easily cause error propagation. In addition, for the search and consultation of chemical plant related issues, if it is not a professional such as a consulting expert, it will rely more on Internet search engines to find relevant knowledge, which is inefficient and inaccurate.

[0032] In view of the above problems, the scheme of the present invention provides a chemical plant question-and-answer method that integrates a knowledge graph. It is used to solve the technical problem that the current unstructured data of chemical plants, such as emergency plans, process cards, accident cases, etc., are difficult to handle. In view of the fact that the current knowledge extraction methods all perform entity recognition and relationship extraction separately, which easily causes the problem of error propagation, the present invention provides a knowledge joint extraction method to extract entities and relationships at the same time. During the extraction process, the model uses a shared underlying feature method to represent the underlying features of entities and their contained relationships. By constructing a safety risk knowledge graph for chemical plants, the automatic extraction, management and analysis of safety risk knowledge of chemical plants can be achieved; at the same time, a question-and-answer system is constructed based on the knowledge graph to facilitate users to ask questions and improve the utilization rate of knowledge.

[0033] Figure 1 This is a method flow chart of a chemical plant question-answering method integrating a knowledge graph provided by an embodiment of the present invention. Figure 1 As shown, an embodiment of the present invention provides a chemical plant question-answering method integrating a knowledge graph, the method comprising:

[0034] Step S10: Collect chemical plant knowledge information and perform vector representation on the words in the knowledge information.

[0035] Specifically, the chemical plant knowledge information includes: structured, semi-structured and unstructured data extracted from the open source chemical plant indicator library. Figure 2 , including the following steps:

[0036] Step S11: Collect structured, semi-structured and unstructured data related to chemical plants.

[0037] Specifically, the collected data include semi-structured and unstructured data such as relevant technical specifications, operating documents, standard specifications, annual reports, emergency plans, accident cases, etc.; structured data such as the name, number, alias, and attributes of the device.

[0038] In the embodiment of the present invention, improving the accuracy of the knowledge graph helps to improve the accuracy of matching. To construct a knowledge graph, it is first necessary to extract knowledge from raw data including structured data, unstructured data and semi-structured data. The knowledge graph is constructed by forming triple relationships through named entity recognition, named entity linking and named entity relationship extraction.

[0039] Named Entity refers to an entity with a specific meaning or strong referentiality in the text, usually including names of people, organizations, proper nouns, etc. The Sixth Message Understanding Conference (MUC 6) and MUC 7 divided it into three categories and seven subcategories. Later, it was expanded to include geopolitics, facilities, transportation, weapons, ethnic and political groups, product names, artwork names, laws, languages, ordinal numbers, cardinal numbers, etc. For specific fields, such as biology: including proteins, ribonucleotides, deoxyribonucleotides and genes, etc. Chemistry: chemical substances and ingredients. Medicine: drug names and diseases. E-commerce: product name, place of origin, manufacturer.

[0040] Embodiment 1:

[0041] Take a company's catalytic cracking unit as an example. The unit is a high-low parallel riser catalytic cracking unit with a processing capacity of 160×104t / a. It is mainly composed of reaction regeneration, fractionation, absorption stabilization, air compressor unit, main air blower unit, flue gas energy recovery, desulfurization, etc. The unit uses hydrogenated wax oil as the main raw material and vacuum residue oil, deasphalted oil and atmospheric residue oil as auxiliary raw materials. The main products are dry gas, liquefied gas, gasoline, diesel and oil slurry.

[0042] The entities identified by the present invention include: devices, raw materials and products. Devices include: reaction regeneration devices, fractionation devices, gas compressor units, etc.; raw materials include hydrogenated wax oil, vacuum residue oil, deasphalted oil, etc.; products include: dry gas, liquefied gas, gasoline, diesel, etc. Traditional entity naming methods are generally divided into three categories: dictionary and rule-based methods, unsupervised methods, and supervised machine learning methods based on feature engineering. In natural language, there are polysemy, multiple words with one meaning, and aliases.

[0043] In the chemical industry, there are often multiple words with one meaning, such as 3# catalytic cracking unit, 3# unit, P03, etc.; there are also complex entity nesting, such as: ethylene glycol recovery tower top separator, which has four nestings: ethylene glycol, separator, ethylene glycol recovery tower and ethylene glycol recovery tower top separator, and their types are not consistent. Ethylene glycol is a material, and the other three entities are equipment. In addition, most entities are unique professional terms, such as rich gas and backwash gas in materials, Fischer-Tropsch reactor and rich liquid flash tank in equipment. In addition, some terms are very long, such as antifreeze dehydration tower top air cooler, release gas compressor reflux cooler and ethylene glycol recovery tower top separator. Finally, there are a large number of special terms, such as "run", "emerge", "drip" and "leak". Therefore, a wide variety of professional vocabulary and its composition are a major problem for information extraction.

[0044] Named entity linking is the process of linking named entity mentions in a text to unambiguous entities in a knowledge base. Traditional non-deep learning methods require a lot of manual work and have poor portability. Traditional named entity relationship extraction methods often require a lot of manual work and manually designed features or rules, which will cause error accumulation and propagation problems, affecting the performance of named entity extraction.

[0045] Since the deep learning-based method does not require the design of artificial features, the present invention uses a deep learning-based method to perform named entity recognition, named entity connection, and named entity relationship extraction. The deep learning-based named entity recognition method generally includes three steps: distributed representation of input, context encoding, and label decoding. The deep learning-based entity linking method mainly has three modules, namely, candidate entity generation (Candidate Entity Generation), candidate entity ranking (Candidate Entity Ranking), and unlinkable mention prediction.

[0046] Named entity relationships refer to semantic relationships between entities. The relationship between two entities is called a binary relationship, and the relationship between three or more entities is called a multi-relationship or a higher-order relationship. Binary relationships are the most basic and common, and multi-relationships can often be converted into binary relationships for processing.

[0047] Step S12: Use the pre-trained model to perform distributed representation on the words in the acquired data.

[0048] Specifically, before representing the data in text form, the data must be preprocessed and enhanced. The text information of unstructured data must be annotated according to the standard pattern of the information extraction model, and the annotated text is used as the training set, verification set, and test set. The data volume is expanded through artificial construction for data enhancement. The expansion methods include original text expansion, random truncation, same-sex word replacement, and pre-trained model output replacement.

[0049] Furthermore, the pre-trained model is used to represent words in a distributed manner. Consider using Bert and Ernie as pre-trained models for knowledge representation. It uses the encoder structure of a bidirectional transfomer and is pre-trained based on Chinese Wikipedia as the relevant corpus. Ernie is a model released by Baidu that is pre-trained based on corpus such as Baidu Tieba. It uses word-level MASK and can better recognize the semantics of Chinese.

[0050] Step S20: Based on the pre-trained entity-relationship joint extraction model, feature extraction is performed on the knowledge information represented by the vector, and a knowledge graph is constructed based on the extracted features.

[0051] Specifically, the knowledge information represented by the vector is BIO labeled to obtain a labeled sequence; the vector of each word is input into the Bi-LSTM layer; wherein the Bi-LSTM layer includes two LSTM models, which are respectively used for extracting forward hidden features and extracting reverse hidden features; the sentence features of the labeled sequence are extracted based on the merging processing of the extracted forward hidden features and the extracted reverse hidden features in the Bi-LSTM layer; the feature relationship between each word vector is extracted based on the sentence features in the conditional random field layer; the head entity and the tail entity are identified in the labeled sequence, and the head entity vector and the tail entity vector are executed with a score function to extract attribute features; the feature extraction result is output based on the sentence features, feature relationships and attribute features.

[0052] In the embodiment of the present invention, a joint entity and relationship extraction model based on transfer learning is used. Specifically, a feature-sharing neural network is used to jointly extract entity relationships, where Bi-LSTM and CRF are used to extract entity features and label each entity; a relationship extraction model based on transfer learning is used to handle complex relationships between different entity pairs. Finally, the py2neo library is used to automatically construct the obtained entities, relationships, and attributes in the neo4j database and store them in the cloud.

[0053] Furthermore, the BIO annotation of the knowledge information represented by the vector includes:

[0054] The segmented data is annotated with BIO, for example, three-digit annotation; B begin, I inside, O outside; BX represents the beginning of entity X, IX represents the end of entity, and O represents not belonging to any type.

[0055] Specifically, first import the labeled sequence into the embedding layer. For an input sequence X = (x1, x2, ... x n ), n represents the length of the sequence. For each word in the sentence, the joint extraction model inputs the embedding of each word into a Bi-LSTM, and then uses the output vector of the nth layer of the Bi-LSTM model as the overall feature of the word.

[0056] Embodiment 2:

[0057] The main function of the Bi-LSTM layer is to extract features from sentences. In the embedding layer, the extraction model can obtain the embedding of each word and use it as a sequence I = (i1, i2, ...i n). Bi-LSTM contains two LSTM models, one extracting forward hidden features Another method to extract reverse hidden features The Bi-LSTM layer can be represented as follows:

[0058]

[0059]

[0060] in, To extract forward hidden features; To extract reverse hidden features; i t is the tth word vector. Combining the forward and reverse hidden layer features can get the output Use an activation function to extract Then map it to a d l dimensional space. l is the number of entity types that need to be identified.

[0061] Furthermore, in the conditional random field layer, the feature relationship between each word vector is extracted based on the sentence features. For each feature output by Bi-LSTM, the model uses the conditional random field to identify the specific label yt at each specific position in the sequence. The conditional random field can better consider the dependency between different labels and thereby improve the accuracy of entity recognition. Specifically, the model merges the output l of the Bi-LSTM layer into a sequence L = [l1, l2, ... l n ] T , input into the CRF (conditional random field) model, where the size of L is s×d l .

[0062] Embodiment three:

[0063] Identify the head entity and the tail entity in the labeled sequence, and perform the score function on the head entity vector and the tail entity vector to extract attribute features, including: for the triples in the sequence, the joint extraction model obtains the embedding vector e1 of the head entity and the embedding vector e2 of the tail entity in the embedding layer by the embedding summation method, and generates the corresponding relationship embedding vector r according to their corresponding relationship r. In order to obtain the representation vectors of entities and relationships at the same time, first assume that e1+r and vector e2 are relatively close, specifically e1+r≈e2. Its score function can be expressed as follows:

[0064]

[0065] The model can also obtain the representation vector of the head and tail entities in the Bi-LSTM layer and And we hope that the entity representation vector in the Bi-LSTM layer also satisfies the constraint and Therefore, the score function in the Bi-LSTM layer can be expressed as follows:

[0066]

[0067]

[0068] In general, the loss function of the model can be regarded as a combination of the entity extraction loss function and the relationship extraction loss function:

[0069] L=L e +λL r

[0070] Where Le is the entity extraction loss function, Lr is the relationship extraction loss function, and λ is the weight hyperparameter to measure the weight of entity relationship extraction.

[0071] Step S30: Use the constructed knowledge graph as a knowledge base for entity linking.

[0072] Specifically, the artificially constructed knowledge graph is regarded as a knowledge base, which contains an entity set E; the entities extracted from the chemical plant related data in the present invention are considered to be potential associated objects of the entity set E, and the set is denoted as M; the task objective is to correspond each entity m∈M to an unambiguous target entity e∈E. If the set E does not contain the target entity of m in the actual calculation, m is linked to a new entity (as a supplement to the original entity library). In terms of specific model selection, the present invention adopts a model based on a graph convolutional network, which uses a graph convolutional network to model local context and global consistency information. An optimal allocation is found for the candidate entity through function calculation.

[0073]

[0074] in, is the variable that outputs the candidate entity, P() is the probability function, is the normalized adjacency matrix of the topological graph, and f is the feature representation of the candidate entity.

[0075]

[0076] Among them, F() is a mapping function under the parameter ω, which is obtained by using an encoder, a subgraph convolutional network and a decoder.

[0077] Step S40: Recover the user's question text information, perform answer matching based on the knowledge base, and feed back the matching answer to the user end.

[0078] Specifically, feature extraction is performed on the question text information, the question and entity are embedded to obtain their dense vector representation, and relevant matching is performed in the candidate knowledge graph; the corresponding node relationship is embedded to obtain its dense vector representation, and the matching degree between the question and the candidate answer is calculated, and the most matching answer is screened out based on the matching degree.

[0079] Embodiment 4:

[0080] Through user input questions, semantic questions are matched with entities and relationships in the chemical plant safety risk map, and answers are inferred. The pre-trained model is used to extract question features, and the single word vectors obtained are added to obtain the vector representation of the corresponding sentence; the answer entity is vectorized using the TransE series model; a multi-column convolutional network is used to extract three feature vectors as the three dimensions of the answer, namely the path of the answer, the contextual information of the answer, and the type of the answer. Through user input questions, the questions and entities are embedded to obtain their dense vector representation, and relevant matching is performed in the candidate knowledge graph. After obtaining the matching knowledge, the corresponding node relationship is embedded to obtain its dense vector representation, and the matching degree between the question and the candidate answer is calculated, and the most matching answer is selected through the model. Bert and other pre-trained models are used to extract question features. The single word vectors obtained are added to obtain the vector representation of the corresponding sentence. This method can effectively identify the part of speech and order. At the same time, in order to improve the performance of the model, additional information can be introduced.

[0081] Furthermore, for different answer features (entity, relationship, type, context), a score function S(q,a) is used to measure the degree of association between the question and the candidate answer; the distributed expression for answer i is:

[0082] g i (a)∈{g e (a), g r (a), g t (a), g c (a)}

[0083] g e (a) is the entity vector, g r (a) is the relationship vector, g t (a) is the type vector, g c (a) is the context vector. For the word vector of the jth word in question i (h output by Bert) j ), the distributed expression of the corresponding problem is recorded as:

[0084]

[0085] a ijis the attention weight of sentence i for word j. The final score function is the dot product of the question and the answer:

[0086]

[0087] In the embodiment of the present invention, the question-answering system of the present invention uses Python as the server-side development language, opens the Http request interface, and uses the semantic analysis model and knowledge graph model established in the previous article to finally achieve the basic question-answering effect. The client uses Java as the development language. When the user enters a question, the client packages the question into an Http message and then requests data from the server. After asynchronous echo, the answer to the question is output on the client.

[0088] Figure 3 : is a system structure diagram of a chemical plant question-answering system integrating a knowledge graph provided by an embodiment of the present invention. Figure 3 As shown, an embodiment of the present invention provides a chemical plant question-answering system integrating a knowledge graph, the system comprising:

[0089] The collection unit is used to collect chemical plant knowledge information and perform vector representation on the words in the knowledge information.

[0090] Specifically, the chemical plant knowledge information includes: structured, semi-structured and unstructured data extracted from the open source chemical plant indicator library. The following steps are included:

[0091] Step S11: Collect structured, semi-structured and unstructured data related to chemical plants.

[0092] Specifically, the collected data include semi-structured and unstructured data such as relevant technical specifications, operating documents, standard specifications, annual reports, emergency plans, accident cases, etc.; structured data such as the name, number, alias, and attributes of the device.

[0093] In the embodiment of the present invention, improving the accuracy of the knowledge graph helps to improve the accuracy of matching. To construct a knowledge graph, it is first necessary to extract knowledge from raw data including structured data, unstructured data and semi-structured data. The knowledge graph is constructed by forming triple relationships through named entity recognition, named entity linking and named entity relationship extraction.

[0094] Named Entity refers to an entity with a specific meaning or strong referentiality in the text, usually including names of people, organizations, proper nouns, etc. The Sixth Message Understanding Conference (MUC 6) and MUC 7 divided it into three categories and seven subcategories. Later, it was expanded to include geopolitics, facilities, transportation, weapons, ethnic and political groups, product names, artwork names, laws, languages, ordinal numbers, cardinal numbers, etc. For specific fields, such as biology: including proteins, ribonucleotides, deoxyribonucleotides and genes, etc. Chemistry: chemical substances and ingredients. Medicine: drug names and diseases. E-commerce: product name, place of origin, manufacturer.

[0095] Take a company's catalytic cracking unit as an example. The unit is a high-low parallel riser catalytic cracking unit with a processing capacity of 160×104t / a. It is mainly composed of reaction regeneration, fractionation, absorption stabilization, air compressor unit, main air blower unit, flue gas energy recovery, desulfurization, etc. The unit uses hydrogenated wax oil as the main raw material and vacuum residue oil, deasphalted oil and atmospheric residue oil as auxiliary raw materials. The main products are dry gas, liquefied gas, gasoline, diesel and oil slurry.

[0096] The entities identified by the present invention include: devices, raw materials and products. Devices include: reaction regeneration devices, fractionation devices, gas compressor units, etc.; raw materials include hydrogenated wax oil, vacuum residue oil, deasphalted oil, etc.; products include: dry gas, liquefied gas, gasoline, diesel, etc. Traditional entity naming methods are generally divided into three categories: dictionary and rule-based methods, unsupervised methods, and supervised machine learning methods based on feature engineering. In natural language, there are polysemy, multiple words with one meaning, and aliases.

[0097] In the chemical industry, there are often multiple words with one meaning, such as 3# catalytic cracking unit, 3# unit, P03, etc.; there are also complex entity nesting, such as: ethylene glycol recovery tower top separator, which has four nestings: ethylene glycol, separator, ethylene glycol recovery tower and ethylene glycol recovery tower top separator, and their types are not consistent. Ethylene glycol is a material, and the other three entities are equipment. In addition, most entities are unique professional terms, such as rich gas and backwash gas in materials, Fischer-Tropsch reactor and rich liquid flash tank in equipment. In addition, some terms are very long, such as antifreeze dehydration tower top air cooler, release gas compressor reflux cooler and ethylene glycol recovery tower top separator. Finally, there are a large number of special terms, such as "run", "emerge", "drip" and "leak". Therefore, a wide variety of professional vocabulary and its composition are a major problem for information extraction.

[0098] Named entity linking is the process of linking named entity mentions in a text to unambiguous entities in a knowledge base. Traditional non-deep learning methods require a lot of manual work and have poor portability. Traditional named entity relationship extraction methods often require a lot of manual work and manually designed features or rules, which will cause error accumulation and propagation problems, affecting the performance of named entity extraction.

[0099] Since the deep learning-based method does not require the design of artificial features, the present invention uses a deep learning-based method to perform named entity recognition, named entity connection, and named entity relationship extraction. The deep learning-based named entity recognition method generally includes three steps: distributed representation of input, context encoding, and label decoding. The deep learning-based entity linking method mainly has three modules, namely, candidate entity generation (Candidate Entity Generation), candidate entity ranking (Candidate Entity Ranking), and unlinkable mention prediction.

[0100] Named entity relationships refer to semantic relationships between entities. The relationship between two entities is called a binary relationship, and the relationship between three or more entities is called a multi-relationship or a higher-order relationship. Binary relationships are the most basic and common, and multi-relationships can often be converted into binary relationships for processing.

[0101] Step S12: Use the pre-trained model to perform distributed representation on the words in the acquired data

[0102] Specifically, before representing the data in text form, the data must be preprocessed and enhanced. The text information of unstructured data must be annotated according to the standard pattern of the information extraction model, and the annotated text is used as the training set, verification set, and test set. The data volume is expanded through artificial construction for data enhancement. The expansion methods include original text expansion, random truncation, same-sex word replacement, and pre-trained model output replacement.

[0103] Furthermore, the pre-trained model is used to represent words in a distributed manner. Consider using Bert and Ernie as pre-trained models for knowledge representation. It uses the encoder structure of a bidirectional transfomer and is pre-trained based on Chinese Wikipedia as the relevant corpus. Ernie is a model released by Baidu that is pre-trained based on corpus such as Baidu Tieba. It uses word-level MASK and can better recognize the semantics of Chinese.

[0104] The graph construction unit is used to extract features of knowledge information represented by vectors based on a pre-trained entity-relationship joint extraction model, and to construct a knowledge graph based on the extracted features.

[0105] Specifically, the knowledge information represented by the vector is BIO labeled to obtain a labeled sequence; the vector of each word is input into the Bi-LSTM layer; wherein the Bi-LSTM layer includes two LSTM models, which are respectively used for extracting forward hidden features and extracting reverse hidden features; the sentence features of the labeled sequence are extracted based on the merging processing of the extracted forward hidden features and the extracted reverse hidden features in the Bi-LSTM layer; the feature relationship between each word vector is extracted based on the sentence features in the conditional random field layer; the head entity and the tail entity are identified in the labeled sequence, and the head entity vector and the tail entity vector are executed with a score function to extract attribute features; the feature extraction result is output based on the sentence features, feature relationships and attribute features.

[0106] In the embodiment of the present invention, a joint entity and relationship extraction model based on transfer learning is used. Specifically, a feature-sharing neural network is used to jointly extract entity relationships, where Bi-LSTM and CRF are used to extract entity features and label each entity; a relationship extraction model based on transfer learning is used to handle complex relationships between different entity pairs. Finally, the py2neo library is used to automatically construct the obtained entities, relationships, and attributes in the neo4j database and store them in the cloud.

[0107] Furthermore, the BIO annotation of the knowledge information represented by the vector includes:

[0108] The segmented data is annotated with BIO, for example, three-digit annotation; B begin, I inside, O outside; BX represents the beginning of entity X, IX represents the end of entity, and O represents not belonging to any type.

[0109] Specifically, first import the labeled sequence into the embedding layer. For an input sequence X = (x1, x2, ... x n ), n represents the length of the sequence. For each word in the sentence, the joint extraction model inputs the embedding of each word into a Bi-LSTM, and then uses the output vector of the nth layer of the Bi-LSTM model as the overall feature of the word.

[0110] The main function of the Bi-LSTM layer is to extract features from sentences. In the embedding layer, the extraction model can obtain the embedding of each word and use it as a sequence I = (i1, i2, ...i n ). Bi-LSTM contains two LSTM models, one extracting forward hidden features Another method to extract reverse hidden features The Bi-LSTM layer can be represented as follows:

[0111]

[0112]

[0113] in, To extract forward hidden features; To extract reverse hidden features; i t is the tth word vector. Combining the forward and reverse hidden layer features can get the output Use an activation function to extract Then map it to a d l dimensional space. l is the number of entity types that need to be identified.

[0114] Furthermore, in the conditional random field layer, the feature relationship between each word vector is extracted based on the sentence features. For each feature output by Bi-LSTM, the model uses the conditional random field to identify the specific label yt at each specific position in the sequence. The conditional random field can better consider the dependency between different labels and thereby improve the accuracy of entity recognition. Specifically, the model merges the output l of the Bi-LSTM layer into a sequence L = [l1, l2, ... l n ] T , input into the CRF (conditional random field) model, where the size of L is s×d l .

[0115] Identify the head entity and the tail entity in the labeled sequence, and perform the score function on the head entity vector and the tail entity vector to extract attribute features, including: for the triples in the sequence, the joint extraction model obtains the embedding vector e1 of the head entity and the embedding vector e2 of the tail entity in the embedding layer by the embedding summation method, and generates the corresponding relationship embedding vector r according to their corresponding relationship r. In order to obtain the representation vectors of entities and relationships at the same time, first assume that e1+r and vector e2 are relatively close, specifically e1+r≈e2. Its score function can be expressed as follows:

[0116]

[0117] The model can also obtain the representation vector of the head and tail entities in the Bi-LSTM layer and And we hope that the entity representation vector in the Bi-LSTM layer also satisfies the constraint and Therefore, the score function in the Bi-LSTM layer can be expressed as follows:

[0118]

[0119]

[0120] In general, the loss function of the model can be regarded as a combination of the entity extraction loss function and the relationship extraction loss function:

[0121] L=L e +λL r

[0122] Where Le is the entity extraction loss function, Lr is the relationship extraction loss function, and λ is the weight hyperparameter to measure the weight of entity relationship extraction.

[0123] Linking unit, used to link entities in the constructed knowledge graph as a knowledge base.

[0124] Specifically, the artificially constructed knowledge graph is regarded as a knowledge base, which contains an entity set E; the entities extracted from the chemical plant related data in the present invention are considered to be potential associated objects of the entity set E, and the set is denoted as M; the task objective is to correspond each entity m∈M to an unambiguous target entity e∈E. If the set E does not contain the target entity of m in the actual calculation, m is linked to a new entity (as a supplement to the original entity library). In terms of specific model selection, the present invention adopts a model based on a graph convolutional network, which uses a graph convolutional network to model local context and global consistency information. An optimal allocation is found for the candidate entity through function calculation.

[0125]

[0126] in, is the variable that outputs the candidate entity, P() is the probability function, is the normalized adjacency matrix of the topological graph, and f is the feature representation of the candidate entity.

[0127]

[0128] Among them, F() is a mapping function under the parameter ω, which is obtained by using an encoder, a subgraph convolutional network and a decoder.

[0129] The feedback unit is used to collect the user's question text information, match the answer based on the knowledge base, and feed back the matching answer to the user end.

[0130] Specifically, feature extraction is performed on the question text information, the question and entity are embedded to obtain their dense vector representation, and relevant matching is performed in the candidate knowledge graph; the corresponding node relationship is embedded to obtain its dense vector representation, and the matching degree between the question and the candidate answer is calculated, and the most matching answer is screened out based on the matching degree.

[0131] Through user input questions, semantic questions are matched with entities and relationships in the chemical plant safety risk map, and answers are inferred. The pre-trained model is used to extract question features, and the single word vectors obtained are added to obtain the vector representation of the corresponding sentence; the answer entity is vectorized using the TransE series model; a multi-column convolutional network is used to extract three feature vectors as the three dimensions of the answer, namely the path of the answer, the contextual information of the answer, and the type of the answer. Through user input questions, the questions and entities are embedded to obtain their dense vector representation, and relevant matching is performed in the candidate knowledge graph. After obtaining the matching knowledge, the corresponding node relationship is embedded to obtain its dense vector representation, and the matching degree between the question and the candidate answer is calculated, and the most matching answer is selected through the model. Bert and other pre-trained models are used to extract question features. The single word vectors obtained are added to obtain the vector representation of the corresponding sentence. This method can effectively identify the part of speech and order. At the same time, in order to improve the performance of the model, additional information can be introduced.

[0132] Furthermore, for different answer features (entity, relationship, type, context), a score function S(q,a) is used to measure the degree of association between the question and the candidate answer; the distributed expression for answer i is:

[0133] g i (a)∈{g e (a), g r (a), g t (a), g c (a)}

[0134] g e (a) is the entity vector, g r (a) is the relationship vector, g t (a) is the type vector, g c (a) is the context vector. For the word vector of the jth word in question i (h output by Bert) j ), the distributed expression of the corresponding problem is recorded as:

[0135]

[0136] a ij is the attention weight of sentence i for word j. The final score function is the dot product of the question and the answer:

[0137]

[0138] The embodiment of the present invention further provides a computer-readable storage medium, on which instructions are stored, which, when executed on a computer, enable the computer to execute the above-mentioned xx.

[0139] Those skilled in the art will understand that all or part of the steps in the method for implementing the above-mentioned embodiments can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions for making a single-chip microcomputer, a chip or a processor (processor) perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0140] The optional embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the technical concept of the embodiments of the present invention, the technical scheme of the embodiments of the present invention can be subjected to a variety of simple modifications, and these simple modifications all belong to the protection scope of the embodiments of the present invention. It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the embodiments of the present invention will not further describe various possible combinations.

[0141] In addition, various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the embodiments of the present invention, they should also be regarded as the contents disclosed in the embodiments of the present invention.

Claims

1. A chemical equipment question-answering method integrating knowledge graph, characterized in that: The method comprises: Collecting chemical plant knowledge information and performing vector representation on words in the knowledge information; Based on the pre-trained entity-relationship joint extraction model, feature extraction is performed on the knowledge information represented by the vector, and a knowledge graph is constructed based on the extracted features; Use the constructed knowledge graph as a knowledge base for entity linking; The user's question text information is collected, and answer matching is performed based on the knowledge base, and the matching answer is fed back to the user end.

2. The method according to claim 1, characterized in that The chemical plant knowledge information includes: Structured, semi-structured and unstructured data extracted from the open source chemical plant indicator library.

3. The method according to claim 2, characterized in that Before vectorizing the words in the knowledge information, the solution further includes: Identify the data type and select a corresponding preprocessing solution based on the identified data type; Preprocessing the knowledge information based on a selected preprocessing scheme; including: Text annotation for unstructured data; The unstructured data and structured data after text annotation are combined into training samples.

4. The method according to claim 1, characterized in that: The vector representation of the words in the knowledge information includes: Based on the Bert model or the Ernie model, the knowledge information is trained to obtain words represented by vectors.

5. The method according to claim 1, characterized in that The entity relationship joint extraction model based on pre-training performs feature extraction on the knowledge information represented by the vector, including: Perform BIO annotation on the knowledge information represented by the vector to obtain the annotation sequence; The vector of each word is input into the Bi-LSTM layer; The Bi-LSTM layer includes two LSTM models, which are used to extract forward hidden features and reverse hidden features respectively; Extracting the features of the labeled sequence sentences based on the combined processing of the extracted forward hidden features and the extracted reverse hidden features at the Bi-LSTM layer; In the conditional random field layer, the feature relationship between word vectors is extracted based on sentence features; Identify the head entity and the tail entity in the annotation sequence, and perform a scoring function on the head entity vector and the tail entity vector to extract attribute features; Output feature extraction results based on sentence features, feature relationships and attribute features.

6. The method according to claim 5, characterized in that The Bi-LSTM layer is represented as: in, To extract forward hidden features; To extract reverse hidden features; i t is the t-th word vector.

7. The method according to claim 5, characterized in that The step of identifying a head entity and a tail entity in the annotation sequence includes: Identify the triplets in the annotation sequence and obtain the head entity and tail entity by summing the embeddings. The score function is: Among them, e1 is the head entity vector; e2 is the tail entity vector; r is the embedding vector generated based on the feature relationship.

8. The method according to claim 1, characterized in that After the constructed knowledge graph is used as a knowledge base for entity linking, the method further includes: Taking all entities extracted based on the knowledge information as entity sets; Determine whether there is a situation in the connected knowledge base that does not contain the entity target in the entity set; If so, the new entity is linked based on the corresponding entity target and added to the knowledge base.

9. The method according to claim 1, characterized in that: The step of collecting the user's question text information, matching the answer based on the knowledge base, and feeding back the matching answer to the user terminal includes: Extract features from the question text information, embed the question and entity to obtain their dense vector representation, and perform relevant matching in the candidate knowledge graph; After embedding the corresponding node relationship, its dense vector representation is obtained, and the matching degree between the question and the candidate answer is calculated, and the most matching answer is selected based on the matching degree.

10. A chemical equipment question-answering system integrating knowledge graph, characterized in that: The system comprises: A collection unit, used for collecting chemical plant knowledge information and performing vector representation on words in the knowledge information; A graph construction unit is used to extract features from knowledge information represented by vectors based on a pre-trained entity-relationship joint extraction model, and to construct a knowledge graph based on the extracted features; Linking unit, used to link entities by constructing the knowledge graph as a knowledge base; The feedback unit is used to collect the user's question text information, match the answer based on the knowledge base, and feed back the matching answer to the user end.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the chemical plant question-answering method integrating a knowledge graph as described in any one of claims 1 to 9.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the chemical equipment question-answering method integrating the knowledge graph as described in any one of claims 1 to 9 is implemented.

Images