Chemical knowledge base construction method fusing deep learning and knowledge graph

Abstract

The invention relates to the technical field of chemical engineering, in particular to a chemical knowledge base construction method fusing deep learning and a knowledge graph, and the method comprises the steps: obtaining a to-be-input chemical natural language statement; preprocessing the chemical class natural language statement to obtain a preprocessed chemical class natural language vector; obtaining a first sub-graph corresponding to the preprocessed chemical natural language vector according to a knowledge graph constructed in advance; inputting the preprocessed chemical engineering type natural language vector and a first sub-graph corresponding to the preprocessed chemical engineering type natural language vector into a trained sub-graph acquisition model, and acquiring a final sub-graph triple; and storing the to-be-input chemical natural language statement and the corresponding final sub-graph triple in a pre-constructed chemical knowledge base to obtain the chemical knowledge base. According to the invention, the problem of complex data lookup during solution design in the prior art is solved, and the problems of long time consumption and high cost during maintenance of the process equipment knowledge graph are solved.

Description

Technical Field

[0001] The present invention relates to the field of chemical technology, and in particular to a method for constructing a chemical knowledge base integrating deep learning and knowledge graphs. Background Art

[0002] In the existing chemical industry, there is no structure specifically designed for the professional knowledge map used in the actual chemical process production. Therefore, when solving the process solution according to the actual process design, it is necessary to consult a large number of books and relevant regulations, which takes a lot of time.

[0003] In addition, the existing solutions only include placing the corresponding knowledge of various common problems in chemical industry into the knowledge graph in a disorderly manner through entity extraction, relationship extraction, attribute extraction and other technologies. It does not have the professionalism of vertical fields, so the error rate is high.

[0004] Therefore, the present invention urgently needs a method for constructing a chemical knowledge base that integrates deep learning and knowledge graphs. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a method for constructing a chemical knowledge base integrating deep learning and knowledge graphs, which solves the problem of complex data consultation when designing solutions in the prior art, and solves the problem of time-consuming and costly maintenance of process equipment knowledge graphs.

[0007] (II) Technical solution

[0008] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0009] The embodiment of the present invention provides a method for constructing a chemical engineering knowledge base integrating deep learning and knowledge graph, comprising:

[0010] S100, obtaining a chemical engineering natural language sentence to be input; preprocessing the chemical engineering natural language sentence to obtain a preprocessed chemical engineering natural language vector;

[0011] S200, obtaining a first subgraph corresponding to the preprocessed chemical engineering natural language vector according to the knowledge graph constructed in advance;

[0012] S300: Input the preprocessed chemical engineering natural language vector and the first subgraph corresponding to the preprocessed chemical engineering natural language vector into the trained subgraph acquisition model to obtain the final subgraph. Figure 3 Tuple;

[0013] S400: The chemical engineering natural language sentence to be input and the corresponding final sub Figure 3The tuple is saved in a pre-built chemical knowledge base, and the chemical knowledge base is obtained.

[0014] The sub-graph acquisition model includes: an embedded matrix acquisition sub-model and a sub-graph acquisition sub-model.

[0015] Optionally, before S100, the step further includes:

[0016] S000, training the sub-graph acquisition model using a training data set to obtain a trained sub-graph acquisition model;

[0017] The training data set is obtained by preprocessing the chemical data set;

[0018] The chemical industry data set includes: various chemical industry natural language sentences and sub-sentences corresponding to each chemical industry natural language sentence: Figure 3 Tuple;

[0019] Among them, each chemical natural language sentence in the chemical data set corresponds to the sub Figure 3 The tuples were obtained by consulting a large number of books and relevant regulations.

[0020] Optionally, the S000 specifically includes:

[0021] The cross entropy loss function and Adam optimizer are used, the learning rate is set to 0.001, the batch size of each training is 200, and 100 repetitions are performed to obtain the trained subgraph acquisition model.

[0022] Optionally, in S100, preprocessing the chemical engineering natural language sentence to obtain the preprocessed chemical engineering natural language vector includes:

[0023] S110, cleaning and standardizing the chemical engineering natural language sentence, removing irrelevant characters, unifying the text format, removing redundant spaces and line breaks, and obtaining a first natural language;

[0024] S120, performing word segmentation and vocabulary normalization on the first natural language, dividing the first natural language into words, removing stop words, and standardizing the vocabulary form by stem extraction or word form restoration to obtain a second natural language;

[0025] S130: Perform vectorization processing on the second natural language to obtain a preprocessed chemical engineering natural language vector.

[0026] Optionally, the S300 specifically includes:

[0027] S310, inputting the preprocessed chemical engineering natural language vector and the first subgraph corresponding to the preprocessed chemical engineering natural language vector into an embedding matrix acquisition submodel to acquire a corresponding embedding matrix;

[0028] S320, input the embedding matrix into the subgraph acquisition submodel to obtain the final subgraph Figure 3 Tuple;

[0029] The sub-graph acquisition sub-model includes: a mixed graph attention layer, a first feed-forward network layer, a residual connection layer, a layer normalization layer, a second feed-forward network layer and a softmax layer.

[0030] Optionally, the S310 specifically includes:

[0031] S311, according to the word embedding method, converting the preprocessed chemical engineering natural language vector into an embedding vector, and using the attention mechanism to obtain a text attention matrix of the embedding vector;

[0032] S312. Obtain the entity feature vector and adjacency matrix of the first subgraph, and obtain the corresponding embedding matrix according to the text attention matrix, the entity feature vector and the adjacency matrix.

[0033] Optionally, the S312 includes:

[0034] Input the entity feature vector and adjacency matrix into the following formula to obtain the corresponding embedding matrix:

[0035]

[0036] Among them, F r is the embedding matrix, E r is the adjacency matrix, Q A is the query matrix of entity feature vectors, is the bond matrix of the entity eigenvector, V c is the value matrix of the entity feature vector, d is the scaling factor, and softmax is the soft maximum function.

[0037] Optionally, the S320 includes:

[0038] S321, inputting the embedding matrix into the mixed graph attention layer, performing weighted fusion on the embedding matrix, and obtaining a first feature;

[0039] S322, inputting the first feature output by the hybrid graph attention module into the first feedforward network layer, performing linear transformation and linear activation, and obtaining a second feature;

[0040] S333: Input the second feature into the residual connection layer and the layer normalization layer, perform residual processing and standardization processing, and obtain a preliminary output value.

[0041] Optionally, the S320 further includes:

[0042] S334, inputting the preliminary output value into the second feedforward network layer, performing feature enhancement, and obtaining an intermediate output value;

[0043] S335: Input the intermediate output value into the softmax layer, obtain the probability distribution corresponding to each intermediate output value, and obtain the intermediate output value with the largest probability as the final sub-value. Figure 3 Tuple.

[0044] Optionally, the method further comprises:

[0045] S500, according to the latest chemical industry standards and research results, regularly review the chemical industry natural language sentences and corresponding sub-languages ​​in the chemical industry knowledge base. Figure 3 The tuple is verified and updated, and the mismatched content is corrected or replaced.

[0046] (III) Beneficial effects

[0047] The beneficial effects of the present invention are as follows: a method for constructing a chemical knowledge base that integrates deep learning and knowledge graphs can add linkage between chemical process flows compared to the prior art due to the integration of deep learning and knowledge graphs, and plays a role in timely providing potential options for solving prevention devices; in addition, the present invention realizes end-to-end information interaction between natural language and graphs. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of a process for constructing a chemical knowledge base integrating deep learning and knowledge graph according to an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of a process for obtaining an embedding matrix in a method for constructing a chemical knowledge base integrating deep learning and knowledge graph in Example 2 of the present invention;

[0050] Figure 3 is a schematic diagram of an adjacency matrix in Embodiment 2 of the present invention;

[0051] Figure 4 A schematic diagram of a first process of obtaining a sub-model from a sub-graph in Embodiment 2 of the present invention;

[0052] Figure 5 A schematic diagram of a second process of obtaining a sub-model from a sub-graph in Embodiment 2 of the present invention;

[0053] Figure 6 This is a schematic diagram of the knowledge graph in Example 2 of the present invention. DETAILED DESCRIPTION

[0054] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0055] Text embedding: Text embedding vector is a technology that converts natural language text into a fixed-length vector representation. Its core purpose is to convert discrete language symbols (such as words, sentences, paragraphs, etc.) into vectors that can be operated in a continuous space so that machine learning models can process and understand the semantic relationships of text.

[0056] Self-Attention: For each attention head, the input embedding vector is first transformed into query, key and value through three linear transformations:

[0057]

[0058] Among them, W Q , W K , W V are the weight matrices of query, key, and value, with dimensions d, d respectively. k ,d v . Where h is the number of attention heads in the multi-head attention, and R is a set of real numbers.

[0059] For each attention head, the attention weight is calculated using the scaled dot product attention mechanism. The formula for dot product attention is:

[0060]

[0061] Among them, softmax is the soft maximum function, K T is the bond matrix, d k is the scaling factor, V is the value matrix, and Q is the query matrix.

[0062] Adjacency matrix: The adjacency matrix is ​​a matrix representation of a graph, used to represent the connection relationship between nodes. Given a graph with n nodes, the adjacency matrix A∈R n×n is a two-dimensional matrix, each element A of the matrix ij Represents the connection relationship between node i and node j.

[0063] Knowledge Graph: A knowledge graph is a structured semantic knowledge base that is designed to store information related to real-world entities (such as people, places, organizations, products, etc.) and describe the various relationships between these entities. Knowledge graphs are often used to enhance search engines, provide richer data context, improve recommendation systems, support natural language processing tasks, etc.

[0064] Key features and components of knowledge graphs:

[0065] Entities: The basic unit in the knowledge graph, representing objects or concepts in the real world, such as "Eiffel Tower", "Apple", etc.

[0066] Relations: semantic connections between entities, such as "located in", "founded by", "belongs to", etc. Relations connect different entities to form a network.

[0067] Attributes: Specific characteristics or information possessed by an entity, such as a person's name, age, occupation, etc.

[0068] Ontology: A conceptual framework that defines entities, relationships, and attributes. It specifies the structure and content of the knowledge graph, including the classification system and the hierarchical relationship between concepts.

[0069] Reasoning: Using information in the knowledge graph to perform logical reasoning to discover new patterns, relationships, or verify hypotheses.

[0070] Graph Database: A database system that stores knowledge graph data and supports efficient graph query and analysis.

[0071] Data mining and information extraction: Automatically extract information from various data sources (such as text, databases, web pages, etc.) and convert them into entities and relationships in the knowledge graph.

[0072] Visualization: Display the knowledge graph in a graphical way so that users can intuitively understand the relationship between entities.

[0073] Graph Neural Networks (GNN): Graph Neural Networks is a deep learning model for processing graph structured data. It learns the features of nodes, edges, or the entire graph by representing the nodes and edges in the graph as vectors and propagating information in the graph.

[0074] The embodiment of the present invention proposes a method for constructing a chemical knowledge base integrating deep learning and knowledge graph. In order to solve the problems of complicated book reference, time-consuming and high-cost maintenance of process equipment knowledge graph in actual chemical process production, the present invention uses a method combining deep learning and knowledge graph. By inputting a chemical natural language sentence, a first subgraph is obtained according to a knowledge graph constructed in advance. After the first subgraph is obtained, a subgraph is obtained according to a trained subgraph acquisition model. Figure 3 Tuples increase the linkage of chemical process flows, can provide potential options for solutions and devices in a timely manner, and make actual chemical production more convenient.

[0075] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0076] Example 1

[0077] See also Figure 1 , a method for constructing a chemical knowledge base integrating deep learning and knowledge graph in an embodiment of the present invention includes:

[0078] Step S100, obtaining a chemical engineering natural language sentence to be input; preprocessing the chemical engineering natural language sentence to obtain a preprocessed chemical engineering natural language vector;

[0079] Step S200: Obtain a first subgraph corresponding to the preprocessed chemical engineering natural language vector according to the knowledge graph constructed in advance;

[0080] Step S300: Input the preprocessed chemical engineering natural language vector and the first subgraph corresponding to the preprocessed chemical engineering natural language vector into the trained subgraph acquisition model to obtain the final subgraph. Figure 3 Tuple;

[0081] Step S400: Input the chemical engineering natural language sentence and the corresponding final sub Figure 3 The tuple is saved in a pre-built chemical knowledge base, and the chemical knowledge base is obtained.

[0082] The sub-graph acquisition model includes: an embedding matrix acquisition sub-model and a sub-graph acquisition sub-model.

[0083] In this embodiment, a method for constructing a chemical knowledge base that integrates deep learning and knowledge graphs adds the linkage between devices in chemical process flows, supplements the professional information of the knowledge graph relationship, and plays a role in timely providing potential options for solution devices, while realizing end-to-end information interaction between natural language and knowledge graphs.

[0084] Example 2

[0085] A method for constructing a chemical knowledge base integrating deep learning and knowledge graph according to an embodiment of the present invention includes:

[0086] Step S100, obtaining a chemical engineering natural language sentence to be input; preprocessing the chemical engineering natural language sentence to obtain a preprocessed chemical engineering natural language vector;

[0087] Step S200: Obtain a first subgraph corresponding to the preprocessed chemical engineering natural language vector according to the knowledge graph constructed in advance;

[0088] Step S300: Input the preprocessed chemical engineering natural language vector and the first subgraph corresponding to the preprocessed chemical engineering natural language vector into the trained subgraph acquisition model to obtain the final subgraph. Figure 3 Tuple;

[0089] Step S400: Input the chemical engineering natural language sentence and the corresponding final sub Figure 3 The tuple is saved in a pre-built chemical knowledge base, and the chemical knowledge base is obtained.

[0090] The sub-graph acquisition model includes: an embedding matrix acquisition sub-model and a sub-graph acquisition sub-model.

[0091] In this embodiment, constructing a knowledge graph in advance includes:

[0092] Define conceptual entities: Divide concepts into broad categories, such as input raw materials, output raw materials, by-products, devices, etc.

[0093] Define specific entities: Based on the division of conceptual entities, define specific entities, such as: coke tower, distillation tower, crude oil, diesel, etc.

[0094] Define the relationship:

[0095] Transport: Indicates the transfer of materials (such as liquids and gases) from one device to the next. For example, the liquid collector after the packed tower is connected by the "Transport" relationship, which means "the liquid is transported to the liquid collector for storage after passing through the packed tower."

[0096] Heating / Cooling: Indicates the energy transfer relationship between devices, such as temperature control through a heat exchanger.

[0097] Separation / mixing: refers to the separation or mixing operation of different materials in specific equipment.

[0098] Reaction: A chemical reaction that occurs in a specific piece of equipment, such as a reactor and other ancillary equipment.

[0099] In a specific implementation process, in step S200, obtaining the first subgraph corresponding to the preprocessed chemical engineering natural language vector according to the knowledge graph constructed in advance specifically includes:

[0100] Obtain a subgraph corresponding to the preprocessed chemical engineering natural language vector, remove duplicates, merge and rearrange the subgraphs, and obtain a corresponding first subgraph;

[0101] For example, suppose Figure 6It is a knowledge graph constructed in advance in the embodiment of the present invention, wherein A, B, and C are equipment entities, C and D are process entities, and E and F are material entities; 1 represents the relationship between equipment and process, 2 represents the relationship between equipment and equipment, 3 represents the relationship between equipment and material, and 4 represents the relationship between material and equipment.

[0102] The subgraphs obtained starting from a single entity D include:

[0103] (D, E): (B, 1, D), (A, 2, B), (A, 3, E);

[0104] (D, B): (B, 1, D), (A, 2, B);

[0105] (D, C): (B, 1, D), (B, 1, C);

[0106] (D, A): (B, 1, D), (B, 1, C), (A, 1, C);

[0107] (D, F): (B, 1, D), (B, 3, F);

[0108] (D, G): (D, G);

[0109] The subgraph obtained above is further screened to delete two completely overlapping paths. That is, assuming that there are two entities D and F in the natural language vector of the chemical industry, but they share a path: (D, F): (B, 1, D), (B, 3, F); then one of the paths is randomly deleted.

[0110] The subgraphs after further screening are reorganized. For example, assuming that two entities E and G are recalled, there are subgraphs:

[0111] (E, D): (A, 3, E), (A, 2, B), (B, 1, D)

[0112] (A, G): (G, 1, D), (B, 1, D), (A, 2, B)

[0113] The two paths can be merged into: (E, G): (A, 3, E), (A, 2, B), (B, 2, D).

[0114] Finally, the reorganized subgraphs are reordered based on the chemical engineering natural language vectors. Based on the multiple entities found, the closer the distance between adjacent entities, the more common paths are ranked first. If there is a tie, they are arranged according to the order of the chemical engineering natural language vectors:

[0115] The first subgraph obtained is (assuming that there are two entities D and E in the chemical engineering natural language vector):

[0116] (D, E): (B, 1, D), (A, 2, B), (A, 3, E);

[0117] (D, C): (B, 1, D), (B, 1, C);

[0118] (D, A): (B, 1, D), (B, 1, C), (A, 1, C);

[0119] (D, F): (B, 1, D), (B, 3, F);

[0120] (D, G): (D, G);

[0121] (E, B): (A, 1, E), (A, 2, B);

[0122] (E, F): (A, 1, E), (A, 2, B), (B, 3, F). In this embodiment, before step S100, the following steps are also included:

[0123] Step S000: training the sub-graph acquisition model using a training data set to obtain a trained sub-graph acquisition model;

[0124] The training data set is obtained by preprocessing the chemical data set;

[0125] The chemical industry dataset includes: various chemical industry natural language sentences and the corresponding sub-sentences of each chemical industry natural language sentence. Figure 3 Tuple;

[0126] Among them, each chemical natural language sentence in the chemical data set corresponds to the sub Figure 3 The tuples were obtained by consulting a large number of books and relevant regulations.

[0127] Step S000 specifically includes:

[0128] The cross entropy loss function and Adam optimizer are used, the learning rate is set to 0.001, the batch size of each training is 200, and 100 repetitions are performed to obtain the trained subgraph acquisition model.

[0129] In the specific implementation process, the training set is divided into multiple small batches, each batch size is 200 samples, and the following operations are performed on each small batch:

[0130] The chemical engineering natural language vectors in the mini-batch and their corresponding first subgraphs are input into the subgraph acquisition model, and the output of the model, i.e., the final subgraph, is calculated. Figure 3 Tuple;

[0131] The cross entropy loss function is used to calculate the difference between the model output and the actual Figure 3The loss between tuples, the gradient of the loss to the model parameters is calculated through the back-propagation algorithm, and the Adam optimizer is used to update the model parameters according to the calculated gradient.

[0132] Repeat the above mini-batch training until 100 full training rounds are completed.

[0133] After each training round, evaluate the performance of the model to monitor overfitting. If the performance of the model is not improving, consider stopping the training early or adjusting the learning rate to obtain the trained subgraph model.

[0134] In this embodiment, in step S100, preprocessing the chemical engineering natural language sentence, and obtaining the preprocessed chemical engineering natural language vector includes:

[0135] Step S110, cleaning and standardizing the chemical engineering natural language sentences, removing irrelevant characters, unifying the text format, removing redundant spaces and line breaks, and obtaining a first natural language;

[0136] Step S120: performing word segmentation and vocabulary normalization on the first natural language, dividing the first natural language into words, removing stop words, and standardizing the vocabulary form by stemming or word form restoration to obtain a second natural language;

[0137] Step S130: perform vectorization processing on the second natural language to obtain a preprocessed chemical engineering natural language vector.

[0138] In the specific implementation process, step S300 specifically includes:

[0139] Step S310: input the preprocessed chemical engineering natural language vector and the first subgraph corresponding to the preprocessed chemical engineering natural language vector into the embedding vector acquisition submodel to acquire the corresponding embedding matrix;

[0140] Step S320: Input the embedding matrix into the subgraph acquisition submodel to obtain the final subgraph. Figure 3 Tuple;

[0141] The subgraph acquisition submodel includes: a mixed graph attention layer, a first feedforward network layer, a residual connection layer, a layer normalization layer, a second feedforward network layer, and a softmax layer.

[0142] Step S310 specifically includes:

[0143] Step S311: According to the word embedding method, the preprocessed chemical engineering natural language vector is converted into an embedding vector, and the text attention matrix of the embedding vector is obtained by using the attention mechanism;

[0144] Step S312: Obtain the entity feature vector and adjacency matrix of the first subgraph, and obtain the corresponding embedding matrix according to the text attention matrix, the entity feature vector and the adjacency matrix.

[0145] Step S312 includes:

[0146] Input the entity feature vector and adjacency matrix into the following formula to obtain the corresponding embedding matrix:

[0147]

[0148] Among them, F r is the embedding matrix, E r is the adjacency matrix, Q A is the query matrix of entity feature vectors, is the bond matrix of the entity eigenvector, V c is the value matrix of the entity feature vector, d is the scaling factor, and softmax is the soft maximum function.

[0149] In a specific embodiment, the word embedding method is Word2Vec or GloVe method.

[0150] For example, the preprocessed natural language vector of the chemical industry is "catalytic cracking unit fractionation". The text length is 8. Convert it into an embedding vector X = [X1, X2, ...X n ]∈R n×d n is the number of vectors, and d is the dimension of the vector. At this time, the embedding matrix is ​​a matrix with a length of 8 and a dimension of 768.

[0151] See also Figure 2 , convert the embedding vector into a text attention matrix:

[0152]

[0153] Among them, softmax is the soft maximum function, K T is the bond matrix, d k is the scaling factor, V is the value matrix, Q is the query matrix, R is the set of real numbers, n is the number of entity vectors, and d is the dimension of the vector.

[0154] Then, the entity feature vector and adjacency matrix are extracted from the chemical equipment knowledge graph.

[0155] Let the entity feature vector of the graph be C∈R e×d , where e is the number of entities in the recalled subgraph in the graph, and d is the dimension of each feature vector, which is equal to the dimension d of the attention matrix obtained previously.

[0156] Compute the fused graph attention matrix:

[0157] Let WA ∈R e×n , Set a maximum value in advance, mac(e) is 1000, that is, the maximum number of subgraph entities is 1000, and max(n) is 4096, that is, the maximum natural language input length is 4096. Parts larger than these two values ​​are directly truncated; when lower than these two values, the redundant parameters are not included in the calculation.

[0158] The graph attention matrix is ​​obtained as follows:

[0159]

[0160] The adjacency matrix is Figure 3 As shown, the adjacency matrix shows that the subgraph relationship is that e1 is output to e2, e2 is output to e3, and e3 is output to e4.

[0161] Let the adjacency matrix of a single relationship be E r ∈R e×e , where e is the number of entities in the recalled subgraph in the graph, and r is the rth relation.

[0162] So the embedding matrix is:

[0163]

[0164] Among them, F r is the embedding matrix, E r is the adjacency matrix, Q A is the query matrix of entity feature vectors, is the bond matrix of the entity eigenvector, V c is the value matrix of the entity feature vector, d is the scaling factor, and softmax is the soft maximum function.

[0165] In this embodiment, step S320 includes:

[0166] Step S321: input the embedding matrix into the mixed graph attention layer, perform weighted fusion on the embedding matrix, and obtain the first feature;

[0167] Step S322: input the first feature output by the hybrid graph attention module into the first feedforward network layer, perform linear transformation and linear activation, and obtain the second feature;

[0168] Step S333: input the second feature into the residual connection layer and the layer normalization layer, perform residual processing and standardization processing, and obtain a preliminary output value.

[0169] Step S334: input the preliminary output value into the second feedforward network layer, perform feature enhancement, and obtain an intermediate output value;

[0170] Step S335: Input the intermediate output value into the softmax layer, obtain the probability distribution corresponding to each intermediate output value, and obtain the intermediate output value with the largest probability as the final sub- Figure 3 Tuple.

[0171] In the specific implementation process, see Figure 4 , the known embedding matrix F r =(F1, F1, ..., F r ), F r ∈R e×d ,set up are the weight matrices for the query, key, and value of the hybrid graph attention module, respectively.

[0172] Compute queries, keys, and values:

[0173]

[0174] Calculate the single-head image attention matrix Ar:

[0175]

[0176] in, The weight matrices of the query, key, and value corresponding to the embedding matrix are respectively, d is the dimension of the vector, e is the number of entity vectors, and F r is the embedding vector, Q r , K r 、V r are the query, key and value corresponding to the embedding matrix respectively, R is a set of real numbers, A r is the single-head graph attention matrix, softmax is the soft maximum function, is the key matrix corresponding to the embedding matrix, and r is the scaling factor.

[0177] Calculate longs:

[0178] When calculating multi-head attention, we no longer divide it into multiple heads based on the d dimension, but rather divide it into multiple heads based on relationships, that is, one relationship is one head.

[0179] After multiple heads are calculated, they are spliced ​​together, that is, the multi-head graph attention matrix is:

[0180] A R =[A1, A2, A3, ..., A r ]∈R r×e×d ;

[0181] Compute the fully connected layer with normalization and weighting:

[0182] Assume that the parameter matrix of the full link layer is W1∈R d×d , W2∈R d×d . b1, b2 are bias terms.

[0183] The calculation formula is:

[0184] FFN(A R )=selu(A R W1+b1)W2+b2∈R r×e×d ;

[0185] Residuals and normalization:

[0186] Output = LayerNorm(A R +FFN(A R ))∈R r×e×d .

[0187] See also Figure 5 , assuming that the initial output value is O′, the preliminary output value is input into the second feedforward network layer for feature enhancement, and the intermediate output value is obtained as:

[0188]

[0189] Input the intermediate output value into the softmax layer, and obtain the probability distribution corresponding to each intermediate output value:

[0190] Y = softmax(FFN output (O′))∈R r×e×e ;

[0191] Get the index of the maximum value in a matrix:

[0192] i * , j * , k * = argmax(Y i，j,k )∈R r×e×e ;

[0193] Where r represents the relationship index, j is the output entity index, and k is the input entity index.

[0194] The output can be summarized as (r, j, k), which generates the final sub Figure 3 Tuple.

[0195] In this embodiment, a method for constructing a chemical knowledge base integrating deep learning and knowledge graph further includes:

[0196] Step S500: Regularly review the chemical industry natural language sentences and corresponding sub-language sentences in the chemical industry knowledge base according to the latest chemical industry standards and research results. Figure 3 The tuple is verified and updated, and the mismatched content is corrected or replaced.

[0197] During the specific implementation process, the latest chemical industry standards and research results are regularly collected from the latest authoritative data sources. For example, the latest news is collected from authoritative data sources such as chemical industry standards, academic papers, technical reports, etc., so as to ensure the timeliness and accuracy of the chemical knowledge base.

[0198] The present embodiment provides a method for constructing a chemical knowledge base that integrates deep learning and knowledge graph. The method can query existing entities and relationships in the knowledge graph through the chemical knowledge base, and can also complete the hidden relationships within the graph through natural language and the existing chemical equipment knowledge graph, and display the forward and backward connections in the chemical process, thereby solving the problem of complex data consultation when designing solutions in the prior art, and solving the problem of time-consuming and costly maintenance of the process equipment knowledge graph.

[0199] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0200] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0201] In the present invention, unless otherwise clearly specified and limited, when a first feature is “on” or “below” a second feature, it may be that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Moreover, when a first feature is “above”, “above” or “above” a second feature, it may be that the first feature is directly above or obliquely above the second feature, or it may simply mean that the first feature is higher in level than the second feature. When a first feature is “below”, “below” or “below” a second feature, it may be that the first feature is directly below or obliquely below the second feature, or it may simply mean that the first feature is lower in level than the second feature.

[0202] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0203] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for constructing a chemical knowledge base integrating deep learning and knowledge graph, characterized in that: include: S100, obtaining a chemical engineering natural language sentence to be input; preprocessing the chemical engineering natural language sentence to obtain a preprocessed chemical engineering natural language vector; S200, obtaining a first subgraph corresponding to the preprocessed chemical engineering natural language vector according to the knowledge graph constructed in advance; S300, inputting the preprocessed chemical engineering natural language vector and the first subgraph corresponding to the preprocessed chemical engineering natural language vector into a trained subgraph acquisition model to obtain a final subgraph triple; S400: Save the chemical engineering natural language sentence to be input and the corresponding final subgraph triple in a pre-built chemical engineering knowledge base to obtain the chemical engineering knowledge base. The sub-graph acquisition model includes: an embedded matrix acquisition sub-model and a sub-graph acquisition sub-model.

2. The method for constructing a chemical knowledge base integrating deep learning and knowledge graph according to claim 1, characterized in that: The S100 also includes: S000, training the sub-graph acquisition model using a training data set to obtain a trained sub-graph acquisition model; The training data set is obtained by preprocessing the chemical data set; The chemical industry data set includes: various chemical industry natural language sentences and subgraph triples corresponding to each chemical industry natural language sentence; Among them, the subgraph triples corresponding to each chemical natural language sentence in the chemical data set are obtained by consulting a large number of books and relevant regulations.

3. The method for constructing a chemical knowledge base integrating deep learning and knowledge graph according to claim 2, characterized in that: The S000 specifically includes: The cross entropy loss function and Adam optimizer are used, the learning rate is set to 0.001, the batch size of each training is 200, and 100 repetitions are performed to obtain the trained subgraph acquisition model.

4. The method for constructing a chemical knowledge base integrating deep learning and knowledge graph according to claim 1, characterized in that: In S100, preprocessing the chemical engineering natural language sentence to obtain the preprocessed chemical engineering natural language vector includes: S11 0. Clean and standardize the chemical engineering natural language sentence, remove irrelevant characters, unify the text format, remove redundant spaces and line breaks, and obtain a first natural language; S120, performing word segmentation and vocabulary normalization on the first natural language, dividing the first natural language into words, removing stop words, and standardizing the vocabulary form by stemming or word form restoration to obtain a second natural language; S130: Perform vectorization processing on the second natural language to obtain a preprocessed chemical engineering natural language vector.

5. The method for constructing a chemical knowledge base integrating deep learning and knowledge graph according to claim 1, characterized in that: The S300 specifically includes: S310, inputting the preprocessed chemical engineering natural language vector and the first subgraph corresponding to the preprocessed chemical engineering natural language vector into an embedding matrix acquisition submodel to acquire a corresponding embedding matrix; S320, inputting the embedding matrix into a subgraph acquisition submodel to obtain a final subgraph triple; The sub-graph acquisition sub-model includes: a mixed graph attention layer, a first feed-forward network layer, a residual connection layer, a layer normalization layer, a second feed-forward network layer and a softmax layer.

6. The method for constructing a chemical knowledge base integrating deep learning and knowledge graph according to claim 5, characterized in that: The S310 specifically includes: S311, according to the word embedding method, converting the preprocessed chemical engineering natural language vector into an embedding vector, and using the attention mechanism to obtain a text attention matrix of the embedding vector; S312. Obtain the entity feature vector and adjacency matrix of the first subgraph, and obtain the corresponding embedding matrix according to the text attention matrix, the entity feature vector and the adjacency matrix.

7. The method for constructing a chemical knowledge base integrating deep learning and knowledge graph according to claim 6, characterized in that: The S312 includes: Input the entity feature vector and adjacency matrix into the following formula to obtain the corresponding embedding matrix: Among them, F r is the embedding matrix, E r is the adjacency matrix, Q A is the query matrix of entity feature vectors, is the bond matrix of the entity eigenvector, V c is the value matrix of the entity feature vector, d is the scaling factor, softmax is the soft maximum function, and C is the entity feature vector.

8. The method for constructing a chemical knowledge base integrating deep learning and knowledge graph according to claim 5, characterized in that: The S320 includes: S321, inputting the embedding matrix into the mixed graph attention layer, performing weighted fusion on the embedding matrix, and obtaining a first feature; S322, inputting the first feature output by the hybrid graph attention module into the first feedforward network layer, performing linear transformation and linear activation, and obtaining a second feature; S333: Input the second feature into the residual connection layer and the layer normalization layer, perform residual processing and standardization processing, and obtain a preliminary output value.

9. The method for constructing a chemical knowledge base integrating deep learning and knowledge graph according to claim 8, characterized in that: The S320 further includes: S334, inputting the preliminary output value into the second feedforward network layer, performing feature enhancement, and obtaining an intermediate output value; S335. Input the intermediate output value into the softmax layer, obtain the probability distribution corresponding to each intermediate output value, and obtain the intermediate output value with the largest probability as the final sub-graph triplet.

10. The method for constructing a chemical knowledge base integrating deep learning and knowledge graph according to claim 1, characterized in that: The method further comprises: S500. According to the latest chemical industry standards and research results, the chemical natural language sentences and corresponding subgraph triples in the chemical knowledge base are regularly verified and updated, and the unmatched contents are corrected or replaced.

Images