Dangerous chemical accident knowledge graph construction and application method based on multi-task learning

Abstract

Great threats are caused to life and property safety of people by dangerous chemical leakage, explosion and other major events. At present, domestic and overseas scholars have less construction work for a knowledge graph of dangerous chemical accidents, so that the knowledge of the public on the dangerous chemicals is restricted, and the prevention of the dangerous chemical accidents is not facilitated. Based on natural language processing and deep learning technologies, the invention provides a dangerous chemical accident knowledge graph construction and application method based on multi-task learning. Dangerous chemical accident data at home and abroad are collected to construct a dangerous chemical accident ontology, and a deep learning algorithm is used for knowledge extraction; constructing a dangerous chemical accident knowledge graph and carrying out semantic annotation according to the chemical safety technical specification and the extracted knowledge; and carrying out key node analysis on accident causes, accident hazard sources and accident development by virtue of the dangerous chemical accident knowledge graph. According to the method, the elements in the dangerous chemical accident can be automatically extracted, evolution of the dangerous chemical accident knowledge graph is facilitated, and a foundation is laid for accident early warning in the future.

Description

Technical Field

[0001] The present invention relates to the field of chemical safety, and in particular to a hazardous chemical accident knowledge graph construction and application method based on multi-task learning. Background Art

[0002] The chemical industry is an important pillar industry for my country's economic development. In recent years, my country's chemical industry has developed rapidly, and the technical level in the chemical field has continued to improve. In terms of policy, the state has taken a number of measures to strengthen supervision of chemical companies to ensure the safe operation of enterprises. However, due to operational errors caused by insufficient knowledge of hazardous chemicals, hazardous chemical accidents have occurred. The explosions and leaks of hazardous chemical accidents have caused great harm to people's lives and the environment.

[0003] Due to the large amount of data and wide range of data sources in chemical knowledge, even technical operators in this profession find it difficult to take effective measures in the first place when faced with sudden hazardous chemical accidents. The construction of a knowledge graph for hazardous chemical accidents can greatly improve the ability to retrieve massive amounts of knowledge, making it easier for operators to learn and master hazardous chemical knowledge; at the same time, the knowledge graph can be used to predict the direction of the evolution of hazardous chemical accidents, providing reference and reference for accident warning.

[0004] Google first proposed the concept of knowledge graph on May 17, 2012, which aims to describe the relationship between concepts, entities and events in the objective world. Knowledge graph is essentially a semantic network that is used to reveal the relationship between all things. It can extract concepts, entities and relationships from various types of complex data and is called a "computable model" of the relationship between things. Due to the main role and powerful functions of knowledge graph, Google has regarded it as the core foundation for building the next generation of intelligent search engines since the knowledge graph was proposed. Summary of the invention

[0005] The purpose of the present invention is to provide a method for constructing and applying a knowledge graph of hazardous chemical accidents based on multi-task learning based on natural language processing and deep learning technology. By collecting domestic and foreign hazardous chemical accident data to construct a hazardous chemical accident ontology, deep learning algorithms are used to extract knowledge; based on the "Chemical Safety Technical Manual" and the extracted knowledge, a knowledge graph of hazardous chemical accidents is constructed and semantic annotation is performed; with the help of the knowledge graph of hazardous chemical accidents, the causes of accidents, sources of accident hazards and key nodes of accident development are analyzed to lay the foundation for future accident warning. In recent years, major events such as hazardous chemical leaks and explosions have posed a great threat to the safety of people's lives and property, and there are relatively few studies on the construction of knowledge graphs for hazardous chemical accidents by scholars at home and abroad. Therefore, it is very necessary to construct a knowledge graph for hazardous chemical accidents. The method provided by the present invention can not only automatically extract the elements in hazardous chemical accidents, but also contribute to the evolution of the knowledge graph of hazardous chemical accidents; and through the knowledge graph, the public's cognition of hazardous chemicals can be accelerated, so as to better prevent hazardous chemical accidents.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for constructing a knowledge graph of hazardous chemical accidents based on multi-task learning, which specifically includes the following steps:

[0007] S1: Collect domestic and foreign hazardous chemicals accident data and clean the collected data;

[0008] S2: Construct the ontology of the knowledge graph;

[0009] S3: Knowledge extraction based on multi-task learning and end-to-end neural network model;

[0010] S4: Sort out the physical and chemical properties of major hazardous sources according to the Chemical Safety Data Sheet and supplement the extracted knowledge;

[0011] S5: Store entity relationship triples into the graph database to build a knowledge graph of hazardous chemical accidents;

[0012] S6: semantic annotation of hazardous chemicals accident knowledge graph;

[0013] S7: Based on the constructed knowledge graph of hazardous chemicals accidents, analyze the causes of accidents, accident hazards and key nodes of accident development.

[0014] The knowledge graph of hazardous chemical accidents is constructed through the steps of ontology construction, knowledge extraction and knowledge storage. The model layer of the knowledge graph defines the storage form of knowledge. When facing new accident cases, data cleaning, knowledge extraction and knowledge storage are carried out according to the process, which can automatically extract accident instances in the text and update the knowledge graph of hazardous chemical accidents.

[0015] Optionally, the sources of the hazardous chemical accident data in step S1 are: 1) writing crawler code to crawl hazardous chemical accident web page information; 2) collating data from the actual production process of chemical companies; 3) manually collecting and collating information and documents;

[0016] Preferably, in step S1, the following operations are performed to clean the data:

[0017] S1-1: Blank data processing. When the accident description content is blank, the data is considered to be blank data and deleted;

[0018] S1-2: Processing of irrelevant data. If the accident description does not contain similar chemical descriptions in the "Catalogue of Hazardous Chemicals", the data is considered to be irrelevant to the hazardous chemical accident and will be deleted;

[0019] S1-3: Redundant data processing. It includes two steps: 1) extracting the date and the name of the hazardous chemical in the accident from the two data; 2) deleting redundant data. If the time of the accident and the name of the hazardous chemical in the two data are the same, it is considered that the two data describe the same accident, and the data with more detailed description is retained;

[0020] S1-4: Invalid information processing. If a certain accident description does not exceed 15 words, it is considered that the data cannot provide valid information and the data will be deleted;

[0021] Preferably, in step S2, the ontology of the knowledge graph has the following categories and attributes:

[0022] S2-1: Accident type class: Optional attributes include risk type;

[0023] S2-2: Accident category: Optional attributes include accident time, accident location, and accident details;

[0024] S2-3: Major hazard source category: Optional attributes include CAS number, melting point, boiling point, storage conditions, storage form, and inventory;

[0025] S2-4: Equipment and facilities: optional attributes include temperature, pressure, and equipment defects;

[0026] S2-5: Equipment and process: optional attributes include reaction conditions, hazard, and criticality indicators;

[0027] S2-6: Human factors category: Optional attributes include improper operation;

[0028] S2-7: Accident results: optional attributes include number of injured, number of deaths, and economic losses;

[0029] S2-8: Enterprise: Optional attributes include business scope, risk, and contact information.

[0030] The relationship between the categories is as follows:

[0031] S2-9: Causal relationship: There are two types of relationships between equipment facilities, equipment technology, human factors and accidents: direct cause and indirect cause;

[0032] S2-10: Accident result relationship: There is an accident result relationship between the accident result and the accident;

[0033] S2-11: Hazardous chemicals subject relationship: There is a hazardous chemicals subject relationship between major hazard sources and accidents;

[0034] S2-12: Enterprise entity relationship: There is an organizational relationship between the enterprise and the accident;

[0035] S2-13: Accident type relationship: There is an accident type relationship between accident types and accidents.

[0036] Preferably, in step S3, the starting position and the ending position of the entity are marked by the boundary marking method, and the middle position of the entity and the remaining non-entities are uniformly marked with "O". The head entity and the tail entity are marked by the entity type label and the relationship type label respectively. For example, "An explosion occurred in Yunnan xx Company. Due to equipment defects, carbon four materials leaked and encountered the open flame of the incinerator, which ignited and exploded." The manual marking is:

[0037] Head-Entity (HE) annotation:

[0038] Start labeling: "Inci OOOOOOOOOOOOOOOOOOOOOOO OOOOO OO OOOOOOO OO"

[0039] End Markdown: "OOOOOOOOOOOO Inci OOOOOOOOOOOOOOOOO OO OOOOOOOOO"

[0040] Relation and Tail-Entity (RTE) annotation:

[0041] Extract relations and tail entities for the head entity Inci:

[0042] Start labeling: "OOOOOOOOOOOOOO DC OOOOO HC OOOOOOOO OO OO RC OO OO O"

[0043] End mark: "OOOOOOOOOOOOOOOO DC OOOOO HC OOOOO OO OOO RC OOOO"

[0044] Among them, Inci represents an accident, HC represents the hazardous chemicals subject relationship, DC and RC represent the direct cause and indirect cause relationship respectively.

[0045] Preferably, in step S3, the knowledge extraction based on multi-task learning and end-to-end neural network model includes the following steps:

[0046] S3-1: Pre-train the RoBERTa model on the hazardous chemical accident corpus after data preprocessing to build a RoBERTa model for the hazardous chemical accident field;

[0047] The RoBERTa model is an optimized version of the BERT model. Compared with BERT, it uses more training data, more training steps, larger batch sizes, and more model parameters. In terms of the model's training method, RoBERTa removes the Next Sentence Prediction (NSP) task, thereby demonstrating better performance. At the same time, the RoBERTa model changes the static mask to a dynamic mask, so that the position of the mask changes with the number of training rounds, thereby enhancing the model's ability to learn different semantic environments. Compared with the BERT model, RoBERTa can be better generalized to downstream training tasks;

[0048] S3-2: Use the BRAT annotation tool to annotate entity relationships on some sentences in the corpus to generate training corpus;

[0049] BRAT is a text annotation tool based on the B / S mode. It needs to be installed in Linux or OS X environment. It supports annotation tasks including named entity extraction, relationship extraction and event extraction.

[0050] S3-3: To address the problems of uneven label distribution and small data volume in the dataset, synonym replacement and back translation are used for data enhancement;

[0051] S3-4: word vector c t Input RoBERTa model to get rich semantic representation s t ;

[0052] S3-5: Put s t Input the bidirectional long short-term memory network (Bi-LSTM) layer to encode the sentence and obtain the semantic representation h combined with the context t ;

[0053] Bi-LSTM uses a bidirectional LSTM to process the input sequence. The input sequence is input to two LSTM layers in forward and reverse order for feature extraction. The final expression of the sequence is obtained by splicing the outputs of the two LSTMs in opposite directions. Compared with LSTM, which only considers the above limitations, Bi-LSTM integrates the contextual information of words and can better represent the semantics of the input sequence. Structurally, Bi-LSTM is similar to LSTM in structure, both of which contain a forget gate, an input gate, an output gate, and a cell state. The specific calculation formula is as follows:

[0054] Input gate: i t =σ(W i ·[h t-1 , x t ]+b i );

[0055] Forget gate: f t =σ(W f ·[h t-1 , x t ]+b f );

[0056] Output gate: O t =σ(W o ·[h t-1 , x t ]+b o );

[0057] Candidate update content:

[0058] Update cell status:

[0059] The final output hidden state: h t =O t *tanh(C t );

[0060] Among them, x represents the input data, t represents the current time step, and W i , W f , W o and W c Respectively represent the corresponding weight matrix, b i 、b f 、b o and b c They represent the corresponding bias vectors respectively, and σ represents the Sigmoid activation function;

[0061] S3-6: For the hidden state h t Perform maximum pooling and calculate the global context representation g;

[0062] S3-7: The semantic representation h obtained in step S3-5 is t , and the global context representation g obtained in step S3-6 to obtain the final sentence representation x t =[h t , g];

[0063] S3-8: Represent the final sentence as x t , input bidirectional gated recurrent self-attention tagger BGSAT HE In the Bi-GRU of the Bi-GRUSelf-Attention Tagger (BGSAT), the hidden state of the starting position is obtained

[0064] Bi-GRU uses bidirectional neural units and has a more concise structure than Bi-LSTM. The GRU layer of Bi-GRU is a model proposed to solve problems such as long-term memory and gradient in back propagation. Its input and output structure is the same as that of ordinary RNN. At time t, input x t and the previous hidden layer state h t-1 , output y t and the hidden state h to be passed to the next node t The internal structure of GRU includes a reset gate and an update gate. The reset gate determines how to combine the new input information with the previous memory, and the update gate defines the amount of previous memory saved to the current time step. The specific formula is as follows:

[0065] Reset gate: r t =σ(x t W xr +h t-1 W hr +b r );

[0066] Update gate: z t =σ(x t W xz +h t-1 W hz +b z );

[0067] Among them, x represents the input data, t represents the current time step, and W xr , W hr , W xz and W hz Respectively represent the corresponding weight matrix, b r and b z They represent the corresponding bias vectors respectively, and σ represents the Sigmoid activation function;

[0068] The candidate hidden layer state formula is as follows:

[0069] Among them, W hx , W hh Respectively represent the corresponding weight matrix, b h represents the corresponding bias vector;

[0070] The final hidden layer state formula is as follows:

[0071]

[0072] S3-9: The output vector in step S3-8 is Input the self-attention mechanism layer (Self-Attention), calculate the attention score to get the vector representation The calculation formula of the attention mechanism is as follows:

[0073] Among them, dk is the dimension of K;

[0074] The calculation formulas for Q, K, and V are as follows:

[0075] Q=W Q X;

[0076] K=W K X;

[0077] V=W V X;

[0078] Among them, W Q , W K and W V Respectively represent the corresponding weight matrices;

[0079] S3-10: Use a normalized exponential function (Softmax) to process the vector output in step S3-9 Normalize to obtain the start position labeling sequence of the head entity;

[0080] S3-11: Calculate the distance embedding from the current position to the starting position Obtained by finding the trainable distance embedding matrix:

[0081]

[0082] Among them, s* is the starting position closest to the current position. This method explicitly limits the length of the extracted entity to no more than 20, indicating that the end position of the entity cannot be before the starting position;

[0083] S3-12: Embedding distance The same as in step S3-9 Phase splicing, as the input for the end position extraction, we get

[0084] S3-13: Input the Bi-GRU layer to get the hidden state of the end position

[0085] S3-14: Hidden state of the end position Calculate the attention score and output vector representation

[0086] S3-15: Use the Softmax function to process the vector output in step S3-14 Normalize to get the end position label sequence of the head entity;

[0087] S3-16: Match the start position and end position of the same type according to the proximity principle, extract the information from the start position to the end position as the head entity, and obtain the head entity list HE = [he1, he2, ...he m}. Where m represents the number of head entities extracted after head entity extraction;

[0088] S3-17: For a certain entity he extracted i Extract relations and tail entities. Here, i represents the i-th head entity in the head entity list;

[0089] S3-18: According to the head entity he i Get the hidden state h of the head entity he and distance embedding Phase splicing Among them, h he =(h sh +h eh ) / 2,h sh and h eh is the hidden state of the start position and the hidden state of the end position of the head entity; s he Indicates the starting position of the head entity;

[0090] S3-19: b t As Q, the vector x in steps 3-7 t As K and V, calculate the attention score and get the head entity he i The sentence-level vector representation a t ;

[0091] S3-20: Place at Input Bidirectional Gated Recurrent Self-Attention Annotator BGSAT RTE , get the head entity he i The start and end position of the tail entity are marked in sequence. Among them, BGSAT RTE With BGSAT HE Using the same architecture;

[0092] S3-21: According to the proximity principle, the start and end positions of the same type are matched, and the information from the start position to the end position is extracted as the tail entity to obtain the head entity he i List of relationships and tail entities Among them, z represents the number of tail entities extracted after relationship and tail entity extraction, j and k represent predefined relationship types;

[0093] S3-22: BGSAT HE and BGSAT RTE The extractor performs model training and calculates the loss function:

[0094]

[0095]

[0096] in, and are the true start and end tags of the tth word, n represents the length of the input sentence, α and β represent the loss weights of the head entity extraction and relation and tail entity extraction tasks respectively;

[0097] S3-23: Using the trained BGSAT HE Decimator and BGSAT RTE The extractor extracts entity relationships from the data in step S1;

[0098] S3-24: Concatenate the extracted head entity and tail entity and the relationship between them to obtain an entity relationship triple.

[0099] Preferably, in step S4, sorting out the physical and chemical properties of major hazard sources according to the Chemical Safety Data Sheet includes the following steps:

[0100] S4-1: Find the major hazard source entity extracted in step s3, and query its corresponding CAS number in the "Catalogue of Hazardous Chemicals";

[0101] S4-2: Check the Material Safety Data Sheet (MSDS) according to the CAS number corresponding to the major hazard source entity to find out its corresponding melting point, boiling point and storage conditions;

[0102] S4-3: Associate the major hazard source entity with its corresponding attributes (CAS number, melting point, boiling point and storage conditions) to establish an "entity-attribute-attribute value" triple as a supplement to the knowledge extracted in step s3.

[0103] Preferably, in step s5, storing the entity relationship triples into the graph database comprises the following steps:

[0104] S5-1: Save the knowledge in step S3 and step S4 into a csv file, import it into the import folder, and change the encoding format to "UTF-8";

[0105] S5-2: Import the knowledge in step S5-1 into the database, and use the "LOAD_CSV" command to batch import the entities and relationships in the import directory;

[0106] Preferably, in step S6, the semantic annotation of the hazardous chemicals accident knowledge graph includes the following steps:

[0107] S6-1: Count the frequency of entities in the knowledge graph of hazardous chemical accidents and find the entities with the highest frequency and their corresponding triples;

[0108] S6-2: Mark the accident description sentence where the triple in step S6-1 is located, and extract topics from the original sentence using the LDA topic model;

[0109] The LDA topic model is mainly used to summarize the topic distribution of documents, and give the topic of each document in the document set in the form of probability distribution. The LDA topic model does not care about the order of words in the document, and usually uses bag-of-words features to represent documents. The document generation process of the LDA model is as follows:

[0110] According to φ k Sampling the topics:

[0111] φ k ~Dir(β);

[0112] Among them, K is the number of topics;

[0113] Generate topic distribution for document i:

[0114] θ i ~Dir(α);

[0115] Determine the topic of each word j:

[0116] T ij ~Multinomial(θ i );

[0117] Sample the final words from the multinomial distribution of words:

[0118] W ij ~Multinomial(φ-T ij );

[0119] S6-3: In the hazardous chemicals accident knowledge graph, select general keywords and semantically annotate the triples.

[0120] Preferably, in step S7, the accident cause analysis includes taking the accident type as a classification condition and the direct cause of the accident as a search condition, counting the number of occurrences of each direct cause in different types of accidents, and returning the cause with the largest number of occurrences as the main cause of the accident;

[0121] Preferably, in step S7, the accident hazard source analysis includes taking the accident type as a classification condition and the major hazard source as a search condition, respectively counting the number of occurrences of each major hazard source in different types of accidents, and returning the hazard source with the largest number of occurrences as the main hazard source of the accident;

[0122] Preferably, in step S7, the key node analysis of the accident development includes analyzing the associated paths in the accident, focusing on analyzing the overlapping parts of the paths during the accident development process, exploring the laws of accident development, and preparing for accident warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0123] The following is a further understanding of the embodiments of the present invention in conjunction with the accompanying drawings and embodiments:

[0124] Figure 1 A schematic diagram of the life cycle of the hazardous chemicals accident knowledge graph of the present invention;

[0125] Figure 2 It is a structural diagram of the types, attributes and relationships of the hazardous chemicals accident entity described in the present invention;

[0126] Figure 3 This is a structural diagram of entity relationship extraction based on multi-task learning and end-to-end neural network model according to the present invention;

[0127] Figure 4 It is a structural diagram of the BGSAT extractor of the present invention;

[0128] Figure 5 This is a schematic diagram of an accident case described in the present invention; DETAILED DESCRIPTION

[0129] The present invention is described in detail below in conjunction with the accompanying drawings and embodiments, which does not mean that the present invention is limited to this embodiment and cannot be used to limit the protection scope of the present invention. Under the premise of not departing from the principle of the invention, several modifications can be made to the present invention, and these modifications are also within the protection scope of the claims of the present invention.

[0130] The method for constructing and applying a knowledge graph of hazardous chemical accidents based on multi-task learning specifically includes the following steps:

[0131] S1: Collect domestic and foreign hazardous chemicals accident data and clean the collected data;

[0132] In step S1, the following operations are performed to clean the data:

[0133] S1-1: Blank data problem. When the accident description is blank, the data is considered blank and deleted;

[0134] S1-2: Irrelevant data issue. If the accident description does not contain similar chemical descriptions in the "Catalogue of Hazardous Chemicals", the data is considered irrelevant to the hazardous chemical accident and deleted;

[0135] S1-3: Redundant data problem. Extract the date and the name of the hazardous chemical in the accident from two data. If the time and name of the hazardous chemical are the same, it is considered that the two data describe the same accident, and the one with more detailed description is retained;

[0136] S1-4: Invalid information problem. If an accident description does not exceed 15 words, it is considered that the data does not provide valid information and is deleted;

[0137] S2: Construct the ontology of the knowledge graph;

[0138] In step S2, the ontology of the knowledge graph has the following categories and attributes:

[0139] S2-1: Accident type class: Optional attributes include risk type;

[0140] S2-2: Accident category: Optional attributes include accident time, accident location, and accident details;

[0141] S2-3: Major hazard source category: Optional attributes include CAS number, melting point, boiling point, storage conditions, storage form, and inventory;

[0142] S2-4: Equipment and facilities: optional attributes include temperature, pressure, and equipment defects;

[0143] S2-5: Equipment and process: optional attributes include reaction conditions, hazard, and criticality indicators;

[0144] S2-6: Human factors category: Optional attributes include improper operation;

[0145] S2-7: Accident results: optional attributes include number of injured, number of deaths, and economic losses;

[0146] S2-8: Enterprise: Optional attributes include business scope, risk, and contact information.

[0147] The relationship between the categories is as follows:

[0148] S2-9: Causal relationship: There are two types of relationships between equipment facilities, equipment technology, human factors and accidents: direct cause and indirect cause;

[0149] S2-10: Accident result relationship: There is an accident result relationship between the accident result and the accident;

[0150] S2-11: Hazardous chemicals subject relationship: There is a hazardous chemicals subject relationship between major hazard sources and accidents;

[0151] S2-12: Enterprise entity relationship: There is an organizational relationship between the enterprise and the accident;

[0152] S2-13: Accident type relationship: There is an accident type relationship between accident types and accidents.

[0153] S3: Knowledge extraction based on multi-task learning and end-to-end neural network model;

[0154] In step S3, the knowledge extraction based on multi-task learning and end-to-end neural network model includes the following steps:

[0155] S3-1: Pre-train the RoBERTa model on the hazardous chemical accident corpus after data preprocessing to build a RoBERTa model for the hazardous chemical accident field;

[0156] The RoBERTa model is an optimized version of the BERT model. Compared with BERT, it uses more training data, more training steps, larger batch sizes, and more model parameters. In terms of the model's training method, RoBERTa removes the Next Sentence Prediction (NSP) task, thereby demonstrating better performance. At the same time, the RoBERTa model changes the static mask to a dynamic mask, so that the position of the mask changes with the number of training rounds, thereby enhancing the model's ability to learn different semantic environments. Compared with the BERT model, RoBERTa can be better generalized to downstream training tasks;

[0157] S3-2: Use the BRAT annotation tool to annotate entity relationships on some sentences in the corpus to generate training corpus;

[0158] BRAT is a text annotation tool based on the B / S mode. It needs to be installed in Linux or OS X environment. It supports annotation tasks including named entity extraction, relationship extraction and event extraction.

[0159] S3-3: To address the problems of uneven label distribution and small data volume in the dataset, synonym replacement and back translation are used for data enhancement;

[0160] S3-4: word vector c t Input RoBERTa model to get rich semantic representation s t ;

[0161] S3-5: Put s t Input the bidirectional long short-term memory network (Bi-LSTM) layer to encode the sentence and obtain the semantic representation h combined with the context t ;

[0162] Bi-LSTM uses a bidirectional LSTM to process the input sequence. The input sequence is input to two LSTM layers in forward and reverse order for feature extraction. The final expression of the sequence is obtained by splicing the outputs of the two LSTMs in opposite directions. Compared with LSTM, which only considers the above limitations, Bi-LSTM integrates the contextual information of words and can better represent the semantics of the input sequence. Structurally, Bi-LSTM is similar to LSTM in structure, both of which contain a forget gate, an input gate, an output gate, and a cell state. The specific calculation formula is as follows:

[0163] Input gate: i t =σ(W i ·[h t-1 , x t ]+b i );

[0164] Forget gate: f t =σ(W f ·[h t-1 , x t ]+b f );

[0165] Output gate: O t =σ(W o ·[h t-1 , x t ]+b o );

[0166] Candidate update content:

[0167] Update cell status:

[0168] The final output hidden state: h t =o t *tanh(C t );

[0169] Among them, x represents the input data, t represents the current time step, and W i , W f , W o and W c Respectively represent the corresponding weight matrix, b i 、b f 、b o and b c They represent the corresponding bias vectors respectively, and σ represents the Sigmoid activation function;

[0170] S3-6: For the hidden state h t Perform maximum pooling and calculate the global context representation g;

[0171] S3-7: The semantic representation obtained in step S3-5 is It is concatenated with the global context representation g obtained in step S3-6 to obtain the final sentence representation x t =[h t , g];

[0172] S3-8: Represent the final sentence as x t , input bidirectional gated recurrent self-attention tagger BGSAT HE In the Bi-GRU of the Bi-GRUSelf-Attention Tagger (BGSAT), the hidden state of the starting position is obtained

[0173] Bi-GRU uses bidirectional neural units and has a more concise structure than Bi-LSTM. The GRU layer of Bi-GRU is a model proposed to solve problems such as long-term memory and gradient in back propagation. Its input and output structure is the same as that of ordinary RNN. At time t, input x t and the previous hidden layer state h t-1 , output y t and the hidden state h to be passed to the next node t The internal structure of GRU includes a reset gate and an update gate. The reset gate determines how to combine the new input information with the previous memory, and the update gate defines the amount of previous memory saved to the current time step. The specific formula is as follows:

[0174] Reset gate: r t =σ(x t Wxr +h t-1 W hr +b r );

[0175] Update gate: z t =σ(x t W xz +h t-1 W hz +b z );

[0176] Among them, x represents the input data, t represents the current time step, and W xr , W hr , W xz and W hz Respectively represent the corresponding weight matrix, b r and b z They represent the corresponding bias vectors respectively, and σ represents the Sigmoid activation function;

[0177] The candidate hidden layer state formula is as follows:

[0178] Among them, W hx , W hh Respectively represent the corresponding weight matrix, b h represents the corresponding bias vector;

[0179] The final hidden layer state formula is as follows:

[0180]

[0181] S3-9: The output vector in step s3-8 is Input the self-attention mechanism layer (Self-Attention), calculate the attention score to get the vector representation The calculation formula of the attention mechanism is as follows:

[0182] Among them, d k is the dimension of K;

[0183] The calculation formulas for Q, K, and V are as follows:

[0184] Q=W Q X;

[0185] K=W K X;

[0186] V=W V X;

[0187] Among them, W Q , W K and W VRespectively represent the corresponding weight matrices;

[0188] s3-10: Use the normalized exponential function (Softmax) to process the vector output in step S3-9 Normalize to obtain the start position labeling sequence of the head entity;

[0189] S3-11: Calculate the distance embedding from the current position to the starting position Obtained by finding the trainable distance embedding matrix:

[0190]

[0191] Among them, s* is the starting position closest to the current position. This method explicitly limits the length of the extracted entity to no more than 20, indicating that the end position of the entity cannot be before the starting position;

[0192] S3-12: Embedding distance The same as in step S3-9 Phase splicing, as the input for the end position extraction, we get

[0193] S3-13: Input the Bi-GRU layer to get the hidden state of the end position

[0194] S3-14: Hidden state of the end position Calculate the attention score and output vector representation

[0195] S3-15: Use the Softmax function to process the vector output in step S3-14 Normalize to get the end position label sequence of the head entity;

[0196] S3-16: Match the start position and end position of the same type according to the proximity principle, extract the information from the start position to the end position as the head entity, and obtain the head entity list HE = {he1, he2, ...he m}. Where m represents the number of head entities extracted after head entity extraction;

[0197] S3-17: For a certain entity he extracted j Extract relations and tail entities. Here, i represents the i-th head entity in the head entity list;

[0198] s3-18: According to the head entity he i Get the hidden state h of the head entityhe and distance embedding Phase splicing Among them, h he =(h sh +h eh ) / 2,h sh and h eh is the hidden state of the start position and the hidden state of the end position of the head entity; s he Indicates the starting position of the head entity;

[0199] S3-19: b t As Q, the vector x in steps 3-7 t As K and V, calculate the attention score and get the head entity he i The sentence-level vector representation a t ;

[0200] S3-20: Place a t Input Bidirectional Gated Recurrent Self-Attention Annotator BGSAT RTE , get the head entity he i The start and end position of the tail entity are marked in sequence. Among them, BGSAT RTE With BGSAT HE Using the same architecture;

[0201] S3-21: According to the proximity principle, the start and end positions of the same type are matched, and the information from the start position to the end position is extracted as the tail entity to obtain the head entity he i List of relationships and tail entities Among them, z represents the number of tail entities extracted after relationship and tail entity extraction, j and k represent predefined relationship types;

[0202] S3-22: BGSAT HE and BGSAT RTE The extractor performs model training and calculates the loss function:

[0203]

[0204]

[0205] in, and are the true start and end tags of the tth word, n represents the length of the input sentence, α and p represent the loss weights of the head entity extraction and relation and tail entity extraction tasks respectively;

[0206] S3-23: Using the trained BGSAT HEDecimator and BGSAT RTE The extractor extracts entity relationships from the data in step s1;

[0207] S3-24: Concatenate the extracted head entity and tail entity and the relationship between them to obtain an entity relationship triple.

[0208] S4: Sort out the physical and chemical properties of major hazardous sources according to the Chemical Safety Data Sheet and supplement the extracted knowledge;

[0209] In step S4, sorting out the physical and chemical properties of major hazardous sources according to the Chemical Safety Data Sheet includes the following steps:

[0210] S4-1: Find the major hazard source entity extracted in step S3, and query its corresponding CAS number in the "Catalogue of Hazardous Chemicals";

[0211] S4-2: Check the Material Safety Data Sheet (MSDS) according to the CAS number corresponding to the major hazard source entity to find out its corresponding melting point, boiling point and storage conditions;

[0212] S4-3: Associate the major hazard source entity with its corresponding attributes (CAS number, melting point, boiling point and storage conditions) to establish an "entity-attribute-attribute value" triple as a supplement to the knowledge extracted in step s3.

[0213] S5: Store entity relationship triples into the graph database to build a knowledge graph of hazardous chemical accidents;

[0214] In step s5, storing the extracted entity relationship triples into the graph database comprises the following steps:

[0215] S5-1: Save the knowledge in step S3 and step S4 into a csv file, import it into the import folder, and change the encoding format to "UTF-8";

[0216] S5-2: Import the knowledge in step S5-1 into the database, and use the "LOAD_CSV" command to batch import the entities and relationships in the import directory;

[0217] S6: semantic annotation of hazardous chemicals accident knowledge graph;

[0218] In step S6, the semantic annotation of the hazardous chemicals accident knowledge graph includes the following steps:

[0219] S6-1: Count the frequency of entities in the knowledge graph of hazardous chemical accidents and find the entities with the highest frequency and their corresponding triples;

[0220] S6-2: Mark the accident description sentence where the triple in step S6-1 is located, and extract topics from the original sentence using the LDA topic model;

[0221] The LDA topic model is mainly used to summarize the topic distribution of documents, and give the topic of each document in the document set in the form of probability distribution. The LDA topic model does not care about the order of words in the document, and usually uses bag-of-words features to represent documents. The document generation process of the LDA model is as follows:

[0222] According to φ k Sampling the topics:

[0223] φ k ~Dir(β);

[0224] Among them, K is the number of topics;

[0225] Generate topic distribution for document i:

[0226] θ i ~Dir(α);

[0227] Determine the topic of each word j:

[0228] T ij ~Multinomial(θ i );

[0229] Sample the final words from the multinomial distribution of words:

[0230] W ij ~Multinomial(φ-T ij );

[0231] S6-3: In the hazardous chemicals accident knowledge graph, select general keywords and semantically annotate the triples.

[0232] S7: Based on the constructed knowledge graph of hazardous chemical accidents, analyze the causes of accidents, sources of accident hazards and key nodes of accident development;

[0233] In step S7, the accident cause analysis includes taking the accident type as a classification condition and the direct cause of the accident as a search condition, counting the number of occurrences of each direct cause in different types of accidents, and returning the cause with the largest number of occurrences as the main cause of the accident;

[0234] In step S7, the accident hazard source analysis includes taking the accident type as the classification condition and the major hazard source as the search condition, counting the number of occurrences of each major hazard source in different types of accidents, and returning the hazard source with the largest number of occurrences as the main hazard source of the accident;

[0235] In step S7, the key node analysis of the accident development includes analyzing the associated paths in the accident, focusing on analyzing the overlapping parts of the paths during the accident development process, exploring the laws of accident development, and preparing for accident warning.

[0236] The specific embodiments described in the present invention are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications, supplements, explanations, or substitute similar methods to the specific embodiments described, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

[0237] The specific embodiments of the present invention are for illustration only and are not intended to be limiting. Those skilled in the art may make several modifications to the specific embodiments or replace them with similar methods without departing from the general principles of the present invention or exceeding the scope defined in the claims.

Claims

1. A method for constructing and applying a knowledge graph of hazardous chemical accidents based on multi-task learning, characterized in that: The specific steps include: S1: Collect domestic and foreign hazardous chemicals accident data and clean the collected data; S2: Construct the ontology of the knowledge graph; S3: Knowledge extraction based on multi-task learning and end-to-end neural network model; S4: Sort out the physical and chemical properties of major hazardous sources according to the Chemical Safety Data Sheet and supplement the extracted knowledge; S5: Store entity relationship triples into the graph database to build a knowledge graph of hazardous chemical accidents; S6: semantic annotation of hazardous chemicals accident knowledge graph; S7: Based on the constructed knowledge graph of hazardous chemicals accidents, analyze the causes of accidents, accident hazards and key nodes of accident development.

2. According to claim 1, a method for constructing and applying a knowledge graph of hazardous chemical accidents based on multi-task learning is characterized in that: In step S2, the basic terms and their relationships are defined using an ontology building tool according to the Stanford seven-step method. The established ontology has the following categories and attributes: (1) Accident type class: optional attributes include risk type; (2) Accident category: Optional attributes include accident time, accident location, and accident details; (3) Major hazard source category: optional attributes include CAS number, melting point, boiling point, storage conditions, storage form, and inventory; (4) Equipment and facilities: optional attributes include temperature, pressure, and equipment defects; (5) Equipment and process: optional attributes include reaction conditions, hazards, and critical indicators; (6) Human factors: optional attributes include improper operation; (7) Accident results: optional attributes include number of injured, number of deaths, and economic losses; (8) Enterprise: Optional attributes include business scope, risk, and contact information. The relationship between the categories is as follows: (1) Causal relationship: There are two types of relationships between equipment facilities, equipment technology, human factors and accidents: direct cause and indirect cause; (2) Accident-result relationship: There is an accident-result relationship between the accident results and the accidents; (3) Hazardous chemicals subject relationship: There is a hazardous chemicals subject relationship between major hazard sources and accidents; (4) Enterprise entity relationship: There is an organizational relationship between the enterprise and the accident; (5) Accident type relationship: There is an accident type relationship between accident types.

3. According to claim 1, a method for constructing and applying a knowledge graph of hazardous chemical accidents based on multi-task learning is characterized in that: In step S3, the boundary labeling method is used to mark the start and end positions of the entity, and the middle position of the entity and the remaining non-entities are uniformly marked with "O". The head entity and the tail entity are marked with entity type labels and relationship type labels respectively. For example, "An explosion occurred at Yunnan xx Company. Due to equipment defects, carbon four materials leaked and encountered the open flame of the incinerator, which ignited and exploded." The manual labeling is: (1) Head-Entity (HE) annotation: Start notation: "Inci OOOOOOOOOOOOOOOOOOOOOOO OOOOOOO OO OOOOOOO" End mark: "OOOOOOOOOOO Inci OOOOOOOOOOOOOOOOOOO OO OOOOOOO" (2) Relation and Tail-Entity (RTE) annotation: Extract relations and tail entities for the head entity Inci: Start labeling: "OOOOOOOOOOOOO DC OOOOO HC OOOOOOOOOO OO RC OOOOO" End mark: "OOOOOOOOOOOOOOOO DC OOOOO HC OOOOOOO OO O RC OOOO" Among them, Inci represents an accident, HC represents the hazardous chemicals subject relationship, DC and RC represent the direct cause and indirect cause relationship respectively.

4. According to claim 1, a method for constructing and applying a knowledge graph of hazardous chemical accidents based on multi-task learning is characterized in that: The specific processing flow in step S3 is as follows: S3-1: Pre-train the RoBERTa model on the hazardous chemical accident corpus after data preprocessing to build a RoBERTa model for the hazardous chemical accident field; S3-2: Use the BRAT annotation tool to annotate entity relationships on some sentences in the corpus to generate training corpus; S3-3: To address the problems of uneven label distribution and small data volume in the dataset, synonym replacement and back translation are used for data enhancement; S3-4: word vector c t Input RoBERTa model to get rich semantic representation s t ; S3-5: t Input Bi-LSTM to encode the sentence and obtain the semantic representation h combined with the context t ; S3-6: For the hidden state h t Perform maximum pooling and calculate the global context representation g; S3-7: The semantic representation h obtained in step S3-5 is t , concatenated with the global context representation g obtained in step S3-6 to obtain the final sentence representation x t =[h t ,g]; S3-8: Represent the final sentence as x t Input Bidirectional Gated Recurrent Self-Attention Annotator BGSAT HE In the bidirectional gated recurrent unit (Bi-GRU) of (Bi-GRU Self-Attention Tagger, BGSAT), the hidden state of the starting position is obtained S3-9: The output vector in step S3-8 is Input the self-attention mechanism layer (Self-Attention), calculate the attention score to get the vector representation S3-10: Use a normalized exponential function (Softmax) to process the vector output in step S3-9 Normalize to obtain the start position labeling sequence of the head entity; S3-11: Calculate the distance embedding from the current position to the starting position Obtained by finding the trainable distance embedding matrix: Among them, s* is the starting position closest to the current position. This method explicitly limits the length of the extracted entity to no more than 20, indicating that the end position of the entity cannot be before the starting position. S3-12: Embedding distance The same as in step S3-9 Phase splicing, as the input for the end position extraction, we get S3-13: Input the Bi-GRU layer to get the hidden state of the end position S3-14: Hidden state of the end position Calculate the attention score and output vector representation S3-15: Use the Softmax function to process the vector output in step S3-14 Normalize to get the end position label sequence of the head entity; S3-16: Match the start position and end position of the same type according to the proximity principle, extract the information from the start position to the end position as the head entity, and obtain the head entity list HE = {he1, he2, ...he m }. Where m represents the number of head entities extracted after head entity extraction; S3-17: For a certain entity he extracted i Extract relations and tail entities. Where i represents the i-th head entity in the head entity list; S3-18: According to the head entity he i Get the hidden state h of the head entity he and distance embedding Phase splicing Among them, h he =(h sh +h eh ) / 2,h sh and h eh is the hidden state of the start position and the hidden state of the end position of the head entity; s he Indicates the starting position of the head entity; S3-19: b t As Q, the vector x in step 3-7 t As K and V, calculate the attention score and get the head entity he i The sentence-level vector representation a t ; S3-20: Place a t Input Bidirectional Gated Recurrent Self-Attention Annotator BGSAT RTE , get the head entity he i The start and end position of the tail entity are marked in sequence. Among them, BGSAT RTE With BGSAT HE Using the same architecture; S3-21: According to the proximity principle, the start and end positions of the same type are matched, and the information from the start position to the end position is extracted as the tail entity to obtain the head entity he i List of relationships and tail entities Among them, z represents the number of tail entities extracted after relationship and tail entity extraction, j and k represent predefined relationship types; S3-22: BGSAT HE and BGSAT RTE The extractor performs model training and calculates the loss function: in, and are the true start and end tags of the tth word, n represents the length of the input sentence, α and β represent the loss weights of the head entity extraction and relation and tail entity extraction tasks respectively; S3-23: Using the trained BGSAT HE Decimator and BGSAT RTE The extractor extracts entity relationships from the data in step S1; S3-24: Concatenate the extracted head entity and tail entity and the relationship between them to obtain an entity relationship triple.

5. According to claim 1, a method for constructing and applying a knowledge graph of hazardous chemical accidents based on multi-task learning is characterized in that: The specific processing flow in step S4 is as follows: S4-1: Find the major hazard source entity extracted in step S3, and query its corresponding CAS number in the "Catalogue of Hazardous Chemicals"; S4-2: Check the Material Safety Data Sheet (MSDS) according to the CAS number corresponding to the major hazard source entity to find out its corresponding melting point, boiling point and storage conditions; S4-3: Associate the major hazard source entity with its corresponding attributes (CAS number, melting point, boiling point, storage conditions) to establish an "entity-attribute-attribute value" triple as a supplement to the knowledge extracted in step S3.

6. According to claim 1, a method for constructing and applying a knowledge graph of hazardous chemical accidents based on multi-task learning is characterized in that: The specific processing flow in step S5 is as follows: S5-1: Save the knowledge in step S3 and step S4 into a csv file, import it into the import folder, and change the encoding format to "UTF-8"; S5-2: Import the knowledge in step S5-1 into the database, and use the "LOAD_CSV" command to batch import the entities and relationships in the import directory.

7. According to claim 1, a method for constructing and applying a knowledge graph of hazardous chemical accidents based on multi-task learning is characterized in that: The specific processing flow in step S6 is as follows: S6-1: Count the frequency of entities in the knowledge graph of hazardous chemical accidents and find the entities with the highest frequency and their corresponding triples; S6-2: Mark the accident description sentence where the triple in step S6-1 is located, and extract topics from the original sentence using the LDA topic model; S6-3: In the hazardous chemicals accident knowledge graph, select general keywords and semantically annotate the triples.

8. The method for constructing and applying a knowledge graph of hazardous chemical accidents based on multi-task learning according to claim 1 is characterized in that: The specific processing flow in step S7 is as follows: The accident cause analysis includes taking the accident type as a classification condition and the direct cause of the accident as a search condition, respectively counting the number of occurrences of each direct cause in different types of accidents, and returning the cause with the largest number of occurrences as the main cause of the accident; The accident hazard source analysis includes taking the accident type as a classification condition and the major hazard source as a search condition, respectively counting the number of occurrences of each major hazard source in different types of accidents, and returning the hazard source with the largest number of occurrences as the main hazard source of the accident; The key node analysis of the accident development includes analyzing the associated paths in the accident, focusing on analyzing the overlapping parts of the paths during the accident development process, exploring the laws of accident development, and preparing for accident warning.

Images