Dangerous chemical accident named entity identification method based on deep learning

Abstract

The invention relates to a dangerous chemical accident named entity identification method based on deep learning. The method comprises the following steps: firstly, acquiring dangerous chemical accident data, preprocessing the dangerous chemical accident data, and further constructing a dangerous chemical data set; based on the hazardous chemical substance data set, constructing a hazardous chemical substance electronic dictionary and a hazardous chemical substance entity corpus; on the basis, an ERNIE-BiGRU-Multi-HeadAttention-CRF deep network model is constructed, and named entity recognition is carried out on the accident description text; meanwhile, performing entity alignment on the accident description text according to a dangerous chemical entity corpus to obtain an identification result; and finally, a result obtained through deep learning model recognition and an entity corpus alignment result are fused to obtain a final entity recognition result. The dangerous chemical accident named entity recognition method has the advantages that the accuracy of dangerous chemical accident named entity recognition can be improved, researchers are helped to deeply analyze accident occurrence reasons and development laws to attract accident training, and reference is provided for follow-up handling and early warning work of dangerous chemical accidents.

Description

Technical Field

[0001] The present invention relates to the fields of natural language processing and chemical safety, and in particular to a method for identifying named entities of hazardous chemical accidents based on deep learning, which is used for extracting accident entities from hazardous chemical accident cases. Background Art

[0002] Named Entity Recognition (NER) is one of the hot research directions in natural language processing and a subtask of information extraction. Its main goal is to locate and classify named entities in text into predefined categories.

[0003] In the chemical industry, chemicals have different properties. Some products are flammable, explosive, toxic, harmful and highly corrosive. Any equipment hidden dangers, system defects, procedural loopholes, work negligence or personal violations may lead to accidents and bring serious consequences. The purpose of accident research is to fully understand the process of accidents, analyze the causes and development laws of accidents, and learn lessons from accidents. Through in-depth research on accidents, identifying the root causes of accidents, revealing possible systemic problems and management loopholes, and providing a basis for formulating targeted rectification plans, it is helpful to improve the safety management capabilities of enterprises and prevent and reduce the occurrence of accidents. The purpose of named entity recognition of hazardous chemical accidents is to extract accident entities from the text of hazardous chemical accident cases. By accurately identifying accident entities, a unified information database can be established to facilitate the collation and analysis of accident cases, provide lessons for similar industries, and promote the improvement of safety management and risk control capabilities in related fields. In summary, named entity recognition of hazardous chemical accidents is of great significance to improving the safety level of the chemical industry and reducing accident risks.

[0004] In the field of natural language processing, named entity recognition methods have gone through a process from early dictionary- and rule-based methods to traditional machine learning methods and then to the current mainstream deep learning methods. Early methods relied on professionals in specific fields to formulate domain knowledge rules for matching and identifying named entities, but were limited by the coverage of rules and the completeness of dictionaries; in machine learning methods, statistical models and classification algorithms were used for feature extraction and entity classification, but there were certain limitations when dealing with complex contexts and large-scale data. In recent years, deep learning methods have been widely used in named entity recognition tasks and have achieved remarkable results. This method can automatically extract feature representations by learning semantic and contextual information in large-scale data according to the characteristics of the task, and achieve more accurate results in the boundaries and type classification of named entities, further improving the performance of named entity recognition. Summary of the invention

[0005] The purpose of the present invention is to supplement the research related to named entity recognition of hazardous chemical accidents in view of the shortcomings of the current technology, and propose a named entity recognition method for hazardous chemical accidents based on deep learning. The ERNIE pre-training model is added to the BiGRU-CRF model to improve the semantic representation ability of Chinese text. BiGRU can avoid the problem of partial entity recognition errors caused by the error propagation of missing semantic information to a certain extent. At the same time, the multi-head self-attention mechanism is introduced in the BiGRU module to enhance the key features in entity recognition and weaken irrelevant features, which can effectively solve the problems of missing information and insufficient utilization in the field of hazardous chemicals in the existing named entity recognition model. In addition, the CRF layer outputs the optimal label sequence according to the constraint relationship between the labels. Finally, the entity corpus constructed by the electronic dictionary of hazardous chemicals is combined to correct the results output by the deep learning model to further improve the accuracy of named entity recognition.

[0006] The method of the present invention is achieved in the following ways:

[0007] Step 1: Obtain accident case data of hazardous chemicals (hereinafter referred to as hazardous chemicals) and construct a data set;

[0008] Step 2: Build an electronic dictionary of hazardous chemicals;

[0009] Step 3: Definition of hazardous chemical accident entities and entity labeling specifications;

[0010] Step 4: Establish a physical corpus based on the electronic dictionary of hazardous chemicals;

[0011] Step 5: Perform entity recognition on hazardous chemicals data based on the constructed deep learning (ERNIE-BiGRU-Multi-HeadAttention-CRF, EBMHA-CRF) model and perform entity alignment based on the hazardous chemicals entity corpus;

[0012] Step 6: Realize the fusion of named entities of hazardous chemical accidents to obtain the final recognition result.

[0013] Preferably, the description of the hazardous chemicals accident case data content should include:

[0014] The source of the hazardous chemical accident, the time of the accident, the location of the accident, the accident unit, the type of accident (at least one of explosion, leakage, and fire), unsafe human behavior, regulatory organizations, rules and regulations, casualties, economic losses, and the relationship between the accidents (direct cause, indirect cause, and root cause).

[0015] Preferably, the sources of obtaining hazardous chemicals accident case data include the following three aspects:

[0016] Through cooperative chemical companies, we sort out accident data during the production process: the companies provide accident records that occur during the production process;

[0017] Web crawling of hazardous chemical accident data: writing crawler code to crawl domestic and foreign hazardous chemical accident web page data;

[0018] Manual collection: Data that cannot be directly obtained from the Internet are manually collected by professionals in the chemical industry as data supplement.

[0019] Preferably, the steps of constructing a data set based on hazardous chemicals accident case data are as follows:

[0020] Step 1.1: Original data construction. The original data set is constructed by integrating the enterprise cooperation data with the hazardous chemicals accident data crawled from the Internet and the manually collected data;

[0021] Step 1.2: Deduplication. Deduplication is performed on the hazardous chemicals accident text data, and the deduplication hazardous chemicals accident text data is stored;

[0022] Step 1.3: Text data preprocessing. Preprocess the text data of the accident with hazardous chemicals removed, including removing useless characters, completing missing information, deleting erroneous sentences, and performing sentence segmentation on overly long text data;

[0023] Step 1.4: Data cleaning. Use the regularized matching algorithm to filter and clean the processed hazardous chemical accident data, remove non-hazardous chemical related information, and obtain filtered and cleaned data.

[0024] Preferably, the steps of constructing the electronic dictionary of hazardous chemicals are as follows:

[0025] Step 2.1: Obtain basic knowledge. Obtain Chinese web encyclopedia corpus and hazardous chemicals corpus as well as chemical industry-related knowledge, including: chemical industry dictionary, basic database and term explanation, etc.;

[0026] Step 2.2: Refer to industry standards. The terminology data of the chemical word segmentation dictionary comes from the hazardous chemicals industry standard issued by the General Office of the State Administration of Work Safety: the Material Safety Data Sheets (MSDS) corresponding to the 2,828 hazardous chemicals in the "Catalogue of Hazardous Chemicals (2015 Edition)". Taking a hazardous chemical as an example, such as synthetic ammonia, the MSDS includes the name of the synthetic ammonia hazardous chemical, alias, CAS number, physical hazards, health hazards, environmental hazards, ingredients / composition information, first aid measures, fire fighting measures, emergency treatment of leaks, stability, reactivity, waste treatment and transportation information, etc.;

[0027] Step 2.3: Terminology completion. Consult industry experts in the field and use the common terms obtained from production data of chemical companies and feedback from front-line workers in the chemical industry as the completion content;

[0028] Step 2.4: Word segmentation pre-training. Use the Word2Vec pre-trained language model to perform Jieba word segmentation training on the encyclopedia corpus to obtain the encyclopedia word vector;

[0029] Step 2.5: Optimization. Perform optimized word segmentation training on the chemical engineering corpus related knowledge to obtain chemical engineering word vectors;

[0030] Step 2.6: Generate a dictionary. Combine the encyclopedia word vector with the chemical word vector, and calibrate them according to the professional dictionary terms in step 2.2 to obtain the final electronic dictionary of hazardous chemicals.

[0031] Preferably, the specific features of the standard definition and entity annotation of the hazardous chemical accident entity in step 3 are as follows:

[0032] Step 3.1: Entity definition. The entity concepts in the hazardous chemical accident data are: accident time, accident location, chemical name, accident cause (direct cause or indirect cause), and result loss. Under the participation and guidance of chemical personnel, the unstructured hazardous chemical accident text is annotated and verified;

[0033] Step 3.2: Entity annotation specification. The annotation structure of BIEO (Begin, Intermediate, End, Other) is used for text data annotation, where "B" represents the entity header, "I" represents the middle part of the entity, "E" represents the end of the entity, and "O" represents non-entity words or punctuation. For example, the manual entity annotation result of "a toxic gas" is "OOB-RISKE-RISKOO";

[0034] Preferably, the steps of establishing a physical corpus based on the electronic dictionary of hazardous chemicals according to step 4 are as follows:

[0035] Step 4.1: Obtain data. Obtain the processed chemical accident data and the constructed hazardous chemicals electronic dictionary data.

[0036] Step 4.2: Database design. Through cooperation with professionals in the chemical industry, the database and table design of the collected data are carried out based on the data in the electronic dictionary of hazardous chemicals.

[0037] Step 4.3: Storage: Based on the electronic dictionary of hazardous chemicals, a corpus of hazardous chemical accident data is constructed and stored in the data table of the database.

[0038] The hazardous chemicals corpus includes hazardous chemicals accident source information and accident information. Among them, hazardous chemicals accident source information includes hazardous chemical name, alias, CAS number, physical hazard, health hazard, environmental hazard, ingredient / composition information, first aid measures, fire fighting measures, leakage emergency treatment, stability and reactivity, waste treatment and transportation information. Accident information includes accident time, accident location, accident unit, accident type (at least one of explosion, leakage, fire), unsafe behavior of people, regulatory organization, rules and regulations, casualties, economic losses and the relationship between accidents (direct cause, indirect cause and root cause).

[0039] Preferably, step 5 trains the data based on the deep learning ERNIE-BiGRU-Multi-Head Attention-CRF model, which has the following characteristics:

[0040] The deep learning model ERNIE-BiGRU-Multi-Head Attention-CRF includes an input representation layer, an information capture layer, an attention layer and a classification output layer.

[0041] Preferably, the training steps are as follows:

[0042] Step 5.1: Input representation layer. Enhanced Representation Through Knowledge Integration (ERNIE) is used as the input layer of the semantic understanding pre-training model. The ERNIE model is optimized based on the BERT model, which can obtain dynamic word vectors, solve the polysemy problem, and enhance the semantic representation ability of the model. In the named entity recognition task, ERNIE has a more accurate understanding of entities, a clearer modeling of entity boundaries, and makes full use of the relationship between characters and words, entities and entities in the corpus, ultimately improving the model's ability to learn the semantics of Chinese text.

[0043] Step 5.2: Information capture layer. The present invention uses a gated recurrent unit (GRU) as the information capture layer model. Compared with the long-short term memory (LSTM) model, the GRU model has the characteristics of faster training speed, fewer parameters, better results, and can achieve global feature extraction of data. This model reduces the three gates to a reset gate and an update gate, where the reset gate determines how to combine the new input information with the previous memory information, and the update gate is used to control the degree to which the state information of the previous moment is input into the current state. The calculations of GRU are expressed as:

[0044] Z t =σ(W z×h t-1 ,X t ]),

[0045] R t =σ(W r ×[h t-1 ,x t ]),

[0046] W t =tanh(W w × t ×h t-1 ,x t ]),

[0047] H t =(1-Z t )×h t-1 +Z t ×W t ,

[0048] In the formula, Z t and R t represents the update gate and reset gate; W t represents the candidate hidden layer state; h t-1 represents the hidden state of the previous node; W represents the weight; H t and h t-1 They represent the hidden layer states at time t and t-1 respectively; σ represents the Sigmoid activation function; tanh represents the tangent activation function; × represents the Hadamard product.

[0049] In order to fully extract the contextual information in the text sequence, a bidirectional GRU model is used. The calculation of BiGRU is expressed as:

[0050]

[0051]

[0052]

[0053] In the formula, and Represent the forward and backward hidden layer states at time t respectively; and Represent the weights of the forward and backward hidden layer states at time t; b t Represents the hidden layer bias at time t.

[0054] Step 5.3: Attention layer. In order to obtain the contextual features of text in multiple subspaces, a multi-head attention mechanism (Multi-HeadAttention) is embedded in BiGRU. The main improvement is that multiple combinations of queries, keys and values ​​are subjected to multiple different linear transformations. The model can learn relevant information in different subspaces, thereby improving the model's focus on expressing the semantics in the field of hazardous chemical information.

[0055]

[0056] MultiHead(Q,K,V)=Concat(head i ,…,head n )W o ,

[0057] in, and Respectively represent the query, key, and value transformation matrices of the i-th head, head i represents the output of the i-th head, Attention() is the attention calculation function, and the Concat function represents the concatenation of each headi; W O Represents the output transformation matrix; MultiHead represents the multi-head attention value.

[0058] Step 5.5: The classification output layer uses a conditional random field (CRF) conditional probability model. For BIEO annotation, "I-PER" will not appear after "B-LOC"; similarly, "I-PER" and "B-LOC" cannot appear in adjacent sequences, so a method is proposed to constrain this condition. CRF constructs a transfer matrix and learns the transition probability between each label. In this way, the generation of invalid label sequences can be greatly reduced. Proofread the correlation between labels to reduce the output probability of incorrect labels. Therefore, adding a CRF layer on the basis of the above model and using CRF for decoding can obtain the globally optimal label sequence. For example, suppose {y1,y2,…,y k} is the corresponding label of the correct sentence, and there exists any input sequence {x1,x2,…,x k}, and its corresponding output sequence {y1,y2,…,y k The conditional probability of} is:

[0059]

[0060] Among them, Z(X) represents the normalization factor, and the basic representation of Z(X) is:

[0061] Z(X)=∑ Yexp(∑ i,k λ k T k (Y i-1 ,Y i,1 )+∑ i,1 u i S i (Y i ,X j (Y i ,X j )),

[0062] Where, T k and S i is the characteristic function, λ i and u i Represent the corresponding weights respectively.

[0063] Step 5.6: Alignment based on the hazardous chemicals entity corpus. Align the data set content with the constructed entity corpus to obtain the same entities in the two; the content is mainly the hazardous chemicals accident source information and accident information mentioned in step 4.3. Among them, the hazardous chemicals accident source information includes the name of hazardous chemicals, alias, CAS number, physical hazards, health hazards, environmental hazards, ingredients / composition information, first aid measures, fire fighting measures, emergency treatment of leaks, stability and reactivity, waste treatment and transportation information. Accident information includes accident time, accident location, accident unit, accident type (at least one of explosion, leak, fire), unsafe human behavior, regulatory organization, rules and regulations, casualties, economic losses and the relationship between accidents (direct cause, indirect cause and root cause).

[0064] Preferably, the step 6 realizes the fusion of named entities of hazardous chemical accidents, which means fusing the results obtained by aligning and matching the entity corpus with the entity recognition results based on the deep learning ERNIE-BiGRU-Multi-HeadAttention-CRF model, as follows:

[0065] For the entity difference part: further analyze its characteristics and find out the reasons why another method cannot recognize it, such as incomplete entity corpus or poor performance of deep learning model on such entities. Consider expanding the corpus or optimizing the deep learning model to improve recognition ability;

[0066] For the parts with the same labels in the entity intersection: compare the corpus alignment matching results with the results generated by the deep learning model, and output the entities with the same labels as the entities of this type to improve the accuracy of entity recognition;

[0067] For different entity labels in entity intersections: perform annotation correction with reference to the entity corpus to ensure the accuracy of the entity.

[0068] The invention proposes a method for identifying named entities of hazardous chemical accidents based on deep learning, which has the following advantages:

[0069] (I) This invention marks the definitions of named entities of hazardous chemical accidents, providing a basis for subsequent research on entity marking and the construction of an entity corpus of an electronic dictionary of hazardous chemicals. Named entity terminology in the chemical industry is relatively standardized, and entity alignment is performed using an entity corpus constructed using an electronic dictionary of hazardous chemicals. At the same time, a deep learning model performs named entity recognition on unstructured accident description texts, saving manual resources. Finally, the entity corpus alignment matching results and the results of deep learning named entity recognition are integrated to improve the accuracy of entity recognition, laying the foundation for clarifying the accident process, analyzing the cause of the accident, studying the law of accident development, drawing lessons, and improving the safety management capabilities of enterprises to prevent accidents from happening again.

[0070] (II) The present invention introduces a multi-head attention mechanism module based on the deep learning ERNIE-BiGRU-CRF model, which mainly solves the problem that the word vector obtained by the model during the training process cannot fully express the semantics of the field of hazardous chemical accidents, and when obtaining contextual semantic information, the semantic information obtained by the BiGRU model cannot fully express the global semantic information of the input text. Therefore, the self-attention mechanism module is introduced as an additional module of BiGRU to enhance the model's ability to mine global information and sentence correlation, and better apply it to the field of hazardous chemical information.

[0071] (III) In the absence of Chinese hazardous chemical accident data resources, the present invention constructs a hazardous chemical accident data set by cooperating with chemical companies and collecting data through crawler programs and manual collection, thus solving the problem of the lack of Chinese hazardous chemical data.

[0072] (IV) Download Material Safety Data Sheets (MSDS) of 2,828 hazardous chemicals to provide professional knowledge support for building the entity corpus. Among them, 2,828 hazardous chemicals are from the Catalogue of Hazardous Chemicals (2015 Edition).

[0073] (V) Build an entity corpus based on the electronic dictionary of hazardous chemicals and align the hazardous chemicals data set; and merge it with the entity results identified by the deep learning model to finally obtain the named entities of hazardous chemical accidents, thereby improving the accuracy of named entity recognition of hazardous chemical accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The present invention is further described below in conjunction with the accompanying drawings, which constitute a part of the specification but do not constitute a limitation to the embodiments of the present invention.

[0075] Attached Figure 1 This is an overall flow chart of a method for named entity recognition of hazardous chemical accidents based on deep learning;

[0076] Attached Figure 2 A schematic diagram of constructing a data set in the method of the present invention;

[0077] Attached Figure 3 A schematic diagram of the deep learning ERNIE-BiGRU-Multi-HeadAttention-CRF model in the method of the present invention;

[0078] Attached Figure 4 Schematic diagram of the BiGRU structure in the method of the present invention. DETAILED DESCRIPTION

[0079] The following detailed description is made of a method for identifying named entities of hazardous chemical accidents based on deep learning according to the present invention with reference to the accompanying drawings and specific implementations of the specification.

[0080] Embodiment 1

[0081] The embodiment of this application is as follows Figure 1 As shown, a method for identifying named entities of hazardous chemical accidents based on deep learning is provided, comprising the following steps:

[0082] S1. Obtain data on hazardous chemical accident cases; details are as follows:

[0083] S101. Sort out accident data during the production process through cooperative chemical companies: The companies provide accident records that occur during the production process;

[0084] S102. Web crawling of hazardous chemical accident data: Write crawler code to crawl domestic and foreign hazardous chemical accident web page data;

[0085] S103. Manual collection: Data that cannot be directly obtained from the Internet are manually collected by professionals in the chemical industry as data supplement.

[0086] S2, such as Figure 2 As shown, obtain the dangerous chemical accident case data and build a database; the specific steps are as follows:

[0087] S201, original data set construction. The enterprise cooperation data described in S1 is integrated with the dangerous chemical accident data crawled from the Internet to construct the original data;

[0088] S202, deduplication. Deduplication processing is performed on the hazardous chemical accident text data, and the deduplication hazardous chemical accident text data is stored;

[0089] S203, text data preprocessing. Preprocess the text data of the accident with hazardous chemicals removed, including removing useless characters, completing missing information, deleting erroneous sentences, and performing sentence segmentation on overly long text data;

[0090] S204, data cleaning. Use the regularized matching algorithm to filter and clean the processed hazardous chemical accident data, remove non-hazardous chemical related information, and obtain filtered and cleaned data.

[0091] S3. Build an electronic dictionary of hazardous chemicals; the specific steps are as follows:

[0092] S301. Obtain basic knowledge. Obtain Chinese web encyclopedia corpus and hazardous chemicals corpus as well as chemical industry-related knowledge, including: chemical industry dictionary, basic database and term explanation, etc.;

[0093] S302. Reference industry standards. The dictionary terminology data comes from the hazardous chemicals industry standard issued by the General Office of the State Administration of Work Safety: the Material Safety Data Sheets (MSDS) corresponding to the 2,828 hazardous chemicals in the Catalogue of Hazardous Chemicals (2015 Edition). Taking a hazardous chemical as an example, such as synthetic ammonia, the MSDS mainly includes: the name of the hazardous chemical synthetic ammonia, alias, CAS number, physical hazards, health hazards, environmental hazards, ingredients / composition information, first aid measures, fire fighting measures, emergency treatment of leaks, stability, reactivity, waste treatment and transportation information;

[0094] S303, term completion. Consult industry experts in this field, and use the common terms obtained from production data of chemical companies and feedback from front-line workers in the chemical industry as the completion content;

[0095] S304, word segmentation pre-training. Use the Word2Vec pre-trained language model to perform Jieba word segmentation training on the encyclopedia corpus to obtain the encyclopedia word vector;

[0096] S305, Optimization: Optimize word segmentation training on the chemical engineering corpus related knowledge to obtain chemical engineering word vectors.

[0097] S306, generate a dictionary. Combine the encyclopedia word vector and the chemical word vector into a dictionary, and calibrate them according to the professional dictionary terms in S302 to obtain a final electronic dictionary of hazardous chemicals.

[0098] S4. Standard definition and entity labeling of hazardous chemical accidents; the specific features are as follows:

[0099] S401, Entity definition. The entity concepts in the hazardous chemical accident data are: time of accident, location of accident, chemical name, cause of accident (direct or indirect), result loss, and casualties. Under the participation and guidance of chemical personnel, the unstructured hazardous chemical accident text is annotated and verified;

[0100] S402, entity annotation specification. The BIEO (Begin, Intermediate, End, Other) annotation method is used for text data annotation, where "B" represents the first word of the entity, "I" represents the middle part of the entity, "E" represents the end part of the entity, and "O" represents non-entity words or punctuation. For example, the manual entity annotation result of "a poisonous gas" is "OOB-RSIKE-RISKOO".

[0101] S5. Establish a physical corpus based on the electronic dictionary of hazardous chemicals. The specific steps are as follows:

[0102] S501, obtaining data. Obtaining processed hazardous chemical accident data and constructed hazardous chemical electronic dictionary data;

[0103] S502, Database design. Through cooperation with professionals in the chemical field, design the database and table for the collected data based on the electronic chemical dictionary data;

[0104] S503, data storage: Based on the electronic dictionary of hazardous chemicals, a corpus of hazardous chemical accident data entities is constructed and stored in a data table of the database.

[0105] S504. The hazardous chemicals corpus includes hazardous chemicals accident source information and accident information. Among them, hazardous chemicals accident source information includes hazardous chemical names, aliases, CAS numbers, physical hazards, health hazards, environmental hazards, ingredients / composition information, first aid measures, firefighting measures, emergency treatment of leaks, stability and reactivity, waste disposal and transportation information. Accident information includes accident time, accident location, accident unit, accident type (at least one of explosion, leak, fire), unsafe human behavior, regulatory organizations, rules and regulations, casualties, economic losses and the relationship between accidents (direct cause, indirect cause and root cause).

[0106] Embodiment 2

[0107] Example 2 of this application Figure 3 As shown, the data is trained based on the deep learning ERNIE-BiGRU-Multi-Head Attention-CRF model; the specific steps are as follows:

[0108] S6. The deep learning model ERNIE-BiGRU-Multi-Head Attention-CRF includes an input representation layer, an information capture layer, an attention layer and a classification output layer.

[0109] S601, input representation layer, uses Enhanced Representation Through Knowledge Integration (ERNIE) as the semantic understanding pre-training model of the input layer. This framework is improved and optimized based on the BERT model, and can obtain dynamic word vectors, solve the problem of polysemy, and enhance the semantic representation ability of the model. In the named entity recognition task, ERNIE has a more accurate understanding of entities, a clearer modeling of entity boundaries, and makes full use of the relationship between characters and words, entities and entities in the corpus, ultimately improving the model's ability to learn the semantics of Chinese text.

[0110] S602, information capture layer. The present invention uses a gated recurrent unit (GRU) as the information capture layer model. Compared with the long-short term memory model (LSTM), the GRU model has the characteristics of faster training speed, fewer parameters, better effect, and can realize global feature extraction of data. The model reduces the three gates to a reset gate and an update gate, where the reset gate determines how to combine the new input information with the previous memory information, and the update gate is used to control the degree to which the state information of the previous moment is input into the current state. The GRU calculation is expressed as:

[0111] Z t =σ(W z ×h t-1 ,X t ]),

[0112] R t =σ(W r ×[h t-1 ,x t ]),

[0113] W t =tanh(W w × t ×h t-1 ,x t ]),

[0114] H t =(1-Z t )×h t-1 +Z t ×W t ,

[0115] S603, where Z t and R t represents the update gate and reset gate; W t represents the candidate hidden layer state; h t-1 represents the hidden state of the previous node; W represents the weight; H t and h t-1 They represent the hidden layer states at time t and t-1 respectively; σ represents the Sigmoid activation function; tanh represents the tangent activation function; × represents the Hadamard product.

[0116] S604: In order to fully extract the context information in the text sequence, a bidirectional GRU model is used. The calculations of BiGRU are expressed as:

[0117]

[0118]

[0119]

[0120] S605, where and Represent the forward and backward hidden layer states at time t respectively; and Represent the weights of the forward and backward hidden layer states at time t; b t Represents the hidden layer bias at time t.

[0121] S606, attention layer, in order to obtain the contextual features of text in multiple subspaces, a multi-head attention mechanism is adopted. The main improvement is that the combination of multiple queries, keys and values ​​is subjected to multiple linear transformations. The model can learn relevant information in different subspaces, thereby improving the model's focus on expressing the semantics in the field of hazardous chemical information.

[0122]

[0123] MultiHead(Q,K,V)=Concat(head i ,…,head n )W o ,

[0124] in, and Respectively represent the query, key, and value transformation matrices of the i-th head, head i Represents the output of the i-th head, Attention() is the attention calculation function, and Concat() function represents each head i Splicing; W ORepresents the output transformation matrix; MultiHead represents the multi-head attention value.

[0125] S607, classification output layer, using Conditional Random Field (CRF), a conditional probability model. For BIEO annotation, "I-PER" will not appear after "B-LOC", and "I-PER" and "B-LOC" cannot appear in adjacent sequences. Therefore, a method is needed to constrain this condition. CRF constructs a transfer matrix and learns the transition probability between each label. This method can greatly reduce the generation of invalid label sequences. Check the correlation between labels to reduce the output probability of incorrect labels. Therefore, adding a CRF layer on the basis of the above model and using CRF for decoding can obtain the globally optimal label sequence. For example, {y1,y2,…,y k} is the label corresponding to the statement. There exists any input sequence {x1,x2,…,x k}, and its corresponding output sequence {y1,y2,…,y k The conditional probability of} is:

[0126]

[0127] Among them, Z(X) represents the normalization factor, and the basic representation of Z(X) is:

[0128] Z(X)=∑ Y exp(∑ i,k λ k T k (Y i-1 ,Y i,1 )+∑ i,1 u i S i (Y i ,X j (Y i ,X j )),

[0129] Where, T k and S i is the characteristic function, λ i and u i Represent the corresponding weights respectively.

[0130] S608. Align and match based on the hazardous chemicals entity corpus. Align the data set content with the constructed entity corpus to obtain the same entities in the two; the information content is mainly the hazardous chemicals accident source information and accident information mentioned in S504. Among them, the hazardous chemicals accident source information includes the name of the hazardous chemical, alias, CAS number, physical hazards, health hazards, environmental hazards, ingredients / composition information, first aid measures, fire fighting measures, emergency treatment of leaks, stability and reactivity, waste disposal and transportation information. Accident information includes the time of the accident, the location of the accident, the accident unit, the type of accident (at least one of explosion, leakage, and fire), unsafe human behavior, regulatory organizations, rules and regulations, casualties, economic losses, and the relationship between the accident (direct cause, indirect cause, and root cause).

[0131] S7. To achieve named entity fusion of hazardous chemical accidents, the results of entity corpus alignment and matching are integrated with entity recognition results based on deep learning ERNIE-BiGRU-Multi-HeadAttention-CRF model; as follows:

[0132] S701. For the entity difference part: further analyze its characteristics and find out the reasons why another method cannot recognize it, such as incomplete entity corpus or poor performance of deep learning model on such entities. Consider expanding the corpus or optimizing the deep learning model to improve recognition ability;

[0133] S702. For the part with the same entity intersection label: compare the corpus alignment matching result and the result generated by the deep learning model, and directly use the entity labels with the same as the final result to improve the accuracy of entity recognition;

[0134] S703. For different entity labels in entity intersections: perform annotation correction with reference to the entity corpus to ensure the accuracy of the entity.

Claims

1. A method for named entity recognition of hazardous chemical accidents based on deep learning, characterized in that: include: Step 1: Obtain accident case data of hazardous chemicals (hereinafter referred to as hazardous chemicals) and construct a data set; Step 2: Build an electronic dictionary of hazardous chemicals; Step 3: Definition of hazardous chemical accident entities and entity labeling specifications; Step 4: Establish a physical corpus based on the electronic dictionary of hazardous chemicals; Step 5: Perform entity recognition on hazardous chemicals data based on the constructed deep learning (ERNIE-BiGRU-Multi-HeadAttention-CRF, EBMHA-CRF) model and perform entity alignment based on the hazardous chemicals entity corpus; Step 6: Realize the fusion of named entities of hazardous chemical accidents to obtain the final recognition result.

2. According to the method of deep learning-based named entity recognition for hazardous chemical accidents in claim 1, it is characterized in that: The main contents of the description of the hazardous chemicals accident case should include: The source of the hazardous chemical accident, the time of the accident, the location of the accident, the accident unit, the type of accident (at least one of explosion, leakage, and fire), unsafe human behavior, regulatory organizations, rules and regulations, casualties, economic losses, and the relationship between the accidents (direct cause, indirect cause, and root cause).

3. According to the method of dangerous chemical accident named entity recognition based on deep learning in claim 1, it is characterized in that: The sources of data on hazardous chemical accident cases in step 1 include the following three aspects: (1) Sort out accident data during the production process through cooperative chemical companies: the companies provide accident records that occur during the production process; (2) Web crawling of hazardous chemical accident data: Writing crawler code to crawl domestic and foreign web pages of hazardous chemical accident data; (3) Manual collection: Data that cannot be directly obtained from the Internet are manually collected by professionals in the chemical industry as a supplement to the data.

4. According to the method of dangerous chemical accident named entity recognition based on deep learning in claim 1, it is characterized in that: The steps of step 1 to obtain the hazardous chemicals accident case data and construct a data set are as follows: Step 1.1: Original data construction. The enterprise cooperation data described in claim 3 is integrated with the hazardous chemicals accident data crawled from the Internet and the manually collected data to construct the original data set; Step 1.2: Deduplication. Deduplication is performed on the hazardous chemicals accident text data, and the deduplication hazardous chemicals accident text data is stored; Step 1.3: Text data preprocessing. Preprocess the text data of the accident with hazardous chemicals removed, including removing useless characters, completing missing information, deleting erroneous sentences, and performing sentence segmentation on overly long text data; Step 1.4: Data cleaning. Use the regularized matching algorithm to filter and clean the processed hazardous chemical accident data, remove non-hazardous chemical related information, and obtain filtered and cleaned data.

5. According to the method of dangerous chemical accident named entity recognition based on deep learning in claim 1, it is characterized in that: The steps for constructing the electronic dictionary of hazardous chemicals in step 2 are as follows: Step 2.1: Obtain basic knowledge. Obtain Chinese web encyclopedia corpus and hazardous chemicals corpus as well as chemical industry-related knowledge, including: chemical industry dictionary, basic database and term explanation, etc.; Step 2.2: Refer to industry standards. The terminology data of the chemical word segmentation dictionary comes from the hazardous chemicals industry standard issued by the General Office of the State Administration of Work Safety: the Material Safety Data Sheets (MSDS) corresponding to the 2,828 hazardous chemicals in the "Catalogue of Hazardous Chemicals (2015 Edition)". Taking a hazardous chemical as an example, such as synthetic ammonia, the MSDS includes the name of the synthetic ammonia hazardous chemical, alias, CAS number, physical hazards, health hazards, environmental hazards, ingredients / composition information, first aid measures, fire fighting measures, emergency treatment of leaks, stability, reactivity, waste treatment and transportation information, etc.; Step 2.3: Terminology completion. Consult industry experts in the field and use the common terms obtained from production data of chemical companies and feedback from front-line workers in the chemical industry as the completion content; Step 2.4: Word segmentation pre-training. Use the Word2Vec pre-training model to perform Jieba word segmentation training on the encyclopedia corpus to obtain the encyclopedia word vector; Step 2.5: Optimization. Perform optimized word segmentation training on the chemical engineering corpus related knowledge to obtain chemical engineering word vectors; Step 2.6: Generate a dictionary. Combine the encyclopedia word vector with the chemical word vector, and calibrate them according to the professional dictionary terms in step 2.2 to obtain the final electronic dictionary of hazardous chemicals.

6. According to the method of dangerous chemical accident named entity recognition based on deep learning in claim 1, it is characterized in that: The specific features of the hazardous chemicals accident entity definition and entity labeling specification in step 3 are as follows: Step 3.1: Entity definition. The entity concepts in the hazardous chemical accident data are: accident time, accident location, accident risk, chemical name, accident cause (direct or indirect cause), result loss, and casualties. Under the participation and guidance of chemical personnel, the unstructured hazardous chemical accident text is annotated and verified; Step 3.2: Entity annotation specification. The annotation structure of BIEO (Begin, Intermediate, End, Other) is used for text data annotation, where "B" represents the entity header, "I" represents the middle part of the entity, "E" represents the end of the entity, and "O" represents non-entity words or punctuation. For example, the manual entity annotation result of "a kind of toxic gas" is "OO B-RISKE-RISK OO".

7. According to the method of dangerous chemical accident named entity recognition based on deep learning in claim 1, it is characterized in that: The steps for establishing a physical corpus based on the electronic dictionary of hazardous chemicals as described in step 4 are as follows: Step 4.1: Get the data. Obtaining the processed hazardous chemical accident data in claim 4 and the hazardous chemical electronic dictionary data in claim 5; Step 4.2: Database design. Through cooperation with professionals in the chemical industry, the database and table design of the collected data are carried out according to the data in the electronic dictionary of hazardous chemicals; Step 4.3: Storage: Based on the electronic dictionary of hazardous chemicals, a corpus of hazardous chemical accident data is constructed and stored in the data table of the database. The hazardous chemicals corpus includes hazardous chemicals accident source information and accident information. Among them, hazardous chemicals accident source information includes hazardous chemical name, alias, CAS number, physical hazard, health hazard, environmental hazard, ingredient / composition information, first aid measures, fire fighting measures, leakage emergency treatment, stability and reactivity, waste treatment and transportation information. Accident information includes accident time, accident location, accident unit, accident type (at least one of explosion, leakage, fire), unsafe behavior of people, regulatory organization, rules and regulations, casualties, economic losses and the relationship between accidents (direct cause, indirect cause and root cause).

8. According to the method of dangerous chemical accident named entity recognition based on deep learning in claim 1, it is characterized in that: The step 5 is to train the data based on the deep learning EBMHA-CRF model, and the specific steps are as follows: The deep learning EBMHA-CRF model includes an input representation layer, an information capture layer, an attention layer and a classification output layer. Step 5.1: Input representation layer. Enhanced Representation Through Knowledge Integration (ERNIE) is used as the input layer of the semantic understanding pre-training model. The ERNIE model is optimized based on the BERT model, which can obtain dynamic word vectors, solve the polysemy problem, and enhance the semantic representation ability of the model. In the named entity recognition task, ERNIE has a more accurate understanding of entities, a clearer modeling of entity boundaries, and makes full use of the relationship between characters and words, entities and entities in the corpus, ultimately improving the model's ability to learn the semantics of Chinese text. Step 5.2: Information capture layer. The present invention uses a gated recurrent unit (GRU) as the information capture layer model. Compared with the long-short term memory (LSTM) model, the GRU model has the characteristics of faster training speed, fewer parameters, better results, and can achieve global feature extraction of data. The model reduces the three gates to a reset gate and an update gate, where the reset gate determines how to combine the new input information with the previous memory information, and the update gate is used to control the degree to which the state information of the previous moment is input into the current state. The GRU calculation is expressed as: Z t =σ(W z ×[h t-1 ,X t ]), R t =σ(W r ×[h t-1 ,x t ]), W t =tanh(W w ×[r t ×h t-1 ,x t ]), H t =(1-Z t )×h t-1 +Z t ×W t , In the formula, Z t and R t represents the update gate and reset gate; W t represents the candidate hidden layer state; h t-1 represents the hidden state of the previous node; W represents the weight; H t and h t-1 They represent the hidden layer states at time t and t-1 respectively; σ represents the Sigmoid activation function; tanh represents the tangent activation function; × represents the Hadamard product. In order to fully extract the contextual information in the text sequence, a bidirectional GRU model is used. The calculation of BiGRU is expressed as: In the formula, and Represent the forward and backward hidden layer states at time t respectively; and Represent the forward and backward hidden layer state weights at time t; b t Represents the hidden layer bias at time t. Step 5.3: Attention layer. In order to obtain the contextual features of text in multiple subspaces, a multi-head attention mechanism (Multi-HeadAttention) is embedded in BiGRU. The main improvement is that multiple combinations of queries, keys and values ​​are subjected to multiple different linear transformations. The model can learn relevant information in different subspaces, thereby improving the model's focus on expressing the semantics of hazardous chemical information. MultiHead(Q,K,V)=Concat(head i ,...,head n )W o , in, and They represent the query, key, and value transformation matrices of the i-th head, headi represents the output of the i-th head, Attention() is the attention calculation function, and Concat() function represents each head i Splicing; W O Represents the output transformation matrix; MultiHead represents the multi-head attention value. Step 5.5: Classification output layer. A conditional probability model called Conditional Random Field (CRF) is used. For BIEO annotation, "I-PER" will not appear after "B-LOC"; similarly, "I-PER" and "B-LOC" cannot appear in adjacent sequences. Therefore, a method is proposed to constrain this condition. CRF constructs a transfer matrix and learns the transition probability between each label. In this way, the generation of invalid label sequences can be greatly reduced. The correlation between labels is checked to reduce the output probability of incorrect labels. Therefore, a CRF layer is added to the above model, and decoding using CRF can obtain the globally optimal label sequence. For example, suppose {y1, y2, ..., y k } is the corresponding label of the correct sentence, there exists any input sequence {x1, x2, ..., x k }, and its corresponding output sequence {y1, y2, ..., y k The conditional probability of} is: Among them, Z(X) represents the normalization factor, and the basic representation of Z(X) is: Z(X)=∑ Y exp(∑ i，k λ k T k (Y i-1 ,Y i,1 )+∑ i，1 u i S i (Y i ,X j (Y i ,X j )), Where, T k and S i is the characteristic function, λ i and u i Represent the corresponding weights respectively. Step 5.6: Alignment based on the hazardous chemicals entity corpus. Align the data set content with the constructed entity corpus to obtain the same entities in the two; the content is mainly the hazardous chemicals accident source information and accident information mentioned in step 4.3 of claim 7. Among them, the hazardous chemicals accident source information includes the name of the hazardous chemical, alias, CAS number, physical hazards, health hazards, environmental hazards, ingredients / composition information, first aid measures, fire fighting measures, emergency treatment of leaks, stability and reactivity, waste treatment and transportation information. Accident information includes accident time, accident location, accident unit, accident type (at least one of explosion, leak, fire), unsafe human behavior, regulatory organization, rules and regulations, casualties, economic losses and the relationship between the accident (direct cause, indirect cause and root cause).

9. According to the method of dangerous chemical accident named entity recognition based on deep learning in claim 1, it is characterized in that: The step 6 realizes the fusion of named entities of hazardous chemical accidents, which refers to the fusion of hazardous chemical entities obtained by aligning the entity corpus with the hazardous chemical entity recognition results obtained based on the deep learning EBMHA-CRF model, as follows: (1) For the entity difference part: further analyze its characteristics, Find out why another method cannot recognize it, such as incomplete entity corpus or poor performance of deep learning model on such entities. Consider corpus expansion or deep learning model optimization to improve recognition ability; (2) For the parts with the same labels in the entity intersection: compare the corpus alignment matching results with the results generated by the deep learning model, and output the entities with the same labels as the entities of this type to improve the accuracy of entity recognition; (3) For different entity labels in entity intersections: perform annotation correction with reference to the entity corpus to ensure the accuracy of the entity.

Images