A dangerous chemical property entity extraction method and system

Abstract

The application provides a dangerous chemical property entity extraction method and system, which comprises the following steps: obtaining the dangerous chemical name in the msds database, searching the chemical property description text through the name retrieval, and obtaining the chemical property description information text data; first, data cleaning is performed, and the repeated information in the information text is removed, then the BIO labeling method is used for labeling to obtain the overall data, and the overall data is divided into a training set and a test set; in the training stage, the test set is used to test the trained model to obtain a test set; and the training result after training and the test set are stored in a graph database Neo4j. Through the construction of the dictionary information of the chemical dangerous goods, the characteristic vector is obtained through the cnn convolution, the extracted data is saved to the Neo4j graph database, the web application is constructed to search the extracted data, and the problems of the poor single model recognition rate and the relative vacancy of the chemical dangerous goods property entity extraction model are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a property entity extraction method and system of dangerous chemicals. BACKGROUND

[0002] Physical extraction of dangerous chemicals in NLP (NER) is an active research area. Due to the complexity of chemical compounds and the related language describing them, it is a difficult task. There are some open source tools available to support the physical extraction of dangerous chemicals (NER). For example, ChemDataExtractor is an open source library for extracting chemical information from text. It provides a set of rules and algorithms for extracting chemical names, formulas and other features from text. However, it also has some problems, including only being able to handle documents in supported formats and extracting specific types of data, and not being able to extract all types of data, while the efficiency of extraction is also a problem. The latest developments in deep learning and natural language processing provide new opportunities to improve existing methods. Current entity extraction (NER) methods for dangerous chemicals in NLP can include the use of dictionaries and rule-based methods, as well as machine learning algorithms such as support vector machines (SVM), conditional random fields (CRF) and recurrent neural networks (RNN). Due to the complexity of the text, a single simple model may not be able to accurately identify certain chemical names or entities. In addition, these models may not be able to identify different variants of the same chemical name or entity, which can lead to incorrect results. Finally, these methods are limited in identifying chemical entities that are not explicitly referenced in the text. SUMMARY

[0003] The present application provides a property entity extraction method for dangerous chemicals, which can solve the problems of poor recognition rate of single model and relative vacancy of entity extraction model for chemical property in the prior art. The present application also provides a property entity extraction system for dangerous chemicals.

[0004] Technical solution: The property entity extraction method for dangerous chemicals provided by the present application comprises the following steps:

[0005] S1 data acquisition: Obtain the name of the dangerous chemical in the msds database, retrieve the description text of the chemical property through the name, and obtain the property description information text data of the chemical;

[0006] S2 data preprocessing: first, data cleaning is performed, and at the same time, repeated information in the information text is removed, then BIO labeling method is used for labeling to obtain overall data, and the overall data is divided into a training set and a test set;

[0007] S3 training phase:

[0008] S31 word embedding model: input each information text of the training set into the bilstm-based character level model respectively, obtain the joint character vector feature representation through the bilstm-based character level model, input the joint character vector feature representation into the long short-term memory network, and obtain the result Q of modeling the input sequence l ,K l ,V l , among them K l , V l is transmitted to the attention layer, and Q l is transmitted to the transformer encoder layer in the component feature acquisition model;

[0009] S32 component feature acquisition model: first, the Chinese characters in each information text are divided into each single root, the word frequency of the root is counted, then the component in the root is input into the convolutional neural network, and the convolutional neural network is used to extract the component level embedding feature, and the feature is input into the transformer encoder layer;

[0010] S33, the features through the attention layer and the encoder layer are fused, and the label information is output through the conditional random field CRF;

[0011] S34, steps S31-S33 are iterated until the end;

[0012] S4, the trained model is tested by using the test set, and the test set is obtained;

[0013] S5, the training result after training and the test set are stored in the graph database Neo4j.

[0014] Further, comprising:

[0015] In step S31, the bilstm-based character level model comprises:

[0016] Classify the matched characters: first, select a sentence S_1, and cut the sentence S_1 into single characters, after cutting, obtain chars={C1, C2, C3, …, Cn-1, Cn}, wherein n is the length of the sentence character, then take each character Ci, establish the corresponding position information In_i according to the chemical dictionary, and set the position information into four categories, namely: {B: indicates the first position of the matched word; M: indicates the middle position of the matched word; E: indicates the end position of the matched word; S: indicates the single character information}, and the corresponding formula is:

[0017]

[0018]

[0019]

[0020]

[0021] wherein, L represents a dictionary, 1≤i≤n;

[0022] Compression: After obtaining the {B, M, E, S} word sets of the given character, each word set is compressed into a fixed-dimensional vector, and the overall features are obtained using the word weighting method. The corresponding formula of the weighting function is:

[0023]

[0024] wherein, z=∑ w∈BUMUEUS z(w), here the frequency of each word appearing on a static database is used as the weight;

[0025] Joint representation of characters: v s (B), v s (M), v s (E) and v s (S) are respectively calculated, and the information is integrated into the character representation, through the following formula:

[0026] e s (B, M, E, S)=[v s (B);v s (M);v s (E);v s (S)]

[0027] x c ←[x c ;e s (B, M, E, S)].

[0028] Further, including:

[0029] In step S31, the joint character vector feature representation is input into the long short-term memory network, including:

[0030] In each time step of the input sequence, the long short-term memory network takes the joint character vector as input, and retains the information in the previous time step in the hidden state. The long short-term memory network outputs a vector representing the entire sequence, and obtains the attention mechanism Q, K, V, which is represented by the following formula:

[0031] [Q, K, V]=e i [W, I, W] (1)

[0032] Wherein, e represents the word vector feature obtained by the long short-term memory network, I is a unit matrix, and W is a variable parameter learned.

[0033] Further comprising:

[0034] In step S32, the input is input into the convolutional neural network, and the first-level component embedding feature is extracted by the convolutional neural network extraction part, which specifically includes:

[0035] First, the component of the root is input into the convolutional neural network, and the relevant features are extracted by the feature extraction layer of the convolutional neural network. The feature extraction layer includes 30 one-dimensional convolution kernels with a size of 3, and the Bayesian optimization algorithm based on random forest is used to search for the optimal hyperparameters.

[0036] Then, the feature embedding of the first-level component of Chinese characters is realized through the max-pooling layer and the fully connected layer, so as to obtain the component feature of each Chinese character. Finally, the first-level component embedding feature obtained from the convolutional neural network is input into the transformer encoder layer.

[0037] Further comprising:

[0038] The transformer encoder layer includes a head self-attention mechanism and a feedforward neural network. The first-level component embedding feature is obtained by formula (1) Q r , K r , V r , and Q l , K r , V r are input into the self-attention mechanism, Q r is input into the attention layer, the result of the multi-head attention mechanism is output to the feedforward neural network, the activation function is set, and the corresponding feature is output.

[0039] Further comprising:

[0040] The method further comprises:

[0041] The API of the open entity extraction system is opened, and then the task is established to determine whether the demand function is database retrieval or entity extraction;

[0042] If entity extraction is performed, the user inputs the chemical dangerous goods description text to be extracted, creates an extraction task, pre-processes the text, predicts the entity label y through the trained model, and simultaneously retrieves the database to check whether there is already existing information. If there is related information, the extracted result information is referenced to judge the difference, and the existing data information is taken as the correct information to supplement and correct the extracted information. If there is no data, the information is directly input into the database, and the web application program is returned, and the extraction task is completed.

[0043] If it is a database retrieval, it is directly retrieved in the graph database according to the label, and if it exists, the retrieval information is directly returned, and if it does not exist, null is returned.

[0044] On the other hand, the present application also provides a dangerous chemical property entity extraction system, comprising:

[0045] A data acquisition module is configured to obtain the name of a dangerous chemical in an msds database, retrieve the description text of the chemical property through the name, and obtain the property description information text data of the chemical;

[0046] A data preprocessing module is configured to clean the data and remove repetitive information in the information text, and then use BIO labeling to label and obtain overall data, and divide the overall data into a training set and a test set;

[0047] Training phase:

[0048] Word embedding model: each information text in the training set is input into a character-level model based on bilstm, and a joint character vector feature representation is obtained through the character-level model based on bilstm, and the joint character vector feature representation is input into a long short-term memory network to obtain a result Q of modeling the input sequence l ,K l ,V l , wherein K l , V l is transmitted to the attention layer, and Q l is transmitted to the transformer encoder layer in the component feature acquisition model;

[0049] Component feature acquisition model: first, the Chinese characters in each information text are split into each single root, the frequency of the root is counted, and then the component in the root is input into a convolutional neural network to extract the component-level embedding feature through the convolutional neural network, and the component-level embedding feature is input into the transformer encoder layer;

[0050] A fusion module is configured to fuse the features through the attention layer and the encoder layer, and output label information through conditional random field CRF;

[0051] A loop module is configured to loop and iterate steps S31-S33 until the end;

[0052] A test module is configured to test the trained model using the test set to obtain a test set;

[0053] A storage module is configured to store the training result after training and the test set in a graph database Neo4j.

[0054] Further, comprising:

[0055] The bilstm-based character-level model comprises:

[0056] The classification module: first, select a sentence S_1, and perform single-character segmentation on the sentence S_1, and obtain chars={C1, C2, C3,..., Cn-1, Cn} after segmentation, wherein n is the length of the characters in the sentence, then take each word Ci, and establish corresponding position information In_i according to the chemical dictionary, and set the position information into four categories, namely: {B: indicating the first position of the matching word; M: indicating the middle position of the matching word; E: indicating the end position of the matching word; S: indicating the single word information}, and the corresponding formula is:

[0057]

[0058]

[0059]

[0060]

[0061] Wherein, L represents a dictionary, 1≤i≤n;

[0062] The compression module: after obtaining the {B, M, E, S} word set of each character, each word set is compressed into a fixed-dimensional vector, and a word weighting method is used to obtain the overall feature, and the formula corresponding to the weighting function is:

[0063]

[0064] Wherein, Z=∑ wz∈BUMUEUS Z(w), wherein the frequency of each word appearing on a static database is used as the weight;

[0065] The joint representation module of the character: respectively solve v s (B), v s (M), v s (E) and v s (S), and integrate the information into the character representation, through the following formula:

[0066] e s (B, M, E, S)=[v s (B); v s (M); v s (E); v s(S)]

[0067] x c ←[x c ;e s (B, M, E, S)].

[0068] Further comprising:

[0069] In the word embedding model, the combined character vector feature representation is input into the long short-term memory network, which includes:

[0070] In each time step of the input sequence, the long short-term memory network takes the combined character vector as input and retains the information in the hidden state in the previous time step, and the long short-term memory network outputs a vector representing the entire sequence, obtaining the Q, K, V of the attention mechanism, and the formula is as follows:

[0071] [Q, K, V] = e i [W, I, W] (1)

[0072] Wherein, e represents the word vector feature obtained by the long short-term memory network, I is a unit matrix, and W is a learned variable parameter;

[0073] Input into the convolutional neural network to extract the radical primary embedding feature through the convolutional neural network, specifically including:

[0074] First, input the radical in the word root into the convolutional neural network to extract relevant features through the feature extraction layer of the convolutional neural network, and the feature extraction layer includes 30 one-dimensional convolution kernels with a size of 3. The Bayesian optimization algorithm based on random forest is used to search for the optimal hyperparameters;

[0075] Then, the feature embedding of the Chinese character radical is realized through the max-pooling layer and the fully connected layer to obtain the component feature of each Chinese character. Finally, the radical primary embedding feature obtained from the convolutional neural network is input into the transformer encoder layer.

[0076] Further comprising:

[0077] The system further comprises:

[0078] The WEB retrieval module includes: the API of the open entity extraction system, then establishes the task, and judges whether the demand function is database retrieval or entity extraction;

[0079] If entity extraction is performed, the user inputs the chemical dangerous goods description text to be extracted, creates an extraction task, pre-processes the text, predicts the entity label y through the trained model, and simultaneously retrieves the database to check whether there is already existing information. If there is already existing information, the result information of the entity extraction is referenced to judge the difference, and the existing data information is taken as the correct information to supplement and correct the extraction information. If there is no data, the information is directly input into the database, and the extraction task is returned to the web application and ends.

[0080] If the database is retrieved, the label is directly retrieved in the graph database, if it exists, the retrieval information is directly returned, and if it does not exist, null is returned.

[0081] Beneficial effects: the present application realizes the extraction of chemical product names by relying on the dictionary information and the characteristics of the character components. The method first constructs the dictionary information of the chemical dangerous goods, then refers to the Chinese disassembled word dictionary to find the character components, then obtains the feature vector through the cnn convolution, combines the word group features and the Chinese character features to realize the extraction of the entity information, then saves the extracted data to the Neo4j graph database, constructs the web application to retrieve the extracted data, thereby solving the problems of poor single model recognition rate and the relative vacancy of the entity extraction model of the chemical dangerous goods properties. BRIEF DESCRIPTION OF DRAWINGS

[0082] Figure 1 The chemical dangerous goods property entity extraction method flow chart of the present application;

[0083] Figure 2 The word embedding model structure schematic diagram of the present application;

[0084] Figure 3 The character level model structure schematic diagram based on bilstm of the present application;

[0085] Figure 4 The training model structure schematic diagram of the present application;

[0086] Figure 5 The WEB retrieval method flow chart schematic diagram of the present application. DETAILED DESCRIPTION

[0087] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0088] AsFigure 1 As shown, the present application discloses an entity extraction method for chemical hazardous property based on text character level and component level features, comprising the following steps:

[0089] S1 data collection: obtain the name of hazardous chemical in the msds database, retrieve the description text of the chemical property through the name, and obtain the property description information text data of the chemical;

[0090] S2 data preprocessing: first, data cleaning is performed, and at the same time, repeated information in the information text is removed, then BIO labeling method is used for labeling, and overall data is obtained, which is divided into training set and test set;

[0091] Specifically, unstructured text fragments of hazardous chemicals are collected, the obtained files are manually labeled, and key word matching is performed using professional vocabulary in the chemical field, each text fragment is distinguished, text and labels are separated, and finally integrated together as a data set D1.

[0092] The obtained data set D1 is processed, the spaces in the text of the data set are removed, and the text in each row of D1 is separated and stored as a words list file T1, the labels corresponding to the text are marked, and a labels list file L1 is generated.

[0093] S3, according to the model network based on bidirectional long short-term attention mechanism, a Chinese field hazardous chemical property entity extraction Model is constructed, that is, a training stage, as shown in Figures 2-4 :

[0094] S31 word embedding model: each information text of the training set is input into the character level model based on bilstm, and the joint character vector feature representation is obtained through the character level model based on bilstm, the joint character vector feature representation is input into the long short-term memory network, and the result Q of modeling the input sequence is obtained l ,K l ,V l , among them K l , V l is transmitted to the attention layer, and Q l is transmitted to the transformer encoder layer in the component feature acquisition model;

[0095] S32 component feature acquisition model: first, the Chinese characters in each information text are divided into each single root, the word frequency of the root is counted, then the component in the root is input into the convolutional neural network, and the component level embedding feature is extracted through the convolutional neural network, and input into the transformer encoder layer;

[0096] S33 will fuse the features through the attention layer and the encoder layer, and output label information through conditional random field CRF;

[0097] S34 iterates steps S31-S33 until the end;

[0098] Specifically, the model integrates the word and structural features in the text to improve the accuracy of entity extraction of the text. For the name of dangerous chemicals, the text structure is relatively regular, and most of them are combinations of the names of multiple chemical products, for example: chemical products containing ethylene in the name, such as "4-ethylene-1-cyclohexene", "styrene", "4-vinylpyridine", etc. can make full use of its chemical name as a feature for extraction. At the same time, for Chinese characters, Chinese characters are pictographic characters, and usually their character shape and meaning have a very close relationship. Especially in chemical names, for example, for olefin substances, most of them belong to unsaturated hydrocarbons and can undergo addition reactions such as hydrogenation, halogenation, hydration, and epoxidation. The "olefin" word can also be split into "fire", "x", and "cloth" structures as features, which are input into the CNN feature extractor. Then using max pooling and fully connected network can get the basic features of each Chinese character. The article uses an entity extraction model based on a long short-term memory model, as shown in Figure Two to obtain the semantic information of the text, and a character feature model based on bert-based component level embedding is used to cross the features of the two models. Finally, crf is used to judge the label, and the results are output to the Neo4j graph database to build a web application for visual display.

[0099] (1) Character-level model based on bilstm

[0100] First, select the sentence S_1, and perform single character segmentation on the sentence S_1. After segmentation, chars={C1, C2, C3, …, Cn-1, Cn} is obtained, where n is the length of the sentence character. Then take each word Ci, and establish the corresponding position information In_i according to the chemical dictionary. In the position matrix, there are 4 tags in In_i {B: indicates the first position of the matching word; M: indicates the middle position of the matching word; E: indicates the end position of the matching word; S: indicates the single character information}, and the formula is represented as:

[0101]

[0102]

[0103]

[0104]

[0105] where L denotes the dictionary, and then compress the set to compress the word embedding of each label into an embedding, and the overall feature is obtained using the word weighting method, and the formula is:

[0106]

[0107] where Z = ∑ w∈BUMUEUS z(w), word weighting is a method proposed due to the general performance of word average, and the frequency of each word appearing in a static database is used as the weight, and a static weight is used to speed up the training rate, and v s (B), v s (M), v s (E) and v s (S) are obtained respectively, and the information is integrated into the character representation, and the formula is:

[0108] e s (B, M, E, S) = [v s (B); v s (M); v s (E); v s (S)]

[0109] x c ← [x c ; e s (B, M, E, S)]

[0110] The model diagram is shown in Figure Two .

[0111] (2) Obtain the component feature through CNN

[0112] Chinese characters are pictographic characters, and generally, Chinese characters and character shapes have great correlation, and there are certain requirements in the naming rules of chemicals, so the embedding features containing roots can be used to represent the structural information of Chinese characters, for example: “fire”, “Yuan”, “Vian” can be divided into “fire”, “Yuan”, “Vian”;

[0113] In the sentence, we first split the Chinese characters into each single root, count the word frequency of the root, input the information, and then input the component to the CNN, so that the relevant features are extracted through the CNN feature extraction layer. At present, 30 one-dimensional convolution kernels with a size of 3 are used, and the SMAC algorithm is used to search for the optimal hyperparameters in the process. After the convolution kernel, the obtained vector is subjected to the maximum pooling layer to reduce redundant information, and the component-level embedding features are obtained through the full connection layer. Finally, the component-level embedding features obtained from the CNN layer are input into the cross_attention model.

[0114] (3) using cross_attention model

[0115] The model combines two features of the text, realizes entity extraction for the properties of dangerous chemicals, and the model diagram is as shown in Figure 4

[0116] First, in terms of word embedding features, first get the word set to which each word belongs, calculate its word weighting feature, and after the word goes through the embedding layer, the word weighting feature of the word is fused into the character feature. Then input the word embedding into the LSTM (Long Short Term Memory Network) to get the modeling result of the input sequence. LSTM is a recurrent neural network that maintains long-term dependencies in sequences by passing states in loops. At each time step of the input sequence, LSTM receives an embedding vector, which represents the input word. LSTM takes the embedding vector as input and retains information from previous time steps in the hidden state. Finally, LSTM outputs a vector representing the entire sequence, which is linearly mapped to the attention Q, K, and V. The formula is as follows:

[0117] [Q, K, V] = e i [W, I, W] (3.1)

[0118] Where e represents the word vector feature obtained by LSTM, I is the identity matrix, and W is a variable parameter that can be learned. k and v are passed to the attention layer, and q is passed to the transformer encoder layer of the component embedding.

[0119] In the component feature acquisition part, the component vector is subjected to component feature extraction by cnn, and then passes through the encoder layer of the transformer. This layer mainly includes two parts, multi-head self-attention mechanism (Multi-Head Self-Attention) and feedforward neural network (Feedforward Neural Network). First, the component features output by the cnn convolution are output to get Q, K, and V through formula 3.1, but only the Q output by the word vector needs to be output to the multi-head attention mechanism. Here, the head is set to 8 to obtain multiple features. Then the result of the multi-head attention mechanism is output to the feedforward neural network, and the activation function is set to ReLU, and the output feature is output.

[0120] Finally, the features obtained by the word vector attention and the encoder are fused, and the label information is output by CRF.

[0121] S4 uses the test set to test the trained model to obtain the test set;​

[0122] S5 stores the training result after training and the test set in the graph database Neo4j

[0123] The API of the open chemical dangerous goods property entity extraction system acquires the network text of the user input dangerous chemical goods, utilizes the entity extraction model Model to extract data for processing, obtains the extraction result of the dangerous chemical goods property, calls the graph database Neo4j for storage, and constructs a WEB application to realize the visualization of the entity information through Echarts.

[0124] Specifically, as shown in the figure, Figure 5 The API of the open entity extraction system is established, and the task is judged to determine whether the demand function is database retrieval or entity extraction. When the entity extraction is performed, the user inputs the chemical dangerous goods description text to be extracted, creates an extraction task, and pre-processes the text. The entity label y is predicted through the trained model, and the database is retrieved to check whether there is already existing information. If there is related information, the information is referenced to judge whether the result is correct. If there is no retrieval information, the obtained information is stored in the database. If it already exists, the final entity encapsulation extraction result is established, the index item is returned to the WEB application program, and the domain expert entity extraction task Task is ended. If it is retrieval information, the label is directly retrieved in the graph database. If it exists, the retrieval information is directly returned. If it does not exist, null is returned.

[0125] On the other hand, the application also provides a property entity extraction system of dangerous chemicals, which comprises:

[0126] The data acquisition module is used for acquiring the dangerous chemical goods name in the msds database, retrieving the chemical property description text through the name, and obtaining the chemical property description information text data;

[0127] The data preprocessing module cleans the data and removes the repeated information in the information text, and then uses the BIO labeling method to label to obtain the overall data, and divides the overall data into a training set and a test set;

[0128] Training stage:

[0129] Word embedding model: input each information text of the training set into the character level model based on bilstm, obtain the joint character vector feature representation through the character level model based on bilstm, input the joint character vector feature representation into the long short-term memory network, and obtain the result Q of modeling the input sequence l ,K l ,V l , V among them K l , Vl to the attention layer, and Q l The transformer encoder layer in the transform bias feature acquisition model is transmitted;

[0130] The bias feature acquisition model first splits each Chinese character in the information text into each single root, counts the word frequency of the root, and then inputs the radicals in the root to the convolutional neural network, allowing it to extract radical-level embedding features through the convolutional neural network, and inputting the features to the transformer encoder layer;

[0131] The fusion module is used to fuse the features through the attention layer and the encoder layer, and output label information through the conditional random field CRF;

[0132] The loop module is used to loop and iterate steps S31-S33 until the end;

[0133] The test module is used to test the trained model using the test set to obtain a test set;

[0134] The storage module is used to store the training results and the test set after training in the graph database Neo4j.

[0135] The character-level model based on bilstm includes:

[0136] The classification module: first, select the sentence S_1, and perform single character segmentation on the sentence S_1. After segmentation, chars={C1, C2, C3,..., Cn-1, Cn} is obtained, where n is the length of the sentence character. Then, take each word Ci, and establish the corresponding position information In_i according to the chemical dictionary. The position information is set to four categories: {B: indicating the first position of the matching word; M: indicating the middle position of the matching word; E: indicating the end position of the matching word; S: indicating the single word information}, and the corresponding formula is represented as:

[0137]

[0138]

[0139]

[0140]

[0141] where L represents the dictionary, and 1≤i≤n;

[0142] Compressing module: after obtaining the {B, M, E, S} word set of the given character, each word set is compressed into a fixed dimension vector, and the overall feature is obtained by using the word weighting method, and the formula corresponding to the weighting function is:

[0143]

[0144] Where, Z = ∑ w∈BUMUEUS Z(w), here the frequency of each word appearing on a static database is used as the weight;

[0145] Joint representation module of characters: respectively obtain v s (B), v s (M), v s (E) and v s (S), integrate the information into the character representation, and pass through the following formula:

[0146] e s (B, M, E, S) = [v s (B); v s (M); v s (E); v s (S)]

[0147] x c ← [x c ; e s (B, M, E, S)].

[0148] In the word embedding model, the joint character vector feature representation is input into the long short-term memory network, including:

[0149] In each time step of the input sequence, the long short-term memory network takes the joint character vector as input and retains the information in the previous time step in the hidden state. The long short-term memory network outputs a vector representing the entire sequence, and obtains the attention mechanism Q, K, V, which is represented by the following formula:

[0150] [Q, K, V] = e i [W, I, W] (1)

[0151] Where, e represents the word vector feature obtained by the long short-term memory network, I is the unit matrix, and W is the learned variable parameter;

[0152] Input into the convolutional neural network, let it extract the radical primary embedding feature through the convolutional neural network, specifically including:

[0153] Firstly, the component of the root is input into the convolutional neural network, so that the relevant features are extracted through the feature extraction layer of the convolutional neural network, and the feature extraction layer includes 30 one-dimensional convolution kernels with a size of 3, and a Bayesian optimization algorithm based on a random forest is used to search for optimal hyperparameters;

[0154] Then, the feature embedding of the Chinese character component level is realized through the maximum pooling layer and the full connection layer to obtain the component feature of each Chinese character, and finally, the component embedding feature obtained from the convolutional neural network is input into the transformer encoder layer.

[0155] The system also includes:

[0156] The WEB retrieval module includes: the API of the open entity extraction system, then the task is established, and it is judged whether the demand function is database retrieval or entity extraction;

[0157] If entity extraction is performed, the user inputs the chemical dangerous goods description text to be extracted, creates an extraction task, pre-processes the text, predicts the entity label y through the trained model, and simultaneously retrieves the database to check whether there is existing information, if there is related information, the reference of the entity extraction result information is performed, the difference is judged, the existing data information is taken as correct information, the extraction information is supplemented and corrected, if there is no data, the information is directly input into the database, the web application program is returned, and the extraction task is completed.

[0158] If it is database retrieval, retrieval is directly performed in the graph database according to the label, if it exists, the retrieval information is directly returned, and if it does not exist, null is returned.

[0159] The data set used in the experiment of the application is obtained by obtaining the dangerous chemical name in the msds database, retrieving the online description text of the chemical property through the name, obtaining the property description information text of the chemical, pre-processing the data, removing part of the connecting words and other information for data cleaning, and removing the repeated information, and summarizing 1753 pieces of data, then using the BIO labeling method for labeling, taking 70% of the total as a training set, and taking 30% as a test set, wherein the label of each text information is:

[0160] {name, molecular formula, boiling point, ignition point, toxicity, volatile, color and odor, ph value, solubility, relative density}, and the public ner data set resume is used as the evaluation of the advantage of the model under the ordinary task.

[0161] The model is built based on the entity extraction network of the transformer encoder layer of the bidirectional long short-term memory network under the Pytorh framework. In the case of 50 epoch training, the learning rate lr is set to 0.0014, and the ablation experiment is set, which is: 1, removing the text component embedding feature 2, not using the cross_attention model, but giving the corresponding Q, K and V to the corresponding attention model, and the result shows that, in the collected hazardous chemical property data set, the correct rate of the test set label reaches 96%, and the effects of different ablation experiments are as follows:

[0162]

[0163] In the table, paper represents the chemical data set summarized in this paper, and Resume represents the public data set, and the data meanings are as follows:

[0164] Precision (p): Precision is the ratio of the number of entities correctly predicted by the model to the total number of predicted entities. Specifically, precision is equal to the intersection of the number of predicted entities and the number of true entities divided by the number of predicted entities. Recall (rec): Recall is the ratio of the number of entities correctly predicted by the model to the total number of true entities. Specifically, recall is equal to the intersection of the number of predicted entities and the number of true entities divided by the number of true entities. F1 score: F1 score is the weighted harmonic mean of precision and recall. Specifically, F1 score is equal to 2 times the product of precision and recall divided by the sum of precision and recall.

[0165] According to the naming rules of chemicals and the structural characteristics of the text, the chemical property entity extraction model based on the transformer encoder model structure is adopted, and the extracted results are automatically filled into the graph database of Neo4j after checking, and the web is constructed for visualization and convenient information retrieval, which reduces the cost and error rate of manual feature extraction

[0166] The present application deeply combines natural language processing and industry knowledge, better mines related knowledge in the field of chemical text, thereby improving the processing efficiency of complex professional text data, and combining the text rule and the naming rule, the word features and the component features of the text can also have good effects in other fields.

[0167] Although the preferred embodiments of the present application have been described, those skilled in the art can make further changes and modifications to the embodiments once they know the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0168] It will be apparent to those skilled in the art that various modifications and variations can be made to the present embodiments without departing from the spirit or scope of the present embodiments. Thus, it is intended that the present embodiments cover the modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

Claims

1. A method for extracting the properties of hazardous chemicals, characterized in that, The method comprises the following steps: S1 data collection: obtain the name of the dangerous chemical in the msds database, retrieve the chemical property description text by name, and obtain the property description text data of the chemical; S2 data preprocessing: first, data cleaning is performed, and at the same time, repeated information in the information text is removed, then BIO labeling is used for labeling to obtain overall data, and the overall data is divided into a training set and a test set; S3 training stage: The S31 word embedding model: input each information text of the training set into a character level model based on bilstm respectively, obtain a joint character vector feature representation through the character level model based on bilstm, input the joint character vector feature representation into a long short-term memory network, and obtain a result Q of modeling the input sequence l ,K l ,V l , and K l , V l in it are transmitted to an attention layer, and Q l is transmitted to a transformer encoder layer in the component feature acquisition model; S32 bias feature acquisition model: first, the Chinese characters in each information text are split into each single root, the word frequency of the root is counted, then the radicals in the root are input into the convolutional neural network, the radicals are extracted through the convolutional neural network to obtain the radical-level embedding features, and the features are input into the transformer encoder layer; S33 fuse the features through the attention layer and the encoder layer, and output label information through the conditional random field CRF; S34 iteratively loop steps S31-S33 until the end; S4 test the trained model using the test set to obtain a test set; S5 store the training results after training and the test set in the graph database Neo4j; In step S31, the character-level model based on bilstm comprises: Classify the matched characters: first, select a sentence S_1, and perform single character segmentation on the sentence S_1, after segmentation, obtain chars={C1, C2, C3, …, Cn-1, Cn}, wherein n is the length of the sentence character, then take each character Ci, establish corresponding position information In_i according to the chemical dictionary, and set the position information into four categories: {B: indicates the first position of the matched word; M: indicates the middle position of the matched word; E: indicates the end position of the matched word; S: indicates the single character information}, and the corresponding formula is: B(Ci) = {wi,k, ∀wi,k∈L, i<k ≤ n}; M(Ci) ={wj,k, ∀wj,k∈ L, 1 ≤ j <i<k ≤ n}; E(Ci) = {wj,i, ∀wj,i∈ L, 1 ≤ j <i}; ； wherein L represents a lexicon, ; Compression: after obtaining each character's {B, M, E, S} word set, each word set is compressed into a fixed-dimensional vector, and a word weighting method is used to obtain its overall features, and the formula corresponding to the weighting function is: ； wherein Here, the frequency of each word's occurrence in a static database is used as the weight. The joint representation of the character is given by and combining the information into the character representation by the equation: ； 。 2. The method of claim 1, wherein, In step S31, the combined character vector features are input into the long short-term memory network, comprising: In each time step of the input sequence, the long short-term memory network takes the combined character vector as input, and retains the information in the previous time step in the hidden state, the long short-term memory network outputs a vector representing the entire sequence, and obtains the attention mechanism Q, K, and V, and the formula is as follows: （1）； Wherein e represents the word vector feature obtained by the long short-term memory network, I is a unit matrix, and W is a learned variable parameter.

3. The method of claim 2, wherein, In step S32, the input is input into the convolutional neural network, and the first-level component feature is extracted by the convolutional neural network extraction part, which specifically includes: First, the component in the root is input into the convolutional neural network, and the relevant features are extracted by the feature extraction layer of the convolutional neural network. The feature extraction layer includes 30 one-dimensional convolution kernels with a size of 3, and the Bayesian optimization algorithm based on random forest is used to search for the optimal hyperparameters; Then, the first-level feature embedding of Chinese character components is realized through the max pooling layer and the full connection layer to obtain the component feature of each Chinese character. Finally, the first-level component embedding feature obtained from the convolutional neural network is input into the transformer encoder layer.

4. The method of claim 3, wherein, The transformer encoder layer comprises: a head self-attention mechanism and a feedforward neural network, which obtain Q r , K r , and V r by formula (1) from the radical-level embedding features, input Q l , K r , and V r into the self-attention mechanism, input Q r into the attention layer, output the result of the multi-head attention mechanism to the feedforward neural network, set an activation function, and output corresponding features.

5. The method of claim 1, wherein, The method further includes: Open the API of the entity extraction system, then establish a task, and determine whether the required function is database retrieval or entity extraction; If entity extraction is performed, the user inputs the chemical dangerous goods description text to be extracted, creates an extraction task, pre-processes the text, predicts the entity label y through the trained model, and simultaneously retrieves the database to check whether there is already existing information. If there is related information, the result information of the entity extraction is referenced to judge the difference, and the existing data information is taken as the correct information to supplement and correct the extracted information. If there is no data, the information is directly input into the database, and the web application program is returned, and the extraction task is completed; If it is database retrieval, the retrieval is directly performed in the graph database according to the label, and if the retrieval information exists, the retrieval information is directly returned, and if the retrieval information does not exist, null is returned.

6. A dangerous chemical property entity extraction system characterized by, The system includes: A data acquisition module is configured to acquire the name of the dangerous chemical in the msds database, retrieve the chemical property description text through the name, and obtain the property description information text data of the chemical; A data preprocessing module is configured to clean the data and remove repeated information in the information text, and then mark the total data by using a BIO marking method to obtain the total data, and divide the total data into a training set and a test set; Training stage: The word embedding model: input each information text of the training set into a bilstm-based character-level model respectively, obtain a joint character vector feature representation through the bilstm-based character-level model, input the joint character vector feature representation into a long short-term memory network, and obtain a result Q of modeling the input sequence l ,K l ,V l , and K l , V l are transmitted to an attention layer, and Q l is transmitted to a transformer encoder layer in the component feature acquisition model; A component feature acquisition model is configured to first split the Chinese characters in each information text into single roots, count the word frequency of the roots, input the components in the roots into a convolutional neural network, extract the first-level component embedding feature by the convolutional neural network, and input the first-level component embedding feature into a transformer encoder layer; A fusion module is configured to fuse the features output by the attention layer and the features output by the encoder layer, and output label information by using a conditional random field (CRF); A loop module is configured to loop and iterate steps S31-S33 until the end; A test module is configured to test the trained model by using the test set to obtain a test set; A storage module is configured to store the training result after training and the test set in a graph database Neo4j; The character-level model based on the bilstm includes: Classification module: first, select the sentence S_1, the sentence S_1 is divided into single character, after cutting, get chars={C1, C2, C3, …, Cn-1, Cn}, wherein, n is the length of the sentence character, then take each character Ci, according to the chemical dictionary to establish the corresponding position information In_i, set the position information to four categories, respectively: {B: indicates the first position of the matching word; M: indicates the middle position of the matching word; E: indicates the end position of the matching word; S: indicates the single word information}, the corresponding formula is represented as: B(Ci) = {wi,k, ∀wi,k∈L, i<k ≤ n}; M(Ci) ={wj,k, ∀wj,k∈ L, 1 ≤ j <i<k ≤ n}; E(Ci) = {wj,i, ∀wj,i∈ L, 1 ≤ j <i}; ； wherein L represents a lexicon, ; Compression: after obtaining the {B, M, E, S} word set of each character, each word set is compressed into a fixed dimension vector, and the overall characteristics are obtained by using the word weighting method, and the formula corresponding to the weighting function is: ； wherein Here, the frequency of each word's occurrence in a static database is used as the weight. The joint representation of the characters is given by and combining the information into the character representation by the equation: ； 。 7. The system of claim 6, wherein, In the word embedding model, the combined character vector features are input into the long short-term memory network, including: In each time step of the input sequence, the long short-term memory network takes the combined character vector as input and retains the information in the previous time step in the hidden state. The long short-term memory network outputs a vector representing the entire sequence, and the attention mechanism Q, K and V are obtained, which are represented by the following formulas: （1）； Wherein, e represents the word vector feature obtained by the long short-term memory network, I is the unit matrix, and W is the variable parameter learned; Input into the convolutional neural network to extract the component level embedding features by the convolutional neural network, specifically including: First, input the component in the root into the convolutional neural network to extract relevant features by the feature extraction layer of the convolutional neural network. The feature extraction layer includes 30 one-dimensional convolution kernels with a size of 3. The Bayesian optimization algorithm based on random forest is used to search for the optimal hyperparameters. Then, the component level feature embedding is realized through the maximum pooling layer and the full connection layer to obtain the component feature of each Chinese character. Finally, the component level embedding features obtained from the convolutional neural network are input into the transformer encoder layer.

8. The system of claim 7, wherein: Also includes: The WEB retrieval module includes: the API of the open entity extraction system, then establishes the task, judges whether the demand function is database retrieval or entity extraction; If entity extraction is performed, the user inputs the chemical dangerous goods description text to be extracted, creates an extraction task, pre-processes the text, predicts the entity label y through the trained model, and searches the database to see if there is already existing information. If there is related information, the entity extraction result information is referenced to judge the difference, and the existing data information is used as the correct information to complete and correct the extraction information. If there is no data, the information is directly input into the database, and the WEB application program is returned, and the extraction task is completed. If it is a database search, it is directly searched in the graph database according to the label, and if it exists, the search information is directly returned, and if it does not exist, null is returned.

Images