Aluminum alloy scientific literature automatic knowledge extraction method based on natural language processing

Abstract

The invention relates to the technical field of artificial intelligence, in particular to an aluminum alloy scientific literature automatic knowledge extraction method based on natural language processing. The method comprises the steps that aluminum alloy material literature data are collected and preprocessed; labeling the preprocessed literature data by adopting a data annotation tool according to a set entity relationship type to obtain an aluminum alloy material literature corpus; constructing an entity naming recognition model based on a bidirectional encoder and a long and short memory network in combination with an attention mechanism and a random vector field; and inputting the aluminum alloy material literature corpus into the entity naming recognition model for training, and extracting aluminum alloy material literature data according to the trained entity naming recognition model. The brand-new named entity recognition model is provided, and information in aluminum alloy material scientific literatures can be accurately and efficiently processed through all built-in functional structures of the brand-new named entity recognition model.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for automatic knowledge extraction from aluminum alloy scientific literature based on natural language processing. Background Art

[0002] In the field of materials science, aluminum alloy, as an important type of metal material, is widely used in aerospace, automobile manufacturing, building structures, electronic products and other industries due to its light weight, high strength, good corrosion resistance and machinability. With the rapid development of science and technology, the number of scientific literature on aluminum alloys has shown a blowout growth. These literatures contain rich technical details, experimental data, theoretical innovations and future trend forecasts. The data and knowledge contained in the literature have extremely high application value for building aluminum alloy material databases, knowledge bases, and training libraries for machine learning models.

[0003] The information in the scientific literature of aluminum alloy materials is mainly text data and chart data. The text is unstructured and semi-structured information expressed in natural language, which is difficult to be directly parsed using conventional algorithms. In the past, data and knowledge were extracted from the literature through manual reading and parsing. This process not only requires researchers to have a deep professional knowledge background, but also requires a lot of time and energy to read, extract, and fill in data and knowledge templates one by one. It is very time-consuming and labor-intensive, with high cost and low efficiency. Therefore, a natural language processing technology is needed that can quickly and effectively extract knowledge from the literature. Summary of the invention

[0004] In order to solve the problems existing in the prior art, the present invention provides an automated knowledge extraction method for aluminum alloy scientific literature based on natural language processing, the method comprising: collecting aluminum alloy material literature data and preprocessing; using a data annotation tool to annotate the preprocessed literature data according to the set entity relationship type to obtain an aluminum alloy material literature corpus; constructing an entity naming recognition model based on a bidirectional encoder and a long short-term memory network, and combining an attention mechanism with a random vector field; inputting the aluminum alloy material literature corpus into the entity naming recognition model for training, and extracting aluminum alloy material literature data according to the trained entity naming recognition model. The present invention proposes a new named entity recognition model, which can accurately and efficiently process information in aluminum alloy material scientific literature through its built-in functional structures.

[0005] The present invention adopts the following technical scheme, which is an automated knowledge extraction method for aluminum alloy scientific literature based on natural language processing, including:

[0006] Collect literature data on aluminum alloy materials and perform preprocessing;

[0007] According to the set entity relationship type, the preprocessed literature data is annotated using a data annotation tool to obtain an aluminum alloy material literature corpus;

[0008] Based on the bidirectional encoder and long short-term memory network, an entity naming recognition model is constructed by combining the attention mechanism and random vector field;

[0009] Inputting the aluminum alloy material literature corpus into the entity naming recognition model for training, and extracting aluminum alloy material literature data according to the trained entity naming recognition model;

[0010] The entity naming recognition model includes: an input layer, a bidirectional encoder layer, a bidirectional long short-term memory network layer, an attention layer and a random vector field layer, wherein:

[0011] Input layer, reads sentence sequence data from the aluminum alloy material literature corpus;

[0012] The bidirectional encoder layer converts the sentence sequence data read by the input layer into a text vector;

[0013] The bidirectional long short-term memory network takes the text vector converted by the bidirectional encoder layer as the feature input, labels the entities in the sentence sequence corresponding to the vector, and obtains the predicted label vector;

[0014] The attention layer assigns attention weights to the predicted label vector output by the bidirectional long short-term memory network, and sends the weighted predicted label vector to the random vector field layer;

[0015] The random vector field layer calculates the score of the predicted label vector after assigning weights, and outputs the prediction result of the model according to the score of the predicted label vector.

[0016] Furthermore, the method further comprises:

[0017] The bidirectional long short-term memory network layer, the attention layer and the random vector field layer in the entity naming extraction model are replaced with a convolutional neural network, a bidirectional gated recurrent unit, an attention layer and an output layer to establish an entity relationship extraction model;

[0018] The entity relationship extraction model includes:

[0019] A convolutional neural network layer, including a convolutional layer, a pooling layer and a fully connected layer. The convolutional layer performs a convolution operation on the text vector converted by the bidirectional encoder layer to extract features from the text vector; the extracted features are pooled by the pooling layer and then sent to the fully connected layer for serialization to obtain a text feature vector;

[0020] The bidirectional gated recurrent unit layer takes the text feature vector extracted by the convolutional neural network layer as input, and extracts the word feature vector in the text feature vector through the forward gated recurrent unit and the backward gated recurrent unit;

[0021] Attention layer, which assigns attention weights to the word feature vectors extracted by the bidirectional gated recurrent unit layer;

[0022] The output layer normalizes the word feature vectors with attention weights output by the attention layer to obtain the probability distribution value of each word feature vector, and outputs the word vector with the highest probability value as the prediction result of the corresponding sentence sequence data.

[0023] Furthermore, the bidirectional encoder layer converts the sentence sequence data read by the input layer into a text vector, specifically:

[0024] The text vector includes a word vector, a sentence vector and a sequence position vector of sentence sequence data.

[0025] Furthermore, the random vector field layer calculates the score of the predicted label vector after the weight is assigned, and outputs the prediction result of the model according to the score of the predicted label vector, including:

[0026] Suppose the sentence sequence data is X(x 1 ,x 2 ,…,x m );

[0027] The predicted label vector output by the bidirectional long short-term memory network is Y(y 1 ,y 2 ,…,y n ); after weighting by the attention layer, we get sequence A(a 1 ,a 2 ,…,a n );

[0028] The score for each predicted label vector is calculated as:

[0029]

[0030] Among them, m is the length of the sentence sequence, n is the number of predicted labels, Indicates the jth label y j Convert to the j+1th label y j+1 The probability of Indicates that the label corresponding to the i-th word vector is y j The probability of

[0031] The score of each predicted label vector is normalized, and the maximum conditional probability of each predicted label vector is obtained as the label prediction result of the sentence sequence data.

[0032] Furthermore, the convolutional neural network layer includes convolutional layer, pooling layer and fully connected layer, specifically:

[0033] The convolution layer performs a convolution operation on the text vector to extract features from the text vector. The calculation method is:

[0034] c i =f(w·x i:i+s-1 +b)

[0035] Among them, w is the convolution kernel, s represents the size, x i represents the i-th word vector of the input, b is the bias term, and f() represents the convolution function;

[0036] The convolution layer calculates the feature matrix of the text vector and sends it to the pooling layer. The pooling layer uses the maximum pooling method to select the maximum weight M. i As the local optimal feature matrix;

[0037] The local optimal feature matrix is ​​sent to the fully connected layer for serialization, and the text feature vector is output.

[0038] Furthermore, the output layer normalizes the word feature vector with attention weights output by the attention layer to obtain the probability distribution value of each word feature vector, including:

[0039] g i =softmax(l·c i +b)

[0040] Among them, l represents the attention weight matrix, b represents the bias term; g i It is the probability distribution value of the feature vector of the i-th word after Softmax normalization.

[0041] Furthermore, the random vector field layer calculates the score of the predicted label vector after the weight is assigned, and outputs the prediction result of the model according to the score of the predicted label vector, and also includes:

[0042] Under the guidance of material experts, the entity relationship type rule base is constructed for the collected literature data, and the data structure of hash table is used for storage;

[0043] The prediction results of the model are verified in the rule base.

[0044] The beneficial effects of the present invention are as follows: the present invention proposes a named entity recognition model based on a bidirectional encoder and a bidirectional long short-term memory network combined with an attention mechanism and a random vector field, namely, a BBAC named entity recognition model; compared with traditional static word vector technologies such as Word2Vec, the bidirectional encoder uses its built-in bidirectional converter structure as a feature extraction mechanism to more effectively deal with the ambiguity of vocabulary and the challenge of long-distance dependency, thereby more accurately capturing the contextual semantic information of the text and improving the efficiency and accuracy of feature extraction. In addition, in order to further enhance the performance of the material entity recognition model, the present invention introduces an attention layer between the bidirectional long short-term memory layer and the random vector field layer as an effective means of enhancing the model features, thereby improving the performance and adaptability of the model in entity recognition tasks; in addition, a rule base and a dictionary are customized, and the predicted label category output data is verified and calibrated after decoding the random vector field layer, thereby improving the accuracy of the final prediction result.

[0045] The present invention also proposes an entity relationship extraction model based on a bidirectional encoder and a convolutional neural network combined with a bidirectional gated recurrent unit and an attention mechanism, namely the BCBA entity relationship extraction model. In the architecture of the bidirectional gated recurrent unit model, an attention layer is introduced as a pre-processing step of the output layer. This design allows the model to dynamically assign attention weights to each word vector in the text, thereby achieving a fine distinction in the influence of different words. Through this process, the model can more effectively extract key features and significantly enhance the ability to recognize important information in sentences, so as to improve the accuracy and performance of the entity relationship extraction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0047] Figure 1 A schematic flow chart of an automated knowledge extraction method for aluminum alloy scientific literature based on natural language processing according to an embodiment of the present invention;

[0048] Figure 2 A schematic diagram of the structure of an entity naming recognition model according to an embodiment of the present invention;

[0049] Figure 3 A schematic diagram of the structure of an entity relationship extraction model according to an embodiment of the present invention;

[0050] Figure 4A schematic diagram showing a comparison of macro average values ​​of different entity naming recognition models according to an embodiment of the present invention;

[0051] Figure 5 Schematic diagram of macro-average comparison of different entity relationship extraction models in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] The flowchart of the method for automatic knowledge extraction of aluminum alloy scientific literature based on natural language processing according to an embodiment of the present invention is as follows: Figure 1 As shown, including:

[0054] S1. Collect aluminum alloy material literature data and perform preprocessing;

[0055] S2. Using a data annotation tool to annotate the preprocessed literature data according to the set entity relationship type to obtain an aluminum alloy material literature corpus;

[0056] S3, based on the bidirectional encoder and long short-term memory network, combined with the attention mechanism and random vector field to build an entity naming recognition model;

[0057] S4, inputting the aluminum alloy material literature corpus into the entity naming recognition model for training, and extracting aluminum alloy material literature data according to the trained entity naming recognition model.

[0058] In a specific embodiment of the present invention, aluminum alloy material literature data is first collected, and then the literature data is preprocessed. Under the guidance of material experts, the entity relationship types required for the corpus are formulated, and the processed text data is annotated according to the entity relationship types specified in the corpus using data annotation tools. After the annotation is completed, the data is exported and formatted accordingly to obtain the corpus.

[0059] The aluminum alloy material literature data collected in the embodiment of the present invention can be stored in PDF format, and the literature data can be preprocessed to clean and remove duplicate and incomplete documents; further, it can be converted into text format using function libraries including but not limited to PDFPlumber, PyPDF2, etc. in Python, and then regular expressions are used for pattern matching to remove irrelevant content and normalize the text, such as removing some redundant line breaks; under the guidance of material experts, the entity relationship types required for the corpus are formulated, and the processed text data is annotated according to the entity relationship types specified in the corpus using annotation tools. After the annotation is completed, it is exported to Json format data, and the corresponding formatting processing is performed on it to obtain a corpus.

[0060] The embodiment of the present invention proposes an entity naming model, referred to as the BBAC (BERT-BiLSTM-Att-CRF) model, whose structure is as follows: Figure 2 As shown, the advantages of the bidirectional encoder (Bidirectional Encoder Representations from Transformers, BERT) of the integrated transformer (Transformer) network, the bidirectional long short-term memory network (Bidirectional Long Short-Term Memory, BiLSTM), the attention mechanism (Attention) and the conditional random field (CRF) are used for named entity recognition. The present invention inputs the training corpus into the model for training. The model mainly includes an input layer, a bidirectional encoder layer, a bidirectional long short-term memory network layer, an attention and random vector field layer. The specific training process is as follows:

[0061] Input layer: read the input sentence sequence data X(x 1 ,x 2 ,…,x m ).

[0062] Bidirectional encoder layer: converts the input sentence sequence into vector form (word vector, sequence position vector, sentence vector). The word vector uses special symbols [CLS] and [SEP] to represent the start and end positions of the sentence respectively; each unit of the Transformer converter feature extractor is composed of a self-attention mechanism and a feedforward neural network, and the units can be stacked continuously.

[0063] Bidirectional long short-term memory layer: Using the word vectors extracted by the bidirectional encoder layer as rich feature input, this layer is committed to learning the deep association between words and their corresponding entity labels during the training phase; the bidirectional long short-term memory layer structure uses its bidirectional characteristics to traverse the input sequence from front to back and from back to front in parallel, capturing and storing the forward hidden layer output and the backward hidden layer output of each time step respectively. Subsequently, the feature vectors in these two directions are concatenated and fused into a more comprehensive word representation. This step effectively improves the ability to understand words in context. Next, a linear transformation is performed on the concatenated output vector, and its dimension is adjusted to match the preset number of entity categories. Finally, the Softmax function is used to normalize these transformed vectors so that each vector can be mapped to the corresponding entity category probability, thereby completing the task of labeling entities in the entire sentence sequence.

[0064] Attention layer: Assume a sentence sequence data is X(x 1 ,x 2 ,…,x m ), the output of the bidirectional long short-term memory layer network is H(h 1 ,h 2 ,…,h n ), Y(y 1 ,y 2 ,…,y n ) is the corresponding entity category label; (x 2 ,x 3 ,x 4 ) together constitute the entity of category X. In the context of the sentence sequence data, x 2 With x 4 The position vector of x 3 The precise marking of x has a more significant influence. 3 The attention network cleverly allocates different attention weights according to the importance of each word in the H sequence, thereby subtly adjusting and optimizing x 3 The labeling result is obtained by the formula: That is, we can calculate x 3 Similarly, for all words in the sequence, the attention mechanism is used to weight the output of the bidirectional long short-term memory layer, and finally the feature vector is sent to the random vector field layer.

[0065] Random Vector Field Layer: Add it after the bidirectional long short-term memory layer network to constrain the output vector of the bidirectional long short-term memory layer to conform to certain grammatical rules, prevent illegal tags, and solve the grammatical constraint problem in the NER process; suppose a sentence sequence is X(x 1 ,x 2 ,…,x m), and after bidirectional long and short memory layer training, the predicted label sequence Y(y 1 ,y 2 ,…,y n ), using the scoring matrix Indicates that the label type corresponding to the i-th word vector is y j The probability of, after weighting by the attention layer, we get the sequence A(a 1 ,a 2 ,…,a n ); random vector fields use the transfer matrix Represents the transition probability within the label, which represents the jth label y j Convert to the j+1th label y j+1 The probability of using and The final score can be found: Among them, m is the length of the sentence sequence, n is the number of predicted labels, Indicates the jth label y j Convert to the j+1th label y j+1 The probability of Indicates that the label type corresponding to the i-th word vector is y j The probability of m n The result is then subjected to Softmax normalization to determine the maximum conditional probability of each label combination. The label combination corresponding to the maximum value is the final determination result of the sequence labeling: Then simplify it by the log-likelihood function: The superscript ^ represents the predicted value. Through continuous learning to optimize the parameters to maximize the likelihood function, this layer outputs the final predicted label category.

[0066] The embodiment of the present invention further constructs entity relationship types and rule bases under the guidance of material experts, such as writing regular expressions to improve the rule base; in addition, a dictionary is constructed through the collected literature data, which is stored in a hash table data structure. Before the final output result, the entity label type predicted by the named entity recognition model is verified and corrected according to the patterns in the dictionary and rule base to improve the accuracy of the final prediction result.

[0067] In another specific embodiment of the present invention:

[0068] The present invention also introduces an entity relationship extraction model based on a bidirectional encoder and a convolutional neural network combined with a bidirectional gated recurrent unit and an attention mechanism, referred to as the BCBA (BERT-CNN-BiGRU-Attention) model, whose structural diagram is shown in the figure. Figure 3As shown in the figure, the advantages of bidirectional encoder, convolutional neural network (CNN), bidirectional gated recurrent unit (BiGRU) layer neural network and attention mechanism are combined to extract entity relations, including:

[0069] Input layer: read the data of the training corpus in units of sentences, X(x 1 ,x 2 ,…,x m ) represents the input sequence.

[0070] Bidirectional encoder layer: Consistent with the bidirectional encoder layer in the entity naming recognition model, it maps the input data into a vector form and passes the obtained feature vector to the next layer of the network.

[0071] Convolutional neural network layer: Convolutional neural network is a special form of feedforward neural network (FNN). Its core network mainly includes convolution layer, pooling layer and fully connected layer. The convolution layer uses a series of sliding windows, namely convolution kernels, to continuously perform convolution operations on the data to capture and extract diverse feature patterns. Each unique convolution kernel plays a specific role and works together in the convolutional neural network to comprehensively parse and extract various features contained in the sentence. The calculation method is: i =f(w·x i:i+s-1 +b); where w is the convolution kernel, s is the size, and x i represents the i-th word vector of the input, b is the bias term, f() represents the convolution function, and the final calculated C(c 1 ,c 2 ,…,c n ) is the output feature matrix; the pooling layer selects the maximum weight M by downsampling the local feature matrix after convolution in a maximum pooling (MaxPooling) manner i As the local optimal feature, discard the remaining values, effectively reduce the size of the feature matrix, improve calculation efficiency and reduce the risk of overfitting. The calculation formula of the maximum pooling is: M i =

[0072] max{c 1 ,c 2 ,...,c n-s+1}. Finally, the M output by the pooling layer i After combination, it is sent to the fully connected layer for serialization as the feature input of the bidirectional gated recurrent unit layer.

[0073] Bidirectional Gated Recurrent Unit Layer: The text feature vector extracted by the bidirectional encoder layer is used as the input of the bidirectional Gated Recurrent Unit layer network. The bidirectional Gated Recurrent Unit layer network further mines and learns the deep features of these input vectors. This process is achieved by running the forward Gated Recurrent Unit and the backward Gated Recurrent Unit in parallel. The forward Gated Recurrent Unit captures the context information from left to right to generate forward features, while the backward Gated Recurrent Unit captures the context information from right to left to generate backward features. The bidirectional processing method enables the bidirectional Gated Recurrent Unit layer to understand and encode the semantic relationships in the text more comprehensively and deeply.

[0074] Attention layer: In the architecture of the bidirectional gated recurrent unit layer, the attention layer is introduced before the output layer. The function of this layer is to dynamically assign different attention weights to each input word vector. Through this process, the model can further refine the text features and distinguish the influence of different words in the sentence, thereby emphasizing and highlighting those features that are more critical to the overall meaning of the sentence, and achieving more refined processing and understanding of text information.

[0075] Output layer: In the output layer, the Softmax function is used to normalize the output of the attention layer to calculate the probability distribution tendency corresponding to each relationship category. Finally, the category with the highest probability value is selected as the result of the text relationship prediction. The specific calculation process follows the conversion logic of the Softmax function as follows:

[0076] g i =softmax(l·c i +b)

[0077] g = arg max (g)

[0078] Among them, l represents the attention weight matrix, b represents the bias term; g i is the probability distribution value of the i-th word feature vector after Softmax normalization.

[0079] The embodiment of the present invention further reflects the extraction effect of the model proposed in this application by evaluating the indicators precision, recall, and F1 value. The calculation method of the above indicators is:

[0080]

[0081] TP refers to the number of entities that the model can correctly identify; FP refers to the number of entities that can be identified but have incorrect category or boundary judgments; FN refers to the number of entities that should have been recognized but were not recognized by the model.

[0082] In the process of entity naming recognition, the embodiment of the present invention conducts a multi-model comparison experiment on the constructed aluminum alloy material corpus. CRF, BiLSTM-CRF, and BERT-BiLSTM-CRF are the current mainstream entity recognition models, and BBCA is the entity naming recognition model proposed in this application. The comparison results are shown in Figure 2. Figure 4 shown.

[0083] In the process of entity relationship extraction, the embodiment of the present invention conducts a multi-model comparison experiment on the constructed aluminum alloy material corpus. BERT-SVM, BERT-RNN, BERT-BiGRU, and BERT-BiGRU-Attention are the current mainstream relationship extraction models, and BCBA is the entity relationship extraction model proposed in this application. The results are as follows: Figure 5 shown.

[0084] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An automated knowledge extraction method for aluminum alloy scientific literature based on natural language processing, characterized in that: include: Collect literature data on aluminum alloy materials and perform preprocessing; According to the set entity relationship type, the preprocessed literature data is annotated using a data annotation tool to obtain an aluminum alloy material literature corpus; Based on the bidirectional encoder and long short-term memory network, an entity naming recognition model is constructed by combining the attention mechanism and random vector field; Inputting the aluminum alloy material literature corpus into the entity naming recognition model for training, and extracting aluminum alloy material literature data according to the trained entity naming recognition model; The entity naming recognition model includes: an input layer, a bidirectional encoder layer, a bidirectional long short-term memory network layer, an attention layer and a random vector field layer, wherein: Input layer, reads sentence sequence data from the aluminum alloy material literature corpus; The bidirectional encoder layer converts the sentence sequence data read by the input layer into a text vector; The bidirectional long short-term memory network takes the text vector converted by the bidirectional encoder layer as the feature input, labels the entities in the sentence sequence corresponding to the vector, and obtains the predicted label vector; The attention layer assigns attention weights to the predicted label vector output by the bidirectional long short-term memory network, and sends the weighted predicted label vector to the random vector field layer; The random vector field layer calculates the score of the predicted label vector after assigning weights, and outputs the prediction result of the model according to the score of the predicted label vector.

2. The method for automatic knowledge extraction from aluminum alloy scientific literature based on natural language processing according to claim 1 is characterized in that: The method further comprises: The entity naming extraction model is replaced with an entity relationship extraction model, wherein the entity relationship extraction model includes: an input layer, a bidirectional encoder layer, a convolutional neural network, a bidirectional gated recurrent unit, an attention layer, and an output layer; wherein: Input layer, reads sentence sequence data from the aluminum alloy material literature corpus; The bidirectional encoder layer converts the sentence sequence data read by the input layer into a text vector; A convolutional neural network layer, including a convolutional layer, a pooling layer and a fully connected layer. The convolutional layer performs a convolution operation on the text vector converted by the bidirectional encoder layer to extract features from the text vector; the extracted features are pooled by the pooling layer and then sent to the fully connected layer for serialization to obtain a text feature vector; The bidirectional gated recurrent unit layer takes the text feature vector extracted by the convolutional neural network layer as input, and extracts the word feature vector in the text feature vector through the forward gated recurrent unit and the backward gated recurrent unit; Attention layer, which assigns attention weights to the word feature vectors extracted by the bidirectional gated recurrent unit layer; The output layer normalizes the word feature vectors with attention weights output by the attention layer to obtain the probability distribution value of each word feature vector, and outputs the word vector with the highest probability value as the prediction result of the corresponding sentence sequence data.

3. The method for automatic knowledge extraction from aluminum alloy scientific literature based on natural language processing according to claim 1 is characterized in that: The bidirectional encoder layer converts the sentence sequence data read by the input layer into a text vector, specifically: The text vector includes a word vector, a sentence vector and a sequence position vector of sentence sequence data.

4. The method for automatic knowledge extraction from aluminum alloy scientific literature based on natural language processing according to claim 1 is characterized in that: The random vector field layer calculates the score of the predicted label vector after the weight is assigned, and outputs the prediction results of the model according to the score of the predicted label vector, including: Suppose the sentence sequence data is X(x1,x2,…,x m ); The predicted label vector output by the bidirectional long short-term memory network is Y(y1,y2,…,y n ); after weighting by the attention layer, we get the sequence A(a1,a2,…,a n ); The score for each predicted label vector is calculated as: Among them, m is the length of the sentence sequence, n is the number of predicted labels, Indicates the jth label y j Convert to the j+1th label y j+1 probability; Indicates that the label corresponding to the i-th word vector is y j probability; The score of each predicted label vector is normalized, and the maximum conditional probability of each predicted label vector is obtained as the label prediction result of the sentence sequence data.

5. The method for automatic knowledge extraction from aluminum alloy scientific literature based on natural language processing according to claim 2, Features: Convolutional neural network layers include convolutional layers, pooling layers, and fully connected layers, specifically: The convolution layer performs a convolution operation on the text vector to extract features from the text vector. The calculation method is: c i =f(w·x i:i+s-1 +b) Among them, w is the convolution kernel, s represents the size, x i represents the i-th word vector of the input, b is the bias term, and f() represents the convolution function; The convolution layer calculates the feature matrix of the text vector and sends it to the pooling layer. The pooling layer uses the maximum pooling method to select the maximum weight M. i As the local optimal feature matrix; The local optimal feature matrix is ​​sent to the fully connected layer for serialization, and the text feature vector is output.

6. The method for automatic knowledge extraction from aluminum alloy scientific literature based on natural language processing according to claim 2 is characterized by: The output layer normalizes the word feature vector with attention weights output by the attention layer to obtain the probability distribution value of each word feature vector, including: <h2 style=";text-align:left;direction:ltr">g<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> =softmax(l·c<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> +b) Among them, l represents the attention weight matrix, b represents the bias term; g i is the probability distribution value of the i-th word feature vector after Softmax normalization.

7. The method for automatic knowledge extraction from aluminum alloy scientific literature based on natural language processing according to claim 1 is characterized by: The random vector field layer calculates the score of the predicted label vector after assigning weights, and outputs the prediction result of the model according to the score of the predicted label vector. It also includes: Under the guidance of material experts, the entity relationship type rule base is constructed for the collected literature data, and the data structure of hash table is used for storage; The prediction results of the model are verified in the rule base.

Images