Process standard entity relation extraction method fusing deep learning and dependency syntax

Abstract

The invention discloses a deep learning and dependency syntax fused process standard entity relationship extraction method, which comprises the following steps: collecting process production process standard unstructured texts, and labeling entities needing to be extracted to establish an entity relationship data set; the method comprises the following steps: building a MacBERT-BiGRU-IDCNN-CRF entity extraction model, and carrying out training; performing entity extraction on a to-be-extracted process production process standard unstructured text through the trained deep learning model; importing the extracted entities into a natural language processing tool through a dictionary; performing word segmentation on sentences of the to-be-extracted process production process standard unstructured text by taking the sentences as units; performing dependency analysis on each constituent part according to a word segmentation result to obtain a dependency relationship of the sentences; according to the dependency relationship of the sentences, performing hyponymy division on each constituent part in the sentences to obtain a hyponymy relationship; and converting the sentence into a triple structure according to a triple establishment rule established according to the hyponymy and the hyponymy, and importing the triple structure into a graph database. The method can effectively solve the problem of dividing the entity boundary of the text by a natural language processing tool; and through a proposed triple construction rule, the method can better adapt to text characteristics in the field of Chinese process standards.

Description

Technical Field

[0001] The present invention relates to a process standard entity relationship extraction method integrating deep learning and dependency syntax, and belongs to the field of knowledge graph technology. Background Art

[0002] Process knowledge has an important impact on the product development, quality control, innovation and comprehensive competitiveness of modern enterprises, and is an important intellectual resource of enterprises. In the production process of process manufacturing enterprises, a large amount of process requirements, operating specifications, process standards and other information have been accumulated, which plays an important role in modern industrial production. It is directly related to product quality and the competitiveness of production enterprises. Therefore, constructing a production process standard knowledge map is of great significance for realizing enterprise digital production and controlling product production quality.

[0003] One of the key technologies for building knowledge graphs is to extract triple structures from unstructured data to represent semantic relationships between entities. At present, there are still many challenges in the automated construction of knowledge graphs in vertical fields. On the one hand, the data knowledge structure in vertical fields is more complex, usually containing more complex ontology engineering and rule-based knowledge; on the other hand, the quality requirements for knowledge extraction in vertical fields are higher, and they rely more on joint extraction from structured and unstructured data within the enterprise. In addition, vertical field knowledge is highly professional, and different fields have different proprietary vocabulary and concepts that require special processing and extraction. In response to the above problems, some scholars have proposed to construct triple relationships by combining the dependency relationship of entities in sentences. However, due to the strong professionalism of process standard texts and the fuzzy boundaries of entities, it is difficult for traditional word segmentation tools to accurately divide the boundaries of entities. Summary of the invention

[0004] In order to solve the properties of fuzzy entity boundaries and complex ontology engineering in the knowledge text in the field of production process standards, the present invention provides a process standard entity relationship extraction method that integrates deep learning and dependency syntax to solve the problem of entity boundary division of text by natural language processing tools; and proposes triple construction rules to adapt to the text characteristics in the field of Chinese process standards.

[0005] The technical solution of the present invention is:

[0006] In the first aspect, a method for extracting entity relations of process standards by integrating deep learning and dependency syntax is provided, including: step 1: collecting unstructured texts of process production process standards, and annotating entities to be extracted to establish an entity relationship dataset; step 2: building a MacBERT-BiGRU-IDCNN-CRF entity extraction model, adjusting hyperparameters, and dividing the labeled entity relationship dataset into a training set and a test set for training the MacBERT-BiGRU-IDCNN-CRF entity extraction model; step 3: extracting entities from the unstructured texts of process production process standards to be extracted through a trained deep learning model; step 4: importing the extracted entities into a natural language processing tool through a dictionary; segmenting the unstructured texts of process production process standards to be extracted on a sentence-by-sentence basis; performing dependency analysis on each component based on the results of the segmentation to obtain the dependency relationship of the sentence; step 5: dividing each component in the sentence into upper and lower parts based on the dependency relationship of the sentence to obtain the upper and lower relationship; converting the sentence into a triple structure based on the triple construction rule constructed based on the upper and lower relationship and importing it into a graph database.

[0007] Furthermore, the text is annotated using text annotation tools and the BMEO annotation strategy. The entity relationship types contained in the text are divided into several categories: equipment, process parameters, standards, processes, methods, materials, and functions. Based on these seven categories, the unstructured text of the process production process is annotated.

[0008] Furthermore, the MacBERT-BiGRU-IDCNN-CRF entity extraction model is constructed using a MacBERT layer, a BIGRU layer, an IDCNN layer, a feature fusion layer, a fully connected layer, and a CRF model. The MacBERT layer is a pre-trained deep bidirectional Transformer model for dividing the input labeled data into sentences and characters, and then converting them into character features and sentence features respectively; the BiGRU layer uses two GRU layers, and the character features converted by the MacBERT layer are input into the BiGRU layer for processing; the sentence features output by the MacBERT layer are spliced ​​into the character features to achieve the purpose of dimensionality upgrade of the sentence features, and then the spliced ​​character features are input into the IDCNN layer using iterative dilated convolution; the character features containing syntactic and sentence structure information output by the IDCNN and the character features containing upper and lower semantic information output by the BiGRU are fused in the feature fusion layer by a weighted average method; the fused character features are then input into the fully connected layer to be converted into the dimensions required for the output result, and finally the CRF model outputs the label result predicted for each character.

[0009] Furthermore, the triple construction rule is specifically as follows: first, determine whether the sentence can independently construct a triple based on the hierarchical relationship. The specific method can be determined by calculating the difference between the maximum and minimum values ​​of hops. If the difference is greater than or equal to 2, it is determined that the triple can be independently constructed. At this time, the head entity and the tail entity are determined based on the hierarchical relationship of the entity, and then the triple is constructed with the predicate as the corresponding relationship; if the difference is less than 2, it means that the sentence components cannot independently construct a triple. At this time, it is necessary to introduce the superior name of the sentence as the head entity of the triple, the predicate in the sentence as the relationship, and the remaining components as the tail entities to construct the triple; wherein, hops represents the distance from any word to the root word Root.

[0010] Secondly, a process standard entity relationship extraction system integrating deep learning and dependency syntax is provided, including: a labeling module, used to execute step 1: collect unstructured text of process production process standards, and label the entities to be extracted to establish an entity relationship dataset; a building module, used to execute step 2: build a MacBERT-BiGRU-IDCNN-CRF entity extraction model, adjust hyperparameters, and divide the labeled entity relationship dataset into a training set and a test set for training the MacBERT-BiGRU-IDCNN-CRF entity extraction model; an extraction module, used to execute step 3: divide the entities to be extracted into The unstructured text of the process production technology standard is used to extract entities through the trained deep learning model; the acquisition module is used to execute step 4: the extracted entities are imported into the natural language processing tool through the dictionary; the extracted unstructured text of the process production technology standard is segmented on a sentence basis; according to the results of the segmentation, each component is subjected to dependency analysis to obtain the dependency relationship of the sentence; the conversion module is used to execute step 5: according to the dependency relationship of the sentence, each component in the sentence is divided into upper and lower levels to obtain the upper and lower level relationship; according to the triple construction rules constructed based on the upper and lower level relationship, the sentence is converted into a triple structure and imported into the graph database.

[0011] In a third aspect, a terminal is provided, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to execute any one of the process standard entity relationship extraction methods integrating deep learning and dependency syntax as described above.

[0012] The beneficial effects of the present invention are as follows: in response to the problem of blurred entity boundaries in process standard knowledge text, the present invention proposes a deep learning model MacBERT-BiGRU-IDCNN-CRF, which enhances the model's ability to perform entity extraction tasks by training and integrating the contextual features of the text with sentence structure information, thereby enhancing the accuracy of entity boundary division in dependency analysis tasks; and proposes a triple construction rule based on the characteristics of the process standard text. The entity relationship extracted by this method is more flexible, which can solve the complex problem of domain knowledge graph ontology engineering to a certain extent, and the knowledge graph constructed by this method is more refined in knowledge representation, so that the knowledge graph contains richer semantic information, which is helpful for subsequent descriptive data analysis and data analysis based on logical reasoning methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a flowchart of the present invention;

[0014] Figure 2 It is a schematic diagram of the entity relationship extraction process that integrates deep learning and dependency syntax;

[0015] Figure 3 This is an example of labeling using the Label Studio labeling tool;

[0016] Figure 4 is an example of dependency analysis results;

[0017] Figure 5 It is a standard triplet for some cigarette silk making processes. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be noted that the embodiments in this application and the features in the embodiments can be combined with each other arbitrarily without conflict.

[0019] Example 1: Figure 1-5As shown, according to a first aspect of an embodiment of the present invention, a method for extracting entity relations of process standards integrating deep learning and dependency syntax is provided, comprising: step 1: collecting unstructured text of process production process standards, and annotating the entities to be extracted to establish an entity relationship dataset; step 2: building a MacBERT-BiGRU-IDCNN-CRF entity extraction model, adjusting hyperparameters, and dividing the labeled entity relationship dataset into a training set and a test set for training the MacBERT-BiGRU-IDCNN-CRF entity extraction model; step 3: dividing the entities to be extracted into a training set and a test set; and step 4: dividing the entities to be extracted into a training set and a test set. The unstructured text of the process production technology standard is extracted for entity extraction through the trained deep learning model; Step 4: Import the extracted entities into the natural language processing tool through the dictionary; Segment the extracted unstructured text of the process production technology standard on a sentence basis; Perform dependency analysis on each component based on the results of segmentation to obtain the dependency relationship of the sentence; Step 5: Based on the dependency relationship of the sentence, divide the components in the sentence into upper and lower levels to obtain the upper and lower level relationship; Convert the sentence into a triple structure based on the triple construction rule constructed based on the upper and lower level relationship and import it into the graph database.

[0020] Furthermore, the Label Studio text annotation tool was used to annotate the text using the BMEO annotation strategy, and the entity relationship types contained in the text were divided into several categories: equipment, process parameters, standards, processes, methods, materials, and functions. Based on these seven categories, the process production technology-related knowledge text was annotated.

[0021] Furthermore, the MacBERT-BiGRU-IDCNN-CRF entity extraction model is constructed by using MacBERT layer, BIGRU layer, IDCNN layer, feature fusion layer, fully connected layer, and CRF model. The MacBERT layer is a pre-trained deep bidirectional Transformer model, which is used to divide the input labeled data into sentences and characters, and then convert them into character features and sentence features respectively; the BiGRU layer uses two GRU layers to input the character features converted by the MacBERT layer into the BiGRU layer for processing; in order to apply sentence features to entity recognition for annotated characters The task requires dimensionality upgrading of sentence features. The sentence features output by the MacBERT layer are concatenated into character features to achieve the purpose of dimensionality upgrading of sentence features. The concatenated character features are then input into the IDCNN layer using iterative dilated convolution. The character features containing syntax and sentence structure information output by IDCNN and the character features containing upper and lower semantic information output by BiGRU are fused in the feature fusion layer by weighted averaging. The fused character features are then input into the fully connected layer to be converted into the dimensions required for the output result. Finally, the CRF model outputs the predicted label results for each character. The label results include entity boundaries and entity categories.

[0022] Specifically, the main components of the MacBERT-BiGRU-IDCNN-CRF entity extraction model are introduced as follows:

[0023] The MacBERT layer is a pre-trained deep bidirectional Transformer model. In order to adapt to the characteristics of Chinese text, MacBERT improves the masking strategy and uses similar words instead of [MASK] for masking, which improves the model's ability to represent vocabulary and sentences. In addition, MacBERT is a general language representation model pre-trained on large-scale text corpora. This transfer learning capability enables the model to perform well in entity recognition tasks with small data volumes or in specific fields. In this model, the MacBERT encoder layer is used to convert the input sentence into a character feature sequence and sentence features.

[0024] The BiGRU layer is an important variant of the recurrent neural network RNN. BiGRU uses two GRU layers to process the forward and backward information of the sequence respectively. This structure enables it to capture richer contextual information when processing sequences, and is particularly suitable for tasks such as natural language processing, time series prediction, and speech recognition. In terms of unit structure, GRU is a simplified version of LSTM. Each LSTM unit has three gating structures (input gate, forget gate, output gate), while GRU has only two gating structures (update gate, reset gate). Due to its simple structure, fewer parameters, and higher computational efficiency, the GRU unit can more efficiently solve the problem of gradient disappearance or explosion in the RNN network structure. For BiGRU, the specific formula involved is as follows:

[0025] r t =σ(W r ·[h t-1 ,x t ])

[0026] z t =σ(W z ·[h t-1 ,x t ])

[0027]

[0028] In the formula, x is input from the current position t and the output h of the previous hidden layer t-1 After adding the linear changes, it is connected to the sigmoid activation function σ, W r and W z Represent the weights of the reset gate and the update gate respectively. Due to the existence of σ, its output value is between 0 and 1, which is used to select how much information to keep. The calculation process of the update gate is similar. is the updated value, which is composed of the weight W of the new candidate state and the reset gate r t 、Previous position output h t-1 , input x at this position t Joint decision; the final output of the current position is h t , by the updated value Update gate z t , enter h in the previous position t-1 The calculation process of the forward and reverse GRU is the same, and the final output of BiGRU is the concatenation of the forward hidden state and the reverse hidden state.

[0029] In the entity recognition task, the label of each character is not only related to the character features of the previous character, but also has a great correlation with the features of the following character. Therefore, the model of the present invention selects a bidirectional GRU, which can take into account the text features of the context when training character features. In addition, compared with LSTM, GRU optimizes the model structure, which can improve the training efficiency of the model while ensuring the model's learning effect on text features.

[0030] The IDCNN layer uses an iterative dilated convolution structure; dilated convolution refers to a technique that expands the receptive field of a convolution operation without increasing the number of parameters by adding a hole (dilation) in the middle of the convolution kernel in a traditional convolution operation. Iterative dilated convolution builds on this by further increasing the receptive field through multi-level iterations to capture longer-range contextual information. Compared to traditional convolutional neural networks, IDCNN can significantly improve the model's ability to model long-distance dependencies without adding too many parameters, thereby improving the model's performance in processing text sequence tasks. The formula for calculating the receptive field of the ID-CNN dilated convolution is as follows:

[0031] F i+1 =(2 i+1 -1)×(2 i+1 -1)

[0032] In the formula, i is the step size. By adding an iterative convolutional neural network and taking advantage of its parallel computing characteristics, the feature vector combined with the sentence features is trained so that the trained features contain feature representations of different scales, which helps the network to understand the input data more comprehensively and improve its ability to represent data. Feature fusion with the character feature vector trained by BiGRU can make these abstract feature representations more discriminative and generalizable, which helps the subsequent CRF layer to perform classification and labeling tasks more accurately.

[0033] The feature fusion layer fuses the feature vector with context features trained by BiGRU with the character feature vector containing syntax and sentence structure features trained by IDCNN by weighted averaging. The formula for weighted averaging is as follows:

[0034] h i =w·x i +(1-w)·y i

[0035] Where h i Represents the output hidden layer feature vector, x i represents the vector input by the BiGRU layer, y i represents the vector input by the IDCNN layer, and w is the weight factor representing the weight of the BiGRU input.

[0036] The CRF model (conditional random field model) is a logarithmic linear model for sequence labeling problems. Its function is to constrain the labels by capturing the dependencies between the context. As a conditional probability distribution model, CRF can be used to express the core principle formula of the linear chain conditional random field as follows:

[0037]

[0038] Among them, x is the input variable, which represents the labeled observation sequence; y is the output sequence, which represents the label sequence corresponding to x one by one; f k represents the characteristic function; ω k Represents the weight corresponding to the feature function. The training set is estimated through maximum likelihood to obtain the conditional probability model, and then the Viterbi algorithm is used to output the label sequence y with the maximum conditional probability according to the given observation sequence.

[0039] After the feature vector is transformed by the fully connected layer, the CRF layer is responsible for decoding the predicted label sequence to achieve the conversion from features to probabilities. In this process, it takes into account the transfer scores between labels and uses this to correct the emission scores given by the feature extraction layer. This means that it not only considers the probability of a single tag, but also the overall structure of the tag sequence and the transfer between different tags, which enables the model to capture the correlation and constraints between tags, such as a certain entity category is more likely to appear after another entity category in a specific context. CRF can help ensure that the entire label sequence has a reasonable structure, thereby improving the performance of the model.

[0040] Furthermore, the precision rate P, recall rate R and F1 value are used as evaluation indicators to comprehensively evaluate the experimental results. The calculation formula is as follows:

[0041]

[0042] It should be noted that, in the above evaluation indicators, the number of correctly identified entities involved in the calculation is counted based on "when the type and start and end boundaries of the entity are successfully predicted".

[0043] Furthermore, in step 4, for the dependency relationship of the sentence, any word element has a dependency relationship with the root word Root directly or indirectly. The root word Root of the sentence is obtained through the interface of the natural language processing tool ltp, and the upper and lower relationships are reproduced with Root as the word element. The single-step dependency relationship between two word elements is taken as a hop, and the distance from any word element to Root is recorded as hops, and hops is used as the basis for judging the upper and lower relationships. Let the hops of the core predicate of the sentence be 0, the component that forms a subject-predicate relationship with the predicate in front of the predicate is hops-1, and the component that forms a verb-object relationship with the predicate after the predicate is hops+1, and so on. The modified component is reduced by one, and the modified component is increased by one. The hops of the parallel relationship remain unchanged, thereby obtaining a clear upper and lower relationship between the components in the sentence.

[0044] Furthermore, the triple building rules are specifically as follows:

[0045] First, determine whether the sentence can independently construct a triple based on the hierarchical relationship. The specific method can be determined by calculating the difference between the maximum and minimum values ​​of hops. If the difference is greater than or equal to 2, it is determined that the triple can be independently constructed. At this time, the head entity and the tail entity are determined based on the hierarchical relationship of the entity, and then the triple is constructed with the predicate as the corresponding relationship; if the difference is less than 2, it means that the sentence components cannot independently construct a triple. At this time, it is necessary to introduce the superior name of the sentence as the head entity of the triple, the predicate in the sentence as the relationship, and the remaining components as the tail entities to construct the triple; among them, hops represents the distance from any word to the root word Root.

[0046] It should be noted that the traditional methods of constructing triples based on dependency syntax include subject-verb-object relationship (Zhang San, eat, apple), possessive relationship of noun (book, author, Zhang San), preposition-object relationship (Zhang San, in, Beijing), etc. However, in the specific process standard unstructured text, there are cases where components are missing in sentences, such as the missing of subject or object. The premise of constructing triples by hierarchical relationship is that the sentence components must be divided into at least three layers, so that the triple relationship can be extracted independently from the sentence. In the above technical scheme, difference judgment is made according to the hierarchical relationship to form two different triple construction rules. The two rules cooperate with each other to effectively make up for the deficiency of the method of determining the head entity and the tail entity only based on the hierarchical relationship of the entity, which cannot construct triples due to component missing; at the same time, in view of the fact that the process standard unstructured text has a strong structure due to the particularity of its text, the present invention introduces the superior name of the sentence as the head entity of the triple, which is obviously more in line with the structural characteristics of the process standard unstructured text. For example, the silk-making process includes the process of heating up the tobacco leaves, which in turn includes process information such as process tasks and incoming material standards, and below that is specific unstructured text knowledge. Therefore, the sentence is a description and supplement of the process information to which it belongs. Therefore, for sentences with missing components, the name of the process information to which it belongs can be used as the head entity to construct a triple relationship.

[0047] According to a second aspect of an embodiment of the present invention, a process standard entity relationship extraction system integrating deep learning and dependency syntax is provided, including: a labeling module, used to execute step 1: collect unstructured text of process production process standards, and label the entities to be extracted to establish an entity relationship data set; a building module, used to execute step 2: build a MacBERT-BiGRU-IDCNN-CRF entity extraction model, adjust hyperparameters, and divide the labeled entity relationship data set into a training set and a test set for training the MacBERT-BiGRU-IDCNN-CRF entity extraction model; an extraction module, used to execute step 3: The unstructured text of the process production technology standard to be extracted is used to extract entities through the trained deep learning model; an acquisition module is used to execute step 4: the extracted entities are imported into the natural language processing tool through the dictionary; the unstructured text of the process production technology standard to be extracted is segmented into sentences as units; dependency analysis is performed on each component based on the result of segmentation to obtain the dependency relationship of the sentence; a conversion module is used to execute step 5: based on the dependency relationship of the sentence, each component in the sentence is divided into upper and lower levels to obtain the upper and lower level relationship; according to the triple construction rules constructed based on the upper and lower level relationship, the sentence is converted into a triple structure and imported into the graph database.

[0048] According to a third aspect of an embodiment of the present invention, there is provided a terminal, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to execute any one of the above-described process standard entity relationship extraction methods integrating deep learning and dependency syntax. In an exemplary embodiment, the terminal may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0049] Example 2: Figure 1-5 As shown, the present invention provides a process standard entity relationship extraction method integrating deep learning and dependency syntax, and the specific steps are as follows Figure 1 , 2 Specifically including:

[0050] Step (1): Taking the unstructured text of the cigarette production process standard as an example, the entities to be extracted are annotated to establish an entity relationship dataset;

[0051] For example, for the knowledge text related to the production process standard of cigarette silk, the entities extracted include equipment, process parameters, standards, processes, methods, materials and functions. Based on these seven categories, Label Studio annotation tool is used to annotate the knowledge related to the silk production process. In order to better display, the annotated results are displayed from one column to multiple columns, as shown in the following example. Figure 3 As shown, taking the first column as an example, B-process parameter represents the entity start boundary under the category of process parameter, and E-process parameter represents the entity end boundary under the category of process parameter.

[0052] Step (2): Divide the labeled entity relationship dataset into a training set and a test set to train the MacBERT-BiGRU-IDCNN-CRF entity extraction model (MBIC entity extraction model). According to the characteristics of the experimental corpus and multiple experiments, the specific hyperparameters are as follows: max_len is 30, batch_size is 20, embedding_num is 768, hidden_num is 128, epoch is 100, number of convolution kernels is 3, weight factor is 0.5, dropout is 0.15, and output prediction result dimension is 24.

[0053] Step (3): extract entities from the unstructured text of the process production technology standard to be extracted through the trained deep learning model;

[0054] For example, take "the water flow rate of the drum-type dehumidification equipment is adjustable and controllable, and the steam pressure, hot air speed, drum speed, and dehumidification damper opening are adjustable." as an example, and use MacBERT-BiGRU-IDCNN-CRF to extract entities. The extraction results are "drum-type dehumidification equipment", "water flow rate", "steam pressure", "hot air speed", "drum speed", and "dehumidification damper opening".

[0055] Step (4): Import the extracted entities into the Chinese Language Technology Platform LTP4.2.0 in the form of a dictionary, and perform dependency analysis on the original sentence. The result of dependency analysis is as follows: Figure 4 shown.

[0056] Step (5): Pass Figure 4 The dependency analysis results obtained can be used to preliminarily divide the components in the sentence into upper and lower levels. The results are as follows:

[0057] Drum rehumidification equipment: Dependency: (2, 'ATT'), hops: -2;

[0058] Water addition flow: Dependency: (3, 'SBV'), hops: -1;

[0059] Adjustable and controllable: Dependency: (0, 'HED'), hops: 0;

[0060] Steam pressure: Dependency: (13, 'SBV'), hops: -1;

[0061] Hot air speed: Dependency: (5, 'COO'), hops: -1;

[0062] Cylinder speed: Dependency: (5, 'COO'), hops: -1;

[0063] Tidal damper opening: Dependency: (5, 'COO'), hops: -1;

[0064] etc.: dependencies: (11, 'RAD'), hops: -1;

[0065] Adjustable: Dependencies: (3, 'COO'), hops: 0.

[0066] The above division results represent the dependency relationship obtained by LTP4.2.0 analysis after the MacBERT-BiGRU-IDCNN-CRF entity extraction model divides the entity extraction and the hypernymy relationship obtained based on the dependency relationship. In the dependency relationship column, the first number in the bracket represents the serial number of the hypernym of the current word, and the following letter group represents the relationship between the current word and the hypernym, such as ATT represents the attributive relationship, SBV represents the subject-predicate relationship, COO represents the parallel relationship, and HED represents the core relationship, which is usually the content pointed to by the predicate. First, according to the difference between the maximum and minimum hops values ​​greater than or equal to 2, it is judged that this sentence can construct a complete triple relationship. At this time, the head entity and the tail entity are determined according to the hypernymy relationship of the entity, and then the predicate is used as the corresponding relationship to construct the triple: from the dependency relationship between the entities, it is judged that "drum-type moisture recovery equipment" is set as the attributive modifying the subject "water flow" as the head entity Subject, then "water flow" is set as the modified object as the tail entity Object, and its corresponding predicate "adjustable and controllable" hops value is 0 as the relationship predicate pointing to the water flow. Then "adjustable" forms a parallel relationship with the core predicate verb, so its hops value is also 0, set as predicate, and the hops hierarchical rule can be inferred that "steam pressure", "hot air speed", "drum speed", and "damp exhaust door opening" all form a parallel relationship with the water addition flow, so they are also modified by the sentence's attributive "drum rehumidification equipment" and are set as the tail entity Object, and are connected to the attributive "drum rehumidification equipment" through the second predicate "adjustable". Import some of the generated triples into the graph database neo4j in graph form, and the effect is as follows Figure 5 shown.

[0067] In order to verify the effectiveness of the combined model of the present invention, it is compared with four other existing entity extraction models. The comparative experimental results are shown in Table 1.

[0068] Table 1 Overall entity recognition results of each model

[0069] Model Training time( / s) Accuracy Recall F1 value Bert-BiLSTM-CRF 47.23 0.902 0.920 0.911 Bert-BiGRU-CRF 42.55 0.917 0.880 0.898 Bert-IDCNN-CRF 41.87 0.846 0.880 0.863 Bert-IDCNN-BiGRU-CRF 43.97 0.938 0.920 0.918 MacBERT-BiGRU-IDCNN-CRF 45.52 0.959 0.940 0.949

[0070] Table 1 shows the training time, accuracy, recall rate and F1 value of several existing algorithm models and the model proposed in the present invention. By combining the basic Bert model with the neural network models such as BiLSTM-CRF, BiGRU-CRF and IDCNN-CRF, the entity recognition effect is compared. From Table 1, it can be seen that the F1 value of the traditional Bert-BiLSTM-CRF is the highest, indicating that its named entity recognition effect is relatively the best, but the training time is much higher than Bert-BiGRU-CRF and Bert-IDCNN-CRF. This is determined by the network structure of LSTM, which cannot guarantee high efficiency while taking into account the extraction accuracy. Although Bert-IDCNN-CRF has the shortest training time, its F1 value is the lowest, indicating that the best prediction effect cannot be achieved if the IDCNN model is used alone in combination with the sentence features for label prediction. Combining Bert with BiGRU and IDCNN for prediction takes into account both training efficiency and prediction accuracy. Its training time is shorter than that of Bert-BiLSTM-CRF, and its training accuracy is higher than that of Bert-BiLSTM-CRF.

[0071] For entity extraction tasks, it is necessary to ensure accuracy. It is also important to improve the comprehensiveness of recognition while ensuring recognition accuracy, so the F1 score is used as the most important evaluation basis. The model of the present invention is formed by replacing Bert in "Bert-IDCNN-BiGRU-CRF" with MacBert. The F1 value of the model of the present invention is 0.031 higher than that of the BERT encoding model, indicating that the effect of the MacBert pre-trained semantic model in the field of Chinese encoding is better than the traditional Bert model; at the same time, it can be seen that the MacBert-BiGRU-IDCNN-CRF model has the best F1 value compared with other combined models, and the training speed is also faster than the traditional BiLSTM model. In addition, the accuracy of this model is slightly higher than the recall rate. From a macro perspective, it shows that the knowledge extraction of the present invention is more cautious, which means that the extracted knowledge is less wrong, which meets the objective needs of the knowledge graph application within the enterprise, and is more inclined to avoid the extraction of wrong information and cause errors and losses in the enterprise production process.

[0072] Furthermore, in order to verify the rationality and effectiveness of the MacBERT-BiGRU-IDCNN-CRF network structure design, the present invention conducts experiments on the established GRU-CRF, BiGRU-CRF, IDCNN-CRF, IDCNN-BiGRU-CRF, MacBert-CRF, and MacBert-IDCNN-BiGRU-CRF models with and without sentence features through ablation experiments (it should be noted that the “MacBert-IDCNN-BiGRU-CRF model without sentence features” means that on the basis of the present invention, the extracted sentence features are discarded; the step of feature splicing using sentence features in the present invention is removed, that is, the character features output by MacBert are directly used as the input of IDCNN), the hyperparameters of each ablation part are the same as the parameter settings under the experiment in Table 1 above, and the above models are trained under the same experimental conditions. The experimental results are shown in Table 2.

[0073] Table 2 Prediction results of ablation experiments

[0074] Model Accuracy Recall F1 value GRU-CRF 0.518 0.580 0.547 BiGRU-CRF 0.745 0.760 0.752 IDCNN-CRF 0.485 0.640 0.552 IDCNN-BiGRU-CRF 0.704 0.760 0.731 MacBert-CRF 0.808 0.840 0.824 MacBert-IDCNN-BiGRU-CRF (without sentence features) 0.885 0.920 0.902 MacBert-IDCNN-BiGRU-CRF (including sentence features) 0.959 0.940 0.949

[0075] The prediction results in Table 2 comprehensively compare the three prediction indicators. From Table 2, it can be found that the prediction effect of BiGRU-CRF is significantly better than that of GRU-CRF, indicating that the bidirectional GRU structure can learn the context information in the text more effectively than the unidirectional GRU structure. In addition, since the general Embedding tool cannot extract the sentence features of the text for encoding, the prediction structure of the model using IDCNN alone or the model combined with BiGRU for prediction is relatively poor, all below 80%. In addition, it can be seen from Table 2 that the reference of the MacBert-CRF model greatly improves the prediction accuracy of the model. When MacBert-CRF is used alone, its predicted F1 value reaches 0.824, which is significantly better than the combined model without MacBert encoding. In addition, this experiment also compares the MacBert-IDCNN-BiGRU-CRF model with or without sentence features. It can be seen that after integrating sentence features into model training, the F1 value increases from 0.902 to 0.949, indicating that sentence features greatly enhance the entity extraction effect of the model in the production process standard scenario. That is, the above experiments verify the superiority of the MacBert-IDCNN-BiGRU-CRF proposed in the present invention in predicting the accuracy of process standard text labels and the effectiveness of the network structure.

[0076] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A process standard entity relationship extraction method integrating deep learning and dependency syntax, characterized in that: include: Step 1: Collect the unstructured text of process production technology standards and annotate the entities to be extracted to establish an entity relationship dataset; Step 2: Build the MacBERT-BiGRU-IDCNN-CRF entity extraction model, adjust the hyperparameters, and divide the labeled entity relationship dataset into a training set and a test set for training the MacBERT-BiGRU-IDCNN-CRF entity extraction model; Step 3: Use the trained deep learning model to extract entities from the unstructured text of the process production technology standard to be extracted; Step 4: Import the extracted entities into the natural language processing tool through the dictionary; segment the extracted process production technology standard unstructured text into sentences; perform dependency analysis on each component based on the segmentation results to obtain the dependency relationship of the sentence; Step 5: Based on the dependency relationship of the sentence, divide the components in the sentence into upper and lower levels to obtain the upper and lower level relationship; according to the triple construction rules constructed based on the upper and lower level relationship, convert the sentence into a triple structure and import it into the graph database.

2. According to claim 1, the process standard entity relationship extraction method integrating deep learning and dependency syntax is characterized in that: The text is annotated using text annotation tools and the BMEO annotation strategy. The entity relationship types contained in the text are divided into several categories: equipment, process parameters, standards, processes, methods, materials, and functions. Based on these seven categories, the unstructured text of the process production process is annotated.

3. The process standard entity relationship extraction method integrating deep learning and dependency syntax according to claim 1 is characterized in that: The MacBERT-BiGRU-IDCNN-CRF entity extraction model is constructed using a MacBERT layer, a BIGRU layer, an IDCNN layer, a feature fusion layer, a fully connected layer, and a CRF model. The MacBERT layer is a pre-trained deep bidirectional Transformer model that is used to divide the input labeled data into sentences and characters, and then convert them into character features and sentence features respectively. The BiGRU layer uses two GRU layers to input the character features converted by the MacBERT layer into the BiGRU layer for processing; The sentence features output by the MacBERT layer are concatenated into the character features to achieve the purpose of dimensionality upgrade of the sentence features, and then the concatenated character features are input into the IDCNN layer using iterative dilated convolution; the character features containing syntax and sentence structure information output by IDCNN and the character features containing upper and lower semantic information output by BiGRU are fused in the feature fusion layer by weighted averaging; the fused character features are then input into the fully connected layer to be converted into the dimension required for the output result, and finally the CRF model outputs the predicted label result for each character.

4. The process standard entity relationship extraction method integrating deep learning and dependency syntax according to claim 1 is characterized in that: The triple construction rule is specifically as follows: first, determine whether a sentence can independently construct a triple based on the hierarchical relationship. The specific method can be determined by calculating the difference between the maximum and minimum values ​​of hops. If the difference is greater than or equal to 2, it is determined that the triple can be independently constructed. At this time, the head entity and the tail entity are determined based on the hierarchical relationship of the entity, and then the triple is constructed with the predicate as the corresponding relationship; if the difference is less than 2, it means that the sentence components cannot independently construct a triple. At this time, it is necessary to introduce the superior name of the sentence as the head entity of the triple, the predicate in the sentence as the relationship, and the remaining components as the tail entities to construct the triple; wherein, hops represents the distance from any word to the root word Root.

5. A process standard entity relationship extraction system integrating deep learning and dependency syntax, characterized in that: include: The annotation module is used to perform step 1: collect the unstructured text of the process production process standard and annotate the entities to be extracted to establish an entity relationship data set; Build a module to perform step 2: build the MacBERT-BiGRU-IDCNN-CRF entity extraction model, adjust the hyperparameters, and divide the labeled entity relationship dataset into a training set and a test set for training the MacBERT-BiGRU-IDCNN-CRF entity extraction model; An extraction module is used to execute step 3: extract entities from the unstructured text of the process production process standard to be extracted through a trained deep learning model; The acquisition module is used to execute step 4: import the extracted entities into the natural language processing tool through the dictionary; segment the extracted process production technology standard unstructured text into sentences; perform dependency analysis on each component according to the segmentation result to obtain the dependency relationship of the sentence; The conversion module is used to execute step 5: according to the dependency relationship of the sentence, each component in the sentence is divided into upper and lower levels to obtain the upper and lower level relationship; according to the triple construction rules constructed based on the upper and lower level relationship, the sentence is converted into a triple structure and imported into the graph database.

6. A terminal, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to execute the process standard entity relationship extraction method integrating deep learning and dependency syntax as described in any one of claims 1 to 4.

Images