A knowledge graph construction method and device for intelligent decision-making, an electronic device, and a storage medium

Abstract

This invention discloses a method, apparatus, electronic device, and storage medium for constructing a knowledge graph for intelligent decision-making, belonging to the field of industrial intelligent technology. The knowledge graph construction method of this invention includes: inputting text data; using a BERT-BiLSTM-CRF entity recognition model that integrates word segmentation features to obtain sequence labels for the input text; using an R-BertTransformer entity relation extraction model to extract domain text event relations; using a BERT-BiLSTM-CRF sequence labeling model to extract event trigger words, followed by a rule-based method to extract event core words, and finally a rule-based method to extract text numerical knowledge; and using a deep learning-based event relation extraction model to extract the logical relationships between events. Using the knowledge graph construction method of this invention, the event extraction accuracy can reach over 80%, and the logical relationship recognition accuracy F1 value can reach up to 0.75, meeting the needs of practical applications.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of industrial intelligence, and particularly relates to a knowledge graph construction method and device for intelligent decision-making, an electronic device and a storage medium. BACKGROUND

[0002] Knowledge graph technology has become an important infrastructure in the field of industrial intelligence and a prerequisite for intelligent decision-making. However, industrial scenarios not only involve a huge number of knowledge rules that are scattered in distribution, but also contain rich numerical information and logic. However, existing knowledge graph technology is often limited to describing static qualitative concepts and their associations, lacking the ability to describe numerical states and being difficult to effectively summarize the logic of the dynamic evolution between these states, thereby severely limiting the effective application of knowledge graph technology in complex industrial scenarios.

[0003] After searching, the application case with the Chinese patent application number 201910575949.7 discloses a construction method of a new type of cement clinker production heat consumption knowledge graph. The application case takes cement clinker production heat consumption knowledge as the center, divides the relationship between basic knowledge, deep knowledge and comprehensive application knowledge, and constructs a structured new type of cement clinker production heat consumption knowledge graph. Distributed agent mode is adopted, and agents and corresponding sub-agents are designed based on the characteristics of the three levels of basic knowledge, deep knowledge and comprehensive application knowledge. By assigning instructions to agents for execution, users can obtain relevant knowledge. The present application analyzes heat consumption from three levels of basic level, deep level and application level, discovers the potential relationship between entities, and then finds out the key factors affecting heat consumption and the effective and efficient method to reduce heat consumption, so as to achieve the direct purpose of reducing energy consumption, and finally realize the optimization of cement industry. However, the application case does not fully consider the Chinese vocabulary information, thereby affecting the entity recognition effect. SUMMARY

[0004] 1. Problem to be solved

[0005] The purpose of the present application is to provide a knowledge graph construction method and device for intelligent decision-making, an electronic device and a storage medium, thereby solving the technical problem that the knowledge graph constructed by using the existing knowledge graph construction method is difficult to apply in complex industrial scenarios.

[0006] 2. Technical solution

[0007] In order to solve the above problems, the technical solution adopted by the present application is as follows:

[0008] The present application provides a knowledge graph construction method for intelligent decision-making, which comprises the following steps:

[0009] Step 1, entity recognition

[0010] Input text data, and use the BERT-BiLSTM-CRF entity recognition model fused with word segmentation features to obtain the sequence label of the input text, i.e., the named entity type, based on the character-based method and fused with Chinese word segmentation features;

[0011] Step two, entity relation extraction

[0012] An R-BertTransformer entity relation extraction model is used to extract the event relations in the domain text;

[0013] Step three, event extraction

[0014] A sequence labeling model BERT-BiLSTM-CRF is used to extract event trigger words, then a rule-based method is used to extract event core words, and finally a rule-based method is used to extract numerical knowledge in the text;

[0015] Step four, event relation extraction

[0016] An event relation extraction model based on deep learning is used to extract the event relations.

[0017] Further, the operation process of the BERT-BiLSTM-CRF entity recognition model fused with word segmentation features for entity recognition is as follows:

[0018] The BERT is used to obtain token vector representation of each character in the input text sequence, and the jieba word segmentation tool is used to obtain corresponding word segmentation information and map it into an embedding vector;

[0019] The character vector and the word segmentation feature vector are concatenated and input into the BiLSTM to extract features in the text sequence;

[0020] The CRF conditional random field is used to label the feature vector output by the upper BiLSTM, i.e., output a label sequence.

[0021] Further, when using BERT to obtain character vector representation of the text sequence, position information is integrated, wherein the special characters “[CLS]” and “[SEP]” are used to mark the start and end positions of the text sequence, and if the input text sequence is composed of multiple sentences, a “[SEP]” special character is added between adjacent sentences to mark.

[0022] Further, when extracting entity relations in step two, the entity representation is concatenated with the input text sequence representation in a feature aggregation manner, the concatenated sequence features are input into the Transformer encoder structure to obtain the context information between the text sequence and the entity, and the BERT position information encoding method is used to embed the entity position information.

[0023] Further, the R-BertTransformer entity relation extraction model comprises:

[0024] The input layer refers to the input text sequence, and the input information also comprises position information of two entities in a sentence, wherein entity character positions are marked with 1, and other character positions are marked with 0.

[0025] The entity position information embedding layer: the character embedding vector of the input text sequence is obtained by using the BERT model, and the text sequence embedding is represented by Seq Embeds.

[0026] The Transformer Encoder layer: the text sequence information is input into a plurality of Encoders with the same structure, each Encoder is composed of a plurality of self-attention layers and a feedforward neural network, and parallel calculation is performed on the global position information of the input sequence.

[0027] The classification layer: the output layer is a full connection layer, and the specific result is output.

[0028] Further, when the rule-based method is used to extract the event core word, a rule library is established by analyzing data, and the rule library is updated and iterated according to the error part of the extraction result.

[0029] Further, when the rule-based method is used to extract the text numerical knowledge, firstly, the node serial number of the numerical value in the dependency syntax tree is determined, the noun closest to the serial number is taken as the core word of the numerical knowledge, and the core word of the numerical knowledge is supplemented through additional rules, and the numerical description information is described by using the upper and lower limits of the numerical interval.

[0030] The application further provides a knowledge graph construction device for intelligent decision-making, comprising:

[0031] The entity recognition module: used for adopting a BERT-BiLSTM-CRF entity recognition model fused with word segmentation features, fusing Chinese word segmentation features on the basis of a character method, and acquiring sequence labels of the input text;

[0032] The entity relation extraction module is used for extracting the event relation of the domain text according to the R-BertTransformer entity relation extraction model.

[0033] The event extraction module is used for extracting the event trigger word by using a sequence labeling model BERT-BiLSTM-CRF, then extracting the event core word by using a rule-based method, and finally extracting the text numerical knowledge by using a rule-based method.

[0034] The event relationship extraction module is used to extract the event relationships between events based on a deep learning-based event relationship extraction model.

[0035] The present invention also provides an electronic device, including a processor and a storage medium, wherein a computer program is stored on the storage medium, and the processor is configured to execute the method of the present invention when the computer program is invoked.

[0036] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, is used to implement the method of the present invention.

[0037] 3. Beneficial effects

[0038] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0039] (1) When constructing a knowledge graph for intelligent decision-making, this invention uses a BERT-BiLSTM-CRF entity recognition model that integrates word segmentation features for entity recognition. Compared with the problem that existing entity extraction models cannot effectively utilize Chinese vocabulary information, this model can utilize Chinese word segmentation features to improve the recognition effect, thus having certain advantages.

[0040] (2) In view of the problem that traditional entity location information embedding is affected by domain text data, the present invention uses the R-BertTransformer entity relationship extraction model to extract domain text event relationships, so that it is not affected by the text length, i.e. the location embedding vector table.

[0041] (3) This invention proposes a rule-based method to extract numerical knowledge from text. It uses the sequence labeling model BERT-BiLSTM-CRF to extract event trigger words, and then combines dependency parsing and rule templates to extract event elements. This effectively addresses the problem of complex domains, such as the cement production process, which involves a large amount of numerical and logical knowledge. Furthermore, this invention defines textual numerical knowledge as a binary form consisting of core words plus numerical description information. Since numerical knowledge and logical knowledge have the same format, this facilitates the extraction of logical relationships and makes model processing easier. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating a knowledge graph construction method for intelligent decision-making.

[0043] Figure 2 A diagram of the Bert-BiLSTM-CRF model that integrates word segmentation features.

[0044] Figure 3 A model diagram for entity relation extraction using R-BERTTransformer.

[0045] Figure 4 This is a basic flowchart of natural language processing for rules.

[0046] Figure 5 This is a schematic diagram of a dependency syntax tree for a rule-based numerical knowledge extraction method.

[0047] Figure 6 Extract a model diagram for event relationships.

[0048] Figure 7 Embedding process for character information and word segmentation information.

[0049] Figure 8 This describes the algorithm flow for mining expert control rules based on Apriori production operation data. Detailed Implementation

[0050] To address the problem that existing knowledge graphs are difficult to apply to complex industrial applications, this invention provides a method for constructing a knowledge graph for intelligent decision-making. The knowledge graph constructed by this invention can not only be effectively applied to complex industrial applications, but also assist in the effective modeling and real-time evaluation of complex scenarios such as cement production processes, thereby improving the level of intelligent decision-making. The overall concept of the knowledge graph construction method of this invention is as follows:

[0051] (1) This invention introduces the concept of event graphs. First, it uses dependency parsing tools to identify core event words in production records for event categorization. Then, it captures numerical knowledge from the text and uses it for event "state" description. Based on a deep learning model, it extracts the event relationships between events. Simultaneously, it combines expert control rules from domain production operation data to ultimately form event triples of state events and their evolutionary relationships. Currently, the event extraction accuracy can reach over 80%, and the event relationship recognition accuracy F1 value can reach up to 0.75, meeting the needs of practical applications.

[0052] (2) This invention expands the event nodes and logical relationships on the basis of qualitative knowledge in the graph to more comprehensively express the numerical description knowledge and events existing in the cement clinker production process. Its advantages are specifically reflected in the fact that, for domain numerical knowledge extraction tasks, it extracts numerical knowledge from text using a rule-based method and utilizes the Apriori algorithm to extract expert control rules contained in the historical operation data of cement clinker production (the specific algorithm flow is as follows). Figure 8 As shown in the figure, this can effectively reduce labor costs.

[0053] (3) This invention targets the task of extracting knowledge about events. It extracts event trigger words based on a sequence labeling model, and then extracts event elements based on a rule template. The rule template is updated and improved by analyzing error instances. The R-BERT Transformer is used to extract domain text event relationships. The model performance can be further improved by adding text paragraph information. The superiority of the method is verified by ablation experiments.

[0054] Combination such as Figure 1 As shown, one embodiment of the present invention provides a knowledge graph construction method for intelligent decision-making in cement production processes, comprising the following steps:

[0055] Step 1: Entity Recognition

[0056] Named entity recognition is a crucial step in knowledge extraction, aiming to identify named entities—words with specific meanings or high referentiality—from unstructured text data. This invention extracts entities such as equipment and materials that meet business requirements from unstructured input text within a specific domain.

[0057] This invention proposes a BERT-BiLSTM-CRF entity recognition model that integrates word segmentation features. It integrates Chinese word segmentation features on the basis of character-based methods and can improve the model performance by utilizing lexical information to include textual and positional information. Moreover, it is not affected by the pre-trained word vectors due to the differences in domain text data that this method addresses in the aforementioned issues, thus possessing certain advantages.

[0058] The model uses BERT to obtain token vector representations of each character in the input text sequence. It then uses the jieba word segmentation tool to obtain the corresponding word segmentation information and maps it to embedding vectors. The character vectors and word segmentation feature vectors are concatenated and input into a BiLSTM. The BiLSTM encodes the input information. Finally, a CRF (Conditional Random Field) performs syntactic computation at the sentence level, capturing the current state of each token and the contextual information of the preceding and following token states. The final output is a labeled sequence. The model structure is as follows: Figure 2 As shown, the specific operation is as follows:

[0059] Step 1: Character Embedding and Word Segmentation Feature Embedding

[0060] The model's input consists of batches of text sequences, with a sentence or paragraph from the domain text serving as a single text sequence. For example... Figure 7The figure below shows the embedding process of character information and word segmentation information when the model inputs a sample. Steps 1-3 in the processing flow are used to obtain the character vector representation of the text sequence. The special characters "[CLS]" and "[SEP]" mark the start and end positions of the text sequence. If the input text sequence consists of multiple sentences, a "[SEP]" special character needs to be added between adjacent sentences. The text information (token ids) assigns a unique ID to each character, and the paragraph information (token type ids) is the segmentation information added to different sentences in a sample input. For example, in an input sequence containing two sentences, the segmentation information added to the embedding vector of the first sentence is 0, and the segmentation information added to the embedding vector of the second sentence is 1; the position information (attention mask) shows which positions of the characters in the input sequence are useful.

[0061] Sub-step 2: BiLSTM Feature Extraction Layer

[0062] Bidirectional long short-term memory networks are suitable for processing sequence data. The BiLSTM model processes the input sequence by using two LSTM structures, taking into account both past and future information in the sequence to better extract features and patterns in the text sequence.

[0063] Sub-step 3: CRF Classification Layer

[0064] If the result of the previous layer is input to the fully connected layer and the output size of the fully connected layer is set to the number of label categories, the prediction result may be: the predicted labels for the input sequence ["return", "rotary kiln"] are ["I-EQ", "I-EQ", "I-EQ"], where I-EQ represents the middle character of the device class entity, lacking the correlation between tags. The CRF layer can solve this problem. By considering the features of the current state and the context features of the previous and next states, it models and annotates the feature vectors output by the upper-layer BiLSTM, that is, marking the output of each character as a specific named entity type, such as device class entities, material class entities, etc.

[0065] By jointly modeling the entire sequence through CRF and considering the correlation between the current state and the previous and next states, for example: non-entity labels and entity internal character labels should not be adjacent, effectively solving similar situations where the predicted labels for the input sequence ["return", "rotary kiln"] are ["I-EQ", "I-EQ", "I-EQ"], so as to obtain more accurate sequence labels and improve the accuracy and robustness of the NER task.

[0066] Step 2: Entity Relationship Extraction

[0067] To address the issue of traditional entity location information embedding being affected by domain text data, the R-BertTransformer relation extraction proposed in this invention offers certain advantages, as it is unaffected by text length (i.e., the position embedding vector table). Considering that the Transformer structure can obtain contextual representations between different characters, this invention employs a feature aggregation approach to concatenate entity representations with the input text sequence representations. The concatenated sequence features are then input into the Transformer encoder structure to obtain contextual information between the text sequence and entities. Furthermore, the BERT positional information encoding method replaces the entity location information embedding methods used in models such as BiLSTM-ATT and PCNN. R-BertTransformer relation extraction is as follows: Figure 3 As shown below, the processing steps for each layer of the model will be described in detail.

[0068] Sub-step 1: Input layer

[0069] The input layer refers to the input text sequence. The input information also includes the position information of the two entities in the sentence, represented by Entity1 Mask and Entity2 Mask, where the position of the entity character is marked with 1 and other character positions are marked with 0.

[0070] Sub-step 2: Entity location information embedding layer

[0071] The character embedding vectors of the input text sequence are obtained using the BERT model, and the text sequence embeddings are represented by Seq Embeds. Based on the positional information of the two input entities, Entity1 Mask and Entity2 Mask, the embedding vectors of the two entities are calculated respectively, and are represented by the mean of the vectors corresponding to the entities in Seq Embeds. The entity embedding vectors are then concatenated with the Seq Embeds.

[0072] When processing input text sequences, the Transformer architecture does not explicitly encode the positional information of the text sequence. Furthermore, this embodiment of the invention uses the positional encoding method from the BERT model to encode the information of each position in the input text sequence into a vector form. The purpose is to explicitly represent the positional information of the input sequence, thereby enabling the model to better understand the relative positions within the sequence data. Specifically, for an input sequence of length L, an L×d matrix is ​​used to represent the positional encoding, where dmodel is the dimension of the hidden layers in the model. The encoding of each position i is a dmodel-dimensional vector, which can be represented as:

[0073]

[0074]

[0075] Here, pos represents the position of the character in the text sequence; 2i refers to the even-numbered dimension of the word vector, and 2i+1 refers to the odd-numbered dimension. This encoding method, based on sine and cosine functions, allows for significant differences in the encoded vectors at different positions in a high-dimensional space, thus helping the model better distinguish words at different positions. Furthermore, using sine and cosine functions ensures the periodicity of the encoding, enabling the model to process longer input sequences.

[0076] Sub-step 3: Transformer Encoder layer

[0077] The model uses only Transformer Encoders, with text sequence information fed into multiple encoders of the same structure. Each encoder consists of multiple self-attention layers and a feedforward neural network, allowing for parallel computation on the global positional information of the input sequence. Within each self-attention layer, the model computes an attention matrix based on the input sequence, then performs a weighted average with the input sequence to obtain the encoder's output, extracting a higher-level text sequence vector representation.

[0078] Sub-step four: Classification layer

[0079] The output layer is a fully connected layer that outputs the specific results.

[0080] Furthermore, this embodiment uses the cross-entropy loss function as the loss function during the training phase, and the argmax function is used to predict the final class during the prediction phase. The cross-entropy loss function measures the difference between the model's prediction and the true label, and is widely used in classification problems. For a classification problem with C classes, assuming the true label of sample i is yi, and the model's predicted output is yi, then the cross-entropy loss function is defined as:

[0081]

[0082] Where N represents the number of samples, and if the i-th sample belongs to the c-th class, then yi,c is 1; otherwise, it is 0. This represents the probability that the model predicts sample i belongs to class c. The meaning of the cross-entropy loss function is: the closer the model's prediction is to the true label, the smaller the value of the loss function. In deep learning, the cross-entropy loss function is often used as the training loss function for classification models.

[0083] Step 3: Event Extraction

[0084] This invention uses the sequence labeling model BERT-BiLSTM-CRF to extract event trigger words, then employs a rule-based method to extract core event words. A rule base is established through data analysis, and the rule base is updated iteratively based on errors found in the extraction results. Finally, rule-based text numerical knowledge extraction is used. This invention defines text numerical knowledge as a binary form consisting of core words and numerical description information. Since numerical knowledge and logical knowledge share the same format, this facilitates the extraction of logical relationships and streamlines model processing.

[0085] Sub-step 1: Extracting event trigger words

[0086] Both event trigger word extraction and named entity recognition can be viewed as sequence labeling tasks. Furthermore, this embodiment of the invention uses the sequence labeling model BERT-BiLSTM-CRF to extract event trigger words, but it differs from the NER task in terms of objectives, labeling methods, and difficulty. The objective of entity recognition in the intelligent decision-making domain of cement clinker production is to extract named entities from text sequences in the building materials manufacturing domain, such as rotary kiln and limestone; while the objective of event trigger word extraction is to extract words that identify the occurrence of events from the domain's text corpus data, i.e., event trigger words.

[0087] Sub-step 2: Rule-based event keyword extraction

[0088] To further reduce the cost of manual annotation, this embodiment of the invention employs a rule-based method to extract core words for events. The basic process of rule-based natural language processing is as follows: Figure 4 As shown, this is an iterative process. A rule base is built manually by analyzing data, and then the rule base is updated and iterated based on the errors in the extraction results. The quality of the rule base will directly affect the effectiveness of the knowledge extraction task. The establishment of the rule base is a very important step.

[0089] Sub-step 3: Rule-based text numerical knowledge extraction

[0090] The rule-based text numerical knowledge extraction process of this invention is as follows: First, determine the node number of the quantity value in the analysis tree. The noun closest to the node number is taken as the core word of the numerical knowledge. Then, add additional rules similar to those in the event extraction section to complete the core words in the quantity knowledge. The numerical description information is described using the upper and lower limits of the numerical range.

[0091] Specifically, when the quantifier is a certain numerical value, the upper and lower limits of the numerical description information are the same. For example, the numerical description information of the text "the decomposition rate in the kiln is 95%" is [95%, 95%]. To standardize the format of quantitative knowledge, two sets are predefined to identify the upper and lower limits of quantitative knowledge. Set 1: {">", "≥", "greater than", "greater than or equal to", "not less than", "exceeding", "higher than"}, and Set 2: {<", "≤", "less than", "less than or equal to", "not higher than", "not exceeding", "lower than"}. When the core word of quantitative knowledge and the word between the quantifiers appear in Set 1, "H" is used to indicate the upper limit. For example, the numerical description information of "greater than 350℃" is [350℃, H]. When it appears in Set 2, "L" is used to indicate the lower limit. For example, the numerical description information of "less than 30%" is [L, 30%].

[0092] This paper introduces a rule-based numerical knowledge extraction method using a specific example. The text sequence is: "Kiln head exhaust fan inlet temperature > 350℃.", and the dependency syntax tree is as follows: Figure 5 As shown.

[0093] Regularization formulas are used to determine quantifiers. The noun "temperature" is closest to the quantifier "350℃," but "temperature" as the core quantifier in the text obviously cannot describe this knowledge. By adding rules to complete the core quantifier, the nodes with an ATT relationship to "temperature" are "kiln exhaust fan" and "inlet," resulting in the completed core quantifier being "kiln exhaust fan inlet temperature." The relationship between the core quantifier and the numerical quantifier is ">," so the numerical description information can be represented as [350℃, H]. This invention defines textual numerical knowledge as a binary form consisting of a core quantifier and numerical description information. The format of numerical knowledge is the same as that of logical knowledge, which facilitates the extraction of logical relationships and makes model processing easier.

[0094] Step 4: Extracting logical relationships

[0095] This invention adjusts the R-BERT Transformer to address the problem of discrepancies between event and entity descriptions. Simultaneously, the model incorporates paragraph information from the text at the classification layer, specifically categorized as: input text taken from the same paragraph, input text taken from adjacent paragraphs, and text taken from two sibling paragraphs. The event relationship extraction model for the cement clinker production process is as follows: Figure 6 As shown, specifically:

[0096] (1) Input layer

[0097] The model input includes: a text sequence (Input Seq), location information of two event elements (Event1 Mask, Event2 Mask), and text sequence paragraph information (Paragraph Info). In the event element location information, the position of the core event word is marked with 1, the position of the event trigger word is marked with 2, and other positions are marked with 0; the position of the core quantitative knowledge word is marked with 1, the quantitative information descriptive words are marked with 2, and other positions are marked with 0. One-hot encoding is used for the text paragraph information, specifically [1,0,0] indicates that the text is taken from the same paragraph, [0,1,0] indicates that the text is taken from a superior-inferior paragraph, and [0,0,1] indicates that the text is taken from two sibling paragraphs.

[0098] (2) Information embedding layer

[0099] The character embedding information of the input text sequence is obtained using the BERT model from the previous step, and the text information embedding is represented by Seq Embeds. Based on the two entity location information Event1 Mask and Event2 Mask, the embedding information of the two event elements is calculated respectively. This embedding information is represented by the mean vector of the event element or quantitative knowledge element in Seq Embeds. The mean vector is passed through a fully connected layer and an activation function to obtain the event element embedding vector. The event element embedding vector is concatenated with the Seq Embeds to obtain the new input sequence Seq Embeds. The BERT positional encoding method is used to encode the information of each position in the input sequence into a vector, and this vector is concatenated with the Seq Embeds to obtain the output of this layer.

[0100] (3) Transformer Encoder layer

[0101] The input sequence is fed into multiple encoders with the same structure. The encoder, composed of stacked self-attention layers and feedforward neural networks, allows for parallel computation based on the global positional information of the input sequence. Within each self-attention layer, the model computes an attention matrix based on the input sequence, then performs a weighted average with the input sequence to obtain the encoder's output, extracting a higher-level text sequence vector representation.

[0102] (4) Output layer

[0103] The paragraph information in one-hot encoding from the input layer is transformed into an embedding vector and concatenated with the sentence vector representation output from the Transformer layer. The concatenated result is then fed into a fully connected layer, using softmax as the activation function. Cross-entropy loss is used as the loss function during training, while argmax is used to predict the final class during the prediction phase.

[0104] Another embodiment of the present invention provides a knowledge graph construction device for intelligent decision-making in cement production processes, comprising:

[0105] Entity recognition module: Used to employ the BERT-BiLSTM-CRF entity recognition model that integrates word segmentation features, and integrates Chinese word segmentation features on the basis of character-based methods to obtain sequence labels of input text;

[0106] The entity relation extraction module is used to extract domain text event relations based on the R-BertTransformer entity relation extraction model.

[0107] The event extraction module is used to extract event trigger words using the sequence labeling model BERT-BiLSTM-CRF, then extract event core words using a rule-based method, and finally extract text numerical knowledge using a rule-based method.

[0108] The event relationship extraction module is used to extract the event relationships between events based on a deep learning-based event relationship extraction model.

[0109] Another embodiment of the present invention provides an electronic device, including a processor and a storage medium, wherein a computer program is stored on the storage medium, and the processor is used to execute the knowledge graph construction method of the present invention when the computer program is invoked.

[0110] Another embodiment of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, is used to implement the knowledge graph construction method of the present invention.

Claims

1. A method for constructing a knowledge graph for intelligent decision-making, characterized in that, The method includes: Step 1: Entity Recognition The input text data is processed using the BERT-BiLSTM-CRF entity recognition model, which integrates word segmentation features. Based on the character-based method, Chinese word segmentation features are fused to obtain the sequence label of the input text, i.e., the named entity type. Step 2: Entity Relationship Extraction The R-BertTransformer entity relation extraction model is used to extract domain text event relations; Step 3: Event Extraction The BERT-BiLSTM-CRF sequence labeling model was used to extract event trigger words, followed by the extraction of event core words using a rule-based method, and finally the extraction of text numerical knowledge using a rule-based method. Step 4: Extracting logical relationships A deep learning-based event relationship extraction model extracts the logical relationships between events; specifically, the event relationship extraction model is as follows: (1) Input layer The model input includes: a text sequence, the location information of two event elements, and paragraph information of the text sequence; in the location information of event elements, the position of the core event word is marked with 1, the position of the event trigger word is marked with 2, and other positions are marked with 0; the position of the core text quantity knowledge word is marked with 1, the quantity information descriptive words are marked with 2, and other positions are marked with 0; one-hot encoding is used for the text paragraph information, specifically [1,0,0] indicates that the text is taken from the same paragraph, [0,1,0] indicates that the text is taken from the upper and lower level paragraphs, and [0,0,1] indicates that the text is taken from two sibling paragraphs; (2) Information embedding layer The character embedding information of the input text sequence is obtained using the BERT model in step three, and the text information embedding is represented by SeqEmbeds. The embedding information of the two event elements is calculated based on the two entity position information Event1 Mask and Event2 Mask respectively. It is represented by the mean vector of the event element or the quantitative knowledge element in Seq Embeds. The mean vector is passed through a fully connected layer and an activation function to obtain the embedding vector of the event element. The event element embedding vector is concatenated with Seq Embeds to obtain the new input sequence Seq Embeds. The BERT positional encoding method is used to encode the information of each position in the input sequence into a vector, and concatenated with Seq Embeds to obtain the output of this layer. (3) Transformer Encoder layer The input sequence is fed into multiple encoders with the same structure; the encoder is composed of multiple self-attention layers and feedforward neural networks stacked together, and performs parallel computation on the global positional information of the input sequence; in each self-attention layer, the model calculates an attention matrix based on the input sequence, and then performs a weighted average with the input sequence to obtain the encoder output, extracting a higher-level text sequence vector representation; (4) Output layer The paragraph information in one-hot format from the input layer is transformed into an embedding vector and concatenated with the sentence vector representation output from the Transformer layer. The concatenated result is then input into a fully connected layer, with softmax as the activation function. Cross-entropy loss is used as the loss function during the training phase, while argmax is used to predict the final category during the prediction phase.

2. The knowledge graph construction method for intelligent decision-making according to claim 1, characterized in that, The process of entity recognition using the BERT-BiLSTM-CRF entity recognition model that integrates word segmentation features is as follows: BERT is used to obtain the token vector representation of each character in the input text sequence, and the corresponding word segmentation information is obtained by the jieba word segmentation tool and mapped to the embedding vector; The character vector and the word segmentation feature vector are concatenated and then input into BiLSTM to extract features from the text sequence; The feature vectors output by the upper-layer BiLSTM are labeled using a CRF (Conditional Random Field), i.e., a labeled sequence is output.

3. The knowledge graph construction method for intelligent decision-making according to claim 2, characterized in that, When using BERT to obtain the character vector representation of a text sequence, positional information is incorporated. The start and end positions of the text sequence are marked with the special characters "[CLS]" and "[SEP]", respectively. If the input text sequence consists of multiple sentences, a special character "[SEP]" is added between adjacent sentences.

4. The knowledge graph construction method for intelligent decision-making according to claim 3, characterized in that, In step two, when extracting entity relationships, the entity representation is concatenated with the input text sequence representation using a pooled feature approach. The concatenated sequence features are then input into the Transformer encoder structure to obtain contextual information between the text sequence and the entities. Finally, the entity location information is embedded using the BERT location information encoding method.

5. The knowledge graph construction method for intelligent decision-making according to claim 4, characterized in that, The R-BertTransformer entity relation extraction model includes: Input layer: The input layer refers to the input text sequence. The input information also includes the position information of the two entities in the sentence, where the position of the entity character is marked with 1 and the position of other characters is marked with 0. Entity location information embedding layer: The character embedding vectors of the input text sequence are obtained using the BERT model, and the text sequence embedding is represented by Seq Embeds; Transformer Encoder layer: The text sequence information is input into multiple encoders with the same structure. Each encoder consists of multiple self-attention layers and feedforward neural networks, and performs parallel computation on the global position information of the input sequence. Classification layer: The output layer is a fully connected layer that outputs specific results.

6. The knowledge graph construction method for intelligent decision-making according to any one of claims 1-5, characterized in that, When using a rule-based approach to extract core keywords for events, a rule base is established by analyzing the data, and the rule base is updated and iterated based on the errors in the extraction results.

7. The knowledge graph construction method for intelligent decision-making according to claim 6, characterized in that, When extracting numerical knowledge from text using a rule-based approach, the node number of the quantity value in the dependency syntax tree is first determined. The noun closest to the node number is taken as the core word of the numerical knowledge. The core words of the quantity knowledge are then supplemented by additional rules. The numerical description information is described using the upper and lower limits of the numerical range.

8. A knowledge graph construction device for intelligent decision-making, characterized in that, include: Entity recognition module: Used to employ the BERT-BiLSTM-CRF entity recognition model that integrates word segmentation features, it integrates Chinese word segmentation features on the basis of character-based methods to obtain sequence labels of input text; The entity relation extraction module is used to extract domain text event relations based on the R-BertTransformer entity relation extraction model. The event extraction module is used to extract event trigger words using the sequence labeling model BERT-BiLSTM-CRF, then extract event core words using a rule-based method, and finally extract text numerical knowledge using a rule-based method. The event relationship extraction module is used to extract the event relationships between events based on a deep learning-based event relationship extraction model. Specifically, the event relationship extraction model is as follows: (1) Input layer The model input includes: a text sequence, the location information of two event elements, and paragraph information of the text sequence; in the location information of event elements, the position of the core event word is marked with 1, the position of the event trigger word is marked with 2, and other positions are marked with 0; the position of the core text quantity knowledge word is marked with 1, the quantity information descriptive words are marked with 2, and other positions are marked with 0; one-hot encoding is used for the text paragraph information, specifically [1,0,0] indicates that the text is taken from the same paragraph, [0,1,0] indicates that the text is taken from the upper and lower level paragraphs, and [0,0,1] indicates that the text is taken from two sibling paragraphs; (2) Information embedding layer The character embedding information of the input text sequence is obtained using the BERT model in step three, and the text information embedding is represented by SeqEmbeds. The embedding information of the two event elements is calculated based on the two entity position information Event1 Mask and Event2 Mask respectively. It is represented by the mean vector of the event element or the quantitative knowledge element in Seq Embeds. The mean vector is passed through a fully connected layer and an activation function to obtain the embedding vector of the event element. The event element embedding vector is concatenated with Seq Embeds to obtain the new input sequence Seq Embeds. The BERT positional encoding method is used to encode the information of each position in the input sequence into a vector, and concatenated with Seq Embeds to obtain the output of this layer. (3) Transformer Encoder layer The input sequence is fed into multiple encoders with the same structure; the encoder is composed of multiple self-attention layers and feedforward neural networks stacked together, and performs parallel computation on the global positional information of the input sequence; in each self-attention layer, the model calculates an attention matrix based on the input sequence, and then performs a weighted average with the input sequence to obtain the encoder output, extracting a higher-level text sequence vector representation; (4) Output layer The paragraph information in one-hot format from the input layer is transformed into an embedding vector and concatenated with the sentence vector representation output from the Transformer layer. The concatenated result is then input into a fully connected layer, with softmax as the activation function. Cross-entropy loss is used as the loss function during the training phase, while argmax is used to predict the final category during the prediction phase.

9. An electronic device comprising a processor and a storage medium, characterized in that, The storage medium stores a computer program, and the processor is used to execute the method of any one of claims 1-7 when the computer program is invoked.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it is used to implement the method of any one of claims 1-7.

Images