A power technology standard entity relationship extraction method based on multi-round automatic question answering

Abstract

This invention proposes a method for extracting entity relations from power technology standards based on multi-turn automatic question answering, comprising: Step 1. Constructing a corpus of power technology standards; Step 2. Constructing information extraction element templates for different types of power technology standards corpus; Step 3. Constructing a question-and-answer corpus of power technology standards based on the information extraction element templates; Step 4. Constructing a text classification module for the power technology standards corpus described in Step 1; Step 5. Automatically constructing a multi-turn question-and-answer and question module by matching the text classification results with the information extraction element templates in Step 4. This module breaks down complex questions into simpler ones and provides step-by-step reasoning answers; Step 6. Constructing a machine reading comprehension module for the power technology standards question-and-answer corpus, which provides step-by-step reasoning answers to the questions automatically constructed in Step 5, thus completing the extraction method of this invention. This invention can effectively alleviate the phenomena of overlapping relations and cross-sentence dependencies in complex texts.

Description

Technical Field

[0001] This invention relates to the field of entity relation extraction technology in the field of power technology, and in particular to a method for extracting entity relations of power technology standards based on multi-turn automatic question answering. Background Technology

[0002] Power technology standards are a crucial guarantee for safe production and high-quality development of equipment management, a significant reflection of an enterprise's outstanding competitiveness, and a prerequisite for all technical work. "Accelerating the digitization and modularization of technical standards, building core applications such as precise retrieval, intelligent question answering, and auxiliary analysis, and gradually achieving deep integration of technical standards with on-site operations to strengthen the empowerment of grassroots levels" is an important requirement for the construction of a modern equipment management system. Due to factors such as insufficient accuracy of existing automatic entity relationship extraction modules, significant performance loss of modules trained on general corpora in specialized fields, and insignificant transfer learning effects, constructing a high-performance entity relationship extraction module specifically for the power technology standards field has become an urgent problem to be solved.

[0003] Entity relation extraction, a crucial foundational task in information extraction, has long garnered significant attention from researchers. It aims to extract entities and their relationships from unstructured text. The extracted information provides the knowledge base for downstream tasks in natural language processing, such as knowledge graph construction, information retrieval, and intelligent question answering. Early on, researchers typically divided it into two subtasks: named entity recognition and relation extraction, employing a pipelined framework to complete them separately. However, because the subtasks were completely independent within this framework, the error accumulation problem of the modules has been a persistent concern. With the rise of end-to-end modules, a joint entity relation extraction framework has attracted considerable attention. This framework integrates the two subtasks into a structured prediction framework or performs multi-task learning through shared representations, achieving joint modeling of the subtasks. However, recent research indicates that joint modeling does not necessarily outperform the pipelined approach.

[0004] Despite significant advancements in entity relation extraction (ERP) research, most current studies focus on general-domain datasets such as news reports. Entities are typically categorized into common types like time, location, names, and organizations, where dependencies are well-examined and triple schemas are well-defined. For example, in the sentence "Beijing is the capital of China," a module can extract the relation using a single dependency step, obtaining (Beijing-city, capital-ReL, China-Country). However, such short texts or single-relation samples are rare in practical applications. Most samples are long texts with multiple relations, and entity pairs may have significant dependency distances, located in different sentences (cross-sentence entity pairs). Furthermore, entity pairs may have multiple relations, and different relations may share the same entity (relation overlap). This is prevalent in complex knowledge samples within specific domains. Existing methods struggle to address relation overlap effectively.

[0005] Existing entity relationship extraction frameworks mainly employ the following methods:

[0006] 1. Entity Relationship Extraction Based on Pipeline Framework

[0007] Early entity-relation extraction modules mostly adopted a pipeline framework, where researchers decomposed the task into two subtasks: named entity recognition and relation extraction. Depending on the execution order of the subtasks, these can be further divided into two modes: entity-first, relation-second; and relation-first, entity-second. The former first extracts entities of the intended category through the entity recognition module, and then uses these extracted entities as priors to predict possible relationships; the latter first extracts relationships from the input text through relation classification, and then uses these as priors to perform entity recognition. While the pipeline framework has broad applicability and can flexibly handle entity-relation extraction tasks in almost all domains, the completely independent execution of subtasks within this framework ignores the interaction between the entity and relation modules. This leads to errors from upstream tasks accumulating in downstream tasks, and such error propagation is irreversible.

[0008] 2. Joint Entity Relationship Extraction

[0009] To address the error accumulation problem in pipelined frameworks, jointly trained entity-relation extraction modules have been proposed. Numerous excellent studies have demonstrated that these modules can model the interactions between entities and relations, effectively mitigating the error accumulation issue. Researchers typically categorize them into two types: structured prediction and multi-task learning.

[0010] Structured prediction methods integrate these two tasks into a unified framework, including: (1) employing an action-based system that can identify new entities and link them to previous entities; (2) using a table-filling method where the diagonal of the table is filled with entity tags and the rest with relation tags representing possible relationships between pairs of entities; (3) using a graph neural network-based approach to jointly predict entity and relation types; and (4) transforming the task into a multi-turn question-answering problem. However, all of these methods require solving a global optimization problem.

[0011] Most multi-task learning methods adopt a cascaded approach to solve joint problems, still requiring the construction of two separate entity recognition and relation extraction modules. These modules are optimized together through parameter sharing strategies, including: (1) using a sequence labeling module for entity prediction and a tree-based LSTM module for relation extraction; and (2) using span representation to predict relations. However, these methods still perform pipelined decoding, i.e., first extracting entities and then using the relation module to predict the dependencies between entities. Summary of the Invention

[0012] To address the shortcomings of existing technologies, this invention proposes a multi-turn automatic question-answering-based entity relation extraction method. Based on current standard machine reading comprehension frameworks and combined with the automatic question construction strategy proposed in this invention, entity relation extraction is transformed into a multi-turn question-answering task: each entity type and relation type is represented by a question, and the entity and relation are extracted by the machine reading comprehension framework. The question can provide additional prior knowledge for semantic understanding, namely entity type and relation type; the answer is the text span corresponding to the question.

[0013] The technical solution adopted by the present invention to achieve the above objectives is as follows:

[0014] A method for extracting entity relationships from power technology standards based on multi-turn automatic question answering includes the following steps:

[0015] Step 1. Construct a corpus of power technology standards;

[0016] Step 2. Construct information extraction element templates for different types of power technology standard corpora;

[0017] Step 3. Construct a corpus of questions and answers on power technology standards based on the information extraction element template;

[0018] Step 4. Construct a text classification module based on the power technology standard corpus described in Step 1;

[0019] Step 5. Automatically construct a multi-round question and answer module based on the text classification results matching information extraction element template in Step 4. Its function is to break down complex questions into simple questions and perform reasoning and answering step by step.

[0020] Step 6. Construct a machine reading comprehension module based on the power technology standard question-and-answer corpus to gradually reason and answer the questions automatically constructed in Step 5, thus completing a power technology standard entity relation extraction method based on multi-turn automatic question answering.

[0021] Furthermore, the construction of the power technology standard corpus mentioned in step 1 involves selecting power technology standards based on actual business operations at the grassroots level, processing them, and then storing them as corpus data.

[0022] Furthermore, step 2, which involves constructing information extraction element templates for different types of power technology standard corpora, divides the data samples into extraction, numerical, statistical, and judgment categories based on the types of key information they contain. Then, it sets the extraction elements for each template to provide question templates for the subsequent power technology standard question-and-answer corpus.

[0023] Furthermore, step 3, which involves constructing a question-and-answer corpus for power technology standards based on the information extraction element template, provides training data for the machine reading comprehension module during the training phase. Each power technology standard generates multiple rounds of questions and answers based on the information extraction element template as input, and the corresponding entity relationships serve as answers. After processing, the data is stored as corpus data in the form of question-and-answer pairs.

[0024] Furthermore, the text classification module constructed for the power technology standard corpus in step 4 is to predict the category of key information in the input text and match it with the information extraction element template described in step 2. Based on the category of the information, the corresponding information extraction element template is assigned.

[0025] Furthermore, the automatic construction of the multi-round question-and-answer module for extracting elements from the text classification results matching information described in step 5 is automatically constructed based on the corpus categories output by the text classification module. In the multi-round question-and-answer stage, the module first extracts the head entity, and then automatically constructs the question for the next round of question-and-answer based on the tail entity category, relation category, and the answers obtained in each round of question-and-answer as set in the template, thereby jointly extracting relations and tail entities. When multiple answers are obtained in the question-and-answer stage, it is necessary to traverse and construct the questions corresponding to all answers in the next round of question-and-answer stage.

[0026] Furthermore, the machine reading comprehension module for constructing the question-and-answer corpus for power technology standards described in step 6 extracts the answer range corresponding to the question from the input samples, including input, output, parameter settings, and evaluation criteria.

[0027] Furthermore, the input is for each given sentence sample C = {w1, w2, ... w...} n}, and the question used to ask the question Q={Qw1,Qw2,…Qw n}, which is the input sequence S = {[CLS], Qw1, Qw2, ..., Qw} required to construct the BERT pre-training module. n [SEP], w1, w2, … w n The tokens are first encoded using the RoBERTa pre-training module to obtain word vector representations H. Finally, a fully connected layer (FC) is used to obtain the probability of each token as the start and end positions of the predicted answer span, which are respectively... The expression is as follows:

[0028]

[0029] In the formula, H = {h1, h2, ... h} n} represents the word vectors of all tokens in the input sequence; W is the weight parameter matrix of the fully connected layer, and b is the bias vector. Let represent the probabilities that the token at position i in the input sequence S is the starting and ending position of the answer, respectively.

[0030] The complete expression for predicting the span probability of the answer is as follows:

[0031]

[0032] In the formula, p(r) i |S,i) represents the probability that the token at position i in the input sequence S is predicted to be true as both the start and end position of the answer. Let represent the probabilities that the token at position i in the input sequence S is the starting and ending position of the answer, respectively.

[0033] During the training phase, the crossEntropyLoss function is used to calculate the loss, as shown in the following expression:

[0034]

[0035] In the formula, crossEntropyLoss is the cross-entropy loss function, i.e., the loss in the above formula. Let p(r) be the summation function. i |S, i ) represents the probability that the token at position i in the input sequence S is predicted to be true as both the start and end position of the answer, and log() is the logarithmic function.

[0036] Furthermore, the output is based on the output of the machine reading comprehension module, which yields the answer span set A = {A1, A2, ..., A...} for each question Q. k The probability scores are standardized using the Softmax function, and the answer with the largest score span is the optimal answer to question Q. The expression is as follows:

[0037] p(best_A)=Softmax(A1,A2,…A k (1-4)

[0038] In the formula, p(best_A) represents the span of the answer with the highest probability in the Q-question, and A = {A1, A2, ..., A}. k} represents the set of answer spans corresponding to question Q. Since multiple answer spans will be output, it is set to k. Softmax is the normalization function.

[0039] Furthermore, the parameter settings involve using the linear scheduling Adam with a warmup rate of 0.05 for parameter optimization during the training phase; the pre-training module used is the RoBERTa-wwm-ext pre-training module, which has 12 layers and a hidden layer vector dimension of 768; the hidden layer vector dimension of the Bi-LSTM network in the text classification module is 128, and the learning rate is 5e-5; in the machine reading comprehension module of the answer extraction module, the maximum sentence length is set to 256, and the learning rate is 3e-5; the answer extraction module is implemented by calling the core algorithm of the machine reading comprehension module.

[0040] The evaluation criteria use precision, recall, and F1 score to assess the module's performance, expressed as follows:

[0041]

[0042]

[0043]

[0044] In the formula, P is precision, R is recall, F1 is the F1 score, and T is the F1 score. p F is the number of positive samples that are classified as positive samples. p F is the number of negative samples that are classified as positive samples. n It is the number of positive samples that are classified as negative samples.

[0045] Compared with the prior art, the present invention has the following beneficial effects and advantages:

[0046] This invention employs a multi-turn question-answering mechanism, enabling the module to perform multi-step reasoning. Through multi-step reasoning, it extracts entities and their relationships from complex knowledge samples, effectively constructing entity dependencies within these samples. Unlike traditional machine reading comprehension methods that use pre-defined schemas for general domains, this invention proposes an automatic question construction mechanism that automatically constructs machine reading comprehension framework questions based on the categories of key information in the input text. Addressing the common issues of overlapping relationships and entity pairs spanning multiple sentences, a complex knowledge sample dataset oriented towards power technology standards is also constructed. This effectively alleviates the problems of long dependency distances between entity pairs and overlapping relationships, providing a research sample for entity relation extraction from complex knowledge samples and laying a data foundation for downstream natural language processing tasks in the power technology standards field.

[0047] Furthermore, this invention constructs a power technology standard dataset that includes master-slave standards, achieving optimal performance on this dataset. The extracted information provides an accurate knowledge foundation for downstream tasks in the field of natural language processing, such as knowledge graph construction, information retrieval, and intelligent question answering. Attached Figure Description

[0048] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0049] Figure 1 This is an overall architecture diagram of a module proposed by a method for extracting entity relationships in power technology standards based on multi-turn automatic question answering. Detailed Implementation

[0050] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0051] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0052] The following reference Figure 1 Tables 1-5 describe the technical solutions of some embodiments of the present invention.

[0053] Example 1

[0054] like Figure 1 The diagram shown is an overall architecture diagram of the present invention, which specifically includes the following steps;

[0055] Step 1. Construct a corpus of power technology standards;

[0056] Step 2. Construct information extraction element templates for different types of power technology standard corpora;

[0057] Step 3. Construct a question-and-answer corpus for power technology standards;

[0058] Step 4. Construct a text classification module for the power technology standard corpus;

[0059] Step 5. Automatically construct multi-round questions by extracting feature templates from text classification results and matching information;

[0060] Step 6. Construct a machine reading comprehension module based on the question-and-answer corpus of power technology standards.

[0061] Example 2

[0062] The construction of the power equipment corpus mentioned in step 1 specifically includes a total of 37 power technical standards, with the selected technical annotations mainly revolving around actual grassroots business. After processing, these are stored as corpus data.

[0063] Example 3

[0064] Based on the data samples, the data is categorized into extraction, numerical, statistical, and judgment types according to the types of key information contained. Step 2 constructs corresponding information extraction element templates for these four types of corpora. The extraction elements set for each template are shown in Table 1. The main goal is to provide question templates for the subsequent automatic question-answering module.

[0065] Table 1: Summary of Information Extraction Template Elements

[0066]

[0067] Example 4

[0068] Step 3 describes the construction of a question-and-answer corpus for power technology standards. This corpus primarily provides training material for the machine reading comprehension module during the training phase. Each technology standard extracts templates based on the information provided in Step 2 to generate multi-round question-and-answer questions as input, and the corresponding entity relationships serve as answers. After processing, the data is stored as corpus data in the form of question-and-answer pairs.

[0069] Example 5

[0070] Step 4 describes constructing a text classification module based on the power technology standard corpus; from Figure 1 As can be seen, the text classification module is the head of the module proposed in this invention. Its main goal is to predict the category of key information in the input text and match it with the information extraction templates of the four categories of corpus constructed in step 2. The corresponding information extraction template is assigned according to the entity category of the information.

[0071] This invention employs a BERT+Bi-LSTM+Attention network architecture for text classification, which has been widely applied in various text classification and relation extraction tasks and has proven to have excellent performance. The text classification module may output multiple categories of corpus, depending on the types of key information contained in the input samples.

[0072] Example 6

[0073] Step 5 describes the automatic construction of a multi-round problem involving feature templates for extracting matching information from text classification results. This step corresponds to... Figure 1 The module for multi-turn question answering and automatic question construction is described in the text classification module of step 4. The main objective of this module is to automatically construct questions for a multi-turn question answering framework based on the corpus categories output by the text classification module in step 4. In the multi-turn question answering stage, the module first extracts head entities. In the field of power technology standards, the head entity for the first round of question answering is fixed as the transformer category. Then, based on the tail entity category, relation category, and the answers (head entities) obtained in each round of question answering set in the template, the module automatically constructs the questions for the next round of question answering, thereby jointly extracting relations and tail entities. When multiple answers are obtained, it is necessary to iterate through and construct the questions corresponding to all answers in the next round of question answering.

[0074] Example 7

[0075] Step 6 describes the construction of a machine reading comprehension module based on a question-and-answer corpus for power technology standards. This step corresponds to... Figure 1 The answer extraction module in the code aims to extract the answer span corresponding to the question from the input sample. It employs a machine reading comprehension module based on answer extraction, with a network structure of "BERT+FC". The specific execution process of each layer of this module is as follows:

[0076] Input: For each given sentence sample C = {w1, w2, ... w...} n}, and the question used to ask the question Q={Qw1,Qw2,…Qw n The present invention constructs the input sequence S = {[CLS], Qw1, Qw2, ... Qw} required for the BERT pre-training module. n [SEP], w1, w2, … w n [SEP]}. Then, the word vector representation H is obtained by encoding it through the RoBERTa pre-training module used in this invention. Finally, a fully connected layer (FC) is used to obtain the probability of each token as the start and end position of the predicted answer span, respectively.

[0077]

[0078] In the formula, H = {h1, h2, ... h} n} represents the word vectors of all tokens in the input sequence. W is the weight parameter matrix of the fully connected layer, and b is the bias vector. Let represent the probabilities that the token at position i in the input sequence S is the starting and ending position of the answer, respectively.

[0079] The complete expression for predicting the span probability of the answer is as follows:

[0080]

[0081] In the formula, p(r) i |S,i) represents the probability that the token at position i in the input sequence S is predicted to be true as both the start and end position of the answer. Let represent the probabilities that the token at position i in the input sequence S is the starting and ending position of the answer, respectively.

[0082] During the training phase, the crossEntropyLoss function is used to calculate the loss, as shown in the following expression:

[0083]

[0084] In the formula, crossEntroyLoss is the cross-entropy loss function, i.e., the loss in the above formula. Let p(r) be the summation function. i |S,i) represents the probability that the token at position i in the input sequence S is predicted to be true as both the start and end position of the answer. log() is the logarithmic function.

[0085] Output: Based on the output of the machine reading comprehension module, the answer span set A = {A1, A2, ..., A} corresponding to each question Q can be obtained. k The probability scores are standardized using the Softmax function, and the answer span with the largest score is the optimal answer to question Q, as shown in the following expression.

[0086] p(best_A)=Softmax(A1,A2,…A k (1-4)

[0087] In the formula, p(best_A) represents the span of the answer with the highest probability in the Q-question. A={A1,A2,…A k} represents the set of answer spans corresponding to question Q. Since the module will output multiple answer spans, it is set to k. Softmax is the normalization function.

[0088] Parameter settings: During the training phase, Adam, a linear scheduler with a warmup rate of 0.05, was used for parameter optimization. The pre-training module used was the Chinese ROBERTa-wwm-ext pre-training module released by the Harbin Institute of Technology-iFlytek Joint Laboratory, which has 12 layers and a hidden layer vector dimension of 768. The hidden layer vector dimension of the Bi-LSTM network in the text classification module is 128, and the learning rate is 5e-5. In the machine reading comprehension module of the answer extraction module, the maximum sentence length is set to 256, and the learning rate is 3e-5.

[0089] Evaluation criteria: Precision, recall, and F1 score are used to evaluate the module's performance. The calculation formulas are as follows:

[0090]

[0091]

[0092]

[0093] In the formula, T p F is the number of positive samples that are classified as positive samples. p This is the number of negative samples that are classified as positive samples. F n It is the number of positive samples that are classified as negative samples.

[0094] Example 8

[0095] This embodiment will select an extraction corpus and use specific examples to illustrate the working principle of the automatic question construction mechanism proposed in this invention, as well as the execution process of the multi-round automatic question answering method for extracting entity relationships in power technology standards proposed in this invention.

[0096] Example of an extraction-type maintenance procedure: Detailed list of maintenance items for deterioration of the physical condition of an oil-immersed transformer (reactor), including: Situation 1: Blockage of the breathing circuit (Category C maintenance): Inspect the breather and pipelines, and perform oil treatment. Situation 2: High oil temperature (Category D maintenance): Check whether the cooler is operating normally, conduct core grounding current, infrared, and chromatographic tests to determine the cause. Situation 3: Normal oil temperature (Category B maintenance): Drain or add oil.

[0097] Includes the following steps:

[0098] Step 1. First, use the text classification module to determine which core information elements in the corpus belong to the extraction category;

[0099] Step 2. Based on the information extraction template in Table 1, obtain the information elements to be extracted from this corpus as follows: entity categories: transformer category, standard details, attribute, attribute value; relation categories: regulation, inclusion, attribute value.

[0100] Step 3. Extract entity relationships from the input sample using a multi-round question-and-answer and automatic question construction module, as follows:

[0101] First question: What types of transformers are covered by this power technology standard?

[0102] First round answer: Oil-immersed transformer

[0103] Second round of questions: What specific details are specified in this power standard for oil-immersed transformers?

[0104] Second round answer: Deterioration of ontological state variables

[0105] Third question: What attributes are included in the degradation of the physical state variables of an oil-immersed transformer?

[0106] Third round of answers: Breathing circuit blockage, oil temperature too high, oil temperature normal.

[0107] Fourth question: When the physical state variables of an oil-immersed transformer deteriorate and the breathing circuit is blocked, what is the corresponding attribute value?

[0108] Fourth round answer: Category C maintenance

[0109] Fourth question: When the physical state of an oil-immersed transformer deteriorates and the oil temperature is too high, what is the corresponding attribute value?

[0110] Fourth round answer: Category D maintenance

[0111] Fourth question: When the physical state of an oil-immersed transformer deteriorates while the oil temperature remains normal, what are the corresponding attribute values?

[0112] Fourth round answer: Category B maintenance

[0113] Following the setting that the head entity of the first round of question-and-answer in the power standards field is fixed as the transformer category, the first step is to obtain the corresponding answer as "oil-immersed transformer" through the answer extraction module. Then, based on the entity relationship category settings in the extraction element template of the extraction corpus information, the second round of questions is constructed according to the first round answer "oil-immersed transformer", the entity category "standard details", and the relationship "regulation". The tail entity and relationship are jointly extracted by the answer extraction module to obtain entity relationship triples (oil-immersed transformer - transformer category, regulation - relationship, entity state quantity degradation - standard details). This process is repeated to extract entity relationships in multiple rounds of automatic question-and-answer, thereby extracting all entity relationships. It is worth noting that not all entity relationship dependencies are presented in the form of triples. There are many hierarchical entity relationships in power standards. For example, in this standard, when the entity state quantity degradation of an oil-immersed transformer occurs under different faults, the required maintenance types are also different. This also proves that this invention is applicable to the extraction of hierarchical entity relationships. In summary, if the current round of questions returns multiple answers through the answer extraction module, and the information extraction template for this sample still exists for the next round of questions and answers, the automatic question construction mechanism will iterate through the answers of this round to construct multiple questions for the next round of questions and answers. Through the above method, this invention completes the entire entity relation extraction task, and the results are summarized as follows:

[0114] Table 2: Summary of Entity Relation Extraction Examples for Extracted Samples

[0115] Transformer categories Standard Details property Attribute value Oil-immersed transformer Deterioration of the bulk state quantity Breathing circuit blockage Category C maintenance Oil-immersed transformer Deterioration of the bulk state quantity Oil temperature is too high Category D maintenance Oil-immersed transformer Deterioration of the bulk state quantity Oil temperature is normal Category B maintenance

[0116] Example 9

[0117] This embodiment will select digital corpus and use specific examples to illustrate the working principle of the automatic question construction mechanism proposed in this invention, as well as the execution process of the multi-round automatic question answering method for extracting entity relationships of power technology standards proposed in this invention.

[0118] Numerical example: The transformer should undergo an oil tank cracking test. The test requirements are as follows: Select one transformer oil tank from the series of products and apply a positive pressure of 103 kPa (hydraulic pressure) to it. After 10 minutes, no cracking should occur.

[0119] Includes the following steps:

[0120] Step 1. First, the core information elements in the corpus are identified by the text classification module as belonging to the number category;

[0121] Step 2. Based on the information extraction template in Table 1, obtain the information elements to be extracted from this corpus as follows: entity category: transformer category, standard details, numerical attribute value; relation category: regulation, attribute value.

[0122] Step 3. Extract entity relationships from the input sample using a multi-round question-and-answer and automatic question construction module, as follows:

[0123] First question: What type of transformer is covered by this power standard?

[0124] First round answer: Oil-immersed power transformer

[0125] Second round of questions: What specific details are specified in this power standard for oil-immersed power transformers?

[0126] Second round of answers: Fuel tank cracking test

[0127] Third question: What numerical attribute values ​​correspond to the oil tank cracking test of an oil-immersed power transformer?

[0128] Third round answer: 103 kPa 10 min

[0129] Unlike extractive corpora where key information elements are generally categorized, numerical corpora exhibit strong category bias. The criteria for this category emphasize entities related to numbers, and these numbers often include units of measurement. Traditional entity relation extraction methods struggle to handle entity pairs with long dependency distances, and numerical entities typically require multiple rounds of reasoning for accurate location. Therefore, the text classification and multi-turn question-answering framework employed in this invention significantly alleviates the challenges of cross-sentence dependencies and multi-turn reasoning for entity pairs. The results are summarized below:

[0130] Table 3: Summary of Entity Relationship Extraction Examples from Numeric Samples

[0131] Transformer categories Standard Details Numeric attribute value Oil-immersed power transformer Fuel tank cracking test 103kPa Oil-immersed power transformer Fuel tank cracking test 10min

[0132] Example 10

[0133] This embodiment will select statistical corpus and use specific examples to illustrate the working principle of the automatic question construction mechanism proposed in this invention, as well as the execution process of the multi-round automatic question answering method for extracting entity relationships of power technology standards proposed in this invention.

[0134] Statistical examples: The main core of a 1000KV single-phase oil-immersed autotransformer can be divided into: five-column type (three main columns and two side yokes) and four-column type (two main columns and two side yokes).

[0135] Specifically, the following steps are included:

[0136] Step 1. First, use the text classification module to determine which core information elements in the corpus belong to the statistical category;

[0137] Step 2. Based on the information extraction template in Table 1, obtain the information elements to be extracted from this corpus as follows: entity category: transformer category, standard details, statistical attribute value; relation category: regulation, statistical attribute value.

[0138] Step 3. Extract entity relationships from the input sample using a multi-round question-and-answer and automatic question construction module, as follows:

[0139] First question: What type of transformer is covered by this power standard?

[0140] First round answer: 1000KV single-phase oil-immersed autotransformer

[0141] Second round of questions: What specific details are specified in this power standard for 1000kV single-phase oil-immersed autotransformers?

[0142] Second round answer: The main body becomes an iron core.

[0143] Third round of questions: What statistical attribute values ​​correspond to the main transformer core of a 1000KV single-phase oil-immersed autotransformer?

[0144] Third round answer: Five-pillar or four-pillar

[0145] In statistical corpora, the dependencies between entity pairs are mostly conceptual hierarchical relationships, and the dependencies between entity pairs are usually relatively simple. A pipelined framework can flexibly adapt to different semantic scenarios. However, in statistical corpora, entities of the same type are often densely packed, and the distance between entities is very close. If a two-stage framework is used for entity relation extraction, although the two subtasks are flexible in execution, they often get stuck in the dilemma of difficult entity pair matching or overlapping relations. On the other hand, since the distance between the same entities is almost adjacent, it also leads to the problem of entity nesting. Therefore, the text classification and multi-turn question answering framework adopted in this invention greatly alleviates the problems of entity pair matching and relation overlap in complex knowledge samples. The results are summarized as follows:

[0146] Table 4: Summary of Entity Relationship Extraction Examples for Statistical Samples

[0147]

[0148] Example 11

[0149] This embodiment will select judgment-type corpus and use specific examples to illustrate the working principle of the automatic question construction mechanism proposed in this invention, as well as the execution process of the multi-round automatic question answering method for extracting entity relationships of power technology standards proposed in this invention.

[0150] Judgment-based example: Oil-immersed shunt reactors used in 1000kV AC systems typically do not have winding thermometers. The specific steps include:

[0151] Step 1. First, use the text classification module to determine which core information elements in the corpus belong to the judgment class;

[0152] Step 2. Based on the information extraction template in Table 1, obtain the information elements to be extracted from this corpus as follows: entity category: transformer category, component category; relation category: judgment relation.

[0153] Step 3. Extract entity relationships from the input sample using a multi-round question-and-answer and automatic question construction module, as follows:

[0154] First question: What type of transformer is covered by this power standard?

[0155] First round answer: Oil-immersed shunt reactor for 1000kV AC system

[0156] Second round of questions: What is the judgment relationship for oil-immersed shunt reactors in 1000kV AC systems in this power standard?

[0157] Second round answer: None

[0158] Third question: What types of components are typically not used in oil-immersed shunt reactors for 1000kV AC systems?

[0159] Third round answer: Winding thermometer

[0160] In judgment-based corpora, the dependencies between entity pairs are often complex and diverse, lacking a unified relation type. Traditional entity relation extraction methods typically require a pre-defined schema, which is impractical for incorporating all relation types and increases computational burden during module training. Furthermore, an excessive number of target words in the relation vocabulary can decrease the overall accuracy of the module. Therefore, the text classification and multi-turn question-answering framework employed in this invention significantly alleviates the complexity of entity pair relationships. The results are summarized below:

[0161] Table 5: Summary of Entity Relationship Extraction Examples for Judgment Class Samples

[0162]

[0163] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for extracting entity relationships from power technology standards based on multi-turn automatic question answering, characterized in that, Includes the following steps: Step 1. Construct a corpus of power technology standards; Step 2. Construct information extraction element templates for different types of power technology standard corpora; based on the data samples, divide them into extraction, numerical, statistical, and judgment categories according to the types of key information they contain, and set the extraction elements for each template to provide question templates for the subsequent power technology standard question and answer corpus. Step 3. Construct a corpus of questions and answers on power technology standards based on the information extraction element template; Step 4. Construct a text classification module for the power technology standard corpus mentioned in Step 1. Its function is to divide the questions into four major categories in order to locate the question construction template. The text classification module is to predict the category of key information in the input text and match it with the information extraction element template in Step 2. According to the category of the information, the corresponding information extraction element template is assigned. Step 5. Automatically construct multi-round questions based on the text classification results matching information extraction element template in Step 4; In the multi-round question-and-answer stage, the module first extracts the head entity, and then automatically constructs the questions for the next round of question-and-answer based on the tail entity category, relation category and the answers obtained in each round of question-and-answer as set in the template, thereby jointly extracting relations and tail entities. When multiple answers are obtained in the question-and-answer stage, it is necessary to traverse and construct the questions corresponding to all answers in the next round of question-and-answer stage; Step 6. Construct a machine reading comprehension module based on the power technology standard question-and-answer corpus to gradually reason and answer the questions automatically constructed in Step 5, thus completing a power technology standard entity relation extraction method based on multi-turn automatic question answering.

2. The method for extracting entity relationships of power technology standards based on multi-turn automatic question answering as described in claim 1, characterized in that, The construction of the power technology standard corpus mentioned in step 1 involves selecting power technology standards based on actual business operations at the grassroots level, processing them, and storing them as corpus data.

3. The method for extracting entity relationships of power technology standards based on multi-turn automatic question answering as described in claim 1, characterized in that, Step 3 describes constructing a question-and-answer corpus for power technology standards based on the information extraction element template. This provides training data for the machine reading comprehension module during the training phase. Each power technology standard generates multi-round question-and-answer questions based on the information extraction element template as input, and the corresponding entity relationships as answers. After processing, the data is stored as corpus data in the form of question-and-answer pairs.

4. The method for extracting entity relationships of power technology standards based on multi-turn automatic question answering as described in claim 1, characterized in that, The machine reading comprehension module for the question-and-answer corpus of power technology standards described in step 6 is to extract the answer range corresponding to the question from the input sample, including input, output, parameter settings and evaluation criteria.

5. The method for extracting entity relationships of power technology standards based on multi-turn automatic question answering as described in claim 4, characterized in that, The input is for each given sentence sample C = {w1, w2, ... w n }, and the question used to ask the question Q = {Q w1, Q w2,…Q w n }, which is used to construct the input sequence S = {[CLS], Q w1, Q w2, ..., Q w} required for the BERT pre-training module. n [SEP], w1, w2, ... w n [SEP]}, the word vector representation H is obtained by encoding it through the RoBERTa pre-training module. Finally, a fully connected layer (FC) is used to obtain the probability of each token as the start and end position of the predicted answer span, respectively. , The expression is as follows: （1-1）； In the formula, H = {h1,h2,…h} n } is the word vector representation of all tokens in the input sequence; W is the weight parameter matrix of the fully connected layer, and b is the bias vector. Let represent the probabilities that the token at position i in the input sequence S is the starting and ending position of the answer, respectively. The complete expression for predicting the span probability of the answer is as follows: （1-2）； In the formula, p(r) i |S,i) represents the probability that the token at position i in the input sequence S is predicted as true as both the start and end positions of the answer. Let represent the probabilities that the token at position i in the input sequence S is the starting and ending positions of the answer, respectively. During the training phase, the crossEntropyLoss function is used to calculate the loss, as shown in the following expression: （1-3）； In the formula, crossEntropyLoss is the cross-entropy loss function, i.e., the loss in the above formula. `log()` is the summation function, and `log()` is the logarithmic function.

6. The method for extracting entity relationships of power technology standards based on multi-turn automatic question answering according to claim 4, characterized in that, The output is based on the output of the machine reading comprehension module, which yields the answer span set A = {A1, A2, ..., A} for each question Q. k The probability scores are standardized using the Softmax function, and the answer with the largest score span is the optimal answer to question Q. The expression is as follows: p(best_A)= Softmax{A1,A2,…A k } (1-4); In the formula, p(best_A) represents the span of the answer with the highest probability for question Q, and A = {A1, A2, ... A}. k } represents the set of answer spans corresponding to question Q. Since multiple answer spans will be output, it is set to k. Softmax is the normalization function.

7. The method for extracting entity relationships of power technology standards based on multi-turn automatic question answering according to claim 4, characterized in that, The parameter settings are as follows: during the training phase, Adam, a linear scheduler with a warmup rate of 0.05, is used for parameter optimization; the pre-training module used is the RoBERTa-wwm-ext pre-training module, which has 12 layers and a hidden layer vector dimension of 768; the hidden layer vector dimension of the Bi-LSTM network in the classification module is 128, and the learning rate is 5e-5; in the machine reading comprehension module of the answer extraction module, the maximum sentence length is set to 256, and the learning rate is 3e-5. The evaluation criteria use precision, recall, and F1 score to assess module performance, as shown in the following expression: （1-5）； （1-6）； （1-7）； In the formula, P is precision, R is recall, F1 is the F1 score, and T is the F1 score. P F is the number of positive samples that are classified as positive samples. p F is the number of negative samples that are classified as positive samples. n It is the number of positive samples that are classified as negative samples.

Images