A machine reading comprehension method for power professional knowledge

Abstract

The application relates to a machine reading comprehension method for electric power professional knowledge, wherein an original question is input into an answer retrieval module for preliminary retrieval to obtain a plurality of first answers and a prediction probability of each first answer; a mother, the original question and the first answer are input into an evidence extraction module to extract an evidence label corresponding to the first answer from text content of the mother; wherein the mother comprises original text, text content in a drawing and text content in a picture; the original question, the first answer and the corresponding evidence label are input into an answer verification module to determine a formal answer with the maximum probability from the plurality of first answers. The application is especially suitable for machine reading comprehension of multiple documents and has good accuracy.

Description

TECHNICAL FIELD

[0001] The application relates to a machine reading comprehension method of electric power professional technical knowledge and belongs to the technical field of artificial intelligence of electric power professionals. BACKGROUND

[0002] At present, the main position of enterprise innovation is strengthened, and various innovation elements are gathered to enterprises. By constructing a system and mechanism fully reflecting the value of knowledge, the aggregation of funds, technology, talent, management, knowledge and data innovation elements is promoted, and data is taken as an important production factor, and artificial intelligence, machine learning and the like are key technologies of digital transformation.

[0003] The unit constructs a knowledge management system of a collection of engineering document library, expert library, problem library, label library and standard system library, changes the implicit knowledge in the field of electric power professional technical knowledge into explicit knowledge, improves the willingness of unit staff knowledge sharing and learning, and solves the problem that knowledge cannot be effectively deposited and inherited.

[0004] 1. Taking the project and the power transformation profession as a first test field, personalized intelligent knowledge management application of electric power professional technology is created, the service quality and efficiency of engineering accumulated knowledge experience are effectively improved, the existing intelligent knowledge base is perfected, and convenient, intelligent and personalized knowledge service ability support is provided for all project engineering and professional personnel.

[0005] 2. The knowledge point association between the professional knowledge base and the intelligent question answering is established, the quality of engineering knowledge retrieval service is improved, the existing knowledge resources are efficiently utilized, and the system barrier is eliminated.

[0006] 3. The intelligent knowledge base is implemented in the project and the power transformation profession, so that engineering problems are better solved, engineering experience is summarized, engineering data is integrated, engineering construction is high-quality and efficient, an interactive dynamic data management center is formed, the problems that knowledge and experience follow people are solved, knowledge and experience are solidified, and a complete knowledge system is established.

[0007] 4. Based on the existing artificial intelligence semantic understanding technology, the reading comprehension key technology suitable for electric power professional technical knowledge is researched, the interactive function of using a machine to help users quickly organize massive engineering knowledge and complete intelligent question answering is realized, the understanding of electric power professional documents by professional personnel is facilitated, the quality and efficiency of learning are improved, and the professional skills of employees are improved.

[0008] 5. Develop an intelligent knowledge management application for power professional technology, realize the functions of connecting with the unit, knowledge retrieval and query, and individualized knowledge management and display for engineering, etc. The application modules include intelligent dialogue management, engineering problem tracking and experience sharing, visual knowledge map, employee learning and training, data statistical analysis, and personal center.

[0009] In the field of accumulation and intelligent application of power professional technology, machine reading comprehension is one of the key technologies. Due to the rapid development of deep learning, machine translation, text generation, human-computer dialogue, sentiment analysis and other aspects have made major breakthroughs. As the most critical part of natural language processing, machine reading comprehension has also developed unprecedentedly. Machine reading comprehension is in the cognitive intelligent stage of machine. Cognitive means that the machine has the ability to understand and think.

[0010] Machine reading comprehension is essentially the ability of computers to understand natural language. It has very high requirements for artificial intelligence-related theories and computer hardware, so the rapid development of deep learning and hardware upgrades can better solve some problems in machine reading comprehension. Machine reading comprehension is the process of inputting text materials into a computer-related system, and then hoping that the computer system can respond accordingly and output the correct answer we expect. In simple terms, the goal of machine reading comprehension is to enable computers to read articles and understand their meaning, find clues to problems in the articles, and then answer the questions.

[0011] Machine reading comprehension has applications in many industries and has great practical significance. In our real life, there are many successful examples of machine reading comprehension. For example, the chat robot in Taobao can automatically answer many questions, saving manpower and reducing costs to some extent. Machine reading comprehension can also be applied to search engines. By understanding the problem and text through machine reading comprehension models, more accurate information can be presented to users, greatly improving efficiency. In the industrial industry, computers can read complex instructions and understand the work steps to provide more accurate guidance. Therefore, the research on machine reading comprehension is of great significance.

[0012] Early machine reading comprehension tasks were relatively simple and did not require complex semantic understanding and reasoning. Simple neural network models were used to predict answers. Early research on machine reading comprehension was still based on traditional features of machine reading comprehension. For example: based on the primary school reading comprehension material, a small-scale data set was sorted out, and the bag-of-words model was used to extract information and pattern match each sentence in the question and the article. The sentence in the article with the highest relevance to the question is the answer. Because the question is matched with each sentence in the article, the efficiency is not very good.

[0013] With the development of technology, various data are greatly increased, and many data sets are proposed, so that the above research cannot effectively solve the problem of expression diversity. The words and sentences in the article are not independent, and each word and each sentence has a mutual relationship and a dependent relationship, so it is also impossible to solve the problem of long-term dependence between words and sentences.

[0014] In order to solve the above problems, researchers begin to consider using neural networks to study machine reading comprehension. Using neural networks to study machine reading comprehension requires a large amount of training corpus, which also promotes the generation of data sets. In the early stage, such as the MCTest data set, the data content of which is mainly children's stories, the data set is suitable for machine reading comprehension of multiple choice tasks. Among them, its form is to give the question and the article, and the goal is to select the option containing the correct answer. The data set also has certain defects, that is, the data size is too small, so it can only be used to test the model and cannot be used to train the model.

[0015] In the prior art, the pre-training model can obtain the effect of reading comprehension, but the existing reading comprehension mode may have the following problems: if the answer to the question comes from multiple documents, or there are many questions, the corresponding parent is more, and because the problem and the answer are similar, if there are multiple similar answers in the text, it is difficult for the machine reading model to output accurate answers, and the situation of answering the wrong question is easy to occur, which affects the effect of machine reading comprehension; in addition, some questions do not have a standard answer in the text or the answer is subjective, which also makes it difficult for the machine reading model to output accurate results.

[0016] Based on this, the present application is proposed. SUMMARY

[0017] The present application provides a machine reading comprehension method for power professional technical knowledge to solve the problems in the prior art, and the specific technical solutions are as follows:

[0018] A machine reading comprehension method for power professional technical knowledge, comprising the following steps:

[0019] Step S1, input the original question to the answer retrieval module for preliminary retrieval to obtain a plurality of first answers and the prediction probability of each first answer;

[0020] Step S2, input the parent, the original question and the first answer to the evidence extraction module to extract the evidence label corresponding to the first answer from the text content of the parent; wherein the parent includes the original text, the text content in the drawing and the text content in the picture;

[0021] Step S3, input the original question, the first answer and the corresponding evidence label into the answer verification module to determine the most probable one from the several first answers as the formal answer.

[0022] Further improvement, the mother is machine reading comprehension and evidence extraction, extract the required answer content and evidence content from the mother, then input the evidence content, original question, answer content into the sequence generation model, after encoding by the encoder, input into the decoder; the decoder fuses the answer content and the evidence content, and finally decodes and gets the first sequence;

[0023] The first sequence is taken as the second answer of the reading comprehension mother and is evaluated, and the evaluation model is trained by deep learning with a multi-label classifier.

[0024] Further improvement, the mother and the original question are reading comprehension, and the answer segment of the original question is obtained, then the classification model and the pre-training encoder are used to pre-train and encode each sentence in the mother and extract the evidence, and the evidence segment corresponding to each sentence in the mother is obtained, and all the evidence segments are weighted in the decoder to obtain the evidence set information; the answer segment, the original question and the evidence set information are input into the decoder, and the decoding and output sequence prediction of the decoder are performed, and finally the first answer is obtained.

[0025] Further improvement, the answer retrieval module comprises a keyword retrieval unit, a word vector retrieval unit and an expert verification unit, and the original question is divided into a plurality of keywords and a plurality of word vectors;

[0026] The keyword extraction unit is used to extract keywords from the mother to obtain a plurality of keyword labels, the keywords and the keyword labels are compared in similarity, the mother with the largest similarity keyword label is selected as the preliminary prediction answer, and the answerability of the original question is judged;

[0027] The word vector extraction unit is used to extract word vectors from the mother to obtain a plurality of word vector labels and sentences in the text where the word vector labels are located, the sentences are label sentences, the process of reading the text by substituting the problem is simulated by interacting information between the original question and the label sentence, the mother with the highest matching degree label sentence is selected as the re-predicted answer based on the understanding of the text, and the prediction accuracy rate of the re-predicted answer is generated;

[0028] The different label sentences in the mother are respectively processed by covering, the re-predicted answer with the smallest change in the prediction accuracy rate of the re-predicted answer is selected as the closest answer after the label sentence is covered.

[0029] The preliminary prediction answer, the re-predicted answer and the closest answer are adjusted in answer span, and the final first answer is given.

[0030] In the answer span adjustment, the preliminary predicted answer, the re-predicted answer and the closest answer are weighted and optimized in the adversarial training,

[0031] Is=γ1I1+γ2I2+γ3I3, I1 is the loss function corresponding to the preliminary predicted answer, I2 is the loss function corresponding to the re-predicted answer, I3 is the loss function corresponding to the closest answer, γ1, γ2, γ3 are the weights, γ1+γ2+γ3=1, Is is the span loss corresponding to the first answer.

[0032] Further improvement, when judging that the original question is unanswerable, γ1=0, γ2 / γ3≥3;

[0033] When judging that the original question is answerable, 0.3≤γ1≤0.4, 1.5≤γ2 / γ3≤2.5, 1.1≤γ1 / γ3≤2.

[0034] Further improvement, the neural network module used in the adversarial training is a deep neural network model for constructing machine reading comprehension, and the deep neural network model comprises one or more of a self-attention model, a bidirectional attention flow model and a model layer based on a multi-head attention mechanism.

[0035] Further improvement, the female parent includes original text, text content in drawings and text content in pictures.

[0036] The beneficial effects of the present application are:

[0037] 1. The present application allows the user of the machine reading system to upload power industry documents and specifications, etc., to pre-train a deep network structure on a large amount of annotated text or annotated data in the early stage, and to provide intelligent answers and reading of specific document content.

[0038] 2. By providing a knowledge graph display and editing interaction in a specific knowledge field, the structured degree of the specific field document is improved.

[0039] 3. Through machine learning, the system's document library related to power and substation, equipment account, product data, maintenance technical documents, technical standards, operation and maintenance information, fault / defect archives, etc. are trained by cognitive construction technology.

[0040] 4. The present application converts the implicit knowledge in the field of power professional technical knowledge into explicit knowledge, perfects the existing knowledge base, establishes the knowledge point association between the professional knowledge base and the intelligent question answering through the machine reading comprehension technology, improves the quality of engineering knowledge retrieval service, realizes efficient use of existing knowledge resources and eliminates system barriers.

[0041] 5、The application is suitable for machine reading comprehension of multiple documents, extracts and integrates multiple documents, assigns different weight coefficients to loss functions in three preliminary predicted answers, re-predicted answers and the closest answers, performs joint optimization, improves prediction performance, further guarantees the reliability and rationality of answers, and improves the accuracy of machine reading comprehension. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical scheme and advantages of the application more clear, the application is further described in detail below in combination with embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application.

[0043] Embodiment 1

[0044] Arrangement of the mother document

[0045] I. Document library

[0046] 1.1. Document uploading

[0047] 1) Click the upload button to pop up the document uploading page, fill in the corresponding content, and click OK.

[0048] 2) Click the batch upload button to pop up the batch document uploading page, fill in the corresponding content, and click OK.

[0049] 1.2. Document search

[0050] The document supports document search according to file name, label, document permission, search content, upload date and other filtering conditions.

[0051] 1.3. Other operations

[0052] The documents in the document library support details and preview viewing and downloading.

[0053] II. Standard system

[0054] 2.1. Document uploading

[0055] 1) Click the upload button to pop up the document uploading page, fill in the corresponding content, and click OK.

[0056] 2.2. Document search

[0057] The standard system supports searching according to file name, label, search content and other filtering conditions.

[0058] 2.3. Other operations

[0059] The documents in the document library support details and preview viewing and downloading.

[0060] Three, drawing library

[0061] 3.1, add directory

[0062] 1) Click the plus button, pop up the new drawing library directory page, fill in the corresponding content and click confirm.

[0063] 2) Click the plus button on the right side of the specific directory, pop up the add sub drawing library directory page, fill in the corresponding content and click confirm.

[0064] 3.2, drawing upload

[0065] 1) Click the upload button, pop up the upload document page, fill in the corresponding content and click confirm.

[0066] 2) Click the batch upload button, pop up the batch upload document page, fill in the corresponding content and click confirm.

[0067] 3.3, drawing search

[0068] The drawing library supports keyword filtering of the directory and keyword search of the file name.

[0069] 3.4, other operations

[0070] The documents of the drawing library support detail and preview viewing, downloading, and like collection.

[0071] The application allows the user of the machine reading system to upload power industry documents and specifications, etc., to pre-train a deep network structure on a large amount of annotated text or annotated data in the early stage, and to provide intelligent answers and reading of specific document content questions for users.

[0072] By providing knowledge graph display and editing interaction in a specific knowledge field, the structured degree of the specific field document is improved.

[0073] Through machine learning, the document library, equipment account, product data, maintenance technical file, technical standard, operation information, fault / defect archive, etc. related to power and substation in the system are trained by cognitive construction technology.

[0074] The above are the mother's related content, acquisition and operation mode, the mother includes the original text, the text content in the drawing, and the text content in the picture.

[0075] By turning the invisible knowledge in the power professional technical knowledge field into explicit knowledge, perfecting the existing knowledge base, establishing the knowledge point association between the professional knowledge base and the intelligent question answering through machine reading understanding technology, improving the quality of engineering knowledge retrieval service, realizing efficient use of existing knowledge resources, and eliminating system barriers.

[0076] Embodiment 2

[0077] A machine reading comprehension method for power professional knowledge, comprising the following steps:

[0078] Step S1, input the original question to the answer retrieval module for preliminary retrieval to obtain a plurality of first answers and the prediction probability of each first answer;

[0079] Step S2, input the mother, the original question and the first answer to the evidence extraction module to extract the evidence label corresponding to the first answer from the text content of the mother;

[0080] Step S3, input the original question, the first answer and the corresponding evidence label to the answer verification module to determine the most probable one as the formal answer from the plurality of first answers.

[0081] Embodiment 3

[0082] In embodiment 2, the mother is subjected to machine reading comprehension and evidence extraction, and the required answer content and evidence content are extracted from the mother. Then, the evidence content, the original question and the answer content are input into a sequence generation model, and after being encoded by an encoder, they are input into a decoder. The decoder fuses the answer content and the evidence content, and finally decodes and obtains a first sequence.

[0083] The first sequence is taken as a second answer of the reading comprehension mother and is evaluated. The evaluation model performs adversarial training by means of a multi-label classifier through deep learning.

[0084] The mother and the original question are subjected to reading comprehension to obtain an answer segment of the original question. Then, a classification model and a pre-training encoder are used to pre-train and encode each sentence in the mother and extract evidence segments corresponding to each sentence in the mother in the decoder. The evidence segments are weighted in the decoder to obtain evidence set information. The answer segment, the original question and the evidence set information are input into the decoder, and after decoding and output sequence prediction by the decoder, a first answer is finally obtained.

[0085] The neural network module used in the adversarial training is a deep neural network model for constructing machine reading comprehension, which comprises one or more of a self-attention model, a bidirectional attention flow model and a model layer based on a multi-head attention mechanism.

[0086] The first answer is an answer sequence fused with multiple levels of evidence. Compared with the original answer segment, the fused first answer has more abundant and accurate information and is more consistent with the original question.

[0087] Embodiment 4

[0088] The answer retrieval module includes a keyword retrieval unit, a word vector retrieval unit, and an expert verification unit, which breaks down the original question into several keywords and several word vectors;

[0089] The parent text is used to extract keywords through a keyword extraction unit to obtain several keyword tags. The keywords and keyword tags are compared for similarity. The parent text containing the keyword tag with the highest similarity is selected as the preliminary predicted answer, and the answerability of the original question is judged.

[0090] The word vector extraction unit extracts word vectors from the original text to obtain several word vector labels and the sentences in the text containing the word vector labels. These sentences are called label sentences. By interacting with the original question and the label sentences, the process of reading the text with the question is simulated. Based on the understanding of the text, the original text containing the label sentence with the highest matching degree is selected as the re-predicted answer, and the prediction accuracy of the re-predicted answer is generated.

[0091] Different labeled sentences in the parent text are masked. After the labeled sentences are masked, the re-predicted answer with the smallest change in prediction accuracy is selected as the closest answer.

[0092] The answer range is adjusted for the initial predicted answer, the re-predicted answer, and the closest answer, and the final first answer is given.

[0093] In the answer range adjustment, the initial predicted answer, the re-predicted answer, and the closest answer are weighted and optimized in adversarial training.

[0094] Is = γ1I1 + γ2I2 + γ3I3, where I1 is the loss function corresponding to the initial predicted answer, I2 is the loss function corresponding to the re-predicted answer, I3 is the loss function corresponding to the closest answer, γ1, γ2, and γ3 are their respective weights, γ1 + γ2 + γ3 = 1, and Is is the span loss corresponding to the first answer.

[0095] When the original question is determined to be unanswerable, γ1 = 0, and γ2 / γ3 ≥ 3; this indicates that the answer is relatively subjective and should be adjusted as much as possible. Using this joint optimization method, the true accuracy rate of the first answer can reach 86%. However, if γ2 / γ3 = 1, the true accuracy rate of the first answer can only reach 67%; if γ2 / γ3 = 2, the true accuracy rate of the first answer can only reach 39%.

[0096] The true accuracy rate of the first answer refers to the accuracy rate of the first answer being the true answer after final human verification, such as in a test of 100 times, based on correct answers.

[0097] When it is judged that the original question is answerable, it indicates that the answer is relatively objective, 0.3<=g1<=0.4, 1.5<=g2 / g3<=2.5, 1.1<=g1 / g3<=2. For example, one of the application scenarios (three document cross extraction comparison): g1=0.32, g2=0.47, g3=0.21, the final true accuracy rate of the first answer can reach 77%. If g1=g2=g3, the final true accuracy rate of the first answer can reach 51%. If g1=0.1, g2=g3, the final true accuracy rate of the first answer can only reach 37%. If g1=0.2, g2=3g3, the final true accuracy rate of the first answer can only reach 25%. If g1=0.32, g2=3g3, the final true accuracy rate of the first answer can only reach 16%.

[0098] Firstly, for a single master of equipment account, product data, maintenance technical documents, technical standards, operation and maintenance information, fault / defect archives, etc., the similarity comparison can be carried out through keywords and keyword tags, and the accuracy is high.

[0099] In addition, it should be noted that the embodiment is suitable for the master of multiple documents, and is especially suitable for machine reading understanding of multiple documents. If it is still a single document, if it is judged that the original question is not answerable, even if g1=0, g2 / g3>=3, the final true accuracy rate of the first answer can only reach 57%.

[0100] In the embodiment, the cross-entropy loss function is used to measure the difference between the true probability distribution and the predicted probability distribution, and the smaller the value is, the better the prediction effect of the model is. The span loss should be as small as possible.

[0101] In the embodiment, for the master of multiple documents, the preliminary predicted answer often has low accuracy, and the re-predicted answer and the closest answer as a supplement can maximize the accuracy. Among them, the re-predicted answer and the closest answer have high accuracy, but if the weight is too high, the situation of "missing detection" is easy to occur. Different weight coefficients are given to the loss functions in the three preliminary predicted answers, the re-predicted answer and the closest answer, and joint optimization is carried out, and the experimental results show that not only the prediction performance is improved, but also the reliability and rationality of the answer are further ensured.

[0102] The above only describes the preferred embodiments of the present application and should not be used to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for machine reading comprehension of electrical power expertise, characterized in that Includes the following steps: Step S1: Input the original question into the answer retrieval module for preliminary retrieval, and obtain several first answers and the predicted probability of each first answer; Step S2: Input the master text, the original question, and the first answer into the evidence extraction module, and extract the evidence tags corresponding to the first answer from the text content of the master text; Step S3: Input the original question, the first answer, and the corresponding evidence label into the answer verification module, and determine the one with the highest probability from several first answers as the official answer; The answer retrieval module includes a keyword retrieval unit, a word vector retrieval unit, and an expert verification unit, which breaks down the original question into several keywords and several word vectors; The parent text is used to extract keywords through a keyword extraction unit to obtain several keyword tags. The keywords and keyword tags are compared for similarity. The parent text containing the keyword tag with the highest similarity is selected as the preliminary predicted answer, and the answerability of the original question is judged. The word vector extraction unit extracts word vectors from the original text to obtain several word vector labels and the sentences in the text containing the word vector labels. These sentences are called label sentences. By interacting with the original question and the label sentences, the process of reading the text with the question is simulated. Based on the understanding of the text, the original text containing the label sentence with the highest matching degree is selected as the re-predicted answer, and the prediction accuracy of the re-predicted answer is generated. Different labeled sentences in the parent text are masked. After the labeled sentences are masked, the re-predicted answer with the smallest change in prediction accuracy is selected as the closest answer. The answer range is adjusted for the initial predicted answer, the re-predicted answer, and the closest answer, and the final first answer is given; In the answer range adjustment, the initial predicted answer, the re-predicted answer, and the closest answer are weighted and optimized in adversarial training. Is = γ1I1 + γ2I2 + γ3I3, where I1 is the loss function corresponding to the initial predicted answer, I2 is the loss function corresponding to the re-predicted answer, I3 is the loss function corresponding to the closest answer, γ1, γ2, and γ3 are their respective weights, γ1 + γ2 + γ3 = 1, and Is is the span loss corresponding to the first answer.

2. The method of claim 1, wherein the method comprises: Machine reading comprehension and evidence extraction are performed on the source text to extract the required answer and evidence content. Then, the evidence content, original question, and answer content are all input into the sequence generation model. After being encoded by the encoder, they are input into the decoder. The decoder merges the answer content with the evidence content, and finally decodes and obtains the first sequence; The first sequence is used as the second answer to the reading comprehension text and evaluated. The evaluation model is trained adversarially using a multi-label classifier through deep learning.

3. The method of claim 1, wherein: The mother and the original question are read and understood to obtain an answer fragment of the original question, then each sentence in the mother is pre-trained and encoded and evidence extraction is performed using a classification model and a pre-trained encoder to obtain an evidence fragment corresponding to each sentence in the mother, and all evidence fragments are weighted in the decoder to obtain evidence set information; the answer fragment, the original question and the evidence set information are jointly input into the decoder, and decoding and output sequence prediction are performed in the decoder to finally obtain a first answer.

4. The machine reading comprehension method for power professional technical knowledge according to claim 1, characterized in that: When it is judged that the original question has no answerability, γ1=0, γ2 / γ3≥3; When it is judged that the original question has answerability, 0.3≤γ1≤0.4, 1.5≤γ2 / γ3≤2.5, 1.1≤γ1 / γ3≤2.

5. The method of claim 2, wherein the method further comprises: The neural network module used in the adversarial training is a deep neural network model for constructing machine reading comprehension, and the deep neural network model comprises one or more of a self-attention model, a bidirectional attention flow model and a model layer based on a multi-head attention mechanism.

6. The method of claim 1, wherein: The mother includes original text, text content in drawings and text content in pictures.