Machine reading comprehension method, device, electronic device and computer storage medium

Abstract

The present invention relates to a machine reading comprehension method, device, equipment and medium, wherein the machine reading comprehension method comprises: receiving a question and an article, converting the question into a question vector through a coding layer calculation, converting the article into an article vector through the coding layer calculation, calculating the question vector and the article vector through a multiple attention layer to obtain an interactive information vector, wherein the multiple attention layer comprises a trained self-attention model and multiple attention matching models, calculating the article through a trained BTM topic model to obtain a topic word, encoding the topic word to obtain a topic feature vector, calculating the interactive information vector and the topic feature vector through a nonlinear output layer, and obtaining an answer related to the question. Through the present application, the problem of low accuracy of answers when using a single word matching method for machine reading comprehension is solved, and the accuracy of machine reading comprehension answers is improved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a machine reading comprehension method, device, electronic device and computer storage medium. Background Art

[0002] Machine reading comprehension is a technology that uses algorithms to enable computers to understand the semantics of articles and answer related questions. In related technologies, a word matching method is used to match questions with words in articles, and then obtain answers related to the questions from the articles. However, this single word matching method makes the interactive information between questions and words in articles too simple, resulting in the obtained answers being irrelevant to the questions, resulting in low accuracy of the answers.

[0003] Regarding the related art, using a single word matching method for machine reading comprehension has the problem of low accuracy of answers, and no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a machine reading comprehension method, apparatus, electronic device, and computer storage medium to at least solve the problem of low accuracy of answers in the related art when using a single word matching method for machine reading comprehension.

[0005] In a first aspect, an embodiment of the present application provides a machine reading comprehension method, the method comprising:

[0006] Receiving a question and an article, converting the question into a question vector through coding layer calculation, and converting the article into an article vector through coding layer calculation;

[0007] The question vector and the article vector are calculated through a multiple attention layer to obtain an interactive information vector, wherein the multiple attention layer includes a trained self-attention model and multiple attention matching models;

[0008] The article is calculated through the trained BTM topic model to obtain topic words, and the topic words are encoded to obtain a topic feature vector;

[0009] The interactive information vector and the topic feature vector are calculated through a nonlinear output layer to obtain an answer related to the question.

[0010] In some embodiments, the encoding layer includes a BERT model and a preset gated hole convolution layer, and converting the question into a question vector through the encoding layer calculation includes:

[0011] The question is learned based on the BERT model to obtain a first word vector of each word in the question, and a first intermediate vector is formed by the first word vectors of each word;

[0012] Calculating the first intermediate vector through the gated dilated convolutional layer to obtain the problem vector;

[0013] The converting the article into an article vector by the encoding layer calculation comprises:

[0014] The article is learned based on the BERT model to obtain a second word vector of each word in the article, and the second word vectors of each word are used to form a second intermediate vector;

[0015] The second intermediate vector is calculated through the gated hole convolution layer to obtain the article vector.

[0016] In some of the embodiments, the gated atrous convolution layer includes multiple layers of atrous convolution gating units connected in sequence, each of the atrous convolution gating units is used to perform interval sampling on the input vector to obtain an output vector, and use the output vector as the input vector of the next layer of atrous convolution gating units.

[0017] In some embodiments, when the multiple attention matching models include a dot product attention model and a Concat-based attention model, calculating the question vector and the article vector through multiple attention layers to obtain an interactive information vector includes:

[0018] Calculate the question vector and the article vector through a trained dot product attention model to obtain a first vector and a second vector, wherein the first vector is a dot product attention vector of the question with respect to the article, and the second vector is a dot product attention vector of the article with respect to the question;

[0019] The question vector and the article vector are calculated by a trained Concat-based attention model to obtain a third vector and a fourth vector, wherein the third vector is a Concat attention vector of the question with respect to the article, and the fourth vector is a Concat attention vector of the article with respect to the question;

[0020] Merge the first vector, the second vector, the third vector and the fourth vector to obtain a merged vector;

[0021] The merged vector is calculated through the trained self-attention model to obtain the mutual information vector.

[0022] In some embodiments, the trained BTM topic model is obtained by:

[0023] Get the article corpus training set;

[0024] The topic distribution on the article corpus training set is sampled by using the Dirichlet distribution function;

[0025] Sampling the distribution of terms under multiple topics through the Dirichlet distribution function;

[0026] A target topic is extracted from the topic distribution, and word pairs are extracted from the target topic so that the word pairs obey the term distribution under the target topic.

[0027] In some embodiments, encoding the topic words to obtain a topic feature vector includes:

[0028] Acquire a training corpus, learn the training corpus based on the BERT model to obtain a third word vector of each word in the training corpus, and form a word vector library by the third word vectors of each word;

[0029] The third word vector of each word in the subject word is obtained from the word vector library, and the subject feature vector is formed by the third word vector of each word in the subject word.

[0030] In some embodiments, the nonlinear output layer includes a preset hyperbolic tangent function, and the calculation of the interactive information vector and the topic feature vector through the nonlinear output layer to obtain an answer related to the question includes:

[0031] According to the interactive information vector and the topic feature vector, nonlinear mapping is performed through the hyperbolic tangent function to obtain the answer extracted from the article.

[0032] In a second aspect, an embodiment of the present application provides a machine reading comprehension device, the device comprising: an encoding module, a multiple attention module, a topic acquisition module and an answer calculation module;

[0033] The encoding module is used to receive a question and an article, convert the question into a question vector through encoding layer calculation, and convert the article into an article vector through encoding layer calculation;

[0034] The multiple attention module is used to calculate the question vector and the article vector through a multiple attention layer to obtain an interactive information vector, wherein the multiple attention layer includes a trained self-attention model and multiple attention matching models;

[0035] The topic acquisition module is used to calculate the topic words of the article through the trained BTM topic model, and encode the topic words to obtain a topic feature vector;

[0036] The answer calculation module is used to calculate the interactive information vector and the topic feature vector through a nonlinear output layer to obtain an answer related to the question.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the machine reading comprehension method as described in the first aspect above is implemented.

[0038] In a fourth aspect, an embodiment of the present application provides a computer storage medium on which a computer program is stored, and when the program is executed by a processor, the machine reading comprehension method as described in the first aspect above is implemented.

[0039] Compared with the related art, the machine reading comprehension method provided in the embodiment of the present application receives a question and an article, converts the question into a question vector through a coding layer calculation, converts the article into an article vector through the coding layer calculation, calculates the question vector and the article vector through a multiple attention layer to obtain an interactive information vector, wherein the multiple attention layer includes a trained self-attention model and multiple attention matching models, calculates the article through a trained BTM topic model to obtain topic words, encodes the topic words to obtain a topic feature vector, calculates the interactive information vector and the topic feature vector through a nonlinear output layer to obtain an answer related to the question, thereby solving the problem of low answer accuracy when using a single word matching method for machine reading comprehension, and improving the accuracy of machine reading comprehension answers. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 is a flow chart of a machine reading comprehension method according to an embodiment of the present application;

[0042] Figure 2 is a schematic diagram of the structure of a dilated convolution gating unit according to an embodiment of the present application;

[0043] Figure 3 is a flowchart of obtaining an interactive information vector according to an embodiment of the present application;

[0044] Figure 4is a flow chart of a machine reading comprehension method according to a preferred embodiment of the present application;

[0045] Figure 5 is a schematic diagram of the structure of a machine reading comprehension device according to an embodiment of the present application;

[0046] Figure 6 It is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0048] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0049] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0050] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application are not limited in quantity and can be singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, and there can be three relationships. For example, "A and / or B" can be: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally associates objects before and after in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0051] This embodiment provides a machine reading comprehension method. Figure 1 is a flow chart of a machine reading comprehension method according to an embodiment of the present application. Figure 1 As shown, the process includes the following steps:

[0052] S110, receiving questions and articles, converting questions into question vectors through coding layer calculation, and converting articles into article vectors through coding layer calculation. The coding layer is used to perform bottom-level processing on articles and questions respectively, digitally encode the texts of articles and questions, and convert articles and questions into information units that can be processed by computers. The coding layer is based on understanding the context to encode each word, phrase and sentence in articles and questions, so as to retain the semantics of the original sentences in the articles. In the coding layer, first, the questions and articles are segmented to obtain multiple words in their respective texts, and then each word is converted into a corresponding word vector, and then the word vector of each word is further encoded to obtain the word-level vector representation of the questions and articles. The conversion of each word to word vector can use any one of the models in word2vec model, GloVe (Global Vectors for Word Representation) model, and ELMo (Embedding from Language Models) model. The word-level vector representation of questions and articles can be bidirectionally encoded using BiGRU (Bi-directional GatedRNN, also known as bidirectional gated recurrent unit network).

[0053] S120, the question vector and the article vector are calculated through multiple attention layers to obtain an interactive information vector. The multiple attention layer includes a trained self-attention model and multiple attention matching models. Since there is a correlation between the article and the question, it is necessary to establish a connection between the article and the question so as to obtain answers related to the question from the article based on the interactive information between the article and the question. By considering the semantics of the article and the question together, the understanding of the question can be deepened by the semantic analysis of the article, and the understanding of the article can be deepened by the semantic analysis of the question, thereby focusing on the semantic connection between the article and the question. The attention mechanism in deep learning is essentially similar to the selective visual attention mechanism of humans. The core goal is to select more critical information for the current task goal from a large amount of information. The multiple attention layer integrates multiple attention mechanisms. By using multiple attention matching models, the interaction at the word level in the sentence can be enhanced, and the interactive information between words can be enriched. In addition, the self-attention mechanism can establish long-distance dependencies and discover important word features within the sentence, which is suitable for establishing the relationship between long articles and questions. The interactive information vector obtained by the multiple attention layers can contain richer and more effective interactive information between questions and articles.

[0054] S130, calculate the article through the trained BTM topic model to obtain the topic words, and encode the topic words to obtain the topic feature vector. The BTM (Biterm Topic Model) topic model takes two words accordingly after determining the topic distribution and word distribution, that is, after a text is segmented, any two words within the window length are paired, and these two words are called a pair of biterms, and then the topic words in the document are obtained according to the probability that each pair of biterms belongs to each topic. The BTM topic model can effectively solve the sparsity problem by relaxing the constraint that the entire document must belong to the same topic to two words within the window length belonging to the same topic. The topic words reflect the topic content of the article, so that the topic feature vector contains the topic information of the article.

[0055] S140, the interactive information vector and the topic feature vector are calculated through a nonlinear output layer to obtain an answer related to the question. By integrating the topic information of the article into the interactive information between the article and the question, it is possible to better guide the extraction of answers related to the question from the article.

[0056] Through the above steps, by integrating multiple attention mechanisms and using multiple attention matching models to enhance the interactive information at the word level between the question and the article, the relevance between the question and the article is enhanced. At the same time, the subject information of the article is incorporated into the interactive information to guide the extraction of the answer from the article, thereby improving the relevance of the answer to the question and solving the problem of low answer accuracy when using a single word matching method for machine reading comprehension.

[0057] In some embodiments, the encoding layer includes a BERT model and a preset gated hole convolution layer. The question is converted into a question vector through the encoding layer calculation, specifically including: obtaining the first word vector of each word in the question by learning the question based on the BERT model, and forming a first intermediate vector from the first word vector of each word and recording it as query: Among them, m represents the number of words in question q, Represents the first word vector of the i-th word in question q, i = 1, ..., m, and then the first intermediate vector is calculated through the gated hole convolution layer to obtain the question vector V q : Among them, Conv res Represents a gated atrous convolutional layer. Similarly, the article is converted into an article vector through the encoding layer calculation, specifically including: obtaining the second word vector of each word in the article by learning the article based on the BERT model, and forming a second intermediate vector composed of the second word vectors of each word, recorded as Document: Where n represents the number of words in article d. Represents the second word vector of the jth word in article d, j = 1, ..., n, and then the second intermediate vector is calculated through the gated hole convolution layer to obtain the article vector V d : Conv res Represents a gated atrous convolutional layer operation.

[0058] The BERT (Bidirectional Encoder Representations from Transformers) model is a deep bidirectional pre-trained language understanding model that uses the Transformer model as a feature extractor. In essence, it learns a good feature representation for a word by running a self-supervised learning method based on a massive corpus. The Transformer model is a classic NLP model proposed by the Google team. Because the Transformer model uses a self-attention mechanism and does not adopt the sequential structure of a recurrent neural network, the Transformer model can be trained in parallel and has global information. Therefore, the first intermediate vector learned based on the BERT model can effectively express the semantics between each word in the question. Similarly, the second intermediate vector can effectively express the semantics between each word in the article. The dilated convolution is a variant of the convolution. Unlike ordinary convolution, the dilated convolution can sample the input text interval across the text segment with the same dilated rate. By stacking dilated convolutions with exponentially increasing dilated rates, it is possible to cover most sentence lengths with fewer layers. The BERT model and gated atrous convolutional layer improve the representation capabilities of question vectors and article vectors, which helps obtain more accurate interactive information vectors through question vectors and article vectors, further improving the accuracy of the answers.

[0059] In some embodiments, the gated atrous convolution layer includes multiple layers of sequentially connected atrous convolution gating units. Figure 2 is a schematic diagram of the structure of a dilated convolution gating unit according to an embodiment of the present application, such as Figure 2 As shown in the figure, each dilated convolution gating unit includes a dilated convolution network and a residual network. Each dilated convolution gating unit is used to perform interval sampling on the input vector to obtain an output vector, and use the output vector as the input vector of the next layer of dilated convolution gating units. The operation of each dilated convolution gating unit can be expressed by the following formula: Among them, X represents the input vector of the dilated convolution gating unit, Y represents the output vector of the dilated convolution gating unit, Conv1 represents convolution operation 1, Conv2 represents convolution operation 2, Conv1 and Conv2 are both dilated convolutions, the number of filters and window sizes set by the two are the same, but the weights are not shared, and Sig represents the sigmoid activation function.

[0060] Ordinary convolution operations can process text in parallel and significantly shorten the model training time, but in order to achieve good results for long-distance dependencies, multiple convolution layers are often required, which increases the risk of gradient vanishing. The gated atrous convolution layer introduces a residual mechanism based on the atrous convolution network, allowing information to be transmitted in multiple channels, which can strengthen effective information, eliminate useless information, and improve the gradient vanishing problem caused by deep networks.

[0061] In some embodiments, Figure 3 is a flowchart of obtaining an interactive information vector according to an embodiment of the present application, such as Figure 3 As shown, in the case where multiple attention matching models include a dot product attention model and a Concat-based attention model, calculating the question vector and the article vector through multiple attention layers to obtain the interactive information vector includes the following steps:

[0062] S310: Calculate the question vector and the article vector through the trained dot product attention model to obtain a first vector and a second vector. The first vector is the dot product attention vector of question q with respect to article d, denoted by V m1 , the second vector is the dot product attention vector of article d on question q, denoted by V m2 Specifically, the first vector V m1 It is obtained by: taking two vectors and After the dot product, the dimension remains the same as the original dimension, and then the hyperbolic tangent function is used for nonlinear transformation, and then v d The original weight is obtained after the linear change of Finally, we get the normalized weighted vector expression V of question q about article d m1 , that is, the first vector V m1 Obtained by the following formula:

[0063]

[0064]

[0065]

[0066] in, Represents the vector of the jth word in the question vector, Represents the vector of the tth word in the article vector, v d is the trained parameter, tanh(·) represents the hyperbolic tangent function. Similarly, the second vector V can be obtained m2 .

[0067] S320: Calculate the question vector and the article vector using the trained Concat-based attention model to obtain a third vector and a fourth vector. The third vector is the Concat attention vector of question q with respect to article d, denoted by V m3 , the fourth vector is the Concat attention vector of article d about question q, denoted by V m4 Specifically, the third vector V m3 Obtained by: go through The vector after matrix mapping and go through The vectors after matrix mapping are merged. In order to ensure nonlinearity, the hyperbolic tangent function is used for nonlinear changes, and then v is used to c The vector is mapped to get the original weight Finally, we get the normalized weighted vector expression V of question q about article d m3 , that is, the third vector V m3 It can be obtained by the following formula:

[0068]

[0069]

[0070]

[0071] in, Represents the vector of the jth word in the question vector, Represents the vector of the tth word in the article vector, and v c is the trained parameter, tanh(·) represents the hyperbolic tangent function. Similarly, we can get the fourth vector V m4 .

[0072] S330, merge the first vector, the second vector, the third vector and the fourth vector to obtain a merged vector: V c =Concat(V m1 ,V m2 ,V m3 ,V m4 ), where V c represents the merge vector, V m1 、V m2 、V m3 and V m4 denote the first vector, the second vector, the third vector and the fourth vector respectively, and Concat(·) denotes a merge operation.

[0073] S340, the merged vector is calculated by the trained self-attention model to obtain the mutual information vector. The mutual information vector is obtained by the following formula:

[0074] V s =Attention(V c W c ,V s W s ), where V c represents the merge vector, V s represents the mutual information vector, W c is the weight of the merge vector, W s is the weight of the interactive information vector, and Attention(·) represents the self-attention operation. The self-attention mechanism can discover important word features within a sentence and continuously adjust and change the weights during the back propagation process. At the same time, parallel computing can be achieved through matrix multiplication during the self-attention calculation process, which speeds up the training of the model.

[0075] In some embodiments, the trained BTM topic model is obtained by the following steps: obtaining an article corpus training set; sampling the topic distribution on the article corpus training set through the Dirichlet distribution function; sampling the term distribution under multiple topics through the Dirichlet distribution function; extracting a target topic from the topic distribution, and extracting word pairs from the target topic so that the word pairs obey the term distribution under the target topic.

[0076] Specifically, we use a large-scale article corpus to build an article corpus training set, and use Dirichlet distribution to sample the word distribution under a topic from parameter β. k=1,...,K,K is the number of topics,Dir(·) represents the Dirichlet distribution function,in the Dirichlet distribution with parameter α,sample the topic distribution θ~Dir(α) of the article corpus training set,extract the target topic Z from the common parameter θ of the article corpus training set,Z is the Dirichlet distribution of topic θ,obeying Z~Mult(θ),Mult(·) represents the multinomial distribution function,let a word pair in the corpus be denoted as b, Extract from the above extracted topic Z and Two words, and make them obey

[0077] In some embodiments, the subject words are encoded through the following steps to obtain a subject feature vector: obtain training corpus, learn the training corpus based on the BERT model to obtain the third word vector of each word in the training corpus, and form a word vector library by the third word vector of each word. The third word vector of each word in the subject words is obtained from the word vector library, and the third word vector of each word in the subject words constitutes a subject feature vector. The subject feature vector obtained based on the BERT model can effectively express the semantics between each word in the subject words, improve the representation ability of the subject feature vector, and integrate the subject feature vector into the interactive information vector, which can further improve the accuracy of the answer.

[0078] In some embodiments, the nonlinear output layer includes a preset hyperbolic tangent function. The interactive information vector and the topic feature vector are calculated through the nonlinear output layer to obtain an answer related to the question, including: according to the interactive information vector and the topic feature vector, a nonlinear mapping is performed through the hyperbolic tangent function to obtain an answer extracted from the article. The answer is predicted by the following formula:

[0079] g IR (d,q)=tanh(W s V s +W t V t ), where V s represents the mutual information vector, V t represents the topic feature vector, W s and W t are the trained parameters, tanh(·) represents the hyperbolic tangent function, g IR (d,q) represents the answer related to question q extracted from article d.

[0080] The embodiments of the present application are described and illustrated below through preferred embodiments.

[0081] Figure 4 is a flow chart of a machine reading comprehension method according to a preferred embodiment of the present application. Figure 4First, it is necessary to train the parameters of the models and functions in the encoding layer, the multiple attention layer, and the nonlinear output layer. The training process includes the following steps: based on a large amount of customer service question and answer corpus, obtain product documents in a specific field, clean and segment these corpus, and then use the BERT model to pre-train word vectors to generate word vectors based on specific vertical fields. Set the parameters for word vector learning model training, which include the dimension of word vectors, batch processing parameters, window size, initial learning rate, word vector matrix, and auxiliary vector matrix. For large-scale training data including training sets and development sets, the training problems and training articles in the training set are respectively segmented and encoded, and the word-level vector representation of the training problem and the word-level vector representation of the training article are respectively obtained. The word-level vector representation of each single word in the training problem and the training article is respectively calculated by a dilated convolution, and the word-level vector representation of the training problem and the word-level vector representation of the training article are respectively calculated by a gated dilated convolution layer, so that the training problem vector and the training article vector can be obtained. The gated dilated convolution layer includes multiple layers of dilated convolution gating units connected in sequence. Then the training question vector and the training article vector are input into the dot product attention model and the Concat-based attention model to obtain the training dot product attention vector V1 and the training Concat attention vector V3 of the training question about the training article, and the training dot product attention vector V2 and the training Concat attention vector V4 of the training article about the training question. V1, V2, V3 and V4 are merged to obtain the training merged vector V c-train , V c-train The training interaction information vector is calculated by the self-attention model. The training article is calculated by the trained BTM topic model to obtain the training topic words, and the training topic words are encoded to obtain the training topic feature vector. The training interaction information vector and the training topic feature vector are nonlinearly combined for answer prediction. Finally, the training set is used to train the parameters of the models and functions in the encoding layer, multiple attention layer, and nonlinear output layer, and the parameters with the best F1 index on the development set are selected and saved, and the training is completed.

[0082] Then, the received questions and articles are input into the model and function with trained parameters, and the correct answer to the question can be extracted from the article. The prediction process includes the following steps: The received questions and articles are converted into question vectors V through the BERT model in the encoding layer and the gated void convolution layer with trained parameters. q and article vector V d . The problem vector V q and the article vector V d Input the trained dot product attention model and Concat-based attention model in the multiple attention layers to get the first vector V m1, the second vector V m2 , the third vector V m3 and the fourth vector V m4 , and V m1 、V m2 、V m3 and V m4 After merging, input the trained self-attention model for calculation to obtain the interactive information vector V s The article is then encoded after being calculated using the trained BTM topic model to obtain the topic feature vector V t . The topic feature vector V t and the mutual information vector V s By using the hyperbolic tangent function with trained parameters in the nonlinear output layer, we can get the answer to this question from the article.

[0083] An embodiment of the present application provides a machine reading comprehension device. Figure 5 is a schematic diagram of the structure of a machine reading comprehension device according to an embodiment of the present application. Figure 5 As shown, the device includes an encoding module 510, a multiple attention module 520, a topic acquisition module 530 and an answer calculation module 540: the encoding module 510 is used to receive a question and an article, and convert the question into a question vector through an encoding layer calculation, and convert the article into an article vector through the encoding layer calculation; the multiple attention module 520 is used to calculate the question vector and the article vector through a multiple attention layer to obtain an interactive information vector, wherein the multiple attention layer includes a trained self-attention model and multiple attention matching models; the topic acquisition module 530 is used to calculate the article through a trained BTM topic model to obtain topic words, and encode the topic words to obtain a topic feature vector; the answer calculation module 540 is used to calculate the interactive information vector and the topic feature vector through a nonlinear output layer to obtain an answer related to the question.

[0084] For the specific definition of the machine reading comprehension device, please refer to the definition of the machine reading comprehension method above, which will not be repeated here. Each module in the above-mentioned machine reading comprehension device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0085] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0086] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0087] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0088] In addition, in combination with the machine reading comprehension method in the above embodiment, the embodiment of the present application can provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any one of the machine reading comprehension methods in the above embodiment is implemented.

[0089] In one embodiment, Figure 6 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application, such as Figure 6 As shown, an electronic device is provided, which may be a server, and its internal structure diagram may be as shown in Figure 6 As shown. The electronic device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the electronic device is used to store data. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a machine reading comprehension method is implemented.

[0090] Those skilled in the art will understand that Figure 6 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0091] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0092] Those skilled in the art should understand that the technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0093] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A machine reading comprehension method, characterized in that: The method comprises: Receiving a question and an article, converting the question into a question vector through coding layer calculation, and converting the article into an article vector through coding layer calculation; The question vector and the article vector are calculated through multiple attention layers to obtain an interactive information vector, wherein the multiple attention layers include a trained self-attention model and multiple attention matching models, and when the multiple attention matching models include a dot product attention model and a Concat-based attention model, the interactive information vector is calculated through multiple attention layers to obtain the question vector and the article vector, including: The question vector and the article vector are calculated using a trained dot product attention model to obtain a first vector and a second vector, wherein the first vector is a dot product attention vector of the question with respect to the article, and the second vector is a dot product attention vector of the article with respect to the question. The question vector and the article vector are calculated by a trained Concat-based attention model to obtain a third vector and a fourth vector, wherein the third vector is a Concat attention vector of the question with respect to the article, and the fourth vector is a Concat attention vector of the article with respect to the question. Merge the first vector, the second vector, the third vector and the fourth vector to obtain a merged vector, Calculating the merged vector through the trained self-attention model to obtain the interactive information vector; The article is calculated through the trained BTM topic model to obtain topic words, and the topic words are encoded to obtain a topic feature vector; The interactive information vector and the topic feature vector are calculated through a nonlinear output layer to obtain an answer related to the question.

2. The method according to claim 1, characterized in that The encoding layer includes a BERT model and a preset gated hole convolution layer, and converting the problem into a problem vector through the encoding layer calculation includes: The question is learned based on the BERT model to obtain a first word vector of each word in the question, and a first intermediate vector is formed by the first word vectors of each word; Calculating the first intermediate vector through the gated atrous convolutional layer to obtain the problem vector; The converting the article into an article vector by the encoding layer calculation comprises: The article is learned based on the BERT model to obtain a second word vector of each word in the article, and the second word vectors of each word are used to form a second intermediate vector; The second intermediate vector is calculated through the gated hole convolution layer to obtain the article vector.

3. The method according to claim 2, characterized in that The gated atrous convolution layer includes multiple layers of atrous convolution gating units connected in sequence, each of which is used to perform interval sampling on an input vector to obtain an output vector, and use the output vector as an input vector of a next layer of atrous convolution gating units.

4. The method according to claim 1, characterized in that: The trained BTM topic model is obtained in the following way: Get the article corpus training set; The topic distribution on the article corpus training set is sampled by using the Dirichlet distribution function; Sampling the distribution of terms under multiple topics through the Dirichlet distribution function; A target topic is extracted from the topic distribution, and word pairs are extracted from the target topic so that the word pairs obey the term distribution under the target topic.

5. The method according to claim 1, characterized in that The encoding of the subject words to obtain a subject feature vector comprises: Acquire a training corpus, learn the training corpus based on the BERT model to obtain a third word vector of each word in the training corpus, and form a word vector library by the third word vectors of each word; The third word vector of each word in the subject word is obtained from the word vector library, and the subject feature vector is formed by the third word vector of each word in the subject word.

6. The method according to claim 1, characterized in that The nonlinear output layer includes a preset hyperbolic tangent function, and the calculation of the interactive information vector and the topic feature vector through the nonlinear output layer to obtain an answer related to the question includes: According to the interactive information vector and the topic feature vector, nonlinear mapping is performed through the hyperbolic tangent function to obtain the answer extracted from the article.

7. A machine reading comprehension device, characterized in that: The device comprises: an encoding module, a multiple attention module, a topic acquisition module and an answer calculation module; The encoding module is used to receive a question and an article, convert the question into a question vector through encoding layer calculation, and convert the article into an article vector through encoding layer calculation; The multiple attention module is used to calculate the question vector and the article vector through a multiple attention layer to obtain an interactive information vector, wherein the multiple attention layer includes a trained self-attention model and multiple attention matching models. When the multiple attention matching models include a dot product attention model and a Concat-based attention model, the interactive information vector is calculated by the multiple attention layers. The interactive information vector includes: The question vector and the article vector are calculated using a trained dot product attention model to obtain a first vector and a second vector, wherein the first vector is a dot product attention vector of the question with respect to the article, and the second vector is a dot product attention vector of the article with respect to the question. The question vector and the article vector are calculated by a trained Concat-based attention model to obtain a third vector and a fourth vector, wherein the third vector is a Concat attention vector of the question with respect to the article, and the fourth vector is a Concat attention vector of the article with respect to the question. Merge the first vector, the second vector, the third vector and the fourth vector to obtain a merged vector, Calculating the merged vector through the trained self-attention model to obtain the interactive information vector; The topic acquisition module is used to calculate the topic words of the article through the trained BTM topic model, and encode the topic words to obtain a topic feature vector; The answer calculation module is used to calculate the interactive information vector and the topic feature vector through a nonlinear output layer to obtain an answer related to the question.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the machine reading comprehension method according to any one of claims 1 to 6 is implemented.

9. A computer storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the machine reading comprehension method according to any one of claims 1 to 6 is implemented.

Images