Question and answer processing method, device, computer equipment and readable storage medium

Abstract

The present invention provides a question-answering processing method, apparatus, computer equipment and readable storage medium. The question-answering processing method comprises: obtaining a question part and a content part of a target document, wherein the content part comprises at least two text blocks; constructing an initial input vector according to the question part and the content part; inputting the initial input vector into a preset question-answering machine model, wherein the question-answering machine model comprises a pre-training model layer, an attention model layer and a final output layer; and determining whether the content part includes an answer corresponding to the question part and the position of the answer in the content part according to question-answering result information. Through the present invention, the accuracy of question-answering processing can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a question-answering processing method, apparatus, computer equipment and readable storage medium. Background Art

[0002] Machine Reading Comprehension Question Answering (MRCQA) refers to the process of finding answers to user questions from documents by understanding the user's questions given a document library. The premise of MRCQA processing here is that the answer can be clearly found in the document, and it is best to be a continuous segment. Document-based question answering processing methods are usually used to solve the problem of finding answers to questions from a large amount of unstructured text.

[0003] At present, for question-answering processing methods, machine learning models are usually used for processing, wherein the machine learning model is a complex function composed of simple functions. When establishing a machine learning model used in the question-answering processing method, a machine learning model architecture is usually designed first, and then the parameters of the architecture are obtained by training the given question-answering training data using a computer. These parameters ensure that the trained machine learning model can achieve the expected design effect in the actual question-answering processing process and has generalization capabilities. The inventor has found that due to the large number of parameters of the machine learning model used in the question-answering processing method, overfitting is prone to occur during the training process, which makes the obtained machine learning model unable to accurately perform question-answering processing, thereby causing the technical problem of low accuracy of the question-answering processing method.

[0004] Therefore, how to improve the accuracy of question-answering processing methods has become a technical problem that needs to be urgently solved in this field. Summary of the invention

[0005] The purpose of the present invention is to provide a question-answering processing method, apparatus, computer device and readable storage medium to solve the above-mentioned technical problems in the prior art.

[0006] On the one hand, to achieve the above objectives, the present invention provides a question-answering processing method.

[0007] The question-answering processing method includes: obtaining a question part and a content part of a target document, wherein the content part includes at least two text blocks; constructing an initial input vector according to the question part and the content part; inputting the initial input vector into a preset question-answering machine model, wherein the question-answering machine model includes a pre-trained model layer, an attention model layer and a final output layer, wherein the pre-trained model layer is used to obtain a first word vector after processing the input initial input vector, and output it to the attention model layer and the final output layer, wherein the attention model layer is used to obtain a second word vector after processing the input first word vector, and output it to the final output layer, wherein the second word vector is used to represent the attention size of a first word to other words in the text block where the first word is located, wherein the first word is a word in the content part, and wherein the final output layer is used to output question-answering result information according to the first word vector and the second word vector; and determining whether the content part includes an answer corresponding to the question part and the position of the answer in the content part according to the question-answering result information.

[0008] Furthermore, the text block is a paragraph or a sentence.

[0009] Furthermore, the second word vector is also used to represent the attention of the first word to the words in the question part.

[0010] Furthermore, the attention model layer calculates the second word vector based on the first word vector by the following steps: inputting the first word vector into a first fully connected function to obtain a first vector; inputting the first word vector into a second fully connected function to obtain a second vector; inputting the first word vector into a third fully connected function to obtain a third vector, wherein the first fully connected function, the second fully connected function and the third fully connected function are respectively different fully connected functions in the attention model layer; constructing an attention matrix based on the question part and the text block; calculating the second word vector based on the first vector, the second vector, the third vector and the attention matrix.

[0011] Furthermore, the second word vector is calculated using the following formula:

[0012] H′=[h′1,h′2,...,h′ A ]=L(C(W1,W2,...,W n ))

[0013] W i =A i V i

[0014]

[0015] V=[V1,V2,...,V n ]=L1(H)

[0016] Q=[Q1,Q2,...,Q n ]=L2(H)

[0017] K=[K1,K2,…,K n ]=L3(H)

[0018] H=[h l , h2,…,h n ],

[0019] The input of the final output layer is calculated using the following formula:

[0020]

[0021]

[0022] Wherein, H′ is the second word vector, n is the processing threshold of the initial input vector, C is the concatenation function, L is the full connection function, V is the first vector, L1 is the first full connection function, H is the first word vector, S is the normalization function, M is the attention matrix, Q is the second vector, L2 is the second full connection function, K is the third vector, L3 is the third full connection function, d k is a normalization constant, i=1,2,…n, is the input of the final output layer, and α is the preset weight.

[0023] Furthermore, the step of constructing an attention matrix according to the question part and the text block includes: constructing an n*n matrix M0; constructing a problem flag matrix corresponding to the problem flag position, and setting the problem flag matrix to the first matrix block of the matrix M0, wherein the first matrix block is a 1*1 matrix; constructing a problem matrix corresponding to the question part, and setting the problem matrix to the second matrix block of the matrix M0, wherein the second matrix block is a QU*QU matrix, and QU is the number of words in the question part; constructing a content start flag matrix corresponding to the content start flag position, and setting the content start flag matrix to the third matrix block of the matrix M0, wherein the third matrix block is a 1*1 matrix; constructing a text block corresponding to the text block A text matrix is ​​constructed, and the text matrix is ​​set as the fourth matrix block of the matrix M0, wherein the fourth matrix block is an AN*AN matrix, and AN is the number of words in the text block; a content termination flag matrix corresponding to the content termination flag is constructed, and the content termination flag matrix is ​​set as the fifth matrix block of the matrix M0, wherein the third matrix block is a 1*1 matrix, wherein the elements in the first matrix block to the fifth matrix block are all 1, and the first matrix block to the fifth matrix block are sequentially arranged in the row direction of the matrix M0; the elements in the matrix M0 that are located in the same column as the second matrix block and in the same row as the fourth matrix block are set to 1; the elements in other positions of the matrix M0 are set to 0.

[0024] Further, the question and answer result information includes a first probability, a second probability vector and a third probability vector, wherein the first probability is the probability that the content part includes the answer, the second probability vector includes the probability that each word in the content part is the starting word of the answer, and the third probability vector includes the probability that each word in the content part is the ending word of the answer. The step of determining whether the content part includes the answer corresponding to the question part and the position of the answer in the content part according to the question and answer result information includes: judging whether the first probability is greater than a preset probability threshold, and when the first probability is greater than the preset probability threshold, the content part includes the answer; obtaining the word corresponding to the maximum probability value in the second probability vector as the starting word of the answer; obtaining the word corresponding to the maximum probability value in the third probability vector as the ending word of the answer; obtaining the part between the starting word and the ending word to obtain the answer.

[0025] On the other hand, to achieve the above-mentioned purpose, the present invention provides a question and answer processing device.

[0026] The question-answering processing device includes: an acquisition module, used to acquire a question part and a content part of a target document, wherein the content part includes at least two text blocks; a processing module, used to construct an initial input vector according to the question part and the content part; an input module, used to input the initial input vector into a preset question-answering machine model, wherein the question-answering machine model includes a pre-trained model layer, an attention model layer and a final output layer, the pre-trained model layer is used to obtain a first word vector after processing the input initial input vector, and output it to the attention model layer and the final output layer, the attention model layer is used to obtain a second word vector after processing the input first word vector, and output it to the final output layer, the second word vector is used to represent the attention size of a first word to other words in the text block where the first word is located, the first word is a word in the content part, and the final output layer is used to output question-answering result information according to the first word vector and the second word vector; and a determination module, used to determine whether the content part includes an answer corresponding to the question part and the position of the answer in the content part according to the question-answering result information.

[0027] To achieve the above objectives, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0028] To achieve the above object, the present invention also provides a computer-readable storage medium on which a computer program is stored, and the computer program implements the steps of the above method when executed by a processor.

[0029] The question-answering processing method, apparatus, computer device and readable storage medium provided by the present invention preset a question-answering machine model, which includes a pre-trained model layer, an attention model layer and a final output layer. When the question part and the content part of the target document are obtained, the initial input vector constructed according to the question part and the content part is input into the question-answering machine model, the pre-trained model layer processes the initial input vector to obtain a first word vector, and outputs the first word vector to the attention model layer and the final output layer. The attention model layer processes the first word vector to obtain a second word vector, and outputs the second word vector to the final output layer. The final output layer outputs question-answering result information according to the first word vector and the second word vector. The output of the final output layer is also the output of the question-answering machine model. Finally, it is determined whether the content part of the target document includes the answer corresponding to the question part, and when the answer is included, the position of the answer in the content part. Among them, as an important inventive point of the present invention, the content part includes at least two text blocks, and the second word vector represents the attention of a word to other words in the text block where it is located, that is, the content part is divided into blocks, and the attention parameters are constrained in one text block through the attention model layer. Compared with the prior art in which the attention calculation scope is the entire text, the present invention effectively reduces the parameters of the question and answer machine model, reduces the overfitting of the question and answer machine model training process, is conducive to obtaining an accurate question and answer machine model, and improves the accuracy of the question and answer machine model, that is, improves the accuracy of the question and answer processing method. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0031] Figure 1 A flowchart of a question-and-answer processing method provided in Embodiment 1 of the present invention;

[0032] Figure 2 A schematic diagram of an attention matrix in the question-answering processing method provided in Embodiment 1 of the present invention;

[0033] Figure 3 A block diagram of a question-answer processing device provided in Embodiment 2 of the present invention;

[0034] Figure 4 This is a hardware structure diagram of a computer device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0036] In order to improve the accuracy of question and answer processing, the inventors conducted the following research on the machine learning models used for question and answer processing in the prior art: the prior art usually adopts an attention-based neural network model in question and answer processing, wherein the calculation scope of attention is the entire text of the content part of the document. In fact, when the distance between two words in the text is far, the attention of one word to another word will be very small. If the calculation scope of attention is reduced, that is, the distance between the two words targeted by the attention is reduced, the parameters of the neural network model will be reduced, thereby reducing overfitting in the training process, thereby improving the accuracy of the neural network model and improving the accuracy of the question and answer processing method.

[0037] Based on this, the present invention proposes a question-answering processing method, apparatus, computer device and readable storage medium. In the question-answering processing method provided by the present invention, the question part and the content part of the target document are first obtained, and the content part includes at least two text blocks. Then, an initial input vector is constructed according to the question part and the content part, and the initial input vector is input into a preset question-answering machine model. Finally, according to the output vector of the question-answering machine model, that is, the question-answering result information, it is determined whether the content part includes the answer corresponding to the question part and the position of the answer in the content part. The question-answering machine model includes a pre-trained model layer, an attention model layer and a final output layer. The input of the pre-trained model layer includes the initial The input vector, the output of the pre-trained model layer includes the first word vector, the pre-trained model layer is used to obtain the first word vector after processing the initial input vector, the input of the attention model layer includes the first word vector, the output of the attention model layer includes the second word vector, the attention model layer is used to obtain the second word vector after processing the first word vector, the input of the final output layer includes the first word vector and the second word vector, the final output layer is used to output the question-answering result information according to the first word vector and the second word vector, the question-answering result information output by the final output layer is also the output vector of the question-answering machine model, and the question-answering result information can be used to determine whether the content part includes the answer corresponding to the question part and the position of the answer in the content part. Among them, the second word vector output by the attention model layer is used to represent the attention size of a word to other words in the text block where the word is located, that is, the attention parameter is constrained in a text block, so that the parameters can be effectively reduced, overfitting can be reduced, and the focus of attention is placed on the inside of the text block, the accuracy of the neural network model is improved, and the accuracy of the question-answering processing method is improved.

[0038] Specific embodiments of the question-answer processing method, apparatus, computer device, and readable storage medium provided by the present invention will be described in detail below.

[0039] Embodiment 1

[0040] The embodiment of the present invention provides a question-answering processing method, through which the accuracy of question-answering processing can be improved. Specifically, Figure 1 This is a flowchart of the question-answering processing method provided in Embodiment 1 of the present invention, such as Figure 1 As shown, the question-answer processing method provided in this embodiment includes the following steps S101 to S104.

[0041] Step S101: Obtain the question part and content part of the target document.

[0042] The target document may be a text for reading comprehension, and the content part includes at least two text blocks. The number of words in each text block included in the content part may be the same or different. For example, when the content part is a document, the text blocks may be divided according to the logical structure of the document, such as a chapter, a section, a paragraph or a sentence of the document, which may all be regarded as a text block. Alternatively, the document may be subjected to natural semantic analysis, and the part with strong semantic correlation may be regarded as a text block.

[0043] Step S102: construct an initial input vector according to the question part and the content part.

[0044] Specifically, the question part and the content part are segmented respectively, and each word is used as a dimension of the initial input vector. At the same time, flag bits are set before the words in the question part, between the words in the question part and the words in the content part, and after the words in the content part.

[0045] For example, the initial input vector constructed based on the question part and the content part is: [CLS]+question+[SEP]+Context+[SEP], where [CLS], [SEP] and [SEP] are all flags, question includes the words of the question part, and Context includes the words of the content part.

[0046] Step S103: Input the initial input vector into the preset question-answering machine model.

[0047] Among them, the question-answering machine model includes a pre-training model layer, an attention model layer and a final output layer.

[0048] The input of the pre-trained model layer includes an initial input vector, and the output of the pre-trained model layer includes a first word vector. The pre-trained model layer is used to obtain the first word vector after processing the input initial input vector, and output it to the attention model layer and the final output layer.

[0049] The input of the attention model layer includes a first word vector, and the output of the attention model layer includes a second word vector. The second word vector is used to represent the attention of the first word to other words in the text block where the first word is located. The first word is a word in the content part. The attention model layer is used to process the input first word vector to obtain the second word vector and output it to the final output layer.

[0050] The input of the final output layer includes the first word vector and the second word vector, and the final output layer is used to output question and answer result information according to the first word vector and the second word vector.

[0051] Specifically, the input of the question-answering machine model is also the input of the pre-trained model layer, and the output of the pre-trained model layer is also the input of the attention model layer. The output of the pre-trained model layer and the output of the attention model layer are superimposed as the input of the final output layer, and the output of the final output layer is also the output of the question-answering machine model.

[0052] Among them, the pre-trained model layer converts each word in the initial input vector and outputs the first word vector corresponding to each word. The attention model layer calculates the attention of each word to other words in the text block where it is located based on the first word vector to obtain the second word vector. The final output layer outputs question and answer result information that can characterize whether the content part includes the answer corresponding to the question part and the position of the answer in the content part based on the superposition of the first word vector and the second word vector and the goal of the document task, so that it can be determined whether the content part includes the answer and the answer position based on the question and answer result information.

[0053] In addition, the question-answering machine model needs to be trained first. During training, an initial question-answering machine model with the above-mentioned pre-training model layer, attention model layer and final output layer is first constructed, training data is obtained, and then the initial question-answering machine model is trained with the training data to determine the values ​​of the parameters in the initial question-answering machine model, that is, to obtain the question-answering machine model required in step S103.

[0054] Specifically, the training data includes multiple training samples, each of which includes a question part and a content part. For each training sample, firstly, a training input vector is constructed according to the question part and the content part in the training sample in the same way as in the above step S102, and then a training output vector is constructed according to whether the answer corresponding to the question part is included in the content part and the position of the answer in the content part, and the training input vector is used as the input of the initial question-answering machine model, and the training output vector is used as the output of the initial question-answering machine model. The initial question-answering machine model is trained using multiple training samples, wherein during the training process, the optimal values ​​of the parameters in the initial question-answering machine model can be determined according to the gradient descent method to determine the optimal question-answering machine model.

[0055] Step S104: Determine whether the content part includes the answer corresponding to the question part and the position of the answer in the content part according to the question and answer result information.

[0056] Optionally, the output of the final output layer, i.e., the question-answer result information, includes a probability representing whether the content part includes an answer, the value of this probability is greater than or equal to 0 and less than or equal to 1, and the larger the value, the greater the possibility that the content part includes an answer; the output of the final output layer also includes, for each word in the content part, a probability representing that the word is the starting word of the answer, similarly, the value of this probability is also greater than or equal to 0 and less than or equal to 1, and the larger the value, the greater the possibility that the word corresponding to the probability is the starting word of the answer; the output of the final output layer also includes, for each word in the content part, a probability representing that the word is the ending word of the answer, similarly, the value of this probability is also greater than or equal to 0 and less than or equal to 1, and the larger the value, the greater the possibility that the word corresponding to the probability is the ending word of the answer. In this step, whether the content part includes the answer corresponding to the question part is determined based on the probability representing whether the content part includes the answer, and the starting word and ending word of the answer are determined based on the probability that each word is the starting word of the answer and the probability that the answer ends.

[0057] Accordingly, during the training process, when constructing a training output vector based on whether the content part includes the answer corresponding to the question part and the position of the answer in the content part, when the content part includes the answer corresponding to the question part, the probability value representing whether the content part includes the answer is 1, the probability value corresponding to the starting word of the answer in the content part is 1, and the probability value corresponding to other words in the content part is 0. The probability value corresponding to the ending word of the answer in the content part is 1, and the probability value corresponding to other words in the content part is 0. When the content part does not include the answer corresponding to the question part, the probability value representing whether the content part includes the answer is 0, and the probability of each word in the content part representing that it is the starting word of the answer and the probability of each word representing that it is the ending word of the answer are both 0, and the construction of the training output vector is completed.

[0058] In the question and answer processing method provided in this embodiment, a question and answer machine model is preset, and the question and answer machine model includes a pre-trained model layer, an attention model layer and a final output layer. When the question part and the content part of the target document are obtained, the initial input vector constructed according to the question part and the content part is input into the question and answer machine model, and the pre-trained model layer converts the initial input vector into a first word vector and inputs it into the attention model layer. The attention model layer outputs a second word vector. The first word vector and the second word vector are superimposed and input into the final output layer. The question and answer result information output by the final output layer, that is, the output of the question and answer machine model, can be used to determine whether the content part of the target document includes the answer corresponding to the question part, and when the answer is included, the position of the answer in the content part. Among them, as an important inventive point of the present invention, the content part includes at least two text blocks, and the second word vector represents the attention of a word to other words in the text block where it is located, that is, the content part is divided into blocks, and the attention parameters are constrained in one text block through the attention model layer. Compared with the prior art in which the attention calculation scope is the entire text, the parameters of the question and answer machine model are effectively reduced, the overfitting of the question and answer machine model training process is reduced, and it is conducive to obtaining an accurate question and answer machine model and improving the accuracy of the question and answer machine model, that is, improving the accuracy of the question and answer processing method.

[0059] Optionally, in one embodiment, the second word vector is also used to represent the attention of the first word to the words in the question part.

[0060] Specifically, in this embodiment, for each word, its corresponding first word vector includes the attention size of the word to the question part and the words in the text block where it is located. Since the semantic correlation between the question part and the answer is strong, the question and answer processing method provided by this embodiment is adopted, and the second word vector is set to also be used to characterize the attention size of the word to the words in the question part, so that the difference between the first word vector corresponding to the words in the answer and the first word vector corresponding to the words in the non-answer increases, which helps to improve the accuracy of determining the answer.

[0061] Optionally, in one embodiment, the attention model layer calculates the second word vector based on the first word vector using the following steps: input the first word vector into the first fully connected function to obtain the first vector; input the first word vector into the second fully connected function to obtain the second vector; input the first word vector into the third fully connected function to obtain the third vector, wherein the first fully connected function, the second fully connected function and the third fully connected function are different fully connected functions in the attention model layer, respectively; construct an attention matrix based on the question part and the text block; calculate the second word vector based on the first vector, the second vector, the third vector and the attention matrix. It should be noted that the "different fully connected functions" here can specifically be the same function model with different parameter values, or can also be different function models, and this application does not limit this.

[0062] Optionally, in one embodiment, the second word vector is calculated using the following formula:

[0063] H′=[h′1,h′2,...,h′ n ]=L(C(W1,W2,...,W n ))

[0064] W i =A i V i

[0065]

[0066] V=[V1,V2,...,V n ]=L1(H)

[0067] Q=[Q1,Q2,...,Q n ]=L2(H)

[0068] K=[K1,K2,...,K n ]=L3(H)

[0069] H=[h1,h2,...,h n ],

[0070] The input of the final output layer is calculated using the following formula:

[0071]

[0072]

[0073] Where H′ is the second word vector, n is the processing threshold of the initial input vector, C is the concatenation function, L is the full connection function, V is the first vector, L1 is the first full connection function, H is the first word vector, S is the normalization function, M is the attention matrix, Q is the second vector, L2 is the second full connection function, K is the third vector, L3 is the third full connection function, d k is a normalization constant, i=1, 2, ...n, is the input of the final output layer, and α is the preset weight.

[0074] Optionally, in one embodiment, the step of constructing an attention matrix according to the question part and the text block includes: constructing an n*n matrix M0; constructing a problem flag matrix corresponding to the problem flag position, and setting the problem flag matrix to the first matrix block of the matrix M0, wherein the first matrix block is a 1*1 matrix; constructing a problem matrix corresponding to the problem part, and setting the problem matrix to the second matrix block of the matrix M0, wherein the second matrix block is a QU*QU matrix, and QU is the number of words in the question part; constructing a content start flag matrix corresponding to the content start flag position, and setting the content start flag matrix to the third matrix block of the matrix M0, wherein the third matrix block is a 1*1 matrix; constructing a text matrix corresponding to the text block, and setting the text matrix is the fourth matrix block of matrix M0, wherein the fourth matrix block is a matrix of AN*AN, and AN is the number of words in the text block; constructs a content termination flag matrix corresponding to the content termination flag position, and sets the content termination flag matrix to the fifth matrix block of matrix M0, wherein the third matrix block is a 1*1 matrix, wherein the elements in the first matrix block to the fifth matrix block are all 1, and the first matrix block to the fifth matrix block are arranged sequentially in the row direction of matrix M0; sets the elements in the matrix M0 that are in the same column as the second matrix block and in the same row as the fourth matrix block to 1; sets the elements in other positions in the matrix M0 to 0, and obtains the attention matrix M. Through the attention matrix M, it is possible to constrain the attention parameters in a text block.

[0075] For example, Figure 2 This is a schematic diagram of the attention matrix in the question-answering processing method provided in the first embodiment of the present invention, such as Figure 2 As shown, when n=512, when constructing the attention matrix M, first construct a 512*512 square matrix M0, and then set the value of each element in the square matrix M0 to obtain the matrix M. The specific steps are as follows:

[0076] Construct a problem flag matrix corresponding to the problem flag bit. The problem flag matrix is ​​set as the first matrix block of the matrix M0. The first matrix block is a 1*1 matrix, which is the element of the first row and first column of the square matrix M0. The value of the element is 1.

[0077] Construct a question matrix corresponding to the question part, which is set as the second matrix block of the matrix M0. The question part includes 20 words. The second matrix block is a 20*20 matrix, which is set after the first matrix block, that is, as the elements of the 2nd to 21st rows and the 2nd to 21st columns of the square matrix M0, and the values ​​of these elements are set to 1;

[0078] Construct a content start flag matrix corresponding to the content start flag bit. The content start flag matrix is ​​set as the third matrix block of the matrix M0. The third matrix block is a 1*1 matrix. As the element of the 22nd row and 22nd column of the square matrix M0, the value of the element is 1.

[0079] Construct a text matrix corresponding to each text block. When the content part includes multiple text blocks, construct a text matrix for each text block, wherein each text matrix forms the fourth matrix block of the matrix M0. The content part includes two text blocks. The first text block includes 200 words. The fourth matrix block corresponding to the first text block is a 200*200 matrix, which is used as the elements of the 23rd to 222nd rows and the 23rd to 222nd columns of the square matrix M0. The second text block includes 150 words. The fourth matrix block corresponding to the second text block is a 150*150 matrix, which is used as the elements of the 223rd to 372nd rows and the 223rd to 372nd columns of the square matrix M0. The values ​​of these elements are all set to 1.

[0080] Construct a content termination flag matrix corresponding to the content termination flag bit. The content termination flag matrix is ​​set as the fifth matrix block of the matrix M0. The fifth matrix block is a 1*1 matrix. As the element of the 373rd row and 373rd column of the square matrix M0, the value of the element is 1.

[0081] Set the elements from row 23 to row 372 and column 2 to column 21 in matrix M0 to 1;

[0082] In the matrix M0, all elements at positions other than the above positions are set to 0, and the attention matrix M is obtained.

[0083] Optionally, in one embodiment, the question and answer result information includes a first probability, a second probability vector and a third probability vector, wherein the first probability is the probability that the content part includes an answer, the second probability vector includes the probability that each word in the content part is the starting word of the answer, and the third probability vector includes the probability that each word in the content part is the ending word of the answer, and the step of determining whether the content part includes the answer corresponding to the question part and the position of the answer in the content part according to the question and answer result information includes: judging whether the first probability is greater than a preset probability threshold, and when the first probability is greater than the preset probability threshold, the content part includes the answer, wherein the preset probability threshold is a dimensionless parameter, which can be set to be greater than 0.5 and less than 1; obtaining the word corresponding to the maximum probability value in the second probability vector as the starting word of the answer; obtaining the word corresponding to the maximum probability value in the third probability vector as the ending word of the answer; obtaining the part between the starting word and the ending word to obtain the answer.

[0084] Embodiment 2

[0085] Corresponding to the above-mentioned embodiment 1, the second embodiment of the present invention provides a question-answering processing device. The technical feature details and corresponding technical effect descriptions can refer to the above-mentioned embodiment 1, which will not be repeated here. Figure 3 A block diagram of a question-answer processing device provided in Embodiment 2 of the present invention, such as Figure 3 As shown, the device includes: an acquisition module 201, a processing module 202, an input module 203 and a determination module 204.

[0086] The acquisition module 201 is used to obtain the question part and the content part of the target document, wherein the content part includes at least two text blocks; the processing module 202 is used to construct an initial input vector based on the question part and the content part; the input module 203 is used to input the initial input vector into a preset question-answering machine model, wherein the question-answering machine model includes a pre-trained model layer, an attention model layer and a final output layer, the pre-trained model layer is used to process the input initial input vector to obtain a first word vector, and output it to the attention model layer and the final output layer, the attention model layer is used to process the input first word vector to obtain a second word vector, and output it to the final output layer, the second word vector is used to represent the attention size of the first word to other words in the text block where the first word is located, the first word is a word in the content part, and the final output layer is used to output question-answering result information based on the first word vector and the second word vector; and the determination module 204 is used to determine whether the content part includes the answer corresponding to the question part and the position of the answer in the content part based on the question-answering result information.

[0087] Optionally, in one embodiment, the text block is a paragraph or a sentence.

[0088] Optionally, in one embodiment, the second word vector is also used to represent the attention of the first word to the words in the question part.

[0089] Optionally, in one embodiment, the attention model layer calculates the second word vector based on the first word vector by the following steps: inputting the first word vector into a first fully connected function to obtain a first vector; inputting the first word vector into a second fully connected function to obtain a second vector; inputting the first word vector into a third fully connected function to obtain a third vector, wherein the first fully connected function, the second fully connected function and the third fully connected function are different fully connected functions in the attention model layer, respectively; constructing an attention matrix based on the question part and the text block; calculating the second word vector based on the first vector, the second vector, the third vector and the attention matrix.

[0090] Optionally, in one embodiment, the second word vector is calculated using the following formula:

[0091] H′=[h′1,h′2,...,h′ n ]=L(C(W1,W2,...,W n ))

[0092] W i =A i V i

[0093]

[0094] V=[V1,V2,...,V n ]=L1(H)

[0095] Q=[Q1,Q2,...,Q n ]=L2(H)

[0096] K=[K1,K2,...,K n ]=L3(H)

[0097] H=[h1,h2,...,h n ],

[0098] The input of the final output layer is calculated using the following formula:

[0099]

[0100]

[0101] Where H′ is the second word vector, n is the processing threshold of the initial input vector, C is the concatenation function, L is the full connection function, V is the first vector, L1 is the first full connection function, H is the first word vector, S is the normalization function, M is the attention matrix, Q is the second vector, L2 is the second full connection function, K is the third vector, L3 is the third full connection function, d k is a normalization constant, i=1,2,…n, is the input of the final output layer, and α is the preset weight.

[0102] Optionally, in one embodiment, the step of constructing an attention matrix according to the question part and the text block includes: constructing an n*n matrix M0; constructing a problem flag matrix corresponding to the problem flag position, and setting the problem flag matrix to the first matrix block of the matrix M0, wherein the first matrix block is a 1*1 matrix; constructing a problem matrix corresponding to the problem part, and setting the problem matrix to the second matrix block of the matrix M0, wherein the second matrix block is a QU*QU matrix, and QU is the number of words in the question part; constructing a content start flag matrix corresponding to the content start flag position, and setting the content start flag matrix to the third matrix block of the matrix M0, wherein the third matrix block is a 1*1 matrix; constructing Construct a text matrix corresponding to the text block, and set the text matrix to the fourth matrix block of the matrix M0, wherein the fourth matrix block is an AN*AN matrix, and AN is the number of words in the text block; construct a content termination flag matrix corresponding to the content termination flag, and set the content termination flag matrix to the fifth matrix block of the matrix M0, wherein the third matrix block is a 1*1 matrix, wherein the elements in the first matrix block to the fifth matrix block are all 1, and the first matrix block to the fifth matrix block are sequentially arranged in the row direction of the matrix M0; set the elements in the matrix M0 that are in the same column as the second matrix block and in the same row as the fourth matrix block to 1; set the elements in other positions of the matrix M0 to 0.

[0103] Optionally, in one embodiment, the question and answer result information includes a first probability, a second probability vector and a third probability vector, wherein the first probability is the probability that the content part includes an answer, the second probability vector includes the probability that each word in the content part is the starting word of the answer, and the third probability vector includes the probability that each word in the content part is the ending word of the answer, and the determination module 204 includes a judgment unit, a first acquisition unit, a second acquisition unit and a third acquisition unit, wherein the judgment unit is used to judge whether the first probability is greater than a preset probability threshold, and when the first probability is greater than the preset probability threshold, the content part includes an answer; the first acquisition unit is used to obtain the word corresponding to the maximum probability value in the second probability vector as the starting word of the answer; the second acquisition unit is used to obtain the word corresponding to the maximum probability value in the third probability vector as the ending word of the answer; the third acquisition unit is used to obtain the part between the starting word and the ending word to obtain the answer.

[0104] Embodiment 3

[0105] This third embodiment also provides a computer device, such as a smart phone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server or cabinet server (including an independent server or a server cluster composed of multiple servers) that can execute programs. Figure 4 As shown, the computer device 01 of this embodiment includes at least but is not limited to: a memory 011 and a processor 012 which can be interconnected through a system bus. Figure 4 It should be pointed out that Figure 4 Only a computer device 01 having components memory 011 and processor 012 is shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0106] In this embodiment, the memory 011 (i.e., readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 011 may be an internal storage unit of the computer device 01, such as a hard disk or memory of the computer device 01. In other embodiments, the memory 011 may also be an external storage device of the computer device 01, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. equipped on the computer device 01. Of course, the memory 011 may also include both the internal storage unit of the computer device 01 and its external storage device. In this embodiment, the memory 011 is generally used to store the operating system and various application software installed on the computer device 01, such as the program code of the question-answering processing device of the second embodiment. In addition, the memory 011 can also be used to temporarily store various types of data that have been output or are to be output.

[0107] In some embodiments, the processor 012 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 012 is generally used to control the overall operation of the computer device 01. In this embodiment, the processor 012 is used to run the program code stored in the memory 011 or process data, such as the question-answering processing method.

[0108] Embodiment 4

[0109] The fourth embodiment also provides a computer-readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a disk, an optical disk, a server, an App application store, etc., on which a computer program is stored, and the program realizes the corresponding function when executed by the processor. The computer-readable storage medium of this embodiment is used to store the question-answer processing device, and when executed by the processor, the question-answer processing method of the first embodiment is realized.

[0110] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0111] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0112] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0113] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A question-answering processing method, characterized in that: include: Acquire a question part and a content part of a target document, wherein the content part includes at least two text blocks; Constructing an initial input vector according to the question part and the content part; Inputting the initial input vector into a preset question-answering machine model, wherein the question-answering machine model includes a pre-trained model layer, an attention model layer, and a final output layer, wherein the pre-trained model layer is used to obtain a first word vector after processing the input initial input vector, and output it to the attention model layer and the final output layer, wherein the attention model layer is used to obtain a second word vector after processing the input first word vector, and output it to the final output layer, wherein the second word vector is used to represent the attention size of the first word to other words in the text block where the first word is located and the attention size of the first word to words in the question part, wherein the first word is a word in the content part, and wherein the final output layer is used to output question-answering result information according to the first word vector and the second word vector; and Determine, according to the question-answer result information, whether the content portion includes the answer corresponding to the question portion and the position of the answer in the content portion, The attention model layer calculates the second word vector according to the first word vector by the following steps: inputting the first word vector into a first fully connected function to obtain a first vector; inputting the first word vector into a second fully connected function to obtain a second vector; inputting the first word vector into a third fully connected function to obtain a third vector, wherein the first fully connected function, the second fully connected function and the third fully connected function are different fully connected functions in the attention model layer respectively; constructing an attention matrix according to the question part and the text block; calculating the second word vector according to the first vector, the second vector, the third vector and the attention matrix; The step of constructing an attention matrix according to the question part and the text block includes: constructing an n*n matrix M0; constructing a question flag matrix corresponding to the question flag bit, and setting the question flag matrix to the first matrix block of the matrix M0, wherein the first matrix block is a 1*1 matrix; constructing a question matrix corresponding to the question part, and setting the question matrix to the second matrix block of the matrix M0, wherein the second matrix block is a QU*QU matrix, and QU is the number of words in the question part; constructing a content start flag matrix corresponding to the content start flag bit, and setting the content start flag matrix to the third matrix block of the matrix M0, wherein the third matrix block is a 1*1 matrix; constructing a text start flag matrix corresponding to the text block. This matrix, and set the text matrix to the fourth matrix block of the matrix M0, wherein the fourth matrix block is an AN*AN matrix, and AN is the number of words in the text block; construct a content termination flag matrix corresponding to the content termination flag, and set the content termination flag matrix to the fifth matrix block of the matrix M0, wherein the third matrix block is a 1*1 matrix, wherein the elements in the first matrix block to the fifth matrix block are all 1, and the first matrix block to the fifth matrix block are arranged in sequence in the row direction of the matrix M0; set the elements in the matrix M0 that are in the same column as the second matrix block and in the same row as the fourth matrix block to 1; set the elements in other positions of the matrix M0 to 0.

2. The question-answer processing method according to claim 1, characterized in that: The text block is a paragraph or a sentence.

3. The question-answer processing method according to claim 1, characterized in that: The second word vector is calculated using the following formula: ， The input of the final output layer is calculated using the following formula: ， in, is the second word vector, n is the processing threshold of the initial input vector, is the concatenation function, is the full connection function, V is the first vector, L1 is the first full connection function, H is the first word vector, is a normalization function, M is the attention matrix, Q is the second vector, L2 is the second fully connected function, K is the third vector, L3 is the third fully connected function, is a normalization constant, i=1,2,…n, is the input of the final output layer, The preset weights.

4. The question-answer processing method according to claim 1, characterized in that: The question-and-answer result information includes a first probability, a second probability vector, and a third probability vector, wherein the first probability is the probability that the content part includes the answer, the second probability vector includes the probability that each word in the content part is the starting word of the answer, and the third probability vector includes the probability that each word in the content part is the ending word of the answer. The step of determining whether the content part includes the answer corresponding to the question part and the position of the answer in the content part according to the question-and-answer result information includes: determining whether the first probability is greater than a preset probability threshold, and when the first probability is greater than the preset probability threshold, the content portion includes the answer; Obtaining a word corresponding to a maximum probability value in the second probability vector as a starting word of the answer; Obtaining the word corresponding to the maximum probability value in the third probability vector as the ending word of the answer; The part between the starting word and the ending word is obtained to obtain the answer.

5. A question-answer processing device, characterized in that: include: An acquisition module, used for acquiring a question part and a content part of a target document, wherein the content part includes at least two text blocks; A processing module, used for constructing an initial input vector according to the question part and the content part; An input module, used to input the initial input vector into a preset question-answering machine model, wherein the question-answering machine model includes a pre-trained model layer, an attention model layer and a final output layer, the pre-trained model layer is used to obtain a first word vector after processing the input initial input vector, and output it to the attention model layer and the final output layer, the attention model layer is used to obtain a second word vector after processing the input first word vector, and output it to the final output layer, the second word vector is used to represent the attention size of the first word to other words in the text block where the first word is located and the attention size of the first word to words in the question part, the first word is a word in the content part, and the final output layer is used to output question-answering result information according to the first word vector and the second word vector; and a determination module, configured to determine, based on the question-answer result information, whether the content portion includes an answer corresponding to the question portion and a position of the answer in the content portion, The attention model layer calculates the second word vector according to the first word vector by the following steps: inputting the first word vector into a first fully connected function to obtain a first vector; inputting the first word vector into a second fully connected function to obtain a second vector; inputting the first word vector into a third fully connected function to obtain a third vector, wherein the first fully connected function, the second fully connected function and the third fully connected function are different fully connected functions in the attention model layer respectively; constructing an attention matrix according to the question part and the text block; calculating the second word vector according to the first vector, the second vector, the third vector and the attention matrix; The step of constructing an attention matrix according to the question part and the text block includes: constructing an n*n matrix M0; constructing a question flag matrix corresponding to the question flag bit, and setting the question flag matrix to the first matrix block of the matrix M0, wherein the first matrix block is a 1*1 matrix; constructing a question matrix corresponding to the question part, and setting the question matrix to the second matrix block of the matrix M0, wherein the second matrix block is a QU*QU matrix, and QU is the number of words in the question part; constructing a content start flag matrix corresponding to the content start flag bit, and setting the content start flag matrix to the third matrix block of the matrix M0, wherein the third matrix block is a 1*1 matrix; constructing a text start flag matrix corresponding to the text block. This matrix, and set the text matrix to the fourth matrix block of the matrix M0, wherein the fourth matrix block is an AN*AN matrix, and AN is the number of words in the text block; construct a content termination flag matrix corresponding to the content termination flag, and set the content termination flag matrix to the fifth matrix block of the matrix M0, wherein the third matrix block is a 1*1 matrix, wherein the elements in the first matrix block to the fifth matrix block are all 1, and the first matrix block to the fifth matrix block are arranged in sequence in the row direction of the matrix M0; set the elements in the matrix M0 that are in the same column as the second matrix block and in the same row as the fourth matrix block to 1; set the elements in other positions of the matrix M0 to 0.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images