Answer text extraction method, device, equipment, medium and product

By using the target context encoder and multiple neural networks to calculate word vectors of word elements in documents, the problem that existing question-and-answer systems cannot handle multi-section answers is solved, and the answer extraction of multiple segments in the document is realized, improving user experience .

CN118627506BActive Publication Date: 2024-11-08BEIJING BEIDA SOFTWARE ENG DEV CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411110723.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-14
Publication Date
2024-11-08
Estimated Expiration
2044-08-14

AI Technical Summary

Technical Problem

The existing question and answer system cannot effectively handle the task of answer paragraphs in a document that are composed of multiple paragraphs and cannot accurately extract multi-paragraph answers.

Method used

Use target context encoder and multiple neural networks (feedforward neural network, dual affine transformation, convolutional neural network) to obtain word vectors in the document, and calculate the similarity between the representation vectors of candidate answer segments and the expression vectors of the problem representation vectors , distinguish answer segments through global threshold representation vectors, and output target candidate answer segments.

Benefits of technology

It realizes accurate extraction of answer segments composed of multiple segments in a document, improves user experience, and can handle tasks of single segments and multiple segments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118627506B_ABST
    Figure CN118627506B_ABST
Patent Text Reader

Abstract

The present application discloses a method, device, equipment, medium and product for extracting answer text, which relates to the field of natural language processing technology. The method includes: receiving a user question and a document containing a corresponding answer; respectively obtaining the word vector of a word unit in the user question, the word vector of a word unit in the document and the word vector of a global word unit; obtaining the representation vectors of all candidate answer text units corresponding to the word vector of the word unit in the document; obtaining the question representation vector corresponding to the word vector of the word unit in the user question and the global threshold representation vector corresponding to the word vector of the global word unit; obtaining the first similarity between the representation vector of each candidate answer text unit and the question representation vector, and the second similarity between the global threshold representation vector and the question representation vector; obtaining a target similarity greater than the second similarity in the first similarity; and outputting the target candidate answer text unit corresponding to the target similarity. The present application can handle tasks in which the answer text unit in the document is composed of multiple text units.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a method, device, equipment, medium and product for extracting answer text. Background Art

[0002] Currently, question answering systems have become a research hotspot in industry and academia.

[0003] In the related art, when obtaining the answer text that the user wants from a document through a question-and-answer system, the question-and-answer system uses a pre-trained language model to encode the document, and then directly predicts the starting and ending positions of the answer text in the form of classification, and finally obtains and outputs the answer text based on the starting and ending positions of the answer text.

[0004] However, this method can only handle tasks where the answer paragraph in the document consists of a single paragraph, and cannot handle tasks where the answer paragraph in the document consists of multiple paragraphs. Summary of the invention

[0005] The purpose of this application is to provide a method, device, equipment, medium and product for extracting answer segments, which can handle tasks in which the answer segment in a document is composed of multiple segments.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a method for extracting an answer paragraph, comprising:

[0008] Receive a user question and a document containing an answer corresponding to the user question;

[0009] Obtaining, through a target context encoder, a word vector of a word unit in the user question, a word vector of a word unit in the document, and a word vector of a global word unit respectively;

[0010] Obtaining representation vectors of all candidate answer passages corresponding to word vectors of word units in the document through a first target feedforward neural network, a target biaffine transformation, and a target convolutional neural network;

[0011] Obtaining, by a second target feedforward neural network, a question representation vector corresponding to the word vector of the word unit in the user question and a global threshold representation vector corresponding to the word vector of the global word unit;

[0012] Obtaining a first similarity between a representation vector of each candidate answer text segment and the question representation vector, and a second similarity between the global threshold representation vector and the question representation vector;

[0013] Acquire a target similarity, where the target similarity is a similarity among the first similarities that is greater than the second similarity;

[0014] Obtaining a representation vector of the target candidate answer text segment corresponding to the target similarity;

[0015] Output the target candidate answer segments corresponding to the representation vector of the target candidate answer segments, wherein the target candidate answer segments are all answer segments corresponding to the user question.

[0016] In a second aspect, the present application provides an answer text extraction device, comprising:

[0017] A receiving module, used to receive a user question and a document containing an answer corresponding to the user question;

[0018] A word vector acquisition module, used to respectively acquire the word vector of the word unit in the user question, the word vector of the word unit in the document, and the word vector of the global word unit through a target context encoder;

[0019] A first representation vector acquisition module, used to acquire the representation vectors of all candidate answer passages corresponding to the word vectors of the word elements in the document through a first target feedforward neural network, a target biaffine transformation and a target convolutional neural network;

[0020] A second representation vector acquisition module is used to acquire, through a second target feedforward neural network, a question representation vector corresponding to the word vector of the word unit in the user question and a global threshold representation vector corresponding to the word vector of the global word unit;

[0021] A similarity acquisition module, used to acquire a first similarity between the representation vector of each candidate answer text segment and the question representation vector, and a second similarity between the global threshold representation vector and the question representation vector;

[0022] A target similarity acquisition module, used to acquire target similarity, where the target similarity is a similarity among the first similarities that is greater than the second similarity;

[0023] A target representation vector acquisition module, used to acquire the representation vector of the target candidate answer text segment corresponding to the target similarity;

[0024] The answer text output module is used to output the target candidate answer text corresponding to the representation vector of the target candidate answer text.

[0025] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the above-described answer text extraction methods.

[0026] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the above-described answer text extraction methods.

[0027] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned answer text extraction methods.

[0028] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0029] The present application provides an answer paragraph extraction method, device, equipment, medium and product. By representing each paragraph in a document in the form of a vector, the representation vectors of all candidate answer paragraphs in the document can be obtained, and then the similarity between the question representation vector and the representation vector of each candidate answer paragraph can be calculated, and the representation vector of the target candidate answer paragraph in the representation vectors of all candidate answer paragraphs can be determined based on the similarity. The target candidate answer paragraph corresponding to the obtained representation vector of the target candidate answer paragraph is the answer paragraph of the user's question. In this way, not only can the task of processing the answer paragraph in the document consisting of a single paragraph be handled, but also the task of processing the answer paragraph in the document consisting of multiple paragraphs can be handled, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 A schematic diagram of a flow chart of a method for extracting an answer text segment provided in one embodiment of the present application;

[0032] Figure 2 A flowchart of a method for extracting answer text provided in the second embodiment of the present application;

[0033] Figure 3 A schematic flow chart of a method for extracting answer text provided in the third embodiment of the present application;

[0034] Figure 4 A schematic diagram of functional modules of an answer text extraction device provided in one embodiment of the present application;

[0035] Figure 5 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0037] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0038] Figure 1 A flowchart of a method for extracting an answer text provided in an embodiment of the present application is shown below. Figure 2 A schematic diagram of a method for extracting an answer text provided in the second embodiment of the present application, combined with Figure 1 and Figure 2 The answer text extraction method includes the following steps 101 to 108:

[0039] Step 101: Receive a user question and a document containing an answer to the user question.

[0040] The document containing the answer to the user's question may be a document input by the user himself, or may be a document retrieved based on the user's question through an existing information retrieval method.

[0041] It is worth noting that in the present disclosure, each algorithm model in the answer text extraction method needs to be trained in advance, that is, before receiving the user's question, the answer text extraction method also includes the following model training steps 201 to 205:

[0042] Step 201, through the context encoder, respectively obtain the word vector of the word unit in the sample question, the word vector of the word unit in the sample document and the word vector of the global sample word unit.

[0043] The sample document contains the answer that can answer the sample question. The answer is in the form of a span in the sample document, that is, a continuous segment of tokens. The number of spans contained in the answer is one or more. The goal of the training task is to extract all the answer spans contained in the sample document based on the sample question.

[0044] Specifically, the context encoder can use a pre-trained language model, such as the BERT model:

[0045] ;

[0046] in, Represents the BERT model used as a context encoder. [CLS] and [SEP] are two special tokens related to the pre-trained language model. [CLS] integrates the global input information, and [SEP] is used to distinguish sample questions and sample documents in the input. From the formula corresponding to the BERT model, it can be seen that each input token is encoded into a vector.

[0047] For example: There is a sample question , the sample problem The word element in is , that is, the sample problem is expressed as , Similarly, the word unit in sample document C is , that is, the sample document is represented as , then, through Then we can get the word vector of the word unit in the sample question, the word vector of the word unit in the sample document and the word vector of the global sample word unit. .

[0048] exist middle, For global sample word input to The word vector of the global sample word obtained, for Input to The word vector of the word unit in the sample question obtained , for Input to The word vectors of the word units in the sample documents obtained .

[0049] Step 202, through the first feedforward neural network, double affine transformation and convolutional neural network, obtain the representation vectors of all candidate sample answer paragraphs corresponding to the word vectors of the word units in the sample document.

[0050] In one embodiment, step 202 includes the following sub-steps A1-A3:

[0051] A1. Extract the starting position feature vector of each candidate sample answer paragraph and the ending position feature vector of each candidate sample answer paragraph from the word vector of the word unit in the sample document through the first feedforward neural network.

[0052] For the word vectors of the word units in the sample documents , a starting position feature vector of a candidate sample answer paragraph and an ending position feature vector of a candidate sample answer paragraph are extracted from the word vectors of word units in the sample document through two different first feedforward neural networks, so that the starting position feature vector of the candidate sample answer paragraph and the ending position feature vector of the candidate sample answer paragraph can be independently learned to distinguish when used.

[0053] The starting position feature vector of the candidate sample answer paragraph can be extracted from the word vector of the word unit in the sample document by the following formula:

[0054] ;

[0055] in, A first feedforward neural network network for extracting a starting position feature vector of a candidate sample answer paragraph from a word vector of a word unit in a sample document; To use the i-th word vector in the sample document word unit as the starting position feature vector of the candidate sample answer paragraph, .

[0056] The end position feature vector of the candidate sample answer paragraph can be extracted from the word vector of the word unit in the sample document by the following formula:

[0057] ;

[0058] in, A first feedforward neural network network for extracting the end position feature vector of the candidate sample answer paragraph from the word vector of the word unit in the sample document; To use the jth word vector in the word unit in the sample document as the end position feature vector of the candidate sample answer paragraph, .

[0059] A2. According to the starting position feature vector of each candidate sample answer paragraph and the ending position feature vector of each candidate sample answer paragraph, a preliminary representation vector of each candidate sample answer paragraph is obtained through double affine transformation.

[0060] Specifically, the preliminary representation vector of each candidate sample answer paragraph can be obtained by the following formula:

[0061] ;

[0062] in, Represents the preliminary representation vector of the candidate sample answer paragraph composed of the i-th word vector to the j-th word vector in the sample document; Indicates that the i-th word vector in the sample document is used as the starting position feature vector of the candidate sample answer paragraph, Indicates that the j-th word vector in the sample document is used as the end position feature vector of the candidate sample answer paragraph; U represents the parameter matrix in the double affine transformation, W represents the parameter matrix of the linear transformation, b represents the bias term, and ⊕ represents the concatenation operation of the vectors.

[0063] A3. Based on the preliminary representation vectors of each candidate sample answer paragraph, a convolutional neural network is used to obtain the interactive paragraph representation vectors after the interaction between each candidate sample answer paragraph.

[0064] In order to make full use of the local semantic information of the paragraph context and the global information of the document, a convolutional neural network is used to interact between paragraphs in the two-dimensional grid form presented by the candidate answer paragraph arrangement. The interactive paragraph representation vector after the interaction between each candidate sample answer paragraph is obtained through the following formula:

[0065] ;

[0066] in, The interactive paragraph representation vector after the interaction between the preliminary representation vector of the candidate sample answer paragraph composed of the i-th word vector to the j-th word vector in the sample document and the preliminary representation vectors of other candidate sample answer paragraphs; Conv2d represents a two-dimensional convolutional neural network; ×N means stacking N layers of CNNBlock; the value of N is determined according to the length of the candidate sample answer paragraph and the size of the convolution kernel, and is generally 3~5; Each i, j in is composed of All i, j in CNNBlock are obtained by convolution, that is, convolution operation is performed on the whole. CNNBlock is specifically expressed as follows:

[0067] ;

[0068] Among them, CNNBlock represents the convolution module; Activation represents the nonlinear activation function, and LayerNorm represents the layer normalization of the feature dimension. represents the initial representation vector of all candidate sample answer paragraphs, that is, all possible sample paragraphs in sample document C, and the above express The i-th row and j-th column in represents the preliminary representation vector of the candidate sample answer paragraph composed of the i-th word vector to the j-th word vector in the sample document C.

[0069] Since a single-layer two-dimensional convolutional neural network can only achieve small-scale text interaction, in order to achieve multi-level and long-distance information interaction, multiple layers of two-dimensional convolutional neural networks are stacked in A3.

[0070] Furthermore, in order to improve the performance and stability of the model, a residual module is added between each layer of the two-dimensional convolutional neural network, for example: The input contains , this is the residual.

[0071] A 4. Obtain the representation vector of each candidate sample answer paragraph based on the preliminary representation vector of each candidate sample answer paragraph and the corresponding interactive paragraph representation vector of each candidate sample answer paragraph.

[0072] Finally, by adding the preliminary representation vector of each candidate sample answer paragraph to the corresponding interactive paragraph representation vector of each candidate sample answer paragraph, the representation vector of each candidate sample answer paragraph can be obtained:

[0073] ;

[0074] in, Represents the final representation vector of the candidate sample answer paragraph composed of the i-th word vector to the j-th word vector in the sample document.

[0075] Step 203: Obtain, through a second feedforward neural network, a sample question representation vector corresponding to the word vector of the word unit in the sample question and a global sample threshold vector corresponding to the word vector of the global sample word unit.

[0076] In one embodiment, the second feedforward neural network includes a first sub-feedforward neural network and a second sub-feedforward neural network, and step 203 includes the following sub-steps B1-B3:

[0077] B1. The features of the word vectors of all word units in the sample questions are integrated through the maximum pooling operation to obtain the comprehensive sample word vector.

[0078] Based on the word vector of the word in the sample question obtained in step 201 , we can first use the maximum pooling operation to combine the features of all word units. The maximum pooling operation (Max Pooling) is to take the largest number according to the dimension for a group of vectors of the same dimension, and finally obtain a vector. The size of each dimension of the vector is the largest value of the dimension in the vector group.

[0079] B2. Use the first sub-feedforward neural network to project the comprehensive sample word vector into the same representation space as the representation vector of the candidate sample answer paragraph to obtain the sample question representation vector.

[0080] Use the first sub-feedforward neural network to project the comprehensive sample word vector into the same representation space of the candidate sample answer paragraph to obtain the sample question representation vector :

[0081] ;

[0082] in, correspond , is the whole word vector of all the words in the sample question. The maximum aggregation operation is performed on these vectors to obtain a vector, which represents the entire question.

[0083] B3. Use the second sub-feedforward neural network to project the word vector of the global sample word unit into the same representation space as the representation vector of the candidate sample answer paragraph to obtain the global sample threshold vector.

[0084] The word vector of the special word [CLS] representing the global sample information obtained by encoding in step 201 , because it contains the information of sample questions, each candidate sample answer document in the sample document, and the global information of the interaction of each candidate sample answer document in the sample document during the encoding process, it is used as a threshold vector to distinguish the sample answer vector and the non-sample answer vector in the candidate sample answer paragraph. It is projected into the same representation space of the candidate sample answer paragraph and the sample question vector through a feedforward neural network to obtain the representation vector of the global sample threshold :

[0085] .

[0086] Among them, the sample answer vector corresponds to the answer paragraph of the sample question in the candidate sample answer paragraph, and the non-sample answer vector corresponds to the answer paragraph of the candidate sample answer paragraph that is not a sample question.

[0087] Step 204, obtaining a loss function value according to the representation vectors of all candidate sample answer passages, the sample question representation vector and the global sample threshold vector.

[0088] After encoding in steps 201 to 203, the sample question, the candidate sample answer paragraph, and the global sample word are all encoded as vectors and projected into the same representation space. The goal of the training phase is to make the similarity between the representation vector of the sample answer paragraph in the candidate sample answer paragraph and the representation vector of the sample question higher than the similarity between the representation vector of the non-sample answer paragraph in the candidate sample answer paragraph and the representation vector of the sample question through comparative learning. At the same time, the global threshold vector can accurately segment the representation vector of the sample answer paragraph and the representation vector of the non-sample answer paragraph.

[0089] In this disclosure, the measure of similarity between vectors uses the scaled dot product:

[0090] ;

[0091] in, and is the vector for similarity calculation, is a learnable scaling factor.

[0092] Specifically, the representation vectors of all sample answer paragraphs and the representation vectors of all non-sample answer paragraphs can be obtained from the representation vectors of all candidate sample answer paragraphs, and then the loss function value can be obtained by the following formula:

[0093] ;

[0094] in, Represents the sample problem representation vector; represents the global sample threshold vector, P represents the set of representation vectors of all sample answer paragraphs, represents the representation vector of the i-th sample answer paragraph; N represents the set of representation vectors of all non-sample answer paragraphs; represents the representation vector of the jth non-sample answer paragraph, ; ; ; Indicates the first scaling factor.

[0095] Among them, the loss function in the training stage includes two parts, L1 and L2. The first part L1 is that the similarity between the representation vector of the sample answer text and the representation vector of the sample question is higher than the similarity between the global sample threshold vector and the representation vector of the sample question, as well as the similarity between the representation vector of the non-sample answer text and the representation vector of the sample question; the second part L2 of the loss function ensures that the similarity between the global sample threshold vector and the representation vector of the sample question is higher than the similarity between the representation vector of the non-sample answer text and the representation vector of the sample question.

[0096] In the present disclosure, the loss function is used to optimize the model parameters during the training phase, and the stochastic gradient descent algorithm in deep learning can be used.

[0097] Step 205, based on the loss function value, update the parameters of the context encoder, the parameter matrix of the dual affine transformation, the convolutional neural network, the first feedforward neural network and the second feedforward neural network, and repeat the above process until the loss function value meets the preset requirements to obtain the target context encoder, the parameter matrix of the target dual affine transformation, the target convolutional neural network, the first target feedforward neural network and the second target feedforward neural network.

[0098] Step 102, through the target context encoder, respectively obtain the word vector of the word unit in the user question, the word vector of the word unit in the document, and the word vector of the global word unit.

[0099] Specifically, the implementation of this step is similar to step 201 and will not be repeated here.

[0100] Step 103, obtaining the representation vectors of all candidate answer passages corresponding to the word vectors of word units in the document through the first target feedforward neural network, the target bi-affine transformation and the target convolutional neural network.

[0101] Specifically, the implementation of this step is similar to step 202 and will not be repeated here.

[0102] Step 104: Obtain, through the second target feedforward neural network, a question representation vector corresponding to the word vector of the word unit in the user's question and a global threshold representation vector corresponding to the word vector of the global word unit.

[0103] Specifically, the implementation of this step is similar to step 203 and will not be repeated here.

[0104] Step 105: Obtain a first similarity between the representation vector of each candidate answer text segment and the question representation vector, and a second similarity between the global threshold representation vector and the question representation vector.

[0105] In an implementable manner, step 105 includes the following sub-steps: obtaining the first similarity and the second similarity respectively according to the scaled dot product.

[0106] Specifically, the first similarity is obtained by the following formula:

[0107] ;

[0108] in, The representation vector representing the i-th candidate answer paragraph; Representation problem represents vector; represents the second scaling factor;

[0109] The second similarity is obtained by the following formula:

[0110] ;

[0111] in, represents the global threshold representation vector; Represents the third scaling factor.

[0112] Step 106: Obtain target similarity, where the target similarity is a similarity in the first similarities that is greater than the second similarity.

[0113] The representation vector of the candidate answer paragraph whose similarity between the prediction and question representation vector is higher than the global threshold representation vector and the question representation vector is the final prediction result.

[0114] ;

[0115] Among them, A represents the representation vector of all candidate answer paragraphs, and the candidate answer paragraph corresponding to the prediction vector is the predicted answer paragraph.

[0116] Step 107, obtaining the representation vector of the target candidate answer segment corresponding to the target similarity.

[0117] Step 108, output the target candidate answer segments corresponding to the representation vectors of the target candidate answer segments, where the target candidate answer segments are all the answer segments corresponding to the user's question.

[0118] In the present disclosure, each paragraph in the document will be represented in the form of a vector, so that the representation vectors of all candidate answer paragraphs in the document can be obtained, and then the similarity between the question representation vector and the representation vector of each candidate answer paragraph can be calculated, and the representation vector of the target candidate answer paragraph in the representation vectors of all candidate answer paragraphs can be determined based on the similarity. The target candidate answer paragraph corresponding to the obtained representation vector of the target candidate answer paragraph is the answer paragraph of the user's question. In this way, not only can the task in which the answer paragraph in the document is composed of a single paragraph be handled, but also the task in which the answer paragraph in the document is composed of multiple paragraphs can be handled, thereby improving the user experience.

[0119] The following is an introduction to the model training method in the present disclosure through detailed examples.

[0120] For example: The sample question is: What campuses does M University have?

[0121] The sample document is: (from the Wikipedia entry "M University"), ... (some paragraphs), in addition to the main campus A, M University now has campus B, campus C, graduate school D, campus E, and campus F, a total of six campuses. ... (some paragraphs). (For the sake of simplicity, the sample document omits other paragraphs that do not contain the answer.)

[0122] From the sample document, we can see that the answers to the sample question are: A Main Campus, B Campus, C Campus, D Graduate School, E Campus, and F Campus. That is, the answer to the sample question is the six paragraphs in the sample document.

[0123] First, concatenate the sample question with the sample document, special word units [CLS] and [SEP] (note that [CLS] and [SEP] are a whole, similar to a word), and get the input of the context encoder as:

[0124] [CLS]What campuses does M University have? [SEP]In addition to the main campus A, M University now has campus B, campus C, graduate school D, campus E, and campus F, a total of six campuses. [SEP].

[0125] Input the above information into the context encoder (such as the BERT model) to obtain the word vector for each word unit:

[0126] ;

[0127] in, The encoded word vector corresponding to “[CLS]”; The encoded word vector corresponding to "What campuses does M University have?" The encoded word vector corresponding to "In addition to the main campus A, M University now has campus B, campus C, graduate school D, campus E, and campus F, a total of six campuses."

[0128] pass and Extract the starting position feature vector of each candidate sample answer paragraph and the ending position feature vector of each candidate sample answer paragraph.

[0129] pass Get the preliminary representation vector of each candidate sample answer paragraph.

[0130] pass Obtain the interactive text representation vector after the interaction between each candidate sample answer text.

[0131] according to Get the representation vector of each candidate sample answer paragraph.

[0132] R(i, j) represents the representation vector of the candidate answer paragraph composed of the i-th to j-th words in the sample document C. For example, R(2, 6) represents the representation vector of the candidate answer paragraph "A main campus" composed of the second to sixth words in "In addition to the A main campus, M University now has B campus, C campus, D graduate school, E campus, and F campus, a total of six campuses". R(1, 3) represents the representation vector of the candidate answer paragraph "Except A" composed of the first to third words in "In addition to the A main campus, M University now has B campus, C campus, D graduate school, E campus, and F campus, a total of six campuses". R(2, 6) is a sample answer paragraph, included in the set P of sample answer paragraphs, and R(1, 3) is a non-sample answer paragraph, included in the set N of non-sample answer paragraphs.

[0133] right , that is, for the encoded word vector of "What campuses does M University have?", first undergoes maximum aggregation and then projection to obtain the sample question representation vector .

[0134] right , that is, project the encoded word vector of "[CLS]" to get .

[0135] The loss function value is calculated according to the loss function formula, and then the stochastic gradient descent algorithm of the neural network in deep learning is used to update the model gradient.

[0136] The answer text extraction method in the present disclosure is introduced below through detailed examples.

[0137] For example: The user's question is: What campuses does M University have?

[0138] The document containing the answer to the user's question is: (from the Wikipedia entry "M University"), ... (some paragraphs), in addition to the main campus A, M University now has campus B, campus C, graduate school D, campus E, and campus F, a total of six campuses. ... (some paragraphs). (For the sake of simplicity, the document containing the answer to the user's question omits other paragraphs that do not contain the answer.)

[0139] From the document, we can see that the answers to the user's question are: A Main Campus, B Campus, C Campus, D Graduate School, E Campus, and F Campus. That is, the answer to the user's question is the six paragraphs in the document.

[0140] First, the user question is concatenated with the document, special words [CLS] and [SEP] to obtain the input of the target context encoder:

[0141] [CLS]What campuses does M University have? [SEP]In addition to the main campus A, M University now has campus B, campus C, graduate school D, campus E, and campus F, a total of six campuses. [SEP].

[0142] Input the above information into the target context encoder (such as the BERT model) to obtain the word vector for each word unit:

[0143] ;

[0144] in, The encoded word vector corresponding to “[CLS]”; The encoded word vector corresponding to "What campuses does M University have?" The encoded word vector corresponding to "In addition to the main campus A, M University now has campus B, campus C, graduate school D, campus E, and campus F, a total of six campuses."

[0145] pass and Extract the starting position feature vector of each candidate answer paragraph and the ending position feature vector of each candidate answer paragraph.

[0146] pass Get the preliminary representation vector of each candidate answer paragraph.

[0147] pass Obtain the interactive text representation vector after the interaction between each candidate answer text.

[0148] according to Get the representation vector of each candidate answer paragraph.

[0149] This step, yes , which corresponds to "What campuses does M University have?" The encoded word vector is subjected to maximum aggregation and projection to obtain the question representation vector .

[0150] right , that is, the encoded word vector corresponding to "[CLS]" is projected to obtain .

[0151] The similarity between the representation vector of all candidate answer paragraphs and the representation vector of the question is calculated according to the formula. For example, the representation vector of the candidate answer paragraph "A Main Campus" represented by R (2, 6) is similar to the representation vector of the question. The similarity is 0.80. The representation vector of the candidate answer passage "Except A" represented by R (1, 3) is similar to the question representation vector The similarity is 0.20.

[0152] At the same time, the similarity between the global threshold representation vector and the question representation vector is calculated as the threshold for judging whether the candidate answer segment is the final answer. For example, the calculated threshold is 0.52. Since 0.80>0.52, it is inferred that the candidate answer segment "A main campus" is the answer. Since 0.20<0.52, it is inferred that the candidate answer segment "except A" is not the answer. The similarity between the representation vector of each candidate answer segment and the question representation vector is compared with the threshold, so that all candidate answer segments can be obtained.

[0153] Based on the same inventive concept, the embodiment of the present application also provides a device for implementing the above-mentioned answer text extraction. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more embodiments of the answer text extraction device provided below can refer to the limitations of the answer text extraction method above, and will not be repeated here.

[0154] In an exemplary embodiment, Figure 4 As shown, a device for extracting an answer paragraph is provided, comprising:

[0155] A receiving module 11 is used to receive a user question and a document containing an answer corresponding to the user question;

[0156] A word vector acquisition module 12 is used to respectively acquire the word vector of the word unit in the user question, the word vector of the word unit in the document, and the word vector of the global word unit through a target context encoder;

[0157] A first representation vector acquisition module 13, used to acquire the representation vectors of all candidate answer text segments corresponding to the word vectors of the word units in the document through a first target feedforward neural network, a target biaffine transformation and a target convolutional neural network;

[0158] A second representation vector acquisition module 14 is used to acquire, through a second target feedforward neural network, a question representation vector corresponding to the word vector of the word unit in the user question and a global threshold representation vector corresponding to the word vector of the global word unit;

[0159] A similarity acquisition module 15, configured to acquire a first similarity between the representation vector of each candidate answer text segment and the question representation vector, and a second similarity between the global threshold representation vector and the question representation vector;

[0160] A target similarity acquisition module 16, configured to acquire a target similarity, wherein the target similarity is a similarity among the first similarities that is greater than the second similarity;

[0161] A target representation vector acquisition module 17 is used to acquire a representation vector of a target candidate answer text segment corresponding to the target similarity;

[0162] The answer text output module 18 is used to output the target candidate answer text corresponding to the representation vector of the target candidate answer text.

[0163] As an optional implementation, the above-mentioned answer text extraction device further includes:

[0164] A first acquisition module is used to obtain, by using a context encoder, word vectors of word units in the sample question, word vectors of word units in the sample document, and word vectors of global sample word units before the receiving module receives the user question;

[0165] A second acquisition module is used to acquire the representation vectors of all candidate sample answer text segments corresponding to the word vectors of the word units in the sample document through the first feedforward neural network, the double affine transformation and the convolutional neural network;

[0166] A third acquisition module is used to acquire, through a second feedforward neural network, a sample question representation vector corresponding to the word vector of the word unit in the sample question and a global sample threshold vector corresponding to the word vector of the global sample word unit;

[0167] A fourth acquisition module, configured to obtain a loss function value according to the representation vectors of all the candidate sample answer passages, the sample question representation vector and the global sample threshold vector;

[0168] An updating module is used to update the parameters of the context encoder, the parameter matrix of the dual affine transformation, the convolutional neural network, the first feedforward neural network and the second feedforward neural network based on the loss function value, and repeat the above process until the loss function value meets the preset requirements to obtain the target context encoder, the parameter matrix of the target dual affine transformation, the target convolutional neural network, the first target feedforward neural network and the second target feedforward neural network.

[0169] As an optional implementation, the second acquisition module includes:

[0170] A first extraction submodule is used to extract, through a first feedforward neural network, a starting position feature vector of each candidate sample answer paragraph and an ending position feature vector of each candidate sample answer paragraph from the word vector of the word unit in the sample document;

[0171] A first acquisition submodule is used to acquire a preliminary representation vector of each candidate sample answer paragraph through the double affine transformation according to the starting position feature vector of each candidate sample answer paragraph and the ending position feature vector of each candidate sample answer paragraph;

[0172] A second acquisition submodule is used to acquire, based on the preliminary representation vectors of the candidate sample answer segments, the interactive segment representation vectors after the interaction between the candidate sample answer segments through the convolutional neural network;

[0173] The third acquisition submodule is used to acquire the representation vector of each candidate sample answer paragraph according to the preliminary representation vector of each candidate sample answer paragraph and the corresponding interactive paragraph representation vector of each candidate sample answer paragraph.

[0174] As an optional implementation, the first acquisition submodule includes:

[0175] The preliminary representation vector of each candidate sample answer paragraph is obtained by the following formula:

[0176] ;

[0177] in, A preliminary representation vector representing the candidate sample answer paragraph composed of the i-th word vector to the j-th word vector in the sample document; Indicates that the i-th word vector in the sample document is used as the starting position feature vector of the candidate sample answer paragraph, represents the j-th word vector in the sample document as the end position feature vector of the candidate sample answer paragraph; U represents the parameter matrix in the double affine transformation, W represents the parameter matrix of the linear transformation, b represents the bias term, and ⊕ represents the concatenation operation of the vectors.

[0178] As an optional implementation, the second acquisition submodule includes:

[0179] The interactive text segment representation vector after the interaction between the candidate sample answer segments is obtained by the following formula:

[0180] ;

[0181] ;

[0182] in, The interactive paragraph representation vector after the interaction between the preliminary representation vector of the candidate sample answer paragraph composed of the i-th word vector to the j-th word vector in the sample document and the preliminary representation vectors of other candidate sample answer paragraphs; Conv2d represents a two-dimensional convolutional neural network; CNNBlock represents a convolution module; ×N represents stacking N layers of CNNBlock; the value of N is determined according to the length of the candidate sample answer paragraph and the size of the convolution kernel; Activation represents a nonlinear activation function, and LayerNorm represents layer normalization of feature dimensions. A preliminary representation vector representing all candidate sample answer passages.

[0183] As an optional implementation, the third acquisition module includes:

[0184] A fourth acquisition submodule is used to synthesize the features of the word vectors of all word units in the sample questions through a maximum aggregation operation to obtain a comprehensive sample word vector;

[0185] a fifth acquisition submodule, configured to use the first sub-feedforward neural network to project the comprehensive sample word vector into the same representation space as the representation vector of the candidate sample answer paragraph to obtain the sample question representation vector;

[0186] a sixth acquisition submodule, configured to use the second sub-feedforward neural network to project the word vector of the global sample word unit into the same representation space as the representation vector of the candidate sample answer paragraph, so as to obtain the global sample threshold vector;

[0187] The second feedforward neural network includes the first sub-feedforward neural network and the second sub-feedforward neural network.

[0188] As an optional implementation manner, the fourth acquisition module includes:

[0189] A seventh acquisition submodule, used to acquire the representation vectors of all sample answer segments and the representation vectors of all non-sample answer segments from the representation vectors of all the candidate sample answer segments;

[0190] The eighth acquisition submodule is used to obtain the loss function value through the following formula:

[0191] ;

[0192] in, Representing the sample problem representation vector; represents the global sample threshold vector, P represents the set of representation vectors of all the sample answer passages, represents the representation vector of the i-th sample answer paragraph; N represents the set of representation vectors of all non-sample answer paragraphs; represents the representation vector of the jth non-sample answer paragraph, ; ; ; Indicates the first scaling factor.

[0193] As an optional implementation, the similarity acquisition module includes:

[0194] A ninth acquisition submodule is configured to respectively acquire the first similarity and the second similarity according to the scaled dot product.

[0195] As an optional implementation manner, the ninth acquisition submodule includes:

[0196] The first similarity is obtained by the following formula:

[0197] ;

[0198] in, A representation vector representing the i-th candidate answer paragraph; Representing the problem representation vector; represents the second scaling factor;

[0199] The second similarity is obtained by the following formula:

[0200] ;

[0201] Among them, the represents the global threshold representation vector; Represents the third scaling factor.

[0202] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video tag processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for extracting an answer paragraph is implemented.

[0203] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0204] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0205] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0206] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0207] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0208] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0209] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0210] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0211] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for extracting answer text, characterized in that: The answer text extraction method comprises: Receive a user question and a document containing an answer corresponding to the user question; Obtaining, through a target context encoder, a word vector of a word unit in the user question, a word vector of a word unit in the document, and a word vector of a global word unit respectively; Obtaining representation vectors of all candidate answer text segments corresponding to word vectors of word units in the document through a first target feedforward neural network, a target biaffine transformation, and a target convolutional neural network; Obtaining, by a second target feedforward neural network, a question representation vector corresponding to the word vector of the word unit in the user question and a global threshold representation vector corresponding to the word vector of the global word unit; Obtaining a first similarity between a representation vector of each candidate answer text segment and the question representation vector, and a second similarity between the global threshold representation vector and the question representation vector; Acquire a target similarity, where the target similarity is a similarity among the first similarities that is greater than the second similarity; Obtaining a representation vector of the target candidate answer text segment corresponding to the target similarity; Outputting the target candidate answer segment corresponding to the representation vector of the target candidate answer segment, wherein the target candidate answer segment is all the answer segments corresponding to the user question; The step of obtaining the representation vectors of all candidate answer passages corresponding to the word vectors of the word units in the document includes: Extracting, by means of the first target feedforward neural network, the starting position feature vector of each candidate answer paragraph and the ending position feature vector of each candidate answer paragraph from the word vector of the word unit in the document; According to the starting position feature vector of each candidate answer paragraph and the ending position feature vector of each candidate answer paragraph, a preliminary representation vector of each candidate answer paragraph is obtained through the target double affine transformation; According to the preliminary representation vectors of the candidate answer paragraphs, obtaining the interaction paragraph representation vectors after the interaction between the candidate answer paragraphs through the target convolutional neural network; The representation vector of each candidate answer paragraph is obtained according to the preliminary representation vector of each candidate answer paragraph and the corresponding interactive paragraph representation vector of each candidate answer paragraph.

2. The answer text extraction method according to claim 1, characterized in that: Before receiving the user's question, the answer text extraction method further includes: Through the context encoder, the word vectors of the word units in the sample question, the word vectors of the word units in the sample document, and the word vectors of the global sample word units are obtained respectively; Obtaining representation vectors of all candidate sample answer passages corresponding to word vectors of word units in the sample document through a first feedforward neural network, a double affine transformation, and a convolutional neural network; Obtaining, by a second feedforward neural network, a sample question representation vector corresponding to the word vector of the word unit in the sample question and a global sample threshold vector corresponding to the word vector of the global sample word unit; Obtaining a loss function value according to the representation vectors of all the candidate sample answer passages, the sample question representation vector and the global sample threshold vector; Based on the loss function value, the context encoder, the parameter matrix of the dual affine transformation, the convolutional neural network, the parameters of the first feedforward neural network and the second feedforward neural network are updated, and the above process is repeated until the loss function value meets the preset requirements to obtain the target context encoder, the parameter matrix of the target dual affine transformation, the target convolutional neural network, the first target feedforward neural network and the second target feedforward neural network.

3. The answer text extraction method according to claim 2, characterized in that: Obtaining representation vectors of all candidate sample answer passages corresponding to word vectors of word units in the sample document through a first feedforward neural network, a double affine transformation, and a convolutional neural network, including: Extracting, by a first feedforward neural network, the starting position feature vector of each candidate sample answer paragraph and the ending position feature vector of each candidate sample answer paragraph from the word vector of the word unit in the sample document; According to the starting position feature vector of each candidate sample answer paragraph and the ending position feature vector of each candidate sample answer paragraph, the preliminary representation vector of each candidate sample answer paragraph is obtained through the double affine transformation; According to the preliminary representation vectors of each candidate sample answer paragraph, obtaining the interaction paragraph representation vector after the interaction between each candidate sample answer paragraph through the convolutional neural network; The representation vector of each candidate sample answer paragraph is obtained according to the preliminary representation vector of each candidate sample answer paragraph and the corresponding interactive paragraph representation vector of each candidate sample answer paragraph.

4. The answer text extraction method according to claim 3, characterized in that: The step of obtaining the preliminary representation vector of each candidate sample answer paragraph through the double affine transformation according to the starting position feature vector of each candidate sample answer paragraph and the ending position feature vector of each candidate sample answer paragraph comprises: The preliminary representation vector of each candidate sample answer paragraph is obtained by the following formula: ; in, A preliminary representation vector representing the candidate sample answer paragraph composed of the i-th word vector to the j-th word vector in the sample document; Indicates that the i-th word vector in the sample document is used as the starting position feature vector of the candidate sample answer paragraph, represents the j-th word vector in the sample document as the end position feature vector of the candidate sample answer paragraph; U represents the parameter matrix in the double affine transformation, W represents the parameter matrix of the linear transformation, b represents the bias term, and ⊕ represents the concatenation operation of the vectors.

5. The answer text extraction method according to claim 4, characterized in that: The step of obtaining the interaction segment representation vector after the interaction between the candidate sample answer segments through the convolutional neural network includes: The interactive text segment representation vector after the interaction between the candidate sample answer segments is obtained by the following formula: ; ; in, Represents the interactive paragraph representation vector after the interaction between the preliminary representation vector of the candidate sample answer paragraph composed of the i-th word vector to the j-th word vector in the sample document and the preliminary representation vectors of other candidate sample answer paragraphs; Conv2d represents a two-dimensional convolutional neural network; CNNBlock represents a convolution module; ×N represents stacking N layers of CNNBlock; the value of N is determined according to the length of the candidate sample answer paragraph and the size of the convolution kernel; Activation represents a nonlinear activation function, LayerNorm represents layer normalization of the feature dimension, and represents the preliminary representation vectors of all candidate sample answer paragraphs.

6. The answer text extraction method according to claim 5, characterized in that: The step of obtaining, by the second feedforward neural network, a sample question representation vector corresponding to the word vector of the word unit in the sample question and a global sample threshold vector corresponding to the word vector of the global sample word unit includes: The features of the word vectors of all word units in the sample questions are integrated through a maximum pooling operation to obtain a comprehensive sample word vector; Using the first sub-feedforward neural network, projecting the comprehensive sample word vector into the same representation space as the representation vector of the candidate sample answer paragraph to obtain the sample question representation vector; Using a second sub-feedforward neural network, projecting the word vector of the global sample word unit into the same representation space as the representation vector of the candidate sample answer paragraph to obtain the global sample threshold vector; The second feedforward neural network includes the first sub-feedforward neural network and the second sub-feedforward neural network.

7. An answer text extraction device, characterized in that: The answer text extraction device comprises: A receiving module, used to receive a user question and a document containing an answer corresponding to the user question; A word vector acquisition module, used to respectively acquire the word vector of the word unit in the user question, the word vector of the word unit in the document, and the word vector of the global word unit through a target context encoder; A first representation vector acquisition module, used to acquire the representation vectors of all candidate answer passages corresponding to the word vectors of the word elements in the document through a first target feedforward neural network, a target biaffine transformation and a target convolutional neural network; A second representation vector acquisition module is used to acquire, through a second target feedforward neural network, a question representation vector corresponding to the word vector of the word unit in the user question and a global threshold representation vector corresponding to the word vector of the global word unit; A similarity acquisition module, used to acquire a first similarity between the representation vector of each candidate answer text segment and the question representation vector, and a second similarity between the global threshold representation vector and the question representation vector; A target similarity acquisition module, used to acquire target similarity, where the target similarity is a similarity among the first similarities that is greater than the second similarity; A target representation vector acquisition module, used to acquire the representation vector of the target candidate answer text segment corresponding to the target similarity; An answer text output module, used to output the target candidate answer text corresponding to the representation vector of the target candidate answer text; The first representation vector acquisition module is specifically used for: Extracting, by means of the first target feedforward neural network, the starting position feature vector of each candidate answer paragraph and the ending position feature vector of each candidate answer paragraph from the word vector of the word unit in the document; According to the starting position feature vector of each candidate answer paragraph and the ending position feature vector of each candidate answer paragraph, a preliminary representation vector of each candidate answer paragraph is obtained through the target double affine transformation; According to the preliminary representation vectors of the candidate answer paragraphs, obtaining the interaction paragraph representation vectors after the interaction between the candidate answer paragraphs through the target convolutional neural network; The representation vector of each candidate answer paragraph is obtained according to the preliminary representation vector of each candidate answer paragraph and the corresponding interactive paragraph representation vector of each candidate answer paragraph.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the answer text extraction method described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the answer text extraction method described in any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the answer text extraction method described in any one of claims 1 to 6 are implemented.