Question answer generation method and device, equipment and storage medium

Abstract

The application relates to the technical field of natural language processing, and discloses a question-answer generation method and device, equipment and a storage medium, which are used for deeply mining semantic information of a question and improving the precision performance of a question-answer system. The method comprises the following steps: inputting a target text into a pre-trained text feature extraction network to output text features of the target text; inputting a question text corresponding to the target text into a pre-trained question feature extraction network to output question features of the question text; the question feature extraction network is trained based on a preset training sample set; the training sample set comprises a plurality of question sample pairs; a sample label of the question sample pair is used for indicating whether the question in the question sample pair has similar semantics; and the text features and the question features are input into an information fusion network to output an answer text corresponding to the question text. In addition, the application also relates to the blockchain technology, and data of the question-answer generation can be stored in a blockchain node.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, and more particularly to a method, apparatus, device, and storage medium for generating answers to questions. Background Technology

[0002] Question answering systems (QA) are an advanced form of information retrieval systems. Their design goal is to answer user questions posed in natural language accurately and concisely, making them a highly relevant topic in artificial intelligence and natural language processing. With the development of AI, QA tasks have significant application value in natural language processing. For example, they are increasingly used in various scenarios such as intelligent customer service in mobile applications and chatbots in outbound telephone calls. In machine reading comprehension scenarios, QA tasks extract information from text based on the question as the answer. This places higher demands on natural language models, requiring them to have a sufficient semantic understanding of the question.

[0003] Currently, the mainstream approach to generating answers in question-answering systems involves concatenating the question and text content as input to a pre-trained model. A pointer network is then connected to the language model to determine the start and end positions. However, existing technologies do not fully explore the semantic information of the question or fully understand the rich information within it. They simply concatenate and combine the question and text information. As a result, the language model cannot effectively utilize the information inherent in the question itself, making it prone to incorrect answers and leading to low accuracy in the question-answering system. Furthermore, the ability to recognize complex questions needs improvement. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for generating question answers, which is used to deeply mine the semantic information of questions, improve the understanding of questions, thereby improving the accuracy and performance of question-answering systems and ensuring the accuracy of generated question answers.

[0005] To achieve the above objectives, the first aspect of the present invention provides a method for generating a question answer, comprising: inputting target text into a pre-trained text feature extraction network and outputting text features of the target text; inputting question text corresponding to the target text into a pre-trained question feature extraction network and outputting question features of the question text; wherein the question feature extraction network is trained based on a preset training sample set; the training sample set includes multiple question sample pairs; the sample labels of the question sample pairs are used to indicate whether the questions in the question sample pairs have similar semantics; inputting the text features and question features into an information fusion network and outputting the answer text corresponding to the question text.

[0006] Optionally, in a first implementation of the first aspect of the present invention, the above-mentioned problem feature extraction network is trained in the following manner: determining target problem sample pairs from the training sample set; wherein, the sample labels of the target problem sample pairs include positive samples or negative samples; positive samples are used to indicate that the problems in the target problem sample pairs have similar semantics; negative samples are used to indicate that the problems in the target problem sample pairs do not have similar semantics; inputting the target problem sample pairs into the initial network, and outputting the problem features of each problem in the target problem pair; classifying the problems based on the problem features of each problem, and obtaining the classification result of each problem; calculating the loss value based on the classification result of each problem, the sample label, and the preset loss function; updating the parameters of the initial network based on the loss value until the initial network converges, and determining the converged initial network as the problem feature extraction network.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the step of classifying the problems based on the problem features of each problem to obtain the classification result of each problem includes: inputting the problem features of each problem into a similarity comparison network to obtain intermediate results; inputting the intermediate results into a preset fully connected layer to output the classification result of each problem.

[0008] Optionally, in a third implementation of the first aspect of the present invention, the step of inputting the question text corresponding to the target text into a pre-trained question feature extraction network and outputting the question features of the question text includes: inputting the question text corresponding to the target text into the pre-trained question feature extraction network, extracting initial features of the question text through the question feature extraction network, mapping the initial features to a specified feature space to obtain the question features of the question text; wherein, in the specified feature space, if the first question and the second question are semantically similar, the feature distance between the question features of the first question and the second question is lower than a first threshold; if the first question and the second question are semantically dissimilar, the feature distance between the question features of the first question and the second question is higher than a second threshold; the second threshold is greater than the first threshold.

[0009] Optionally, in a fourth implementation of the first aspect of the present invention, the step of inputting text features and question features into an information fusion network and outputting the answer text corresponding to the question text includes: concatenating the text features and question features to obtain concatenated features; inputting the concatenated features into the information fusion network and fusing the concatenated features through a fully connected layer network in the information fusion network to obtain fused features; identifying the features corresponding to the answer text in the fused features through a pointer network in the information fusion network; and outputting the answer text corresponding to the question text based on the features corresponding to the answer text.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, the step of identifying the features corresponding to the answer text in the fusion features through the pointer network in the information fusion network includes: identifying the starting feature and the ending feature in the fusion features through the pointer network in the information fusion network; and determining the starting feature, the ending feature, and the intermediate feature between the starting feature and the ending feature as the features corresponding to the answer text.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, the step of inputting the target text into a pre-trained text feature extraction network and outputting the text features of the target text includes: encoding the target text to obtain the encoded features of the target text; inputting the encoded features into a pre-trained language representation model, transforming the encoded features through the language representation model, and performing feature fusion processing on the transformed encoded features based on a self-attention mechanism to obtain the text features of the target text.

[0012] A second aspect of the present invention provides a question-answer generation apparatus, comprising a first input module for inputting target text into a pre-trained text feature extraction network and outputting text features of the target text; a second input module for inputting question text corresponding to the target text into a pre-trained question feature extraction network and outputting question features of the question text; wherein the question feature extraction network is trained based on a preset training sample set; the training sample set includes multiple question sample pairs; the sample labels of the question sample pairs are used to indicate whether the questions in the question sample pairs have similar semantics; and a third input module for inputting the text features and question features into an information fusion network and outputting answer text corresponding to the question text.

[0013] Optionally, in the first implementation of the second aspect of the present invention, the second input module is further configured to: determine target problem sample pairs from the training sample set; wherein the sample labels of the target problem sample pairs include positive samples or negative samples; the positive samples are used to indicate that the problems in the target problem sample pairs have similar semantics; the negative samples are used to indicate that the problems in the target problem sample pairs do not have similar semantics; input the target problem sample pairs into the initial network and output the problem features of each problem in the target problem pair; classify the problems based on the problem features of each problem to obtain the classification result of each problem; calculate the loss value based on the classification result of each problem, the sample label, and the preset loss function; update the parameters of the initial network based on the loss value until the initial network converges, and determine the converged initial network as the problem feature extraction network.

[0014] Optionally, in a second implementation of the second aspect of the present invention, the second input module is further configured to: input the problem features of each problem into a similarity comparison network to obtain intermediate results; input the intermediate results into a preset fully connected layer to output the classification results of each problem.

[0015] Optionally, in a third implementation of the second aspect of the present invention, the second input module is further configured to: input the question text corresponding to the target text into a pre-trained question feature extraction network, extract initial features of the question text through the question feature extraction network, and map the initial features to a specified feature space to obtain the question features of the question text; wherein, in the specified feature space, if the first question and the second question are semantically similar, the feature distance between the question features of the first question and the second question is lower than a first threshold; if the first question and the second question are semantically dissimilar, the feature distance between the question features of the first question and the second question is higher than a second threshold; the second threshold is greater than the first threshold.

[0016] Optionally, in a fourth implementation of the second aspect of the present invention, the third input module is specifically used for: concatenating text features and question features to obtain concatenated features; inputting the concatenated features into an information fusion network, and fusing the concatenated features through a fully connected layer network in the information fusion network to obtain fused features; identifying the features corresponding to the answer text in the fused features through a pointer network in the information fusion network; and outputting the answer text corresponding to the question text based on the features corresponding to the answer text.

[0017] Optionally, in a fifth implementation of the second aspect of the present invention, the third input module is further configured to: identify the starting feature and the ending feature in the fusion features through the pointer network in the information fusion network; and determine the starting feature, the ending feature, and the intermediate feature between the starting feature and the ending feature as the features corresponding to the answer text.

[0018] Optionally, in a sixth implementation of the second aspect of the present invention, the first input module is further configured to: encode the target text to obtain the encoded features of the target text; input the encoded features into a pre-trained language representation model, transform the encoded features through the language representation model, and perform feature fusion processing on the transformed encoded features based on a self-attention mechanism to obtain the text features of the target text.

[0019] A third aspect of the present invention provides a question-answer generation device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the question-answer generation device to execute the question-answer generation method described above.

[0020] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the above-described method for generating answers to questions.

[0021] In the technical solution provided by this invention, the target text is input into a pre-trained text feature extraction network, which outputs the text features of the target text; the question text corresponding to the target text is input into a pre-trained question feature extraction network, which outputs the question features of the question text; wherein, the question feature extraction network is trained based on a preset training sample set; the training sample set includes multiple question sample pairs; the sample labels of the question sample pairs are used to indicate whether the questions in the question sample pairs have similar semantics; the text features and question features are input into an information fusion network, which outputs the answer text corresponding to the question text. In this method, the target text and the question text corresponding to the target text are input into the text feature extraction network and the question feature extraction network, respectively, to obtain the text features of the target text and the question features of the question text. The text features and question features are then input into the information fusion network to obtain the answer text corresponding to the question text. This method can deeply mine the semantic information of the question, improve the understanding of the question, thereby improving the accuracy performance of the question-answering system and ensuring the accuracy of the generated question answers. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of one embodiment of the method for generating answers to questions in this invention.

[0023] Figure 2 This is a schematic diagram of another embodiment of the method for generating answers to questions in this invention;

[0024] Figure 3 This is a schematic diagram of one embodiment of the question answer generation device in this invention;

[0025] Figure 4 This is a schematic diagram of one embodiment of the device for generating answers to questions in this invention. Detailed Implementation

[0026] This invention provides a method, apparatus, device, and storage medium for generating question answers, which is used to deeply mine the semantic information of questions, improve the understanding of questions, thereby improving the accuracy and performance of question-answering systems and ensuring the accuracy of generated question answers.

[0027] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the method for generating answers to questions in this invention includes:

[0029] Step S101: Input the target text into the pre-trained text feature extraction network and output the text features of the target text.

[0030] Question-answering systems integrate technologies such as knowledge representation, information retrieval, and natural language processing. They enable users to ask questions in natural language, and based on the user's information query needs, the system automatically finds accurate answers from various data resources through question analysis. These data resources include a significant amount of text data, which is composed of a series of characters. After word segmentation, these characters form a set of words, which constitutes the raw data. Machine learning algorithms cannot directly use this raw data; therefore, it is necessary to transform it into numerical features that machine learning algorithms can recognize.

[0031] In this embodiment of the invention, the target text is input into a pre-trained text feature extraction network, which outputs the text features of the target text. Specifically, the text feature extraction network is one layer of the pre-trained language representation model BERT (Bidirectional Encoder Representations from Transformer). The target text is a word sequence representing a single text sentence or a pair of texts. For a given word, its input representation is obtained by summing three embeddings: token embeddings, position embeddings, and segmentation embeddings. Token embeddings are word vectors after word segmentation. Position embeddings encode the positional information of words into feature vectors, which is a crucial step in introducing word positional relationships into the pre-trained language representation model. Segmentation embeddings distinguish between two sentences; for a sentence pair input, the feature values ​​of the first sentence and the second sentence are different. After converting the target text into numerical features that can be recognized by machine learning algorithms, it is input into the pre-trained text feature extraction network, which extracts and outputs the text features of the target text.

[0032] In actual implementation, the target text is first encoded, that is, each character in the target text is converted into an ID code to obtain the encoded features of the target text. Then, the encoder of the Transformer in the pre-trained language representation model completes the transformation of the encoded vector. Based on the self-attention mechanism, the transformed encoded features are fused to obtain the text features of the target text. The self-attention mechanism is an improvement on the attention mechanism, which reduces the dependence on external information and is better at capturing the correlation within the data or features.

[0033] Compared with existing technologies, this step encodes the target text, transforms the encoded vector through an encoder, and then uses a self-attention mechanism to fuse features between characters, thereby extracting deep features of the target text.

[0034] Step S102: Input the question text corresponding to the target text into the pre-trained question feature extraction network, and output the question features of the question text; wherein, the question feature extraction network is trained based on a preset training sample set; the training sample set includes multiple question sample pairs; the sample labels of the question sample pairs are used to indicate whether the questions in the question sample pairs have similar semantics;

[0035] After obtaining the text features of the target text, the corresponding question text is further input into a pre-trained question feature extraction network. This question feature extraction network is also one layer of the pre-trained language representation model BERT, and is trained based on a preset training sample set. The training sample set includes multiple question sample pairs, each containing two questions. Each question sample pair carries a sample label to indicate whether the two questions in the question sample pair have similar semantics. The sample label includes positive and negative samples. If the sample label is positive, it indicates that the questions in the target question sample pair have similar semantics; if the sample label is negative, it indicates that the questions in the target question sample pair do not have similar semantics.

[0036] In actual implementation, the question text corresponding to the target text is input into a pre-trained question feature extraction network. This network extracts initial features from the question text and maps these initial features to a specified feature space. In this specified feature space, if the first question and the second question are semantically similar, the feature distance between the features of the first question and the second question is lower than a first threshold; conversely, if the first question and the second question are semantically dissimilar, the feature distance between the features of the first question and the second question is higher than a second threshold. The larger the feature distance, the smaller the similarity. The second threshold is greater than the first threshold.

[0037] Compared with existing technologies, this step can fully mine the problem text. By mapping the initial features of the problem text extracted by the problem feature extraction network to a specified feature space, the problem features of problem texts with similar expressions are closer in feature distance, while the problem features of problem texts with dissimilar expressions are further apart in feature distance, thereby effectively improving the understanding of the problem semantics.

[0038] Step S103: Input the text features and question features into the information fusion network and output the answer text corresponding to the question text.

[0039] In practical implementation, the aforementioned text features and question features are input into an information fusion network. This network fuses and concatenates the text features and question features. Specifically, the question features and text features are first concatenated, and then connected to a fully connected layer for interactive information fusion. Feature fusion can obtain the most differentiated information from multiple feature sets involved in the fusion, eliminating redundant information caused by the correlation between different feature sets. Finally, a pointer network is connected after the fully connected layer to output the answer text corresponding to the question text. The pointer network can identify the start and end features in the fused features. Based on this, the start and end fields in the answer text can be identified. Then, the start features, end features, and intermediate features between the start and end features are identified as the features corresponding to the answer text, thereby obtaining the answer text.

[0040] This step, by connecting to a pointer network, eliminates the need to rely on the decoder's output dictionary, thus solving the problem of the output dictionary depending on the length of the input sequence. Optionally, the pointer network can be a fully connected layer. Based on this, by connecting two fully connected layers, the classification judgment of whether each character is the beginning or the end can be made separately.

[0041] In this embodiment of the invention, the target text is input into a pre-trained text feature extraction network, which outputs the text features of the target text. The question text corresponding to the target text is input into a pre-trained question feature extraction network, which outputs the question features of the question text. The question feature extraction network is trained based on a preset training sample set, which includes multiple question sample pairs. The sample labels of the question sample pairs are used to indicate whether the questions in the question sample pairs have similar semantics. The text features and question features are input into an information fusion network, which outputs the answer text corresponding to the question text. In this method, the target text and the question text corresponding to the target text are input into the text feature extraction network and the question feature extraction network, respectively, to obtain the text features of the target text and the question features of the question text. The text features and question features are then input into the information fusion network to obtain the answer text corresponding to the question text. This method can deeply mine the semantic information of the question, improve the understanding of the question, thereby improving the accuracy performance of the question-answering system and ensuring the accuracy of the generated question answers.

[0042] Optionally, the above-mentioned problem feature extraction network is trained in the following manner: Target problem sample pairs are determined from the training sample set; wherein the sample labels of the target problem sample pairs include positive or negative samples; positive samples indicate that the problems in the target problem sample pairs have similar semantics; negative samples indicate that the problems in the target problem sample pairs do not have similar semantics; the target problem sample pairs are input into the initial network, and the problem features of each problem in the target problem pair are output; based on the problem features of each problem, the problems are classified to obtain the classification result for each problem; based on the classification result of each problem, the sample labels, and the preset loss function, the loss value is calculated; the parameters of the initial network are updated based on the loss value until the initial network converges, and the converged initial network is determined as the problem feature extraction network.

[0043] In practice, the feature extraction network for the above problem is trained in the following way:

[0044] First, target question sample pairs are determined from the training samples, which are used to train the question feature network. In one approach, a question sample pair consists of two questions. Each question sample pair has a sample label, which includes positive and negative samples. Positive samples indicate that the two questions in the target question sample pair have similar semantics, while negative samples indicate that the two questions in the target question sample pair do not have similar semantics. If the two questions in the sample question pair have similar semantics, it means that the two questions in the sample question pair express the same meaning. Conversely, if the two questions in the sample question pair do not have similar semantics, it means that the two questions in the sample question pair express different meanings.

[0045] Secondly, the target question sample pairs are input into the initial network, and the question features of each question in the target question pair are output. Specifically, the question text in the target question sample pairs is converted into numerical features that can be recognized by machine learning algorithms and then input into the initial network. The initial network is one layer of a pre-trained language representation model, which can extract the question features in the target question samples and then output the question features of each question in the target question pair.

[0046] Then, based on the characteristics of each problem, the problems are classified to obtain the classification results for each problem. Specifically, the problems are classified according to their characteristics. There are multiple options for the classification criteria. For example, problems can be classified according to different fields, such as education, healthcare, finance, shopping, etc. Alternatively, they can be classified according to different aspects of the same field, and so on.

[0047] Optionally, the problem features of each problem can be input into a similarity comparison network to obtain intermediate results; the intermediate results can then be input into a pre-defined fully connected layer to output the classification results for each problem.

[0048] Specifically, the problem features of each of the above problems are input into a similarity comparison network to obtain intermediate results. This similarity comparison network is used to compare the similarity of problem features in the above problem sample pairs. Specifically, it can be a fully connected layer. Then, the intermediate results are input into a preset fully connected layer. This fully connected layer can implement a binary classification model to judge whether the problems in the problem sample pairs have similar semantics. That is, it judges whether the problems in the problem sample pairs have similar semantics and outputs a classification model with only 0 and 1, and then outputs the classification result of each problem.

[0049] Then, based on the classification results, sample labels, and preset loss functions for each problem, the loss value is calculated. Specifically, the classification results for each problem clarify the domain and target aspect of each problem. The sample labels clarify whether the problem statements in the problem pairs are semantically similar. The preset loss function is used to calculate the degree of difference between the predicted value and the true value. There are various choices for the loss function, such as the Cross Entropy Loss function and the Contrastive Loss function. Based on the classification results, sample labels, and preset loss functions for each problem, the loss value can be calculated. This loss value is a relatively direct indicator to evaluate the fit of the feature extraction network on the training set. Within a certain range, the larger the loss value, the larger the difference between the predicted value and the true value; the smaller the loss value, the smaller the difference between the predicted value and the true value. When the loss value no longer decreases, the best fit is achieved.

[0050] Finally, the parameters of the initial network are updated based on the loss value until the initial network converges. The converged initial network is then designated as the problem feature extraction network. Specifically, the parameters of the initial network are updated and adjusted according to the aforementioned loss value until the loss value no longer decreases, achieving the best fit, i.e., the initial network converges. At this point, the predicted value output by the initial network is closest to the true value, with the smallest difference. This initial network is then designated as the problem feature network. Based on this, inputting the problem text corresponding to the target text into the pre-trained problem feature extraction network ensures the accuracy of the problem features in the output problem text.

[0051] Optionally, the question text corresponding to the target text is input into a pre-trained question feature extraction network. The question feature extraction network extracts the initial features of the question text and maps the initial features to a specified feature space to obtain the question features of the question text. In the specified feature space, if the first question and the second question are semantically similar, the feature distance between the question features of the first question and the second question is lower than a first threshold; if the first question and the second question are semantically dissimilar, the feature distance between the question features of the first question and the second question is higher than a second threshold; the second threshold is greater than the first threshold.

[0052] In actual implementation, the question text corresponding to the target text is input into a pre-trained question feature extraction network. The question feature extraction network extracts the initial features of the question text and maps these initial features to a specified feature space. In this specified feature space, if the first question and the second question are semantically similar, that is, they express the same meaning, then the feature distance between the question features of the first question and the second question is lower than a first threshold. The smaller the feature distance, the more similar the features are. That is, questions with similar semantics have similar question features. If the first question and the second question are not semantically similar, then the feature distance between the question features of the first question and the second question is higher than a second threshold. The larger the feature distance, the lower the similarity. The second threshold is greater than the first threshold.

[0053] This step makes the problem features of semantically similar problems more similar and the problem features of semantically dissimilar problems more dissimilar, thereby effectively improving the semantic understanding of the problems.

[0054] Optionally, the text features and question features are concatenated to obtain concatenated features; the concatenated features are input into an information fusion network, and the concatenated features are fused through a fully connected layer network in the information fusion network to obtain fused features; the pointer network in the information fusion network identifies the features corresponding to the answer text in the fused features; and the answer text corresponding to the question text is output based on the features corresponding to the answer text.

[0055] In practice, the text features and question features mentioned above are concatenated to obtain concatenated features, which are then input into an information fusion network. This information fusion network is also one layer in a pre-trained language representation model. The fully connected layer in the information fusion network can fuse the concatenated features. Feature fusion can obtain the most differentiated information from multiple feature sets involved in the fusion, and can eliminate redundant information caused by the correlation between different feature sets. Furthermore, by connecting a pointer network after the fully connected layer in the information fusion network, the pointer network can identify the features corresponding to the answer text in the fused features, and then output the answer text corresponding to the question answer text based on the features corresponding to the answer text.

[0056] Specifically, the starting and ending features in the fusion features are identified through the pointer network in the information fusion network; the starting features, ending features, and intermediate features between the starting and ending features are identified as the features corresponding to the answer text.

[0057] In practical implementation, the pointer network in the information fusion network can identify the starting and ending features among the aforementioned fusion features. Based on this, the starting and ending fields in the answer text can be identified, and the starting features, ending features, and intermediate features between the starting and ending features can be identified as the features corresponding to the answer text, thereby obtaining the answer text.

[0058] In this step, the answer text is obtained based on the pointer network in the information fusion network, which is based on the in-depth mining of the question text. This improves the understanding of the question, enhances the accuracy of the question-answering system, and ensures the accuracy of the generated question answer.

[0059] Optionally, the target text is encoded to obtain its encoded features; the encoded features are then input into a pre-trained language representation model, which transforms the encoded features and performs feature fusion processing on the transformed encoded features based on a self-attention mechanism to obtain the text features of the target text.

[0060] In actual implementation, the target text is first encoded, that is, each character in the target text is converted into an ID code to obtain the encoded features of the target text. Then, the encoder of the transformer Trsnsformer in the pre-trained language representation model completes the transformation of the encoded vector. Based on the self-attention mechanism, the transformed encoded features are fused to obtain the text features of the target text. The self-attention mechanism is an improvement on the attention mechanism, which reduces the dependence on external information and is better at capturing the correlation within the data or features.

[0061] Compared with existing technologies, this step encodes the target text, transforms the encoded vector through an encoder, and then uses a self-attention mechanism to fuse features between characters, thereby extracting deep features of the target text.

[0062] Another embodiment of the invention is provided below, which may be found in [reference needed]. Figure 2 .

[0063] Step S201: Encode the target text to obtain its encoded features; input the encoded features into a pre-trained language representation model, transform the encoded features through the language representation model, and perform feature fusion processing on the transformed encoded features based on a self-attention mechanism to obtain the text features of the target text.

[0064] In actual implementation, the target text is first encoded, that is, each character in the target text is converted into an ID code to obtain the encoded features of the target text. Then, the encoder of the Transformer in the pre-trained language representation model completes the transformation of the encoded vector. Based on the self-attention mechanism, the transformed encoded features are fused to obtain the text features of the target text. The self-attention mechanism is an improvement on the attention mechanism, which reduces the dependence on external information and is better at capturing the correlation within the data or features.

[0065] Step S202: Input the question text corresponding to the target text into a pre-trained question feature extraction network. Extract initial features from the question text using the question feature extraction network. Map the initial features to a specified feature space to obtain the question features of the question text. In the specified feature space, if the first question and the second question are semantically similar, the feature distance between the question features of the first question and the second question is lower than a first threshold. If the first question and the second question are semantically dissimilar, the feature distance between the question features of the first question and the second question is higher than a second threshold. The second threshold is greater than the first threshold.

[0066] In actual implementation, the question text corresponding to the target text is input into a pre-trained question feature extraction network. This network extracts initial features from the question text and maps these initial features to a specified feature space. In this specified feature space, if the first question and the second question are semantically similar, the feature distance between the features of the first question and the second question is lower than a first threshold; conversely, if the first question and the second question are semantically dissimilar, the feature distance between the features of the first question and the second question is higher than a second threshold. The larger the feature distance, the smaller the similarity. The second threshold is greater than the first threshold.

[0067] Step S203: The text features and question features are concatenated to obtain concatenated features; the concatenated features are input into the information fusion network, and the concatenated features are fused through the fully connected layer network in the information fusion network to obtain fused features;

[0068] In actual implementation, the above text features and question features are input into the information fusion network. This information fusion network can fuse and concatenate the text features and question features. Specifically, the question features and text features are concatenated first, and then connected to a fully connected layer for information interaction and fusion. Feature fusion can obtain the most differentiated information from multiple feature sets involved in the fusion, and can eliminate redundant information caused by the correlation between different feature sets.

[0069] Step S204: Identify the starting and ending features in the fusion features through the pointer network in the information fusion network; determine the starting features, ending features, and intermediate features between the starting and ending features as the features corresponding to the answer text;

[0070] In practical implementation, the pointer network in the information fusion network can identify the starting and ending features among the aforementioned fusion features. Based on this, the starting and ending fields in the answer text can be identified, and the starting features, ending features, and intermediate features between the starting and ending features can be determined as the features corresponding to the answer text.

[0071] Step S205: Output the answer text corresponding to the question text based on the features corresponding to the answer text.

[0072] In this embodiment of the invention, the target text and the corresponding question text are input into a text feature extraction network and a question feature extraction network, respectively, to obtain the text features of the target text and the question features of the question text. These text features and question features are then input into an information fusion network to obtain the answer text corresponding to the question text. This method allows for in-depth mining of the semantic information of the question, improving the understanding of the question and thus enhancing the accuracy and performance of the question-answering system, ensuring the accuracy of the generated question answers.

[0073] The method for generating answers to questions in the embodiments of the present invention has been described above. The apparatus for generating answers to questions in the embodiments of the present invention will be described below. Please refer to [link / reference]. Figure 3 One embodiment of the question answer generation device in this invention includes:

[0074] The first input module 301 is used to input the target text into a pre-trained text feature extraction network and output the text features of the target text.

[0075] The second input module 302 is used to input the question text corresponding to the target text into a pre-trained question feature extraction network and output the question features of the question text; wherein, the question feature extraction network is trained based on a preset training sample set; the training sample set includes multiple question sample pairs; the sample labels of the question sample pairs are used to indicate whether the questions in the question sample pairs have similar semantics;

[0076] The third input module 303 is used to input text features and question features into the information fusion network and output the answer text corresponding to the question text.

[0077] In this embodiment of the invention, the target text is input into a pre-trained text feature extraction network, which outputs the text features of the target text. The question text corresponding to the target text is input into a pre-trained question feature extraction network, which outputs the question features of the question text. The question feature extraction network is trained based on a preset training sample set, which includes multiple question sample pairs. The sample labels of the question sample pairs are used to indicate whether the questions in the question sample pairs have similar semantics. The text features and question features are input into an information fusion network, which outputs the answer text corresponding to the question text. In this method, the target text and the question text corresponding to the target text are input into the text feature extraction network and the question feature extraction network, respectively, to obtain the text features of the target text and the question features of the question text. The text features and question features are then input into the information fusion network to obtain the answer text corresponding to the question text. This method can deeply mine the semantic information of the question, improve the understanding of the question, thereby improving the accuracy performance of the question-answering system and ensuring the accuracy of the generated question answers.

[0078] Optionally, the second input module 302 described above can also be specifically used for:

[0079] Target question sample pairs are determined from the training sample set mentioned above; wherein the sample labels of the target question sample pairs include positive samples or negative samples; the positive samples are used to indicate that the questions in the target question sample pairs have similar semantics; the negative samples are used to indicate that the questions in the target question sample pairs do not have similar semantics; the target question sample pairs are input into the initial network, and the problem features of each question in the target question pair are output; based on the problem features of each question, the questions are classified to obtain the classification result of each question; based on the classification result of each question, the sample label, and the preset loss function, the loss value is calculated; the parameters of the initial network are updated based on the loss value until the initial network converges, and the converged initial network is determined as the problem feature extraction network mentioned above.

[0080] Optionally, the second input module 302 described above may also be specifically used for:

[0081] The problem features of each problem are input into a similarity comparison network to obtain intermediate results; the intermediate results are then input into a pre-defined fully connected layer to output the classification results for each problem.

[0082] Optionally, the second input module 302 described above may also be specifically used for:

[0083] The question text corresponding to the target text is input into a pre-trained question feature extraction network. The network extracts initial features of the question text and maps these initial features to a specified feature space to obtain the question features of the question text. In the specified feature space, if the first question and the second question are semantically similar, the feature distance between the question features of the first question and the second question is lower than a first threshold; if the first question and the second question are semantically dissimilar, the feature distance between the question features of the first question and the second question is higher than a second threshold; the second threshold is greater than the first threshold.

[0084] Optionally, the third input module 303 described above may also be specifically used for:

[0085] The text features and question features are concatenated to obtain concatenated features. These concatenated features are then input into an information fusion network, where they are fused using a fully connected layer to obtain fused features. A pointer network within the information fusion network identifies the features corresponding to the answer text within the fused features. Based on these features, the answer text corresponding to the question text is output.

[0086] Optionally, the third input module 303 described above may also be specifically used for:

[0087] By using the pointer network in the information fusion network, the starting and ending features in the fusion features are identified; the starting features, ending features, and intermediate features between the starting and ending features are determined as the features corresponding to the answer text.

[0088] Optionally, the first input module 301 described above may also be specifically used for:

[0089] The target text is encoded to obtain its encoded features. These encoded features are then input into a pre-trained language representation model, which transforms the encoded features. Finally, based on a self-attention mechanism, the transformed encoded features are fused to obtain the textual features of the target text.

[0090] The above figure describes in detail the question answer generation device in the embodiment of the present invention. The following describes in detail the question answer generation device in the embodiment of the present invention from the perspective of hardware processing.

[0091] Figure 4This is a schematic diagram of a question-answer generation device 400 provided in an embodiment of the present invention. The question-answer generation device 400 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 410 (e.g., one or more processors) and a memory 420, and one or more storage media 430 (e.g., one or more mass storage devices) for storing application programs 433 or data 432. The memory 420 and storage media 430 can be temporary or persistent storage. The program stored in the storage media 430 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the question-answer generation device 400. Furthermore, the processor 410 may be configured to communicate with the storage media 430 and execute the series of instruction operations in the storage media 430 on the question-answer generation device 400.

[0092] The device 400 for generating the answer to the question may also include one or more power supplies 440, one or more wired or wireless network interfaces 450, one or more input / output interfaces 460, and / or one or more operating systems 431, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 4 The illustrated device structure for generating answers to questions does not constitute a limitation on the device for generating answers to questions. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0093] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of a method for generating a question answer.

[0094] The present invention also provides a question answer generation device, which includes a memory and a processor. The memory stores instructions, and when the instructions are executed by the processor, the processor performs the steps of the question answer generation method in the above embodiments.

[0095] Furthermore, the computer-readable storage medium may primarily include a stored program area and a stored data area, wherein the stored program area may store the operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of blockchain nodes, etc.

[0096] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0098] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating answers to a question, characterized in that, The methods for generating the answers to the questions include: The target text is input into a pre-trained text feature extraction network, which outputs the text features of the target text. The question text corresponding to the target text is input into a pre-trained question feature extraction network, which outputs the question features of the question text. The question feature extraction network is trained based on a preset training sample set, which includes multiple question sample pairs. The sample labels of the question sample pairs are used to indicate whether the questions in the question sample pairs have similar semantics. The text features and the question features are input into an information fusion network, and the answer text corresponding to the question text is output. The problem feature extraction network was trained in the following manner: Target question sample pairs are determined from the training sample set. The sample labels of the target question sample pairs include positive samples or negative samples. The positive samples are used to indicate that the questions in the target question sample pairs have similar semantics, and the negative samples are used to indicate that the questions in the target question sample pairs do not have similar semantics. The target problem sample pairs are input into the initial network, and the problem features of each problem in the target problem pair are output. The problems are classified based on their characteristics to obtain a classification result for each problem; Based on the classification result of each problem, the sample label, and the preset loss function, the loss value is calculated. The parameters of the initial network are updated based on the loss value until the initial network converges. The converged initial network is then determined as the problem feature extraction network. The step of inputting the question text corresponding to the target text into a pre-trained question feature extraction network and outputting the question features of the question text includes: inputting the question text corresponding to the target text into a pre-trained question feature extraction network, extracting initial features of the question text through the question feature extraction network, and mapping the initial features to a specified feature space to obtain the question features of the question text. In the specified feature space, if the first question and the second question are semantically similar, the feature distance between the question features of the first question and the second question is lower than a first threshold; if the first question and the second question are semantically dissimilar, the feature distance between the question features of the first question and the second question is higher than a second threshold, and the second threshold is greater than the first threshold. The step of inputting the text features and the question features into an information fusion network and outputting the answer text corresponding to the question text includes: concatenating the text features and the question features to obtain concatenated features; inputting the concatenated features into the information fusion network; fusing the concatenated features through a fully connected layer network in the information fusion network to obtain fused features; identifying the features corresponding to the answer text in the fused features through a pointer network in the information fusion network; and outputting the answer text corresponding to the question text based on the features corresponding to the answer text.

2. The method for generating answers to questions according to claim 1, characterized in that, The step of classifying the problems based on their characteristics and obtaining classification results for each problem includes: The problem features of each problem are input into a similarity comparison network to obtain intermediate results; The intermediate results are input into a preset fully connected layer, which outputs the classification results for each question.

3. The method for generating answers to questions according to claim 1, characterized in that, The step of identifying the features corresponding to the answer text in the fused features through the pointer network in the information fusion network includes: The starting and ending features in the fused features are identified through the pointer network in the information fusion network. The starting feature, the ending feature, and the intermediate feature between the starting feature and the ending feature are determined as the features corresponding to the answer text.

4. The method for generating answers to questions according to claim 1, characterized in that, The steps of inputting the target text into a pre-trained text feature extraction network and outputting the text features of the target text include: The target text is encoded to obtain its encoded features; The encoded features are input into a pre-trained language representation model, which transforms the encoded features. The transformed encoded features are then fused using a self-attention mechanism to obtain the text features of the target text.

5. A device for generating answers to questions, used to implement the method for generating answers to questions as described in any one of claims 1-4, characterized in that, The device for generating the answer to the question includes: The first input module is used to input the target text into a pre-trained text feature extraction network and output the text features of the target text. The second input module is used to input the question text corresponding to the target text into a pre-trained question feature extraction network and output the question features of the question text; wherein, the question feature extraction network is trained based on a preset training sample set; the training sample set includes multiple question sample pairs; the sample labels of the question sample pairs are used to indicate whether the questions in the question sample pairs have similar semantics; The third input module is used to input the text features and the question features into the information fusion network and output the answer text corresponding to the question text.

6. A device for generating answers to questions, characterized in that, The device for generating the answer to the question includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the question answer generation device to perform the question answer generation method as described in any one of claims 1-4.

7. A computer-readable storage medium having instructions stored thereon, characterized in that, When the instruction is executed by the processor, it implements the method for generating the answer to the question as described in any one of claims 1-4.

Images