A question and answer method, apparatus, device and medium

Abstract

The application discloses a question and answer method and device, equipment and medium, and relates to the technical field of computers, which comprises the following steps: cutting a target question and answer text to obtain each target word, and vectorizing the target word to obtain each target vector; generating a question text matrix based on the target vector through a question encoder in a target question and answer model; calculating the target similarity between the target question text and each reference text based on the question text matrix and each reference text matrix through a similarity calculation module in the target question and answer model, and selecting the target number of reference texts with the highest target similarity; the reference text matrix is a matrix constructed by a reference text encoder in the target question and answer model based on the reference text; each reference text is a standard question text and a reference answer text, or a reference answer text; and generating a target answer text based on the target number of reference texts through an answer generator. The application can improve the accuracy and generality of question and answer, and speed up the efficiency of question and answer.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a question-and-answer method, apparatus, device, and medium. Background Technology

[0002] Currently, in question-answering tasks across various fields, people mainly pursue two types of effects: one is accuracy, and the other is the generalization of question-answering, that is, when a user inputs a question that is semantically similar but expressed differently in a real-world scenario, can the question-answering system provide an answer based on similar questions and answers in historical data?

[0003] Traditional question-and-answer matching methods mostly involve recording question-and-answer pair data and calculating coverage using regular expression matching, determining matching degree using discriminant classification models, and determining similarity using clustering models. These traditional methods cannot avoid drawbacks such as low efficiency, unstable results due to randomness, and the need for a large amount of manual intervention to develop answer frameworks.

[0004] In summary, improving the accuracy and generalization of question-and-answer systems, and accelerating their efficiency, are pressing issues that need to be addressed. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a question-and-answer method, apparatus, device, and medium that can improve the accuracy and generalization of question-and-answer, and accelerate the efficiency of question-and-answer. The specific solution is as follows:

[0006] Firstly, this application discloses a question-and-answer method, including:

[0007] The target question text is obtained, and the target question and answer text is segmented into words to obtain target words. Then, the target words are vectorized to obtain target vectors.

[0008] The question encoder in the target question answering model generates a question text matrix based on each target vector;

[0009] The similarity calculation module in the target question-answering model calculates the target similarity between the target question text and each reference text based on the question text matrix and each reference text matrix. The reference texts are then sorted from highest to lowest similarity, and the top-ranked reference texts are selected. The reference text matrix is ​​a matrix pre-constructed by the reference text encoder in the target question-answering model based on reference texts in a preset reference text library. Each reference text consists of a standard question text and a reference answer text, or the reference answer text.

[0010] The answer generator generates target answer text corresponding to the target question text based on the target number of reference texts.

[0011] Optionally, before obtaining the target question text, the process further includes:

[0012] During the deployment phase of the target question-answering model, each reference text in the preset reference text library is segmented to obtain reference words, and each reference word is vectorized to obtain reference vectors.

[0013] The reference text encoder generates a reference text matrix corresponding to each reference text based on the reference vectors corresponding to each reference text in the preset reference text library.

[0014] Optionally, the reference text matrix and the question text matrix are two-dimensional matrices; wherein, the first dimension represents the position information of the corresponding word segmentation, and the second dimension represents the word segmentation feature information of the corresponding word segmentation.

[0015] Optionally, the step of calculating the target similarity between the target question text and each reference text based on the question text matrix and each reference text matrix through the similarity calculation module in the target question answering model includes:

[0016] The dot product of the question text matrix and the transposed reference text matrix is ​​calculated by the similarity calculation module in the target question answering model to obtain a first similarity matrix representing the similarity between each target word segment and each reference word segment;

[0017] The second similarity matrix is ​​constructed by selecting the maximum similarity corresponding to each target word in the first similarity matrix;

[0018] The summation operation is performed on all similarities in the second similarity matrix to obtain the total similarity;

[0019] The maximum similarity in the second similarity matrix is ​​selected as the key similarity.

[0020] The average similarity is obtained by averaging all similarities in the second similarity matrix.

[0021] The target similarity between the target question text and the reference text corresponding to the reference text matrix is ​​determined based on the sum similarity, key similarity, and average similarity, so as to calculate the target similarity between the target question text and each of the reference texts.

[0022] Optionally, before obtaining the target question text, the process further includes:

[0023] Obtain the text of the question to be trained, the text of the correct answer to be trained, and the text of the incorrect answer to be trained;

[0024] The first combination of the question text to be trained and the correct answer text to be trained is used as the first positive sample for calculating the similarity between the question text to be trained and the correct answer text to be trained, and the second combination of the question text to be trained and the first combination is used as the second positive sample for calculating the similarity between the question text to be trained and the first combination.

[0025] The third combination of the question text to be trained and the incorrect answer text to be trained is used as the first negative sample for calculating the similarity between the question text to be trained and the incorrect answer text to be trained, and the fourth combination of the question text to be trained and the third combination is used as the second negative sample for calculating the similarity between the question text to be trained and the third combination.

[0026] The text of the question to be trained, the text of the correct answer to be trained, and the text of the incorrect answer to be trained are processed by word segmentation to obtain several words to be trained;

[0027] The target question answering model is obtained by training an initial question answering model based on the word segment to be trained, and by training an initial question answering model according to the first positive sample, the second positive sample, the first negative sample, and the second negative sample.

[0028] Optionally, the indexer in the initial question-answering model includes a pre-training dual-tower model and the pre-training similarity calculation module; wherein, the dual-tower model is a model based on a single-layer encoding layer in the converter model; the question encoder and the reference text encoder in the target question-answering model are obtained by splitting the post-training dual-tower model.

[0029] Optionally, the step of calculating the target similarity between the target question text and each reference text based on the question text matrix and each reference text matrix through the similarity calculation module in the target question answering model includes:

[0030] If the reference text matrix is ​​stored in several graphics processors, the similarity calculation module calculates the target similarity between the target question text and each of the reference texts based on the question text matrix and the reference text matrices stored in the graphics processor.

[0031] Accordingly, the step of sorting the reference texts according to the target similarity from high to low, and then selecting the top-ranked target number of reference texts, includes:

[0032] The reference texts stored in each graphics processor are sorted in descending order of target similarity, and the number of reference texts with the highest target similarity corresponding to each graphics processor are selected.

[0033] Using any one of the plurality of graphics processors, all the selected reference texts are reordered according to the target similarity from high to low, and the number of reference texts with the highest ranking among the reordered reference texts are selected.

[0034] Secondly, this application discloses a question-and-answer device, comprising:

[0035] The vector generation module is used to acquire the target question text, perform word segmentation on the target question-and-answer text to obtain target words, and then vectorize each target word to obtain a target vector.

[0036] The matrix generation module is used to generate a question text matrix based on each target vector using the question encoder in the target question answering model.

[0037] The reference text selection module is used to calculate the target similarity between the target question text and each reference text based on the question text matrix and each reference text matrix using the similarity calculation module in the target question answering model. The reference texts are then sorted according to their target similarity from high to low, and the top-ranked reference texts are selected. The reference text matrix is ​​a matrix pre-constructed by the reference text encoder in the target question answering model based on reference texts in a preset reference text library. Each reference text is either a standard question text and a reference answer text, or the reference answer text.

[0038] The answer generation module is used to generate target answer text corresponding to the target question text based on the target number of reference texts through an answer generator.

[0039] Thirdly, this application discloses an electronic device, including:

[0040] Memory, used to store computer programs;

[0041] A processor for executing the computer program to implement the aforementioned question-and-answer method.

[0042] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned question-and-answer method.

[0043] As can be seen, this application obtains the target question text, performs word segmentation on the target question-answer text to obtain target words, and then vectorizes each target word to obtain target vectors; generates a question text matrix based on each target vector through the question encoder in the target question-answer model; calculates the target similarity between the target question text and each reference text based on the question text matrix and each reference text matrix through the similarity calculation module in the target question-answer model, and sorts them according to the target similarity from high to low, and then selects the target number of reference texts with the highest sorting; the reference text matrix is ​​a matrix pre-constructed by the reference text encoder in the target question-answer model based on the reference texts in a preset reference text library; each reference text is a standard question text and a reference answer text, or the reference answer text; and generates the target answer text corresponding to the target question text through the answer generator based on the target number of reference texts. Therefore, this application generates target words instead of target characters, which can more precisely represent the meaning of the target question text, improve accuracy, and the number of words is less than the number of characters, which can improve the processing speed. When the reference text mentioned in this application is the standard question text and the reference answer text, text similarity is emphasized. When the reference text is the reference answer text, semantic similarity is emphasized. Considering both types of similarity improves the generalization and accuracy of question answering. This application selects the top-ranked target number of reference texts instead of setting a preset threshold, which reduces manual intervention and the influence of human factors, and can improve accuracy. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0045] Figure 1 This is a flowchart of a question-and-answer method disclosed in this application;

[0046] Figure 2 This is a schematic diagram of a reference text matrix generation method disclosed in this application;

[0047] Figure 3 This is a schematic diagram of target similarity calculation disclosed in this application;

[0048] Figure 4 This is a flowchart of a specific question-and-answer method disclosed in this application;

[0049] Figure 5 This is a schematic diagram of a model training method disclosed in this application;

[0050] Figure 6 This is a flowchart of a specific question-and-answer method disclosed in this application;

[0051] Figure 7 This is a schematic diagram of a method for calculating the hierarchical similarity of a graphics processor disclosed in this application;

[0052] Figure 8 This is a schematic diagram of a question-answering method based on graphics processor-based hierarchical similarity calculation disclosed in this application;

[0053] Figure 9 This is a schematic diagram of the structure of a question-and-answer device disclosed in this application;

[0054] Figure 10 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] Currently, in question-answering tasks across various fields, people mainly pursue two types of effects: one is accuracy, and the other is the generalization of question-answering, that is, when a user inputs a question that is semantically similar but expressed differently in a real-world scenario, can the question-answering system provide an answer based on similar questions and answers in historical data?

[0057] Traditional question-and-answer matching methods mostly involve recording question-and-answer pair data and calculating coverage using regular expression matching, determining matching degree using discriminant classification models, and determining similarity using clustering models. These traditional methods cannot avoid drawbacks such as low efficiency, unstable results due to randomness, and the need for a large amount of manual intervention to develop answer frameworks.

[0058] Therefore, this application proposes a question-and-answer scheme that can improve the accuracy and generalization of question-and-answer, and accelerate the efficiency of question-and-answer.

[0059] This application discloses a question-and-answer method, see [link to relevant documentation]. Figure 1 As shown, the method includes:

[0060] Step S11: Obtain the target question text, and perform word segmentation on the target question and answer text to obtain each target word segment. Then, vectorize each target word segment to obtain each target vector.

[0061] In this embodiment, the Jieba word segmentation tool is used to segment the target question text; each target word segment is vectorized, that is, embedded; Embedding refers to the process of encoding each input text into word vectors after segmentation.

[0062] It should be noted that Jieba is a commonly used Chinese word segmentation tool. Before embedding the corpus, the text needs to be segmented by character or word. Word segmentation is used instead of character segmentation because word segmentation can more precisely represent the meaning of the target text. Therefore, word segmentation can improve the accuracy of model recognition and also reduce the total number of target vectors.

[0063] In this embodiment, the preset lexicon stores all the word segments used in the process of training the initial question-answering model to obtain the target question-answering model. When there are unknown word segments in the target question text that are not in the preset lexicon, the known word segments are first vectorized, and then the vectors corresponding to the unknown word segments are labeled as [unk] unknown markers or [pad] padding markers for subsequent padding.

[0064] In this embodiment, before obtaining the target question text, the method further includes: during the deployment phase of the target question-answering model, performing word segmentation on each reference text in the preset reference text library to obtain reference words, and vectorizing each reference word to obtain reference vectors; and generating the reference text matrix corresponding to each reference text based on the reference vectors corresponding to each reference text in the preset reference text library through the reference text encoder.

[0065] See Figure 2 The diagram shows a schematic of a reference text matrix generation method. Each reference text in the preset reference text library is segmented and vectorized, and then processed by position encoding and reference text encoder to obtain a reference text matrix. All reference text matrices can be stored in the reference text matrix library.

[0066] Step S12: Generate a question text matrix based on each target vector using the question encoder in the target question answering model.

[0067] In this embodiment, the reference text matrix and the question text matrix are two-dimensional matrices. The first dimension represents the positional information of the corresponding word segmentation, and the second dimension represents the segmentation feature information of the corresponding word segmentation. Specifically, the two-dimensional matrix adds a dimension of text feature information. Therefore, the positional information of a 512-bit word or bag of tokens is preserved to the greatest extent in the first dimension, while the feature representation of a 128-bit word or bag of tokens is represented in the second dimension. This method significantly improves the accuracy of text similarity calculation due to the increased dimensional information, and has been verified to have a hit rate approximately 10% higher than the one-dimensional matrix similarity algorithm.

[0068] In this embodiment, since the model regenerates its vocabulary based on the input corpus during training, it inevitably encounters uncommon words in user questions that are not included in the vocabulary, resulting in [unk] tokens (unknown target words) in the sequence. To address this issue, an automated empty word (uncommon word) imputation mechanism is designed into the model architecture. When the input question is embedded by the embedding module, if there are [unk] unknown tokens or [pad] imputation tokens in the sequence, the model automatically calculates the mean in a one-dimensional direction based on the feature representations of other words or word bags in the preceding and following context (i.e., calculates the mean of known feature representations in each of the 128 feature dimensions) and imputs them to all [unk] token positions. Optionally, the same operation is performed on [pad] imputation positions, i.e., the mean is calculated based on other values ​​in the question text matrix for imputation. The advantage of this approach is that it increases the information density of the question text matrix and the reference text matrix within limited input information, improving the accuracy of similarity calculation. Of course, we do not enable this mode during the training phase. Firstly, because our vocabulary is generated based on the training corpus, there are no uncommon words like "[unk]". Secondly, during training, we need to pursue the purity of the input information rather than the information density, otherwise the training data will be distorted and the effect will not be good in actual applications. Therefore, the "[pad]" markers will not be filled in during training.

[0069] Step S13: Using the similarity calculation module in the target question-answering model, calculate the target similarity between the target question text and each reference text based on the question text matrix and each reference text matrix, and sort them according to the target similarity from high to low, then select the target number of reference texts with the highest sorting; the reference text matrix is ​​a matrix pre-constructed by the reference text encoder in the target question-answering model based on the reference texts in the preset reference text library; each reference text is a standard question text and a reference answer text, or the reference answer text.

[0070] In this embodiment, before obtaining the target question text, the method further includes: during the deployment phase of the target question-answering model, segmenting each reference text in the preset reference text library to obtain reference words, and vectorizing each reference word to obtain reference vectors; and generating a reference text matrix corresponding to each reference text based on the reference vectors corresponding to each reference text in the preset reference text library through the reference text encoder. It should be noted that the process of generating the reference text matrix is ​​consistent with the process of generating the question text matrix.

[0071] In this embodiment, the reference text encoder in the target question answering model pre-builds the reference text matrix before acquiring the question text file, rather than during the processing of the target question text. This reduces the complexity of the target question answering system and improves its speed.

[0072] In this embodiment, the step of calculating the target similarity between the target question text and each reference text based on the question text matrix and each reference text matrix using the similarity calculation module in the target question-answering model includes: calculating the dot product of the question text matrix and the transposed reference text matrix using the similarity calculation module in the target question-answering model to obtain a first similarity matrix representing the similarity between each target word and each reference word; selecting the maximum similarity corresponding to each target word in the first similarity matrix to form a second similarity matrix; performing a summation operation on all similarities in the second similarity matrix to obtain a total similarity; selecting the maximum similarity in the second similarity matrix as the key similarity; performing an average operation on all similarities in the second similarity matrix to obtain an average similarity; determining the target similarity between the target question text and the reference texts corresponding to the reference text matrix based on the total similarity, key similarity, and average similarity, so as to calculate the target similarity between the target question text and each reference text.

[0073] It should be noted that before calculating the target similarity between the target question text and each reference text based on the question text matrix and each reference text matrix using the similarity calculation module in the target question-answering model, a 1D-Layernorm operation (layer normalization with dimension 1) needs to be performed on the second dimension (128 bits) of the question text matrix and the reference text matrix. This is to reduce the misleading effect of meaningless and noisy word repetition on the overall similarity, thereby playing a noise reduction role. The beneficial effects of the 1D-Layernorm operation include: 1. Making the model more likely to converge and avoiding gradient explosion; 2. Reducing the influence of noisy words. For example, some common non-keywords may appear multiple times in the reference text. If normalization is not performed, such a large number of repeated high scores will mislead the one-dimensional similarity matrix calculated in the second step, diluting the proportion of effective word similarity in the MixSim similarity. Therefore, adding a layer normalization operation with dimension 1 between the first and second steps of the matrix similarity calculation can ensure the concentration of effective information.

[0074] It should be noted that the first similarity matrix is ​​a two-dimensional matrix, obtained by normalizing the dot product of the question text matrix (size n×m, i.e., the number of tokens (number of word segments in the question text) × feature dimension) and the transposed reference text matrix (size m×n). The size of the first similarity matrix is ​​n×n (i.e., the number of tokens (number of word segments in the question text) × the number of tokens (number of word segments in the reference text)). It should be noted that the first similarity matrix represents the pairwise similarity relationships between each word or word group in the question text and the reference text, possessing a higher information density and a more explicit and intuitive computational logic, making model training easier to converge and question-answer matching more accurate.

[0075] It should be noted that the second similarity matrix is ​​a one-dimensional matrix of size n×1 obtained by performing a maximum value operation on the second dimension (128 bits) of the first similarity matrix of n×n (i.e., finding the most similar word or word group in the question matrix to all words or word groups in the text matrix and returning its score) (i.e., selecting the maximum similarity corresponding to each target word in the first similarity matrix).

[0076] It should be noted that the summation operation of all similarities in the second similarity matrix to obtain the total similarity is specifically to perform a summation operation on the obtained one-dimensional matrix to obtain the sum of the highest pairwise similarities of words or word packs, which is the total similarity (MaxSum).

[0077] It should be noted that the maximum similarity in the second similarity matrix is ​​selected as the key similarity. Specifically, the maximum similarity value in the one-dimensional matrix is ​​taken, which is the similarity score of the most similar pair between the question matrix and the text matrix, and is the key similarity (MaxMax).

[0078] It should be noted that the mean similarity is obtained by averaging all similarities in the second similarity matrix. Specifically, the mean is applied to the one-dimensional matrix, which represents the average score of the similarity between all words or word packs between the question matrix and the text matrix. This is the mean similarity (MaxMean).

[0079] It should be noted that the target similarity uses three similarity algorithms—MaxSum, MaxMax, and MaxMean—to calculate the similarity between the question text matrix and the reference text matrix, forming the MixSim similarity index to achieve higher accuracy.

[0080] It should be noted that the determination of the target similarity between the target question text and the reference text corresponding to the reference text matrix based on the total similarity, key similarity and average similarity can be based on the weights set for the three similarities, and the determination of the target similarity between the target question text and the reference text corresponding to the reference text matrix based on the total similarity, key similarity and average similarity.

[0081] In one specific embodiment, see Figure 3 The diagram illustrates a target similarity calculation. It shows a two-dimensional matrix of a question text (1, 512, 128) and n transposed two-dimensional matrices of reference texts (n, 128, 512). The first similarity matrix (512, 512) is obtained by multiplying the question text matrix with one of the reference text matrices. A layernorm normalization operation of dimension 1 is then performed on the first similarity matrix to obtain the second similarity matrix. Based on the second similarity matrix, the summation similarity, key similarity, and average similarity are calculated. Finally, the target similarity is obtained by calculating weights through a fully connected layer based on the summation similarity, key similarity, and average similarity.

[0082] It should be noted that this matrix multiplication method for calculating similarity is several times faster and requires fewer resources compared to the traditional discriminative model, which requires a complete algorithmic process to calculate the problem and each reference text.

[0083] In this embodiment, each reference text is either a standard question text and a reference answer text, or the reference answer text. This is because using standard question text and reference answer text as reference text allows for sufficient consideration of text similarity, while using reference answer text as reference text fully considers semantic similarity. Considering both types of similarity makes the result more accurate. Furthermore, the number of reference texts that are standard question texts and reference answer texts, and the number of reference texts that are reference answer texts, can be adjusted according to specific requirements (whether text similarity or semantic similarity is emphasized). If text similarity is emphasized, the number of reference texts that are standard question texts and reference answer texts is increased; if semantic similarity is emphasized, the number of reference texts that are reference answer texts is increased.

[0084] Step S14: Generate the target answer text corresponding to the target question text based on the target number of reference texts using the answer generator.

[0085] In this embodiment, the answer generator's specific function is to generate answers based on the top-n relevant reference texts and questions indexed by the retriever, using a GPT (Generative Pre-Trained Transformer) language model, referencing the given text, and organizing the language accordingly. The answer generator can be a chatglm model. Using chatglm as the answer generator makes the algorithm's expression closer to human language habits, and the temperature can be adjusted to determine whether the model generates more creative and random answers. Of course, stability and randomness need to be balanced; generally, setting it to the most rigorous generation mode is sufficient. It should be noted that the chatglm model's pre-training corpus contains more Chinese text than other open-source GPT models from both domestic and international sources, giving it better Chinese language organization capabilities.

[0086] In this embodiment, the step of inputting the target question text and the target number of reference files into the answer generator to obtain the target answer text can be replaced by the following steps: training a text summarization model, and then using the text summarization model to extract the key information of the target number of reference files to obtain the target answer text; however, the expression of the target answer text obtained by using the text summarization model is not as close to human expression as the generative language model, and the effect may not meet expectations.

[0087] In this embodiment, similarity calculation based on 2D matrix can be replaced by 1D sentence vector similarity calculation. The drawback is that the semantic information of 1D sentence vector is compressed, resulting in low accuracy and difficulty in training convergence.

[0088] It is evident that generating target word segments instead of target characters in this application provides a more refined representation of the meaning of the target question text, improving accuracy. Furthermore, the number of word segments is less than the number of characters, which can increase the processing speed. When the reference text in this application is a standard question text or a reference answer text, text similarity is emphasized; when the reference text is a reference answer text, semantic similarity is emphasized. Considering both types of similarity improves the generalization and accuracy of the question-and-answer process. This application selects the top-ranked target number of reference texts instead of setting a preset threshold, reducing manual intervention and minimizing the impact of human factors, thus improving accuracy.

[0089] This application discloses a specific question-and-answer method. Compared to the previous embodiment, this embodiment further explains and optimizes the technical solution. See also... Figure 4 As shown, it specifically includes:

[0090] Step S21: Obtain the text of the question to be trained, the text of the correct answer to be trained, and the text of the incorrect answer to be trained.

[0091] In this embodiment, the existing question texts, correct answer texts, and incorrect answer texts of the corresponding domain are used for training, which means training is performed using the corpus of the corresponding domain.

[0092] It should be noted that, for different input corpora, a domain-specific lexicon is automatically regenerated, focusing on corpus-based words, reducing redundant vocabulary, and making the model lightweight; different domains have their own lexicons. It should also be noted that, during training, the model parameters are entirely driven by the corpus within the specific domain, without interference from redundant information from other pre-trained model parameters.

[0093] Step S22: Use the first combination of the question text to be trained and the correct answer text to be trained as the first positive sample for calculating the similarity between the question text to be trained and the correct answer text to be trained, and use the second combination of the question text to be trained and the first combination as the second positive sample for calculating the similarity between the question text to be trained and the first combination.

[0094] In this embodiment, the first positive sample is (Q, D); the second positive sample is (Q, QD); Q represents the question text, and D represents the answer text; the ratio of the first positive sample to the second positive sample can be 2:1, and the ratio can be changed according to the actual situation.

[0095] Step S23: Use the third combination of the question text to be trained and the incorrect answer text to be trained as the first negative sample for calculating the similarity between the question text to be trained and the incorrect answer text to be trained, and use the fourth combination of the question text to be trained and the third combination as the second negative sample for calculating the similarity between the question text to be trained and the third combination.

[0096] In this embodiment, negative samples are configured similarly to positive samples. It should be noted that the simultaneous presence of both positive and negative samples ensures sample balance.

[0097] Step S24: Perform word segmentation on the question text to be trained, the correct answer text to be trained, and the incorrect answer text to be trained to obtain several words to be trained; based on the words to be trained, and according to the first positive sample, the second positive sample, the first negative sample, and the second negative sample, train an initial question answering model to obtain the target question answering model.

[0098] In this embodiment, jieba is used for word segmentation for training instead of character segmentation, which simplifies the training difficulty and improves the training convergence speed and model accuracy. In addition, segmenting into words results in fewer words and fewer corresponding vectors than segmenting into characters, which greatly shortens the length of the input text vector.

[0099] In this embodiment, the indexer in the initial question-answering model includes a pre-training dual-tower model and the pre-training similarity calculation module; wherein, the dual-tower model is a model based on a single-layer encoding layer in the converter model; the question encoder and the reference text encoder in the target question-answering model are obtained by splitting the post-training dual-tower model.

[0100] It's important to note that the dual-tower model is built upon a single-layer encoder layer in the transformer model. Specifically, it extracts the Transformer-encoder layer from the Transformer module, the most popular basic paradigm in natural language processing, and uses one encoder-layer on each side to construct a parallel dual-tower model. This is because the high dimensionality of two-dimensional matrices increases memory or GPU memory usage during computation. Therefore, a single-layer encoder structure is used to maximize the lightweight nature of model parameters, improve inference speed, and reduce resource consumption. This allows for improved hit rate by leveraging the advantages of two-dimensional matrices while maintaining low resource consumption. Furthermore, using only one transformer-encoder layer (one encoder layer) for text encoding and feature extraction is more lightweight, faster to train, and requires fewer resources compared to BERT (Bidirectional Encoder Representation from Transformers) models (12 or 24 layers). Of course, the number of layers can be increased as needed to achieve the ability to include more information in large corpora.

[0101] In this embodiment, after model training is complete, model deployment is required. The question encoder and reference text encoder, derived from the dual-tower model after training, are deployed during this step. The reference text encoder (Qencoder), after being split, is not deployed online. Instead, it first encodes existing reference texts into a reference text matrix. It is only called again when new reference texts need updating, thus improving the system's computational efficiency in terms of algorithm model complexity. Furthermore, after the reference text set is encoded, the reference texts are converted into a reference text matrix and stored in memory or GPU memory. This is used for the similarity calculation step after the user's input is encoded into a question matrix. The question encoder (Dencoder), after being split, is deployed online as an upstream component of the answer generator to encode the question text in real time. In summary, the later-stage update and maintenance of the reference text library matrix in this application is completely decoupled from the online question-answering system. If new texts need updating, the text library can be directly expanded without affecting other modules.

[0102] See Figure 5The diagram illustrates a model training process. The process involves segmenting and vectorizing the question text, the correct answer text, and the incorrect answer text. After positional encoding, the text is processed by a question encoder or a reference text encoder to calculate similarity, sort the texts, and obtain a target number of reference texts. These reference texts are then input into an answer generator to obtain temporary answer text. The parameters of the segmentation and vectorization steps, the question encoder, and the reference text encoder are updated via backpropagation based on the temporary answer text. This process can be repeated multiple times until the obtained answer meets the preset accuracy rate.

[0103] In this embodiment, the dual-tower model (Qencoder, Dencoder) in the indexer model can be replaced by BERT. The drawback is that the model is too large and has high resource requirements. It is unnecessary to use such a large model as BERT for domain-specific question answering.

[0104] As can be seen, this application obtains the question text to be trained, the correct answer text to be trained, and the incorrect answer text to be trained; a first combination of the question text to be trained and the correct answer text to be trained is used as a first positive sample for calculating the similarity between the question text to be trained and the correct answer text to be trained, and a second combination of the question text to be trained and the first combination is used as a second positive sample for calculating the similarity between the question text to be trained and the first combination; a third combination of the question text to be trained and the incorrect answer text to be trained is used as a first negative sample for calculating the similarity between the question text to be trained and the incorrect answer text to be trained, and a fourth combination of the question text to be trained and the third combination is used as a second negative sample for calculating the similarity between the question text to be trained and the third combination; word segmentation is performed on the question text to be trained, the correct answer text to be trained, and the incorrect answer text to be trained to obtain several word segments to be trained; based on the word segments to be trained, and according to the first positive sample, the second positive sample, the first negative sample, and the second negative sample, an initial question answering model is trained to obtain the target question answering model. It is evident that the problem text encoder and the reference text encoder of this application utilize a single encoding layer, thereby improving training speed and obtaining a more lightweight model.

[0105] This application discloses a specific question-and-answer method. Compared to the previous embodiment, this embodiment further explains and optimizes the technical solution. See also... Figure 6 As shown, it specifically includes:

[0106] Step S31: Obtain the target question text, and perform word segmentation on the target question and answer text to obtain each target word segment, and then vectorize each target word segment to obtain each target vector.

[0107] For a more detailed description of the process of step S31, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0108] Step S32: Generate a question text matrix based on each target vector using the question encoder in the target question answering model.

[0109] For a more detailed description of the process of step S32, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0110] Step S33: Calculate the target similarity between the target question text and each of the reference texts using the similarity calculation module and each of the graphics processors based on the question text matrix and the reference text matrices stored in the graphics processor; the reference text matrix is ​​a matrix pre-constructed by the reference text encoder in the target question-answering model based on the reference texts in a preset reference text library; each of the reference texts is a standard question text and a reference answer text, or the reference answer text.

[0111] Step S34: Sort the reference texts stored in each graphics processor in descending order of target similarity, and select the number of reference texts that are at the top of the sorted list for each graphics processor.

[0112] Step S35: Using any one of the graphics processors, reorder all the selected reference texts according to the target similarity from high to low, and select the number of reference texts with the highest target similarity from the reordered reference texts.

[0113] In this embodiment, if the reference text matrix is ​​stored in the graphics card, and all the secure text matrices are divided into multiple matrix packages and stored in different graphics cards, then the similarity can be calculated layer by layer by the graphics processor. The number of graphics processor layers corresponds to the number of graphics cards. First, each graphics processor calculates the similarity between the reference text matrix and the question text matrix in its corresponding matrix package, obtaining the target number of reference texts. Therefore, each graphics processor obtains the target number of reference files. Then, a graphics processor selects the target number of reference files from multiple sets of target reference files. For details, see [link to documentation]. Figure 7The diagram illustrates a method for hierarchical similarity calculation using graphics processors. It shows three graphics processors (GPU0, GPU1, and GPU2) and three matrix packets. GPU0, GPU1, and GPU2 each process one matrix packet to obtain a target number of reference texts. Then, GPU0 selects a target number of reference texts from multiple sets of target number of reference texts. These target number of reference texts are the top target number of reference texts with the highest similarity to the target question text among all corresponding reference texts.

[0114] It should be noted that the hierarchical parallel computing method for efficient similarity calculation is faster because multiple image processors operate in parallel. It is at least 10 times more efficient than computing in a CPU (Central Processing Unit) (which is usually computed in a CPU).

[0115] It should be noted that each graphics processor needs to automatically clear its cache after completing its corresponding calculations.

[0116] See Figure 8 The diagram illustrates a question-answering method based on graphics processor-based hierarchical similarity calculation. First, the question text is segmented and vectorized, then positionally encoded, and finally encoded by a question encoder to obtain a question text matrix. Then, similarity is calculated using a graphics processor-based hierarchical similarity calculation method to derive a final target number of reference texts. The question text and the target number of reference texts are then input into an answer generator to obtain the target answer text. The reference text matrix is ​​obtained from a pre-set reference text matrix library. The reference texts are obtained from the pre-set reference text library. During the question encoding process, a dynamic padding module is added to fill in the [unk] unknown markers or [pad] padding markers in the question text matrix. The [unk] unknown markers or [pad] padding markers are used to mark the target unknown vectors in the target vectors. The target unknown vectors are words in the target question text that do not exist in the corresponding domain's lexicon.

[0117] Step S36: Generate the target answer text corresponding to the target question text based on the target number of reference texts using the answer generator.

[0118] For a more detailed description of the process of step S36, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0119] As can be seen, this application generates target words rather than target characters, which can more precisely represent the meaning of the target question text, improve accuracy, and the number of words is less than the number of characters, which can improve the speed of subsequent processing. When the reference text mentioned in this application is the standard question text and the reference answer text, text similarity is emphasized. When the reference text is the reference answer text, semantic similarity is emphasized. Considering both types of similarity improves the generalization and accuracy of question answering. This application selects the top-ranked target number of reference texts instead of setting a preset threshold, reducing manual intervention and the influence of human factors, which can improve accuracy. In addition, this application uses multiple GPUs to perform similarity detection separately, which speeds up the process and improves the efficiency of question answering.

[0120] Accordingly, this application also discloses a question-and-answer device, see [link to relevant documentation]. Figure 9 As shown, the device includes:

[0121] The vector generation module 11 is used to acquire the target question text, perform word segmentation on the target question-and-answer text to obtain target words, and then vectorize the target words to obtain target vectors.

[0122] Matrix generation module 12 is used to generate a question text matrix based on each target vector through the question encoder in the target question answering model;

[0123] The reference text selection module 13 is used to calculate the target similarity between the target question text and each reference text based on the question text matrix and each reference text matrix through the similarity calculation module in the target question answering model, and sort them according to the target similarity from high to low, and then select the target number of reference texts with the highest sorting; the reference text matrix is ​​a matrix pre-constructed by the reference text encoder in the target question answering model based on the reference texts in the preset reference text library; each reference text is a standard question text and a reference answer text, or the reference answer text;

[0124] Answer generation module 14 is used to generate target answer text corresponding to the target question text based on the target number of reference texts through an answer generator.

[0125] The more specific working process of each of the above modules can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0126] It is evident that generating target word segments instead of target characters in this application provides a more refined representation of the meaning of the target question text, improving accuracy. Furthermore, the number of word segments is less than the number of characters, which can increase the processing speed. When the reference text in this application is a standard question text or a reference answer text, text similarity is emphasized; when the reference text is a reference answer text, semantic similarity is emphasized. Considering both types of similarity improves the generalization and accuracy of the question-and-answer process. This application selects the top-ranked target number of reference texts instead of setting a preset threshold, reducing manual intervention and minimizing the impact of human factors, thus improving accuracy.

[0127] Furthermore, embodiments of this application also provide an electronic device. Figure 10 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0128] Figure 10 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the question-and-answer method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0129] In this embodiment, the power supply 26 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 25 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 24 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0130] Furthermore, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon can include computer programs 221, and the storage method can be temporary storage or permanent storage. The computer programs 221 may include, in addition to computer programs capable of performing the question-and-answer method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, computer programs capable of performing other specific tasks.

[0131] Furthermore, embodiments of this application also disclose a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned question-and-answer method.

[0132] The specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0133] The various embodiments in this application are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. For the same or similar parts between the various embodiments, refer to each other. As for the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and relevant parts can be referred to in the method section.

[0134] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0135] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0136] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0137] The above provides a detailed description of the question-and-answer method, apparatus, device, and storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A question-and-answer method, characterized in that, include: Obtain the target question text, and perform word segmentation on the target question text to obtain each target word segment. Then, vectorize each target word segment to obtain each target vector. The question encoder in the target question answering model generates a question text matrix based on each target vector; The similarity calculation module in the target question answering model calculates the target similarity between the target question text and each reference text based on the question text matrix and each reference text matrix. The reference texts are then sorted in descending order of the target similarity and the number of reference texts that are ranked first are selected. The reference text matrix is ​​a matrix pre-constructed by the reference text encoder in the target question-answering model based on the reference text in a preset reference text library; each reference text is a standard question text and a reference answer text, or the reference answer text; The answer generator generates target answer text corresponding to the target question text based on the target number of reference texts; The step of calculating the target similarity between the target question text and each reference text based on the question text matrix and each reference text matrix using the similarity calculation module in the target question answering model includes: The similarity calculation module in the target question answering model calculates the dot product between the question text matrix and the transposed reference text matrix to obtain a first similarity matrix representing the similarity between each target word segment and each reference word segment; the reference word segment is a word segment obtained by segmenting each reference text in the preset reference text library. The second similarity matrix is ​​constructed by selecting the maximum similarity corresponding to each target word in the first similarity matrix; The sum of all similarities in the second similarity matrix is ​​obtained by summing them up. The maximum similarity in the second similarity matrix is ​​selected as the key similarity. The average similarity is obtained by averaging all similarities in the second similarity matrix. The target similarity between the target question text and the reference text corresponding to the reference text matrix is ​​determined based on the sum similarity, key similarity, and average similarity, so as to calculate the target similarity between the target question text and each of the reference texts.

2. The question-and-answer method according to claim 1, characterized in that, Before obtaining the target question text, the process also includes: During the deployment phase of the target question-answering model, each reference text in the preset reference text library is segmented to obtain reference words, and each reference word is vectorized to obtain reference vectors. The reference text encoder generates a reference text matrix corresponding to each reference text based on the reference vectors corresponding to each reference text in the preset reference text library.

3. The question-and-answer method according to claim 2, characterized in that, The reference text matrix and the question text matrix are two-dimensional matrices; wherein, the first dimension represents the position information of the corresponding word segmentation, and the second dimension represents the word segmentation feature information of the corresponding word segmentation.

4. The question-and-answer method according to claim 1, characterized in that, Before obtaining the target question text, the process also includes: Obtain the text of the question to be trained, the text of the correct answer to be trained, and the text of the incorrect answer to be trained; The first combination of the question text to be trained and the correct answer text to be trained is used as the first positive sample for calculating the similarity between the question text to be trained and the correct answer text to be trained, and the second combination of the question text to be trained and the first combination is used as the second positive sample for calculating the similarity between the question text to be trained and the first combination. The third combination of the question text to be trained and the incorrect answer text to be trained is used as the first negative sample for calculating the similarity between the question text to be trained and the incorrect answer text to be trained, and the fourth combination of the question text to be trained and the third combination is used as the second negative sample for calculating the similarity between the question text to be trained and the third combination. The text of the question to be trained, the text of the correct answer to be trained, and the text of the incorrect answer to be trained are processed by word segmentation to obtain several words to be trained; The target question answering model is obtained by training an initial question answering model based on the word segment to be trained, and by training an initial question answering model according to the first positive sample, the second positive sample, the first negative sample, and the second negative sample.

5. The question-and-answer method according to claim 4, characterized in that, The indexer in the initial question-answering model includes a pre-training dual-tower model and the pre-training similarity calculation module; wherein, the dual-tower model is a model based on a single-layer encoding layer in the converter model; the question encoder and the reference text encoder in the target question-answering model are obtained by splitting the post-training dual-tower model.

6. The question-and-answer method according to any one of claims 1 to 5, characterized in that, The step of calculating the target similarity between the target question text and each reference text based on the question text matrix and each reference text matrix using the similarity calculation module in the target question answering model includes: If the reference text matrix is ​​stored in several graphics processors, the similarity calculation module calculates the target similarity between the target question text and each of the reference texts based on the question text matrix and the reference text matrices stored in the graphics processor. Accordingly, the step of sorting the reference texts according to the target similarity from high to low, and then selecting the top-ranked target number of reference texts, includes: The reference texts stored in each graphics processor are sorted in descending order of target similarity, and the number of reference texts with the highest target similarity corresponding to each graphics processor are selected. Using any one of the plurality of graphics processors, all the selected reference texts are reordered according to the target similarity from high to low, and the number of reference texts with the highest ranking among the reordered reference texts are selected.

7. A question-and-answer device, characterized in that, include: The vector generation module is used to acquire the target question text, perform word segmentation on the target question text to obtain target words, and then vectorize each target word to obtain a target vector. The matrix generation module is used to generate a question text matrix based on each target vector using the question encoder in the target question answering model. The reference text selection module is used to calculate the target similarity between the target question text and each reference text based on the question text matrix and each reference text matrix through the similarity calculation module in the target question answering model, and sort them in order from high to low according to the target similarity, and then select the target number of reference texts with the highest sorting. The reference text matrix is ​​a matrix pre-constructed by the reference text encoder in the target question-answering model based on the reference text in a preset reference text library; each reference text is a standard question text and a reference answer text, or the reference answer text; The answer generation module is used to generate target answer text corresponding to the target question text based on the target number of reference texts using an answer generator; Specifically, the reference text selection module is used to calculate the dot product of the question text matrix and the transposed reference text matrix through the similarity calculation module in the target question answering model, to obtain a first similarity matrix representing the similarity between each target word segment and each reference word segment; the reference word segment is a word segment obtained by segmenting each reference text in the preset reference text library. The second similarity matrix is ​​constructed by selecting the maximum similarity corresponding to each target word in the first similarity matrix; The sum of all similarities in the second similarity matrix is ​​obtained by summing them up. The maximum similarity in the second similarity matrix is ​​selected as the key similarity. The average similarity is obtained by averaging all similarities in the second similarity matrix. The target similarity between the target question text and the reference text corresponding to the reference text matrix is ​​determined based on the sum similarity, key similarity, and average similarity, so as to calculate the target similarity between the target question text and each of the reference texts.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the question-and-answer method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the question-and-answer method as described in any one of claims 1 to 6.

Images