Automatic question and answer similar question generation method and device, storage medium and electronic equipment

Abstract

The invention relates to an automatic question and answer similar question generation method and device, a storage medium and electronic equipment, and the method comprises the steps: extracting a target standard question from an FAQ library, and generating a similar greeting selection set comprising a plurality of candidate similar questions based on a first large language model; obtaining all questions except the target standard questions and the corresponding target similar question set in the FAQ library, constructing an external question pool, and constructing an interference set based on target candidate similar questions in the similar questions and the external question pool; obtaining model questions and answers based on the target standard questions, the interference set and a second large language model, and determining a to-be-optimized similar question set of the target standard questions based on the target standard questions and answers corresponding to the target standard questions and the model questions and answers; and based on the to-be-optimized similar question set and the similar question set corresponding to the target standard question, placing similar questions conforming to the optimization strategy in the to-be-optimized similar question set in the similar question set corresponding to the target standard question. The generation efficiency of similar questions can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence natural language processing, and in particular to an automatic question and answer similar question generation method and device, a storage medium and an electronic device. BACKGROUND

[0002] In intelligent customer service, search engines, knowledge base systems and other application scenarios, a frequently-asked questions (FAQ) library is a core component. A high-quality FAQ library not only needs to be accurate in content, but also needs to cover similar questions that users may ask, such as various synonymous and near-synonymous questions. In related technologies, when building an FAQ library, similar questions are generally manually written by domain experts or operation personnel, or generated based on rules / templates, or generated using large models. However, the above methods for generating similar questions have the following defects: For the method of manually writing similar questions by domain experts or operation personnel, the cost is high, the efficiency is low, and it is difficult to ensure the comprehensiveness of similar questions.

[0003] For the method of generating similar questions according to rules / templates, since similar questions are generated based on predefined rules or synonym replacement templates, the diversity of generated similar questions is poor, the naturalness is low, and it is difficult to deal with complex and variable natural language expressions.

[0004] For the method of generating similar questions using a large model, it is an improvement on the method of generating similar questions according to rules / templates. A batch of similar questions are directly generated using a large language model, which can make the generated similar questions diverse, but has the following defects: Uncontrollability: The generated similar questions may have slight semantic deviations from the original questions, or even completely deviate from the original meaning.

[0005] Evaluation difficulty: There is a lack of effective automated evaluation mechanism, and manual review is still needed, which makes the generation efficiency low.

[0006] Repetition and redundancy: A large number of questions may be generated that express different things but have highly repetitive semantics, resulting in a bloated FAQ library and affecting subsequent retrieval efficiency. SUMMARY

[0007] Therefore, the present application provides an automatic question and answer similar question generation method and device, a storage medium and an electronic device.

[0008] Specifically, the present application is implemented by the following technical solutions: According to a first aspect of the present application, an automatic question and answer similar question generation method is provided, and the automatic question and answer similar question generation method comprises: Extract the target standard question from the pre-built FAQ library, use the pre-trained first large language model to generate multiple candidate similar questions for the target standard question, and construct a similar question candidate set, wherein the target standard question is one of the multiple standard questions contained in the FAQ library; Obtain all questions from the FAQ library except for the target standard question and the target similar question set corresponding to the target standard question, construct an external question pool, and construct an interference set based on the target candidate similar questions in the similar question candidate set and the external question pool; Based on the target standard question, the interference set, and the pre-constructed second language model, obtain the model question and answer corresponding to the target candidate similar question. Based on the target standard question and answer corresponding to the target standard question and the model question and answer, determine the set of similar questions to be optimized for the target standard question. Based on the set of similar questions to be optimized and the set of similar questions corresponding to the target standard question, and according to the preset optimization strategy, the similar questions in the set of similar questions to be optimized that conform to the optimization strategy are placed in the set of similar questions corresponding to the target standard question.

[0009] Optionally, the step of placing similar questions in the set of similar questions to be optimized that conform to the optimization strategy into the set of similar questions corresponding to the target standard question, based on the set of similar questions to be optimized and the set of similar questions corresponding to the target standard question, according to a pre-set optimization strategy, includes: Based on the target standard question and the set of similar questions corresponding to the target standard question, construct a target vector subspace; For each similar question to be optimized in the set of similar questions to be optimized, calculate the specific index value of the similar question to be optimized and the target vector subspace. If the calculated specific index value satisfies the optimization strategy, place the similar question to be optimized in the similar question set corresponding to the target standard question.

[0010] Optionally, calculating the specificity index value between the similarity question to be optimized and the target vector subspace includes: The similarity question to be optimized is vectorized to obtain the similarity question vector; Calculate the cosine similarity between the similarity vector to be optimized and each target vector in the target vector subspace; Obtain the maximum cosine similarity among all cosine similarities, calculate the difference between 1 and the maximum cosine similarity, and obtain the specificity index value.

[0011] Optionally, calculating the specificity index value between the similarity question to be optimized and the target vector subspace includes: The similarity question to be optimized is vectorized to obtain the similarity question vector; Obtain the projection vector of the similarity vector to be optimized in the target vector subspace; Calculate the residual vector between the similarity vector to be optimized and the projection vector, obtain the L2 norm of the residual vector, and obtain the specificity index value.

[0012] Optionally, calculating the specificity index value between the similarity question to be optimized and the target vector subspace includes: Calculate the first Euclidean distance between the similarity question to be optimized and the nearest neighbor vector in the target vector subspace; Calculate the second Euclidean distance between any two target vectors in the target vector subspace; Calculate the mean and standard deviation of the Euclidean distance for each second Euclidean distance; The specificity index value is obtained based on the mean and standard deviation of the first Euclidean distance and the Euclidean distance.

[0013] Optionally, constructing an interference set based on the target candidate similar questions in the similar question candidate set and the external question pool includes: Extract the target candidate similar question from the candidate similar question set, retrieve multiple questions with matching degree to the target candidate similar question from the external question pool, and construct an interference set based on the target candidate similar question.

[0014] Optionally, retrieving multiple questions from the external question pool that match the target candidate similarity question includes: The target candidate similarity question is vectorized using a pre-built embedding model to obtain the target candidate similarity question vector; For each question in the external question pool, the embedding model is used to vectorize them respectively to obtain the question vector corresponding to each question; Calculate the cosine similarity between the target candidate similarity question vector and each question vector, and determine multiple questions that match the target candidate similarity question based on the cosine similarity.

[0015] The automated question-and-answer similar question generation method in this technical solution extracts a target standard question from a pre-built FAQ database, uses a pre-trained first language model to generate multiple candidate similar questions for the target standard question, and constructs a candidate similar question set. The target standard question is one of the multiple standard questions included in the FAQ database. It then acquires all questions from the FAQ database except for the target standard question and the target similar question set corresponding to the target standard question, constructs an external question pool, and builds an interference set based on the target candidate similar questions in the candidate similar question set and the external question pool. Based on the target standard question, the interference set, and the pre-built second language model, it obtains the model question-and-answer corresponding to the target candidate similar question. Based on the target standard question-and-answer and the model question-and-answer, it determines the set of similar questions to be optimized for the target standard question. Based on the set of similar questions to be optimized and the set of similar questions corresponding to the target standard question, according to a pre-set optimization strategy, it places the similar questions in the set of similar questions to be optimized that conform to the optimization strategy into the set of similar questions corresponding to the target standard question. In this way, the first language model is used to generate a candidate set of similar questions, and the second language model is used to obtain the model questions and answers corresponding to the candidate similar questions in the candidate set. Then, based on the model questions and answers corresponding to the candidate similar questions and the standard questions and answers, the set of similar questions to be optimized is determined. Finally, based on the set of similar questions to be optimized and the set of similar questions corresponding to the target standard questions, the optimization strategy is used to perform redundancy removal optimization. This can ensure that the generated similar questions are semantically consistent with the target standard questions and answers, avoid semantic drift, improve the efficiency and accuracy of similar question generation, and thus improve the accuracy of question answers.

[0016] According to a second aspect of the present invention, an automated question-and-answer similar question generation apparatus is provided, the apparatus comprising: The candidate similar question acquisition module is used to extract the target standard question from a pre-built FAQ library, generate multiple candidate similar questions for the target standard question using a pre-trained first language model, and construct a similar question candidate set, wherein the target standard question is one of the multiple standard questions included in the FAQ library; The interference set construction module is used to obtain all questions in the FAQ library except for the target standard question and the target similar question set corresponding to the target standard question, construct an external question pool, and construct an interference set based on the target candidate similar questions in the similar question candidate set and the external question pool; The module for obtaining similar questions to be optimized is used to obtain model question answers corresponding to the target candidate similar questions based on the target standard question, the interference set, and the pre-built second large language model, and to determine the set of similar questions to be optimized for the target standard question based on the target standard question and the model question answer. The module for processing similar questions to be optimized is used to place similar questions that conform to the optimization strategy in the set of similar questions to be optimized and the set of similar questions corresponding to the target standard question, according to a pre-set optimization strategy.

[0017] According to a third aspect of the present invention, a storage medium is provided having a computer program stored thereon, wherein when the program is executed by a processor, it implements the steps of the automated question-answering similar question generation method in any possible implementation of the first aspect.

[0018] According to a fourth aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the automated question-answering similar question generation method in any possible implementation of the first aspect. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating an automated question-answering similar question generation method provided in an embodiment of the present invention; Figure 2 A schematic diagram of an automated question-and-answer similar question generation device provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] In related technologies, for improved methods of generating similar questions using large models, a batch of similar questions can be generated using large language models, making the generated similar questions diverse. However, since the generated similar questions contain a large number of questions with different expressions but highly repetitive semantics, they need to be manually reviewed and evaluated one by one, which makes the time required to determine similar questions long and the generation efficiency low. Furthermore, the generated similar questions may have slight semantic deviations or deviate from the original questions, making the accuracy of the generated similar questions low and making it difficult to achieve both high generation quality and coverage.

[0024] In this embodiment, to address the aforementioned technical problems, an automated, efficient, and high-quality method for generating similar questions is provided. By combining large model generation and discrimination, and introducing specificity evaluation of the vector space, automated quality control, evaluation, and redundancy removal of generated similar questions are achieved, thereby significantly reducing manual intervention and improving the efficiency and accuracy of generating similar questions, ultimately enhancing the coverage and quality of the FAQ library.

[0025] See Figure 1 This invention provides an automated question-and-answer similar question generation method, which may include the following steps: S101. Extract the target standard question from the pre-built FAQ library, and use the pre-trained first language model to generate multiple candidate similar questions for the target standard question, and construct a similar question candidate set, wherein the target standard question is one of the multiple standard questions included in the FAQ library; In this embodiment, the pre-built FAQ library can be a FAQ library built using relevant technologies, and this embodiment does not limit this. The FAQ library contains multiple standard questions, each corresponding to a standard question and answer and a set of similar questions. Standard questions and standard questions and answers form a standard question-answer pair. The set of similar questions can be empty. If the set of similar questions is not empty, then similar questions that have been manually reviewed are retained. For similar questions generated using a large language model, since they contain a large number of questions with different expressions but highly repetitive semantics, they are removed from the similar set.

[0026] In this embodiment, to improve the coverage of similar questions, for each standard question in the FAQ library, multiple candidate similar questions are generated using the first large language model. As an optional embodiment, the generated multiple candidate similar questions can also be merged with the similar questions in the FAQ library that correspond to the standard question removed from the similarity set in this embodiment, and then used as the candidate set of similar questions for that standard question.

[0027] In this embodiment, as an optional implementation, a target standard question is extracted from a pre-built FAQ library, and a first pre-trained language model is used to generate multiple candidate similar questions for the target standard question, constructing a candidate set of similar questions, including: For each standard question in the FAQ library, a first pre-trained language model is used to generate multiple candidate similar questions for that standard question, and a candidate set of similar questions is constructed, which includes the standard question and the multiple candidate similar questions generated accordingly.

[0028] In this embodiment, the FAQ library typically consists of a large number of standard questions. Each standard question Corresponding to a standard question and answer and the corresponding set of similar questions In this embodiment, each standard question is taken as the target standard question and processed sequentially according to the method of this embodiment.

[0029] In this embodiment, for a given target standard question-answer pair Generate using the largest language model These candidate similar questions constitute a candidate set of similar questions for the standard question. .

[0030] In this embodiment, as an optional implementation, the first large language model can be a large language model in related technologies, and this embodiment does not limit it.

[0031] S102. Obtain all questions in the FAQ database except for the target standard question and the target similar question set corresponding to the target standard question, construct an external question pool, and construct an interference set based on the target candidate similar questions in the similar question candidate set and the external question pool; In this embodiment, the FAQ library is used to retrieve questions other than the target standard questions. and its own set of similar questions All questions other than those in the FAQ database constitute an external question pool. For example, if the FAQ database contains 100 standard questions and their corresponding sets of similar questions, then the constructed external question pool contains the remaining 99 standard questions and their corresponding sets of similar questions.

[0032] In this embodiment, an interference set is constructed based on the target candidate similar questions in the similar question candidate set and the external question pool, including: Extract the target candidate similar question from the candidate similar question set, retrieve multiple questions with matching degree to the target candidate similar question from the external question pool, and construct an interference set based on the target candidate similar question.

[0033] In this embodiment, as an optional implementation, multiple questions with matching degree to the target candidate similarity question are retrieved from the external question pool, including: The target candidate similarity question is vectorized using a pre-built embedding model to obtain the target candidate similarity question vector; For each question in the external question pool, the embedding model is used to vectorize them respectively to obtain the question vector corresponding to each question; Calculate the cosine similarity between the target candidate similarity question vector and each question vector, and determine multiple questions that match the target candidate similarity question based on the cosine similarity.

[0034] In this embodiment, each candidate similarity question corresponds to an interference set. For each candidate similarity question... Vectorize it using an embedding model to obtain Then, from the vectorized external problem pool, retrieve the relevant... The highest cosine similarity These questions constitute a set of interference. Each retrieved question has a corresponding standard question and answer in the FAQ database. These distractor questions are similar to the candidate similar questions in vector representation, but theoretically should correspond to the standard question and answer of the candidate similar questions. Mismatch.

[0035] S103. Based on the target standard question, the interference set, and the pre-constructed second large language model, obtain the model question and answer corresponding to the target candidate similar question. Based on the target standard question and answer corresponding to the target standard question and the model question and answer, determine the set of similar questions to be optimized for the target standard question. In this embodiment, as an optional implementation, based on the target standard question, the interference set, and the pre-built second large language model, the model question answer corresponding to the target candidate similar question is obtained, including: A11, merge the target standard question with the interference set to obtain a merged set, retrain the pre-built second language model based on the merged set, and construct structured prompt words based on the target candidate similarity question; In this embodiment, as an optional implementation, the target standard set is merged with the interference set to obtain a merged set, including: The target standard question is placed in the interference set, and the questions in the interference set are randomly shuffled to obtain a merged set.

[0036] In this embodiment, the target standard question The interference set is merged and randomly shuffled, then input into the second largest language model for training.

[0037] In this embodiment, the structured prompt words are prompt words that meet the input requirements of the second largest language model.

[0038] In this embodiment, taking the example of a merge set containing multiple instances, the corresponding template example is as follows: Example 1: Question 1: [q_i1] Standard Q&A: [a_i1] Example 2: The target standard is: [Q_t] Target Standard Questions and Answers: [A_t] … Example K Question K: [q_ik] Standard Q&A: [a_ik].

[0039] In this embodiment, the structured prompt words are as follows: User question: [Q_ti] In this embodiment, the user question is a candidate similar question to the target question.

[0040] A12, input the structured prompt words into the second large language model to obtain model question answering.

[0041] In this embodiment, structured prompt words are used as input to the second language model to obtain the model question-and-answer output by the second language model.

[0042] In this embodiment, as an optional embodiment, based on the target standard question and answer corresponding to the target standard question and the model question and answer, the set of similar questions to be optimized for the target standard question is determined, including: Calculate the similarity between the target standard question and the model question and answer. If the calculated similarity is greater than a preset similarity threshold, place the target candidate similar question corresponding to the model question and answer into the set of similar questions to be optimized in the target standard question.

[0043] In this embodiment, for each candidate similar question in the candidate similar question set, the model question answer output by the second largest language model is obtained by constructing the corresponding interference set, and the similarity is calculated with the target standard question answer. Candidate similar questions that meet the similarity threshold are placed in the similar question set to be optimized, thus obtaining the similar question set to be optimized for the target standard question.

[0044] In this embodiment, the constructed complete prompt words are input into the second large language model (discriminative model) to obtain the model question-and-answer generated by the second large language model, denoted as . Compare the question-and-answer generated by the second largest language model. With standard Q&A The similarity is determined by either exact matching or cosine similarity based on embedding. If the similarity is higher than a similarity threshold, then the candidate similarity question is considered valid in the cognition of the second language model. Questions about target standards If the semantics are consistent, it is deemed valid and placed in the set of similar questions to be optimized in the target standard question; otherwise, it is deemed invalid and the candidate similar question is discarded.

[0045] S104. Based on the set of similar questions to be optimized and the set of similar questions corresponding to the target standard question, according to the preset optimization strategy, place the similar questions in the set of similar questions to be optimized that conform to the optimization strategy into the set of similar questions corresponding to the target standard question.

[0046] In this embodiment, a challenging testing environment is constructed to rigorously verify the semantic correctness of the candidate similarity set to be optimized.

[0047] In this embodiment, as an optional implementation, the testing environment is used to evaluate the novelty of the candidate similarity question to be optimized from the representation level, ensuring that it is not covered by the corresponding similarity question set.

[0048] In this embodiment, as an optional embodiment, based on the set of similar questions to be optimized and the set of similar questions corresponding to the target standard question, and according to a pre-set optimization strategy, similar questions in the set of similar questions to be optimized that conform to the optimization strategy are placed in the set of similar questions corresponding to the target standard question, including: B11, construct a target vector subspace based on the target standard question and the similar question set corresponding to the target standard question; In this embodiment, the target standard question The vector and all similar questions in the similar question set corresponding to the target standard question. The set of vectors formed by the vectors is defined as the target vector subspace. .

[0049] B12, for each similar question to be optimized in the set of similar questions to be optimized, calculate the specific index value of the similar question to be optimized and the target vector subspace. If the calculated specific index value satisfies the optimization strategy, place the similar question to be optimized in the similar question set corresponding to the target standard question.

[0050] In this embodiment, for the similar questions to be optimized in the selected set of similar questions to be optimized... Calculate its vector The specific index value relative to the target vector subspace.

[0051] In this embodiment, as an optional embodiment, calculating the specificity index value between the similarity space to be optimized and the target vector subspace includes: The similarity question to be optimized is vectorized to obtain the similarity question vector; Calculate the cosine similarity between the similarity vector to be optimized and each target vector in the target vector subspace; Obtain the maximum cosine similarity among all cosine similarities, calculate the difference between 1 and the maximum cosine similarity, and obtain the specificity index value.

[0052] In this embodiment, as an optional embodiment, the specificity index value is the specificity based on the minimum cosine distance.

[0053] In this embodiment, calculation and The cosine similarity of each target vector is taken as the maximum value. The specificity index value is: .

[0054] In this embodiment, as another optional embodiment, calculating the specificity index value of the similarity space to be optimized and the target vector subspace includes: The similarity question to be optimized is vectorized to obtain the similarity question vector; Obtain the projection vector of the similarity vector to be optimized in the target vector subspace; Calculate the residual vector between the similarity vector to be optimized and the projection vector, obtain the L2 norm of the residual vector, and obtain the specificity index value.

[0055] In this embodiment, as an optional embodiment, the specificity index value is based on the norm of the vector projection residual.

[0056] In this embodiment, the target vector subspace is... Each target vector in the algorithm is considered as an empirical distribution, and the similarity vector to be optimized is calculated. Projection vector on the target vector subspace and calculate Residual: .

[0057] In this embodiment, the residual vector can effectively represent the semantic part of the similarity problem to be optimized that is not covered by the existing problem. This is achieved by calculating the L2 norm of the residual vector. It can directly measure the absolute distance of the similarity question to be optimized from the target subspace.

[0058] In this embodiment, as another optional embodiment, calculating the specificity index value of the similarity interval to be optimized and the target vector subspace includes: Calculate the first Euclidean distance between the similarity question to be optimized and the nearest neighbor vector in the target vector subspace; Calculate the second Euclidean distance between any two target vectors in the target vector subspace; Calculate the mean and standard deviation of the Euclidean distance for each second Euclidean distance; The specificity index value is obtained based on the mean and standard deviation of the first Euclidean distance and the Euclidean distance.

[0059] In this embodiment, as an optional embodiment, the specificity index value is the Z-score standardized score based on the nearest neighbor distance.

[0060] In this embodiment, the similarity vector to be optimized is calculated. to target vector subspace First Euclidean distance of the nearest neighbor vector Calculate the target vector subspace. The second Euclidean distance between all pairs of vectors within the interior is calculated, and the mean Euclidean distance of this distribution is calculated based on the interior distance distribution of the second Euclidean distance. and standard deviation Calculate the Z-score standardized score.

[0061] In this embodiment, as an optional implementation, the Z-score standardized score is calculated using the following formula: .

[0062] In this embodiment, The larger the value, the greater the distance between the similar problem to be optimized and the existing problem is, which is much greater than the typical distance within the similar problem set, and the stronger the specificity.

[0063] In this embodiment, as an optional embodiment, if the calculated specificity index value satisfies the optimization strategy, the similar question to be optimized is placed in the similar question set corresponding to the target standard question, including: If the calculated specificity index value corresponding to the similar question to be optimized is greater than the preset specificity index threshold, the similar question to be optimized is placed in the similar question set corresponding to the target standard question to which the similar question to be optimized belongs; If the specificity index value corresponding to the calculated similarity question to be optimized is not greater than the pre-set specificity index threshold, the similarity question to be optimized is discarded.

[0064] In this embodiment, the optimization strategy is a pre-set specificity index threshold. If the specificity index value corresponding to the similar question to be optimized is higher than the specificity index threshold, it is placed in the similar question set, that is, the similar question set is updated using the similar question to be optimized. Otherwise, it is determined that the similar question to be optimized is redundant with a certain similar question in the similar question set, and therefore, the similar question to be optimized is discarded.

[0065] In this embodiment, after updating the new similar question to the FAQ library, each similar question to be optimized in the set of similar questions to be optimized is traversed to determine whether the similar question to be optimized meets the optimization strategy. After the specificity judgment of the set of similar questions to be optimized is completed, another target standard question is selected and the steps of this embodiment are repeated until no similar questions that meet the optimization strategy are generated in multiple consecutive candidate similar questions in the similar question candidate set. Then the process is stopped, thereby realizing the continuous optimization of FAQ.

[0066] The automated question-and-answer similar question generation method in this embodiment uses large language models (first and second large language models) for cognitive discrimination, ensuring that the generated similar questions are strictly consistent with the original question-and-answer pairs semantically, thus avoiding semantic drift. Furthermore, through vector space specificity evaluation, the novelty of the new similar questions is mathematically quantified, effectively avoiding redundancy in the FAQ library and ensuring its simplicity and efficiency. Iterative generation further expands the semantic range of the FAQ library, continuously improving the intelligence level of the question-and-answer system. Therefore, the method in this embodiment frees manual labor from tedious writing and review work, automating the entire process of similar question generation, review, and selection, greatly improving the efficiency and accuracy of similar question generation.

[0067] Based on the same inventive concept, such as Figure 2 As shown, this embodiment of the invention also provides an automated question-and-answer similar question generation device, the device comprising: The candidate similar question acquisition module 201 is used to extract the target standard question from the pre-built FAQ library, generate multiple candidate similar questions for the target standard question using the pre-trained first large language model, and construct a similar question candidate set, wherein the target standard question is one of the multiple standard questions included in the FAQ library; In this embodiment, as an optional embodiment, the candidate similarity question acquisition module 201 is specifically used for: For each standard question in the FAQ library, a first pre-trained language model is used to generate multiple candidate similar questions for that standard question, and a candidate set of similar questions is constructed, which includes the standard question and the multiple candidate similar questions generated accordingly.

[0068] Interference set construction module 202 is used to obtain all questions in the FAQ library except for the target standard question and the target similar question set corresponding to the target standard question, construct an external question pool, and construct an interference set based on the target candidate similar questions in the similar question candidate set and the external question pool; In this embodiment, as an optional embodiment, the interference set construction module 202 is specifically used for: Extract the target candidate similar question from the candidate similar question set, retrieve multiple questions with matching degree to the target candidate similar question from the external question pool, and construct an interference set based on the target candidate similar question.

[0069] In this embodiment, as an optional implementation, multiple questions with matching degree to the target candidate similarity question are retrieved from the external question pool, including: The target candidate similarity question is vectorized using a pre-built embedding model to obtain the target candidate similarity question vector; For each question in the external question pool, the embedding model is used to vectorize them respectively to obtain the question vector corresponding to each question; Calculate the cosine similarity between the target candidate similarity question vector and each question vector, and determine multiple questions that match the target candidate similarity question based on the cosine similarity.

[0070] The module 203 for obtaining similar questions to be optimized is used to obtain model question answers corresponding to the target candidate similar questions based on the target standard question, the interference set and the pre-built second large language model, and to determine the set of similar questions to be optimized for the target standard question based on the target standard question and the model question answer. In this embodiment, as an optional embodiment, the similarity question acquisition module 203 to be optimized is specifically used for: The target standard question is merged with the interference set to obtain a merged set. The pre-built second language model is retrained based on the merged set, and structured prompt words are constructed based on the target candidate similarity question. Input the structured prompts into the second large language model to obtain model question answers.

[0071] In this embodiment, as an optional implementation, the target standard set is merged with the interference set to obtain a merged set, including: The target standard question is placed in the interference set, and the questions in the interference set are randomly shuffled to obtain a merged set.

[0072] In this embodiment, as an optional embodiment, based on the target standard question and answer corresponding to the target standard question and the model question and answer, the set of similar questions to be optimized for the target standard question is determined, including: Calculate the similarity between the target standard question and the model question and answer. If the calculated similarity is greater than a preset similarity threshold, place the target candidate similar question corresponding to the model question and answer into the set of similar questions to be optimized in the target standard question.

[0073] The similar question processing module 204 is used to place similar questions that conform to the optimization strategy in the similar question set to be optimized into the similar question set corresponding to the target standard question, based on the similar question set to be optimized and the similar question set corresponding to the target standard question, according to a pre-set optimization strategy.

[0074] In this embodiment, as an optional embodiment, the similarity processing module 204 to be optimized includes: A vector subspace construction unit is used to construct a target vector subspace based on the target standard question and the similar question set corresponding to the target standard question; The unit for processing similar questions to be optimized is used to calculate the specific index value of each similar question to be optimized in the set of similar questions to be optimized and the target vector subspace. If the calculated specific index value satisfies the optimization strategy, the similar question to be optimized is placed in the similar question set corresponding to the target standard question.

[0075] In this embodiment, as an optional embodiment, the similarity processing unit to be optimized is specifically used for: The similarity question to be optimized is vectorized to obtain the similarity question vector; Calculate the cosine similarity between the similarity vector to be optimized and each target vector in the target vector subspace; Obtain the maximum cosine similarity among all cosine similarities, calculate the difference between 1 and the maximum cosine similarity, and obtain the specificity index value.

[0076] In this embodiment, as another optional embodiment, the similarity processing unit to be optimized is specifically used for: The similarity question to be optimized is vectorized to obtain the similarity question vector; Obtain the projection vector of the similarity vector to be optimized in the target vector subspace; Calculate the residual vector between the similarity vector to be optimized and the projection vector, obtain the L2 norm of the residual vector, and obtain the specificity index value.

[0077] In this embodiment, as another optional embodiment, the similarity processing unit to be optimized is specifically used for: Calculate the first Euclidean distance between the similarity question to be optimized and the nearest neighbor vector in the target vector subspace; Calculate the second Euclidean distance between any two target vectors in the target vector subspace; Calculate the mean and standard deviation of the Euclidean distance for each second Euclidean distance; The specificity index value is obtained based on the mean and standard deviation of the first Euclidean distance and the Euclidean distance.

[0078] Based on the same inventive concept, embodiments of the present invention also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the automated question-answering similar question generation method in any of the above possible implementations.

[0079] Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0080] Based on the same inventive concept, see [link to inventive concept] Figure 3 This invention also provides an electronic device, including a memory 101 (e.g., non-volatile memory), a processor 102, and a computer program stored on the memory 101 and executable on the processor 102. When the processor 102 executes the program, it implements the steps of the automated question-and-answer similar question generation method described in any of the above possible implementations, which is equivalent to the aforementioned automated question-and-answer similar question generation device. Of course, the processor can also be used to process other data or perform calculations. This electronic device can be a PC, server, terminal, or other similar device.

[0081] like Figure 3 As shown, the electronic device may also include: memory 103, network interface 104, and internal bus 105. In addition to these components, other hardware may also be included, which will not be described in detail here.

[0082] It should be noted that the aforementioned automated question-and-answer similar question generation device can be implemented by software. As a device in a logical sense, it is formed by the processor 102 of the electronic device in which it is located reading the computer program instructions stored in the non-volatile memory into the memory 103 for execution.

[0083] The embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0084] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by special-purpose logic circuitry—such as FPGA (Field Programmable Gate Array) or ASIC (Application-Specific Integrated Circuit), and the device can also be implemented as special-purpose logic circuitry.

[0085] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0086] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0087] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily used to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0088] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0089] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0090] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0091] The above are merely specific embodiments of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An automated question-and-answer similar question generation method, characterized in that, include: Extract the target standard question from the pre-built FAQ library, use the pre-trained first large language model to generate multiple candidate similar questions for the target standard question, and construct a similar question candidate set, wherein the target standard question is one of the multiple standard questions contained in the FAQ library; Obtain all questions from the FAQ library except for the target standard question and the target similar question set corresponding to the target standard question, construct an external question pool, and construct an interference set based on the target candidate similar questions in the similar question candidate set and the external question pool; Based on the target standard question, the interference set, and the pre-constructed second language model, obtain the model question and answer corresponding to the target candidate similar question. Based on the target standard question and answer corresponding to the target standard question and the model question and answer, determine the set of similar questions to be optimized for the target standard question. Based on the set of similar questions to be optimized and the set of similar questions corresponding to the target standard question, and according to the preset optimization strategy, the similar questions in the set of similar questions to be optimized that conform to the optimization strategy are placed in the set of similar questions corresponding to the target standard question.

2. The automated question-answering similar question generation method according to claim 1, characterized in that, The step of placing similar questions in the set of similar questions to be optimized that conform to the optimization strategy into the set of similar questions corresponding to the target standard question, based on the set of similar questions to be optimized and the set of similar questions corresponding to the target standard question, according to a pre-set optimization strategy, includes: Based on the target standard question and the set of similar questions corresponding to the target standard question, construct a target vector subspace; For each similar question to be optimized in the set of similar questions to be optimized, calculate the specific index value of the similar question to be optimized and the target vector subspace. If the calculated specific index value satisfies the optimization strategy, place the similar question to be optimized in the similar question set corresponding to the target standard question.

3. The automated question-answering similar question generation method according to claim 2, characterized in that, The calculation of the specificity index value between the similarity space to be optimized and the target vector subspace includes: The similarity question to be optimized is vectorized to obtain the similarity question vector; Calculate the cosine similarity between the similarity vector to be optimized and each target vector in the target vector subspace; Obtain the maximum cosine similarity among all cosine similarities, calculate the difference between 1 and the maximum cosine similarity, and obtain the specificity index value.

4. The automated question-answering similar question generation method according to claim 2, characterized in that, The calculation of the specificity index value between the similarity space to be optimized and the target vector subspace includes: The similarity question to be optimized is vectorized to obtain the similarity question vector; Obtain the projection vector of the similarity vector to be optimized in the target vector subspace; Calculate the residual vector between the similarity vector to be optimized and the projection vector, obtain the L2 norm of the residual vector, and obtain the specificity index value.

5. The automated question-answering similar question generation method according to claim 2, characterized in that, The calculation of the specificity index value between the similarity space to be optimized and the target vector subspace includes: Calculate the first Euclidean distance between the similarity question to be optimized and the nearest neighbor vector in the target vector subspace; Calculate the second Euclidean distance between any two target vectors in the target vector subspace; Calculate the mean and standard deviation of the Euclidean distance for each second Euclidean distance; The specificity index value is obtained based on the mean and standard deviation of the first Euclidean distance and the Euclidean distance.

6. The automated question-answering similar question generation method according to any one of claims 1 to 5, characterized in that, The construction of an interference set based on the target candidate similar questions in the similar question candidate set and the external question pool includes: Extract the target candidate similar question from the candidate similar question set, retrieve multiple questions with matching degree to the target candidate similar question from the external question pool, and construct an interference set based on the target candidate similar question.

7. The automated question-answering similar question generation method according to claim 6, characterized in that, The step of retrieving multiple questions from the external question pool that match the target candidate similarity question includes: The target candidate similarity question is vectorized using a pre-built embedding model to obtain the target candidate similarity question vector; For each question in the external question pool, the embedding model is used to vectorize them respectively to obtain the question vector corresponding to each question; Calculate the cosine similarity between the target candidate similarity question vector and each question vector, and determine multiple questions that match the target candidate similarity question based on the cosine similarity.

8. An automated question-and-answer similar question generation device, characterized in that, The automated question-and-answer similar question generation device includes: The candidate similar question acquisition module is used to extract the target standard question from a pre-built FAQ library, generate multiple candidate similar questions for the target standard question using a pre-trained first language model, and construct a similar question candidate set, wherein the target standard question is one of the multiple standard questions included in the FAQ library; The interference set construction module is used to obtain all questions in the FAQ library except for the target standard question and the target similar question set corresponding to the target standard question, construct an external question pool, and construct an interference set based on the target candidate similar questions in the similar question candidate set and the external question pool; The module for obtaining similar questions to be optimized is used to obtain model question answers corresponding to the target candidate similar questions based on the target standard question, the interference set, and the pre-built second large language model, and to determine the set of similar questions to be optimized for the target standard question based on the target standard question and the model question answer. The module for processing similar questions to be optimized is used to place similar questions that conform to the optimization strategy in the set of similar questions to be optimized and the set of similar questions corresponding to the target standard question, according to a pre-set optimization strategy.

9. A storage medium, characterized in that, The program or instructions are stored on the storage medium, and the program or instructions are executed by the processor to implement the steps of the automated question-answering similar question generation method as described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the automated question-answering similar question generation method according to any one of claims 1 to 7.

Images