Extended question screening method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide an extended question screening method and device, electronic equipment and storage medium, comprising: obtaining a standard question and a plurality of randomly generated extended questions, wherein the standard question includes a target standard question and a plurality of remaining standard questions, screening a candidate extended question corresponding to the target standard question from the plurality of extended questions, and clustering the candidate extended question into a plurality of clusters, wherein the first similarity between the candidate extended question and the target standard question is within a first predetermined range, and determining the target extended question in each cluster according to the second similarity between the candidate extended question and the remaining standard questions. The present application determines the target extended question that meets the conditions in a large number of extended questions through similarity screening and batch clustering, meets the demand of a complex interactive system for screening high-quality extended questions in content complex and large number of extended questions, and further improves the user human-computer interaction experience.

Description

Extended question: Screening methods, devices, electronic equipment, and storage media Technical Field

[0001] This invention belongs to the field of human-computer interaction technology, and in particular relates to an extended question screening method, device, electronic device and storage medium. Background Technology

[0002] Human-Computer Interaction (HCI) refers to the information exchange process between humans and computers using a certain conversational language and interactive methods to complete a specific task. Various artificial intelligence systems, such as voice control systems, can be implemented through HCI. To improve the accuracy of user intent recognition, a large number of pre-written samples (i.e., standard questions) are often prepared for each intent. Since the number of manually prepared standard questions is limited, a large number of related samples (i.e., extended questions) need to be recommended based on the standard questions.

[0003] Currently, expanded sentences are obtained by manually processing questions collected from data sources. The data source is existing data, and the quantity and quality of expanded sentences obtained from existing data through manual processing are limited. To ensure the quality of expanded questions, quality testing of expanded questions is required. Existing technology uses word frequency statistics to analyze the quality of expanded questions on a limited number of manually processed expanded questions. However, this method is only applicable to a limited number of manually processed expanded questions. When the content of the expanded questions generated by the system is complex and the number is large, the quality detection method is not effective, affecting the quality of expanded questions and further impacting the user experience. Summary of the Invention

[0004] This invention provides an extended question screening method, apparatus, electronic device, and storage medium to solve the problem that existing extended question quality testing only supports the detection of a limited number of manually processed extended questions, which affects the quality of extended questions.

[0005] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0006] In a first aspect, the present invention provides an extended query screening method, the method comprising:

[0007] Obtain a standard question and multiple randomly generated extended questions, wherein the standard question includes a target standard question and multiple remaining standard questions;

[0008] Candidate extended questions corresponding to the target standard question are selected from multiple extended questions, and the candidate extended questions are clustered into multiple clusters, wherein the first similarity between the candidate extended questions and the target standard question is within a first preset range;

[0009] The target extension question in each cluster is determined based on the second similarity between the candidate extension question and the remaining standard questions.

[0010] Optionally, the step of filtering candidate extended questions corresponding to the target standard question from a plurality of extended questions, and clustering the candidate extended questions into a plurality of clusters, includes:

[0011] Each of the extended questions and the target standard question is obtained with a first similarity score.

[0012] Filter the extended questions corresponding to the first similarity within a first preset range to obtain candidate extended questions corresponding to the target standard question;

[0013] The candidate expansion questions are clustered into multiple clusters using a preset clustering algorithm.

[0014] Optionally, determining the target extension question in each cluster based on the second similarity between the candidate extension questions and the remaining standard questions includes:

[0015] The candidate extended question in each of the clusters is compared with the remaining standard questions, and the maximum similarity between the candidate extended question and the remaining standard questions is determined as the second similarity.

[0016] If the difference between the first similarity and the second similarity is greater than the first preset threshold, the candidate expansion question is retained;

[0017] The candidate extension question with the highest similarity among the retained candidates is identified as the target extension question in the cluster.

[0018] Optionally, before obtaining the standard question and multiple randomly generated extended questions, the process includes:

[0019] Collect a standard question, input the standard question into a pre-trained first generative model, and randomly generate multiple extended questions corresponding to the standard question; or...

[0020] The standard question and the first extended question are input into the first generation model, and multiple extended questions corresponding to the standard question are randomly generated. The first extended question is pre-annotated based on the standard question.

[0021] Optionally, training the first generative model includes:

[0022] Obtain the standard question and the extended questions associated with the standard question;

[0023] The standard question is input into the text generation model to randomly generate actual extended questions;

[0024] Determine the contrast loss of the extended question associated with the actual extended question and the standard question;

[0025] Based on the contrast loss, the network parameters of the text generation model are adjusted to obtain a pre-trained first generation model.

[0026] Optionally, if the plurality of extended questions are randomly generated based on the standard question and the first extended question, the process of obtaining the standard question and the randomly generated plurality of extended questions further includes:

[0027] Candidate extended questions are selected from multiple extended questions, and the candidate extended questions are clustered into multiple clusters, wherein the first similarity between the candidate extended questions and the target standard question is within a second preset range;

[0028] The target extension question in each cluster is determined based on the average similarity between the candidate extension question, the standard question, and the first extension question.

[0029] Optionally, determining the target extended question in each cluster based on the average similarity between the candidate extended question, the standard question, and the first extended question includes:

[0030] The candidate extended question in each of the clusters is compared with the target standard question and the first extended question to determine the first average similarity between the candidate extended question and the target standard question and the first extended question.

[0031] The candidate expanded question is compared with the remaining standard questions and the first expanded question to determine the second average similarity between the candidate expanded question, the remaining standard questions and the first expanded question;

[0032] If the first average similarity is greater than the second average similarity, the candidate extended question is retained;

[0033] The candidate extension question with the highest average similarity among the retained candidates is identified as the target extension question in the cluster.

[0034] Secondly, the present invention provides an extended screening device, the device comprising:

[0035] The first acquisition module is used to acquire a standard question and a number of randomly generated extended questions, wherein the standard question includes a target standard question and a number of remaining standard questions;

[0036] The first filtering module is used to filter candidate extended questions corresponding to the target standard question from a plurality of extended questions, and to cluster the candidate extended questions into a plurality of clusters, wherein the first similarity between the candidate extended questions and the target standard question is within a first preset range;

[0037] The first determining module is used to determine the target extended question in each of the clusters based on the second similarity between the candidate extended questions and the remaining standard questions.

[0038] Optionally, the first filtering module includes:

[0039] The first acquisition submodule is used to acquire the first similarity between the multiple extended questions and the target standard question respectively;

[0040] The filtering submodule is used to filter the extended questions corresponding to the first similarity within a first preset range to obtain candidate extended questions corresponding to the target standard question;

[0041] The clustering submodule is used to cluster the candidate expansion questions into multiple clusters using a preset clustering algorithm.

[0042] Optionally, the first determining module includes:

[0043] The first determining submodule is used to compare the candidate extended question in each of the clusters with the remaining standard questions respectively, and determine the maximum similarity between the candidate extended question and the remaining standard questions as the second similarity.

[0044] The first judgment submodule is used to retain the candidate expansion question if the difference between the first similarity and the second similarity is greater than a first preset threshold.

[0045] The second determining submodule is used to determine the candidate extension question with the highest similarity among the retained candidate extension questions as the target extension question in the cluster.

[0046] Optionally, the device further includes:

[0047] The first generation module is used to collect standard questions, input the standard questions into a pre-trained first generation model, and randomly generate multiple extended questions corresponding to the standard questions; or...

[0048] The second generation module is used to input the standard question and the first extended question into the first generation model and randomly generate multiple extended questions corresponding to the standard question, wherein the first extended question is pre-annotated based on the standard question.

[0049] Optionally, training the first generative model includes:

[0050] The acquisition unit is used to acquire the standard question and the extended questions associated with the standard question;

[0051] A generation unit is used to input the standard question into the text generation model and randomly generate an actual extended question.

[0052] A determining unit is used to determine the contrast loss of the extended question associated with the actual extended question and the standard question;

[0053] The training unit is used to adjust the network parameters of the text generation model according to the contrast loss to obtain a pre-trained first generation model.

[0054] Optionally, if the plurality of extended questions are randomly generated based on the standard question and the first extended question, the apparatus further includes:

[0055] The second filtering module is used to filter candidate extended questions from multiple extended questions and cluster the candidate extended questions into multiple clusters, wherein the first similarity between the candidate extended questions and the target standard question is within a second preset range;

[0056] The second determining module is used to determine the target extended question in each of the clusters based on the average similarity between the candidate extended question and the standard question and the first extended question.

[0057] Optionally, the second determining module includes:

[0058] The third determining submodule is used to compare the candidate extended question in each of the clusters with the target standard question and the first extended question respectively, and determine the first average similarity between the candidate extended question and the target standard question and the first extended question;

[0059] The fourth determining submodule is used to compare the candidate extended question with the remaining standard questions and the first extended question respectively, and determine the second average similarity between the candidate extended question and the remaining standard questions and the first extended question;

[0060] The second judgment submodule is used to retain the candidate expansion question if the first average similarity is greater than the second average similarity.

[0061] The fifth determining submodule is used to determine the candidate extension question with the highest average similarity among the retained candidate extension questions as the target extension question in the cluster.

[0062] Thirdly, the present invention provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described extended query filtering method.

[0063] Fourthly, the present invention provides a readable storage medium that, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the aforementioned extended query filtering method.

[0064] This invention, through obtaining a standard question and a plurality of randomly generated extended questions, wherein the standard question includes a target standard question and a plurality of remaining standard questions, selects candidate extended questions corresponding to the target standard question from the plurality of extended questions, and clusters the candidate extended questions into a plurality of clusters. The first similarity between the candidate extended question and the target standard question is within a first preset range. Based on the second similarity between the candidate extended question and the remaining standard questions, the target extended question in each cluster is determined. This invention avoids manual processing of extended sentences by obtaining a large number of automatically and randomly generated extended questions, saving labor costs. It uses clustering to determine target extended questions that are related to but different from the target standard question from the selected candidate extended questions, resulting in diverse and less confusing target extended questions, fluent and coherent sentences, and improved quality of extended questions. Furthermore, by using similarity filtering and batch clustering to determine the target extended questions that meet the conditions from a large number of extended questions, it meets the needs of complex interactive systems to select high-quality extended questions from complex and numerous extended questions, further improving the user's human-computer interaction experience. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 is one of the flowcharts of an extended question screening method provided in an embodiment of the present invention;

[0067] Figure 2 is a flowchart of step 103 of an extended question screening method provided in the embodiment of the present invention in Figure 1;

[0068] Figure 3 is a second flowchart of an extended question screening method provided in an embodiment of the present invention;

[0069] Figure 4 is a flowchart of an extended question filtering method provided in the embodiment of the present invention shown in Figure 1;

[0070] Figure 5 is a flowchart of model training in an extended question screening method provided in the embodiment of the present invention shown in Figure 1;

[0071] Figure 6 is a flowchart of the third step of an extended question screening method provided in an embodiment of the present invention;

[0072] Figure 7 is a flowchart of an extended question filtering method provided in the embodiment of the present invention shown in Figure 6;

[0073] Figure 8 is a schematic diagram of an extended screening device provided in an embodiment of the present invention;

[0074] Figure 9 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0075] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0076] Referring to Figure 1, a flowchart of one of the steps of the extended question filtering method provided in an embodiment of the present invention is shown. The method may include:

[0077] Step 101: Obtain the standard question and multiple randomly generated extended questions, wherein the standard question includes the target standard question and multiple remaining standard questions.

[0078] In this embodiment of the invention, in order to select a small number of high-quality extended questions that have certain differences from each other from a large number of randomly generated extended questions and add them to the knowledge base, it is first necessary to determine the standard questions to be extended and obtain the standard questions and a number of randomly generated extended questions.

[0079] It should be noted that the standard questions can be pre-collected from a historical dialogue corpus. The standard questions include a target standard question and multiple remaining standard questions. The target standard question is the standard question currently to be expanded. After determining the standard question to be expanded, expanded questions that match it can be obtained. For example, expanded questions whose keywords are consistent with or related to the keywords of the standard question to be expanded are identified as the expanded questions that match the standard question to be expanded and stored as a pair of standard questions and corresponding expanded questions.

[0080] Specifically, in this embodiment, multiple extended questions corresponding to the standard question are randomly generated by a pre-trained text generation model. In this embodiment, the standard question is collected in advance based on the historical dialogue corpus, and the standard question is input into the text generation model to randomly generate multiple extended questions. In some embodiments, the text generation model is input with the standard question and pre-annotated extended questions, and multiple extended questions corresponding to the standard question are randomly generated. This embodiment will not be described in detail here.

[0081] Step 102: Select candidate extended questions that correspond to the target standard question from multiple extended questions, and cluster the candidate extended questions into multiple clusters, wherein the first similarity between the candidate extended questions and the target standard question is within a first preset range.

[0082] In this embodiment of the invention, if multiple extended questions are randomly generated based on standard questions, candidate extended questions corresponding to the target standard question are selected from the multiple extended questions. Specifically, in this embodiment, a semantic similarity model can be used to select candidate extended questions that meet the similarity conditions.

[0083] Specifically, step 102 may include the following steps:

[0084] First, obtain the first similarity between multiple extended questions and the target standard question;

[0085] Secondly, the extended questions corresponding to the first similarity within the first preset range are selected to obtain candidate extended questions corresponding to the target standard question;

[0086] Secondly, the candidate expansion questions are clustered into multiple clusters using a preset clustering algorithm.

[0087] For example, the target standard question and multiple extended questions are input into the sentence semantic similarity model, and the similarity between each extended question and the target standard question is compared. Extended questions with similarity within a first preset range are retained as candidate extended questions, and the remaining extended questions are filtered out or stored for later use. The first preset range is preset according to the system's quality testing requirements for extended questions, and is not specifically limited here. In this embodiment, the first preset range can be 0.6 to 0.8.

[0088] In this embodiment of the invention, the candidate expansion question can be clustered into multiple clusters using the K-means clustering algorithm. It should be noted that the K-means clustering algorithm is an iterative clustering analysis algorithm. It divides the selected candidate expansion questions into K groups, randomly selects K candidate expansion question objects as initial cluster centers, calculates the distance between the center candidate expansion question of each group and the center candidate expansion question of other groups, and assigns each candidate expansion question to the nearest cluster center. Of course, the above is only a specific example. In this embodiment, any clustering method that can cluster candidate expansion questions into multiple clusters can be used, and no specific limitation is made here.

[0089] This embodiment filters candidate extended questions that correspond to the target standard question from multiple extended questions, and clusters the candidate extended questions into multiple clusters. Without any prior sample labels, the samples are divided into several categories based on the inherent relationships between the data, so that the similarity between samples of the same category is high and the similarity between samples of different categories is low. This makes it easier to filter out high-quality extended questions from similar samples and improves the screening efficiency.

[0090] Step 103: Determine the target expansion question in each cluster based on the second similarity between the candidate expansion question and the remaining standard questions.

[0091] In this embodiment of the invention, after clustering the candidate expansion questions into multiple clusters, in order to filter out high-quality expansion questions from the clusters, it is necessary to consider the second similarity between the candidate expansion questions and the remaining standard questions, and determine the target expansion question from the retained candidate expansion questions.

[0092] It should be noted that the candidate extended questions in each cluster are compared with all other remaining standard questions. For example, the candidate extended questions and multiple remaining standard questions are input into the semantic similarity model, and the similarity between the candidate extended questions and each remaining standard question is compared. If the difference between the similarity between the candidate extended question and the target standard question and the maximum similarity between the candidate extended question and each remaining standard question is greater than a preset threshold, the candidate extended question is retained. Finally, the candidate extended question with the highest similarity among the retained candidate extended questions is selected as the target extended question. This ensures that the selected target extended questions are diverse and less prone to confusion, resulting in fluent and coherent sentences and improving the quality of the extended questions. The preset threshold is pre-set according to the system's requirements for the quality testing of extended questions. In this embodiment, the preset threshold can be 0.1 or 0.15, and is not specifically limited here.

[0093] Further, referring to Figure 2, which is a flowchart of step 103 of the extended question filtering method provided in the embodiment of the present invention in Figure 1, step 103 includes:

[0094] Step 1031: Compare the candidate extended question with the remaining standard questions in each cluster, and determine the maximum similarity between the candidate extended question and the remaining standard questions as the second similarity.

[0095] In this embodiment of the invention, the candidate extended questions in each cluster are compared with the remaining standard questions, and the maximum similarity between the candidate extended questions and the remaining standard questions is determined as the second similarity, so that the candidate extended questions are both associated with the standard questions and distinguished from other standard questions, and high-quality candidate extended questions are selected and retained.

[0096] Specifically, the candidate expanded question and multiple remaining standard questions are input into the sentence semantic similarity model, the similarity between the candidate expanded question and each remaining standard question is compared, and the maximum similarity between the candidate expanded question and the remaining standard questions is determined as the second similarity.

[0097] Step 1032: If the difference between the first similarity and the second similarity is greater than the first preset threshold, the candidate extended question is determined to be the target extended question in the cluster.

[0098] In this embodiment, if the difference between the first similarity between the candidate extended question and the target standard question and the second similarity between the candidate extended question and the maximum value of the similarity between each remaining standard question is greater than the first preset threshold, then the candidate extended question is retained. Finally, the candidate extended question with the highest first similarity among the retained candidate extended questions is taken as the target extended question, thereby making the selected target extended questions diverse and less prone to confusion, with fluent and coherent sentences, thus improving the quality of the extended questions.

[0099] This invention, through obtaining a standard question and a plurality of randomly generated extended questions, wherein the standard question includes a target standard question and a plurality of remaining standard questions, selects candidate extended questions corresponding to the target standard question from the plurality of extended questions, and clusters the candidate extended questions into a plurality of clusters. The first similarity between the candidate extended question and the target standard question is within a first preset range. Based on the second similarity between the candidate extended question and the remaining standard questions, the target extended question in each cluster is determined. This invention avoids manual processing of extended sentences by obtaining a large number of automatically and randomly generated extended questions, saving labor costs. It uses clustering to determine target extended questions that are related to but different from the target standard question from the selected candidate extended questions, resulting in diverse and less confusing target extended questions, fluent and coherent sentences, and improved quality of extended questions. Furthermore, by using similarity filtering and batch clustering to determine the target extended questions that meet the conditions from a large number of extended questions, it meets the needs of complex interactive systems to select high-quality extended questions from complex and numerous extended questions, further improving the user's human-computer interaction experience.

[0100] Referring to Figure 3, a second flowchart of the extended question filtering method provided in an embodiment of the present invention is shown. The method may include:

[0101] Step 104: Collect standard questions, input the standard questions into the pre-trained first generative model, and randomly generate multiple extended questions corresponding to the standard questions; or,

[0102] Step 105: Input the standard question and the first extended question into the first generative model, and randomly generate multiple extended questions corresponding to the standard question, wherein the first extended question is pre-annotated based on the standard question.

[0103] It should be noted that in steps 104 and 105 above, the large number of extended questions generated need to be screened. A small number of extended questions that are different from each other and of high quality are selected and added to the knowledge base. There are two situations for screening extended questions: one is that the input of the generation model consists entirely of standard questions, and the other is that the input of the generation model includes a small number of extended questions in addition to standard questions.

[0104] Therefore, in this embodiment, a standard question is first collected and input into a pre-trained first generative model to randomly generate multiple extended questions corresponding to the standard question; or, the standard question and the first extended question are input into the first generative model to randomly generate multiple extended questions corresponding to the standard question, wherein the first extended question is pre-labeled based on the standard question. The purpose is to obtain a large number of randomly generated extended questions for further filtering.

[0105] Step 101: Obtain the standard question and multiple randomly generated extended questions, wherein the standard question includes the target standard question and multiple remaining standard questions.

[0106] Step 102: Select candidate extended questions that correspond to the target standard question from multiple extended questions, and cluster the candidate extended questions into multiple clusters, wherein the first similarity between the candidate extended questions and the target standard question is within a first preset range.

[0107] Step 103: Determine the target expansion question in each cluster based on the second similarity between the candidate expansion question and the remaining standard questions.

[0108] Steps 101 to 103 above are the same as those described above, and will not be repeated here.

[0109] Compared to existing technologies, this embodiment can automatically and randomly generate a large number of extended questions using a pre-trained text generation model, saving the cost of manually processing extended questions.

[0110] To enable those skilled in the art to more clearly understand the extended question filtering method described above, please refer to Figure 4, which is a flowchart of the extended question filtering method provided in the embodiment of the present invention in Figure 1. The following description takes the example of multiple extended questions being randomly generated based on standard questions:

[0111] S201, obtain the target standard question and several randomly generated extended questions;

[0112] S202, obtain the first similarity between multiple extended questions and the target standard question;

[0113] S203, filter out the extended questions with the first similarity within the first preset range as candidate extended questions;

[0114] S204, cluster the candidate expansion question into multiple clusters;

[0115] S205, obtain the second similarity between the candidate extended question and the remaining standard questions;

[0116] S206, if the difference between the first similarity and the second similarity of the candidate extended question is greater than a preset threshold, retain the candidate extended question;

[0117] S207, the candidate expansion question with the highest similarity among the retained candidates is determined as the target expansion question.

[0118] This invention, through the acquisition of a large number of automatically and randomly generated extended questions, avoids manual processing of extended sentences, saving labor costs. It utilizes clustering to identify target extended questions that are related to but differ from the target standard question from a pool of candidate extended questions. This results in diverse and less confusing target extended questions, with fluent and coherent sentences, thus improving the quality of the extended questions. Furthermore, by using similarity filtering and batch clustering, it identifies target extended questions that meet the criteria from a large number of extended questions, satisfying the need of complex interactive systems to filter high-quality extended questions from a large number of complex content, further enhancing the user's human-computer interaction experience.

[0119] Referring to Figure 5, which is a flowchart of the first generative model training process of the extended question filtering method provided in the embodiment of the present invention in Figure 1, including:

[0120] Step 301: Obtain the standard question and the extended questions associated with the standard question.

[0121] In this embodiment of the invention, in order to optimize the text generation model on standard questions and extended question data, and enable the model to randomly generate extended questions based on standard questions, it is necessary to obtain standard questions and extended questions associated with standard questions in the historical corpus in advance as training data to optimize and train the text generation model.

[0122] It should be noted that this embodiment does not specifically limit the text generation model; it can be any neural network model that can automatically generate text, and will not be described in detail here.

[0123] Step 302: Input the standard question into the text generation model to randomly generate the actual extended question.

[0124] Step 303: Determine the contrast loss of the extended question associated with the actual extended question and the standard question.

[0125] In this embodiment, standard questions and extended questions are used to fine-tune the text generation model. A standard question is given, and the text generation model outputs the corresponding actual extended question. The contrast loss between the actual extended question and the extended question associated with the standard question is determined.

[0126] Step 304: Adjust the network parameters of the text generation model according to the contrast loss to obtain the pre-trained first generation model.

[0127] In this embodiment, the text generation model is iteratively trained based on the comparative loss between the actual extended question and the extended question associated with the standard question. By adjusting the network parameters of the text generation model, the loss between the actual extended question and the extended question corresponding to the standard question can be ignored. This allows the extended questions corresponding to the standard question to be selected from a batch of extended question samples, thus obtaining a pre-trained first generation model. Specifically, the decoding method of the extended questions generated by the first generation model adopts random decoding. A certain number of sentence beginnings are given according to the usage scenario, and characters are added to the beginning of the sentences, such as: how, how, is, etc. The optimized first generation model is used to randomly generate a large number of extended questions related to the standard question.

[0128] This invention relates to an extended question filtering method, which is basically the same as the extended question filtering method provided in the first embodiment. The difference is that, as shown in Figure 6, if multiple extended questions are randomly generated based on the standard question and the first extended question, after step 101, the method further includes:

[0129] Step 106: Select candidate extended questions from multiple extended questions and cluster the candidate extended questions into multiple clusters, wherein the first similarity between the candidate extended questions and the target standard question is within a second preset range.

[0130] In this embodiment, the target standard question and multiple extended questions are input into the semantic similarity model. The similarity between each extended question and the target standard question is compared. Extended questions with similarity within a second preset range are retained as candidate extended questions, while the remaining extended questions are filtered out or stored for later use. The second preset range is pre-set according to the system's quality testing requirements for extended questions and is not specifically limited here. In this embodiment, the second preset range can be between 0.6 and 0.85. The candidate extended questions are clustered into multiple clusters using the K-means clustering algorithm.

[0131] Step 107: Determine the target expansion question in each cluster based on the average similarity between the candidate expansion question, the standard question, and the first expansion question.

[0132] In this embodiment, after clustering the candidate expansion questions into multiple clusters, in order to filter out high-quality expansion questions from the clusters, it is necessary to consider the average similarity between the candidate expansion questions, the standard question, and the first expansion question, and determine the target expansion question from the retained candidate expansion questions.

[0133] Specifically, step 107 may include the following steps:

[0134] First, the candidate extended question in each cluster is compared with the target standard question and the first extended question to determine the first average similarity between the candidate extended question and the target standard question and the first extended question.

[0135] Secondly, the candidate extended question is compared with the remaining standard questions and the first extended question to determine the second average similarity between the candidate extended question and the remaining standard questions and the first extended question.

[0136] Secondly, if the first average similarity is greater than the second average similarity, the candidate extended question is retained;

[0137] Finally, the candidate expansion question with the highest average similarity among the retained candidates was identified as the target expansion question in the cluster.

[0138] For example, the candidate extended question in each cluster is compared with all standard questions and the pre-labeled first extended question in the knowledge base to generate the knowledge (standard question + extended question) of the candidate extended question as set P, and the other knowledge (standard question + extended question) in the knowledge base as set Q1, Q2... If the first average similarity between the candidate extended question and all sentences in P is greater than the second average similarity between the candidate extended question and sentences in Q1, Q2..., then the candidate extended question is retained as a candidate, and the one with the highest first average similarity is selected as the target extended question in the cluster.

[0139] In this embodiment, a screening method is designed to improve the quality of the final selected extended questions, taking into account the characteristic of the first generation model randomly generating extended questions. This results in the final selected extended questions having greater diversity, making it less likely for knowledge points to be confused, and ensuring that the sentences are fluent and coherent.

[0140] Of course, it should be noted that the target expansion question filtering method described in this embodiment differs from the filtering method described in the first embodiment in terms of similarity comparison, and there is no order between the two. This embodiment is only used as an example where multiple expansion questions are randomly generated based on the standard question and the first expansion question.

[0141] It should be noted that this embodiment does not limit the specific execution location of steps 106 and 107. In this embodiment, for ease of understanding, steps 106 and 107 are described as being executed after step 101 and before step 102. In actual use, steps 106 and 107 can also be completed as a single step after step 103. Each case will not be described in detail here.

[0142] Referring to Figure 7, to enable those skilled in the art to more clearly understand the extended question filtering method described above, Figure 7 is a flowchart of the extended question filtering method provided in the embodiment of the present invention in Figure 6. The following description takes the example of multiple extended questions being randomly generated based on a standard question and a first extended question:

[0143] S401, obtain the target standard question and several randomly generated extended questions;

[0144] S402, obtain the first similarity between multiple extended questions and the target standard question;

[0145] S403, filter out the extended questions with the first similarity within the second preset range as candidate extended questions;

[0146] S404 clusters the candidate expansion question into multiple clusters;

[0147] S405, obtain the first average similarity between the candidate extended question and the target standard question and the first extended question;

[0148] S406, obtain the second average similarity between the candidate extended question and the remaining standard questions and the first extended question;

[0149] S407, If the first average similarity is greater than the second average similarity, retain the candidate extended question;

[0150] S408, the candidate expansion question with the highest similarity among the retained candidates is determined as the target expansion question.

[0151] In this embodiment, a filtering method is designed to improve the quality of the final selected extended questions, taking into account the random generation of extended questions by the first generative model. This results in a greater diversity of the final selected extended questions, making it less likely for knowledge points to be confused, and ensuring fluent and coherent sentences. Similarity filtering and batch clustering are used to identify target extended questions that meet the criteria from a large number of extended questions, satisfying the need of complex interactive systems to filter high-quality extended questions from a large number of complex questions, further enhancing the user's human-computer interaction experience.

[0152] Referring to Figure 8, a schematic diagram of the structure of the extended screening device provided in an embodiment of the present invention is shown. The device may include:

[0153] The first acquisition module 501 is used to acquire a standard question and a plurality of randomly generated extended questions, wherein the standard question includes a target standard question and a plurality of remaining standard questions;

[0154] The first filtering module 502 is used to filter candidate extended questions corresponding to the target standard question from a plurality of extended questions, and to cluster the candidate extended questions into a plurality of clusters, wherein the first similarity between the candidate extended questions and the target standard question is within a first preset range;

[0155] The first determining module 503 is used to determine the target extended question in each of the clusters based on the second similarity between the candidate extended questions and the remaining standard questions.

[0156] Furthermore, the first screening module 502 includes:

[0157] The first acquisition submodule is used to acquire the first similarity between the multiple extended questions and the target standard question respectively;

[0158] The filtering submodule is used to filter the extended questions corresponding to the first similarity within a first preset range to obtain candidate extended questions corresponding to the target standard question;

[0159] The clustering submodule is used to cluster the candidate expansion questions into multiple clusters using a preset clustering algorithm.

[0160] Furthermore, the first determining module 503 includes:

[0161] The first determining submodule is used to compare the candidate extended question in each of the clusters with the remaining standard questions respectively, and determine the maximum similarity between the candidate extended question and the remaining standard questions as the second similarity.

[0162] The first judgment submodule is used to retain the candidate expansion question if the difference between the first similarity and the second similarity is greater than a first preset threshold.

[0163] The second determining submodule is used to determine the candidate extension question with the highest similarity among the retained candidate extension questions as the target extension question in the cluster.

[0164] Furthermore, the device also includes:

[0165] The first generation module is used to collect standard questions, input the standard questions into a pre-trained first generation model, and randomly generate multiple extended questions corresponding to the standard questions; or...

[0166] The second generation module is used to input the standard question and the first extended question into the first generation model and randomly generate multiple extended questions corresponding to the standard question, wherein the first extended question is pre-annotated based on the standard question.

[0167] Furthermore, the training of the first generative model includes:

[0168] The acquisition unit is used to acquire the standard question and the extended questions associated with the standard question;

[0169] A generation unit is used to input the standard question into the text generation model and randomly generate an actual extended question.

[0170] A determining unit is used to determine the contrast loss of the extended question associated with the actual extended question and the standard question;

[0171] The training unit is used to adjust the network parameters of the text generation model according to the contrast loss to obtain a pre-trained first generation model.

[0172] Furthermore, if the plurality of extended questions are randomly generated based on the standard question and the first extended question, the apparatus further includes:

[0173] The second filtering module is used to filter candidate extended questions from multiple extended questions and cluster the candidate extended questions into multiple clusters, wherein the first similarity between the candidate extended questions and the target standard question is within a second preset range;

[0174] The second determining module is used to determine the target extended question in each of the clusters based on the average similarity between the candidate extended question and the standard question and the first extended question.

[0175] Furthermore, the second determining module includes:

[0176] The third determining submodule is used to compare the candidate extended question in each of the clusters with the target standard question and the first extended question respectively, and determine the first average similarity between the candidate extended question and the target standard question and the first extended question;

[0177] The fourth determining submodule is used to compare the candidate extended question with the remaining standard questions and the first extended question respectively, and determine the second average similarity between the candidate extended question and the remaining standard questions and the first extended question;

[0178] The second judgment submodule is used to retain the candidate expansion question if the first average similarity is greater than the second average similarity.

[0179] The fifth determining submodule is used to determine the candidate extension question with the highest average similarity among the retained candidate extension questions as the target extension question in the cluster.

[0180] This invention, through obtaining a standard question and a plurality of randomly generated extended questions, wherein the standard question includes a target standard question and a plurality of remaining standard questions, selects candidate extended questions corresponding to the target standard question from the plurality of extended questions, and clusters the candidate extended questions into a plurality of clusters. The first similarity between the candidate extended question and the target standard question is within a first preset range. Based on the second similarity between the candidate extended question and the remaining standard questions, the target extended question in each cluster is determined. This invention avoids manual processing of extended sentences by obtaining a large number of automatically and randomly generated extended questions, saving labor costs. It uses clustering to determine target extended questions that are related to but different from the target standard question from the selected candidate extended questions, resulting in diverse and less confusing target extended questions, fluent and coherent sentences, and improved quality of extended questions. Furthermore, by using similarity filtering and batch clustering to determine the target extended questions that meet the conditions from a large number of extended questions, it meets the needs of complex interactive systems to select high-quality extended questions from complex and numerous extended questions, further improving the user's human-computer interaction experience.

[0181] This invention also provides an electronic device, as shown in FIG9, including a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0182] Memory 603 is used to store computer programs;

[0183] When processor 601 executes the program stored in memory 603, it can perform the following steps:

[0184] Obtain a standard question and multiple randomly generated extended questions, wherein the standard questions include a target standard question and multiple remaining standard questions; filter candidate extended questions corresponding to the target standard question from the multiple extended questions, and cluster the candidate extended questions into multiple clusters, wherein the first similarity between the candidate extended questions and the target standard question is within a first preset range; determine the target extended question in each cluster based on the second similarity between the candidate extended questions and the remaining standard questions. 。

[0185] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0186] The communication interface is used for communication between the aforementioned terminal and other devices.

[0187] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0188] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0189] The present invention also provides a readable storage medium, wherein when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is able to perform the extended query filtering method of the foregoing embodiments.

[0190] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0191] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. The structure required to construct such a system is readily apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0192] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0193] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed invention requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0194] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0195] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the sorting device according to the present invention. The present invention can also be implemented as a device or apparatus program for performing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0196] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0197] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0198] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention. However, the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should also be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0199] It should be noted that the various data-related processes in the embodiments of this application are carried out in compliance with the relevant data protection laws and policies of the country where the location is located, and with the authorization granted by the owner of the corresponding device.

Claims

1. An extended question filtering method, characterized in that, The method includes: collecting a standard question; inputting the standard question into a pre-trained first generative model to randomly generate multiple extended questions corresponding to the standard question; or, inputting the standard question and a first extended question into the first generative model to randomly generate multiple extended questions corresponding to the standard question, wherein the first extended question is pre-annotated based on the standard question; obtaining the standard question and multiple randomly generated extended questions, wherein the standard question includes a target standard question and multiple remaining standard questions; if the multiple extended questions are randomly generated based on the standard question and the first extended question, selecting candidate extended questions from the multiple extended questions, and then... The candidate extended questions are clustered into multiple clusters, wherein the first similarity between the candidate extended questions and the target standard questions is within a second preset range; the target extended question in each cluster is determined based on the average similarity between the candidate extended questions and the standard questions and the first extended questions; candidate extended questions corresponding to the target standard questions are selected from the multiple extended questions, and the candidate extended questions are clustered into multiple clusters, wherein the first similarity between the candidate extended questions and the target standard questions is within a first preset range; the target extended question in each cluster is determined based on the second similarity between the candidate extended questions and the remaining standard questions.

2. The method according to claim 1, characterized in that, The step of selecting candidate extended questions corresponding to the target standard question from multiple extended questions and clustering the candidate extended questions into multiple clusters includes: obtaining a first similarity between multiple extended questions and the target standard question; selecting extended questions corresponding to the first similarity within a first preset range to obtain candidate extended questions corresponding to the target standard question; and clustering the candidate extended questions into multiple clusters using a preset clustering algorithm.

3. The method according to claim 1, characterized in that, The step of determining the target expansion question in each cluster based on the second similarity between the candidate expansion question and the remaining standard questions includes: comparing the candidate expansion question in each cluster with the remaining standard questions respectively, and determining the maximum similarity between the candidate expansion question and the remaining standard questions as the second similarity; if the difference between the first similarity and the second similarity is greater than a first preset threshold, retaining the candidate expansion question; and determining the candidate expansion question with the highest first similarity among the retained candidate expansion questions as the target expansion question in the cluster.

4. The method according to claim 1, characterized in that, The training of the first generative model includes: obtaining a standard question and an extended question associated with the standard question; inputting the standard question into a text generation model to randomly generate actual extended questions; determining the contrast loss between the actual extended questions and the extended questions associated with the standard question; and adjusting the network parameters of the text generation model according to the contrast loss to obtain a pre-trained first generative model.

5. The method according to claim 1, characterized in that, The step of determining the target extended question in each cluster based on the average similarity between the candidate extended question, the standard question, and the first extended question includes: comparing the candidate extended question in each cluster with the target standard question and the first extended question respectively to determine a first average similarity between the candidate extended question, the target standard question, and the first extended question; comparing the candidate extended question with the remaining standard questions and the first extended question respectively to determine a second average similarity between the candidate extended question, the remaining standard questions, and the first extended question; if the first average similarity is greater than the second average similarity, retaining the candidate extended question; and determining the candidate extended question with the highest first average similarity among the retained candidate extended questions as the target extended question in the cluster.

6. An extended screening device, characterized in that, The apparatus includes: a first generation module, configured to collect a standard question, input the standard question into a pre-trained first generation model, and randomly generate multiple extended questions corresponding to the standard question; or, a second generation module, configured to input the standard question and a first extended question into the first generation model, and randomly generate multiple extended questions corresponding to the standard question, wherein the first extended question is pre-annotated based on the standard question; an acquisition module, configured to acquire the standard question and the multiple randomly generated extended questions, wherein the standard question includes a target standard question and multiple remaining standard questions; if the multiple extended questions are randomly generated based on the standard question and the first extended question, a second filtering module, configured to filter candidate extended questions from the multiple extended questions. The system first determines the target extended question by clustering the candidate extended questions into multiple clusters, wherein the first similarity between the candidate extended questions and the target standard question is within a second preset range; a second determining module is used to determine the target extended question in each cluster based on the average similarity between the candidate extended questions and the standard question and the first extended question; a filtering module is used to filter candidate extended questions corresponding to the target standard question from the multiple extended questions, and cluster the candidate extended questions into multiple clusters, wherein the first similarity between the candidate extended questions and the target standard question is within a first preset range; and a determining module is used to determine the target extended question in each cluster based on the second similarity between the candidate extended questions and the remaining standard questions.

7. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the program, it implements the extended query filtering method as described in any one of claims 1-5.

8. A readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the extended query filtering method according to any one of claims 1-5.

Images