Question mining methods, devices and equipment in question-answering models

Abstract

This application discloses a method, apparatus, and device for mining questions in a question-answering model, relating to the field of digital healthcare, which can solve the problem of low efficiency in question mining in question-answering models. The method includes: acquiring historical sample questions; initially grouping the historical sample questions into multiple first groups, wherein the distance between historical sample questions within each first group is less than a first preset threshold; further grouping the historical sample questions within each first group using a clustering algorithm to obtain multiple second groups, wherein the distance between historical sample questions within each second group is less than a second preset threshold, and the second preset threshold is less than a first preset threshold; calculating the distance between the second groups; merging the second groups with distances less than the second preset threshold to obtain merged second groups; and determining all merged second groups as the target question set required to build the question-answering model.

Description

Technical Field

[0001] This application relates to the field of digital healthcare, and in particular to a method, apparatus, and device for mining questions in a question-answering model. Background Technology

[0002] Question-answering models are a type of retrieval model that finds a question that matches a user's question in an existing question-answer mapping and returns the corresponding answer to the user. For example, medical question-answering models help doctors answer patients' questions and improve diagnostic efficiency, while financial question-answering models help staff provide services to financial users and improve service efficiency.

[0003] Before building a question-answering model, it is necessary to extract similar questions from a large number of questions into a question set. Currently, the similarity between pairs of questions is calculated using a Siamese network model (the specific process is to input question 1 and question 2 into two sub-network models in the Siamese network model, and the two sub-network models extract the feature information of the corresponding questions and calculate the similarity based on the extracted feature information). However, when processing a large amount of data, calculating the similarity between pairs of questions will result in high resource consumption and low efficiency. Summary of the Invention

[0004] In view of this, this application provides a method, apparatus and device for mining questions in a question-answering model, relating to the field of digital healthcare, which can solve the problem of low efficiency in question mining in question-answering models.

[0005] According to one aspect of this application, a method for mining questions in a question-answering model is provided, the method comprising:

[0006] Historical sample questions are obtained, and the historical sample questions are initially grouped to obtain multiple first groups, wherein the distance between the historical sample questions in the first group is less than a first preset threshold.

[0007] The historical sample questions within each of the first groups are regrouped using a clustering algorithm to obtain multiple second groups. The distance between the historical sample questions within each second group is less than a second preset threshold, and the second preset threshold is less than the first preset threshold.

[0008] Calculate the distance between the second groups, and merge the second groups that are less than the second preset threshold to obtain the merged second group;

[0009] The second group, after merging all the above, is identified as the target question set required to build the question-answering model.

[0010] Preferably, the initial grouping of the historical sample questions to obtain multiple first groups, wherein the distance between the historical sample questions within the first group is less than a first preset threshold, includes:

[0011] Calculate the distance between the historical sample questions, and group the historical sample questions whose distance is less than a first preset threshold into a first-level initial group;

[0012] Calculate the text length of the historical sample questions within the first initial grouping, and then group the text lengths into a second initial grouping according to different preset length intervals to obtain multiple first groups. The distance between the historical sample questions in the first group is less than a first preset threshold and the text lengths are within the same preset length interval.

[0013] Preferably, before addressing the issue of obtaining historical samples, the method further includes:

[0014] Obtain historical sample messages and extract the questions from the historical sample messages;

[0015] The first question is identified as having identical text and a quantity greater than a preset value, and this first question is designated as the second group.

[0016] If a second question is found to have the same text and a number less than or equal to the preset value, the duplicate questions in the first question and the second question are deleted to obtain historical sample questions.

[0017] Preferably, the step of calculating the distance between the second groups, and merging the second groups that are less than the second preset threshold, includes:

[0018] Determine the representative problem from any historical sample problem in the second group;

[0019] Calculate the distance between the representative questions. If the distance between the representative questions is less than the second preset threshold, then combine the second group into one group.

[0020] Preferably, after determining the second group after merging all the aforementioned items as the target question set required to build the question-answering model, the method further includes:

[0021] In the merged second group, select any one to be ranked first, determine the merged second group that is closest to the first group and rank it second, determine the merged second group that is closest to the second group and rank it third, and so on until all the merged second groups are sorted.

[0022] Preferably, it also includes:

[0023] From each of the merged second groups, a standard problem is determined;

[0024] Obtain the standard answer corresponding to each of the standard questions, and establish a mapping between the standard questions and the standard answers.

[0025] According to another aspect of this application, a question-mining apparatus for a question-answering model is provided, the apparatus comprising:

[0026] The initial grouping module is used to obtain historical sample questions, perform initial grouping on the historical sample questions to obtain multiple first groups, wherein the distance between the historical sample questions in the first group is less than a first preset threshold.

[0027] The regrouping module is used to regroup the historical sample questions within each first group using a clustering algorithm to obtain multiple second groups, wherein the distance between the historical sample questions within each second group is less than a second preset threshold, and the second preset threshold is less than the first preset threshold.

[0028] The merging module is used to calculate the distance between the second groups, merge the second groups that are less than the second preset threshold, and obtain the merged second group.

[0029] The determination module is used to identify all the merged second group as the target question set required to build the question-answering model.

[0030] Preferably, the device further includes: a preprocessing module, used to acquire historical sample messages, extract questions from the historical sample messages, determine a first question with identical text and a number greater than a preset value, take the first question as a second group, determine a second question with identical text and a number less than or equal to the preset value, delete duplicate questions in the first question and the second question, and obtain historical sample questions.

[0031] According to another aspect of the present invention, a storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the question mining method in the above-described question-answering model.

[0032] According to another aspect of the present invention, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the question mining method in the above-described question-answering model.

[0033] Using the above technical solution, this application discloses a method, apparatus, and device for mining questions in a question-answering model. First, historical sample questions are acquired and initially grouped to obtain multiple first groups, where the distance between historical sample questions within each first group is less than a first preset threshold. Then, a clustering algorithm is used to further group the historical sample questions within each first group to obtain multiple second groups, where the distance between historical sample questions within each second group is less than a second preset threshold, and the second preset threshold is less than a first preset threshold. Further, the distance between the second groups is calculated, and the second groups with distances less than the second preset threshold are merged to obtain merged second groups. Finally, all merged second groups are determined as the target question set required to build the question-answering model. The technical solution in this application involves initially grouping historical sample problems into multiple first groups, and then clustering the historical sample problems within each first group. The computational cost of clustering is only the number of historical sample problems within the group (to improve the clustering speed, the clustering algorithm can be used once for each historical sample problem within each group, and all first groups can be clustered in parallel). Compared to directly calculating the similarity of all historical sample problems, this method greatly saves computational cost and improves efficiency.

[0034] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0035] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0036] Figure 1 This illustration shows a flowchart of a question mining method in a question-answering model provided in an embodiment of this application;

[0037] Figure 2 This paper illustrates a flowchart of a question mining method in another question-answering model provided in an embodiment of this application.

[0038] Figure 3 The present application provides a schematic diagram of the structure of a question-finding device in a question-answering model;

[0039] Figure 4 A schematic diagram of the structure of a question-mining device in another question-answering model provided in this application embodiment is shown. Detailed Implementation

[0040] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the present application can be combined with each other.

[0041] It should be noted that the question-answering model includes multiple sets of historical sample questions and the corresponding answers for each set of historical sample questions. Since users often ask the same questions, building a question-answering model helps users obtain answers and improves efficiency. Each set of historical sample questions expresses the same semantics, only the wording is different.

[0042] To address the current problems, embodiments of this application provide a method for mining questions in a question-answering model, such as... Figure 1 As shown, the method includes:

[0043] 101. Obtain historical sample questions, and perform initial grouping of the historical sample questions to obtain multiple first groups, wherein the distance between the historical sample questions in the first group is less than a first preset threshold.

[0044] The historical sample questions are obtained from multiple pairs of real dialogues. In specific application scenarios, the medical question-answering model corresponds to multiple pairs of real dialogues between patients and doctors (such as a patient with atrial fibrillation asking a doctor a question and the doctor answering the question), while the financial question-answering model corresponds to multiple pairs of real dialogues between financial users and staff (such as a financial user with loan needs asking a staff member a question and the staff member answering the question).

[0045] Specifically, the dialogue includes personal information, historical sample questions, response text, and stop words (such as thank you and hello). Historical sample questions are obtained from historical sample messages. In contrast, existing twin networks do not learn from real common questions. Therefore, the questions mined in this application are more realistic, and the resulting question-answering model is more practical.

[0046] For example, the historical sample questions are 1, 2, 3, 4, 5, 6, 7, 8, 9. After initial grouping, multiple first groups are obtained: [1, 2, 4, 6], [3, 7, 9], [5, 8]. Among them, the distance between the historical sample questions within each first group is less than a first preset threshold, while the distance between the historical sample questions between each first group is greater than or equal to the first preset threshold. For example, the distance between historical sample question 1 and historical sample question 2 is less than the first preset threshold, while the distance between historical sample question 1 and historical sample question 3 is greater than or equal to the first preset threshold.

[0047] Since historical sample problems are characterized by large amounts of data, and the clustering algorithm has a complexity of O(N^2), directly processing large amounts of data would be very time-consuming. Therefore, initial grouping of historical sample problems can significantly reduce the time consumption of clustering. Although the semantics of historical sample problems in the first group cannot be guaranteed to be completely identical, the semantics of historical sample problems in different first groups can be significantly different.

[0048] Specifically, since this is only the initial grouping, historical sample questions within the first group may contain semantically completely different questions that have been grouped together. In a specific application scenario (such as atrial fibrillation), a historical sample question within the first group might be: What causes atrial fibrillation? and What is atrial fibrillation? The distance between "What causes atrial fibrillation?" and "What is atrial fibrillation?" is less than a first preset threshold. In a specific application scenario (such as corporate loans), a historical sample question within the first group might be: What is a corporate interest-subsidized loan? and How much is the corporate interest-subsidized loan? The distance between "What is a corporate interest-subsidized loan?" and "How much is the corporate interest-subsidized loan?" is less than a first preset threshold. Therefore, it is necessary to regroup the historical sample questions within the first group after the initial grouping using a clustering algorithm.

[0049] 102. Using a clustering algorithm, the historical sample questions within each first group are regrouped to obtain multiple second groups. The distance between the historical sample questions within each second group is less than a second preset threshold, and the second preset threshold is less than a first preset threshold.

[0050] In this embodiment, a clustering algorithm is used to regroup each first group to obtain multiple second groups. Historical sample questions within each second group have the same semantic meaning. In a specific application scenario (such as atrial fibrillation), the historical sample questions within a certain second group are: What causes atrial fibrillation? What causes atrial fibrillation? What leads to atrial fibrillation? What triggers atrial fibrillation? Where the distance between "What causes atrial fibrillation?", "What leads to atrial fibrillation?", "What triggers atrial fibrillation?", and "What triggers atrial fibrillation?" is less than a second preset threshold. In a specific application scenario (such as corporate loans), the historical sample questions within a certain second group are: When is the application period for corporate interest-subsidized loans? What are the application areas for corporate interest-subsidized loans? What are the application conditions for corporate interest-subsidized loans? Where the distance between "When is the application period for corporate interest-subsidized loans?", "What are the application areas for corporate interest-subsidized loans?", and "What are the application conditions for corporate interest-subsidized loans?" is less than a second preset threshold.

[0051] 103. Calculate the distance between the second groups, and merge the second groups that are less than the second preset threshold to obtain the merged second group.

[0052] 104. Determine all merged second groups as the target question set needed to build the question-answering model.

[0053] It should be noted that since the historical sample questions within the first group may have different semantics, when such a first group is regrouped, multiple second groups are obtained. The historical sample questions within these second groups are semantically identical, but there may be semantically identical historical sample questions between these second groups. In order to ensure the accuracy of question mining, that is, the historical sample questions within the target question set required to build the question answering model are all semantically identical, and the historical sample questions between target question sets are all semantically different, it is necessary to merge several semantically identical second groups. Among them, building the question answering model requires the target question set and the standard answer corresponding to the target question set.

[0054] In specific application scenarios (such as atrial fibrillation), one set of historical sample questions within the first group is: What to eat for atrial fibrillation? What medications to take for atrial fibrillation? How to take medication for atrial fibrillation? Another set of historical sample questions within the first group is: What dietary precautions should be taken for atrial fibrillation patients? What are the dietary restrictions for atrial fibrillation?

[0055] When the first group is subdivided, two second groups are obtained. One second group is: What medications are used for atrial fibrillation? How should medications for atrial fibrillation be taken? The other second group is: What should one eat for atrial fibrillation? When the second group is subdivided, one second group is: What dietary precautions should be taken for atrial fibrillation? What are the dietary restrictions for atrial fibrillation?

[0056] The second group, "What to eat for atrial fibrillation?" and "What dietary precautions should be taken for atrial fibrillation? What are the dietary restrictions for atrial fibrillation?", are semantically identical. These two semantically identical second groups can be merged. The result of merging these three second groups is: Second Group 1: What medications are taken for atrial fibrillation? How should medications be taken for atrial fibrillation? Second Group 2: What to eat for atrial fibrillation? What dietary precautions should be taken for atrial fibrillation? What are the dietary restrictions for atrial fibrillation?

[0057] In specific application scenarios (such as corporate loans), historical sample questions within one group are: What are the applicants for corporate interest-subsidized loans? What are the application areas for corporate interest-subsidized loans? Can I apply for a corporate interest-subsidized loan? Can I apply for a corporate interest-subsidized loan now? Another set of historical sample questions within group one is: When does the corporate interest-subsidized loan start? When is the application period for a corporate interest-subsidized loan? What is the application deadline for a corporate interest-subsidized loan?

[0058] Subgrouping the first group yields two second groups. One second group addresses: What are the applicants for subsidized corporate loans? What are the application areas for subsidized corporate loans? Can I apply for a subsidized corporate loan? The other second group addresses: Can I apply for a subsidized corporate loan now? Subgrouping the second group yields: When does the subsidized corporate loan program start? When can I apply for a subsidized corporate loan? What is the application period for a subsidized corporate loan?

[0059] The second set of questions, "Can I apply for a subsidized business loan now?" and "When does the subsidized business loan start?", and "When can I apply for a subsidized business loan?", are semantically identical. Therefore, these two sets of questions can be merged. The result of merging these three sets is: Second Set 1: What are the applicants for subsidized business loans? What are the application areas for subsidized business loans? Can I apply for a subsidized business loan? Second Set 2: Can I apply for a subsidized business loan now? When does the subsidized business loan start? When can I apply for a subsidized business loan? What is the application period for subsidized business loans?

[0060] This application discloses a method, apparatus, and device for mining questions in a question-answering model. First, historical sample questions are acquired and initially grouped to obtain multiple first groups, where the distance between historical sample questions within each first group is less than a first preset threshold. Then, a clustering algorithm is used to further group the historical sample questions within each first group, resulting in multiple second groups, where the distance between historical sample questions within each second group is less than a second preset threshold, and the second preset threshold is less than a first preset threshold. Further, the distances between the second groups are calculated, and the second groups with distances less than the second preset threshold are merged to obtain merged second groups. Finally, all merged second groups are determined as the target question set required to build the question-answering model. Through the technical solution in this application, by initially grouping historical sample questions into multiple first groups, and then clustering the historical sample questions within each first group, the computational cost of clustering is only the number of historical sample questions within a group (to improve clustering speed, the clustering algorithm can be used once for each historical sample question within a group, and all first groups can be clustered in parallel). Compared to directly calculating the similarity of all historical sample questions, this greatly saves computational cost and improves efficiency.

[0061] Furthermore, as a refinement and extension of the specific implementation methods of the above embodiments, in order to fully illustrate the specific implementation process of this embodiment, another method for mining questions in a question-answering model is provided, such as... Figure 2 As shown, the method includes:

[0062] 201. Obtain historical sample questions, calculate the distance between historical sample questions, and perform the first-level initial grouping of historical sample questions whose distance between them is less than a first preset threshold.

[0063] In this embodiment, in order to obtain historical sample questions, the historical sample messages need to be processed beforehand to obtain the historical sample questions. Specifically: obtain the historical sample messages, extract the questions in the historical sample messages, determine the first question with the same text and a number greater than a preset value, take the first question as the second group, determine the second question with the same text and a number less than or equal to a preset value, delete the duplicate questions in the first question and the second question, and obtain the historical sample questions.

[0064] Historical sample messages are the original data obtained, such as dialogue messages between patients and doctors on the platform, or dialogue messages between financial users with loan needs and staff at the window inquiring about handling business. Historical sample messages are usually short texts (first, short texts are generally less than 50 characters, because dialogue messages are back and forth, and usually one party will not say more than 100 words before the other party responds; second, short texts are concise, patients or financial users will ask questions in colloquial and summarizing language, while doctors or financial staff will reply with concise and knowledgeable language. Usually, neither party will speak at length, but what they say can cover professional knowledge).

[0065] Furthermore, questions are extracted from historical sample messages (key information such as "?", "is it?", "does it exist?", "is it not?", "what", etc. are considered questions). Only one question with identical text is retained. However, if a question appears frequently (is a frequently asked question), retaining only one of it will result in it not being assigned to the second group when the distance between it and other questions is greater than or equal to a second preset threshold. This would lead to it being incorrectly ignored, reducing the accuracy of question mining. Therefore, the first question with identical text and a number greater than a preset value is selected as the second group. Then, all first questions are deleted and not retained in the historical sample questions. For the second question with identical text and a number less than or equal to a preset value, only one is retained and placed in the historical sample questions to be processed.

[0066] After obtaining the historical sample questions, the first step is to perform initial grouping. This can be done using the simhash algorithm, a commonly used text deduplication hash algorithm. The simhash algorithm includes: First, segmenting each historical sample question into multiple word vectors and assigning a weight to each word vector (a larger weight indicates higher importance of the word vector in the sentence); Second, mapping the word vectors corresponding to each historical sample question using a hash function to generate a binary string for each word vector (e.g., a binary string corresponding to a word vector might be 100101); Third, multiplying the 1s in the binary string by the weight corresponding to that word vector, and multiplying the 0s by the weight corresponding to that word vector, and then taking the negative of the result, to obtain the weight vector corresponding to each word vector. If the weight is 3, then the weight vector is [3, -3, -3, 3, -3, 3]. Then, the weight vectors of all word vectors in the historical sample question are summed to obtain a new weight vector, for example: [3, 4, 1, 5, -5, 1]. Fourth, the dimensionality of each merged weight vector is reduced, i.e., positions greater than 0 are 1 and positions less than 0 are 0, which gives the simhash value of the historical sample question. Correspondingly, [3, 4, 1, 5, -5, 1] ​​can be converted to the simhash value [1, 1, 1, 1, 0, 1]. Fifth, the distance between two historical sample questions is determined based on the Hamming distance between them. The Hamming distance is the number of different positions at the same location between two binary strings. For example, the Hamming distance between [1, 1, 1, 1, 0, 1] and [1, 1, 1, 0, 0, 0] is 2.

[0067] Accordingly, the first preset threshold can be 3. Since the Hamming distance between [1,1,1,1,0,1] and [1,1,1,0,0,0] is 2, which is less than the first preset threshold of 3, the two historical sample problems corresponding to [1,1,1,1,0,1] and [1,1,1,0,0,0] can be initially grouped into the same group by the first layer.

[0068] 202. Calculate the text length of the historical sample questions within the first initial group, and group the text lengths into second initial groups according to different preset length intervals to obtain multiple first groups. Among them, the distance between the historical sample questions in the first group is less than the first preset threshold and the text length is within the same preset length interval.

[0069] In this embodiment, as one implementation method, since segmenting the text length can improve the efficiency of the clustering algorithm in calculating distance, a second initial grouping is performed before the clustering algorithm and after the first initial grouping. The first initial grouping is based on the distance between historical sample questions (the smaller the distance, the higher the semantic similarity), and the second initial grouping is based on the text length.

[0070] For example, different preset length ranges can be: text length less than 10, greater than or equal to 10 and less than 20, and greater than or equal to 20. After the first level of initial grouping, n groups are obtained. Taking one group as an example, this group includes 10 historical sample questions. The text length is then divided into three preset length ranges for the second level of initial grouping. Correspondingly, there are 3 text lengths less than 10, 6 text lengths greater than or equal to 10 and less than 20, and 1 text length greater than or equal to 20. This second level of initial grouping results in 3 first groups. If the historical sample questions in each of these n groups can be divided into three ranges, then the number of first groups will be n*3.

[0071] 203. Using a clustering algorithm, the historical sample questions within each first group are regrouped to obtain multiple second groups. The distance between the historical sample questions within each second group is less than a second preset threshold, and the second preset threshold is less than a first preset threshold.

[0072] In this embodiment, the clustering algorithm can be the DBSCAN density clustering algorithm. This clustering algorithm clusters samples based on their density. Pre-set clustering parameters include at least the maximum distance threshold for clustering (corresponding to the second preset threshold in this embodiment). Specifically, the DBSCAN density clustering algorithm includes a distance function. The principle of the clustering algorithm is to continuously calculate the distance between samples using the distance function, iteratively summarizing and grouping samples with a distance less than the second preset threshold together. The distance function can be the Jaccard distance, the Edit_distance, or a weighted average of both. No limitation is made here. Taking a weighted average distance function as an example, the formula is as follows:

[0073] Distance(x,y)=[jaccard(x,y)+edit_distance(x,y)] / [length(x)+length(y)]

[0074] Where x and y are two historical sample questions, Distance(x,y) is the distance between the two historical sample questions, length(x) is the text length of x, and length(y) is the text length of y.

[0075] 204. Calculate the distance between the second groups, and merge the second groups that are less than the second preset threshold to obtain the merged second group.

[0076] 205. Determine all merged second groups as the target question set needed to build the question-answering model.

[0077] For step 204 of the embodiment, as one implementation, calculating the distance between the second groups may include: determining a representative question among the historical sample questions of any second group, and calculating the distance between the representative questions. A representative question is one that semantically represents all historical sample questions of the second group. Preferably, the question with the longest text length among all historical sample questions in the second group can be used as the representative question. Calculating the distance between the second groups is equivalent to calculating the distance between the representative questions of each second group. The distance calculation formula is prior art and will not be elaborated here.

[0078] Furthermore, if the distance between representative questions is less than a second preset threshold, the corresponding second groups are merged into one group to obtain a merged second group. There are no semantically identical historical sample questions within the merged second groups, and the historical sample questions within each merged second group have the same semantics. Therefore, the merged second group can serve as the target question set in the question-answering model, where each target question set corresponds to one merged second group.

[0079] 206. From each merged second group, determine a standard question, obtain the standard answer corresponding to each standard question, and establish a mapping between standard questions and standard answers.

[0080] It should be noted that the target question set required to build the question-answering model has already been mined in step 205 of the embodiment. In addition to the target question set, the question-answering model also needs to obtain the answer set corresponding to each target question set. Specifically, since the historical sample questions are obtained from the dialogue (the dialogue includes personal information, historical sample questions, response text, and some stop words such as thank you and hello), the answer set is also obtained from the dialogue (that is, the answers to the historical sample questions can be obtained from the response text). Since there are multiple answers in the answer set, it is necessary to select a standard answer as a method. This standard answer can be the one with the longest text length in the answer set.

[0081] Preferably, to improve the accuracy of the standard answer, manual review can be performed to select the most accurate answer from the answer set. To improve the efficiency of manual review, all the second groups can be sorted. Specifically, in the merged second groups, any one of them can be selected and ranked first. The merged second group closest to the first group is then ranked second, and so on, until all merged second groups are sorted. After sorting, the closer second groups are semantically more similar, thus improving the efficiency of manual review.

[0082] In specific application scenarios (such as atrial fibrillation), the questions should be listed first: Question 1: What causes atrial fibrillation? What causes atrial fibrillation? What triggers atrial fibrillation? Question 2: What precautions should be taken for atrial fibrillation? What precautions should be taken in daily life for atrial fibrillation? Question 3: What medications are used for atrial fibrillation? How is atrial fibrillation managed? Question 4: What are the causes of atrial fibrillation? What are the causes of atrial fibrillation?

[0083] After sorting: Question 1: What causes atrial fibrillation? What causes atrial fibrillation? What causes atrial fibrillation? What triggers atrial fibrillation? Question 4: What are the causes of atrial fibrillation? What are the causes of atrial fibrillation? Question 2: What should be paid attention to regarding atrial fibrillation? What should be paid attention to in daily life regarding atrial fibrillation? Question 3: What medications are used for atrial fibrillation? How is atrial fibrillation treated?

[0084] In specific application scenarios (such as corporate loans), the questions should be listed first: Question 1: What are the applicants for corporate interest-subsidized loans? Can I apply for a corporate interest-subsidized loan? What are the application areas for corporate interest-subsidized loans? Question 2: When does the corporate interest-subsidized loan start? When can I apply for a corporate interest-subsidized loan? What is the application period for a corporate interest-subsidized loan? Can I apply for a corporate interest-subsidized loan now? Question 3: What is a corporate interest-subsidized loan? What are the new regulations for corporate interest-subsidized loans?

[0085] After sorting: Question 3: What is a corporate interest-subsidized loan? What are the new regulations for corporate interest-subsidized loans? Question 2: When does a corporate interest-subsidized loan start? When can I apply for a corporate interest-subsidized loan? What is the application period for a corporate interest-subsidized loan? Can I apply for a corporate interest-subsidized loan now? Question 1: Who is eligible to apply for a corporate interest-subsidized loan? Can I apply for a corporate interest-subsidized loan? What are the application areas for corporate interest-subsidized loans?

[0086] Once the question-answering model is established, in practical applications, such as in medical scenarios, it can be used by patients or doctors (this is not limited); similarly, in financial scenarios, it can be used by staff handling financial transactions or by financial users (this is also not limited). Specifically, a real-time message is sent to the question-answering model. Upon receiving the message, the model extracts the real-time question, calculates its similarity to standard questions, and matches the target question with the highest similarity from multiple standard questions. As one implementation, if a target historical sample question is matched in the second group corresponding to the target question, and a match is found, the standard answer is output. As another implementation, the standard answer corresponding to the standard question is directly output.

[0087] Based on the mapping relationship between standard questions and standard answers, the matching process between user input questions and question-answering model output answers can be explained intuitively, with high accuracy and stability.

[0088] This application discloses a method, apparatus, and device for mining questions in a question-answering model. First, historical sample questions are acquired and initially grouped to obtain multiple first groups, where the distance between historical sample questions within each first group is less than a first preset threshold. Then, a clustering algorithm is used to further group the historical sample questions within each first group, resulting in multiple second groups, where the distance between historical sample questions within each second group is less than a second preset threshold, and the second preset threshold is less than a first preset threshold. Further, the distances between the second groups are calculated, and the second groups with distances less than the second preset threshold are merged to obtain merged second groups. Finally, all merged second groups are determined as the target question set required to build the question-answering model. Through the technical solution in this application, by initially grouping historical sample questions into multiple first groups, and then clustering the historical sample questions within each first group, the computational cost of clustering is only the number of historical sample questions within a group (to improve clustering speed, the clustering algorithm can be used once for each historical sample question within a group, and all first groups can be clustered in parallel). Compared to directly calculating the similarity of all historical sample questions, this greatly saves computational cost and improves efficiency.

[0089] Furthermore, as Figure 1 and Figure 2 The specific implementation of the method shown in this application provides a question mining device in a question-answering model, such as... Figure 3 As shown, the device includes: an initial grouping module 31, a regrouping module 32, a merging module 33, and a determination module 34;

[0090] The initial grouping module 31 can be used to obtain historical sample questions, perform initial grouping on the historical sample questions to obtain multiple first groups, wherein the distance between the historical sample questions in the first group is less than a first preset threshold.

[0091] The regrouping module 32 can be used to regroup the historical sample questions in each first group using a clustering algorithm to obtain multiple second groups, wherein the distance between the historical sample questions in each second group is less than a second preset threshold, and the second preset threshold is less than the first preset threshold.

[0092] The merging module 33 can be used to calculate the distance between the second groups, merge the second groups that are less than the second preset threshold, and obtain the merged second group;

[0093] The determination module 34 can be used to determine all the merged second group as the target question set required to build the question-answering model.

[0094] Accordingly, in order to initially group the historical sample questions to obtain multiple first groups, the initial grouping module 31 can be used to calculate the distance between the historical sample questions, and to group historical sample questions whose distance is less than a first preset threshold into a first-level initial group; calculate the text length of the historical sample questions in the first-level initial group, and to group the text length into a second-level initial group according to different preset length intervals to obtain multiple first groups, wherein the distance between the historical sample questions in the first group is less than the first preset threshold and the text length is within the same preset length interval.

[0095] In specific application scenarios, a question-mining device in a question-answering model, such as Figure 4 As shown, it also includes: a preprocessing module 35, which can be used to obtain historical sample messages, extract questions from the historical sample messages; determine the first question with the same text and a number greater than a preset value, and take the first question as the second group; determine the second question with the same text and a number less than or equal to the preset value, delete the duplicate questions in the first question and the second question, and obtain the historical sample questions.

[0096] Accordingly, in order to calculate the distance between the second groups, the second groups that are less than the second preset threshold are merged. The merging module 33 can be used to determine the representative questions in the historical sample questions of any second group; calculate the distance between the representative questions; and if the distance between the representative questions is less than the second preset threshold, then the second groups are merged into one group.

[0097] In specific application scenarios, a question-mining device in a question-answering model, such as Figure 4 As shown, it also includes: a sorting module 36, which can be used to select any one of the merged second groups to be sorted first, determine the merged second group that is closest to the first group and sort it second, determine the merged second group that is closest to the second group and sort it third, and so on to sort all the merged second groups.

[0098] In specific application scenarios, a question-mining device in a question-answering model, such as Figure 4 As shown, it also includes: a mapping module 37, which can be used to determine a standard question from each of the merged second groups; obtain the standard answer corresponding to each standard question; and establish a mapping between the standard questions and the standard answers.

[0099] It should be noted that other corresponding descriptions of the functional units involved in the question-finding device in the question-answering model provided in this embodiment can be found in [reference]. Figures 1 to 2 The corresponding description will not be repeated here.

[0100] Based on the above, Figures 1 to 2 Accordingly, this embodiment also provides a storage medium, which may be volatile or non-volatile, storing computer-readable instructions thereon. When these instructions are executed by a processor, they implement the above-described method. Figures 1 to 2 The question mining method in the question-answering model shown.

[0101] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a storage medium (such as a CD-ROM, USB flash drive, or portable hard drive) and includes several instructions to cause a computer device (such as a personal computer, server, or network device) to execute the methods of various implementation scenarios of this application.

[0102] Based on the above, Figures 1 to 2 The method shown and Figure 3 , Figure 4 To achieve the above objectives, this embodiment also provides a computer device, which includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to implement the above-described virtual device embodiment. Figures 1 to 2 The question mining method in the question-answering model shown.

[0103] Optionally, the computer device may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0104] Those skilled in the art will understand that the computer device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0105] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned computer device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing device.

[0106] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platform, or it can be implemented by hardware.

[0107] By applying the technical solution of this application, compared with the existing technology, this application discloses a method, apparatus, and device for mining questions in a question-answering model. First, historical sample questions are acquired and initially grouped to obtain multiple first groups, wherein the distance between historical sample questions within each first group is less than a first preset threshold. Then, a clustering algorithm is used to further group the historical sample questions within each first group to obtain multiple second groups, wherein the distance between historical sample questions within each second group is less than a second preset threshold, and the second preset threshold is less than a first preset threshold. Further, the distance between the second groups is calculated, and the second groups with distances less than the second preset threshold are merged to obtain merged second groups. Finally, all merged second groups are determined as the target question set required to build the question-answering model. The technical solution in this application involves initially grouping historical sample problems into multiple first groups, and then clustering the historical sample problems within each first group. The computational cost of clustering is only the number of historical sample problems within the group (to improve the clustering speed, the clustering algorithm can be used once for each historical sample problem within each group, and all first groups can be clustered in parallel). Compared to directly calculating the similarity of all historical sample problems, this method greatly saves computational cost and improves efficiency.

[0108] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the modules in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or can be modified to be located in one or more apparatuses different from this embodiment. The modules of the above-described embodiment can be combined into one module, or further divided into multiple sub-modules.

[0109] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of any particular implementation scenario. The above disclosures are merely a few specific implementation scenarios of this application; however, this application is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. A method for mining questions in a question-answering model, characterized in that, include: Historical sample questions are obtained, and the historical sample questions are initially grouped to obtain multiple first groups, wherein the distance between the historical sample questions in the first group is less than a first preset threshold. The historical sample questions within each of the first groups are regrouped using a clustering algorithm to obtain multiple second groups. The distance between the historical sample questions within each second group is less than a second preset threshold, and the second preset threshold is less than the first preset threshold. Calculate the distance between the second groups, and merge the second groups that are less than the second preset threshold to obtain the merged second group; The merged second group is identified as the target question set required to build the question-answering model; Before addressing the issue of obtaining historical samples, the following is also included: Obtain historical sample messages and extract the questions from the historical sample messages; The first question is identified as having identical text and a number greater than a preset value. Any one of the first questions is taken as the second group, and all the first questions are deleted and not retained in the historical sample questions. Identify a second question with identical text and a number less than or equal to the preset value, delete the duplicate questions in the second question, and obtain the historical sample questions.

2. The method according to claim 1, characterized in that, The initial grouping of the historical sample questions to obtain multiple first groups, wherein the distance between the historical sample questions within the first group is less than a first preset threshold, includes: Calculate the distance between the historical sample questions, and group the historical sample questions whose distance is less than a first preset threshold into a first-level initial group; Calculate the text length of the historical sample questions within the first initial grouping, and then group the text lengths into a second initial grouping according to different preset length intervals to obtain multiple first groups. The distance between the historical sample questions in the first group is less than a first preset threshold and the text lengths are within the same preset length interval.

3. The method according to claim 1, characterized in that, The step of calculating the distance between the second groups, and merging the second groups that are less than the second preset threshold, includes: Determine the representative problem from any historical sample problem in the second group; Calculate the distance between the representative questions. If the distance between the representative questions is less than the second preset threshold, then combine the second group into one group.

4. The method according to claim 1, characterized in that, After determining the second group, which combines all the aforementioned elements, as the target question set required to build the question-answering model, the following is also included: In the merged second group, select any one to be ranked first, determine the merged second group that is closest to the first group and rank it second, determine the merged second group that is closest to the second group and rank it third, and so on until all the merged second groups are sorted.

5. The method according to claim 1 or 4, characterized in that, Also includes: From each of the merged second groups, a standard problem is determined; Obtain the standard answer corresponding to each of the standard questions, and establish a mapping between the standard questions and the standard answers.

6. A question-mining device in a question-answering model, characterized in that, include: The initial grouping module is used to obtain historical sample questions, perform initial grouping on the historical sample questions to obtain multiple first groups, wherein the distance between the historical sample questions in the first group is less than a first preset threshold. The regrouping module is used to regroup the historical sample questions within each first group using a clustering algorithm to obtain multiple second groups, wherein the distance between the historical sample questions within each second group is less than a second preset threshold, and the second preset threshold is less than the first preset threshold. The merging module is used to calculate the distance between the second groups, merge the second groups that are less than the second preset threshold, and obtain the merged second group. The determination module is used to identify all the merged second group as the target question set required to build the question-answering model; The preprocessing module is used to acquire historical sample messages, extract questions from the historical sample messages; determine the first questions with identical text and a number greater than a preset value, take any one of the first questions as the second group, delete all the first questions, and not retain them in the historical sample questions; determine the second questions with identical text and a number less than or equal to the preset value, delete the duplicate questions in the second questions, and obtain the historical sample questions.

7. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the question mining method in the question-answering model as described in any one of claims 1 to 5.

8. A computer device comprising a memory, a processor, and a computer program stored on a storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the question mining method in the question-answering model according to any one of claims 1 to 5.

Images