Question mining method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a problem mining method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a pre-constructed standard question library; the standard question library comprises a first standard question text, and the first standard question text corresponds to a first intention category; according to the importance degree of each word in the first standard question text for the first intention category, mining keywords of the first intention category from the plurality of words; determining co-occurrence words of the keywords according to co-occurrence information of the keywords and the non-keyword words in a standard question library; and according to the keyword and the co-occurrence word of the keyword, mining a target problem text from a pre-acquired target text set. According to the method, the high-quality question text can be efficiently and accurately mined from a large number of texts, so that the standard question library is effectively expanded.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a problem mining method, device, electronic device and storage medium. Background Art

[0002] The technology behind intelligent customer service is mainly based on dialogue interaction technology. Common dialogue tasks can be divided into chat type, task type and question-and-answer type. Among them, the most common is the question-and-answer type intelligent customer service system, namely the FAQ (Frequently asked Questions, retrieval-based question-and-answer) system. When a customer asks a question, the system uses rule engines, model matching and other technologies to identify the intent corresponding to the question raised by the customer, and then automatically returns the pre-set answer based on the intent label (such as the intent number). The advantage of this system is that the answer quality is relatively high, and the disadvantage is that the application of text matching technology for intent recognition requires a sufficiently complete standard question library (i.e., knowledge base) prepared in advance.

[0003] The retrieval-based question-answering system needs to be configured with some commonly used and clearly described questions, called "standard questions (text)". There is a many-to-one mapping relationship between these standard questions and answers. When the questions raised by the customer are matched to the standard questions, the corresponding answers are matched. The set of standard questions constitutes the knowledge base, that is, the standard question base. A common matching method is: when a customer asks a question, the FAQ system calculates the text similarity between the customer's question and all the configured standard questions, and finds the standard question that is most similar to the question raised by the customer. When the question is accurately identified, the intent label corresponding to the question is obtained, and the pre-defined answer is returned. The main purpose of standard question mining is to improve the generalization ability of the FAQ system's intent recognition, so that the FAQ system can identify more complex and diverse questions.

[0004] The data used in standard question mining mainly comes from ASR text data generated by ASR (Automatic Speech Recognition) technology when agents communicate with customers. However, due to the influence of the external environment and the performance of the ASR model itself, the generated ASR text data has problems such as long text and misrecognition. In addition, the randomness of people's conversations and the large number of professional words bring certain difficulties to the standard question mining task. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a question mining method, device, electronic device and storage medium, which are used to efficiently and accurately mine high-quality question texts from a large amount of text, thereby effectively expanding the standard question library.

[0006] To solve the above technical problems, the embodiments of the present application are implemented as follows:

[0007] On the one hand, an embodiment of the present application provides a problem mining method, including:

[0008] Obtain a pre-built standard question library; the standard question library includes a first standard question text, the first standard question text corresponds to a first intention category; the first standard question text includes a plurality of words;

[0009] According to the importance of each word in the first standard question text to the first intent category, mining keywords of the first intent category from the multiple words; the multiple words include the keywords and non-keywords;

[0010] Determining co-occurrence words of the keywords according to co-occurrence information of the keywords and the non-keywords in the standard question library;

[0011] According to the keywords and the co-occurring words of the keywords, target question texts are mined from a pre-acquired target text set.

[0012] On the other hand, an embodiment of the present application provides a problem mining device, including:

[0013] An acquisition module, configured to acquire a pre-built standard question library; the standard question library includes a first standard question text, the first standard question text corresponds to a first intent category; the first standard question text includes a plurality of words;

[0014] A first mining module is used to mine keywords of the first intent category from the multiple words according to the importance of each word in the first standard question text to the first intent category; the multiple words include the keywords and non-keywords;

[0015] A determination module, configured to determine co-occurrence words of the keyword according to co-occurrence information of the keyword and the non-keyword in the standard question library;

[0016] The second mining module is used to mine the target question text from the pre-acquired target text set according to the keywords and the co-occurring words of the keywords.

[0017] On the other hand, an embodiment of the present application provides an electronic device, including a processor and a memory electrically connected to the processor, the memory storing a computer program, and the processor being used to call and execute the computer program from the memory to implement the above-mentioned problem mining method.

[0018] On the other hand, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program can be executed by a processor to implement the above-mentioned problem mining method.

[0019] The technical solution of the embodiment of the present application is adopted, by obtaining a pre-constructed standard question library, the standard question library includes a first standard question text, and the first standard question text corresponds to a first intent category, according to the importance of each word in the first standard question text to the first intent category, the keywords of the first intent category are mined from multiple words (including keywords and non-keywords) in the first standard question text; and the co-occurrence words of the keywords are determined according to the co-occurrence information of the keywords and non-keywords in the standard question library. Since the keywords are mined based on the importance of each word in the first standard question text to the first intent category, the keywords can reflect the first intent category of the first standard question text to a certain extent. Since the co-occurrence words of the keywords are determined based on the co-occurrence information of the keywords and the co-occurrence words in the standard question library, the co-occurrence information is the co-occurrence information, therefore, the co-occurrence words of the keywords can be understood as words with a high degree of relevance to the keywords (such as co-appearing in the same standard question text). Then, according to the keywords of the first intent category and the co-occurrence words of the keywords, the target question text is mined from the pre-acquired target text set. It can be seen that the mining of target question text fully considers the keywords that can represent the intent category of the standard question text and the co-occurring words that are highly correlated with the keywords. For example, the same target question text includes both keywords and their co-occurring words. In this way, the keywords and co-occurring words in the target question text can accurately reflect the intent category corresponding to the target question text, so that the target question text has a high semantic generalization and corresponds to an accurate intent category, achieving the effect of mining high-quality target question text from the target text set. In addition, since the mining process of the target question text does not require human participation, this automated question mining method can greatly reduce the workload for users (such as operators) and is conducive to quickly and accurately expanding the standard question library. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate one or more embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in one or more embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 is a schematic flow chart of a problem mining method according to an embodiment of the present application;

[0022] Figure 2 is a schematic block diagram of a problem mining device according to an embodiment of the present application;

[0023] Figure 3It is a schematic block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] The embodiments of the present application provide a question mining method, device, electronic device and storage medium for efficiently and accurately mining high-quality question texts from a large amount of text, thereby effectively expanding the standard question library.

[0025] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0026] In the field of intelligent question answering technology, mining standard question texts is somewhat difficult. EDA (Easy Data Augmentation) methods or open source BERT-related models (such as Simbert models) are usually used to mine standard question texts. Among them, the EDA method mainly consists of four simple but powerful operations, including synonym replacement, random insertion, random exchange and random deletion. One of the ways is: by replacing the words in the existing standard question text with synonyms, a new question text is obtained; or, by randomly inserting characters, words, etc. into the existing standard question text, a new question text is obtained. For example, the existing standard question text is "The interest rate is so high". Through the EDA method, new question texts are mined as follows: The bank has such a high interest rate, the interest rate is so high, and the interest rate is so high; etc. It can be seen that since the EDA method is relatively simple, it is easy to generate illogical question texts, resulting in semantic errors in the generated question texts. If you want to generate logically accurate question texts, you need to participate in human (such as operators), such as manually deleting question texts with semantic errors. Obviously, this will greatly increase the workload of operators and be inefficient.

[0027] BERT-related models, such as the Simbert model, are based on the open-source BERT model and are trained using a large number of standard question texts. Although BERT-related models can generate sentences with strong semantic expression capabilities, they are unable to generate more informative and diverse standard question texts due to the limitations of the model itself. For example, the existing standard question text is "The interest rate is so high." New questions generated based on the BERT-related model are as follows: Why is the interest rate so high? Why is the interest rate so high now? Why is the interest rate so high? And so on. It can be seen that the information in the question text mined using the BERT-related model is not rich or diverse enough, so the generalization ability is insufficient, which is not very helpful for improving the performance of the intent recognition model.

[0028] Based on the problems existing in the above methods, the present application provides a method for question mining, which obtains a pre-constructed standard question library, the standard question library includes a first standard question text, and the first standard question text corresponds to a first intent category. According to the importance of each word in the first standard question text to the first intent category, the keywords of the first intent category are mined from multiple words (including keywords and non-keywords) in the first standard question text; and the co-occurrence words of the keywords are determined according to the co-occurrence information of the keywords and non-keywords in the standard question library. Since the keywords are mined based on the importance of each word in the first standard question text to the first intent category, the keywords can reflect the first intent category of the first standard question text to a certain extent. Since the co-occurrence words of the keywords are determined based on the co-occurrence information of the keywords and the co-occurrence words in the standard question library, the co-occurrence information is the information of co-occurrence, and therefore, the co-occurrence words of the keywords can be understood as words with a high degree of relevance to the keywords (such as co-appearing in the same standard question text). Then, according to the keywords of the first intent category and the co-occurrence words of the keywords, the target question text is mined from the pre-acquired target text set. It can be seen that the mining of target question text fully considers the keywords that can represent the intent category of the standard question text and the co-occurring words that are highly correlated with the keywords. For example, the same target question text includes both keywords and their co-occurring words. In this way, the keywords and co-occurring words in the target question text can accurately reflect the intent category corresponding to the target question text, so that the target question text has a high semantic generalization and corresponds to an accurate intent category, achieving the effect of mining high-quality target question text from the target text set. In addition, since the mining process of the target question text does not require human participation, this automated question mining method can greatly reduce the workload for users (such as operators) and is conducive to quickly and accurately expanding the standard question library.

[0029] The problem mining method provided in the embodiment of the present application can be executed by an electronic device, or by software installed in the electronic device. Specifically, the electronic device can be a terminal device or a server device. Among them, the terminal device can include a smart phone, a laptop computer, a smart wearable device, a vehicle terminal, etc., and the server device can include an independent physical server, a server cluster composed of multiple servers, or a cloud server capable of cloud computing.

[0030] Figure 1 is a schematic flow chart of a problem mining method according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0031] S102, obtaining a pre-constructed standard question library; the standard question library includes a first standard question text, the first standard question text corresponds to a first intention category; the first standard question text includes a plurality of words.

[0032] Among them, the standard question library may include N standard question texts, where N is an integer greater than 1. Optionally, among the N standard question texts, multiple standard question texts may correspond to the same intent category. Each intent category may correspond to a unique intent label (such as an intent number), and there is a one-to-one correspondence between the intent label and the answer. Therefore, when N standard question texts correspond to the same intent category, the answer to each standard question text in the N standard question texts is the same. The N standard question texts may also correspond to different intent categories.

[0033] The first intention category may be any intention category corresponding to the standard question library. The first standard question text is one or more standard question texts corresponding to the first intention category in the standard question library.

[0034] S104: mining keywords of the first intent category from the plurality of words according to the importance of each word in the first standard question text to the first intent category, the plurality of words including keywords and non-keywords.

[0035] Among them, the importance of each word to the first intent category can be represented by a specific numerical value. The higher the value of the importance of the word to the first intent category, the higher the importance of the word to the first intent category. This embodiment does not limit the form of the value of the importance, for example, it can be a percentage, an integer, etc. Optionally, the importance of each word to the first intent category can be represented based on the appearance information of each word in the first standard question text (such as the number of occurrences, the frequency of occurrence, etc.), and / or, based on the appearance information of each word in the entire standard question library (i.e., N standard question texts) (such as the number of occurrences, the frequency of occurrence, etc.). The number of occurrences of a word in a text (i.e., the first standard question text or the N standard question texts) is the number of times the word appears in the text. The frequency of occurrence of a word in a text (i.e., the first standard question text or the N standard question texts) is the proportion of the number of times the word appears in the text to the total number of all words in the text. For example, if the text includes a total of 10 words and the word appears 3 times in the text, then the frequency of occurrence of the word in the text is 0.3.

[0036] For example, for a certain word in the first standard question text, when characterizing the importance of the word to the first intent category based on the number of times the word appears in the first standard question text, the number of times the word appears in the first standard question text can be determined first, and then the number of times the word appears can be determined as the value of the importance of the word to the first intent category. Assuming that the number of times the word appears in the first standard question text is 3, the value of the importance of the word to the first intent category is 3. Alternatively, the value of the importance of the word to the first intent category can be calculated based on the number of times the word appears in the first standard question text and a preset calculation method (such as a calculation formula). How to calculate the importance of each word to the first intent category will be described in detail in the following embodiments.

[0037] Before executing step S104, the first standard question text may be segmented by a word segmenter to obtain all the words in the first standard question text. Optionally, all the words in the first standard question text may be preprocessed, such as removing stop words and retaining necessary nouns, pronouns, verbs, prepositions, adjectives and words commonly used in business.

[0038] S106: Determine co-occurrence words of the keyword according to co-occurrence information of the keyword and the non-keyword in the standard question database.

[0039] The co-occurrence information is information about the co-occurrence of the keyword and the non-keyword, such as the number of co-occurrences, the co-occurrence frequency, etc. Optionally, when the keyword and the non-keyword appear in the same standard question text, it can be considered that the keyword and the non-keyword co-occur, and the non-keyword that co-occurs with the keyword is the co-occurrence word of the keyword.

[0040] S108: mining a target question text from a pre-acquired target text set according to the keywords and their co-occurring words.

[0041] The target text set includes multiple question texts.

[0042] In this embodiment, the target text set includes multiple dialogue texts, and the dialogue texts include question texts and answer texts. The dialogue texts in the target text set may be historical dialogue texts between the customer and the agent in the current scenario. The current scenario refers to the scenario corresponding to the standard question library, that is, the scenario related to the standard question text in the standard question library. For example, if the scenario corresponding to the standard question library is the telemarketing scenario, then the current scenario is the telemarketing scenario, and the dialogue texts in the target text set may include dialogue texts between the customer and the agent during the telemarketing process.

[0043] The technical solution of the embodiment of the present application is adopted, by obtaining a pre-constructed standard question library, the standard question library includes a first standard question text, and the first standard question text corresponds to a first intent category, according to the importance of each word in the first standard question text to the first intent category, the keywords of the first intent category are mined from multiple words (including keywords and non-keywords) in the first standard question text; and according to the co-occurrence information of keywords and non-keywords in the standard question library, the co-occurrence words corresponding to the keywords are determined. Since the keywords are mined based on the importance of each word in the first standard question text to the first intent category, the keywords can reflect the first intent category of the first standard question text to a certain extent. Since the co-occurrence words of the keywords are determined based on the co-occurrence information of the keywords and the co-occurrence words in the standard question library, the co-occurrence information is the information of co-occurrence, therefore, the co-occurrence words of the keywords can be understood as words with a high degree of relevance to the keywords (such as co-appearing in the same standard question text). Then, according to the keywords of the first intent category and the co-occurrence words of the keywords, the target question text is mined from the pre-acquired target text set. It can be seen that the mining of target question text fully considers the keywords that can represent the intent category of the standard question text and the co-occurring words that are highly correlated with the keywords. For example, the same target question text includes both keywords and their co-occurring words. In this way, the keywords and co-occurring words in the target question text can accurately reflect the intent category corresponding to the target question text, so that the target question text has a high semantic generalization and corresponds to an accurate intent category, achieving the effect of mining high-quality target question text from the target text set. In addition, since the mining process of the target question text does not require human participation, this automated question mining method can greatly reduce the workload for users (such as operators) and is conducive to quickly and accurately expanding the standard question library.

[0044] In one embodiment, according to the importance of each word in the first standard question text to the first intent category, when mining keywords of the first intent category from multiple words in the first standard question text, the following steps A1-A4 may be specifically performed:

[0045] Step A1, determining a target long text corresponding to a first intention category according to a first standard question text; the target long text is composed of at least one first standard question text.

[0046] Among them, the first standard question text corresponding to the first intent category may include one or more. Usually, the first intent category corresponds to multiple first standard question texts. When determining the target long text corresponding to the first intent category, first screen out one or more representative first standard question texts from the multiple first standard question texts corresponding to the first intent category, and then splice the screened one or more first standard question texts to obtain the target long text corresponding to the first intent category. Among them, the representative first standard question text can be a first standard question text with clear semantic logic (that is, high text quality).

[0047] Optionally, there is a situation where, among the multiple first standard question texts corresponding to the first intent category, there are first standard question texts with a large text distance, that is, the text similarity between the first standard question texts is low. In this case, a balanced screening method can be used to screen multiple representative first standard question texts from the first standard question texts corresponding to the first intent category. The purpose of balanced screening is to make the number of first standard question texts with higher similarity as much as possible among the multiple representative first standard question texts screened for the first intent category. Then, the target long text is determined based on the multiple first standard question texts screened out in a balanced manner.

[0048] For example, Table 1 below schematically shows the correspondence between different intent categories and standard question texts. The intent category in Table 1 may be the first intent category or other intent categories, and the standard question text may be the first standard question text or the standard question text corresponding to other intent categories. As shown in Table 1, the standard question texts corresponding to numbers 1-6 correspond to the intent category "temporarily unable to pay back the money"; the standard question texts corresponding to numbers 7-9 correspond to the intent category "processing later at work". Among the standard question texts corresponding to numbers 1-6, the text similarity between the standard question texts corresponding to numbers 1-3 is high, and the text similarity between the standard question texts corresponding to numbers 4-6 is high. It can be seen that among the multiple standard question texts screened out in a balanced manner, the number of standard question texts with high similarity is 3. Then, by splicing the standard question texts corresponding to numbers 1-6, the target long text corresponding to the intent category "temporarily unable to pay back the money" can be obtained. Similarly, by splicing the standard question texts corresponding to numbers 7-9, the target long text corresponding to the intent category "processing later at work" can be obtained. It should be noted that what is shown in Table 1 is only an exemplary method, and it does not limit the number of standard question texts constituting the target long text. In order to make the mining of standard question texts more balanced and accurate, the text lengths of the target long texts corresponding to each intent category should be as similar or close as possible. The text length of the target long text can be determined based on the number of standard question texts that make up the target long text. For example, the target long text corresponding to each intent category is obtained by splicing 10 standard question texts. The text length of the target long text can also be determined based on the total number of words in the target long text. For example, the total number of words in the target long text corresponding to each intent category is between 50 and 60 words.

[0049] Table 1

[0050]

[0051] Step A2, determining the first occurrence information of each word in the target long text in the target long text; and determining the second occurrence information of each word in the target long text in the standard question library.

[0052] The target long text may be segmented by a word segmenter, thereby determining each word in the target long text.

[0053] Step A3: Determine the importance of each word in the target long text to the first intention category according to the first occurrence information and the second occurrence information.

[0054] Step A4, mining keywords of the first intent category from multiple words according to the importance of each word in the target long text to the first intent category; keywords are words whose importance is greater than or equal to a preset importance threshold.

[0055] Optionally, the first occurrence information includes an occurrence frequency. Based on this, when determining the first occurrence information of each word in the target long text, the first occurrence number of the first word in the target long text may be determined first, and then the occurrence frequency of the first word in the target long text may be determined based on the first occurrence number and the total number of words included in the target long text. The first word is any word in the target long text corresponding to the first intent category.

[0056] The second occurrence information includes the inverse text frequency. Based on this, when determining the second occurrence information of each word in the target long text in the standard question library, the first text quantity of the target long text including the second word in all target long texts can be determined for the second word. All target long texts refer to the target long texts corresponding to all intent categories corresponding to the standard question library, and the second word is any word in the target long text corresponding to the first intent category. Then, based on the first text quantity and the total text quantity of the target long text, determine the inverse text frequency corresponding to the second word. Among them, since each intent category corresponding to the standard question library corresponds to a unique target long text, the total text quantity of the target long text is equal to the total number of categories of the intent category corresponding to the standard question library.

[0057] For the target long text corresponding to the first intent category, after determining the occurrence frequency of each word in the target long text and the inverse text frequency of each word in the target long text in the standard question library, the importance of each word in the target long text to the first intent category is determined based on the occurrence frequency and the inverse text frequency.

[0058] Optionally, an improved TF-IDF algorithm is used to determine the importance of each word in the target long text for the first intent category, in which TF represents the word frequency, that is, the frequency of occurrence of the word in the target long text, and IDF represents the inverse text frequency of the word in the N standard question texts.

[0059] First, the word frequency is calculated. The frequency of occurrence of a word in the target long text can be calculated using the following formula (1):

[0060]

[0061] Where i represents any word in the target long text, that is, the frequency of occurrence of word i in the target long text is to be calculated. j represents the target long text where word i is located, and k represents each word included in the target long text. TF i,j Represents the frequency of word i in the target long text j. N i,j represents the number of occurrences of word i in the target long text j, ∑ k N k,jrepresents the sum of the number of occurrences of each word in the target long text j in the target long text j, which is equal to the total number of words included in the target long text j.

[0062] Then the inverse text frequency of the word is calculated. The inverse text frequency of the word can be calculated using the following formula (2):

[0063]

[0064] Where i represents any word in the target long text, that is, the inverse text frequency of word i in the standard question library is to be calculated. j represents the target long text where word i is located. IDF i represents the inverse text frequency of word i in the standard question library, and D represents the total number of texts in the target long text. i | represents the number of first target long texts that include word i in all target long texts.

[0065] For the target long text corresponding to the first intent category, after calculating the occurrence frequency of each word in the target long text and the inverse text frequency of each word in the target long text in the standard question library, the importance of each word in the target long text to the first intent category is calculated. The importance can be calculated according to the following formula (3):

[0066] P i =TF i,j *IDF i (3)

[0067] Among them, P i Indicates the importance of word i to the first intent category. That is, the importance of a word to the first intent category is the product of the frequency of occurrence of the word in the target long text corresponding to the first intent category and the inverse text frequency of the word in the standard question library.

[0068] At this point, according to the method provided in the above embodiment, the keywords of each intent category corresponding to the standard question library can be determined. Optionally, the number n (n is a positive integer) of keywords for each intent category can be set in advance, so that the words ranked in the top n in importance are selected as the keywords of the intent category in order of importance.

[0069] In one embodiment, the co-occurrence information of keywords and non-keywords in the standard question library includes co-occurrence degree. The standard question library includes N standard question texts, where N is an integer greater than 1. When determining the co-occurrence words of keywords based on the co-occurrence information of keywords and non-keywords in the standard question library, the following steps B1-B3 may be performed:

[0070] Step B1, determining the number of second texts of standard question texts including keywords among the N standard question texts; and determining the number of third texts of standard question texts including both keywords and non-keywords among the N standard question texts.

[0071] Step B2, determining the co-occurrence of keywords and non-keywords in the standard question database according to the number of second texts, the number of third texts, and the total number of texts of N standard question texts.

[0072] Step B3: determine that non-keywords whose co-occurrence degree with the keyword is greater than or equal to a preset co-occurrence degree threshold are co-occurrence words.

[0073] Optionally, the PMI (Point-wise mutual information) method is used to calculate the co-occurrence of keywords and non-keywords in the standard question database. The purpose of the PMI method is to find words that appear at the same time as keywords, which can be calculated according to the following formulas (4)-(6):

[0074]

[0075]

[0076]

[0077] Among them, PMI(i,j) represents the co-occurrence of keyword i and non-keyword j in the standard question database, and N represents the total number of texts in the N standard question texts. M(i) represents the number of second texts of standard question texts that include keyword i in the N standard question texts. M(i,j) represents the number of third texts of standard question texts that include both keyword i and non-keyword j in the N standard question texts. p(j) and p(i) are calculated in the same way, both are formula (6), and the only difference is the difference in words.

[0078] After calculating the co-occurrence of keywords and non-keywords in the standard question library, determine the non-keywords whose co-occurrence with the keywords is greater than or equal to the preset co-occurrence threshold as the co-occurrence words of the keywords. Optionally, when the value of PMI(i,j) is a positive number, it means that there is a certain co-occurrence correlation between keyword i and non-keyword j. The larger the value of PMI(i,j), the higher the co-occurrence degree between keyword i and non-keyword j. Assuming that the preset co-occurrence threshold is 0.5, non-keyword j corresponding to a value greater than or equal to 0.5 is screened out from all PMI(i,j), which is the co-occurrence word of keyword i.

[0079] In one embodiment, when executing step S108, that is, mining the target question text from the pre-acquired target text set according to the keywords of the first intent category and the co-occurring words of the keywords, the following steps may be performed:

[0080] First, candidate question texts are screened from the target text set; the candidate question texts are question texts that include both keywords and co-occurring words of the keywords.

[0081] If the keyword corresponds to only one co-occurring word, the candidate question text is the question text that includes both the keyword and the co-occurring word. If the keyword includes multiple co-occurring words, the question text that includes both the keyword and at least one co-occurring word of the keyword can be determined as the candidate question text; or the question text that includes both the keyword and all the co-occurring words of the keyword can be determined as the candidate question text.

[0082] Secondly, the intent category to which the candidate question text belongs is predicted to obtain the intent category prediction result of the candidate question text.

[0083] Again, based on the intent category prediction results, determine whether the candidate question text is the target question text.

[0084] In this embodiment, the intent category prediction result may include at least one of the following: the first predicted intent category, and the probability that the candidate question text belongs to each intent category.

[0085] Optionally, a text length range of the candidate question texts may be preset to filter out candidate question texts that are not within the text length range, and then determine the target question text based on the filtered candidate question texts.

[0086] Table 2 below exemplarily lists the corresponding relationship between the keywords, co-occurring words and candidate question texts corresponding to the first intent category "pay back after salary is paid". It can be seen that the candidate question text includes both keywords and co-occurring words of keywords.

[0087] Table 2

[0088]

[0089] It can be understood that when a keyword has multiple co-occurring words, if the candidate question text needs to include the keyword and all of its co-occurring words at the same time, the more co-occurring words there are, the stricter the constraints on the candidate question text will be, and the higher the accuracy of the candidate question text will be.

[0090] In this embodiment, the intent category of the candidate question text is predicted to obtain the intent category prediction result of the candidate question text, and then whether the candidate question text is the target question text is determined according to the intent category prediction result. The purpose is to further check and screen the candidate question text, so as to screen out high-quality target question texts that are helpful for improving the generalization ability of the intent recognition model, so as to make the mining results of the standard question text more accurate and more diverse.

[0091] In one embodiment, the intent category prediction result includes a first predicted intent category. When predicting the intent category to which the candidate question text belongs and obtaining the intent category prediction result of the candidate question text, the following steps C1-C4 may be performed:

[0092] Step C1, clustering the N standard question texts to obtain a clustering result; the clustering result includes a plurality of question text sets; each question text set includes a plurality of standard question texts.

[0093] Among them, any existing clustering algorithm can be used to cluster the N standard question texts. For example, the k-means clustering algorithm can be used, and the k value in the algorithm can be selected to be greater than or equal to the total number of intent categories corresponding to the standard question library, and the number of standard question texts in each cluster cannot be lower than the preset threshold. Before clustering, the standard question text vector corresponding to each standard question text in the N standard question texts can be determined based on the existing vector representation method, and then the N standard question text vectors can be clustered. K clusters correspond to K question text sets, and the standard question texts corresponding to the standard question text vectors included in each cluster constitute a question text set. Among them, the standard question library includes N standard question texts, and N is an integer greater than 1.

[0094] Step C2, determining the central question text corresponding to each question text set; the central question text is the standard question text closest to the cluster center corresponding to the question text set.

[0095] The clustering method of clustering using the above k-means clustering algorithm is used. After clustering, K clusters and the cluster center (i.e., the center vector) of each cluster can be obtained, so that the standard question text corresponding to the standard question text vector closest to the cluster center in each cluster can be determined as the center question text. K clusters correspond to K center question texts.

[0096] Step C3, selecting the central question text with the highest similarity to the candidate question text from multiple central question texts.

[0097] In this step, when calculating the similarity between the candidate question text and the K central question texts, the vector distance between the text vector corresponding to the candidate question text and the text vector corresponding to each central question text can be calculated, and then based on the vector distance, the text vector corresponding to the central question text with the shortest vector distance between the text vector corresponding to the candidate question text is determined, and the central question text with the highest similarity is determined. The vector distance is inversely proportional to the similarity.

[0098] Step C4, determining the intent category of the central question text having the highest similarity with the candidate question text as the first predicted intent category.

[0099] Among the K central question texts, each central question text corresponds to a unique intent category. Assuming that among the K central question texts, the central question text with the highest similarity to the candidate question text is text A, then the intent category of text A is the first predicted intent category.

[0100] Optionally, when determining whether the candidate question text is the target question text according to the first predicted intent category, if the first predicted intent category and the intent category corresponding to the keyword are the same, the candidate question text is determined to be the target question text. If the first predicted intent category and the intent category corresponding to the keyword are different, the candidate question text is determined not to be the target question text.

[0101] In this embodiment, if the first predicted intent category is different from the intent category corresponding to the keyword, it means that the candidate question text has semantic anomalies, such as semantic illogicality. By screening out the candidate question texts whose first predicted intent category is different from the intent category corresponding to the keyword, it is impossible for the final target question text to have semantic anomalies, thereby ensuring the high quality of the target question text.

[0102] In one embodiment, if the first predicted intent category of the candidate question text is the same as the intent category corresponding to the keyword, the candidate question text may be retained first. That is to say, when the first predicted intent category of the candidate question text is the same as the intent category corresponding to the keyword, the candidate question text is not directly determined as the target question text, but this part of the candidate question text is further screened to determine a target question text with higher quality.

[0103] In one embodiment, the intent category prediction result includes: the probability that the candidate question text belongs to each intent category. When predicting the intent category to which the candidate question text belongs and obtaining the intent category prediction result corresponding to the candidate question text, a pre-trained intent recognition model can be used to predict the intent category to which the candidate question text belongs and obtain the probability that the candidate question text belongs to each intent category. Among them, the intent recognition model is trained based on the sample question text and the sample intent category of the sample question text. Since the intent recognition model is an existing model, the specific model training process will not be described in detail.

[0104] In this embodiment, in the process of predicting the intent category to which the candidate question text belongs, the predicted candidate question text may be all candidate question texts screened from the target text set, that is, the target text set includes both keywords and question texts with co-occurring words of the keywords. It may also be the candidate question texts retained after screening according to the first predicted intent category.

[0105] Optionally, when determining whether a candidate question text is a target question text based on the intent category prediction result, the information entropy of the candidate question text can be calculated based on the probability that the candidate question text belongs to each intent category. If the information entropy is greater than or equal to a preset information entropy threshold, the candidate question text is determined to be the target question text; if the information entropy is less than the preset information entropy threshold, the candidate question text is determined not to be the target question text.

[0106] The calculation formula of information entropy can be expressed as the following formula (7):

[0107] H(X)=-∑p(x i )log(p(x i )) (7)

[0108] Where X represents the candidate question text currently being calculated. Assume that there are n intent categories in total, and different intent categories are represented by (x1, x2, …, x n ), then in formula (7), p(x i ) indicates that the candidate question text belongs to the intent category x i The probability of , i = 1, 2, ..., n. And, ∑p(x i ) is 1. That is, for the same candidate question text, the sum of the probabilities of it belonging to each intent category is 1.

[0109] In this embodiment, information entropy can characterize the direct relationship between the semantic information amount of the candidate question text and its uncertainty to a certain extent, that is, the measure of the semantic information amount is equal to the amount of uncertainty. By inputting the candidate standard question into the intent recognition model, after obtaining the probability that the candidate question text belongs to each intent category, the information entropy of the candidate question text is calculated based on the predicted probability. The larger the value of the information entropy, the more difficult it is for the intent recognition model to predict the candidate question text, and the higher the possibility that the candidate question text will improve the generalization ability of the intent recognition model. Therefore, by determining the candidate question text with a larger information entropy (i.e., greater than or equal to the preset information entropy threshold) as the target question text, the target question text finally determined can be beneficial to improving the generalization ability of the intent recognition model.

[0110] In summary, specific embodiments of the present subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recorded in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing can be advantageous.

[0111] The above is a problem mining method provided in an embodiment of the present application. Based on the same idea, an embodiment of the present application also provides a problem mining device.

[0112] Figure 2 is a schematic block diagram of a problem mining device according to an embodiment of the present application, such as Figure 2 As shown, the device comprises:

[0113] An acquisition module 21 is used to acquire a pre-built standard question library; the standard question library includes a first standard question text, the first standard question text corresponds to a first intention category; the first standard question text includes a plurality of words;

[0114] A first mining module 22, configured to mine keywords of the first intent category from the plurality of words according to the importance of each word in the first standard question text to the first intent category; the plurality of words include the keywords and non-keywords;

[0115] A determination module 23, configured to determine co-occurrence words of the keyword according to co-occurrence information of the keyword and the non-keyword in the standard question library;

[0116] The second mining module 24 is used to mine the target question text from the pre-acquired target text set according to the keywords and the co-occurring words of the keywords.

[0117] In one embodiment, the first mining module 22 performs the following steps when mining keywords of the first intent category from the multiple words according to the importance of each word in the first standard question text to the first intent category:

[0118] Determine a target long text corresponding to the first intention category according to the first standard question text; the target long text is composed of at least one of the first standard question texts;

[0119] Determine the first occurrence information of each word in the target long text in the target long text; and determine the second occurrence information of each word in the target long text in the standard question library;

[0120] Determining the importance of each word in the target long text to the first intention category according to the first occurrence information and the second occurrence information;

[0121] According to the importance of each word in the target long text to the first intent category, the keywords of the first intent category are mined from the multiple words; the keywords are words whose importance is higher than or equal to a preset importance threshold.

[0122] In one embodiment, the first occurrence information includes an occurrence frequency;

[0123] When determining the first occurrence information of each word in the target long text, the first mining module 22 performs the following steps:

[0124] Determine the first occurrence number of a first word in the target long text; the first word is any word in the target long text;

[0125] The occurrence frequency of the first word in the target long text is determined according to the first occurrence number and the total number of words included in the target long text.

[0126] In one embodiment, the second occurrence information includes an inverse text frequency;

[0127] When determining the second occurrence information of each word in the target long text in the standard question library, the first mining module 22 performs the following steps:

[0128] Determine the number of first texts of the target long text including a second word in the target long text corresponding to each intention category in the standard question library; the second word is any word in the target long text;

[0129] The inverse text frequency corresponding to the second word is determined according to the first text quantity and the total text quantity of the target long text.

[0130] In one embodiment, the co-occurrence information includes a co-occurrence degree;

[0131] When determining the co-occurrence words of the keywords according to the co-occurrence information of the keywords and the non-keywords in the standard question library, the determination module 23 performs the following steps:

[0132] Determine the number of second texts of standard question texts including the keyword in the N standard question texts; and determine the number of third texts of standard question texts including both the keyword and the non-keyword in the N standard question texts;

[0133] Determining the co-occurrence of the keywords and the non-keywords in the standard question library according to the second text quantity, the third text quantity, and the total text quantity of the N standard question texts;

[0134] A non-keyword whose co-occurrence degree with the keyword is greater than or equal to a preset co-occurrence degree threshold is determined as the co-occurrence word.

[0135] In one embodiment, when the second mining module 24 mines the target question text from the pre-acquired target text set according to the keywords and the co-occurring words of the keywords, the second mining module 24 performs the following steps:

[0136] Screening candidate question texts from the target text set; the candidate question texts are question texts that include both the keywords and the non-keywords;

[0137] Predicting the intent category to which the candidate question text belongs, and obtaining an intent category prediction result of the candidate question text;

[0138] According to the intention category prediction result, it is determined whether the candidate question text is the target question text.

[0139] In one embodiment, the intent category prediction result includes a first predicted intent category;

[0140] When the second mining module 24 predicts the intention category to which the candidate question text belongs and obtains the intention category prediction result of the candidate question text, the following steps are performed:

[0141] Performing clustering processing on N standard question texts to obtain a clustering result; the clustering result includes a plurality of question text sets; each question text set includes a plurality of the standard question texts;

[0142] Determine the central question text corresponding to each question text set; the central question text is the standard question text closest to the cluster center corresponding to the question text set;

[0143] Selecting, from the plurality of central question texts, the central question text having the highest similarity to the candidate question texts;

[0144] The intent category of the central question text having the highest similarity to the candidate question text is determined as the first predicted intent category.

[0145] In one embodiment, when the second mining module 24 determines whether the candidate question text is the target question text according to the intention category prediction result, the second mining module 24 performs the following steps:

[0146] If the first predicted intention category is the same as the intention category corresponding to the keyword, determining the candidate question text as the target question text;

[0147] If the first predicted intent category and the intent category corresponding to the keyword are different, it is determined that the candidate question text is not the target question text.

[0148] In one embodiment, the intent category prediction result includes: the probability that the candidate question text belongs to each intent category;

[0149] When the second mining module 24 predicts the intent category to which the candidate question text belongs and obtains the intent category prediction result corresponding to the candidate question text, the following steps are performed:

[0150] The pre-trained intent recognition model is used to predict the intent category to which the candidate question text belongs, and the probability that the candidate question text belongs to each intent category is obtained; wherein the intent recognition model is trained based on sample question text and the sample intent category of the sample question text.

[0151] In one embodiment, when the second mining module 24 determines whether the candidate question text is the target question text according to the intention category prediction result, the second mining module 24 performs the following steps:

[0152] Calculating the information entropy of the candidate question text according to the probability that the candidate question text belongs to each intention category;

[0153] If the information entropy is greater than or equal to a preset information entropy threshold, determining the candidate question text as the target question text;

[0154] If the information entropy is less than the preset information entropy threshold, it is determined that the candidate question text is not the target question text.

[0155] The device of the embodiment of the present application is adopted, by obtaining a pre-constructed standard question library, the standard question library includes a first standard question text, and the first standard question text corresponds to a first intent category, according to the importance of each word in the first standard question text to the first intent category, mining the keywords of the first intent category from multiple words (including keywords and non-keywords) in the first standard question text; and determining the co-occurrence words of the keywords according to the co-occurrence information of the keywords and non-keywords in the standard question library. Since the keywords are mined based on the importance of each word in the first standard question text to the first intent category, the keywords can reflect the first intent category of the first standard question text to a certain extent. Since the co-occurrence words of the keywords are determined based on the co-occurrence information of the keywords and the co-occurrence words in the standard question library, the co-occurrence information is the information of co-occurrence, therefore, the co-occurrence words of the keywords can be understood as words with a high degree of relevance to the keywords (such as co-appearing in the same standard question text). Then, according to the keywords of the first intent category and the co-occurrence words of the keywords, the target question text is mined from the pre-acquired target text set. It can be seen that the mining of target question text fully considers the keywords that can represent the intent category of the standard question text and the co-occurring words that are highly correlated with the keywords. For example, the same target question text includes both keywords and their co-occurring words. In this way, the keywords and co-occurring words in the target question text can accurately reflect the intent category corresponding to the target question text, so that the target question text has a high semantic generalization and corresponds to an accurate intent category, achieving the effect of mining high-quality target question text from the target text set. In addition, since the mining process of the target question text does not require human participation, this automated question mining method can greatly reduce the workload for users (such as operators) and is conducive to quickly and accurately expanding the standard question library.

[0156] Those skilled in the art should understand that Figure 2 The problem mining device in can be used to implement the problem mining method described above, and the detailed description thereof should be similar to the description of the method part above. To avoid tediousness, it will not be repeated here.

[0157] Based on the same idea, the embodiment of the present application also provides an electronic device, such as Figure 3As shown. The electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors 301 and memory 302, and the memory 302 may store one or more storage applications or data. Among them, the memory 302 can be a temporary storage or a permanent storage. The application stored in the memory 302 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the electronic device. Furthermore, the processor 301 can be configured to communicate with the memory 302 to execute a series of computer executable instructions in the memory 302 on the electronic device. The electronic device may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more input and output interfaces 305, and one or more keyboards 306.

[0158] Specifically in this embodiment, the electronic device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions for the electronic device, and the one or more programs are configured to be executed by one or more processors, including the following computer executable instructions:

[0159] Obtain a pre-built standard question library; the standard question library includes a first standard question text, the first standard question text corresponds to a first intention category; the first standard question text includes a plurality of words;

[0160] According to the importance of each word in the first standard question text to the first intent category, mining keywords of the first intent category from the multiple words; the multiple words include the keywords and non-keywords;

[0161] Determining co-occurrence words of the keywords according to co-occurrence information of the keywords and the non-keywords in the standard question library;

[0162] According to the keywords and the co-occurring words of the keywords, target question texts are mined from a pre-acquired target text set.

[0163] The technical solution of the embodiment of the present application is adopted, by obtaining a pre-constructed standard question library, the standard question library includes a first standard question text, and the first standard question text corresponds to a first intent category, according to the importance of each word in the first standard question text to the first intent category, the keywords of the first intent category are mined from multiple words (including keywords and non-keywords) in the first standard question text; and according to the co-occurrence information of keywords and non-keywords in the standard question library, the co-occurrence words of the keywords are determined. Since the keywords are mined based on the importance of each word in the first standard question text to the first intent category, the keywords can reflect the first intent category of the first standard question text to a certain extent. Since the co-occurrence words of the keywords are determined based on the co-occurrence information of the keywords and the co-occurrence words in the standard question library, the co-occurrence information is the co-occurrence information, therefore, the co-occurrence words of the keywords can be understood as words with a high degree of relevance to the keywords (such as co-appearing in the same standard question text). Then, according to the keywords of the first intent category and the co-occurrence words of the keywords, the target question text is mined from the pre-acquired target text set. It can be seen that the mining of target question text fully considers the keywords that can represent the intent category of the standard question text and the co-occurring words that are highly correlated with the keywords. For example, the same target question text includes both keywords and their co-occurring words. In this way, the keywords and co-occurring words in the target question text can accurately reflect the intent category corresponding to the target question text, so that the target question text has a high semantic generalization and corresponds to an accurate intent category, achieving the effect of mining high-quality target question text from the target text set. In addition, since the mining process of the target question text does not require human participation, this automated question mining method can greatly reduce the workload for users (such as operators) and is conducive to quickly and accurately expanding the standard question library.

[0164] The embodiment of the present application further proposes a computer-readable storage medium, which stores one or more computer programs, wherein the one or more computer programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to perform each process of the above-mentioned problem mining method embodiment, and are specifically used to perform:

[0165] Obtain a pre-built standard question library; the standard question library includes a first standard question text, the first standard question text corresponds to a first intention category; = the first standard question text includes a plurality of words;

[0166] According to the importance of each word in the first standard question text to the first intent category, mining keywords of the first intent category from the multiple words; the multiple words include the keywords and non-keywords;

[0167] Determining co-occurrence words of the keywords according to co-occurrence information of the keywords and the non-keywords in the standard question library;

[0168] According to the keywords and the co-occurring words of the keywords, target question texts are mined from a pre-acquired target text set.

[0169] The technical solution of the embodiment of the present application is adopted, by obtaining a pre-constructed standard question library, the standard question library includes a first standard question text, and the first standard question text corresponds to a first intent category, according to the importance of each word in the first standard question text to the first intent category, the keywords of the first intent category are mined from multiple words (including keywords and non-keywords) in the first standard question text; and according to the co-occurrence information of keywords and non-keywords in the standard question library, the co-occurrence words of the keywords are determined. Since the keywords are mined based on the importance of each word in the first standard question text to the first intent category, the keywords can reflect the first intent category of the first standard question text to a certain extent. Since the co-occurrence words of the keywords are determined based on the co-occurrence information of the keywords and the co-occurrence words in the standard question library, the co-occurrence information is the co-occurrence information, therefore, the co-occurrence words of the keywords can be understood as words with a high degree of relevance to the keywords (such as co-appearing in the same standard question text). Then, according to the keywords of the first intent category and the co-occurrence words of the keywords, the target question text is mined from the pre-acquired target text set. It can be seen that the mining of target question text fully considers the keywords that can represent the intent category of the standard question text and the co-occurring words that are highly correlated with the keywords. For example, the same target question text includes both keywords and their co-occurring words. In this way, the keywords and co-occurring words in the target question text can accurately reflect the intent category corresponding to the target question text, so that the target question text has a high semantic generalization and corresponds to an accurate intent category, achieving the effect of mining high-quality target question text from the target text set. In addition, since the mining process of the target question text does not require human participation, this automated question mining method can greatly reduce the workload for users (such as operators) and is conducive to quickly and accurately expanding the standard question library.

[0170] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0171] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0172] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0173] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0174] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0175] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0176] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0177] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0178] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0179] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0180] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0181] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0182] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A problem mining method, characterized in that: include: Obtain a pre-built standard question library; the standard question library includes a first standard question text, and the first standard question text corresponds to a first intention category; The first standard question text includes a plurality of words; According to the importance of each word in the first standard question text to the first intent category, mining keywords of the first intent category from the multiple words; the multiple words include the keywords and non-keywords; Determining co-occurrence words of the keywords according to co-occurrence information of the keywords and the non-keywords in the standard question library; According to the keywords and the co-occurring words of the keywords, target question texts are mined from a pre-acquired target text set.

2. The method according to claim 1, characterized in that The step of mining keywords of the first intent category from the plurality of words according to the importance of each word in the first standard question text to the first intent category includes: Determine a target long text corresponding to the first intention category according to the first standard question text; the target long text is composed of at least one of the first standard question texts; Determine the first occurrence information of each word in the target long text in the target long text; and determine the second occurrence information of each word in the target long text in the standard question library; Determining the importance of each word in the target long text to the first intention category according to the first occurrence information and the second occurrence information; According to the importance of each word in the target long text to the first intent category, the keywords of the first intent category are mined from the multiple words; the keywords are words whose importance is higher than or equal to a preset importance threshold.

3. The method according to claim 2, characterized in that The first occurrence information includes occurrence frequency; The determining the first occurrence information of each word in the target long text in the target long text includes: Determine the first occurrence number of a first word in the target long text; the first word is any word in the target long text; The occurrence frequency of the first word in the target long text is determined according to the first occurrence number and the total number of words included in the target long text.

4. The method according to claim 2, characterized in that: The second occurrence information includes an inverse text frequency; The determining the second occurrence information of each word in the target long text in the standard question library includes: Determine the number of first texts of the target long text including a second word in the target long text corresponding to each intention category in the standard question library; the second word is any word in the target long text; The inverse text frequency corresponding to the second word is determined according to the first text quantity and the total text quantity of the target long text.

5. The method according to claim 1, characterized in that The co-occurrence information includes co-occurrence degree; The step of determining the co-occurrence words of the keywords according to the co-occurrence information of the keywords and the non-keywords in the standard question library includes: Determine the number of second texts of standard question texts including the keyword in the N standard question texts; and determine the number of third texts of standard question texts including both the keyword and the non-keyword in the N standard question texts; Determining the co-occurrence of the keywords and the non-keywords in the standard question library according to the second text quantity, the third text quantity, and the total text quantity of the N standard question texts; A non-keyword whose co-occurrence degree with the keyword is greater than or equal to a preset co-occurrence degree threshold is determined as the co-occurrence word.

6. The method according to claim 1, characterized in that The step of mining a target question text from a pre-acquired target text set according to the keyword and the co-occurring words of the keyword includes: Screening candidate question texts from the target text set; the candidate question texts are question texts that include both the keywords and the non-keywords; Predicting the intent category to which the candidate question text belongs, and obtaining an intent category prediction result of the candidate question text; According to the intention category prediction result, it is determined whether the candidate question text is the target question text.

7. The method according to claim 6, characterized in that The intention category prediction result includes a first predicted intention category; The step of predicting the intent category to which the candidate question text belongs to obtain a prediction result of the intent category of the candidate question text, comprises: Performing clustering processing on N standard question texts to obtain a clustering result; the clustering result includes a plurality of question text sets; each question text set includes a plurality of the standard question texts; Determine the central question text corresponding to each question text set; the central question text is the standard question text closest to the cluster center corresponding to the question text set; Selecting, from the plurality of central question texts, the central question text having the highest similarity to the candidate question texts; The intent category of the central question text having the highest similarity to the candidate question text is determined as the first predicted intent category.

8. The method according to claim 7, characterized in that The step of determining whether the candidate question text is the target question text according to the intention category prediction result includes: If the first predicted intention category is the same as the intention category corresponding to the keyword, determining the candidate question text as the target question text; If the first predicted intent category and the intent category corresponding to the keyword are different, it is determined that the candidate question text is not the target question text.

9. The method according to any one of claims 6 to 8, characterized in that: The intention category prediction result includes: the probability that the candidate question text belongs to each intention category; The predicting the intent category to which the candidate question text belongs to obtain a prediction result of the intent category corresponding to the candidate question text, includes: The pre-trained intent recognition model is used to predict the intent category to which the candidate question text belongs, and the probability that the candidate question text belongs to each intent category is obtained; wherein the intent recognition PC2310039CN model is trained based on sample question text and the sample intent category of the sample question text.

10. The method according to claim 9, characterized in that The step of determining whether the candidate question text is the target question text according to the intention category prediction result includes: Calculating the information entropy of the candidate question text according to the probability that the candidate question text belongs to each intention category; If the information entropy is greater than or equal to a preset information entropy threshold, determining the candidate question text as the target question text; If the information entropy is less than the preset information entropy threshold, it is determined that the candidate question text is not the target question text.

11. A problem mining device, characterized in that: include: An acquisition module, configured to acquire a pre-built standard question library; the standard question library includes a first standard question text, and the first standard question text corresponds to a first intention category; The first standard question text includes a plurality of words; A first mining module is used to mine keywords of the first intent category from the multiple words according to the importance of each word in the first standard question text to the first intent category; the multiple words include the keywords and non-keywords; A determination module, configured to determine co-occurrence words of the keyword according to co-occurrence information of the keyword and the non-keyword in the standard question library; The second mining module is used to mine the target question text from the pre-acquired target text set according to the keywords and the co-occurring words of the keywords.

12. An electronic device, characterized in that: It comprises a processor and a memory electrically connected to the processor, the memory stores a computer program, and the processor is used to call and execute the computer program from the memory to implement the problem mining method as described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that: The storage medium is used to store a computer program, and the computer program can be executed by a processor to implement the problem mining method as described in any one of claims 1-10.

Images