Data generation method and device, electronic equipment, medium and program product
By generating candidate keyword lists for corpus processing in corpus sets and selecting target keyword lists, the problem of low accuracy of keyword lists is solved, and the accuracy of intention recognition is improved.
Patent Information
- Application Number
- CN202411040677.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-12-17
AI Technical Summary
In the prior art, the accuracy of the keyword list depends on the accurate keyword list, and how to generate a keyword list with high accuracy is a technical challenge.
By processing the corpus in the corpus set, multiple candidate keyword tables are generated and the target keyword table is selected based on the evaluation value to ensure the completeness and accuracy of the keyword table.
Improve the accuracy and completeness of keyword lists and enhance the accuracy of intention recognition.
Smart Images
Figure CN119150852A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a data generation method, device, electronic device, medium and program product. Background Art
[0002] Intent recognition technology is used in many scenarios, especially in human-computer dialogue scenarios. In the process of human-computer dialogue scenarios, it is usually necessary to first identify the intent of the text sent by the client, and then respond based on the recognition results. In order to improve the accuracy of intent recognition, the client's text is usually fully matched based on the keyword table. If the match is successful, the matching result of the keyword table is directly used as the user's intention. If the match fails, it is further matched based on other means. It can be seen that the accuracy of this method depends on an accurate keyword table, so how to generate a highly accurate keyword table is a technical problem that needs to be solved. Summary of the invention
[0003] The present application provides a data generation method, device, electronic device, medium and program product.
[0004] In a first aspect, an embodiment of the present application provides a data generation method, comprising:
[0005] Processing the corpus in the corpus set to obtain a plurality of keyword information, wherein the plurality of keyword information corresponds one-to-one to a plurality of preset intent tags;
[0006] Generate multiple candidate keyword tables based on the multiple keyword information, the candidate keyword tables including the multiple preset intent tags and the keywords included in the keyword information corresponding to each of the preset intent tags;
[0007] Determining a first evaluation value of each of the candidate keyword tables, wherein the first evaluation value is used to evaluate the accuracy of the candidate keyword table;
[0008] A target keyword table is determined from the plurality of candidate keyword tables, wherein a first evaluation value of the target keyword table is greater than or equal to a first threshold.
[0009] In a second aspect, an embodiment of the present application provides a data generating device, including:
[0010] A processing module, used for processing the corpus in the corpus set to obtain a plurality of keyword information, wherein the plurality of keyword information corresponds one-to-one to a plurality of preset intent tags;
[0011] A generating module, configured to generate a plurality of candidate keyword tables based on the plurality of keyword information, wherein the candidate keyword tables include the plurality of preset intent tags and keywords included in the keyword information corresponding to each of the preset intent tags;
[0012] A determination module, used to determine a first evaluation value of each candidate keyword table, wherein the first evaluation value is used to evaluate the accuracy of the candidate keyword table;
[0013] The determination module is further configured to determine a target keyword table from the plurality of candidate keyword tables, wherein a first evaluation value of the target keyword table is greater than or equal to a first threshold.
[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the method steps of the first aspect described above are implemented.
[0015] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method steps of the first aspect described above are implemented.
[0016] In a fifth aspect, an embodiment of the present application further provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the method described in the first aspect.
[0017] It can be seen that in the embodiment of the present application, each candidate keyword table generated includes multiple preset intent tags and keywords included in the keyword information corresponding to each preset intent tag, so that each candidate keyword table generated covers the keywords corresponding to each preset intent tag, thereby improving the completeness of the keyword table, and then based on the evaluation value of each candidate keyword table, a target keyword table is selected from multiple candidate keyword tables, so that the accuracy of the selected target keyword table is more reasonable, thereby improving the accuracy of the generated keyword table. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0019] Figure 1 is a flow chart of a data generation method provided in an embodiment of the present application;
[0020] Figure 2 It is one of the structural schematic diagrams of the data generating device provided in the embodiment of the present application.
[0021] Figure 3 This is the second structural diagram of the data generating device provided in the embodiment of the present application. DETAILED DESCRIPTION
[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0023] In the related art, the method for generating a keyword table generally includes the following steps: obtaining session data, setting intent tags for the session data, performing word segmentation on the session data, and then using the word segments with higher frequency in the session data corresponding to each intent tag as the keywords corresponding to the intent tag, thereby generating a corresponding keyword table, which includes multiple intent tags and the keywords corresponding to each intent tag.
[0024] It can be seen that in the related art, after determining the keyword information of the intent tag, the determined keyword information is directly used as the keyword information in the keyword table to generate the keyword table. In the embodiment of the present application, after determining the keyword information of the preset intent tag, multiple candidate keyword tables are generated based on the keyword information, and the keyword information in each candidate keyword table is different. Then, the target keyword table is determined from the multiple candidate keyword tables. Compared with the related art, the embodiment of the present application generates multiple keyword tables based on the keyword information. In this way, it is beneficial to select a relatively better solution from multiple solutions to obtain the target keyword table, which is beneficial to improve the generation effect of the keyword table.
[0025] See also Figure 1 , Figure 1 A flow chart of a data generation method provided in an embodiment of the present application, the data generation method comprising:
[0026] Step 101: Process the corpus in the corpus set to obtain a plurality of keyword information, wherein the plurality of keyword information corresponds one-to-one to a plurality of preset intent tags;
[0027] Step 102: Generate multiple candidate keyword tables based on the multiple keyword information, wherein the candidate keyword tables include the multiple preset intent tags and the keywords included in the keyword information corresponding to each of the preset intent tags;
[0028] Step 103: determining a first evaluation value of each candidate keyword table, wherein the first evaluation value is used to evaluate the accuracy of the candidate keyword table;
[0029] Step 104: Determine a target keyword table from the multiple candidate keyword tables, wherein a first evaluation value of the target keyword table is greater than or equal to a first threshold.
[0030] The above-mentioned corpus may be a corpus generated by annotating corpuses in various human-computer conversation scenarios, wherein the annotating of the corpus means setting intent tags for the corpus.
[0031] The above-mentioned multiple preset intention labels can be determined according to specific scenarios. For example, in scenarios related to the financial industry, the multiple preset intention labels may include: withholding approval, deferred repayment, refusal to repay, etc.
[0032] The above-mentioned processing of the corpus in the corpus set to obtain multiple keyword information may specifically include: performing word segmentation processing on the corpus in the corpus set, and generating keyword information corresponding to the preset intent label based on the word segmentation result of the corpus corresponding to each preset intent label.
[0033] The keyword information corresponding to the above-mentioned preset intent tag can be generated based on all the corpora corresponding to the preset intent tag. Specifically, all the corpora corresponding to the preset intent tag can be first determined in the corpus set, and then all the determined corpora are segmented to obtain all the segmented words corresponding to the preset intent tag, and then multiple keywords are determined from the segmented words corresponding to the preset intent tag, and the determined multiple keywords are used as the keyword information corresponding to the preset intent tag.
[0034] Generating multiple candidate keyword tables based on the multiple keyword information may specifically include: generating multiple candidate keyword tables corresponding to the multiple preset intent tags based on different keyword incorporation methods, wherein in different candidate keyword tables, the keywords corresponding to the multiple preset intent tags are different. In the different candidate keyword tables, the different keywords corresponding to the multiple preset intent tags may mean: in different candidate keyword tables, the keywords corresponding to all preset intent tags are different. Alternatively, the different keywords corresponding to the multiple preset intent tags may also mean: in different candidate keyword tables, there are some preset intent tags that correspond to different keywords. For example, the above-mentioned multiple candidate keyword tables include at least two of the following:
[0035] The first keyword table, wherein the first keyword table includes: the multiple preset intent tags, and the keyword information corresponding to each preset intent tag; wherein the first keyword table refers to: a keyword table generated by incorporating all the keywords corresponding to each intent tag into the keyword table, wherein in the first keyword table, the preset intent tags and the corresponding keyword information have a one-to-one correspondence. In the process of intent recognition based on the first keyword table, the text to be recognized can be segmented to obtain a number of segmented words, and then the obtained segmented words are matched with the keyword information in the first keyword table. If the obtained segmented words are exactly the same as the keywords in the target keyword information, the intent indicated by the preset intent tag corresponding to the target keyword information is determined as the intent recognition result. wherein the target keyword information can be the keyword information corresponding to any one of the multiple preset intent tags.
[0036] A second keyword table, wherein the second keyword table includes: the plurality of preset intent tags, and some keywords in the keyword information corresponding to each intent tag; wherein the second keyword table refers to: a keyword table generated by incorporating some keywords corresponding to each intent tag into the keyword table. Since there are multiple combinations of some keywords in the keyword information corresponding to each intent tag, the plurality of candidate keyword tables may include at least one second keyword table.
[0037] A third keyword table, wherein the third keyword table includes: the plurality of preset intent tags, and the first keyword information corresponding to each intent tag, the first keyword information corresponding to the target intent tag includes: at least some of the keywords in the keyword information corresponding to the target intent tag, and at least some of the keywords in the keyword information corresponding to the first intent tag, the target intent tag is any one of the plurality of preset intent tags, and the first intent tag is a preset intent tag among the plurality of preset intent tags whose semantic similarity with the target intent tag is greater than or equal to a preset threshold; wherein, the vector distance between each preset intent tag can be calculated, and the vector distance can be used as the semantic similarity between the two preset intent tags, and the preset threshold can be a threshold set according to actual conditions, for example, when the vector distance is mapped to between 0-1, the preset threshold can be 0.7-0.9. Since there are multiple combinations of at least some of the keywords in the keyword information corresponding to the target intent tag and at least some of the keywords in the keyword information corresponding to the first intent tag, at least one third keyword table can be included in the plurality of candidate keyword tables.
[0038] A fourth keyword table, wherein the fourth keyword table includes: the multiple preset intention tags, and the second keyword information corresponding to each intention tag, the second keyword information corresponding to the target intention tag includes: at least part of the keywords in the keyword information corresponding to the target intention tag, and at least part of the keywords in the keyword information corresponding to the second intention tag, the second intention tag is a preset intention tag among the multiple preset intention tags, the semantic similarity with the target intention tag being less than the preset threshold; since there are multiple combinations of at least part of the keywords in the keyword information corresponding to the target intention tag and at least part of the keywords in the keyword information corresponding to the second intention tag, the above-mentioned multiple candidate keyword tables may include at least one fourth keyword table.
[0039] A fifth keyword table, wherein the fifth keyword table includes: the multiple preset intent tags, and the third keyword information corresponding to each intent tag, and the third keyword information corresponding to the target intent tag includes: at least some of the keywords in the keyword information corresponding to the target intent tag, and at least one randomly generated keyword. Since there are multiple combinations of at least some of the keywords in the keyword information corresponding to the target intent tag and at least one randomly generated keyword, the multiple candidate keyword tables may include at least one fifth keyword table.
[0040] The above-mentioned method for determining the first evaluation value of each candidate keyword table may be various methods for evaluating the accuracy of the keyword table. It can be understood that the higher the first evaluation value is, the higher the accuracy of the candidate keyword table is. The value range of the above-mentioned first threshold value can be set according to actual needs. For example, when the value range of the above-mentioned first evaluation value is 0-1, the first threshold value may be a relatively high value such as 0.9 or 0.95. The above-mentioned target keyword table may be any candidate keyword table among the multiple candidate keyword tables whose first evaluation value is greater than or equal to the first threshold value. Alternatively, the target keyword table may also be: among the candidate keyword tables whose first evaluation value is greater than or equal to the first threshold value, the candidate keyword table with the largest first evaluation value.
[0041] In this implementation, each candidate keyword table generated includes multiple preset intent tags and keywords included in the keyword information corresponding to each preset intent tag, so that each candidate keyword table generated covers the keywords corresponding to each preset intent tag, thereby improving the completeness of the keyword table. Then, based on the evaluation value of each candidate keyword table, a target keyword table is selected from the multiple candidate keyword tables, so that the accuracy of the selected target keyword table is more reasonable, thereby improving the accuracy of the generated keyword table.
[0042] Optionally, the processing of the corpus in the corpus set to obtain multiple keyword information includes:
[0043] Determine, from the corpus set, the corpus corresponding to each of the preset intent labels, and obtain a corpus subset corresponding to each of the preset intent labels;
[0044] Performing word segmentation processing on the corpus in each of the corpus subsets to obtain a first word set corresponding to each of the corpus subsets;
[0045] The multiple keyword information is generated based on the second evaluation value and the third evaluation value of each word in the first word set, the second evaluation value is used to evaluate the weight of the word in the corresponding corpus, and the second evaluation value is used to evaluate the occurrence frequency of the word in the corresponding word set.
[0046] Among them, the weight of the above-mentioned word in the belonging corpus can specifically refer to: evaluating the semantic weight of the word in the belonging corpus. Specifically, the higher the weight of a word in the belonging corpus, the closer the word is to the semantics of the belonging corpus, and accordingly, the more likely the word is a keyword in the corpus.
[0047] Correspondingly, the higher the frequency of a word appearing in the corresponding word set, the more likely it is that the word is a keyword in the corresponding word set.
[0048] In this implementation, the plurality of keyword information is generated based on the second evaluation value and the third evaluation value of each word in the first word set, which is beneficial to improving the accuracy of the generated keyword information.
[0049] Optionally, the plurality of keyword information corresponds to a plurality of the first word sets one by one; for each of the first word sets, generating keyword information based on the second evaluation value and the third evaluation value of the words in the first word set includes:
[0050] Determine a second word set from the first word set based on a second evaluation value of the words in the first word set; and determine a third word set from the first word set based on a third evaluation value of the words in the first word set;
[0051] generating keyword information based on the second word set and the third word set;
[0052] The second word set includes a plurality of first words and a word group formed by two semantically related first words among the plurality of first words, and the second evaluation value of the first word is greater than or equal to a second threshold;
[0053] The third word set includes a plurality of second words and a word group formed by two semantically related second words among the plurality of second words, and a third evaluation value of the second words is greater than or equal to a third threshold.
[0054] It can be understood that each first word set can generate a corresponding keyword information according to the above method. The following further explains the generation process of the plurality of keyword information based on the generation of a keyword information corresponding to one of the first word sets:
[0055] First, the word concentration of each word in the first word set can be calculated, and the word concentration value of each word can be used as the second evaluation value of the word. Then, all words in the first word set whose second evaluation value is greater than or equal to the second threshold are determined as first words, and the determined first words are added to the second word set. Correspondingly, the word frequency of each word in the first word set can be calculated, and the word frequency of each word is used as the third evaluation value of the word, and then, all words in the first word set whose third evaluation value is greater than or equal to the third threshold are determined as second words, and the determined second words are added to the third word set. Among them, the value range of the second threshold and the third threshold can be determined according to actual needs. For example, when the value range of the second evaluation value is 0-1, the second threshold can be 0.5 or 0.6, etc. Correspondingly, the third threshold can also be determined according to the specific situation. For example, in some embodiments of the present application, when the word frequency of the word with the highest word frequency in the first word set is 8, the third threshold can be any value between 2-5.
[0056] Since there may be more than two words from the determined first words that are derived from the same corpus, there may be more than two first words that are adjacent words in the target corpus. When generating the second word set, in addition to using the above first words as keywords in the second word set, two semantically related first words in the determined first words may be spliced, and the spliced words may be used as keywords in the second word set, wherein the two semantically related first words may be two adjacent first words in the target corpus. The splicing method may include positive splicing and reverse splicing, that is, when the first word A and the first word B are two adjacent words in the target corpus, and the first word B is located after the first word A in the target corpus, the words obtained after splicing may include: "first word A first word B" and "first word B first word A". The target corpus may be any corpus in the corpus subset corresponding to the first word set.
[0057] Accordingly, since there may be more than two words from the determined second words that are derived from the same corpus, there may be more than two second words that are adjacent words in the target corpus. When generating the third word set, in addition to using the second words as keywords in the third word set, two semantically related second words in the determined second words may be spliced, and the spliced words may be used as keywords in the third word set, wherein the two semantically related second words may be two adjacent second words in the target corpus. The splicing method may include positive splicing and reverse splicing, that is, when the second word C and the second word D are two adjacent words in the target corpus, and the second word D is located after the second word C in the target corpus, the words obtained after splicing may include: "second word C second word D" and "second word D second word C".
[0058] The generating of keyword information based on the second word set and the third word set may specifically refer to: removing overlaps and merging the second word set with the third word set to obtain the keyword information.
[0059] In some embodiments of the present application, the process of determining the second word set from the first word set based on the second evaluation value of the words in the first word set may also refer to: first, selecting the first n words with larger word concentration values from the first word set as n first words, and then determining the semantically associated first words among the n first words, and splicing the semantically associated first words to obtain spliced words. In this way, the n first words and all spliced words obtained by splicing the first words can be used as keywords in the second word set.
[0060] Accordingly, in some embodiments of the present application, the process of determining the third word set from the first word set based on the third evaluation value of the words in the first word set may also refer to: first, selecting the first m words with higher frequency from the first word set as m second words, and then determining the semantically related second words among the m second words, and splicing the semantically related second words to obtain spliced words. In this way, the m second words and all the spliced words obtained by splicing the second words can be used as keywords in the third word set.
[0061] In this implementation, a second word set and a third word set are generated based on word concentration values and word frequencies respectively, and keyword information is determined based on the second word set and the third word set. In this way, since both word concentration values and word frequencies are considered in the process of generating keyword information, it is beneficial to improve the generation effect of keyword information.
[0062] Optionally, the second evaluation value of a third word included in the first word set is determined based on a distance between word vectors of the third word and a fourth word in the first word set.
[0063] The number of the fourth word can be one or more, and the third word and the fourth word are from the same corpus. If the number of the fourth word is multiple, then the fourth word can be other words in the first word set that belong to the same corpus as the third word. For example, assuming that the words in the first word set that belong to the same corpus as the third word are word 1, word 2, and word 3, then the fourth words are word 1, word 2, and word 3.
[0064] The second evaluation value may specifically refer to a word concentration value, which may be determined by calculating the vector distance between each word in the corpus. The smaller the vector distance between two words, the greater the semantic similarity between the two words. It is understandable that when a word is closer in semantic similarity to other words in the corpus to which it belongs, the greater the word concentration value of the word in the corpus to which it belongs. Correspondingly, the greater the semantic weight of the word in the corpus to which it belongs, the greater the second evaluation value of the word.
[0065] Among them, the second evaluation value of the target word can be determined based on the sum of the word vector distances between the target word and all words in the target corpus, the target corpus is any corpus in the corpus subset corresponding to the first word set, and the target word is any word in the target corpus. Specifically, the second evaluation value of the target word in the above-mentioned first word set can be negatively correlated with "the sum of the word vector distances between the target word and all words in the target corpus".
[0066] It can be understood that when calculating the second evaluation value of the target word, multiple words after word segmentation processing of the target corpus can be obtained from the first word set, and then all the obtained words are input into the pre-trained language model for vector conversion to obtain the word vector corresponding to each word, and the distance between the word vector of each word and the word vectors of other words in the target corpus is calculated. Then, the inverse of the sum of the distances between the target word and all other words is taken as the second evaluation value of the target word. The higher the second evaluation value, the higher the probability that the corresponding word is a keyword.
[0067] In this implementation, the second evaluation value of a word is determined based on the distance between the word vectors of the word and other words in the first word set, which is helpful to improve the accuracy of the determined second evaluation value.
[0068] Optionally, the method further comprises:
[0069] Perform intent recognition on each piece of corpus in the conversation data to obtain the intent label corresponding to each piece of corpus;
[0070] Based on the intent label corresponding to each corpus, ratio information is determined; the ratio information includes the ratio corresponding to each preset intent label, and the ratio is the ratio of the number of corpus corresponding to the preset intent label in the session data to the total amount of data of the session data;
[0071] The corpus is generated based on the ratio information, wherein the number of corpora corresponding to each of the preset intent labels in the corpus matches the ratio information.
[0072] The conversation data may be historical conversation data, and the historical conversation data may include historical conversation data of conversations between the intelligent conversation robot and all customers.
[0073] The above-mentioned performing intent recognition on each corpus in the conversation data and obtaining the intent label corresponding to each corpus may specifically refer to: inputting each corpus in the conversation data into the intent recognition model for recognition, and obtaining the intent label of each corpus. Then, the number of corpuses corresponding to each preset intent label in the conversation data is counted, and the proportion information is generated according to the counted number. For example, when the conversation data includes 1,000 corpuses, if the number of corpuses corresponding to the preset intent label "batch deduction" is 100, then the proportion information corresponding to the preset intent label "batch deduction" can be 10%. Correspondingly, the number of corpuses corresponding to the preset intent label "delayed repayment" is 100, then the proportion information corresponding to the preset intent label "delayed repayment" is also 10%.
[0074] The matching of the number of corpora corresponding to each preset intent label in the corpus with the ratio information means that the ratio of the number of corpora corresponding to each preset intent label in the corpus is the same as the ratio indicated by the ratio information.
[0075] It can be understood that the above corpus can be a data set generated by intentionally annotating part of the corpus in the conversation data. For example, when the conversation data includes 1,000 pieces of corpus, the corpus can include 200 pieces of corpus. Since the number of corpus corresponding to each preset intention label in the corpus matches the proportion information, and the proportion information corresponding to the preset intention label "batch deduction" can be 10%, and the proportion information corresponding to the preset intention label "delayed repayment" is also 10%, therefore, the number of corpus with the intention label "batch deduction" in the corpus is 20, and the number of corpus with the intention label "delayed repayment" in the corpus is also 20, thereby achieving corpus balance.
[0076] In this implementation, by matching the number of corpora corresponding to each of the preset intent labels in the corpus set with the proportion information, it is helpful to balance the corpora in the corpus set, thereby further improving the generation effect of keyword information.
[0077] Optionally, for each of the candidate keyword tables, determining a first evaluation value of the candidate keyword table includes:
[0078] Input the test corpus in the preset test set into the intent recognition model for intent recognition, and obtain the predicted intent of each of the test corpus; wherein the candidate keyword table is the keyword table of the intent recognition model; the intent recognition includes: matching the test corpus with the candidate keyword table; if the test corpus matches the keyword in the candidate keyword table, then taking the intent label corresponding to the matched keyword as the predicted intent of the test corpus; each test corpus in the preset test set corresponds to a real intent;
[0079] Determining the accuracy of intent recognition of the preset test set based on the predicted intent and true label of the test corpus;
[0080] A first evaluation value of the candidate keyword table is determined based on the accuracy rate.
[0081] Specifically, each of the multiple candidate keyword tables can be used as a keyword table corresponding to a preset intent recognition model, and intent recognition is performed on the test corpus in the preset test set to obtain a first evaluation value corresponding to each candidate keyword table. The specific process may include: performing intent recognition on each test corpus in the preset test set based on the candidate keyword table and the preset intent recognition model, obtaining the accuracy of intent recognition of the preset test set, and determining the first evaluation value of the candidate keyword table based on the accuracy.
[0082] Among them, the inputting of the test corpus in the preset test set into the intent recognition model for intent recognition to obtain the predicted intent of each of the test corpus may include: performing word processing on the test corpus to obtain multiple words, and then matching the multiple words obtained with the candidate keyword table. If the match is successful, the preset intent label obtained by matching is used as the predicted intent. If the match fails, the intent recognition of the test corpus is performed based on the preset intent recognition model to obtain the predicted intent, and then the predicted intent is compared with the real intent corresponding to the test corpus to determine the accuracy of the recognition. When the predicted intent is the same as the real intent, it can be determined that the test corpus is correctly recognized. When the predicted intent is different from the real intent, it can be determined that the test corpus is incorrectly recognized. Then, the number of test corpora that are correctly identified and the number of test corpora that are intended to be recognized based on a candidate keyword table in a preset test set can be counted, and based on the number of test corpora that are correctly identified, the accuracy of the candidate keyword table in identifying the intent of the preset test set can be determined. For example, when the preset test set includes 100 test corpora, and the number of test corpora that are correctly identified based on the candidate keyword table is 95, the first evaluation value of the candidate keyword table is 0.95.
[0083] In this way, according to the above method, the first evaluation value of each candidate keyword table in the plurality of candidate keyword tables can be determined.
[0084] In this embodiment, the first evaluation value of each candidate keyword table is determined based on the intention recognition results of each candidate keyword table on the test corpus in the preset test set. This is conducive to accurately evaluating the accuracy of each candidate keyword table, thereby facilitating improving the accuracy of the first evaluation value of each determined candidate keyword table.
[0085] See also Figure 2 , Figure 2 A schematic diagram of the structure of a data generating device 200 provided in an embodiment of the present application, wherein the data generating device 200 comprises:
[0086] A processing module 201 is used to process the corpus in the corpus set to obtain a plurality of keyword information, wherein the plurality of keyword information corresponds to a plurality of preset intent tags one by one;
[0087] A generating module 202, configured to generate a plurality of candidate keyword tables based on the plurality of keyword information, wherein the candidate keyword tables include the plurality of preset intent tags and the keywords included in the keyword information corresponding to each of the preset intent tags;
[0088] A determination module 203, configured to determine a first evaluation value of each candidate keyword table, wherein the first evaluation value is used to evaluate the accuracy of the candidate keyword table;
[0089] The determining module 203 is further configured to determine a target keyword table from the plurality of candidate keyword tables, wherein a first evaluation value of the target keyword table is greater than or equal to a first threshold.
[0090] Optionally, the processing module 201 includes:
[0091] A determination submodule is used to determine the corpus corresponding to each of the preset intention labels from the corpus set, and obtain a corpus subset corresponding to each of the preset intention labels;
[0092] A word segmentation submodule, used for performing word segmentation processing on the corpus in each of the corpus subsets to obtain a first word set corresponding to each of the corpus subsets;
[0093] A generation submodule is used to generate the multiple keyword information based on the second evaluation value and the third evaluation value of each word in the first word set, wherein the second evaluation value is used to evaluate the weight of the word in the corresponding corpus, and the second evaluation value is used to evaluate the frequency of occurrence of the word in the corresponding word set.
[0094] Optionally, the plurality of keyword information corresponds one-to-one to the plurality of first word sets; and the generating submodule includes:
[0095] A determining unit, configured to determine a second word set from the first word set based on a second evaluation value of a word in the first word set; and to determine a third word set from the first word set based on a third evaluation value of a word in the first word set;
[0096] A generating unit, configured to generate keyword information based on the second word set and the third word set;
[0097] The second word set includes a plurality of first words and a word group formed by two semantically related first words among the plurality of first words, and the second evaluation value of the first word is greater than or equal to a second threshold;
[0098] The third word set includes a plurality of second words and a word group formed by two semantically related second words among the plurality of second words, and a third evaluation value of the second words is greater than or equal to a third threshold.
[0099] Optionally, the second evaluation value of a third word included in the first word set is determined based on a distance between word vectors of the third word and a fourth word in the first word set.
[0100] Optionally, the device further comprises:
[0101] The intent recognition module is used to recognize the intent of each piece of corpus in the conversation data and obtain the intent label corresponding to each piece of corpus;
[0102] The determination module 203 is further configured to determine proportion information based on the intent label corresponding to each corpus; the proportion information includes a proportion corresponding to each preset intent label, and the proportion is a ratio of the number of corpuses corresponding to the preset intent label in the session data to the total amount of data in the session data;
[0103] The generating module 202 is further configured to generate the corpus based on the ratio information, wherein the number of corpora corresponding to each of the preset intent labels in the corpus matches the ratio information.
[0104] Optionally, the determination module 203 is specifically used to input the test corpus in the preset test set into the intent recognition model for intent recognition, and obtain the predicted intent of each of the test corpus; wherein the candidate keyword table is the keyword table of the intent recognition model; the intent recognition includes: matching the test corpus with the candidate keyword table; if the test corpus matches the keyword in the candidate keyword table, then taking the intent label corresponding to the matched keyword as the predicted intent of the test corpus; each test corpus in the preset test set corresponds to a real intent;
[0105] The determination module 203 is further configured to determine the accuracy of the intent recognition of the preset test set based on the predicted intent and the true label of the test corpus;
[0106] The determination module 203 is further configured to determine a first evaluation value of the candidate keyword table based on the accuracy rate.
[0107] See also Figure 3 , Figure 3 is a structural diagram of a data generating device 300 provided in another embodiment of the present application, such as Figure 3 As shown, the data generating device 300 includes: a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor. The various components in the database generating device are coupled together via a bus interface 303. When the computer program is executed by the processor 301, the following steps are implemented:
[0108] Processing the corpus in the corpus set to obtain a plurality of keyword information, wherein the plurality of keyword information corresponds one-to-one to a plurality of preset intent tags;
[0109] Generate multiple candidate keyword tables based on the multiple keyword information, the candidate keyword tables including the multiple preset intent tags and the keywords included in the keyword information corresponding to each of the preset intent tags;
[0110] Determining a first evaluation value of each of the candidate keyword tables, wherein the first evaluation value is used to evaluate the accuracy of the candidate keyword table;
[0111] A target keyword table is determined from the plurality of candidate keyword tables, wherein a first evaluation value of the target keyword table is greater than or equal to a first threshold.
[0112] Optionally, the processing of the corpus in the corpus set to obtain multiple keyword information includes:
[0113] Determine, from the corpus set, the corpus corresponding to each of the preset intent labels, and obtain a corpus subset corresponding to each of the preset intent labels;
[0114] Performing word segmentation processing on the corpus in each of the corpus subsets to obtain a first word set corresponding to each of the corpus subsets;
[0115] The multiple keyword information is generated based on the second evaluation value and the third evaluation value of each word in the first word set, the second evaluation value is used to evaluate the weight of the word in the corresponding corpus, and the second evaluation value is used to evaluate the occurrence frequency of the word in the corresponding word set.
[0116] Optionally, the plurality of keyword information corresponds to a plurality of the first word sets one by one; for each of the first word sets, generating keyword information based on the second evaluation value and the third evaluation value of the words in the first word set includes:
[0117] Determine a second word set from the first word set based on a second evaluation value of the words in the first word set; and determine a third word set from the first word set based on a third evaluation value of the words in the first word set;
[0118] generating keyword information based on the second word set and the third word set;
[0119] The second word set includes a plurality of first words and a word group formed by two semantically related first words among the plurality of first words, and the second evaluation value of the first word is greater than or equal to a second threshold;
[0120] The third word set includes a plurality of second words and a word group formed by two semantically related second words among the plurality of second words, and a third evaluation value of the second words is greater than or equal to a third threshold.
[0121] Optionally, the second evaluation value of a third word included in the first word set is determined based on a distance between word vectors of the third word and a fourth word in the first word set.
[0122] Optionally, the method further comprises:
[0123] Perform intent recognition on each piece of corpus in the conversation data to obtain the intent label corresponding to each piece of corpus;
[0124] Based on the intent label corresponding to each corpus, ratio information is determined; the ratio information includes the ratio corresponding to each preset intent label, and the ratio is the ratio of the number of corpus corresponding to the preset intent label in the session data to the total amount of data of the session data;
[0125] The corpus is generated based on the ratio information, wherein the number of corpora corresponding to each of the preset intent labels in the corpus matches the ratio information.
[0126] Optionally, for each of the candidate keyword tables, determining a first evaluation value of the candidate keyword table includes:
[0127] Input the test corpus in the preset test set into the intent recognition model for intent recognition, and obtain the predicted intent of each of the test corpus; wherein the candidate keyword table is the keyword table of the intent recognition model; the intent recognition includes: matching the test corpus with the candidate keyword table; if the test corpus matches the keyword in the candidate keyword table, then taking the intent label corresponding to the matched keyword as the predicted intent of the test corpus; each test corpus in the preset test set corresponds to a real intent;
[0128] Determining the accuracy of intent recognition of the preset test set based on the predicted intent and true label of the test corpus;
[0129] A first evaluation value of the candidate keyword table is determined based on the accuracy rate.
[0130] An embodiment of the present application also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0131] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. Among them, the computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0132] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0133] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for an electronic device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present application.
[0134] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. A data generation method, characterized in that: include: Processing the corpus in the corpus set to obtain a plurality of keyword information, wherein the plurality of keyword information corresponds one-to-one to a plurality of preset intent tags; Generate multiple candidate keyword tables based on the multiple keyword information, the candidate keyword tables including the multiple preset intent tags and the keywords included in the keyword information corresponding to each of the preset intent tags; Determining a first evaluation value of each of the candidate keyword tables, wherein the first evaluation value is used to evaluate the accuracy of the candidate keyword table; A target keyword table is determined from the plurality of candidate keyword tables, wherein a first evaluation value of the target keyword table is greater than or equal to a first threshold.
2. The method according to claim 1, characterized in that: The corpus in the corpus set is processed to obtain multiple keyword information, including: Determine, from the corpus set, the corpus corresponding to each of the preset intent labels, and obtain a corpus subset corresponding to each of the preset intent labels; Performing word segmentation processing on the corpus in each of the corpus subsets to obtain a first word set corresponding to each of the corpus subsets; The multiple keyword information is generated based on the second evaluation value and the third evaluation value of each word in the first word set, the second evaluation value is used to evaluate the weight of the word in the corresponding corpus, and the second evaluation value is used to evaluate the occurrence frequency of the word in the corresponding word set.
3. The method according to claim 2, characterized in that The plurality of keyword information corresponds to the plurality of first word sets one by one; for each of the first word sets, keyword information is generated based on the second evaluation value and the third evaluation value of the words in the first word set, including: Determine a second word set from the first word set based on a second evaluation value of the words in the first word set; and determine a third word set from the first word set based on a third evaluation value of the words in the first word set; generating keyword information based on the second word set and the third word set; The second word set includes a plurality of first words and a word group formed by two semantically related first words among the plurality of first words, and the second evaluation value of the first word is greater than or equal to a second threshold; The third word set includes a plurality of second words and a word group formed by two semantically related second words among the plurality of second words, and a third evaluation value of the second words is greater than or equal to a third threshold.
4. The method according to claim 2, characterized in that: A second evaluation value of a third word included in the first word set is determined based on a distance between word vectors of the third word and a fourth word in the first word set.
5. The method according to claim 1, characterized in that The method further comprises: Perform intent recognition on each piece of corpus in the conversation data to obtain the intent label corresponding to each piece of corpus; Based on the intent label corresponding to each corpus, ratio information is determined; the ratio information includes the ratio corresponding to each preset intent label, and the ratio is the ratio of the number of corpus corresponding to the preset intent label in the session data to the total amount of data of the session data; The corpus is generated based on the ratio information, wherein the number of corpora corresponding to each of the preset intent labels in the corpus matches the ratio information.
6. The method according to claim 1, characterized in that For each of the candidate keyword tables, determining a first evaluation value of the candidate keyword table includes: Input the test corpus in the preset test set into the intent recognition model for intent recognition, and obtain the predicted intent of each of the test corpus; wherein the candidate keyword table is the keyword table of the intent recognition model; the intent recognition includes: matching the test corpus with the candidate keyword table; if the test corpus matches the keyword in the candidate keyword table, then taking the intent label corresponding to the matched keyword as the predicted intent of the test corpus; each test corpus in the preset test set corresponds to a real intent; Determining the accuracy of intent recognition of the preset test set based on the predicted intent and true label of the test corpus; A first evaluation value of the candidate keyword table is determined based on the accuracy rate.
7. A data generating device, characterized in that: include: A processing module, used for processing the corpus in the corpus set to obtain a plurality of keyword information, wherein the plurality of keyword information corresponds one-to-one to a plurality of preset intent tags; A generating module, configured to generate a plurality of candidate keyword tables based on the plurality of keyword information, wherein the candidate keyword tables include the plurality of preset intent tags and keywords included in the keyword information corresponding to each of the preset intent tags; A determination module, used to determine a first evaluation value of each candidate keyword table, wherein the first evaluation value is used to evaluate the accuracy of the candidate keyword table; The determination module is further configured to determine a target keyword table from the plurality of candidate keyword tables, wherein a first evaluation value of the target keyword table is greater than or equal to a first threshold.
8. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the method steps as claimed in any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps according to any one of claims 1 to 6 are implemented.
10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the method steps as claimed in any one of claims 1 to 6.