User intention recognition method and device, computer equipment and storage medium
Through format analysis, text extraction rules and semantic correction of speech text files, automatic identification of user intentions, solving the problem of manual analysis and summary time consumption in the prior art, and achieving the effect of reducing labor costs and improving production efficiency.
Patent Information
- Application Number
- CN202411036618.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-12-20
AI Technical Summary
In the prior art, staff members need to spend time to analyze and summarize after communicating with customers, resulting in increased labor costs and reduced productivity.
Through format parsing, text extraction rules and semantic correction of speech text files, user intent is automatically identified, dialogue text in a unified format is generated, and input it into the language processing model to output user intent.
It saves manual entry operation time, reduces the cost of the enterprise's human resources and improves work efficiency.
Smart Images
Figure CN119166760A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method, device, computer equipment and storage medium for identifying user intention. Background Art
[0002] In the prior art, when conducting business activities, staff usually communicate and interact with customers over the phone. Through conversations with customers, staff understand and record customers' needs and intentions. In this process, staff need to spend time analyzing and summarizing after each call, which takes up staff time and increases labor costs to a certain extent. At the same time, for conversations with high repetitiveness, manual operation efficiency is relatively low.
[0003] Therefore, in view of the shortcomings of the prior art solutions, the present invention provides a method for identifying user intention. Summary of the invention
[0004] Based on this, it is necessary to provide a user intention recognition method, device, computer equipment and storage medium to address the above technical problems.
[0005] On the one hand, a method for identifying user intention is provided, the method comprising: determining a file parsing method corresponding to a format of a speech text file according to the format, parsing speech text files of different formats according to different file parsing methods to obtain speech text in a unified format; extracting dialogue text content from each speech text node of the speech text according to a text extraction rule, converting the dialogue text content into a formatted dialogue to obtain dialogue text, wherein one speech text node includes an answer text and / or a question text; performing semantic correction on the dialogue text to obtain a corrected dialogue text; inputting the corrected dialogue text into a language processing model to output user intention.
[0006] Optionally, before obtaining the voice text in a unified format, the method further includes: converting the voice file into an initial voice text by using a voice recognition technology, wherein the initial voice text contains special characters; and performing escape processing on the special characters in the initial voice text to obtain the voice text.
[0007] Optionally, extracting the dialogue text content from each speech text node of the speech text according to the text extraction rule includes: traversing all speech text nodes of the speech text; matching each speech text node with a preset dialogue regular expression, and extracting initial dialogue text content from the speech text node; comparing the number of characters in the dialogue text content in each speech text node with the number of characters in the corresponding part of the speech text; if the difference between the number of characters in the dialogue text in a speech text node and the number of characters in the corresponding part of the speech text is less than or equal to a first preset value, using the initial dialogue text content as the dialogue text content; if the difference between the number of characters in the dialogue text in a speech text node and the number of characters in the corresponding part of the speech text is greater than the first preset value, re-cutting the speech text node into strings; and extracting the dialogue text content from the speech text node through the string cutting.
[0008] Optionally, the semantic correction of the dialogue text to obtain a corrected dialogue text includes: performing word segmentation processing on each sentence in the dialogue text through an entity sequence labeling model to obtain entity words in each sentence in the dialogue text, wherein the entity sequence labeling model is trained based on correct samples and error samples, and the entity words include wrong words to be corrected; identifying the entity words through search and recall to obtain the wrong words to be corrected in each sentence; and replacing the wrong words to be corrected in turn with replacement words in a preset replacement word library according to the order of the dialogue text to obtain the corrected dialogue text.
[0009] Optionally, the word segmentation processing for each sentence in the dialogue text includes: writing each sentence of the dialogue text into an index table; using continuous pinyin in the sentence as a key value, querying the entity word corresponding to the key value, wherein the continuous pinyin includes continuous pinyin of different lengths; and performing word segmentation processing on each sentence in the dialogue text according to the continuous pinyin of different lengths and the corresponding entity words.
[0010] Optionally, the method replaces the erroneous words to be corrected in sequence with replacement words in a preset replacement word library according to the order of the dialogue text to obtain a corrected dialogue text, including: training a binary classifier model according to features at a preset level; detecting, by the binary classifier model, the confidence of the replacement words in each sentence of the corrected dialogue text in the corrected dialogue text; if the confidence of the replacement words in the corrected dialogue text in the corrected dialogue text is greater than a second preset value, the verification of the corrected dialogue text is completed.
[0011] Optionally, the method also includes: calculating the longest common subsequence of the pinyin of the entity word and the corresponding replacement word based on a common subsequence algorithm; when the longest common subsequence meets a preset condition, determining that the replacement word is a target replacement word; matching the target replacement word with a preset optimization word library to obtain an optimized word; replacing the target replacement word with the optimized word to obtain an optimized dialogue text.
[0012] On the other hand, a user intention recognition device is provided, which includes: a parsing module, which is used to determine a file parsing method corresponding to the format of a voice text file according to the format of the file, and parse voice text files of different formats according to different file parsing methods to obtain voice text in a unified format; an extraction module, which is used to extract dialogue text content from each voice text node of the voice text according to a text extraction rule, and convert the dialogue text content into a formatted dialogue to obtain dialogue text, wherein one voice text node includes an answer text and / or a question text; a correction module, which is used to perform semantic correction on the dialogue text to obtain a corrected dialogue text; and an output module, which is used to input the corrected dialogue text into a language processing model to output the user intention.
[0013] On the other hand, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: determining a file parsing method corresponding to a format of a speech text file according to the format, parsing speech text files of different formats according to different file parsing methods to obtain speech text in a unified format; extracting dialogue text content from each speech text node of the speech text according to a text extraction rule, converting the dialogue text content into a formatted dialogue to obtain dialogue text, wherein one speech text node includes an answer text and / or a question text; performing semantic correction on the dialogue text to obtain a corrected dialogue text; and inputting the corrected dialogue text into a language processing model to output user intent.
[0014] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: according to the format of a voice text file, a file parsing method corresponding to the format is determined, and voice text files of different formats are parsed according to different file parsing methods to obtain a voice text in a unified format; according to a text extraction rule, a dialogue text content is extracted from each voice text node of the voice text, and the dialogue text content is converted into a formatted dialogue to obtain a dialogue text, wherein one voice text node includes an answer text and / or a question text; semantically correcting the dialogue text to obtain a corrected dialogue text; and inputting the corrected dialogue text into a language processing model to output user intention.
[0015] The above-mentioned user intention recognition method, device, equipment and storage medium, the method includes: according to the format of the voice text file, determining the file parsing method corresponding to the format, parsing the voice text files of different formats according to different file parsing methods, and obtaining a voice text in a unified format; according to the text extraction rule, extracting the dialogue text content from each voice text node of the voice text, converting the dialogue text content into a formatted dialogue, and obtaining a dialogue text, wherein one voice text node includes an answer text and / or a question text; semantically correcting the dialogue text to obtain a corrected dialogue text; inputting the corrected dialogue text into a language processing model to output the user intention; in this way, by using the method of intelligently generating user intention by extracting voice dialogue, the operation time of manual input into the system is saved, which to a certain extent liberates productivity and reduces the human resource cost of the enterprise. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 is a flowchart of a method for identifying user intention in one embodiment;
[0017] Figure 2 A schematic diagram of a flow chart of generating user intent in a method for identifying user intent in one embodiment;
[0018] Figure 3 A schematic diagram of an n-gram algorithm flow of a method for identifying user intent in one embodiment;
[0019] Figure 4 A schematic diagram of a dialog text correction process of a method for identifying user intent in one embodiment;
[0020] Figure 5 A schematic diagram of the state flow of a method for identifying user intent in one embodiment;
[0021] Figure 6 is a structural block diagram of a user intention recognition device in one embodiment;
[0022] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0024] It should be understood that in the description of the present invention, unless the context clearly requires otherwise, words such as “include”, “including” and the like throughout the specification should be interpreted as including rather than being exclusive or exhaustive; that is, as including but not limited to “including”.
[0025] It should also be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, "plurality" means two or more.
[0026] It should be noted that the terms "S1", "S2", etc. are only used for the purpose of describing the steps, and do not specifically refer to the order or sequence, nor are they used to limit the present invention. They are only for the convenience of describing the method of the present invention, and cannot be understood as indicating the order of the steps. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.
[0027] In one embodiment, Figure 1 As shown, a method for identifying user intention is provided, comprising the following steps:
[0028] S101: according to the format of the voice text file, determine a file parsing method corresponding to the format, and parse the voice text files of different formats according to different file parsing methods to obtain a voice text of a unified format.
[0029] Here, due to different call service providers, the formats of the obtained voice text files are different. There may be file formats such as JSON format, paragraph format, XML format, etc. Correspondingly, the JSON format file is parsed by JsonDocContextParser, the XML format file is parsed by XmlDocContextParser, and the paragraph format file is parsed by ParagraphDocContextParser.
[0030] Specifically, different formats of voice text files are parsed by file parsers of different formats, and the voice text files of different formats are parsed into a unified preset format, which can be a custom format. Exemplarily, assuming that the voice file text is in JSON format, a corresponding JSON format parsing method is adopted, a Configuration object is created using the Builder mode, JacksonMapper is used as a mapping provider, and a DocumentContext object is initialized to parse the voice file text in JSON format.
[0031] S102: extracting dialogue text content from each speech text node of the speech text according to text extraction rules, converting the dialogue text content into a formatted dialogue to obtain dialogue text, wherein one speech text node includes an answer text and / or a question text.
[0032] Here, text content can be extracted from the speech text by using string operations, regular expressions or third-party libraries; the text content in a node can be customized by the user, and can include a question text or an answer text from either party of the conversation, or a question text and an answer text from both parties of the conversation; the formatted conversation can be in a question-and-answer format including the names of the two parties of the conversation.
[0033] Specifically, it can be assumed that the extracted dialogue text content is "Hello, we have a good product to recommend to you, do you have time now? Sorry, I am busy now." The dialogue text content is formatted as: "Staff asked: Hello, we have a good product to recommend to you, do you have time now?
[0034] Customer: Sorry, I'm busy now. ".
[0035] S103: semantically correcting the dialogue text to obtain a corrected dialogue text.
[0036] Here, semantic correction may include modifying wrong words in the dialogue text into correct words, correcting grammatical errors in the dialogue text, or correcting contextual reasoning errors in the dialogue text.
[0037] Specifically, the conversation text may be input into a trained text processing model to obtain a corrected conversation text; or errors may be directly identified on the conversation text, and the identified errors may be corrected one by one.
[0038] S104: Input the corrected dialogue text into a language processing model and output the user intention.
[0039] Here, the language processing model can be a large language processing model or other models. By inputting a certain amount of dialogue texts from different scenarios into the model from time to time and informing the language processing model of the bonus factors, the language processing model can be continuously learned and improved through manual alignment and tuning.
[0040] Specifically, you can input instructions to ChatGPT through the prompt word engineering to output the customer's intention in the dialogue communication. For example, you can input the corrected dialogue text: "Staff asked: Hello, we have a good product to recommend to you, do you have time now? Customer answered: Sorry, I'm not interested", and through the prompt word engineering, the result "Customer refused, no interest at the moment" is output.
[0041] In one embodiment, Figure 2 FIG. 1 is a flow chart of generating user intent in an embodiment of the present invention, such as Figure 2 As shown, the process of generating user intent includes: the customer conducts a voice conversation, obtains a voice file through channel partners A, B, and C, converts the voice file into voice text through ASR audio-to-text technology, extracts the conversation text from the voice text, and generates a conversation summary (i.e., user intent) based on the conversation text.
[0042] It should be noted that the present invention uses an automated method to replace the staff in automatically entering the analysis and summary of customer intentions, thereby saving the staff's time cost.
[0043] In some specific implementations, before obtaining the speech text in a unified format, the method further includes:
[0044] Converting the voice file into an initial voice text by voice recognition technology, wherein the initial voice text contains special characters;
[0045] The special characters in the initial voice text are escaped to obtain the voice text.
[0046] Here, the speech recognition technology may be ASR (Automatic Speech Recognition) technology, which converts human speech into natural language text by analyzing and decoding speech signals. Special characters that do not conform to semantics in the initial speech text may be removed or replaced by escape processing.
[0047] Specifically, the ASR technology is used to convert the voice file into the initial voice text, the initial voice text is converted into a string, the string is escaped, and the processed string is converted back to the voice text. For example, the initial voice text can be read into the character buffer through the constructor file input stream FileInputStream, and then the character stream is written into a string encoded in UTF-8, and the special character escape processing is performed on the string content. The code method
[0048] StringEscapeUtils.unescapeFileStr removes or replaces the semantically incompatible characters in the initial voice text.
[0049] In this way, the initial voice text can be escaped to remove or replace special characters in the initial voice text.
[0050] In some specific implementations, extracting the dialogue text content from each speech text node of the speech text according to the text extraction rule includes:
[0051] Traversing all speech text nodes of the speech text;
[0052] Matching each speech text node with a preset dialogue regular expression, and extracting initial dialogue text content from the speech text node;
[0053] Compare the number of characters in the dialogue text content in each voice text node with the number of characters in the corresponding part of the voice text;
[0054] If the difference between the number of characters in the dialogue text in a voice text node and the number of characters in the corresponding part of the voice text is less than or equal to a first preset value, the initial dialogue text content is used as the dialogue text content;
[0055] If the difference between the number of characters in the dialogue text in a speech text node and the number of characters in the corresponding part of the speech text is greater than a first preset value, re-segmenting the speech text node;
[0056] By cutting the character string, the dialogue text content is extracted from the voice text node.
[0057] Here, the regular expressions may be regular expressions for staff and customers.
[0058] Specifically, the conversation text is extracted through regular expressions. By constructing regular expressions for staff and customers, the texts in all nodes are traversed in turn to extract the conversation text content. After the voice text of a certain node is extracted through the regular expression, the number of characters in the extracted conversation content and the voice text of the node are checked. When it is detected that the conversation content of the node extracted by the regular expression is incomplete, the voice text of the node will be re-extracted through string cutting to obtain the complete conversation content of the node.
[0059] In this way, the conversation content is extracted by combining regular expressions with string cutting, which improves the accuracy of extracting the conversation content while ensuring the speed of extracting the conversation content.
[0060] In some specific implementations, the semantic correction of the dialogue text to obtain the corrected dialogue text includes:
[0061] Perform word segmentation processing on each sentence in the dialogue text by using an entity sequence labeling model to obtain entity words in each sentence in the dialogue text, wherein the entity sequence labeling model is trained based on correct samples and error samples, and the entity words include error words to be corrected;
[0062] The entity words are identified by search and recall to obtain the wrong words to be corrected in each sentence;
[0063] According to the sequence of the dialogue text, the wrong words to be corrected are replaced in sequence by replacement words in a preset replacement word library to obtain a corrected dialogue text.
[0064] Here, when the entity sequence labeling model is trained based on correct data, the obtained entity sequence labeling model is inconsistent with the actual application distribution of the error correction scenario. There is a situation where wrong words cannot be recognized when they appear in the dialogue text. We can construct samples containing erroneous data. For example, we can use sentences with errors as training samples to make the distribution of training data and actual application scenarios consistent. We can train the entity sequence labeling model by combining correct data and erroneous data as training data.
[0065] Among them, search recall can include local word library recall and business word library recall. The local word library includes fixed-mode words, such as WeChat communication, product price, purchase method, after-sales service, etc.; the business word library includes product data, including product name, product introduction, product pricing, product marketing customer portrait, etc.
[0066] Specifically, the algorithm in the entity sequence labeling model can be used for Elastic Search retrieval, writing the conversation text into the es index table, using the pinyin name of the term as the index item, and sharding the index table according to the length of the term to obtain the entity word.
[0067] In one embodiment, Figure 3 FIG. 1 is a flow chart of a dialog text correction process in an embodiment of the present invention. Figure 3 As shown, the dialogue text correction process includes: semantic understanding of the dialogue text, obtaining entity words from the dialogue text, identifying incorrect words through search recall, which includes local vocabulary recall and business vocabulary recall, and then orderly replacing the incorrect words and optimizing the replacement words.
[0068] In this way, by training the entity sequence labeling model on data containing errors, the training data can be made closer to the actual usage scenario, thereby improving the accuracy of word segmentation through the entity sequence labeling model; the dialogue text can be corrected to obtain the corrected dialogue text, and the accuracy of identifying user intentions can be improved by improving the accuracy of the dialogue text.
[0069] In some specific implementations, the word segmentation processing of each sentence in the conversation text includes:
[0070] Writing each sentence of the dialogue text into an index table;
[0071] Using continuous pinyin in the sentence as a key value, searching for entity words corresponding to the key value, wherein the continuous pinyin includes continuous pinyin of different lengths;
[0072] According to the continuous pinyins of different lengths and the corresponding entity words, each sentence in the dialogue text is segmented.
[0073] Here, Elastic Search is used for retrieval, and pinyin n-grams of various orders are used as key values when establishing the index. Continuous pinyin of different lengths can include 1-grams, 2-grams, and 3-grams, etc. For example, 1-grams can be "jia", 2-grams can be "jia hao", and 3-grams can be "jia hao you". At the same time, when the key value is "jia", the entity words corresponding to the key value can be "add friend communication details", "add friend communication", etc.
[0074] Among them, high-order key values are queried first, and when the quantity threshold cannot be met, the query is returned to the low-order key value query to ensure the query quantity. For example, three query values are returned by default. At this time, the pinyin string is jia hao you gou tong. The query results are "add friend communication details" and "add friend communication", and the query is returned to the low-order key value query to get "add friend", and finally the query results are "add friend communication details", "add friend communication" and "add friend".
[0075] In one embodiment, Figure 4 Schematic diagram of the n-gram algorithm flow in the embodiment of the present invention, as shown in Figure 4 The n-gram algorithm process shown includes: obtaining dictionary entries, generalizing the dictionary entries and obtaining aliases, obtaining index entries, converting the index entries into pinyin strings, using 1-grams, 2-grams and 3-grams as key values, and querying the corresponding texts, specifically including 1-grams with "jia" as the index, querying to obtain text 1 "add friend communication details", text 2 "add friend communication" and text 3 "add friend", 2-grams with "jia hao" as the index, querying to obtain text 1 "add friend communication details", text 2 "add friend communication" and text 3 "add friend", and 3-grams with "jia hao you" as the index, querying to obtain text 1 "add friend communication details", text 2 "add friend communication" and text 3 "add friend".
[0076] In some specific implementations, the replacing the wrong words to be corrected in sequence according to the dialogue text to obtain the corrected dialogue text includes:
[0077] Train a binary classifier model based on the features at the preset level;
[0078] Detecting the confidence of the replacement word in each sentence of the corrected dialogue text in the corrected dialogue text by using the binary classifier model;
[0079] If the confidence of the replacement word in the corrected dialogue text in the corrected dialogue text is greater than a second preset value, the verification of the corrected dialogue text is completed.
[0080] Here, a binary classifier model is used to determine whether the value of the replacement word is the first value or the second value. The first value can indicate a correct result, and the second value can indicate an incorrect result. For example, the first value can be 1, and the second value can be 0, that is, the correct word is at position 1, and the incorrect word is at position 0.
[0081] Furthermore, the binary classifier model may be a pointwise binary classifier model, which detects the rationality of the replacement word in each sentence individually, and when the rationality of the replacement word in the sentence is lower than a preset value, the sentence is transferred to manual processing.
[0082] Specifically, the preset level may include the pinyin level, sub-level and business level, etc. The features of the preset level may include: initial consonants, finals, whole syllables, etc. at the pinyin level, edit distance, common substrings, common subsequences, etc. at the character level, product definition information, marketing methods, etc. at the business level.
[0083] In this way, the rationality of the replacement word in the sentence can be verified through the binary classifier model.
[0084] In some embodiments, the method further comprises:
[0085] Based on the common subsequence algorithm, the longest common subsequence of the pinyin of the entity word and the corresponding replacement word is calculated;
[0086] When the longest common subsequence meets a preset condition, determining the replacement word as a target replacement word;
[0087] Matching the target replacement word with a preset optimization word library to obtain an optimized word;
[0088] The target replacement word is replaced by the optimization word to obtain an optimized dialogue text.
[0089] Here, assuming that the pinyin sequence of the entity word is Oa, and the pinyin sequence of the replacement word is T, a two-dimensional array at is defined, where at[i][j] represents the length of the longest common subsequence of the first i pinyins of Oa and the first j pinyins of T, at[i][0] and at[0][j] are initialized to 0, and each pinyin of the pinyin sequence Oa and T is traversed. If Oa[i] and T[j] are the same, then at[i][j]=at[i-1][j-1]+1; otherwise at[i][j]=max(at[i-1][j],at[i][j-1]), which means the maximum length of the subsequences of different pinyin sequences. Finally, at[n][m] is the length of the longest common subsequence of Oa and T. For example, assuming Oa:["jia","v","gou","tong"]T:["jia","wei","gou","tong"], at[4][4]=3.
[0090] Furthermore, the preset condition may be that when the length of the longest common subsequence is different from the length of the pinyin sequence of the replacement word, the replacement word is optimized.
[0091] The preset optimized word library may include a business word library, a local word library, etc. The optimized word library may be updated by users adding words from time to time.
[0092] Specifically, when the user inputs voice input, what he inputs is "add micro communication", and the replacement word is "add v communication". At this time, "add v communication" is matched with the optimization vocabulary, and "add WeChat communication" is matched from the optimization vocabulary, and "add v communication" is optimized to "add WeChat communication" for echo.
[0093] In this way, by optimizing the echo, the accuracy of the conversation text is further improved, and the accuracy of identifying user intent is improved compared to the method of directly restricting the pinyin of the original string and the replacement string to correspond one to one.
[0094] In one embodiment, Figure 5 Schematic diagram of the state flow in an embodiment of the present invention. Figure 5 As shown, the system initializes, extracts the dialogue text, and accepts the dialogue text. If the acceptance is successful, the speech processing model is called and waits for output. If the output is normal, the generation is successful. If the output fails, the generation fails, the system retries and returns to system initialization. If the acceptance fails, manual intervention is performed and the system returns to system initialization.
[0095] It should be understood that although Figure 1-Figure 5 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-Figure 5 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.
[0096] In one embodiment, Figure 6As shown, a user intention recognition device is provided, and the device includes: a parsing module 601, an extraction module 602, a correction module 603 and an output module 604; wherein the parsing module 601 is used to determine a file parsing method corresponding to the format of a voice text file according to the format of the file, and parse voice text files of different formats according to different file parsing methods to obtain a voice text in a unified format; the extraction module 602 is used to extract the dialogue text content from each voice text node of the voice text according to a text extraction rule, and convert the dialogue text content into a formatted dialogue to obtain a dialogue text, wherein one voice text node includes an answer text and / or a question text; the correction module 603 is used to perform semantic correction on the dialogue text to obtain a corrected dialogue text; the output module 604 is used to input the corrected dialogue text into a language processing model to output the user intention.
[0097] As a preferred implementation manner, in an embodiment of the present invention, the device also includes: an escape module, which is specifically used to: convert the voice file into an initial voice text through voice recognition technology, wherein the initial voice text contains special characters; and escape the special characters in the initial voice text to obtain the voice text.
[0098] As a preferred implementation, in the embodiment of the present invention, the extraction module 602 is specifically used to: traverse all speech text nodes of the speech text; match each speech text node with a preset dialogue regular expression, and extract the initial dialogue text content from the speech text node; compare the number of characters in the dialogue text content in each speech text node with the number of characters in the corresponding part of the speech text; if the difference between the number of characters in the dialogue text in a speech text node and the number of characters in the corresponding part of the speech text is less than or equal to a first preset value, use the initial dialogue text content as the dialogue text content; if the difference between the number of characters in the dialogue text in a speech text node and the number of characters in the corresponding part of the speech text is greater than the first preset value, re-cut the speech text node into strings; and extract the dialogue text content from the speech text node through the string cutting.
[0099] As a preferred implementation manner, in an embodiment of the present invention, the correction module 603 is specifically used for: performing word segmentation processing on each sentence in the dialogue text through an entity sequence labeling model to obtain entity words in each sentence in the dialogue text, wherein the entity sequence labeling model is trained based on correct samples and error samples, and the entity words include wrong words to be corrected; identifying the entity words through search and recall to obtain the wrong words to be corrected in each sentence; according to the order of the dialogue text, replacing the wrong words to be corrected in turn with replacement words in a preset replacement word library to obtain a corrected dialogue text.
[0100] As a preferred implementation manner, in an embodiment of the present invention, the correction module 603 is specifically used to: write each sentence of the dialogue text into an index table; use the continuous pinyin in the sentence as a key value to query the entity word corresponding to the key value, wherein the continuous pinyin includes continuous pinyin of different lengths; and perform word segmentation processing on each sentence in the dialogue text according to the continuous pinyin of different lengths and the corresponding entity words.
[0101] As a preferred implementation, in an embodiment of the present invention, the correction module 603 is specifically used to: train a binary classifier model based on features at a preset level; detect the confidence of the replacement word in each sentence of the corrected dialogue text in the corrected dialogue text through the binary classifier model; if the confidence of the replacement word in the corrected dialogue text in the corrected dialogue text is greater than a second preset value, the verification of the corrected dialogue text is completed.
[0102] As a preferred implementation manner, in an embodiment of the present invention, the device also includes an optimization module, which is specifically used to calculate the longest common subsequence of the pinyin of the entity word and the corresponding replacement word based on the common subsequence algorithm; when the longest common subsequence meets the preset conditions, determine that the replacement word is the target replacement word; match the target replacement word with a preset optimization word library to obtain an optimized word; replace the target replacement word with the optimized word to obtain an optimized dialogue text.
[0103] For the specific definition of the user intention recognition device, please refer to the definition of the user intention recognition method above, which will not be repeated here. Each module in the above-mentioned user intention recognition device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0104] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a user intention recognition method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.
[0105] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0106] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: S1: determining a file parsing method corresponding to a format of a speech text file according to the format, parsing speech text files of different formats according to different file parsing methods, and obtaining speech texts of a unified format; S2: extracting a dialogue text content from each speech text node of the speech text according to a text extraction rule, converting the dialogue text content into a formatted dialogue, and obtaining a dialogue text, wherein one speech text node includes an answer text and / or a question text;
[0107] S3: semantically correcting the dialogue text to obtain a corrected dialogue text; S4: inputting the corrected dialogue text into a language processing model to output user intention.
[0108] In one embodiment, when the processor executes the computer program, the following steps are also implemented: converting the voice file into an initial voice text through voice recognition technology, wherein the initial voice text contains special characters; and performing escape processing on the special characters in the initial voice text to obtain the voice text.
[0109] In one embodiment, when the processor executes the computer program, the following steps are also implemented: traversing all speech text nodes of the speech text; matching each speech text node with a preset dialogue regular expression, and extracting the initial dialogue text content from the speech text node; comparing the number of characters in the dialogue text content in each speech text node with the number of characters in the corresponding part of the speech text; if the difference between the number of characters in the dialogue text in a speech text node and the number of characters in the corresponding part of the speech text is less than or equal to a first preset value, taking the initial dialogue text content as the dialogue text content; if the difference between the number of characters in the dialogue text in a speech text node and the number of characters in the corresponding part of the speech text is greater than the first preset value, re-cutting the speech text node into strings; and extracting the dialogue text content from the speech text node through the string cutting.
[0110] In one embodiment, when the processor executes the computer program, the following steps are also implemented: each sentence in the dialogue text is segmented by an entity sequence labeling model to obtain entity words in each sentence in the dialogue text, wherein the entity sequence labeling model is trained based on correct samples and error samples, and the entity words include wrong words to be corrected; the entity words are identified by search and recall to obtain the wrong words to be corrected in each sentence; and according to the order of the dialogue text, the wrong words to be corrected are replaced in turn by replacement words in a preset replacement word library to obtain a corrected dialogue text.
[0111] In one embodiment, when the processor executes the computer program, the following steps are also implemented: writing each sentence of the dialogue text into an index table; using continuous pinyin in the sentence as a key value to query the entity word corresponding to the key value, wherein the continuous pinyin includes continuous pinyin of different lengths; and performing word segmentation processing on each sentence in the dialogue text according to the continuous pinyin of different lengths and the corresponding entity words.
[0112] In one embodiment, when the processor executes the computer program, the following steps are also implemented: training a binary classifier model based on features at a preset level; detecting, through the binary classifier model, the confidence of the replacement word in each sentence of the corrected dialogue text in the corrected dialogue text; if the confidence of the replacement word in the corrected dialogue text in the corrected dialogue text is greater than a second preset value, the verification of the corrected dialogue text is completed.
[0113] In one embodiment, when the processor executes the computer program, the following steps are also implemented: based on the common subsequence algorithm, the longest common subsequence of the pinyin of the entity word and the corresponding replacement word is calculated; when the longest common subsequence meets the preset conditions, the replacement word is determined to be the target replacement word; the target replacement word is matched with a preset optimization word library to obtain an optimized word; and the target replacement word is replaced by the optimized word to obtain an optimized dialogue text.
[0114] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: S1: according to the format of a voice text file, a file parsing method corresponding to the format is determined, and voice text files of different formats are parsed according to different file parsing methods to obtain a voice text in a unified format; S2: according to a text extraction rule, a dialogue text content is extracted from each voice text node of the voice text, and the dialogue text content is converted into a formatted dialogue to obtain a dialogue text, wherein one voice text node includes an answer text and / or a question text; S3: semantically correcting the dialogue text to obtain a corrected dialogue text; S4: inputting the corrected dialogue text into a language processing model to output user intention.
[0115] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: converting the voice file into an initial voice text through voice recognition technology, wherein the initial voice text contains special characters; performing escape processing on the special characters in the initial voice text to obtain the voice text.
[0116] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: traversing all speech text nodes of the speech text; matching each speech text node with a preset dialogue regular expression, and extracting the initial dialogue text content from the speech text node; comparing the number of characters in the dialogue text content in each speech text node with the number of characters in the corresponding part of the speech text; if the difference between the number of characters in the dialogue text in a speech text node and the number of characters in the corresponding part of the speech text is less than or equal to a first preset value, taking the initial dialogue text content as the dialogue text content; if the difference between the number of characters in the dialogue text in a speech text node and the number of characters in the corresponding part of the speech text is greater than the first preset value, re-cutting the speech text node into strings; and extracting the dialogue text content from the speech text node through the string cutting.
[0117] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: each sentence in the dialogue text is segmented by an entity sequence labeling model to obtain entity words in each sentence in the dialogue text, wherein the entity sequence labeling model is trained based on correct samples and error samples, and the entity words include wrong words to be corrected; the entity words are identified by search and recall to obtain the wrong words to be corrected in each sentence; and according to the order of the dialogue text, the wrong words to be corrected are replaced in turn by replacement words in a preset replacement word library to obtain a corrected dialogue text.
[0118] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: writing each sentence of the dialogue text into an index table; using the continuous pinyin in the sentence as a key value to query the entity word corresponding to the key value, wherein the continuous pinyin includes continuous pinyin of different lengths; and performing word segmentation processing on each sentence in the dialogue text according to the continuous pinyin of different lengths and the corresponding entity words.
[0119] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: training a binary classifier model based on features at a preset level; detecting the confidence of the replacement word in each sentence of the corrected dialogue text in the corrected dialogue text through the binary classifier model; if the confidence of the replacement word in the corrected dialogue text in the corrected dialogue text is greater than a second preset value, the verification of the corrected dialogue text is completed.
[0120] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the common subsequence algorithm, the longest common subsequence of the pinyin of the entity word and the corresponding replacement word is calculated; when the longest common subsequence meets the preset conditions, the replacement word is determined to be the target replacement word; the target replacement word is matched with a preset optimization word library to obtain an optimized word; and the target replacement word is replaced by the optimized word to obtain an optimized dialogue text.
[0121] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0122] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0123] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.
Claims
1. A method for identifying user intention, characterized in that: The method comprises: Determine a file parsing method corresponding to the format of the voice text file according to the format, and parse the voice text files of different formats according to different file parsing methods to obtain a voice text of a unified format; Extracting the dialogue text content from each speech text node of the speech text according to the text extraction rule, converting the dialogue text content into a formatted dialogue to obtain a dialogue text, wherein one speech text node includes an answer text and / or a question text; Performing semantic correction on the dialogue text to obtain a corrected dialogue text; The corrected dialogue text is input into a language processing model to output the user intention.
2. The method for identifying user intention according to claim 1, characterized in that: Before obtaining the speech text in a unified format, the method further includes: Converting the voice file into an initial voice text by voice recognition technology, wherein the initial voice text contains special characters; The special characters in the initial voice text are escaped to obtain the voice text.
3. The method for identifying user intention according to claim 1, characterized in that: The extracting of the dialogue text content from each speech text node of the speech text according to the text extraction rule comprises: Traversing all speech text nodes of the speech text; Matching each speech text node with a preset dialogue regular expression, and extracting initial dialogue text content from the speech text node; Compare the number of characters in the dialogue text content in each voice text node with the number of characters in the corresponding part of the voice text; If the difference between the number of characters in the dialogue text in a voice text node and the number of characters in the corresponding part of the voice text is less than or equal to a first preset value, the initial dialogue text content is used as the dialogue text content; If the difference between the number of characters in the dialogue text in a speech text node and the number of characters in the corresponding part of the speech text is greater than a first preset value, re-segmenting the speech text node; By cutting the character string, the dialogue text content is extracted from the voice text node.
4. The method for identifying user intention according to claim 1, characterized in that: The semantic correction of the dialogue text to obtain the corrected dialogue text includes: Perform word segmentation processing on each sentence in the dialogue text by using an entity sequence labeling model to obtain entity words in each sentence in the dialogue text, wherein the entity sequence labeling model is trained based on correct samples and error samples, and the entity words include error words to be corrected; The entity words are identified by search and recall to obtain the wrong words to be corrected in each sentence; According to the sequence of the dialogue text, the wrong words to be corrected are replaced in sequence by replacement words in a preset replacement word library to obtain a corrected dialogue text.
5. The method for identifying user intention according to claim 4, characterized in that: The word segmentation processing of each sentence in the dialogue text includes: Writing each sentence of the dialogue text into an index table; Using continuous pinyin in the sentence as a key value, searching for entity words corresponding to the key value, wherein the continuous pinyin includes continuous pinyin of different lengths; According to the continuous pinyins of different lengths and the corresponding entity words, each sentence in the dialogue text is segmented.
6. The method for identifying user intention according to claim 4, characterized in that: The step of replacing the wrong words to be corrected in sequence with replacement words in a preset replacement word library according to the order of the dialogue text to obtain a corrected dialogue text includes: Train a binary classifier model based on the features at the preset level; Detecting the confidence of the replacement word in each sentence of the corrected dialogue text in the corrected dialogue text by using the binary classifier model; If the confidence of the replacement word in the corrected dialogue text in the corrected dialogue text is greater than a second preset value, the verification of the corrected dialogue text is completed.
7. The method for identifying user intention according to claim 6, characterized in that: The method further comprises: Based on the common subsequence algorithm, the longest common subsequence of the pinyin of the entity word and the corresponding replacement word is calculated; When the longest common subsequence meets a preset condition, determining the replacement word as a target replacement word; Matching the target replacement word with a preset optimization word library to obtain an optimized word; The target replacement word is replaced by the optimization word to obtain an optimized dialogue text.
8. A user intention recognition device, characterized in that: The device comprises: a parsing module, an extraction module, a correction module and an output module; wherein, The parsing module is used to determine a file parsing method corresponding to the format of the voice text file according to the format of the voice text file, and parse the voice text files of different formats according to different file parsing methods to obtain a voice text of a unified format; The extraction module is used to extract the dialogue text content from each speech text node of the speech text according to the text extraction rule, convert the dialogue text content into a formatted dialogue, and obtain the dialogue text, wherein one speech text node includes an answer text and / or a question text; The correction module is used to perform semantic correction on the dialogue text to obtain a corrected dialogue text; The output module is used to input the corrected dialogue text into a language processing model and output the user intention.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.