A method and apparatus for tts problem mining and iteration

Abstract

The application provides a method and device for TTS problem mining and iteration, comprising: identifying a user query and converting text into a standardized language by combining a dialect mapping; classifying the converted query by a first model for understanding the user's intention; splicing the intention of the classified query after the previous sentence is broadcast and the next sentence spoken by the user to classify by a second model to predict the final TTS question and the corresponding score, establishing the relationship between the TTS and the user feedback by using the real feedback of the user in the voice interaction process, helping the TTS to automatically find the problem and then iteratively output the ASR result, improving the reliability of the ASR output result by using the dictionary tree method for mapping conversion; taking the output of the first model as part of the input of the second model, and integrating the information of the TTS itself in the second prediction process to improve the reliability of the final output result.

Description

Technical Field

[0001] This application relates to the field of vehicle infotainment system technology, and more specifically, to a method and apparatus for TTS problem mining and iteration. Background Technology

[0002] TTS, or Text-to-Speech, refers to the process in intelligent voice dialogue systems where the system reads aloud the generated text, essentially allowing the machine to speak to a human, thus facilitating human-computer interaction. In automotive voice systems, during human-computer interaction, TTS involves the vehicle's system first understanding the user's query and then providing a targeted response based on the current context and the query's semantics. Current TTS problem identification and iteration involve collecting user feedback and conducting data analysis, allowing the system to understand user needs and their evaluation of the TTS system. The problem classification stage categorizes user feedback into different types, helping to address issues in a targeted manner. In root cause analysis, the system locates the cause of the problem, uses analytical techniques to identify anomalies, and continuously optimizes solutions based on experimental results and user feedback. Continuous quality control and assurance ensure that the TTS system provides stable service under various conditions. However, in daily use, TTS responses often become unsuitable, such as inaccurate, incomplete, or ambiguous content, unclear or harsh voices, and inconsistent speech rates. These issues prevent the system from quickly identifying and analyzing the user's true intent, resulting in low reliability of ASR (Automatic Speech Recognition) outputs.

[0003] Therefore, this application provides a method for TTS problem mining and iteration to solve one of the aforementioned technical problems. Summary of the Invention

[0004] The purpose of this application is to provide a method and apparatus for TTS problem mining and iteration, capable of solving at least one of the aforementioned technical problems. The specific solution is as follows:

[0005] According to a specific implementation of this application, in a first aspect, this application provides a method for TTS problem mining and iteration, comprising the following steps:

[0006] S1. Identify user queries and perform text conversion using dialect mapping to convert the text into a standardized language;

[0007] S2. Classify the transformed query using the first model to understand the user's intent;

[0008] S3. The TTS of the previous sentence is concatenated with the intent of the query spoken by the user in the next sentence after classification. The second model is then used to classify the TTS and predict the final TTS question and corresponding score. This is used to understand the context of the previous sentence and combine it with the current question to make the final intent classification.

[0009] S4. Output the TTS content with scores greater than the threshold and the predicted question types.

[0010] Preferably, the text conversion using dialect mapping includes the following steps:

[0011] S11. Store the dialects of each region in a dictionary-mapped format to the specified folder;

[0012] S12. Recognize the user query as text;

[0013] S13. Based on the query obtained from speech recognition, use a trie to perform text mapping for the dialect, and convert the query into easily understandable text.

[0014] Preferably, the first model is the FastText model, and the specific method of the FastText model for intent classification is as follows:

[0015] S21. Train the fasttext model for intent classification;

[0016] S22. The model prediction results output according to the intent classification.

[0017] Preferably, the categories of intent classification include unclear TTS pronunciation, excessively fast TTS speech rate, excessively slow TTS speech rate, ambiguous TTS content, TTS system error, and unnatural TTS voice.

[0018] Preferably, the specific steps for training the fasttext model for intent classification are as follows:

[0019] S211. Collect text samples containing intent category labels;

[0020] S212. Associate the text of each sample with the corresponding intent category label;

[0021] S213. Clean up the text data and remove stop words and punctuation marks;

[0022] S214. Convert the text data into a format that FastText can accept, one line per sample;

[0023] S215, Use the preset prefix;

[0024] S216. Train the model using the prepared dataset, and then use the trained model to classify new text.

[0025] Preferably, the second model is the DistilBERT deep learning model.

[0026] Preferably, the specific method for predicting the final TTS questions and corresponding scores is as follows:

[0027] S31. Combine the TTS of the previous broadcast with the model prediction result of S22 to form a complete message.

[0028] S32. Train the DistilBERT deep learning model to predict intent;

[0029] S33. Set different score thresholds for different TTS question categories;

[0030] S34. Output the TTS content and question categories that exceed the threshold.

[0031] Preferably, the specific steps for outputting TTS scores greater than a threshold and the predicted question types are as follows:

[0032] S41. Define the types of TTS questions that need to be output;

[0033] S42. Set different score thresholds for different TTS question categories;

[0034] S43. Output the TTS content and question type that exceed the threshold.

[0035] According to a specific embodiment of this application, in a second aspect, this application provides an apparatus for TTS question mining and iteration, comprising: an ASR module, a query intent classification module, a TTS question classification module, and a question filtering and output module, wherein the ASR module is unidirectionally connected to the question intent classification module, the question intent classification module is unidirectionally connected to the TTS question classification module, and the TTS question classification module is unidirectionally connected to the question filtering and output module;

[0036] The ASR module is used for speech recognition and converting speech signals into text.

[0037] The query intent classification module is used to classify the natural language questions raised by users into predefined intent categories;

[0038] The TTS problem classification module is used to classify and process problems related to text-to-speech synthesis.

[0039] The question filtering and output module is used to filter the intents corresponding to questions and output the corresponding results.

[0040] Compared with the prior art, the above-described solution of this application has at least the following beneficial effects: By using the fasttext model and the DistilBERT deep learning model to predict the intended speech, based on the user's actual usage scenario, it makes full use of the user's real feedback during the voice interaction process, establishes the relationship between TTS and user feedback, thereby helping TTS to automatically discover problems and iterate on the ASR results. The dictionary tree method is used to perform mapping transformation, which improves the credibility of the ASR output results. The output of the first model is used as part of the input of the second model. The information of TTS itself is integrated in the second prediction process, which improves the reliability and rationality of the final output results. Attached Figure Description

[0041] Figure 1 A flowchart of a method for TTS problem mining and iteration according to an embodiment of this application is shown;

[0042] Figure 2 A block diagram of an apparatus for TTS problem mining and iteration according to an embodiment of this application is shown. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0044] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “said,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0045] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0046] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.

[0047] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0048] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0049] It should be noted that any symbols and / or numbers present in the specification that are not marked in the accompanying drawings are not reference numerals.

[0050] The optional embodiments of this application are described in detail below with reference to the accompanying drawings.

[0051] The embodiments provided in this application are embodiments of a method for TTS problem mining and iteration.

[0052] The following is combined with Figure 1 The embodiments of this application will be described in detail.

[0053] A method for TTS question mining and iteration includes steps S100, S200, S300, and S400. The method identifies user queries and performs text conversion using dialect mapping, transforming the text into a standardized language. The converted query is then used to classify intent using a first model to understand the user's intent. The intents of the previously spoken TTS and the next user-spoken query are concatenated and classified using a second model to predict the final TTS question and its corresponding score, used to understand the context of the previous sentence and combine it with the current question for final intent classification. TTS with scores greater than a threshold and the predicted question types are output. Step S100 involves storing regional dialects in a dictionary-mapped format in a designated folder; identifying user questions as text; and using a trie to map the dialect to easily understood text based on the speech-recognized query, thus identifying user queries and performing text conversion using dialect mapping to transform the text into a standardized language.

[0054] The above process specifically involves: identifying user queries and performing dialect-mapped text conversion.

[0055] Based on the user's input text, dialect recognition and mapping are performed, and the text is then converted into the standard language.

[0056] The first model performs intent classification:

[0057] Use pre-trained models such as fastText, random forest, or support vector machine (SVM) to classify the intent of the transformed user text in order to determine the user's intent.

[0058] TTS broadcast content and user question intent classification:

[0059] The text-to-speech (TTS) content of the previous sentence is combined with the user's subsequent question text to form a combined text.

[0060] Use models like DistilBERT, BERT, or DistilBERT to classify the intent of combined text.

[0061] Predicting the final TTS questions and scores:

[0062] The TTS questions predicted by the intent classification model and their corresponding scores are analyzed, and TTS questions and their predicted question types that exceed a set threshold are selected.

[0063] The specific implementation steps include: ensuring that TTS questions and predicted question types exceeding a preset threshold are selected as the final output; data preprocessing and dialect mapping conversion; model training (fastText, random forest, SVM, DistilBERT, BERT, or DistilBERT); model ensemble and combination of text intent classification; and setting a threshold for filtering and outputting classification results, specifically as follows:

[0064] Dialect recognition and text conversion:

[0065] Dialects are identified using a language recognition model, and then mapped and converted.

[0066] First model intent classification:

[0067] Use the selected fastText, random forest, or SVM model to classify the user's intent from the text.

[0068] TTS broadcast content and user question intent classification:

[0069] Combine the previous TTS broadcast content with the user's question text.

[0070] Use DistilBERT, BERT, or DistilBERT models to classify the intent of combined text.

[0071] Predicting the final TTS questions and scores:

[0072] Analyze the TTS questions predicted by the second intent classification model and their corresponding scores.

[0073] It should be noted that the first model in this invention is the FastText model, Random Forest, or Support Vector Machine, and the second model is DistilBERT, BERT, or DistilBERT. Constructing a trie involves storing regional dialects in a trie format, where each node represents a letter, and the final node contains information about the corresponding dialect. This helps the system map dialects more effectively. The user question text recognition first receives the user's input, which is a text-based question. Intent classification and entity recognition technologies are used to understand the user's question intent and extract relevant entities. If the user's input is speech, it is converted to text, which can be achieved through Automatic Speech Recognition (ASR). The trie structure is used to map the recognized text to dialects. The trie allows for the effective establishment of associations between dialects and standard text. The recognized dialect text is converted to standard text so that the system can better understand the user's intent. The results of the dialect mapping are integrated to ensure that the user's input is correctly understood, even if dialects are involved, it can be processed in standard text form. Based on the understood user intent and entities, the system generates an appropriate response, which can be text, speech, or other forms of output. The storage of the dialect trie requires collecting dialect data from various regions. This can include common vocabulary, phrases, grammatical structures, etc., for each dialect. This data can be obtained from linguistic research, dialect dictionaries, and local spoken language. A trie is constructed using the collected dialect data. A trie is a tree-like data structure where each node represents a character, and the path from the root node to a leaf node forms a word. Here, we treat the character sequence stored at each node as a dialect vocabulary. Dialect mapping of user queries: a. Speech recognition: The user asks a question, and speech is converted to text using speech recognition. b. Trie mapping: The user's text is mapped using a trie. Starting from the root node, the text of the user's question is matched character by character. If a matching path is found, it is mapped to the dialect vocabulary stored in the trie. This helps to convert the user query into easily understandable text. User query recognition and text conversion: The dialect vocabulary obtained through trie mapping is combined with other parts of the user's question text. This step can include text cleaning, vocabulary consolidation, and further text conversion operations.

[0074] Example:

[0075] In response to a user's question: "Where is the S100?"

[0076] Voice recognition: "Where is the S100?"

[0077] Trie mapping: Matching "S" -> "some" maps to "Where is some100?".

[0078] User query recognition and text conversion: Further processing of "Where is some100?" may involve removing "some" or integrating it into other more suitable text formats.

[0079] S200: Collect text samples containing intent category labels; associate the text of each sample with the corresponding intent category label; clean the text data, removing stop words and punctuation marks; convert the text data into a format acceptable to FastText, one line per sample; use a preset prefix to train the FastText model for intent classification, and output the model prediction results based on the intent classification.

[0080] It should be noted that this invention ensures coverage of various user intents that the system is expected to process by collecting text samples containing different intents from various channels; assigning a corresponding intent category label to each text sample; this is key to supervised learning and is used to train the model; removing stop words that have no practical significance for the classification task to reduce noise in model training; removing punctuation marks to maintain the consistency of the text data, wherein assigning intent category labels to text samples is a key step in supervised learning; each text sample needs to be associated with a corresponding intent category label; for example: text: "user's text content" label: "intent" Figure 1"Ensure the accuracy and consistency of labels so that the model can learn patterns from labeled data; Remove stop words: Stop words are words that appear frequently in text but usually have no practical meaning for text classification tasks; this includes common words such as "the," "is," and "and"; removing these stop words can reduce noise in model training, allowing the model to focus more on words with practical information; Remove punctuation: Punctuation marks are usually used for grammar and marking in text, but may not have practical meaning in some text classification tasks; removing punctuation marks helps maintain the consistency of text data and prevents the model from being interfered with by these marks; Example data cleaning process: Assume there is the following text sample: Text: "Hello, this is a sample sentence with some stop words." Label: "Greeting" Assign intent category label to text sample: Greeting: "Hello, this is a sample sentence with some stop words." Remove stop words: Greeting: "Hello, sample sentence stop words." Remove punctuation: Greeting: "Hello sample sentence stop..." The "words" cleanup process can be implemented using text processing libraries (such as NLTK or spaCy); specific operations may include word segmentation, part-of-speech tagging, removal of stop words and punctuation marks; associate the text of each sample with the corresponding intent category label and save it in a format acceptable to FastText; typically, each line represents a sample, including the text and its corresponding label; train a text classification model using tools provided by FastText; use the trained model to make predictions, analyze the model's prediction results, and understand the intent classification of each test sample;

[0081] Example:

[0082] Input: query

[0083] For example: I can't hear what's being said.

[0084] Output: Model prediction results

[0085] For example: TTS speaking too fast.

[0086] S300: Concatenates the TTS (Text-to-Speech) of the previous sentence with the prediction result of the fasttext model; trains the DistilBERT deep learning model to predict intent; sets different score thresholds for different TTS question categories; outputs TTS content and question categories exceeding the threshold, thus concatenating the intent after classifying the TTS of the previous sentence with the user's query in the next sentence, and using the second model for classification to predict the final TTS question and corresponding score, which is used to understand the context of the previous sentence and combine it with the current question to perform the final intent classification.

[0087] It should be noted that this invention applies the FastText model to the previously broadcast TTS to obtain the corresponding model prediction result, which includes the intent and the corresponding score. The broadcast content of the TTS is then concatenated with the prediction result of the FastText model to form a new input text. This text will be used for subsequent DistilBERT model training and prediction. DistilBERT model training is performed using these concatenated TTS and FastText prediction results as input, paired with the corresponding intent labels; this forms a training set. The DistilBERT model is trained using this training set, with the goal of classifying the intent of TTS questions. Different score thresholds are set for different TTS question categories; these thresholds can be adjusted based on model performance and application scenarios. Based on the thresholds, TTS scores in the FastText output that exceed the threshold and their corresponding question categories are selected. The selected TTS and question categories are concatenated to form a new input for the final intent classification of DistilBERT. The trained DistilBERT model is used to classify the intent of these concatenated inputs. The final TTS question and its corresponding predicted intent are output; this result may include the intent category and the corresponding confidence score.

[0088] Example:

[0089] Input: TTS + the output of the previous model.

[0090] For example: All the car windows have been opened for you + TTS content does not match reality.

[0091] Output: Model prediction results.

[0092] For example: All car windows have been opened for you. ***TTS content does not reflect reality*** 0.972.

[0093] S400: Define the TTS question types to be output; set different score thresholds for different TTS question categories; output TTS content and question types that exceed the threshold, thereby outputting TTS with scores exceeding the threshold and the predicted question types.

[0094] It should be noted that this invention first defines the types of TTS questions that the system is expected to handle, such as "check the weather," "get news," and "perform an operation." A corresponding output threshold is defined for each question type; these thresholds can be adjusted based on your application scenario, performance requirements, and the importance of different question types. A model is used for classification: a deep learning model, such as DistilBERT, is used to classify the TTS questions. For each question type, it is determined whether the model's output score is greater than the corresponding threshold. For TTS questions that exceed the threshold, their content and corresponding question type are output. The output is formatted into an easily understandable and processable form; for example, a data structure containing TTS content, question type, and model prediction score can be output. The output of eligible TTS questions and their corresponding question types can be text output, stored in a file, or integrated into other systems.

[0095] Example:

[0096] TTS question types and output threshold definitions:

[0097] Problem Type 1: Function Operation Confirmation

[0098] Output threshold: 0.8

[0099] Description: A user asks for confirmation of whether a certain function has been performed, such as: "All the windows in the car have been opened for you."

[0100] Problem Type 2: Operation Mismatch

[0101] Output threshold: 0.7

[0102] Description: Users pointed out that the TTS content did not match the actual operation, for example: "TTS says that the car window has been opened, but it is not actually done."

[0103] Problem Type 3: Information Request

[0104] Output threshold: 0.75

[0105] Description: A user requests specific information or status, such as: "Are the car windows closed?".

[0106] This application also provides apparatus embodiments that follow the above embodiments, for implementing the method steps described in the above embodiments. The interpretation of the same names is the same as that in the above embodiments, and they have the same technical effects as those in the above embodiments, so they will not be repeated here.

[0107] like Figure 2As shown, this application provides a device for TTS question mining and iteration, including: an ASR module, a question intent classification module, a TTS question classification module, and a question filtering and output module. In specific operation:

[0108] Example 1

[0109] 1. User voice input (ASR module):

[0110] The user asked, "How's the weather today?"

[0111] The ASR module converts speech to text: "How's the weather today?"

[0112] 2. Question Intent Classification (Query Intent Classification Module):

[0113] The question intent classification module categorizes text:

[0114] Type: "How's the weather today?"

[0115] Output: The intent is categorized as "check the weather".

[0116] 3. TTS Question Classification Module (Text-to-Speech Question Classification Module):

[0117] The TTS question classification module categorizes questions into types suitable for TTS:

[0118] Type: "How's the weather today?"

[0119] Output: The TTS question is categorized as "weather query".

[0120] 4. Problem Filtering and Output Module:

[0121] The problem filtering and output module filters and outputs problems according to certain rules:

[0122] Type: "How's the weather today?"

[0123] Question intent: "Check the weather"

[0124] TTS Question Type: "Weather Inquiry"

[0125] 5. Conditional judgment and output:

[0126] Conditional judgment: If the question intent is "check the weather" and the TTS question type is "weather query".

[0127] Output: Output "How's the weather today?" to the TTS engine for a voice response, and may record relevant information.

[0128] 6. Voice output (TTS module):

[0129] Based on the output of the TTS question classification module, the system can use the TTS module to convert text into speech:

[0130] Type: "How's the weather today?"

[0131] TTS output: Generate the corresponding voice: "Today's weather is sunny, with a high of 28 degrees Celsius and a low of 18 degrees Celsius."

[0132] 7. User Voice Response (ASR Module):

[0133] The user might respond: "Thanks for reminding me to bring an umbrella."

[0134] The ASR module converts the user's voice into text: "Thanks for reminding me to bring an umbrella."

[0135] Example 2

[0136] Examples based on the fastText and DistilBERT models:

[0137] 1. User Query Recognition and Dialect Mapping:

[0138] Dialect mapping: For different regional dialects, a mapping table can be used to convert user input into a standardized language. This helps to unify the form of user queries, making them easier to process and recognize.

[0139] Text conversion: Standardize the user's text, remove noise and stop words, and may also require word segmentation or stemming.

[0140] 2. FastText Intent Classification:

[0141] Training the FastText model: Using pre-labeled corpus data, train the FastText model to classify user queries into intent categories. This means the model will be able to categorize user questions into different intent types, such as queries, actions, requests, etc.

[0142] 3. Classification of the preceding sentence and the current question intent in TTS:

[0143] Previous TTS Categorization: Categorize the intent of the previous TTS content to understand the type of information conveyed in the previous TTS. This could be a question, answer, prompt, etc.

[0144] Current Question Intent Classification: The trained FastText model is used to classify the intent of the current user's question.

[0145] 4. DistilBERT Final TTS Questions and Score Prediction:

[0146] Text vectorization: The DistilBERT model is used to vectorize the previous TTS and the current question.

[0147] Concatenation and Classification: The two vectors are concatenated, and the concatenated vector is input into the pre-trained DistilBERT model for classification, predicting the final TTS question type and corresponding score.

[0148] Example implementation process:

[0149] Data preparation and preprocessing: Prepare user questions and TTS data, and preprocess them, including cleaning, standardization, and intent labeling.

[0150] FastText Intent Classifier Training: The FastText model is trained using labeled data so that it can classify user queries into different intent categories when they are received.

[0151] DistilBERT model training: Prepare and label TTS and question data, and then train the DistilBERT model to predict TTS question types and scores.

[0152] Model Integration: The above models are integrated into one process. The system receives the user's query, performs dialect mapping and standardization, then uses the FastText model to classify the intent, and uses the DistilBERT model to classify the previous TTS and the current question, and finally predicts the TTS question type and score.

[0153] Output: Output the TTS question types with scores above the threshold and their corresponding intent categories.

[0154] Example 3

[0155] 1. User Query Recognition and Dialect Mapping:

[0156] Dialect mapping: Using a dialect mapping table to convert user input into a standardized language.

[0157] Text conversion: Standardizes user text, removes noise and stop words, and performs stemming operations.

[0158] 2. Random Forest Intent Classification:

[0159] Model selection: Random Forest was chosen as the intent classifier. Random Forest is an ensemble learning algorithm suitable for tasks such as text classification.

[0160] Training the model: Using labeled corpus data, a random forest model is trained to classify intents for standardized user queries.

[0161] 3. BERT final intent classification:

[0162] Model selection: BERT (or its lightweight version such as DistilBERT) was chosen as the final intent classifier. BERT is a pre-trained deep learning model that performs well in many natural language processing tasks.

[0163] Data preparation: Prepare the labeled TTS and question data, and concatenate the previous TTS with the next user question to form the input sequence.

[0164] Fine-tuning operation: Fine-tuning the prepared data to adapt it to a specific TTS problem classification task.

[0165] 4. Implementation process:

[0166] User Query Processing: Dialect mapping and standardization of user queries.

[0167] Random Forest Intent Classification: Use the random forest model to classify the intent of standardized user queries.

[0168] TTS Question Concatenation and Classification: The previous TTS sentence and the current user question are concatenated, and the BERT model is used to classify the final intent and predict the final TTS question type and score.

[0169] Output: Output the TTS question types with scores greater than the threshold and their corresponding intent categories.

[0170] Example 4

[0171] 1. User Query Recognition and Dialect Mapping:

[0172] Dialect mapping: Using a dialect mapping table to convert user input into a standardized language.

[0173] Text conversion: Standardizes user text, removes noise and stop words, and performs stemming operations.

[0174] 2. Support Vector Machine Intent Classification:

[0175] Model selection: Support Vector Machine (SVM) was chosen as the intent classifier. SVM is a supervised learning algorithm suitable for tasks such as text classification.

[0176] Training the model: Using labeled corpus data, train an SVM model to classify intents for standardized user queries.

[0177] 3. DistilBERT final intent classification:

[0178] Model selection: DistilBERT was chosen as the final intent classifier. DistilBERT is a lightweight version of BERT, suitable for resource-constrained scenarios, and performs excellently in intent classification tasks.

[0179] Data preparation: Prepare the labeled TTS and question data, and concatenate the previous TTS with the next user question to form the input sequence.

[0180] Fine-tuning operation: Fine-tuning the prepared data to adapt it to a specific TTS problem classification task.

[0181] 4. Implementation process:

[0182] User Query Processing: Dialect mapping and standardization of user queries.

[0183] Support Vector Machine Intent Classification: Use the Support Vector Machine model to classify the intent of the standardized user query.

[0184] TTS Question Concatenation and Classification: The previous TTS sentence and the current user question are concatenated, and the DistilBERT model is used to classify the final intent and predict the final TTS question type and score.

[0185] Output: Output the TTS question types with scores greater than the threshold and their corresponding intent categories.

[0186] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for TTS question mining and iteration, characterized in that, Includes the following steps: S1. Identify user queries and perform text conversion using dialect mapping to convert the text into a standardized language; S2. The transformed query is used to classify intent using the first model to understand the user's intent. The first model is the fastText model, which includes: S21. Train the fasttext model for intent classification; S22. Model prediction results output based on intent classification; S3. The TTS (Text-to-Speech) of the previous sentence is concatenated with the intent of the user's query in the next sentence. This concatenation is then used by the second model for classification, predicting the final TTS question and corresponding score. This is used to understand the context of the previous sentence and combine it with the current question for the final intent classification. The second model is the DistilBERT deep learning model, which includes: S31. Combine the TTS of the previous broadcast with the model prediction result of S22 to form a complete message. S32. Train the DistilBERT deep learning model to predict intent; S33. Set different score thresholds for different TTS question categories; S34. Output the TTS content and question categories that exceed the threshold; S4. Output the TTS content with scores greater than the threshold and the predicted question types.

2. The method according to claim 1, characterized in that, The process of text conversion using dialect mapping includes the following steps: S11. Store the dialects of each region in a dictionary-mapped format to the specified folder; S12. Recognize the user query as text; S13. Based on the query obtained from speech recognition, use a trie to perform text mapping for the dialect, and convert the query into easily understandable text.

3. The method according to claim 1, characterized in that, The categories for intent classification include unclear TTS pronunciation, excessively fast TTS speech rate, excessively slow TTS speech rate, ambiguous TTS content, TTS system errors, and unnatural TTS voice.

4. The method according to claim 1, characterized in that, The specific steps for training the fasttext model for intent classification are as follows: S211. Collect text samples containing intent category labels; S212. Associate the text of each sample with the corresponding intent category label; S213. Clean up the text data and remove stop words and punctuation marks; S214. Convert the text data into a format that FastText can accept, one line per sample; S215, Use the preset prefix; S216. Train the model using the prepared dataset, and then use the trained model to classify new text.

5. The method according to claim 1, characterized in that, The specific method for predicting the final TTS questions and corresponding scores is as follows: S31. Combine the TTS of the previous broadcast with the model prediction result of S22 to form a complete message. S32. Train the DistilBERT deep learning model to predict intent; S33. Set different score thresholds for different TTS question categories; S34. Output the TTS content and question categories that exceed the threshold.

6. The method according to claim 1, characterized in that, The specific steps for outputting TTS scores greater than the threshold and the predicted question types are as follows: S41. Define the types of TTS questions that need to be output; S42. Set different score thresholds for different TTS question categories; S43. Output the TTS content and question type that exceed the threshold.

7. A device for TTS problem mining and iteration, characterized in that, The method according to any one of claims 1-6 includes: an ASR module, a query intent classification module, a TTS question classification module, and a question filtering and output module, wherein the ASR module is unidirectionally connected to the question intent classification module, the question intent classification module is unidirectionally connected to the TTS question classification module, and the TTS question classification module is unidirectionally connected to the question filtering and output module. The ASR module is used for speech recognition and converting speech signals into text. The query intent classification module is used to classify the natural language questions raised by users into predefined intent categories; The TTS problem classification module is used to classify and process problems related to text-to-speech synthesis. The question filtering and output module is used to filter the intents corresponding to questions and output the corresponding results.

Images