AI-Driven Personalized Speech Training and Pronunciation Correction System

Abstract

The present invention provides an AI-driven personalized speech training and pronunciation correction system. The system first collects the original speech data of the user during the speech training process and extracts multi-dimensional speech feature vectors including pitch, speech rate, intonation, formant parameters, and corresponding texts. Based on the feature vectors, it identifies the context labels of the current speech and constructs the user's pronunciation feature vectors. The system further calls the standard pronunciation database according to the context labels to generate the target pronunciation feature vectors in the corresponding context, and makes a multi-dimensional comparison between the user's pronunciation and the target pronunciation to obtain a set of difference parameters including pronunciation parts, speech rate, and emotional differences. Finally, the system generates correction suggestions including pronunciation action guidance, intonation adjustment tips, and semantic emotion strengthening instructions, and receives user feedback information to improve the personalized training effect. The present invention can achieve precise pronunciation correction under context awareness and enhance the personalization and intelligence level of speech training.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly to an AI-driven personalized speech training and pronunciation correction system. Background Art

[0002] In the prior art, speech training and pronunciation correction systems usually perform basic comparison and scoring on user speech based on preset acoustic models or static pronunciation templates. Such systems mostly use the calculation of the distance between audio features and target pronunciations as the core evaluation method. Some systems can provide feedback on indicators such as pitch and speech rate, but generally focus on the evaluation and correction of static indicators, lacking dynamic interaction and in-depth understanding capabilities.

[0003] The prior art has obvious deficiencies in dealing with context relevance, emotional expression accuracy, and individual differences. Specifically, traditional systems cannot distinguish pronunciation targets according to the dialogue context, ignore the role of intonation and emotion in speech expression, and cannot dynamically adjust the evaluation criteria based on the user's historical performance, resulting in overly mechanical and un-targeted training feedback, and it is difficult to meet the high-quality speech expression training requirements in complex application scenarios.

[0004] In view of the above problems, there is an urgent need for a new speech training and pronunciation correction solution that can integrate context understanding, emotion perception, and personalized modeling. Summary of the Invention

[0005] The present application provides an AI-driven personalized speech training and pronunciation correction system to achieve accurate pronunciation correction under context awareness.

[0006] The present application provides an AI-driven personalized speech training and pronunciation correction system, including:

[0007] A voice collection unit, configured to collect the original voice data of the user during the speech training process, and extract a multi-dimensional voice feature vector including pitch, speech rate, intonation, formant parameters, and the corresponding text of the speech from the original voice data;

[0008] An identification and construction unit, configured to identify the context label of the user's current speech based on the semantic information and emotional features in the multi-dimensional voice feature vector, and construct a corresponding user pronunciation feature vector based on the multi-dimensional voice feature vector;

[0009] A feature generation unit, configured to call a target pronunciation feature vector corresponding to the context label from a preset standard pronunciation database based on the context label;

[0010] A difference analysis unit is configured to perform multi-dimensional comparison between the user's pronunciation feature vector and the target pronunciation feature vector to obtain a set of difference parameters including pronunciation position deviation, speech rate difference, and emotional expression difference.

[0011] An error correction feedback unit is configured to generate pronunciation correction suggestions based on the set of difference parameters. The pronunciation correction suggestions include pronunciation action guidelines, intonation adjustment tips, and semantic emotion strengthening instructions, and is also configured to output the pronunciation correction suggestions to the user and receive the practice feedback information from the user based on the pronunciation correction suggestions.

[0012] The beneficial effects of this application mainly include: (1) By extracting multi-dimensional speech feature vectors including pitch, speech rate, intonation, formant parameters, and speech text, the system can achieve a comprehensive and refined modeling of the user's pronunciation features, thereby improving the accuracy and coverage of speech analysis. (2) The system introduces a context label recognition mechanism, which can dynamically call corresponding target pronunciation standards according to different speech scenarios (such as dialogue, speech, reading, etc.), realizing context-related personalized training and enhancing the adaptability and practicality of training content. (3) By performing multi-dimensional difference comparison between the user's pronunciation features and context-related target pronunciation features, the system can identify deviations in pronunciation position, speech rate, and emotional expression, etc., providing a highly targeted quantitative basis for subsequent correction and significantly improving the error correction accuracy. (4) The system constructs an interactive closed-loop training mechanism based on the user's feedback information, making the pronunciation correction suggestions not only have real-time and context awareness capabilities, but also continuously optimize the training effect during the user's continuous practice, improving the efficiency and intelligent level of speech learning. Description of the Drawings

[0013] Figure 1 is a schematic diagram of an AI-driven personalized speech training and pronunciation correction system provided by the first embodiment of this application. Detailed Description of the Embodiment

[0014] Many specific details are set forth in the following description in order to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of this application. Therefore, this application is not limited by the specific implementations disclosed below.

[0015] The first embodiment of this application provides an AI-driven personalized speech training and pronunciation correction system. Please refer to Figure 1 , which is a schematic diagram of the first embodiment of this application. The following will be combined with Figure 1 to provide a detailed description of an AI-driven personalized speech training and pronunciation correction system according to the first embodiment of this application.

[0016] The system includes a voice acquisition unit 101, an identification and construction unit 102, a feature generation unit 103, a difference analysis unit 104, and an error correction and feedback unit 105.

[0017] The voice acquisition unit 101 is used to acquire the original voice data of the user during the voice training process, and extract a multi-dimensional voice feature vector including pitch, speech rate, intonation, formant parameters, and the text corresponding to the speech from the original voice data.

[0018] In the AI-driven personalized voice training and pronunciation correction system of this embodiment, the voice acquisition unit 101 is the front-end input component of the system. Its function is to obtain the original voice data of the user during the voice training process and extract a multi-dimensional voice feature vector from this voice data. This unit can include one or more audio acquisition devices, such as an electret microphone array, a MEMS microphone, or a sound pickup module integrated in a mobile device, a computer, or an ear-worn device. The acquisition frequency is preferably 16 kHz or above to ensure that the resolution of the audio data is high enough for subsequent feature extraction.

[0019] The acquired original voice signal is first subjected to signal noise reduction and reverberation removal processing by a preprocessing module. Front-end enhancement algorithms based on spectral subtraction, Wiener filtering, or deep neural networks can be used to remove the interference of environmental background noise and reverberation on the voice training accuracy. The processed voice signal is sent into the acoustic feature extraction process.

[0020] The feature extraction module uses time-domain and frequency-domain analysis methods to extract a multi-dimensional voice feature vector. The multi-dimensional voice features at least include the following: pitch, which can be estimated by the autocorrelation method or a deep learning method based on a masked spectrogram; speech rate, which is obtained by statistically analyzing the time distribution of phonemes, syllables, or words; intonation, that is, the pitch change trend of the speech in a whole sentence, which can be modeled by an average pitch curve; formant parameters, usually the formant frequencies F1, F2, F3, etc. are extracted through LPC (Linear Predictive Coding) analysis; in addition, the system should also extract the text information corresponding to the speech, usually using an ASR (Automatic Speech Recognition) model to identify the text sequence corresponding to the original voice. A preferred ASR model can be based on the CTC structure, the Transformer-Encoder, or the Conformer architecture to achieve higher recognition accuracy and adaptability to accents.

[0021] For the convenience of subsequent module calls, all the above features will be uniformly encoded into a structured multi-dimensional vector, which serves as the output result of the voice acquisition unit 101. This multi-dimensional voice feature vector not only retains the original acoustic information but also incorporates temporal features and semantic tags, having good scalability and system interface compatibility. This vector will be directly passed to the recognition and construction unit 102 for subsequent context analysis and pronunciation feature modeling.

[0022] It is worth noting that the voice acquisition unit 101 can also cooperate with the wake-up mechanism and permission control mechanism of the user device to achieve automatic data collection and upload during the multi-round training process while ensuring user privacy. In addition, to improve the personalized adaptation ability, the voice acquisition unit 101 can also include a user ID binding mechanism to ensure that each training data can be associated with historical data, facilitating the system to accumulate individual pronunciation profiles and supporting subsequent dynamic optimization and long-term tracking and correction.

[0023] To sum up, the voice acquisition unit 101 in this system not only completes high-quality audio input and noise reduction preprocessing but also constructs a high-dimensional voice feature representation required for subsequent recognition, analysis, and training control.

[0024] The recognition and construction unit 102 is used to identify the context label of the user's current speech based on the semantic information and emotional features in the multi-dimensional voice feature vector, and construct a corresponding user pronunciation feature vector based on the multi-dimensional voice feature vector.

[0025] In the AI-driven personalized voice training and pronunciation correction system of this embodiment, the recognition and construction unit 102 is used to process the multi-dimensional voice feature vector output by the voice acquisition unit 101, and its functions mainly include two aspects: one is to identify the context label of the current speech, and the other is to construct the user's pronunciation feature vector based on this feature vector.

[0026] First of all, the recognition and construction unit receives the above-mentioned multi-dimensional voice feature vector including pitch, speech rate, intonation, formant parameters, and the text corresponding to the speech. In order to obtain semantic and emotional information from these features, the recognition and construction unit internally includes a semantic understanding sub-module and an emotion analysis sub-module. The semantic understanding sub-module is used to analyze the text content and language structure of the user's spoken sentence, and preferably adopts a natural language processing model, such as a context-aware model based on BERT, RoBERTa, or Transformer encoder, to encode the sentence meaning. Combining the text information corresponding to the speech, the system can identify the functional type of the current sentence, such as declarative sentence, interrogative sentence, exclamatory sentence, or distinguish the scene type, such as daily conversation, news broadcast, voice command, etc.

[0027] The sentiment analysis sub-module uses signal features such as intonation, rhythm, and formant patterns, which are highly correlated with acoustic features, and combines deep neural network models (such as multi-layer LSTM or CNN-BiGRU structures) to determine the emotional tendency expressed by the current speech, such as neutral, positive, anxious, angry, etc. Based on the dual recognition of semantic labels and emotion labels, the system fuses the results and generates a clear context label through a context discrimination network. This label is selected from a preset context set, such as "children's conversation", "news broadcast", "customer service scenario", "role-playing", etc. Each context label corresponds to a set of subsequent pronunciation targets and evaluation benchmarks.

[0028] After completing context recognition, the recognition and construction unit further aggregates the original multi-dimensional speech feature vectors to construct the user's pronunciation feature vector. This vector is used to characterize the overall speech expression state of the user in the current context, including the pronunciation part, speech rate pattern, pitch change of the actual pronunciation, and the emotional consistency between the pronunciation and the content spoken. Preferably, the user pronunciation feature vector adopts a vector structure with a fixed dimension, and the attention mechanism can be used to assign weights to different feature dimensions to enhance the expression ability of features sensitive to pronunciation differences. In addition, the construction process can also call the user's pronunciation habits or known problems in the historical training records to achieve personalized adjustment of the user model, so that the constructed pronunciation feature vector can reflect the comprehensive expression of the user's current state and past features.

[0029] Finally, the output result of the recognition and construction unit 102 includes two parts: one is the context label, which is used to indicate the expression context where the user's current speech is located; the other is the user pronunciation feature vector, which is used for subsequent difference analysis with the target pronunciation features. These two outputs will be transmitted to the feature generation unit 103 and the difference analysis unit 104 respectively, as the basis for generating subsequent correction suggestions.

[0030] Furthermore, the recognition and construction unit includes a context transformation simulation module. Specifically, the context transformation simulation module is used to analyze the pronunciation feature continuity between different context labels in the user's historical training data, extract the transitional expression patterns between adjacent context categories, and establish a multi-context mixed expression model based on the transitional expression patterns;

[0031] When the recognition and construction unit detects that the current multi-dimensional speech feature vector contains feature indicators corresponding to two or more context labels at the same time, and the feature indicators are highly correlated in the historical data, the context transformation simulation module calls the cross-context interference decoder to decouple and separate the different context components in the multi-dimensional speech feature vector, generate the speech feature sub-vectors corresponding to each context, and calculate the context conflict confidence score for each sub-vector based on the separation result;

[0032] In the case where the confidence score of any context conflict exceeds a preset threshold, the context transformation simulation module triggers a feature masking mechanism to partially mask the context feature sub-vectors with confidence lower than the threshold, and calculates the fusion weights for the retained parts;

[0033] The fusion weights are used to perform weighted combination on the target pronunciation feature vectors corresponding to multiple context labels in a preset standard pronunciation database to generate a dynamic target pronunciation feature vector containing interpretable weight vector annotations. The dynamic target pronunciation feature vector has transitional expression features and is used to compare with the current user pronunciation feature vector to cope with the non-linear expression deviation caused by the rapid context switching during the natural speech training of the user.

[0034] In an AI-driven personalized speech training and pronunciation correction system, the recognition and construction unit includes a context transformation simulation module, which is used to recognize and process the multi-context mixing or context transition expression phenomena that may occur during the natural speech training of the user. The system first analyzes the user's historical speech samples to establish a continuous mapping relationship of pronunciation features between different context labels. Context labels are pre-defined expression scene classifications of the system, such as "news broadcast", "emotional reading", "customer service dialogue", etc. Each context label corresponds to a set of known standard pronunciation feature sequences. These context labels are automatically recognized by the model or can be provided with an initial training set by manually annotating training samples.

[0035] To construct the transitional expression pattern, the system extracts key transition features including the pitch curve change rate, speech rate fluctuation range, emotional parameter mutation points, etc. based on the feature change sequences shown by the user during the natural transition between two adjacent contexts in the historical samples. In the processing flow, the system uses a sliding window mechanism to cut out speech segments with cross-context features from the user's speech data. Subsequently, the dynamic time warping (DTW) algorithm is used to align these segments, and a feature clustering method based on K-means is used to classify similar transitional samples into one category. Each category represents a common context transition path, such as the mode of gradually transitioning from "formal broadcast" to "emotional reading". After clustering, a bidirectional long short-term memory network (BiLSTM) is used to model these time series features, and a transitional expression prediction model is trained to output the predicted "transition state" feature vector when detecting signs of context mixing, so as to be used for subsequent dynamic target generation.

[0036] When the recognition building block receives the multi-dimensional speech feature vector in the current user speech, the context transformation simulation module first calls the existing context recognition model (such as the joint encoding network based on BERT and BiGRU) to analyze the potential context label distribution in the current sample. If it is detected that the key features corresponding to two or more context labels appear simultaneously, and they have a high-frequency co-occurrence relationship in the historical data (such as appearing simultaneously in more than 30% of the transition training samples), the system judges that there may be a context mixing or context jump situation.

[0037] At this time, the context transformation simulation module calls the cross-context interference decoder. This decoder adopts a multi-head attention mechanism structure to decouple the input multi-dimensional speech feature vector in the context dimension, decomposing it into several speech feature sub-vectors, and each sub-vector corresponds to a candidate context component. For example, one sub-vector may mainly reflect the speech rate rhythm of "news broadcast", and another may reflect the pitch jitter and emotional expression of "emotional reading".

[0038] To judge the effectiveness of each context component in the current expression, the system calculates the context conflict confidence score for each speech feature sub-vector respectively. This score comprehensively considers the following three factors: one is the feature similarity between the sub-vector and its standard context sample, and the cosine similarity is preferably used for calculation; the second is the proportion of the sub-vector in the total energy of the current speech, which is used to measure its expression intensity; the third is the matching degree between the semantics expressed by the current speech text and the context label, which is evaluated by sentence vector comparison (such as the vector after BERT encoding). After these three indicators are normalized, the confidence score is calculated by weighted average. The setting of the weights can be based on data statistical experience. For example, the semantic consistency is set to 0.4, the acoustic similarity is set to 0.4, and the energy ratio is set to 0.2, or it can also be trained by a supervised learning method.

[0039] Once the confidence score of a sub-vector is lower than the threshold set by the system (such as 0.55), then this sub-vector will be regarded as an invalid or interfering context component, and the system triggers the feature masking mechanism to down-weight this sub-vector, or directly masks its participation in the subsequent fusion calculation with a zero-value mask. For the remaining high-confidence sub-vectors, the system re-normalizes their confidence scores using the Softmax function to generate the fusion weights.

[0040] The fusion weights are used to retrieve the target pronunciation feature vectors corresponding to multiple context labels from the standard pronunciation database, and these target vectors are weighted and combined according to the weights, and finally a dynamic target pronunciation feature vector is formed. This vector not only expresses the features of the dominant context, but also integrates the pronunciation styles of the retained components in the secondary context, and attaches weight information to the structure for subsequent comparison or visualization.

[0041] The dynamic target pronunciation feature vector has the ability of transitional expression, and is especially suitable for complex situations such as rapid context switching and cross-scenario emotional fluctuations in the user expression in natural language communication scenarios, which helps to improve the accuracy and adaptability of the speech training system under diverse expressions. After the dynamic target pronunciation feature vector is generated, it will be input into the difference analysis unit in the system as a comparison benchmark for multi-dimensional comparison with the pronunciation feature vector of the current user. The system generates a set of difference parameters by quantitatively analyzing the differences between these two vectors in dimensions such as pitch, speech rate, intonation, and emotional expression. This set of difference parameters will be used in the subsequent error correction feedback unit to help the system give pronunciation correction suggestions that are more context-appropriate and take into account the transitional state.

[0042] The following is the reference implementation code for the recognition and construction unit and the context transformation simulation module:

[0043] import numpy as np

[0044] import torch

[0045] import torch.nn as nn

[0046] import torch.nn.functional as F

[0047] from sklearn.cluster import KMeans

[0048] from sklearn.metrics.pairwise import cosine_similarity

[0049] from scipy.spatial.distance import euclidean

[0050] from fastdtw import fastdtw

[0051] # BiLSTM is used for transitional expression modeling

[0052] class BiLSTMTransitionModel(nn.Module):

[0053] def __init__(self, input_dim, hidden_dim, output_dim):

[0054] super(BiLSTMTransitionModel, self).__init__()

[0055] self.lstm = nn.LSTM(input_dim, hidden_dim, num_layers=1,bidirectional=True, batch_first=True)

[0056] self.fc = nn.Linear(hidden_dim 2, output_dim)

[0057] def forward(self, x):

[0058] out, _ = self.lstm(x) # Output shape: [batch_size, seq_len,hidden_dim 2]

[0059] out = self.fc(out[:, -1, :]) # Get the last time step output

[0060] return out # Output is the transition state prediction feature vector

[0061] # Multi-head attention mechanism is used to decouple context components

[0062] class MultiHeadContextSeparator(nn.Module):

[0063] def __init__(self, input_dim, num_heads):

[0064] super(MultiHeadContextSeparator, self).__init__()

[0065] self.attn = nn.MultiheadAttention(embed_dim=input_dim, num_heads=num_heads, batch_first=True)

[0066] def forward(self, x):

[0067] # Treat the input x as a multi-dimensional speech feature vector containing multi-context information

[0068] attn_output, _ = self.attn(x, x, x) # Self-attention, output multiple sub-vectors

[0069] return attn_output # [batch, seq, input_dim], can be split according to the context

[0070] # Context conflict confidence score calculation function

[0071] def calculate_confidence_score(subvector, context_vector, energy_ratio, semantic_similarity,

[0072] weights=(0.4, 0.4, 0.2)):

[0073] acoustic_sim = cosine_similarity(subvector.reshape(1, -1),context_vector.reshape(1, -1))[0][0]

[0074] score = weights[0] semantic_similarity + weights[1] acoustic_sim + weights[2] energy_ratio

[0075] return score

[0076] # Feature masking + fusion weight calculation module

[0077] def apply_mask_and_compute_weights(confidences, threshold):

[0078] masked_confidences = [c if c>= threshold else 0 for c inconfidences]

[0079] masked_confidences = np.array(masked_confidences)

[0080] if masked_confidences.sum() == 0:

[0081] masked_confidences += 1e-6 # To prevent division by zero

[0082] weights = masked_confidences / masked_confidences.sum()

[0083] return weights # The output is the normalized fusion weights

[0084] # Dynamic target vector generation (weighted averaging)

[0085] def generate_dynamic_target_vector(context_vectors, weights):

[0086] # context_vectors: List[np.array], each is the vector of a standard pronunciation sample

[0087] # weights: List[float], the corresponding fusion weights

[0088] weighted_sum = sum(w * v for w, v in zip(weights, context_vectors))

[0089] return weighted_sum # The dynamic target pronunciation feature vector

[0090] # Example: Identifying the building block process control function

[0091] def process_user_input(user_feature_vector, candidate_contexts,context_templates,

[0092] transition_model, separator_model, semantic_model, threshold=0.55):

[0093] """

[0094] user_feature_vector: np.array, the multi-dimensional feature vector of the current speech

[0095] candidate_contexts: List[str], the initially identified context tags

[0096] context_templates: Dict[str, np.array], the standard vectors for each context tag

[0097] transition_model: The transition expression BiLSTM model

[0098] separator_model: The multi-context decoupling module

[0099] semantic_model: The callable sentence vector model (such as BERT)

[0100] """

[0101] # Step 1: Context decoupling

[0102] x_tensor = torch.tensor(user_feature_vector, dtype = torch.float32).unsqueeze(0) # [1, seq_len, dim]

[0103] separated = separator_model(x_tensor).squeeze(0) # [seq_len, dim]

[0104] # Step 2: Calculate the context confidence of each sub-vector

[0105] confidences = []

[0106] context_subvectors = []

[0107] for i, ctx in enumerate(candidate_contexts):

[0108] subvec = separated[i].detach().numpy()

[0109] context_subvectors.append(subvec)

[0110] energy_ratio = np.linalg.norm(subvec) / np.linalg.norm(user_feature_vector)

[0111] semantic_similarity = cosine_similarity(semantic_model.encode(user_feature_vector).reshape(1, -1),

[0112] semantic_model.encode(context_templates[ctx]).reshape(1, -1))[0][0]

[0113] score = calculate_confidence_score(subvec, context_templates[ctx], energy_ratio, semantic_similarity)

[0114] confidences.append(score)

[0115] # Step 3: Mask & normalize fusion weights

[0116] fusion_weights = apply_mask_and_compute_weights(confidences,threshold)

[0117] # Step 4: Generate dynamic target pronunciation vector

[0118] context_vecs = [context_templates[c] for c in candidate_contexts]

[0119] dynamic_target_vector = generate_dynamic_target_vector(context_vecs, fusion_weights)

[0120] return dynamic_target_vector, confidences, fusion_weights

[0121] A feature generation unit 103, configured to call, based on the context label, a target pronunciation feature vector corresponding to the context label from a preset standard pronunciation database.

[0122] In the AI-driven personalized speech training and pronunciation correction system of the present invention, the role of the feature generation unit 103 is to retrieve and call, according to the context label output by the recognition and construction unit 102, a target pronunciation feature vector that matches the context from a preset standard pronunciation database, so as to provide an accurate and context-adaptive reference standard for subsequent difference analysis.

[0123] Specifically, the feature generation unit first receives a context label, which can represent the current speech communication situation of the user, such as children's Q&A, news broadcast, emotional recitation, formal speech, etc. Each context label corresponds to a set of predefined pronunciation reference samples in the system. During the design stage of the system, based on a large amount of high-quality speech data annotated by professional speech teachers, these samples are classified, sorted, and encoded into standardized pronunciation feature vectors. These target pronunciation feature vectors are exactly the same in structure as the user pronunciation feature vectors, and also include multiple dimensions such as pitch, speech rate, intonation contour, formant distribution, rhythm pattern, and their corresponding semantic-emotional labels, ensuring good comparability and algorithm compatibility during comparison.

[0124] The feature generation unit usually embeds an efficient feature retrieval mechanism, which quickly locates the reference pronunciation set in the corresponding context in the database according to the input context label. Preferably, this retrieval mechanism can be implemented based on hash indexing, vector similarity retrieval, or a deep context matching network, ensuring that even when the database scale is large, the system can still retrieve the target feature within milliseconds. In a specific implementation, the feature generation unit can set several calling strategies, such as only calling the optimal reference pronunciation vector in this context, or calling multiple reference samples and generating the final target pronunciation feature vector through an aggregation algorithm (such as average vector, weighted fusion, or confidence optimization) to improve the robustness and flexibility of the comparison.

[0125] To further improve the matching degree, the feature generation unit can also refer to personalized parameters such as the user's age, gender, and pronunciation habits, and call standard samples that are closer to the user's physiological characteristics and expression style in the same context. For example, for child users, the system preferentially calls the standard samples of peers; for users with obvious accents, standard samples with a slight dialect tendency can be called as the target reference. This design makes the training effect not only meet context adaptation but also have individual matching, which helps to improve user acceptance and learning efficiency.

[0126] Once the target pronunciation feature vector is generated, it will be sent to the difference analysis unit 104 as an internal reference standard of the system for multi-dimensional comparison with the current pronunciation feature vector of the user. To ensure the maintainability and scalability of the system, the feature generation unit also needs to consider the update mechanism of the standard pronunciation database during design, support dynamically supplementing context samples according to newly recorded samples by teachers, or automatically optimizing the weights or labels of existing target features based on user training feedback. Such a design can ensure that the system continuously adapts to different users and evolving language expression scenarios during long-term use.

[0127] The difference analysis unit 104 is used to perform multi-dimensional comparison between the user pronunciation feature vector and the target pronunciation feature vector to obtain a set of difference parameters including pronunciation part deviation, speech rate difference, and emotional expression difference.

[0128] In the AI-driven personalized speech training and pronunciation correction system described in this embodiment, the function of the difference analysis unit 104 is to perform multi-dimensional comparison of the speech features between the received user pronunciation feature vector and the target pronunciation feature vector after receiving them, so as to obtain a set of parameters that can quantify the difference between the user's current pronunciation and the standard pronunciation, that is, the difference parameter set. This unit plays a core analysis and accuracy feedback role in the entire system and is the key bridge connecting "recognition construction" and "correction feedback".

[0129] The difference analysis unit first receives the user pronunciation feature vector output by the recognition construction unit. This vector contains multiple parameters of the user's speech in the current context, such as pitch curve, speech rate time series, intonation contour, formant distribution, rhythm structure, and speech feature indicators related to emotion such as short-term energy fluctuation and pitch shift trend. At the same time, the difference analysis unit also receives the target pronunciation feature vector provided by the feature generation unit, which is the feature representation of the standard pronunciation sample matching the user's current context label in the same dimension. To ensure the consistency of the comparison, the data dimensions, sampling lengths, and parameter orders of the two vectors must be strictly aligned, and usually, normalization, resampling, time alignment (such as based on dynamic time warping DTW), etc. are used for standardization processing in the preprocessing stage.

[0130] In the comparison stage, the difference analysis unit uses a dimension-by-dimension comparison method to calculate the distance or deviation between the user's pronunciation and the target pronunciation in each feature dimension. For continuous numerical features (such as pitch, speech rate, intonation curve), metrics such as mean squared error, correlation coefficient, deviation integral, and dynamic matching distance can be used for quantitative analysis; for the matching of the emotion dimension, it can be evaluated based on the KL divergence, cross-entropy, or confidence difference between the emotion classification probability distributions. For discrete event features such as rhythm and stress position, they can be judged through a similarity matrix or event matching rate. Finally, the difference values of all these dimensions will be uniformly encoded into a set of difference parameters, which not only contains the quantitative deviation values of each type of feature, but may also include comprehensive scores, problem priority labels, and recommended intervention intensities automatically generated by the system according to empirical rules or learning models.

[0131] For the convenience of subsequent module calls, the difference analysis unit can also output the set of difference parameters in a structured manner, organizing it into a structure containing three main subsets: The first part is the deviation information of the pronunciation part, which is used to reflect the abnormalities in the use of pronunciation organs such as the tip of the tongue, the root of the tongue, and the lips and teeth; the second part is the speech rate-related parameters, including overall speech rate, local procrastination, uneven rhythm distribution, etc.; the third part is the deviation in emotional expression, which involves aspects such as intonation curve mismatch, emotion inconsistency, and deviation of expression tendency. In addition, this unit can also combine the historical training results with the current comparison results to judge the trend of deviation changes, such as whether a certain type of error appears repeatedly and whether the correction progress is stable, so as to provide a reference basis in the time dimension for personalized recommendations.

[0132] In actual implementation, the difference analysis unit can be deployed as a combined model framework. The front end is a rule engine or an index extraction module built based on expert experience, and the back end is a deep learning model or a graph neural network, which is used to improve the recognition ability of complex pronunciation deviations or emotional inconsistencies. To improve the execution efficiency, the system can adopt a parallel processing architecture, perform comparison operations on different feature dimensions respectively, and merge the difference parameters for output in the final stage.

[0133] In summary, the difference analysis unit 104 outputs a set of difference parameters that can be quantified, analyzed, and applied to personalized feedback through refined multi-dimensional speech feature comparison, providing an accurate, comprehensive, and clearly structured basis for subsequent pronunciation correction suggestions.

[0134] Furthermore, the difference analysis unit includes a user individual behavior deviation prediction network, which is used to identify the long-term expression deviation characteristics of the user in each pronunciation dimension relative to the target pronunciation sample based on the non-uniform pronunciation behavior trajectories that repeatedly appear in the user's historical speech samples under the same semantic content and context expression conditions, and classify and label the confidence level of the deviation characteristics;

[0135] The behavioral offset prediction network includes a sequence modeling structure for constructing a pronunciation stability feature matrix. The feature matrix statistically models the offset direction, offset amplitude, and change trend of each pronunciation dimension in the target semantic-context pair, and outputs an offset trend vector representing the individual's behavioral offset trend. The offset trend vector includes a marker dimension for distinguishing between structural offset and emotional offset;

[0136] The difference analysis unit fuses the offset trend vector with the user's pronunciation feature vector in the current training, and introduces an individual expression boundary weight factor. The individual expression boundary weight factor is a feature tolerance parameter generated based on the "individual expression features that do not need to be corrected" marked or learned and recognized in the user's historical speech data, and is used to suppress the misidentification of this type of expression offset as a pronunciation error during the comparison, thereby generating a corrected user pronunciation feature vector;

[0137] The corrected user pronunciation feature vector is used for multi-dimensional comparison with the target pronunciation feature vector to improve the system's tolerance and comparison accuracy for individual expression differences.

[0138] In the AI-driven personalized speech training and pronunciation correction system of the present invention, in order to improve the system's adaptability to the long-term pronunciation habits of different users, a user individual behavioral offset prediction network is introduced into the difference analysis unit. The core purpose of this network is to identify and model the non-consistent pronunciation behaviors that often occur repeatedly by users under the conditions of the same semantic content and context expression, and reasonably handle such individual differences during the subsequent pronunciation comparison process, so as to prevent the system from misjudging them as pronunciation errors.

[0139] In a specific implementation, the individual behavioral offset prediction network first receives historical speech training data from the speech acquisition unit and the recognition and construction unit. The system preprocesses these historical samples and classifies and organizes them based on the semantic labels (such as "How are you", "Excuse me, how can I get there") and context labels (such as "customer service context", "daily conversation context") corresponding to the user's speech. Under the condition of the same semantic-context pair, the system extracts the pronunciation feature vectors of the user in multiple training rounds and calculates the offset degree of each pronunciation dimension (such as pitch, speech rate, intonation, formant position, emotional intensity, etc.). For example, if a user always shows a situation where the speech rate is about 15% slower and the emotional intensity is 20% lower than the standard speech rate in multiple "customer service contexts", then this offset pattern will be recorded and form a trajectory.

[0140] The sequence modeling structure in this network can adopt mainstream sequence learning models such as bidirectional LSTM (BiLSTM) or Transformer to model the above historical offset trajectories. The system calculates the offset direction (positive or negative), offset amplitude (such as the percentage higher than the standard value), and the change trend in multiple rounds of training (such as linear growth, convergence, fluctuation) respectively on each pronunciation dimension, and generates a pronunciation stability feature matrix. Each row of this matrix corresponds to a pronunciation dimension in a semantic-context pair, and each column represents a certain statistical feature, such as mean offset, variance, slope, etc. Based on the matrix, the system further outputs an offset trend vector to describe the long-term pronunciation offset characteristics of the user in each dimension. At the same time, the system classifies this offset feature into two categories: "structural offset" and "emotional offset". Structural offsets are usually relatively stable, such as the speech rate always being relatively slow, while emotional offsets are susceptible to context fluctuations, such as insufficient emotional expression in the "recitation context" and normal in the "question-and-answer context".

[0141] When performing difference analysis, the system fuses the user's pronunciation feature vector in the current training round with the above offset trend vector. The fusion method can adopt weight weighting or residual compensation model. At this time, to prevent the system from misidentifying the user's personalized long-term expression style as a pronunciation error, the system introduces an individual expression boundary weight factor, which is a feature tolerance parameter generated according to the "expression offsets that do not need to be corrected" automatically identified by manual annotation or model learning in historical pronunciation data. Specifically, in the training stage, the system records the expression habits that are marked as acceptable by experts in historical data (such as a slight downward intonation, slightly prolonging the end of a word, etc.) as sample features, or automatically identifies through clustering algorithms the behaviors where pronunciation offsets exist but do not affect the clarity and acceptability of semantic expression. Under these conditions, the system calculates the offset range of each type of individual expression feature on each pronunciation dimension and forms a feature tolerance boundary. For example, "the pitch is not more than 10Hz lower and the speech rate is not more than 20ms slower" can be regarded as features within the individual expression tolerance range.

[0142] In the difference analysis stage, this weight factor is used to adjust the comparison sensitivity of each pronunciation dimension. For the offset dimensions within the expression boundary range, the system will reduce their difference weights to avoid mislabeling this type of pronunciation offset as a deviation to be corrected, thereby generating a corrected user pronunciation feature vector. This correction process can be implemented by mechanisms such as difference masking, weight decay, or error tolerance threshold in technical implementation.

[0143] In the present invention, an individual expression boundary weight factor is used to adjust the comparison sensitivity of the system to each pronunciation dimension during the differential analysis phase. Its calculation method is based on the long-term pronunciation deviation characteristics shown in the user's historical speech training data and combines the system's judgment result on pronunciation accuracy. The entire calculation process does not require user intervention and is completely automatically executed by the system.

[0144] Specifically, during the process of the user's multi-round speech training, the system first collects and stores the original pronunciation samples of the user under each semantic-context label, and at the same time extracts the characteristic values of the sample in each pronunciation dimension, including but not limited to pitch, speech rate, intonation, formant frequency, and emotional expression parameters, etc. For each dimension, the system records the actual value of the user in this dimension in each round of training, and calculates the difference with the target value of the same dimension in the preset standard pronunciation sample to obtain the deviation value. The system classifies and summarizes these deviation values according to the semantic-context label, and forms a deviation value sequence for each dimension.

[0145] Next, the system performs statistical calculations on the deviation value sequence. For each pronunciation dimension, the system calculates its deviation mean and standard deviation , and records the occurrence frequency of the deviation in this dimension. If the deviation direction of a certain dimension remains consistent in multiple rounds of training, and the deviation values are all within a stable range (that is, the standard deviation is less than the system preset threshold , for example, 5%), and the deviation frequency of this dimension exceeds the set proportion (for example, 70%) in all training samples, and at the same time this deviation is not recognized as an error by the system or does not trigger an artificial correction mark, then the system determines that this deviation belongs to the individual expression difference of the user.

[0146] On this basis, the system assigns a boundary weight factor to this pronunciation dimension according to the following rules. If the absolute value of the deviation mean is less than a certain tolerance range (for example, less than 5% of the average value of this dimension), then set the weight factor of this dimension to 1, indicating that no tolerance processing is performed; if the deviation mean is large but the standard deviation is still in the low fluctuation range, indicating that this deviation is a stable expression style, then set the weight factor to 0.5; if the deviation mean is large and the volatility is high, indicating that this deviation is very likely the result of a specific regional or individual language habit, but does not affect semantic accuracy, then it can be set to a lower weight factor, such as 0.2. All weight factors are finally retained in the system as an individual expression tolerance template for each user and each semantic-context combination, and are called as comparison weights in subsequent differential analysis.

[0147] Finally, the corrected user pronunciation feature vector will be used for multi-dimensional comparison with the standard target pronunciation feature vector. By introducing individual behavior deviation trends and expression boundary factors, the system can accurately distinguish the deviations to be corrected from individual tolerance differences, significantly reducing the false judgment rate while improving the comparison accuracy, enabling the system to have higher human-machine adaptability, personalized feedback capabilities, and speech evaluation reliability in multiple contexts.

[0148] The error correction feedback unit 105 is used to generate pronunciation correction suggestions based on the difference parameter set. The pronunciation correction suggestions include pronunciation action guidelines, intonation adjustment prompts, and semantic emotion strengthening instructions, and are used to output the pronunciation correction suggestions to the user and receive the practice feedback information from the user based on the pronunciation correction suggestions.

[0149] In the AI-driven personalized speech training and pronunciation correction system of the present invention, the function of the error correction feedback unit 105 is to generate targeted pronunciation correction suggestions according to the difference parameter set output by the difference analysis unit 104, and output the suggestions to the user in an understandable and interactive manner, while receiving the practice feedback information generated by the user during the execution of the suggestions, thus constructing a closed-loop speech training process. This unit directly affects the effectiveness of speech training, the intuitiveness of the user experience, and the dynamic adaptation ability of the personalized training mechanism, and is an important link connecting analysis and learning feedback in the system.

[0150] The error correction feedback unit first receives the difference parameter set, which contains the difference information between the user's current pronunciation and the target pronunciation in multiple dimensions, such as the deviation of the pronunciation position, the speed of speech being too fast or too slow, the unnatural rise and fall of intonation, or the inconsistency between emotional expression and context expectations. Based on this difference information, the system calls the rule library and the training strategy model to interpret and make decisions on the deviation of each dimension, thereby generating corresponding pronunciation correction suggestions. Preferably, the error correction suggestions are represented in a structured instruction set, covering the following three categories: First, pronunciation action guidelines, such as prompts like "The tip of the tongue needs to be raised, pay attention to aspiration" and "The lip shape is not round enough, need to open wider", which are usually presented in a way that combines anatomy and phonetics and is applicable to users with different native language backgrounds; second, intonation adjustment prompts, such as "The ending sound of this word needs to rise to indicate a questioning tone" or "The overall intonation needs to drop smoothly to reflect a declarative sentence", and such adjustments are generated based on the pitch curve of standard speech; finally, semantic emotion strengthening instructions, such as "Please enhance the emotional expression of this section and add an appropriate angry tone" or "Need to add a kind tone in this sentence to make it more in line with the children's conversation context".

[0151] The error correction feedback unit not only outputs text prompts, but also enhances the understanding and execution effects through multimodal means. For example, it can demonstrate by combining dynamic graphs, oral structure schematic animations, or standard pronunciation videos, enabling users to more intuitively understand the differences between their own errors and the standards. In addition, to enhance real-time interactivity, this unit supports sentence-by-sentence or word-by-word feedback modes and allows users to conduct repeated training at their own pace. When a user completes a certain exercise, the system will collect their new round of speech data in real time and re-extract their multi-dimensional speech features to form an updated user pronunciation feature vector. This vector will be passed into the difference analysis unit again for comparison, and the system will accordingly judge the correction effect of the user on specific error dimensions.

[0152] During the process of receiving exercise feedback information, the error correction feedback unit also records behavioral metrics such as the number of exercises, error repetition rate, local correction success rate, and user response latency. These feedback information will be used as the basis for subsequent dynamic adjustment of training strategies and model parameters. In implementation, this unit can be deployed as an integrated feedback service module, which contains a feedback generation engine built based on deep reinforcement learning or expert system rules, and also supports remote model updates and local feedback caching.

[0153] To ensure personalized effects, the error correction feedback unit can also dynamically adjust the recommendation strategy by combining the user's historical training data and deviation evolution trajectories. For example, for a certain type of recurring pronunciation error, the system will increase the feedback priority of its error correction suggestions and give more explicit or more basic instructions at the same time; while for the mastered pronunciation points, the prompts can be simplified and the intervention can be reduced to improve the training efficiency and user satisfaction.

[0154] In summary, the error correction feedback unit 105 ensures that users can fully understand the reasons for errors and obtain effective improvement paths by converting the difference parameter set into hierarchical, multi-dimensional, and personalized pronunciation correction suggestions and constructing an interactive feedback channel, ultimately realizing the whole-process closed-loop control of speech training from detection, diagnosis to intervention, and verification.

[0155] Furthermore, the error correction feedback unit includes a feedback trajectory modeling module, which is used to record and generate the error correction history sequence of the user in multiple pronunciation dimensions during multiple training rounds. The pronunciation dimensions include pitch, speech rate, intonation, formant parameters, and emotional expression features;

[0156] The feedback trajectory modeling module is also used to calculate the error correction performance parameters of each type of pronunciation error under the corresponding context label based on the error correction history sequence. The error correction performance parameters include error stability parameters, error volatility parameters, and self-recovery ability parameters, which are calculated through the occurrence frequency variation of errors within a local time window, the error change amplitude between adjacent training rounds, and the error correction rate without prompts respectively;

[0157] The feedback trajectory modeling module is further configured to generate a feedback status code according to the error correction performance parameter, and the feedback status code is used to represent the feedback intervention priority of each type of pronunciation error in a specific context;

[0158] Based on the feedback status code, the error correction feedback unit determines an output strategy for pronunciation correction suggestions, and the output strategy includes intervention frequency, prompt content form, and feedback content structure;

[0159] Wherein, when a specified analog pronunciation error has high stability and low self-recovery ability in multiple training rounds, and similar performances occur under two or more context labels, the error correction feedback unit generates multimodal combined training content including pronunciation part diagrams, standard pronunciation animations, and target context intonation examples, and marks this pronunciation error as a key tracking error, continuously records its performance during subsequent training, and updates the corresponding feedback status code.

[0160] In the AI-driven personalized speech training and pronunciation correction system of the present invention, the error correction feedback unit is not only responsible for outputting correction suggestions for the user's pronunciation deviation, but further includes a feedback trajectory modeling module for continuously tracking, modeling, and regulating the user's pronunciation performance in multiple training rounds, so as to achieve refined personalized error correction management.

[0161] In a specific implementation, the feedback trajectory modeling module records the pronunciation analysis results of the user during each training process, and constructs an error correction history sequence in the form of a time series. This sequence is based on the pronunciation dimension as the structural basis, and respectively tracks multiple types of pronunciation parameters such as pitch, speech rate, intonation, formant parameters (formants, such as F1, F2, F3, etc.), and emotional expression features (such as energy distribution, pitch fluctuation, and emotion category). After each training, the system will identify the types of deviations existing in the user in these dimensions, and record the performance of this error in this round, including information such as error intensity, number of prompts, whether it is corrected, and the context label where it is located, forming a multi-dimensional error correction log with dimension index, time index, and context index.

[0162] Based on the constructed error correction history sequence, the system further calculates the error correction performance parameters of each type of pronunciation error in different contexts to evaluate the evolution trend and intervention difficulty of the error. Among them, the error stability parameter is calculated by setting a time sliding window (such as the last 5 training rounds) to calculate the standard deviation or coefficient of variation of the occurrence frequency of the error within the window, and to judge whether it is a long-term fixed deviation. The error volatility parameter reflects whether there are repeated and uncertain fluctuation behaviors in the user's performance by calculating the error change rate between adjacent training rounds, such as the percentage difference in intonation deviation between the nth round and the n+1th round. The self-recovery ability parameter is used to quantify whether the user can autonomously complete error correction without receiving prompts, and its calculation method can be the proportion of errors that automatically disappear in the unprompted rounds.

[0163] After the above three parameters are normalized, they will be used as inputs to generate the feedback status code. The feedback status code is a structured vector or identifier indicating the feedback priority level of a certain type of pronunciation error in a specific context. The system can set several levels, such as low, medium, high or numerical priority weights, so as to dynamically adjust the feedback behavior of the system when generating pronunciation correction suggestions.

[0164] Based on this feedback status code, the error correction feedback unit selects the corresponding output strategy. The output strategies include the intervention frequency (such as feedback per round or intermittent feedback), the form of the prompt content (such as text instructions, voice replay, pronunciation organ guiding animation) and the structure of the feedback content (such as whether it includes the explanation of the pronunciation features specific to the target context). The system can quickly map the feedback status code to the specific strategy plan through table look-up mapping or strategy rule diagrams to achieve personalized training path adjustment.

[0165] In further reinforcement control, when a certain type of pronunciation error shows high stability in multiple training rounds (that is, the error persists and there is no significant improvement) and has low self-recovery ability (that is, it is difficult for the user to correct without prompts), and similar types of error manifestations are detected under two or more context labels, such as the deviations of too fast speech rate and weak emotional expression in both the "statement context" and the "emotional reading context", the system determines this type of error as the key tracking error. For such key errors, the system no longer only provides brief text prompts, but outputs multi-modal combined training materials containing three types of content: the first is the structural diagram of the pronunciation part, clearly showing the expected states such as tongue position and mouth shape; the second is the standard pronunciation animation, which can repeatedly play the pronunciation process of the standard sample and perform slow decomposition; the third is the target intonation semantic example corresponding to the context in which the pronunciation is located, such as "it is recommended to lengthen the intonation of this sentence to reflect a soothing emotion" or "the end of this sentence needs to rise slightly to reflect the questioning context".

[0166] In addition, this error will be added to the key tracking list by the system, and its performance will be continuously recorded in subsequent training rounds. The feedback status code will be updated after each training to dynamically adjust the training strategy until the performance parameters of this error type reach the set convergence criteria (for example, this error completely disappears or the score significantly decreases in three consecutive training rounds).

[0167] The above feedback trajectory modeling mechanism solves the technical bottlenecks of "rigid feedback", "lack of continuous tracking", and "difficulty in personalized regulation" in traditional speech training systems by introducing technical paths such as quantifying the history of error correction behavior, dynamically adapting feedback strategies, and fusing cross-context performance.

[0168] Furthermore, the error correction feedback unit includes a multi-modal visualization feedback module, which is used to generate pronunciation correction suggestions. Based on the pronunciation part deviation parameters, pitch contour difference curves, and emotional score differences generated by the user in the current training round, it constructs correction guidance content that fuses multiple sources and simultaneously outputs the following three types of visualization feedback information:

[0169] The first type of feedback information constructs a dynamic diagram path of the pronunciation organs corresponding to the current error type of the user based on the built-in three-dimensional pronunciation organ modeling library and pronunciation anatomy mapping template in the system. The path includes the standard movement trajectories of the tongue tip, tongue root, soft palate, pharyngeal wall, and lips, and is adapted to the offset angle and offset amplitude parameters of the user's error part to generate an animation path sequence for contrast correction;

[0170] The second type of feedback information overlays the pitch change contour of the current user's speech and the pitch sequence of the target pronunciation on the unified time axis to form a pitch contrast graph, and marks the segment intervals where the user's pitch is too low or too high, as well as the target adjustment direction and amplitude intervals recommended by the system;

[0171] The third type of feedback information calls the preset emotion correction prompt template according to the current context label and the emotional expression score difference, and dynamically adjusts the intensity and presentation method of the prompt content according to the user's actual performance. The emotion prompt consists of semantic label annotation, intonation dynamic change curve, and tone demonstration audio to form a speech intonation joint animation and is output to the user display terminal in a visual form.

[0172] Among them, the multi-modal visualization feedback module also includes a real-time driving parameter synchronization mechanism, which is used to dynamically update the above diagram content according to the real-time speech input flow during each round of the user's speech training, so that the pronunciation action animation and the pitch annotation curve change in real time with the user's stress behavior, achieving a more targeted and interactive correction guidance effect.

[0173] In the AI-driven personalized speech training and pronunciation correction system described in the present invention, a multimodal visualization feedback module is provided in the error correction feedback unit. This module is used to output correction guidance information in multiple dimensions in a graphical and animated manner during the process of generating pronunciation correction suggestions, enabling users to intuitively perceive the specific location, type, and degree of their own pronunciation deviations, thereby improving the training efficiency and error correction accuracy. This module receives multi-source data inputs from the difference analysis unit and the recognition and construction unit, mainly including three aspects of parameters: First, the pronunciation part deviation parameter generated by the user in the current training round. This parameter is calculated by comparing the pronunciation feature vector with the target standard pronunciation vector and is usually marked with the specific pronunciation part, its offset angle, displacement direction, and offset amplitude in the form of organ structure coding; Second, the pitch contour difference curve. This curve is obtained by performing time normalization on the frame-level pitch of the user's audio signal and calculating the difference value sequence after dynamic time warping with the pitch contour of the target speech, thereby reflecting the relative deviation of the user's pitch from the target pitch throughout the pronunciation segment; Third, the emotional score difference. This difference is obtained by analyzing the emotional expression level of the current speech under the context limitation through an emotion recognition model and comparing it with the emotional expression reference value of the preset target speech in the same context, quantitatively reflecting the deviation degree of the user's speech in terms of emotional intensity or type.

[0174] After receiving the above data, the multimodal visualization feedback module generates three types of graphical feedback information and synchronously outputs it to the user display terminal. The first type of feedback information is the dynamic diagram path of the pronunciation organs. The system calls the pre-constructed oral anatomical structure in the three-dimensional pronunciation organ modeling library, including the standard movement paths of key pronunciation organs such as the tongue tip, tongue root, soft palate, pharyngeal wall, and lips. According to the error type identified in the current training round, the corresponding organ parts are located, and the user's offset parameters are mapped into the model to generate an animation path that contrasts with the standard action. This path can display the error in the spatial displacement of the user's current pronunciation and prompt the user to adjust the pronunciation action through dynamic changes in color, speed, or arrows.

[0175] The second type of feedback information is the pitch comparison graph. The system aligns and overlays the pitch change curve of the user's speech with the target pitch curve along the time axis and clearly marks the intervals where the pitch is too low or too high in the graph, and additionally recommends the adjustment direction and amplitude interval. In the graph, the deviation degree and how to correct it can be indicated by color blocks, gradient bands, or label indication methods. For example, it indicates that the user needs to increase the pitch by about 20 Hz in a certain speech segment or keep it stable instead of a descending tone at the end.

[0176] The third type of feedback information is the emotional expression prompt. The system identifies the context label corresponding to the user's current voice and compares the difference between its emotional expression score and the target emotional baseline value. It calls the emotional prompt template and dynamically generates content strength and output mode in combination with the current deviation type. The output form includes highlighting the position of semantic key words, dynamic lines of intonation change trends, and audio playback demonstrations of the target tone. The system combines these elements into a visual animation sequence and presents it in conjunction with the user's voice clips, allowing users to further correct emotional expression deviations by watching and imitating.

[0177] In order to enhance the responsiveness and interactivity of feedback, the multimodal visual feedback module also includes a real-time driven parameter synchronization mechanism. This mechanism continuously monitors the real-time audio input stream when the user is performing voice training. When it detects that the user has repeated pronunciation or re-reading behavior, it immediately recalculates the relevant difference parameters based on the new real-time data and refreshes the above three types of graphic content synchronously. The system can dynamically update the pronunciation action trajectory, pitch curve or emotional prompts in a local area or the entire content, so that each round of user pronunciation can correspond to a real-time feedback update, thereby achieving a continuous, natural and personalized feedback experience, helping users to more efficiently identify and correct their own pronunciation problems.

[0178] Furthermore, the error correction feedback unit includes a user feedback analysis and training path scheduling module, which is used to extract and accumulate the pronunciation error types, corresponding context labels, prompt response time and correction success or failure mark information that appear in each round of training after receiving multiple rounds of speech feedback samples based on pronunciation correction suggestions from users, construct a structured error performance record set, and generate an error type statistical matrix for training scheduling based on the error performance record set;

[0179] The error correction feedback unit also includes a training response analysis unit, which is used to calculate the training priority score of each error type according to the repetition frequency, correction time interval after prompting, continuous rounds and error amplitude change trend of each error type in the error type statistical matrix in a specific context, and generate a corresponding prompt adjustment weight, wherein the prompt adjustment weight includes the recommended output frequency, prompt content length, visual guidance intensity and interactive tracking cycle length;

[0180] The user feedback analysis and training path scheduling module is configured to construct a dynamic training response label mapping table based on the user's historical training status annotations, divide the user's training response efficiency in different time periods or different contexts into multiple training status labels, and select an adaptive training path module combination according to the current label status, thereby dynamically generating an individualized training prompt plan sequence, and pushing it to the training task scheduling engine in real time in the form of instructions, and updating the calling order, content composition and feedback rhythm of subsequent training tasks.

[0181] In the AI-driven personalized speech training and pronunciation correction system described in the present invention, in order to achieve a dynamically adapted personalized speech correction path, a user feedback analysis and training path scheduling module is set in the error correction feedback unit. This module is mainly used for the structured analysis and path planning of the speech feedback samples generated by the user during multiple rounds of training. Specifically, after the user completes each round of training based on the pronunciation correction suggestions, the system automatically records the feedback data of this round and extracts core elements including the pronunciation error type, the current context label, the response time after the system prompt, and whether the pronunciation in this round is successfully corrected. The identification of the error type is based on the pronunciation dimension deviation results output by the aforementioned difference analysis unit, the context label is determined by semantic parsing combined with the scene classification module, the prompt response time is calculated by the time difference between the system prompt trigger time and the user's start pronunciation time, and the correction result is determined by the system comparing the difference degree between the front and back samples and combining the confidence model score for binary judgment.

[0182] The system stores the above-mentioned extracted multiple pieces of information in the user's individual training file, accumulatively constructs an error performance record set by round, and classifies and counts various pronunciation errors that appear in the record set to form an error type statistical matrix. This matrix represents the error type in the row dimension and statistically counts the occurrence frequency, continuous round distribution, correction delay time after the prompt, and error change curve of this error in each context in the column dimension. The training response analysis unit traverses and normalizes this matrix, calculates the performance stability, improvement speed, and intervention difficulty of each error type at different training stages respectively, and maps these dimensions to a comprehensive training priority score. The higher the value, the greater the impact of this type of error on the learning progress or the more intensive training is required.

[0183] According to the training priority score, the system further assigns corresponding prompt adjustment weights to each error type to adjust the prompt output strategy of this type of error in the subsequent training process. The prompt adjustment weights cover multiple specific dimensions, including the recommended output frequency (such as repeating every n rounds), the length of the prompt content (brief reminder or detailed explanation), the clarity of visual guidance, and the response period of interactive tracking, etc. This weight will be used to control the combination and output rhythm of training content during the task scheduling stage.

[0184] In addition, the user feedback analysis and training path scheduling module also constructs a dynamic training response label mapping table based on the response status marked during the user's historical training process. By evaluating the user's response efficiency in different time periods or contexts, such as the user's quick response in a customer service context and slow correction in emotional reading, the system divides it into multiple training status labels, such as "quick adaptation type", "repeated error-prone type", "rhythm-sensitive type", etc. These labels form a mapping relationship with the training content module. During the actual training task generation stage, the most suitable training path module group is matched according to the current user's label. For example, for users of the "rhythm-sensitive type", the beat-guided prompt is preferentially called, while for "repeated error-prone" users, the multi-round feedback linkage module is called.

[0185] Finally, the system generates an individualized training prompt plan sequence based on the above labels, priority scores, and hint weights, and sends this sequence as an instruction to the training task scheduling engine in real time. The scheduling engine adjusts the call order, content composition structure, and feedback interaction rhythm of subsequent training tasks according to the received prompt plan, achieving precise adaptation to the user's individualized learning rhythm and error correction performance.

[0186] This solution realizes the closed-loop optimization from data collection to training rhythm control through links such as structured extraction of training feedback, multi-dimensional calculation of priorities, construction of status label mapping, and dynamic plan push.

[0187] Furthermore, the training response analysis unit includes a hint adjustment parameter generation module, which is used to dynamically calculate the hint output weight of each error type in the current training task based on the following formula 1 :

[0188] ;

[0189] where represents the historical cumulative hint frequency of this error type under the current context label, reflecting the number of times the system has prompted for this error; it is automatically counted from the system training records and is used to measure the training load density of this error;

[0190] represents the average residual error ratio of this error in the last three rounds of training. Specifically, it is obtained by calculating the relative deviation between the user's pronunciation samples in each round and the target pronunciation in this error dimension and performing average normalization processing;

[0191] represents the error improvement gradient between two consecutive rounds of this error, defined as the difference in residual error between the current round and the previous round, and is used to measure the correction trend; its positive and negative values represent the trends of error exacerbation or weakening respectively, and this value is input as a sensitive parameter for the behavior response direction;

[0192] Indicates the number of context tags involved in this error in the historical training records; obtained by the system through unique counting based on the context annotation information in the training records;

[0193] Indicates the cumulative number of training rounds in which this error has occurred,

[0194] Is the context complexity adjustment coefficient set by the system; its value is set according to the degree of emphasis on the multi-context adaptation ability in the training scenario, usually selected in the range of 0.2 to 1.0.

[0195] The above-mentioned prompt output weight Is used to construct a prompt output control strategy for this error type, including output frequency, graphic display duration, speech rate prompt density, and interaction animation rhythm. The system automatically adjusts the prompt priority and display method of this error in subsequent training tasks according to the weight value to achieve training load balancing, prompt precise adaptation, and feedback path optimization, thereby improving the user's error correction efficiency and the system's intelligent intervention ability.

[0196] After the system completes the calculation of the prompt output weight It will be used as the core control factor and input into the prompt strategy parameter mapping model. This model is a set of parameter mapping function sets established based on experimental experience and machine learning rules. Each type of prompt strategy parameter is based on To adjust and control the final output effect through parametric mapping relationships. Specifically as follows:

[0197] 1. Output frequency:

[0198] The system sets a basic prompt frequency , when Increases, the prompt frequency increases. The calculation of the output frequency Can be calculated using the following formula:

[0199] ;

[0200] Among them, Is the frequency adjustment coefficient, empirically set in the range of 0.1 - 0.3; for example, the basic frequency can be 2 times per minute, and it can also be adjusted according to empirical data.

[0201] 2. Graphic display time:

[0202] The system will dynamically adjust the playing time of each prompt animation or graphic to avoid difficulties in understanding caused by too short prompts or fatigue caused by too long prompts. The display time Can be adjusted in the following way:

[0203] ;

[0204] wherein, is the minimum display time (such as 2 seconds); is the time extension factor (such as 1.5 seconds / unit weight), is the maximum weight corresponding to the upper limit of the display time (such as 3.0), to avoid extremely long prompts caused by outliers.

[0205] 3. Animation rhythm control:

[0206] If is relatively high, it indicates that the current error difficulty is high and the state is complex. The system should slow down the animation rhythm to facilitate the user to observe the action details. The rhythm scaling coefficient is determined by the following rules:

[0207] ;

[0208] where is the rhythm adjustment factor, often set to . This value is multiplied by the original animation frame rate to obtain the final display rhythm. For example, the original speed is 1.0x, when the rhythm is scaled to approximately 0.71x.

[0209] 4. Speech rate feedback density:

[0210] The system can increase or decrease the feedback density of the speech rate, that is, the "speech rate segmented annotation", the number of "slow demonstration", etc. added to the prompt. This density can be obtained through non-linear normalization mapping:

[0211] ;

[0212] where is the maximum insertion density (such as 5 times / prompt), ensuring that it gradually saturates under high weights to avoid over-dense prompts.

[0213] These mapping functions can be optimized through experimental training and user test feedback, and their coefficients or output forms can be dynamically adjusted in combination with the user's training status labels in the system, so as to achieve a truly "variable according to demand" personalized prompt output path.

[0214] The system will finally package these parameters into a prompt strategy parameter group and register them in the training task scheduling module to direct the organization and presentation method of the corresponding prompt content in subsequent training rounds, thus forming a logical closed-loop path of "weight-driven - strategy generation - feedback output".

[0215] By introducing this calculation formula, the system can dynamically generate high-resolution intervention control factors based on the user's historical training status and immediate behavioral responses in different contexts, significantly improving the efficiency and accuracy of prompt resource allocation, thereby enhancing the success rate of user pronunciation correction and the convergence speed of the training path without increasing the training burden.

[0216] Furthermore, the error correction feedback unit includes a training status recognition and feedback adaptation module, which is used to extract correction behavior parameters from the current voice sample after the user completes the voice feedback based on the previous pronunciation correction suggestions. The correction behavior parameters include correction amplitude, response delay, and residual pronunciation error amount, and the user's training status is classified by tags based on preset parameter thresholds, generating training status tags including immediate response type, delayed response type, and structural solidification deviation type.

[0217] Among them, the training status tags are used to select the corresponding pronunciation correction suggestion output strategies. The output strategies include the dynamic combination of text prompts and graphical feedback, rhythm and speech rate assisted guidance, and multi-modal linkage error correction schemes. The training status recognition and feedback adaptation module is also configured with a tag migration trigger mechanism, which is used to detect the change trend of the user's correction behavior parameters in consecutive training rounds. When it is recognized that the tag migration condition is met, the current training status tag is automatically switched and the subsequent feedback strategy is updated until the tag is recognized as entering the stable state interval and then the basic correction output method is restored, thereby realizing the closed-loop control of training feedback based on behavior adaptability.

[0218] In the AI-driven personalized voice training and pronunciation correction system of the present invention, the error correction feedback unit is provided with a training status recognition and feedback adaptation module, which is used to extract multiple indicators related to pronunciation correction behavior from the feedback voice sample submitted by the user after completing the voice training round, and accordingly judge the current user's training status, so as to dynamically match the appropriate pronunciation correction suggestion output strategy to improve the training efficiency and adaptability.

[0219] In the specific implementation process, this module first receives the voice samples recorded by the user based on the previous round of prompt content of the system, and calls the difference analysis unit and the behavior recognition component to extract three key parameters from the samples: The first is the correction amplitude, that is, the degree of difference in multi-dimensional pronunciation features between the user's current pronunciation sample and the previous round of incorrect pronunciation samples, which can be quantitatively represented by the Euclidean distance change between the standard pronunciation feature vector and the current user feature vector, the pitch curve overlap degree, or the alignment offset value of the speech rate time axis, etc.; The second is the response delay, which refers to the time interval between the system outputting the pronunciation correction prompt and the user starting to pronounce. This parameter is obtained by recording the time scale difference between the prompt trigger time and the starting point of the user's voice; The third is the residual pronunciation error amount, that is, the number or degree of pronunciation dimensions that have not been fully corrected in the user's current pronunciation. After the system compares the current sample with the target pronunciation, this data identifies the remaining error types and calculates their quantity and deviation amplitude.

[0220] The system takes these three types of parameters as inputs, compares them with the pre-set numerical thresholds, and classifies the user's current behavior into a certain training status label according to the combined judgment rules. If the correction amplitude is large, the response time is short, and the residual error is low, it is judged as an immediate response type; if the correction amplitude is medium, the response delay is long, and the residual deviation is medium, it is classified as a delayed response type; if the correction amplitude is low and the residual deviation persists and remains stable, it is classified as a structural solidification offset type. The classification criteria for the above labels can be obtained by statistics from a large number of historical sample training data, or formed into a state classification mapping table by manual rule configuration.

[0221] After the label is generated, the system selects an output strategy that matches the user's current status label. For users of the immediate response type, the system defaults to outputting relatively concise text prompts and static graphic information, emphasizing quick confirmation and self-correction; for users of the delayed response type, the system adds speech rate control, rhythm marking, and paragraph segmentation prompt content to help them improve the feedback processing speed and the perception of rhythm stability; for users of the structural solidification offset type, the system calls a multi-modal linkage prompt scheme, including animation of the pronunciation organ movements, simulated audio of emotions and tones, context-adaptive emotion regulation prompts, etc., to form an all-round intervention strategy to break through their inherent expression patterns.

[0222] To adapt to the dynamic characteristics of the user's behavior state changing with the training rounds, the module is also equipped with a label migration trigger mechanism, which continuously monitors the change trend of the corrected behavior parameters in multiple consecutive training rounds. When the system identifies that the correction amplitude under the current label has increased significantly or the response delay has been continuously shortened, and the residual error has been gradually reduced, it automatically determines that the user's training state has changed positively, triggers the state label migration accordingly, updates the current training state label, and synchronously adjusts the output mode of the correction suggestions. This migration process is implemented by setting a threshold window, a moving average model, or a trend-based classifier, etc., to ensure the rationality and stability of label changes.

[0223] When the user's training state is recognized as a stable interval, the system will no longer use the enhanced feedback strategy, but instead revert to the basic correction prompt method to reduce the cognitive load and consolidate the correction results achieved by the user. Through the closed-loop path composed of the above training state recognition, label classification, feedback matching, state migration, and output strategy update, the system can generate flexible prompt schemes for different types of users, realize the training content scheduling driven by behavior responses, and effectively improve the adaptation ability and intervention efficiency of personalized speech training.

[0224] Although this application is disclosed above in preferred embodiments, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the protection scope of this application should be determined by the scope defined by the claims of this application.

Claims

1. An AI-driven personalized speech training and pronunciation correction system, characterized in that, Including: A voice acquisition unit, configured to acquire the original voice data of the user during voice training, and extract a multi-dimensional voice feature vector including pitch, speech rate, intonation, formant parameters, and the text corresponding to the voice from the original voice data; An identification and construction unit, configured to identify the context label of the user's current voice based on the semantic information and emotional features in the multi-dimensional voice feature vector, and construct a corresponding user pronunciation feature vector based on the multi-dimensional voice feature vector; A feature generation unit, configured to call a target pronunciation feature vector corresponding to the context label from a preset standard pronunciation database based on the context label; A difference analysis unit, configured to perform multi-dimensional comparison on the user pronunciation feature vector and the target pronunciation feature vector to obtain a difference parameter set including pronunciation part deviation, speech rate difference, and emotional expression difference; An error correction and feedback unit, configured to generate pronunciation correction suggestions based on the difference parameter set, where the pronunciation correction suggestions include pronunciation action guidance, intonation adjustment tips, and semantic emotion strengthening instructions, and is configured to output the pronunciation correction suggestions to the user and receive the practice feedback information of the user based on the pronunciation correction suggestions; Wherein, the identification and construction unit includes a context transformation simulation module, and the context transformation simulation module is specifically configured to analyze the pronunciation feature continuity between different context labels in the user's historical training data, extract the transitional expression patterns between adjacent context categories, and establish a multi-context mixed expression model based on the transitional expression patterns; When the identification and construction unit detects that the current multi-dimensional voice feature vector contains feature indicators corresponding to two or more context labels at the same time, and the feature indicators have high correlation in historical data, the context transformation simulation module calls a cross-context interference decoder to decouple and separate different context components in the multi-dimensional voice feature vector, generate voice feature sub-vectors corresponding to each context, and calculate the context conflict confidence score of each sub-vector based on the separation result; In the case where any context conflict confidence score exceeds a preset threshold, the context transformation simulation module triggers a feature masking mechanism to partially mask the context feature sub-vectors with confidence lower than the threshold, and calculate the fusion weight for the remaining part; The fusion weight is used to perform weighted combination on the target pronunciation feature vectors corresponding to multiple context labels in the preset standard pronunciation database to generate a dynamic target pronunciation feature vector including an interpretable weight vector annotation, and the dynamic target pronunciation feature vector has a transitional expression feature, which is used to compare with the current user pronunciation feature vector to cope with the non-linear expression deviation caused by the rapid context switching of the user during natural voice training.

2. The AI-driven personalized speech training and pronunciation correction system according to claim 1, characterized in that The error correction and feedback unit includes a feedback trajectory modeling module, and the feedback trajectory modeling module is configured to record and generate the error correction historical sequence of the user in multiple pronunciation dimensions during multiple training rounds, and the pronunciation dimensions include pitch, speech rate, intonation, formant parameters, and emotional expression features; The feedback trajectory modeling module is further configured to calculate, based on the error correction history sequence, error correction performance parameters for each type of pronunciation error under corresponding context tags. The error correction performance parameters include an error stability parameter, an error volatility parameter, and a self-recovery ability parameter, which are calculated respectively by the occurrence frequency variation of the error within a local time window, the error change amplitude between adjacent training rounds, and the error correction rate without prompts. The feedback trajectory modeling module is further configured to generate a feedback status code according to the error correction performance parameters. The feedback status code is used to represent the feedback intervention priority of each type of pronunciation error in a specific context. The error correction feedback unit determines an output strategy for pronunciation correction suggestions based on the feedback status code. The output strategy includes intervention frequency, prompt content form, and feedback content structure. Among them, when a specified analogous pronunciation error has high stability and low self-recovery ability in multiple training rounds and shows similar performance under two or more context tags, the error correction feedback unit generates multimodal combined training content including pronunciation part diagrams, standard pronunciation animations, and target context intonation examples, marks this pronunciation error as a key tracking error, and continuously records its performance and updates the corresponding feedback status code during subsequent training.

3. The AI-driven personalized speech training and pronunciation correction system according to claim 1, characterized in that The difference analysis unit includes a user individual behavior deviation prediction network. The behavior deviation prediction network is configured to identify, based on non-consistent pronunciation behavior trajectories that repeatedly appear in the user's historical speech samples under the same semantic content and context expression conditions, the long-term expression deviation characteristics of the user relative to the target pronunciation sample in each pronunciation dimension, and classify and confidence-label the deviation characteristics. The behavior deviation prediction network includes a sequence modeling structure for constructing a pronunciation stability feature matrix. The feature matrix statistically models the deviation direction, deviation amplitude, and change trend of each pronunciation dimension in the target semantic-context pair, and outputs a deviation trend vector representing the individual behavior deviation trend. The deviation trend vector includes a marked dimension for distinguishing structural deviation from emotional deviation. The difference analysis unit fuses the deviation trend vector with the user's pronunciation feature vector in the current training, and introduces an individual expression boundary weight factor. The individual expression boundary weight factor is a feature tolerance parameter generated based on the "individual expression features that do not need to be corrected" marked or learned and identified from the user's historical speech data, and is used to suppress the misidentification of expression deviation as a pronunciation error during comparison, thereby generating a corrected user pronunciation feature vector. The corrected user pronunciation feature vector is used for multi-dimensional comparison with the target pronunciation feature vector to improve the system's tolerance and comparison accuracy for individual expression differences.

4. The AI-driven personalized speech training and pronunciation correction system according to claim 1, characterized in that, The error correction feedback unit includes a multimodal visualization feedback module, which is configured to construct multi-source fusion correction guidance content based on the pronunciation part deviation parameter, pitch contour difference curve, and emotional score difference generated by the user in the current training round when generating pronunciation correction suggestions, and simultaneously output the following three types of visualization feedback information: The first type of feedback information, based on the system's built-in three-dimensional pronunciation organ modeling library and pronunciation anatomical mapping template, constructs a dynamic graphic path of the pronunciation organ corresponding to the user's current error type. The path includes the standard movement trajectory of the tongue tip, tongue root, soft palate, pharyngeal wall and lips, and is adapted to the offset angle and offset amplitude parameters of the user's error position to generate a contrast-corrected animation path sequence; The second type of feedback information is based on the pitch change contour of the current user's voice and the pitch sequence of the target pronunciation, which are superimposed on a unified time axis to form a pitch comparison chart, and the segment intervals where the user's pitch is too low or too high, as well as the target adjustment direction and amplitude range recommended by the system are marked in the chart; The third type of feedback information, based on the difference between the current context label and the emotion expression score, calls the system's preset emotion correction prompt template, and dynamically adjusts the prompt content intensity and presentation method according to the user's actual performance. The emotion correction prompt is animated by semantic labels, dynamic tone change curves and tone demonstration audio to form a voice and tone joint animation, which is output to the user's display terminal in a visual form; Among them, the multimodal visual feedback module also includes a real-time driving parameter synchronization mechanism, which is used to dynamically update the above-mentioned graphic content according to the real-time voice input stream in each round of user voice training, so that the pronunciation action animation and the tone marking curve change in real time with the user's re-reading behavior, thereby achieving a more targeted and interactive correction and guidance effect.

5. The AI-driven personalized speech training and pronunciation correction system according to claim 1, wherein The error correction feedback unit includes a user feedback analysis and training path scheduling module, which is used to extract and accumulate the pronunciation error types, corresponding context labels, prompt response time and correction success or failure mark information that appear in each round of training after receiving multiple rounds of speech feedback samples based on pronunciation correction suggestions from users, build a structured error performance record set, and generate an error type statistical matrix for training scheduling based on the error performance record set; The error correction feedback unit also includes a training response analysis unit, which is used to calculate the training priority score of each error type according to the repetition frequency, correction time interval after prompting, continuous rounds and error amplitude change trend of each error type in the error type statistical matrix in a specific context, and generate a corresponding prompt adjustment weight, wherein the prompt adjustment weight includes the recommended output frequency, prompt content length, visual guidance intensity and interactive tracking cycle length; The user feedback analysis and training path scheduling module is configured to construct a dynamic training response label mapping table based on the user's historical training status annotations, divide the user's training response efficiency in different time periods or different contexts into multiple training status labels, and select an adaptive training path module combination according to the current label status, thereby dynamically generating an individualized training prompt plan sequence, and pushing it to the training task scheduling engine in real time in the form of instructions, and updating the calling order, content composition and feedback rhythm of subsequent training tasks.

6. The AI-driven personalized speech training and pronunciation correction system according to claim 1, characterized in that, The error correction feedback unit includes a training status recognition and feedback adaptation module, which is used to extract correction behavior parameters from the current voice sample after the user completes the voice feedback based on the pronunciation correction suggestions of the previous round. The correction behavior parameters include the correction amplitude, response delay, and residual pronunciation error amount, and the training status of the user is classified by tags based on preset parameter thresholds, generating training status tags including immediate response type, delayed response type, and structural solidification offset type. Among them, the training status tags are used to select the corresponding pronunciation correction suggestion output strategies, and the output strategies include the dynamic combination of text prompts and graphic feedback, rhythm and speech rate assisted guidance, and multi-modal linkage error correction solutions. The training status recognition and feedback adaptation module is also configured with a tag migration trigger mechanism, which is used to detect the change trend of the user's correction behavior parameters in consecutive training rounds. When it is recognized that the tag migration condition is met, the current training status tag is automatically switched and the subsequent feedback strategy is updated until the tag is recognized as entering the stable state interval and then the basic correction output method is restored, so as to realize the closed-loop control of training feedback based on behavior adaptability.

Images