A mandarin monosyllabic word pronunciation error detection method and system thereof
By combining multi-task learning and multi-scale local feature fusion with the Conformer-MB model, the problem of multi-faceted detection in Mandarin pronunciation error detection is solved, improving the comprehensiveness and accuracy of detection, especially in the detection of phonemes, tones and words.
Patent Information
- Application Number
- CN202211608570.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-14
- Publication Date
- 2026-05-15
- Estimated Expiration
- 2042-12-14
AI Technical Summary
Existing technologies struggle to detect multiple aspects of Mandarin pronunciation errors simultaneously using a single model, such as the overall pronunciation of words, tone, retroflex endings, and stress. Furthermore, traditional Transformer encoders neglect local information during feature extraction.
Using the Conformer-MB model, combined with convolutional neural networks and Transformer models, and through multi-task learning and multi-scale local feature fusion, a Mandarin monosyllabic word pronunciation error detection system is constructed using GOP features and reference text data, which integrates phonemes, tones, and word pronunciation error probabilities.
It improves the pronunciation error detection effect in different aspects of Mandarin, and enhances the comprehensiveness and accuracy of detection, especially in the performance of pronunciation error detection in phonemes, tones and words.
Smart Images
Figure CN116030838B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of language detection technology, specifically relating to a method and system for detecting pronunciation errors of monosyllabic words in Mandarin. Background Technology
[0002] Computer-Assisted Pronunciation Training (CAPT) is an important autonomous language learning technology that facilitates non-native speakers' (L2) learning of their native language's spoken form (L1). Compared to traditional classrooms, CAPT is more economical and convenient, and it also allows language learners to receive immediate feedback on their pronunciation. Due to its practicality, CAPT has been extensively studied, with most work focusing on phoneme-based pronunciation error detection. However, pronunciation error detection encompasses many other aspects, such as whole-word pronunciation, tone, retroflex endings, unstressed syllables, and stress, which have typically been modeled separately in previous studies. However, these different aspects of pronunciation detection tasks are interconnected; therefore, joint modeling might allow the model to learn a more comprehensive representation from different aspects, thereby improving its performance. In reality, we also hope to have a single model capable of detecting pronunciation errors from multiple aspects simultaneously.
[0003] Meanwhile, since GOP features reflect the logarithmic posterior probability relationship between phonemes, there is a certain local correlation between their features. Traditional Transformer encoders often ignore this local information during feature extraction. The Conformer encoder cleverly combines convolutional neural networks and the Transformer model, accurately embedding local features and global representations. The convolutional module in the Conformer model utilizes the BottleNecks convolutional structure often used in ResNet residual networks to reduce computation and parameter count. However, such a structure has limited ability to acquire local features, especially local feature information at different scales, which is useful for various aspects of pronunciation error detection tasks. Against this background, the applicant proposes a method and system for detecting pronunciation errors of monosyllabic words in Mandarin, thereby improving the pronunciation error detection effect of Mandarin in different aspects. Summary of the Invention
[0004] The purpose of this invention is to provide a simple and reasonably designed method and system for detecting pronunciation errors of monosyllabic words in Mandarin in order to solve the above-mentioned problems.
[0005] The present invention achieves the above objectives through the following technical solutions:
[0006] The first aspect of this invention provides a method for detecting pronunciation errors of monosyllabic words in Mandarin Chinese, the method comprising:
[0007] A standard speech recognition model was trained based on Mandarin speech data with mixed regional accents.
[0008] Obtain the audio data to be tested, and based on the audio data to be tested and the standard speech recognition model, obtain GOP feature data and reference text data corresponding to the audio; wherein, the reference text data includes content data and location data;
[0009] By fusing GOP feature data with corresponding audio reference text data, and based on the GOP feature data and the fused data, the Conformer-MB model outputs the pronunciation error probabilities in terms of phonemes, tones, and words.
[0010] Based on a set threshold, the detection result is obtained as to whether the pronunciation is correct or incorrect.
[0011] The standard speech recognition model trained on Mandarin speech data with mixed regional accents is a factorized time-delay neural network standard speech recognition model (Acoustic). The audio data to be tested is used to obtain GOP feature data reflecting pronunciation quality based on the factorized time-delay neural network standard speech recognition model. Based on linear layer mapping, we obtain dimensional data X that is identical to the dimensional data output by the reference text embedding layer. g In this calculation process, X raw The original audio sequence has a length of T. o C is the reference text corresponding to the audio, and its length is T. t The obtained GOP feature data is d g R represents the dimension of the GOP features obtained through the acoustic model, and R is a real vector space.
[0012]
[0013] As a further optimization of the present invention, the reference text data corresponding to the audio is encoded based on the embedding layer to obtain the text content encoding. and text location encoding information It is fused with the GOP feature information. Where, T t d represents the length of the phoneme sequence in the reference text. e Let R be the feature dimension obtained after mapping, and R be a real vector space with vector size T. t *d e ;
[0014] The GOP feature data is fused with the reference text data to obtain the original features.
[0015] and
[0016] X = Concat(X) g ,X t ,X p )
[0017] As a further optimization of the present invention, when calculating the score of the attention mechanism in the Conformer-MB model, X will include the calculation result of the attention mechanism of the previous module to ensure that low-level feature information can be fully preserved. The attention calculation formula can be expressed by the following formula, where α is the weight coefficient of the current attention mechanism module.
[0018] MHSA(Q i ,K i V i )=MHSA(Q i-1 ,K i-1 V i-1 )+(1-α)*MHSA(Q i ,K i V i ).
[0019] As a further optimization of the present invention, local feature extraction is performed based on three dilated convolutions with dilation rates of 1, 5, and 9 and a kernel size of 31*31. The resulting local feature fusion result, Output, is obtained based on the convolution operation. conv Based on Output conv Get the output Output conv This process utilizes residual structures.
[0020] OutPut conv =RESNET(Dilation(X) rate＝1 +Dilation(X) rate＝5 +Dilation(X) rate＝9 )
[0021] OutPut = RESNET(Fn(OutPut) conv )).
[0022] As a further optimization of the present invention, the encoding result of the Conformer-MB encoder is input into the phoneme linear layer, and the output is obtained by averaging the GOP feature encoding results over the sequence length. avg The results are input into the word linear layer and the tone linear layer respectively. The results obtained from each linear layer are mapped to the 0-1 interval based on the Sigmoid function.
[0023] Outputavg =AVENGE(Output)
[0024] P(p) = Sigmod(W) P Output+b p )
[0025] P(w) = Sigmod(W) w Output avg +b w )
[0026] P(t) = Sigmod(W) t Output avg +b t );
[0027] Where P(p) is the probability of correct pronunciation of a phoneme, P(w) is the probability of correct pronunciation of a word, and P(t) is the probability of correct pronunciation of a tone.
[0028] As a further optimization of the present invention, a multi-task learning approach is adopted during the training phase;
[0029] In the reasoning phase, based on the correct pronunciation probabilities obtained from various aspects, a threshold is set, and the final pronunciation error detection result is obtained according to the threshold.
[0030] As a further optimization of the present invention, during the training phase, the loss of each aspect task is calculated using the mean variance (MSE) loss function, and then accumulated to obtain the final loss value Loss, which is then used for backpropagation.
[0031] Loss = Loss phone +Loss word +Loss tone .
[0032] As a further optimization of the present invention, during the reasoning stage, a threshold is set based on the correct pronunciation probability of phonemes, words, and tones obtained from the model, and the correctness of pronunciation is classified according to the threshold.
[0033] result p =Threshold(P(p)) p
[0034] result w =Threshold(P(w)) w .
[0035] result t =Threshold(P(t)) t
[0036] A second aspect of the present invention provides a Mandarin monosyllabic word pronunciation error detection system, characterized in that it includes a memory and a processor, wherein the memory includes a Mandarin monosyllabic word pronunciation error detection method program, and when the Mandarin monosyllabic word pronunciation error detection method program is executed by the processor, it performs the following steps:
[0037] A standard speech recognition model was trained based on Mandarin speech data with mixed regional accents.
[0038] Obtain the audio data to be tested, and based on the audio data to be tested and the standard speech recognition model, obtain GOP feature data and reference text data corresponding to the audio; wherein, the reference text data includes content data and location data;
[0039] By fusing GOP feature data with corresponding audio reference text data, and based on the GOP feature data and the fused data, the Conformer-MB model outputs the pronunciation error probabilities in terms of phonemes, tones, and words.
[0040] Based on a set threshold, the detection result is obtained as to whether the pronunciation is correct or incorrect.
[0041] The standard speech recognition model trained on Mandarin speech data with mixed regional accents is a standard speech recognition model based on factorized time-delay neural networks. The audio data to be tested is used to obtain GOP feature data reflecting the pronunciation quality based on the standard speech recognition model of factorized time-delay neural networks. Based on linear layer mapping, the same dimensional data as the dimensional data output by the embedding layer of the reference text is obtained.
[0042] The beneficial effects of this invention are as follows: This invention discloses a method for detecting multi-faceted pronunciation errors of monosyllabic words in Mandarin based on the Conformer architecture. Utilizing a multi-task learning approach, it supplements current research on multi-faceted joint error detection in Mandarin pronunciation error detection tasks. Furthermore, it proposes a Multiwidthband dilated convolution strategy and applies it to the Conformer model to construct a model called Conformer-MB. This model further improves the pronunciation error detection performance of different aspects of Mandarin by fusing local feature information at multiple scales. Attached Figure Description
[0043] Figure 1 A flowchart of a method for detecting pronunciation errors of monosyllabic words in Mandarin is shown;
[0044] Figure 2 A schematic diagram of the method flow of the Conformer architecture is shown;
[0045] Figure 3A schematic diagram is shown illustrating the improved structure of the convolutional module in the Conformer module using multiple dilated convolutions;
[0046] Figure 4 A block diagram of a Mandarin monosyllabic word pronunciation error detection system is shown. Detailed Implementation
[0047] The present application will now be described in further detail with reference to the accompanying drawings. It should be noted that the following specific embodiments are only used to further illustrate the present application and should not be construed as limiting the scope of protection of the present application. Those skilled in the art can make some non-essential improvements and adjustments to the present application based on the above application content.
[0048] Example 1
[0049] Figure 1 This paper presents a flowchart of a method for detecting pronunciation errors of monosyllabic words in Mandarin.
[0050] A method for detecting pronunciation errors of monosyllabic words in Mandarin, the method includes,
[0051] A standard speech recognition model was trained based on Mandarin speech data with mixed regional accents.
[0052] Obtain the audio data to be tested, and based on the audio data to be tested and the standard speech recognition model, obtain GOP feature data and reference text data corresponding to the audio; wherein, the reference text data includes content data and location data;
[0053] By fusing GOP feature data with corresponding audio reference text data, and based on the GOP feature data and the fused data, the Conformer-MB model outputs the pronunciation error probabilities in terms of phonemes, tones, and words.
[0054] Based on a set threshold, the detection result is obtained as to whether the pronunciation is correct or incorrect.
[0055] Specifically, a standard speech recognition model is first trained using Mandarin speech data with mixed regional accents. The trained standard acoustic model, i-vector feature extractor, and related vocabulary file are then used for the next step of GOP feature extraction.
[0056] In extracting GOP-related features: using the previously trained standard acoustic model, we extract GOP features. The GOP features we use are the Log Phone Posterior Probability (LPP) and the Log Posterior Ratio (LPR), which are defined as follows:
[0057] Specifically, the LPP definition of phoneme p is as shown in Formula 1.
[0058]
[0059] Among them, t e and t s It is the time index that makes up the start and end frames of this phoneme; t is the observation sequence of audio pronunciation states at time t, and s is all pronunciation states corresponding to phoneme p.
[0060] The LPR that makes up the GOP feature sequence is defined as shown in Formula 2, where phoneme p j For the current phoneme, p i For the other phonemes in the vocabulary at that time, O represents the overall observation state sequence of the audio.
[0061] LPR(p j |p i ) = logP(p j |o;t s , t e )-logP(p i |o;t s , t e (2)
[0062] Previously, tone-related information was not considered when calculating GOP features; the pronunciation of tones could be reflected by the [LPP, LPR] features. Further, the previous feature extraction script was improved by adding the vowel and tone as an item in the vocabulary, resulting in a vocabulary size of 218. Therefore, the GOP feature calculated for a certain phoneme p is a 436-dimensional vector, as shown in Equation 3.
[0063] [LPP(p1), ..., LPP(p 218 LPR(p1|p), ..., LPR(p) 218 |p)] (3)
[0064] A multi-faceted pronunciation error detection model was constructed to obtain the pronunciation accuracy of phonemes, tones, and words.
[0065] Based on GOP features, a constructed Conformer-MB model is used for multi-faceted pronunciation error detection. This model has a three-layer Conformer-MB encoder, each consisting of two feedforward network modules, an attention mechanism module, and a convolutional module. The attention mechanism module uses one attention head, and the calculation of attention scores between encoders employs a cross-mode strategy: starting from the second module, the attention score in the current encoder is equal to the attention score in the previous Conformer-MB encoder multiplied by the weights, plus the current encoder's attention score. This effectively avoids the loss of low-level feature information. In the convolutional module of the Conformer-MB encoder, three parallel dilated convolutions are used to obtain local feature information from different scales, and a BatchNorm layer and a Swish activation function are added after each convolutional layer to accelerate model convergence. After obtaining the final feature information from the Conformer-MB encoder, the output is fed into a linear layer, and a Sigmoid activation function is used to map the linear layer output to the probability of correct pronunciation. Simultaneously, the phoneme encoding results output by the encoder are used to obtain the tone and word-level encoding results by averaging the sequence length, and a linear layer and Sigmoid activation function are set for each encoding result.
[0066] Finally, the model outputs a result, and the correctness of the pronunciation is determined by setting a threshold.
[0067] Since the model outputs the probability of correct pronunciation, we need to manually set thresholds during testing to classify the probabilities and ultimately obtain pronunciation error detection results in three aspects (phonemes, tones, and words).
[0068] Specifically, Figure 2 A schematic diagram of the method flow of the Conformer architecture is shown;
[0069] Furthermore, the standard speech recognition model trained on Mandarin speech data with mixed regional accents is a factorized time-delay neural network standard speech recognition model (Acoustic). The audio data to be tested is used to obtain GOP feature data reflecting pronunciation quality based on the factorized time-delay neural network standard speech recognition model. Based on linear layer mapping, we obtain dimensional data X that is identical to the dimensional data output by the reference text embedding layer. g In this calculation process, X raw The original audio sequence has a length of T. o C is the reference text corresponding to the audio, and its length is T. t The obtained GOP feature data is d gR represents the dimension of the GOP features obtained through the acoustic model, and R is a real vector space.
[0070]
[0071] The reference text data corresponding to the audio is encoded based on the embedding layer and then fused with the GOP feature information.
[0072] Specifically, the content of the reference text is encoded using an embedding layer, and the resulting output is... It means that T t d represents the length of the phoneme sequence in the reference text. e The feature dimension obtained after mapping is of size T. t *d e When encoding positions, a truncated normal distribution (trunc_normal) is used to initialize the position parameters of the reference text sequence. The resulting position encoding is obtained by... express.
[0073]
[0074] The GOP feature data is fused with the reference text data to obtain the original features.
[0075] X = Concat(X) g ,X t ,X p ).
[0076] Furthermore, based on the GOP features and text features, the constructed Conformer-MB model is used to obtain pronunciation error detection results from different aspects. The entire encoder includes two feedforward networks, a multi-head self-attention mechanism, and a convolutional module, and uses a cross-module attention strategy. That is, starting from the second module, the output of the multi-head attention mechanism in the i-th module is obtained by summing two parts, specifically, the Q-value in the current module. i K i V i The corresponding attention score is calculated by multiplying the weighting coefficient by the Q value from the previous module. i-1 K i-1 V i-1 The corresponding attention scores are summed to obtain the result, where α is the weighting coefficient of the attention mechanism score of the current module.
[0077] MHSA(Q i ,K i V i )=MHSA(Q i-1 ,K i-1 V i-1)+(1-α)*MHSA(Q i ,K i V i ).
[0078] In the Conformer-MB model, the input vector first passes through a feedforward neural network (Fn), yielding an output X. Then, it passes through a multi-head attention mechanism module to obtain the final vector X. a Furthermore, starting from the second module, when calculating the score of this attention mechanism, X will include the calculation result of the attention mechanism (MHSA) of the previous module to ensure that low-level feature information can be fully preserved. The attention calculation formula can be expressed by the following formula, where α is the weight coefficient of the attention mechanism score of the current module.
[0079] X = Fn(X)
[0080] Q i =X*W q
[0081] K i =X*W k
[0082] V i =X*W v
[0083] X a =(1-α)MHSA(Q) i ,K i V i )+X.
[0084] Specifically, Figure 3 A schematic diagram is shown illustrating the improved structure of the convolutional module in the Conformer module using multiple dilated convolutions;
[0085] Furthermore, local feature extraction is performed based on three dilated convolutions with dilation rates of 1, 5, and 9 and a kernel size of 31*31, ensuring consistent dimensionality after the convolution operation. A residual structure is also used in this step. After the convolution operation, the local feature fusion result, Output, is obtained. conv Based on Output conv Get the output Output (redirect Output) conv The input is fed into the final feedforward module to obtain the output (Output), and a residual structure is used in this process.
[0086] OutPut conv =RESNET(Dilation(X) rate＝1 +Dilation(X) rate＝5+Dilation(X) rate＝9 )
[0087] OutPut = RESNET(Fn(OutPut) conv )).
[0088] Furthermore, the encoding results based on the Conformer-MB encoder are input into the phoneme linear layer. The average of the GOP feature encoding results over the sequence length is then input into the word linear layer and the tone linear layer, respectively. The results obtained from each linear layer are mapped to the 0-1 interval based on the Sigmoid function.
[0089] Output avg =AVENGE(Output)
[0090] P(p) = Sigmod(W) P Output+b p )
[0091] P(w) = Sigmod(W) w Output avg +b w )
[0092] P(t) = Sigmod(W) t Output avg +b t );
[0093] Where P(p) is the probability of correct pronunciation of a phoneme, P(w) is the probability of correct pronunciation of a word, and P(t) is the probability of correct pronunciation of a tone.
[0094] During the training phase, a multi-task learning approach is used for training.
[0095] In the reasoning phase, based on the correct pronunciation probabilities obtained from various aspects, a threshold is set, and the final pronunciation error detection result is obtained according to the threshold.
[0096] During the training phase, the loss for each aspect of the task is calculated using the mean variance (MSE) loss function, and then summed to obtain the final loss value Loss, which is then used for backpropagation.
[0097] Loss = Loss phone +Loss word +Loss tone .
[0098] During the inference phase, based on the correct pronunciation probabilities of phonemes, words, and tones obtained from the model, thresholds are set, and pronunciation errors are classified according to the thresholds.
[0099] result p =Threshold(P(p)) p
[0100] result w =Threshold(P(w)) w
[0101] result t =Threshold(P(t)) t .
[0102] Figure 4 A block diagram of a Mandarin monosyllabic word pronunciation error detection system is shown.
[0103] A Mandarin monosyllabic word pronunciation error detection system includes a memory and a processor. The memory includes a Mandarin monosyllabic word pronunciation error detection method program. When the processor executes the Mandarin monosyllabic word pronunciation error detection method program, it performs the following steps:
[0104] A standard speech recognition model was trained based on Mandarin speech data with mixed regional accents.
[0105] Obtain the audio data to be tested, and based on the audio data to be tested and the standard speech recognition model, obtain GOP feature data and reference text data corresponding to the audio; wherein, the reference text data includes content data and location data;
[0106] By fusing GOP feature data with corresponding audio reference text data, and based on the GOP feature data and the fused data, the Conformer-MB model outputs the pronunciation error probabilities in terms of phonemes, tones, and words.
[0107] Based on a set threshold, the detection result is obtained as to whether the pronunciation is correct or incorrect.
[0108] It should be noted that this method and system for detecting pronunciation errors in Mandarin monosyllabic words first trains a standard speech recognition model using Mandarin speech data with mixed regional accents. Then, the trained standard acoustic model is used to extract GOP features. Next, based on the GOP features and reference text encoding information, a multi-faceted Mandarin pronunciation error detection model is constructed. The Conformer-MB model is used to fuse local GOP features at different scales, thereby improving the error detection performance in various aspects while performing multi-faceted pronunciation error detection. This invention fully utilizes the representation of GOP features in pronunciation error detection, leveraging the characteristics of the Conformer model and dilated convolution to acquire local feature information at various scales, resulting in richer representations. Finally, linear layers and the Sigmoid activation function are used according to different tasks to map the errors to the probability of correct pronunciation, and the pronunciation error detection results are obtained by setting a threshold.
[0109] It should also be noted that in this embodiment, a standard speech recognition model is trained using Mandarin speech data with mixed regional accents; GOP features are extracted using the previously trained standard acoustic model; based on the GOP features, a multi-faceted Mandarin pronunciation error detection model is constructed, and the Conformer-MB model is used to fuse local GOP features at different scales, thereby improving the error detection performance in various aspects while performing multi-faceted pronunciation error detection. This invention fully utilizes the representation of GOP features in pronunciation error detection, and leverages the characteristics of the Conformer model and dilated convolution to acquire local feature information at various scales, resulting in richer representations. On the self-constructed PSC-Monosyllable dataset, the F1 scores for phoneme, tone, and word pronunciation error classification indicators reached 70.01%, 80.20%, and 79.07%, respectively.
[0110] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.
Claims
1. A method for detecting pronunciation errors of monosyllabic words in Standard Mandarin, characterized in that, The method includes, A standard speech recognition model was trained based on Mandarin speech data with mixed regional accents. Obtain the audio data to be tested, and based on the audio data to be tested and the standard speech recognition model, obtain GOP feature data and reference text data corresponding to the audio; wherein, the reference text data includes content data and location data; By fusing GOP feature data with corresponding audio reference text data, and based on the GOP feature data and the fused data, the Conformer-MB model outputs the pronunciation error probabilities in terms of phonemes, tones, and words. Based on a set threshold, the detection result is obtained whether the pronunciation is correct or incorrect; The GOP feature data includes feature data that includes the log-posterior probability of phonemes and the log-posterior ratio.
2. The method for detecting pronunciation errors of monosyllabic words in Mandarin according to claim 1, characterized in that: The standard speech recognition model trained on Mandarin speech data with mixed regional accents is a factorized time-delay neural network standard speech recognition model. The audio data to be tested is used to obtain GOP feature data reflecting pronunciation quality based on the factorized time-delay neural network standard speech recognition model. Based on linear layer mapping, the same dimensional data as the output dimensional data of the reference text embedding layer is obtained. During this calculation process, The original audio sequence has a length of [length missing]. , The reference text corresponding to the audio has a length of [length missing]. The obtained GOP feature data is , The size of the GOP feature dimensions obtained through the acoustic model. It is a real vector space; ； Where Wg is the weight matrix, bg is the bias term, and de is the output dimension of the embedding layer.
3. The method for detecting pronunciation errors of monosyllabic words in Mandarin according to claim 2, characterized in that: The reference text data corresponding to the audio is encoded based on the embedding layer to obtain text content encoding information. and text location encoding information It is fused with the GOP feature information; wherein, The length of the phoneme sequence in the reference text. Output dimension for the embedding layer; The GOP feature data is fused with the reference text data to obtain the original features. The formula is shown below, that is, 。 4. The method for detecting pronunciation errors of monosyllabic words in Mandarin according to claim 3, characterized in that: In the Conformer-MB model, when calculating the score of this attention mechanism, The middle part will include the calculation results of the attention mechanism from the previous module to ensure that low-level feature information is fully preserved. The attention calculation formula can be expressed by the following formula: These are the weight coefficients of the current attention mechanism module; 。 5. The method for detecting pronunciation errors of monosyllabic words in Mandarin according to claim 4, characterized in that: Local feature extraction is performed using three dilated convolutions with dilation rates of 1, 5, and 9 and a kernel size of 31*31. The resulting local feature fusion is then obtained based on the convolution operation. ,based on The output Output is obtained, and a residual structure is used in this process; 。 6. The method for detecting pronunciation errors of monosyllabic words in Mandarin according to claim 5, characterized in that: The encoding results based on the Conformer-MB encoder are input into the phoneme linear layer, and the GOP feature encoding results are averaged over the sequence length to obtain the final result. The results are input into the word linear layer and the tone linear layer respectively. The results obtained from each linear layer are mapped to the 0-1 interval based on the Sigmoid function. ； Where P(p) is the probability of correct phoneme pronunciation, P(w) is the probability of correct word pronunciation, and P(t) is the probability of correct tone pronunciation; Wp and bp are the weight matrices and bias terms of the phoneme linear layer; Ww and bw are the weight matrices and bias terms of the word linear layer; and Wt and bt are the weight matrices and bias terms of the tone linear layer.
7. The method for detecting pronunciation errors of monosyllabic words in Mandarin according to claim 1, characterized in that: During the training phase, a multi-task learning approach is used for training. In the reasoning phase, based on the correct pronunciation probabilities obtained from various aspects, a threshold is set, and the final pronunciation error detection result is obtained according to the threshold.
8. The method for detecting pronunciation errors of monosyllabic words in Mandarin according to claim 7, characterized in that: During the training phase, the loss for each aspect of the task is calculated using the Mean Squared Error (MSE) loss function, and then summed to obtain the final loss value. Then, reverse propagation is performed; 。 9. A method for detecting pronunciation errors of monosyllabic words in Mandarin according to claim 8, characterized in that: During the inference phase, based on the correct pronunciation probabilities of phonemes, words, and tones obtained from the model, thresholds are set, and pronunciation errors are classified according to the thresholds. 。 10. A Mandarin monosyllabic word pronunciation error detection system, characterized in that, The system includes a memory and a processor. The memory contains a program for detecting pronunciation errors of monosyllabic words in Mandarin. When the processor executes the program for detecting pronunciation errors of monosyllabic words in Mandarin, it performs the following steps: A standard speech recognition model was trained based on Mandarin speech data with mixed regional accents. Obtain the audio data to be tested, and based on the audio data to be tested and the standard speech recognition model, obtain GOP feature data and reference text data corresponding to the audio; wherein, the reference text data includes content data and location data; By fusing GOP feature data with corresponding audio reference text data, and based on the GOP feature data and the fused data, the Conformer-MB model outputs the pronunciation error probabilities in terms of phonemes, tones, and words. Based on a set threshold, the detection result is obtained whether the pronunciation is correct or incorrect; The GOP feature data includes feature data that includes the log-posterior probability of phonemes and the log-posterior ratio.