Intelligent foreign language spoken language training method
By preprocessing and augmenting spoken language data, a Generative Adversarial Network (GAN) is constructed to identify and match pronunciation errors. This solves the problem of existing technologies being unable to objectively analyze and expand training content, and significantly improves the effectiveness of spoken language pronunciation training.
Patent Information
- Application Number
- CN202411066462.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-05
- Publication Date
- 2024-11-26
AI Technical Summary
Existing technologies cannot objectively analyze and judge the results of spoken pronunciation training, and the training content is limited and cannot be effectively expanded, resulting in poor spoken pronunciation training effects.
By collecting spoken language data for preprocessing and data augmentation, a generative adversarial network (GAN) is constructed. Standard pronunciation features of the spectral sequence are extracted, user training speech data is collected and pronunciation errors are identified, and similar data is matched for repeated training.
It enables objective analysis and targeted adjustments to spoken pronunciation, significantly improving the effectiveness of spoken pronunciation training.
Smart Images

Figure CN119028325A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to oral training, and in particular to an intelligent foreign language oral training method. Background Art
[0002] At present, in the practice of oral teaching, in order to enable students to master the corresponding oral pronunciation skills, the generally adopted method is to provide students with different categories of oral pronunciation training materials, so that students can repeatedly follow the oral pronunciation training materials, so that their actual oral pronunciation can be close to the standard oral pronunciation corresponding to the oral pronunciation training materials.
[0003] However, the above-mentioned oral pronunciation training method can only rely on students' own subjective feelings to judge whether the training results of oral pronunciation are consistent with standard oral pronunciation, and cannot objectively analyze and judge the students' actual oral pronunciation, which is not conducive to students' targeted adjustment and improvement of their actual oral pronunciation, thereby greatly reducing the training effect of oral pronunciation training. In addition, the training content of oral pronunciation training materials is limited. How to expand oral data based on existing oral pronunciation training materials to help students expand training content is also a problem that needs to be solved. Summary of the invention
[0004] 1. Technical issues to be solved
[0005] In view of the above-mentioned shortcomings of the prior art, the present invention provides a foreign language oral intelligent training method, which can effectively overcome the defects of the prior art that the oral data cannot be amplified and the actual oral pronunciation cannot be objectively analyzed and judged.
[0006] (II) Technical solution
[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions:
[0008] A foreign language oral intelligent training method comprises the following steps:
[0009] S1. Collect spoken data, preprocess the spoken data, and obtain a spoken data set;
[0010] S2, performing data augmentation on the spoken language data set to obtain an augmented spoken language data set, and using the augmented spoken language data set for foreign language spoken training;
[0011] S3, collecting training speech data of the user for foreign language oral training, and extracting feature parameters of the training speech data;
[0012] S4, performing pronunciation error recognition on the training speech data according to the characteristic parameters of the training speech data to obtain a pronunciation error recognition result;
[0013] S5. Match similar spoken data in the augmented spoken data set with respect to the pronunciation error recognition results, and use the similar spoken data to repeat the foreign language spoken training.
[0014] Preferably, in S1, the spoken data is collected and preprocessed to obtain a spoken data set, including:
[0015] S11, collecting spoken data, performing window filtering and frequency domain transformation on the spoken data to obtain a corresponding spectrum sequence, and normalizing the spectrum sequence;
[0016] S12, extracting the standard pronunciation features of the normalized spectrum sequence using a linear prediction coding method;
[0017] S13. Match the standard pronunciation features with the spoken data, and annotate the spoken data to obtain a spoken data set.
[0018] Preferably, in S11, window filtering and frequency domain transformation are performed on the spoken data to obtain a corresponding spectrum sequence, including:
[0019] Fast Fourier transform is used to transform the spoken data into frequency domain to obtain the corresponding spectrum sequence:
[0020]
[0021] Among them, F(A,B) represents the spectrum sequence, A and B represent the two-dimensional vectors of the spectrum sequence, f(a,b) represents spoken data, a and b represent the two-dimensional vectors of spoken data, and m and n represent the number of two-dimensional vectors a and b of spoken data respectively.
[0022] Preferably, in S2, data augmentation is performed on the spoken language data set to obtain an augmented spoken language data set, and the augmented spoken language data set is used for foreign language spoken training, including:
[0023] S21, construct a generative adversarial network GAN, and input the spoken language dataset into the generative adversarial network GAN for data amplification;
[0024] S22, performing window filtering and frequency domain transformation on the new spoken language data in the augmented spoken language data set to obtain a corresponding spectrum sequence, and performing normalization processing on the spectrum sequence;
[0025] S23, extracting the standard pronunciation features of the normalized spectrum sequence using a linear predictive coding method;
[0026] S24, matching the standard pronunciation features with the new spoken data, and annotating the new spoken data to obtain an augmented spoken data set;
[0027] S25. Use the augmented spoken language dataset for foreign language spoken language training.
[0028] Preferably, the generative adversarial network GAN includes a generative network G, a first discriminative network D1 and a second discriminative network D2;
[0029] The generative network G takes the spoken language dataset as input, captures the distribution of sample data, and generates new sample data that is considered to be true in the first discriminant network D1 as much as possible;
[0030] The first discriminant network D1 judges the authenticity of the new sample data generated by the generative network G;
[0031] The second discriminant network D2 controls the authenticity and smoothness of the new sample data generated by the generative network G.
[0032] Preferably, in S3, the training speech data of the user for foreign language oral training is collected, and the characteristic parameters of the training speech data are extracted, including:
[0033] S31, collecting training speech data of a user for foreign language oral training, performing window filtering and frequency domain transformation on the training speech data, and obtaining a corresponding spectrum sequence;
[0034] S32, obtaining a mixed feature vector of the training speech data according to the spectrum sequence of the training speech data;
[0035] Among them, the mixed feature vector includes Mel-frequency cepstral coefficients and optimized voice start time. The optimized voice start time is the difference between the earlier time of vocal cord vibration and the end of the corresponding phoneme and the oral release time. The oral release time is the start time of the phoneme after the speech corpus is segmented by phonemes.
[0036] Preferably, obtaining a mixed feature vector of the training speech data according to the spectrum sequence of the training speech data in S32 includes:
[0037] S321, inputting the spectrum sequence of the training speech data into a set of Mel-scaled triangular filter banks to convert the spectrum from a linear scale to a Mel scale;
[0038] S322, taking the logarithm of the energy output by each filter, and performing discrete cosine transform DCT, taking a preset number of coefficients as Mel-frequency cepstrum coefficients;
[0039] The Mel scale is calculated using the following formula:
[0040]
[0041] In the above formula, f M is the Mel scale, and f is the actual frequency.
[0042] Preferably, in S4, performing pronunciation error recognition on the training speech data according to the characteristic parameters of the training speech data to obtain a pronunciation error recognition result includes:
[0043] S41, obtaining spoken data and its corresponding spectrum sequence in the augmented spoken data set, and inputting the spectrum sequence of the spoken data into a set of Mel-scaled triangular filter banks to convert the spectrum from a linear scale to a Mel scale;
[0044] S42, taking the logarithm of the energy output by each filter, performing discrete cosine transform DCT, and taking a preset number of coefficients as Mel-frequency cepstrum coefficients;
[0045] S43, determining the difference between the time when the vocal cord vibration of the spoken data occurs earlier than the time when the corresponding phoneme ends and the oral discontinuity time, where the oral discontinuity time is the start time of the phoneme after the speech corpus is segmented according to the phoneme, and obtaining a mixed feature vector of the spoken data;
[0046] S44, matching the mixed feature vector of the spoken data with the mixed feature vector of the training speech data to perform pronunciation error recognition on the training speech data to obtain a pronunciation error recognition result.
[0047] Preferably, in S5, matching similar spoken data in the augmented spoken data set with respect to the pronunciation error recognition result, and repeating the foreign language spoken training using the similar spoken data, comprises:
[0048] S51, obtaining a mixed feature vector of the target spoken data corresponding to the mispronounced training speech data, and matching similar spoken data in the augmented spoken data set by calculating the similarity between the mixed feature vectors;
[0049] S52, obtaining a standard pronunciation spectrogram of similar spoken language data, and providing a pre-recorded standard pronunciation oral and tongue position animation for the user to repeat foreign language oral training.
[0050] Preferably, obtaining a mixed feature vector of the target spoken data corresponding to the mispronounced training speech data in S51, and matching similar spoken data in the augmented spoken data set by calculating the similarity between the mixed feature vectors, comprises:
[0051] S511, normalizing the mixed feature vector of the target spoken language data and the mixed feature vector of other spoken language data in the augmented spoken language data set;
[0052] S512: Calculate the similarity between the normalized mixed feature vector of the target spoken language data and the mixed feature vectors of other spoken language data in the augmented spoken language data set using the following formula:
[0053]
[0054] Among them, S(X, Y i ) is the mixed feature vector X of the target spoken data and the mixed feature vector Y of the i-th spoken data in the augmented spoken data set i The similarity between j is the jth component of the mixed feature vector X of the target spoken language data, Y ij is the mixed feature vector Y of the i-th spoken data in the augmented spoken data set i The jth component of , N is the dimension of the mixed feature vector.
[0055] (III) Beneficial effects
[0056] Compared with the prior art, the foreign language oral intelligent training method provided by the present invention has the following beneficial effects:
[0057] 1) Collect spoken data, pre-process the spoken data to obtain a spoken data set, perform data amplification on the spoken data set to obtain an amplified spoken data set, and use the amplified spoken data set for foreign language spoken training, build a generative adversarial network (GAN) to perform data amplification on the spoken data set, and effectively expand the training content. At the same time, by performing window filtering and frequency domain transformation on the spoken data in the amplified spoken data set, the corresponding spectrum sequence is obtained, and the spectrum sequence is normalized. The linear predictive coding method is used to extract the standard pronunciation features of the normalized spectrum sequence, and the spoken data is annotated, so that users can use the standard pronunciation features for standardized foreign language spoken training;
[0058] 2) Collecting training speech data of the user for oral foreign language training, and extracting characteristic parameters of the training speech data, performing pronunciation error recognition on the training speech data according to the characteristic parameters of the training speech data, and obtaining a pronunciation error recognition result, and matching the mixed characteristic vector of the oral data with the mixed characteristic vector of the training speech data, so as to perform pronunciation error recognition on the training speech data and obtain a pronunciation error recognition result, thereby being able to objectively analyze and judge the actual oral pronunciation of the user;
[0059] 3) Match similar spoken data in the augmented spoken data set with the pronunciation error recognition results, use similar spoken data to repeat foreign language oral training, match similar spoken data in the augmented spoken data set by calculating the similarity between mixed feature vectors, obtain the standard pronunciation spectrogram of similar spoken data, and provide pre-recorded standard pronunciation oral tongue position animation for users to repeat foreign language oral training, thereby helping users to make targeted adjustments and improvements to their actual oral pronunciation, greatly improving the training effect of oral pronunciation training. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0061] Figure 1 It is a schematic diagram of the process of the present invention;
[0062] Figure 2 A schematic diagram of a process of performing data amplification on a spoken language data set to obtain an amplified spoken language data set in the present invention;
[0063] Figure 3 It is a flow chart of repeatedly performing foreign language oral training on mispronounced target oral data in the present invention. DETAILED DESCRIPTION
[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0065] A foreign language oral intelligent training method, such as Figure 1 and Figure 2 As shown, S1, collect spoken data, pre-process the spoken data, and obtain a spoken data set, which specifically includes:
[0066] S11, collecting spoken data, performing window filtering and frequency domain transformation on the spoken data to obtain a corresponding spectrum sequence, and normalizing the spectrum sequence;
[0067] S12, extracting the standard pronunciation features of the normalized spectrum sequence using a linear predictive coding method;
[0068] S13. Match the standard pronunciation features with the spoken data, and annotate the spoken data to obtain a spoken data set.
[0069] Specifically, in S11, the spoken data is subjected to window filtering and frequency domain transformation to obtain a corresponding spectrum sequence, including:
[0070] Use fast Fourier transform to transform the spoken data into frequency domain and get the corresponding spectrum sequence:
[0071]
[0072] Among them, F(A,B) represents the spectrum sequence, A and B represent the two-dimensional vectors of the spectrum sequence, f(a,b) represents spoken data, a and b represent the two-dimensional vectors of spoken data, and m and n represent the number of two-dimensional vectors a and b of spoken data respectively.
[0073] S2. Performing data augmentation on the spoken language data set to obtain an augmented spoken language data set, and using the augmented spoken language data set for foreign language spoken training, specifically including:
[0074] S21, construct a generative adversarial network GAN, and input the spoken language dataset into the generative adversarial network GAN for data amplification;
[0075] S22, performing window filtering and frequency domain transformation on the new spoken language data in the augmented spoken language data set to obtain a corresponding spectrum sequence, and performing normalization processing on the spectrum sequence;
[0076] S23, extracting the standard pronunciation features of the normalized spectrum sequence using a linear prediction coding method;
[0077] S24, matching the standard pronunciation features with the new spoken data, and annotating the new spoken data to obtain an augmented spoken data set;
[0078] S25. Use the augmented spoken language dataset for foreign language spoken language training.
[0079] In the technical solution of the present application, the generative adversarial network GAN includes a generating network G, a first discriminant network D1 and a second discriminant network D2;
[0080] The generative network G takes the spoken language dataset as input, captures the distribution of sample data, and generates new sample data that is considered to be true in the first discriminant network D1 as much as possible;
[0081] The first discriminant network D1 judges the authenticity of the new sample data generated by the generative network G;
[0082] The second discriminant network D2 controls the authenticity and smoothness of the new sample data generated by the generative network G.
[0083] The above technical scheme collects spoken data, preprocesses the spoken data to obtain a spoken data set, performs data amplification on the spoken data set to obtain an amplified spoken data set, and uses the amplified spoken data set for foreign language oral training, constructs a generative adversarial network GAN to perform data amplification on the spoken data set, and effectively expands the training content. At the same time, by performing window filtering and frequency domain transformation on the spoken data in the amplified spoken data set, a corresponding spectrum sequence is obtained, and the spectrum sequence is normalized. The linear predictive coding method is used to extract the standard pronunciation features of the normalized spectrum sequence, and the spoken data is annotated, so that users can use the standard pronunciation features to perform standardized foreign language oral training.
[0084] like Figure 1 and Figure 3 As shown, S3, collecting training speech data of users for foreign language oral training, and extracting feature parameters of the training speech data, specifically including:
[0085] S31, collecting training speech data of a user for foreign language oral training, performing window filtering and frequency domain transformation on the training speech data, and obtaining a corresponding spectrum sequence;
[0086] S32, obtaining a mixed feature vector of the training speech data according to the spectrum sequence of the training speech data;
[0087] Among them, the mixed feature vector includes Mel-frequency cepstral coefficients and optimized voice start time. The optimized voice start time is the difference between the earlier time of vocal cord vibration and the end of the corresponding phoneme and the oral release time. The oral release time is the start time of the phoneme after the speech corpus is segmented by phonemes.
[0088] Specifically, obtaining a mixed feature vector of the training speech data according to the spectrum sequence of the training speech data in S32 includes:
[0089] S321, inputting the spectrum sequence of the training speech data into a set of Mel-scaled triangular filter banks to convert the spectrum from a linear scale to a Mel scale;
[0090] S322, taking the logarithm of the energy output by each filter, and performing discrete cosine transform DCT, taking a preset number of coefficients as Mel-frequency cepstrum coefficients;
[0091] The Mel scale is calculated using the following formula:
[0092]
[0093] In the above formula, f M is the Mel scale, and f is the actual frequency.
[0094] S4, performing pronunciation error recognition on the training speech data according to the characteristic parameters of the training speech data to obtain a pronunciation error recognition result, specifically including:
[0095] S41, obtaining spoken data and its corresponding spectrum sequence in the augmented spoken data set, and inputting the spectrum sequence of the spoken data into a set of Mel-scaled triangular filter banks to convert the spectrum from a linear scale to a Mel scale;
[0096] S42, taking the logarithm of the energy output by each filter, performing discrete cosine transform DCT, and taking a preset number of coefficients as Mel-frequency cepstrum coefficients;
[0097] S43, determining the difference between the time when the vocal cord vibration of the spoken data occurs earlier than the time when the corresponding phoneme ends and the oral discontinuity time, where the oral discontinuity time is the start time of the phoneme after the speech corpus is segmented according to the phoneme, and obtaining a mixed feature vector of the spoken data;
[0098] S44, matching the mixed feature vector of the spoken data with the mixed feature vector of the training speech data to perform pronunciation error recognition on the training speech data to obtain a pronunciation error recognition result.
[0099] The above technical scheme collects training voice data of users for foreign language oral training, and extracts characteristic parameters of the training voice data, performs pronunciation error recognition on the training voice data according to the characteristic parameters of the training voice data, and obtains pronunciation error recognition results, and can perform pronunciation error recognition on the training voice data by matching the mixed feature vectors of the oral data with the mixed feature vectors of the training voice data, and obtain pronunciation error recognition results, thereby being able to objectively analyze and judge the user's actual oral pronunciation.
[0100] like Figure 1 and Figure 3 As shown, S5, matching similar spoken data in the augmented spoken data set for the pronunciation error recognition result, and repeating the foreign language spoken training using the similar spoken data, specifically including:
[0101] S51, obtaining a mixed feature vector of the target spoken data corresponding to the mispronounced training speech data, and matching similar spoken data in the augmented spoken data set by calculating the similarity between the mixed feature vectors;
[0102] S52, obtaining a standard pronunciation spectrogram of similar spoken language data, and providing a pre-recorded standard pronunciation oral and tongue position animation for the user to repeat foreign language oral training.
[0103] Specifically, in S51, a mixed feature vector of the target spoken data corresponding to the mispronounced training speech data is obtained, and similar spoken data is matched in the augmented spoken data set by calculating the similarity between the mixed feature vectors, including:
[0104] S511, normalizing the mixed feature vector of the target spoken language data and the mixed feature vector of other spoken language data in the augmented spoken language data set;
[0105] S512: Calculate the similarity between the normalized mixed feature vector of the target spoken language data and the mixed feature vectors of other spoken language data in the augmented spoken language data set using the following formula:
[0106]
[0107] Among them, S(X, Y i ) is the mixed feature vector X of the target spoken data and the mixed feature vector Y of the i-th spoken data in the augmented spoken data set i The similarity between j is the jth component of the mixed feature vector X of the target spoken language data, Y ij is the mixed feature vector Y of the i-th spoken data in the augmented spoken data set i The jth component of , N is the dimension of the mixed feature vector.
[0108] The above technical solution matches similar spoken data in the augmented spoken data set for the pronunciation error recognition results, uses the similar spoken data to repeat foreign language oral training, matches similar spoken data in the augmented spoken data set by calculating the similarity between mixed feature vectors, obtains the standard pronunciation spectrogram of the similar spoken data, and provides a pre-recorded standard pronunciation oral and tongue position animation for users to repeat foreign language oral training, thereby helping users to make targeted adjustments and improvements to their actual oral pronunciation, greatly improving the training effect of oral pronunciation training.
[0109] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A foreign language oral intelligent training method, characterized by: The following steps are involved: S1. Collect spoken data, preprocess the spoken data, and obtain a spoken data set; S2, performing data augmentation on the spoken language data set to obtain an augmented spoken language data set, and using the augmented spoken language data set for foreign language spoken training; S3, collecting training speech data of the user for foreign language oral training, and extracting feature parameters of the training speech data; S4, performing pronunciation error recognition on the training speech data according to the characteristic parameters of the training speech data to obtain a pronunciation error recognition result; S5. Match similar spoken data in the augmented spoken data set with respect to the pronunciation error recognition results, and use the similar spoken data to repeat the foreign language spoken training.
2. The intelligent foreign language oral training method according to claim 1, characterized in that: In S1, oral data is collected and preprocessed to obtain an oral data set, including: S11, collecting spoken data, performing window filtering and frequency domain transformation on the spoken data to obtain a corresponding spectrum sequence, and normalizing the spectrum sequence; S12, extracting the standard pronunciation features of the normalized spectrum sequence using a linear prediction coding method; S13. Match the standard pronunciation features with the spoken data, and annotate the spoken data to obtain a spoken data set.
3. The intelligent foreign language oral training method according to claim 2, characterized in that: In S11, the spoken language data is subjected to window filtering and frequency domain transformation to obtain the corresponding spectrum sequence, including: Fast Fourier transform is used to transform the spoken data into frequency domain to obtain the corresponding spectrum sequence: Among them, F(A,B) represents the spectrum sequence, A and B represent the two-dimensional vectors of the spectrum sequence, f(a,b) represents spoken data, a and b represent the two-dimensional vectors of spoken data, and m and n represent the number of two-dimensional vectors a and b of spoken data respectively.
4. The intelligent foreign language oral training method according to claim 2, characterized in that: In S2, the spoken language data set is augmented to obtain an augmented spoken language data set, and the augmented spoken language data set is used for foreign language spoken training, including: S21, construct a generative adversarial network GAN, and input the spoken language dataset into the generative adversarial network GAN for data amplification; S22, performing window filtering and frequency domain transformation on the new spoken language data in the augmented spoken language data set to obtain a corresponding spectrum sequence, and performing normalization processing on the spectrum sequence; S23, extracting the standard pronunciation features of the normalized spectrum sequence using a linear predictive coding method; S24, matching the standard pronunciation features with the new spoken data, and annotating the new spoken data to obtain an augmented spoken data set; S25. Use the augmented spoken language dataset for foreign language spoken language training.
5. The intelligent foreign language oral training method according to claim 4, characterized in that: The generative adversarial network GAN includes a generating network G, a first discriminant network D1 and a second discriminant network D2; The generative network G takes the spoken language dataset as input, captures the distribution of sample data, and generates new sample data that is considered to be true in the first discriminant network D1 as much as possible; The first discriminant network D1 judges the authenticity of the new sample data generated by the generative network G; The second discriminant network D2 controls the authenticity and smoothness of the new sample data generated by the generative network G.
6. The intelligent foreign language oral training method according to claim 4, characterized in that: S3 collects training speech data of users for foreign language oral training and extracts feature parameters of the training speech data, including: S31, collecting training speech data of a user for foreign language oral training, performing window filtering and frequency domain transformation on the training speech data, and obtaining a corresponding spectrum sequence; S32, obtaining a mixed feature vector of the training speech data according to the spectrum sequence of the training speech data; Among them, the mixed feature vector includes Mel-frequency cepstral coefficients and optimized voice start time. The optimized voice start time is the difference between the earlier time of vocal cord vibration and the end of the corresponding phoneme and the oral release time. The oral release time is the start time of the phoneme after the speech corpus is segmented by phonemes.
7. The intelligent foreign language oral training method according to claim 6, characterized in that: In S32, a mixed feature vector of the training speech data is obtained according to the spectrum sequence of the training speech data, including: S321, inputting the spectrum sequence of the training speech data into a set of Mel-scaled triangular filter banks to convert the spectrum from a linear scale to a Mel scale; S322, taking the logarithm of the energy output by each filter, performing discrete cosine transform DCT, and taking a preset number of coefficients as Mel-frequency cepstrum coefficients; The Mel scale is calculated using the following formula: In the above formula, f M is the Mel scale, and f is the actual frequency.
8. The intelligent foreign language oral training method according to claim 6, characterized in that: In S4, pronunciation errors are identified on the training speech data according to the characteristic parameters of the training speech data to obtain pronunciation error identification results, including: S41, obtaining spoken data and its corresponding spectrum sequence in the augmented spoken data set, and inputting the spectrum sequence of the spoken data into a set of Mel-scaled triangular filter banks to convert the spectrum from a linear scale to a Mel scale; S42, taking the logarithm of the energy output by each filter, performing discrete cosine transform DCT, and taking a preset number of coefficients as Mel-frequency cepstrum coefficients; S43, determining the difference between the time when the vocal cord vibration of the spoken data occurs earlier than the time when the corresponding phoneme ends and the oral discontinuity time, where the oral discontinuity time is the start time of the phoneme after the speech corpus is segmented according to the phoneme, and obtaining a mixed feature vector of the spoken data; S44, matching the mixed feature vector of the spoken data with the mixed feature vector of the training speech data to perform pronunciation error recognition on the training speech data to obtain a pronunciation error recognition result.
9. The intelligent foreign language oral training method according to claim 8, characterized in that: In S5, similar spoken data are matched in the augmented spoken data set for the pronunciation error recognition results, and foreign language spoken training is repeated using similar spoken data, including: S51, obtaining a mixed feature vector of the target spoken data corresponding to the mispronounced training speech data, and matching similar spoken data in the augmented spoken data set by calculating the similarity between the mixed feature vectors; S52, obtaining a standard pronunciation spectrogram of similar spoken language data, and providing a pre-recorded standard pronunciation oral and tongue position animation for the user to repeat foreign language oral training.
10. The intelligent foreign language oral training method according to claim 9, characterized in that: S51 obtains a mixed feature vector of the target spoken data corresponding to the mispronounced training speech data, and matches similar spoken data in the augmented spoken data set by calculating the similarity between the mixed feature vectors, including: S511, normalizing the mixed feature vector of the target spoken language data and the mixed feature vector of other spoken language data in the augmented spoken language data set; S512: Calculate the similarity between the normalized mixed feature vector of the target spoken language data and the mixed feature vectors of other spoken language data in the augmented spoken language data set using the following formula: Among them, S(X, Y i ) is the mixed feature vector X of the target spoken data and the mixed feature vector Y of the i-th spoken data in the augmented spoken data set i The similarity between j is the jth component of the mixed feature vector X of the target spoken language data, Y ij is the mixed feature vector Y of the i-th spoken data in the augmented spoken data set i The jth component of , N is the dimension of the mixed feature vector.
Citation Information
Cited By
Pronunciation recognition learning auxiliary method and system based on voice processing
CN120356485A
