Mandarin training correction method for voice processing

Abstract

The invention discloses a mandarin training correction method for voice processing, and relates to the technical field of voice signal processing, and the method comprises the steps: recording the voice when a user reads a training text, and the amplitude-frequency feature sequence of each read character voice; constructing corresponding vowel feature vectors and initial feature vectors; detecting non-standard vowel syllables of the user; detecting non-standard initial syllables of the user; and prompting the user, and broadcasting the corresponding standard mandarin syllable pronunciation and spelling method to the user so as to train and correct the mandarin of the user. According to the method, the specific syllables of the user are accurately captured and carefully recognized, the pronunciation and spelling problems of the user are effectively pointed out, mandarin training and correction of the user are more efficient and accurate, and the method has great significance in improving the language ability and communication level of the public.

Description

Technical Field

[0001] The present invention relates to the technical field of speech signal processing, and in particular to a Mandarin training and correction method for speech processing. Background Art

[0002] As a standard pronunciation, Mandarin is of great significance in communication and education. However, due to the influence of dialects, regional differences and personal pronunciation habits, many people have varying degrees of phonetic deviation in Mandarin pronunciation.

[0003] Traditional speech correction methods mainly rely on manual teaching, where teachers need to correct learners' pronunciation word by word and sound by sound. This method is not only time-consuming and labor-intensive, but also lacks personalization and accuracy.

[0004] At present, Mandarin training and correction technology based on speech recognition has made some progress, but it still has limitations in accurately capturing and analyzing subtle deviations in individual speech. In particular, existing technologies are often unable to recognize and correct specific syllables such as initials and finals. This limitation makes it difficult for many Mandarin learners to effectively improve their pronunciation problems, hindering the fluency and accuracy of language communication. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a Mandarin training and correction method for speech processing, which solves the problem that the current technology has limitations in accurately capturing and analyzing subtle deviations in individual speech, and is unable to identify and correct specific syllables such as initials and finals.

[0006] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0007] A method for mandarin training and correction of speech processing comprises the following steps:

[0008] S1, recording the user's voice when reading the training text, and obtaining the amplitude-frequency feature sequence of each word after segmentation, analog-to-digital conversion, energy normalization and frequency domain analysis;

[0009] S2, analyzing the frequency of each amplitude peak in each read-aloud character sound amplitude-frequency feature sequence, constructing the corresponding read-aloud character sound vowel feature vector, and analyzing the envelope of each read-aloud character sound amplitude-frequency feature sequence, constructing the corresponding read-aloud character sound initial consonant feature vector;

[0010] S3, comparing the frequency deviation and trend difference of the final feature vector of each pronounced word with the standard Mandarin pronunciation, and detecting the user's non-standard final syllables;

[0011] S4, comparing the initial consonant feature vectors of each pronounced character with the standard Mandarin character pronunciation in a graded manner, and detecting the user's non-standard initial consonant syllables;

[0012] S5. Prompt the user based on the non-standard syllables detected by S3 and S4, and broadcast the corresponding standard Mandarin syllable pronunciation and spelling method to the user to train and correct the user's Mandarin.

[0013] The beneficial effects of the present invention are as follows: the present invention targets the user's pronunciation, performs frequency domain analysis, and based on the different characteristics of initial consonants and finals, accurately captures and meticulously identifies the user's specific syllables, effectively points out the user's pronunciation and spelling problems, and makes the user's Mandarin training and correction more efficient and accurate, which is of great significance to improving the public's language ability and communication level.

[0014] Furthermore, the S1 comprises the following sub-steps:

[0015] S11, recording the user's voice when reading the training text at a uniform speed, obtaining a reading voice electric signal, and dividing it into a plurality of reading word sound electric signals according to the number of words in the training text, wherein the time length of each reading word sound electric signal is:

[0016] ,

[0017] Among them, τ is the time length of reading the electric signal of the word sound, T is the time length of reading the electric signal of the voice, and L is the number of words in the training text;

[0018] S12, converting each read-aloud word sound electrical signal from an analog signal into a digital signal sequence, and performing energy normalization processing according to the following formula to obtain each read-aloud word sound time domain feature sequence:

[0019] ,

[0020] Among them, s i (k) is the value of the kth element of the time-domain feature sequence of the i-th pronunciation, is the value of the kth element of the digital signal sequence of the i-th spoken word before normalization, and M is the number of sampling points of the electrical signal of the spoken word during analog-to-digital conversion;

[0021] S13, calculating the amplitude-frequency feature sequence corresponding to the time-domain feature sequence of each pronounced word.

[0022] Furthermore, the S13 comprises the following sub-steps:

[0023] S131, according to the time domain feature sequence of each pronounced word, through fast Fourier transform FFT, calculate each pronounced word complex frequency domain sequence;

[0024] S132, by the following formula, according to each read word sound complex frequency domain sequence, calculate each read word sound amplitude frequency sequence:

[0025] ,

[0026] in, is the amplitude value of the nth element of the amplitude frequency sequence of the i-th word read aloud, r i (n) is the real part of the nth element of the complex frequency domain sequence of the i-th pronounced word, j i (n) is the imaginary part of the nth element of the complex frequency domain sequence of the i-th pronounced word;

[0027] S133, by using the following formula, a nonlinear transformation is performed on each read word sound amplitude frequency sequence to enhance recognition, and each read word sound amplitude frequency feature sequence is obtained:

[0028] ,

[0029] Among them, p i (n) is the amplitude value of the nth element of the i-th read word frequency characteristic sequence, γ is the gain coefficient, log 10 is the logarithmic function with base 10, b ias is the bias coefficient.

[0030] The beneficial effects of the above further scheme are: analysis and processing are performed on each word sound to avoid crosstalk between words; energy normalization processing is performed to avoid the influence of different user voice volumes on word sound recognition; the nonlinear transformation formula of the amplitude-frequency sequence is a monotonically increasing function, which not only guarantees the original size regularity, but also amplifies the difference near zero point to effectively enhance the recognition.

[0031] Furthermore, in S2, the process of analyzing the frequencies of each amplitude peak in each read-aloud character sound amplitude-frequency feature sequence and constructing the corresponding read-aloud character sound final feature vector includes the following steps:

[0032] A1. Find the first N elements with the highest amplitude value from high to low in the first M / 2 elements in the amplitude-frequency feature sequence of the read-aloud sound, and record their sequence numbers in the amplitude-frequency feature sequence of the read-aloud sound, where M is the number of sampling points of the read-aloud sound electrical signal during analog-to-digital conversion;

[0033] A2, according to the sequence number of each element in the read word sound amplitude frequency feature sequence recorded in A1, the frequency of the corresponding element is calculated by the following formula:

[0034] ,

[0035] Among them, f x is the frequency of the xth highest amplitude element in the amplitude-frequency feature sequence of the read word, f s is the sampling frequency during analog-to-digital conversion, n xis the sequence number of the element with the xth highest amplitude value in the amplitude-frequency feature sequence of the read-aloud sound, where x is a positive integer less than or equal to N;

[0036] A3. Concatenate the frequencies of the N elements in descending order of the corresponding amplitude values ​​to form a final feature vector.

[0037] The beneficial effect of the above further scheme is that the final vowel is a sound formed when the airflow passes freely through the oral cavity without obstruction, and when the final vowel is pronounced, the vocal cord is in a vibrating state. From a technical perspective, the spectrum of the final vowel usually has a relatively obvious resonance peak structure, which is shown as a prominent peak on the spectrum graph. Therefore, the present invention designs a final vowel feature extraction method based on the amplitude-frequency peak frequency point to identify the final vowel features of the user's pronunciation.

[0038] Furthermore, in S2, the process of analyzing the envelope of each read-aloud sound amplitude-frequency feature sequence and constructing the corresponding read-aloud sound initial consonant feature vector is:

[0039] The numbers of the elements in the amplitude-frequency feature sequence of the read-out characters are used as independent variables, and the amplitude value is used as the dependent variable. The least squares method is used to perform polynomial fitting to obtain the envelope function and fitting coefficients of each order expressed as follows. The fitting coefficients of each order are concatenated into the initial consonant feature vector, regardless of the order:

[0040] ,

[0041] in, is the value of the envelope function of the amplitude-frequency feature sequence of the read-out word when the independent variable is n, n is the element number of the amplitude-frequency feature sequence of the read-out word, α m is the mth order fitting coefficient, n m It is the operation of raising n to the power of m, α0 is the 0th order fitting coefficient, R is the maximum power of the polynomial fitting, and m is the order number of the fitting coefficient.

[0042] The beneficial effect of the above further scheme is that the initial consonant is different from the final consonant. When pronouncing, the airflow in the oral cavity is suddenly released, forming an instantaneous explosive sound. Therefore, the energy of the initial consonant is relatively concentrated in the time domain, but very wide in the frequency domain, affecting the shape of the entire spectrum. The present invention effectively extracts the initial consonant characteristics by analyzing the envelope of amplitude and frequency.

[0043] Furthermore, in S3, the method of comparing the frequency deviation and trend difference of the final feature vector of each pronounced pronunciation with the standard Mandarin pronunciation includes the following steps:

[0044] B1, performing vector subtraction on the final feature vectors of the user's pronunciation and the standard Mandarin pronunciation, without distinguishing between the subtrahend and the minuend, to obtain a frequency deviation vector;

[0045] B2. Calculate the enhancement coefficient sequence according to the frequency offset vector;

[0046] B3, processing the frequency offset vector by the enhancement coefficient sequence to obtain the frequency difference value;

[0047] B4, according to the final feature vectors of the user's pronunciation and the standard Mandarin pronunciation, respectively calculate the trend vectors of the user's pronunciation and the standard Mandarin pronunciation;

[0048] B5. Obtaining a trend difference value according to the trend vectors of the pronunciation of the word read aloud by the user and the pronunciation of the word in standard Mandarin;

[0049] B6, performing weighted summation on the frequency difference value and the trend difference value to obtain the non-standard value of the finals of the word read aloud by the user;

[0050] B7. If the non-standard value of the final of the word pronounced by the user is greater than the final decision threshold, it is determined that the pronunciation of the word pronounced by the user is not standard and the final is a problem syllable.

[0051] Furthermore, the expression used in B2 is:

[0052] ,

[0053] Among them, ξ(x) is the value of the xth element of the enhancement coefficient sequence, is the value of the xth element of the frequency deviation vector, where x is a positive integer less than or equal to the length of the final feature vector;

[0054] The expression used in B3 is:

[0055] ,

[0056] Among them, d istc,f is the frequency difference value, and |∙| is the absolute value operation.

[0057] Furthermore, the expression used in B4 is:

[0058] ,

[0059] ,

[0060] Among them, κ(1) is the value of the first element of the trend vector, κ(x)| x>1 is the value of the xth element of the trend vector when x is greater than 1, f(x) is the value of the xth element of the final feature vector, and f(x-1) is the value of the x-1th element of the final feature vector;

[0061] The expression used in B5 is:

[0062] ,

[0063] Among them, d istc,td is the trend gap value, κ user (x) is the xth element value of the pronunciation trend vector of the word read aloud by the user, κ stand (x) is the x-th element value of the standard Mandarin pronunciation trend vector, and |∙| is the absolute value operation.

[0064] The beneficial effect of the above further scheme is: since the essence of the final feature vector is the set of frequency values ​​corresponding to the amplitude-frequency peak point, the frequency deviation and trend difference can effectively characterize the difference in the final feature vector, which can serve as a reasonable evaluation indicator for the accuracy of the user's pronunciation and efficiently and accurately quantify the standard degree of the user's final pronunciation.

[0065] Furthermore, in S4, the method of comparing the initial consonant feature vectors of each pronounced pronunciation with the standard Mandarin pronunciation in a graded manner includes the following steps:

[0066] C1. The initial consonant feature vectors of the user's pronunciation and the standard Mandarin pronunciation are classified according to the following classification strategy, and the ordinary initial consonant feature vectors, the key initial consonant feature vectors and the secondary initial consonant feature vectors of the user's pronunciation and the standard Mandarin pronunciation are obtained respectively:

[0067] The 0th order fitting coefficient and the 5th order fitting coefficients are combined from small to large order to form the common initial consonant feature vector;

[0068] The first-order fitting coefficients and the second-order fitting coefficients are arranged in ascending order to form a key-level initial consonant feature vector;

[0069] The third-order fitting coefficient and the fourth-order fitting coefficient are arranged in ascending order to form a secondary initial consonant feature vector;

[0070] C2, according to the common initial consonant feature vector, the key initial consonant feature vector and the secondary initial consonant feature vector of the user's pronunciation and the standard Mandarin pronunciation, respectively calculate the user's pronunciation pronunciation initial consonant non-standard value third component, the user's pronunciation pronunciation initial consonant non-standard value first component and the user's pronunciation pronunciation initial consonant non-standard value second component;

[0071] C3, adding the first to third components of the non-standard values ​​of the initial consonants of the characters read aloud by the user to obtain the non-standard values ​​of the initial consonants of the characters read aloud by the user;

[0072] C4. If the non-standard value of the initial consonant of the word read aloud by the user is greater than the initial consonant judgment threshold, it is judged that the pronunciation of the word read aloud by the user is not standard and the initial consonant is a problem syllable.

[0073] Furthermore, in C2, the third component of the non-standard value of the initial consonant of the user's pronunciation is calculated according to the common-level initial consonant feature vectors of the user's pronunciation and the standard Mandarin pronunciation through the following formula:

[0074] ,

[0075] Among them, ς3 is the third component of the non-standard value of the initial consonant of the word read aloud by the user, R is the maximum power of the polynomial fitting, and A 3,user (y3) is the y3th element value of the feature vector of the common initial consonant of the word read aloud by the user, A 3,stand (y3) is the y3th element value of the feature vector of the standard Mandarin initial consonant of the ordinary level, y3 is the sequence number, and |∙| is the absolute value operation;

[0076] The first component of the non-standard value of the initial consonant of the user's pronunciation is calculated according to the key-level initial consonant feature vectors of the user's pronunciation and the standard Mandarin pronunciation through the following formula:

[0077] ,

[0078] Among them, ς1 is the first component of the non-standard value of the initial consonant of the word read aloud by the user, A 1,user (y1) is the y1th element value of the feature vector of the key initial consonant of the word read aloud by the user, A 1,stand (y1) is the y1th element value of the feature vector of the key-level initial consonant of the standard Mandarin pronunciation, y1 is the sequence number, and e is a natural constant;

[0079] The second component of the non-standard value of the initial consonant of the user's pronunciation is calculated according to the secondary initial consonant feature vectors of the user's pronunciation and the standard Mandarin pronunciation through the following formula:

[0080] ,

[0081] Among them, ς2 is the second component of the non-standard value of the initial consonant of the word read aloud by the user, A 2,user (y2) is the y2th element value of the secondary initial consonant feature vector of the word read aloud by the user, A 2,stand (y2) is the y2th element value of the feature vector of the secondary initial consonant of the standard Mandarin pronunciation, and y2 is the sequence number.

[0082] The beneficial effects of the above further scheme are as follows: the essence of the initial consonant feature vector is the fitting coefficients of the multiple terms of the amplitude-frequency envelope. In curve fitting, the 0th order fitting coefficient represents the bias, which is insignificant for the present invention. The 5th order and above fitting coefficients have little influence on the curve, so they are used as the ordinary level; the 1st order and 2nd order fitting coefficients represent the slope and parabolic curvature respectively, which have a great influence on the curve, so they are used as the key level; the 3rd order and 4th order fitting coefficients play a regulating role on the curve and also have a great influence on the curve, so they are used as the secondary level. The present invention exponentially amplifies the key level difference, proportionally amplifies the secondary level difference, and conventionally finds the difference for the ordinary level, so as to achieve hierarchical comparison and efficiently and accurately quantify the standard degree of the user's initial consonant pronunciation. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 The present invention is a flowchart of a method for Mandarin training and correction of speech processing. DETAILED DESCRIPTION

[0084] The specific implementation modes of the present invention are described below to facilitate those skilled in the art to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0085] like Figure 1 As shown, in one embodiment of the present invention, a method for Mandarin training and correction of speech processing includes the following steps:

[0086] S1. Record the user's voice when reading the training text, and obtain the frequency-amplitude feature sequence of each word after segmentation, analog-to-digital conversion, energy normalization and frequency domain analysis. This includes the following steps:

[0087] S11, recording the user's voice when reading the training text at a uniform speed, obtaining a reading voice electric signal, and dividing it into a plurality of reading word sound electric signals according to the number of words in the training text, wherein the time length of each reading word sound electric signal is:

[0088] ,

[0089] Among them, τ is the time length of reading the electric signal of the word sound, T is the time length of reading the electric signal of the voice, and L is the number of words in the training text.

[0090] S12, converting each read-aloud word sound electrical signal from an analog signal into a digital signal sequence, and performing energy normalization processing according to the following formula to obtain each read-aloud word sound time domain feature sequence:

[0091] ,

[0092] Among them, s i (k) is the value of the kth element of the time-domain feature sequence of the i-th pronunciation, is the value of the kth element of the digital signal sequence of the i-th spoken word before normalization, and M is the number of sampling points of the spoken word electrical signal during analog-to-digital conversion.

[0093] The number of sampling points cannot be too small. The frequency range of human voice is mainly between 200 Hz and 4000 Hz. Therefore, the sampling frequency of analog-to-digital conversion should be greater than 8 kHz. The present invention adopts 10 kHz as the sampling frequency of analog-to-digital conversion. In this embodiment, the time length of each reading electric signal is 0.3 seconds. Therefore, the value of M is set to 0.3 times 8k, that is, 2400.

[0094] S13, calculating the amplitude-frequency feature sequence corresponding to the time-domain feature sequence of each pronounced word.

[0095] S13 includes the following sub-steps:

[0096] S131, according to the time domain feature sequence of each pronounced word, through fast Fourier transform FFT, calculate each pronounced word complex frequency domain sequence;

[0097] S132, by the following formula, according to each read word sound complex frequency domain sequence, calculate each read word sound amplitude frequency sequence:

[0098] ,

[0099] in, is the amplitude value of the nth element of the amplitude frequency sequence of the i-th word read aloud, r i (n) is the real part of the nth element of the complex frequency domain sequence of the i-th pronounced word, j i (n) is the imaginary part of the nth element of the complex frequency domain sequence of the i-th pronounced word;

[0100] S133, by using the following formula, a nonlinear transformation is performed on each read word sound amplitude frequency sequence to enhance recognition, and each read word sound amplitude frequency feature sequence is obtained:

[0101] ,

[0102] Among them, p i (n) is the amplitude value of the nth element of the i-th read word frequency characteristic sequence, γ is the gain coefficient, log 10 is the logarithmic function with base 10, b ias is the bias coefficient.

[0103] In this embodiment, γ is set to 20, b ias Set to 98.

[0104] The above steps analyze and process each word to avoid crosstalk between words; energy normalization processing avoids the influence of different user voice volumes on word recognition; the nonlinear transformation formula of the amplitude-frequency sequence is a monotonically increasing function, which not only guarantees the original size regularity, but also amplifies the difference near zero point to effectively enhance the recognition.

[0105] S2. Analyze the frequency of each amplitude peak in each amplitude-frequency feature sequence of the read-aloud pronunciation, construct the corresponding feature vector of the read-aloud pronunciation vowel, and analyze the envelope of each amplitude-frequency feature sequence of the read-aloud pronunciation, and construct the corresponding feature vector of the read-aloud pronunciation initial consonant.

[0106] In S2, the process of analyzing the frequencies of each amplitude peak in each read-aloud character sound amplitude-frequency feature sequence and constructing the corresponding read-aloud character sound final feature vector includes the following steps:

[0107] A1. Find the first N elements with the highest amplitude value from high to low in the first M / 2 elements in the amplitude-frequency feature sequence of the read-aloud sound, and record their sequence numbers in the amplitude-frequency feature sequence of the read-aloud sound, where M is the number of sampling points of the read-aloud sound electrical signal during analog-to-digital conversion;

[0108] A2, according to the sequence number of each element in the read word sound amplitude frequency feature sequence recorded in A1, the frequency of the corresponding element is calculated by the following formula:

[0109] ,

[0110] Among them, f x is the frequency of the xth highest amplitude element in the amplitude-frequency feature sequence of the read word, f s is the sampling frequency during analog-to-digital conversion, n x is the sequence number of the element with the xth highest amplitude value in the amplitude-frequency feature sequence of the read-aloud sound, where x is a positive integer less than or equal to N;

[0111] A3. Concatenate the frequencies of the N elements in descending order of the corresponding amplitude values ​​to form a final feature vector.

[0112] In this embodiment, after debugging, N is set to 64.

[0113] The final vowel is the sound formed when the airflow passes through the oral cavity freely without obstruction. When pronouncing the final vowel, the vocal cord is in a vibrating state. From a technical perspective, the spectrum of the final vowel usually has a relatively obvious resonance peak structure, which is shown as a prominent peak on the spectrum graph. Therefore, the present invention designs a final vowel feature extraction method based on the amplitude-frequency peak frequency point to identify the final vowel features of the user's pronunciation.

[0114] In S2, the process of analyzing the envelope of each read-aloud sound frequency feature sequence and constructing the corresponding read-aloud sound initial consonant feature vector is:

[0115] The numbers of the elements in the amplitude-frequency feature sequence of the read-out characters are used as independent variables, and the amplitude value is used as the dependent variable. The least squares method is used to perform polynomial fitting to obtain the envelope function and fitting coefficients of each order expressed as follows. The fitting coefficients of each order are concatenated into the initial consonant feature vector, regardless of the order:

[0116] ,

[0117] in, is the value of the envelope function of the amplitude-frequency feature sequence of the read-out word when the independent variable is n, n is the element number of the amplitude-frequency feature sequence of the read-out word, α m is the mth order fitting coefficient, n m It is the operation of raising n to the power of m, α0 is the 0th order fitting coefficient, R is the maximum power of the polynomial fitting, and m is the order number of the fitting coefficient.

[0118] For those skilled in the art, the least squares method is easy to learn from literature, so this article will not go into details about the specific process.

[0119] Initial consonants are different from finals. When pronouncing, the airflow in the oral cavity is suddenly released, forming an instantaneous explosive sound. Therefore, the energy of the initial consonant is relatively concentrated in the time domain, but very broad in the frequency domain, affecting the shape of the entire spectrum. The present invention effectively extracts the initial consonant characteristics by analyzing the envelope of amplitude and frequency.

[0120] S3. Compare the frequency deviation and trend difference of the final feature vector of each pronounced pronunciation with the standard Mandarin pronunciation to detect the user's non-standard final syllables.

[0121] In S3, the method of comparing the frequency deviation and trend difference of the final feature vector of each pronounced pronunciation with the standard Mandarin pronunciation includes the following steps:

[0122] B1, performing vector subtraction on the final feature vectors of the user's pronunciation and the standard Mandarin pronunciation, without distinguishing between the subtrahend and the minuend, to obtain a frequency deviation vector;

[0123] B2. Calculate the enhancement coefficient sequence according to the frequency offset vector;

[0124] B3, processing the frequency offset vector by the enhancement coefficient sequence to obtain the frequency difference value;

[0125] B4, according to the final feature vectors of the user's pronunciation and the standard Mandarin pronunciation, respectively calculate the trend vectors of the user's pronunciation and the standard Mandarin pronunciation;

[0126] B5. Obtaining a trend difference value according to the trend vectors of the pronunciation of the word read aloud by the user and the pronunciation of the word in standard Mandarin;

[0127] B6, performing weighted summation on the frequency difference value and the trend difference value to obtain the non-standard value of the finals of the word read aloud by the user;

[0128] B7. If the non-standard value of the final of the word pronounced by the user is greater than the final decision threshold, it is determined that the pronunciation of the word pronounced by the user is not standard and the final is a problem syllable.

[0129] The final decision threshold is a threshold set for the non-standard value of the final of the word read aloud by the user. The present invention does not limit the value of the final decision threshold, which is set by the user according to the correction requirements of the final syllable pronunciation.

[0130] The expression used in B2 is:

[0131] ,

[0132] Among them, ξ(x) is the value of the xth element of the enhancement coefficient sequence, is the value of the xth element of the frequency deviation vector, where x is a positive integer less than or equal to the length of the final feature vector;

[0133] The expression used in B3 is:

[0134] ,

[0135] Among them, d istc,f is the frequency difference value, and |∙| is the absolute value operation.

[0136] The expression used in B4 is:

[0137] ,

[0138] ,

[0139] Among them, κ(1) is the value of the first element of the trend vector, κ(x)| x>1 is the value of the xth element of the trend vector when x is greater than 1, f(x) is the value of the xth element of the final feature vector, and f(x-1) is the value of the x-1th element of the final feature vector;

[0140] The expression used in B5 is:

[0141] ,

[0142] Among them, d istc,td is the trend gap value, κ user (x) is the xth element value of the pronunciation trend vector of the word read aloud by the user, κ stand (x) is the x-th element value of the standard Mandarin pronunciation trend vector, and |∙| is the absolute value operation.

[0143] Since the essence of the final feature vector is the set of frequency values ​​corresponding to the amplitude-frequency peak points, the frequency deviation and trend difference can effectively characterize the difference in the final feature vectors, and serve as a reasonable evaluation indicator for the accuracy of user pronunciation, and efficiently and accurately quantify the standard degree of user final pronunciation.

[0144] S4. The initial consonant feature vectors of each pronounced word are compared with the standard Mandarin pronunciation in a graded manner to detect the user's non-standard initial consonant syllables.

[0145] In S4, the method of comparing the initial consonant feature vectors of each pronounced character with the standard Mandarin character in a graded manner includes the following steps:

[0146] C1. The initial consonant feature vectors of the user's pronunciation and the standard Mandarin pronunciation are classified according to the following classification strategy, and the ordinary initial consonant feature vectors, the key initial consonant feature vectors and the secondary initial consonant feature vectors of the user's pronunciation and the standard Mandarin pronunciation are obtained respectively:

[0147] The 0th order fitting coefficient and the 5th order fitting coefficients are combined from small to large order to form the common initial consonant feature vector;

[0148] The first-order fitting coefficients and the second-order fitting coefficients are arranged in ascending order to form a key-level initial consonant feature vector;

[0149] The third-order fitting coefficient and the fourth-order fitting coefficient are arranged in ascending order to form a secondary initial consonant feature vector;

[0150] C2. Calculate the third component of the non-standard value of the initial consonant of the user's pronunciation according to the common initial consonant feature vectors of the user's pronunciation and the standard Mandarin pronunciation through the following formula:

[0151] ,

[0152] Among them, ς3 is the third component of the non-standard value of the initial consonant of the word read aloud by the user, R is the maximum power of the polynomial fitting, and A 3,user (y3) is the y3th element value of the feature vector of the common initial consonant of the word read aloud by the user, A 3,stand (y3) is the y3th element value of the feature vector of the standard Mandarin initial consonant of the ordinary level, y3 is the sequence number, and |∙| is the absolute value operation;

[0153] The first component of the non-standard value of the initial consonant of the user's pronunciation is calculated according to the key-level initial consonant feature vectors of the user's pronunciation and the standard Mandarin pronunciation through the following formula:

[0154] ,

[0155] Among them, ς1 is the first component of the non-standard value of the initial consonant of the word read aloud by the user, A 1,user (y1) is the y1th element value of the feature vector of the key initial consonant of the word read aloud by the user, A 1,stand (y1) is the y1th element value of the feature vector of the key-level initial consonant of the standard Mandarin pronunciation, y1 is the sequence number, and e is a natural constant;

[0156] The second component of the non-standard value of the initial consonant of the user's pronunciation is calculated according to the secondary initial consonant feature vectors of the user's pronunciation and the standard Mandarin pronunciation through the following formula:

[0157] ,

[0158] Among them, ς2 is the second component of the non-standard value of the initial consonant of the word read aloud by the user, A 2,user (y2) is the y2th element value of the secondary initial consonant feature vector of the word read aloud by the user, A 2,stand (y2) is the y2th element value of the secondary initial consonant feature vector of the standard Mandarin Chinese word, and y2 is the sequence number;

[0159] C3, adding the first to third components of the non-standard values ​​of the initial consonants of the characters read aloud by the user to obtain the non-standard values ​​of the initial consonants of the characters read aloud by the user;

[0160] C4. If the non-standard value of the initial consonant of the word read aloud by the user is greater than the initial consonant judgment threshold, it is judged that the pronunciation of the word read aloud by the user is not standard and the initial consonant is a problem syllable.

[0161] The essence of the initial consonant feature vector is the fitting coefficients of the multiple terms of the amplitude-frequency envelope. In curve fitting, the 0th order fitting coefficient represents the bias, which is insignificant for the present invention. The 5th order and above fitting coefficients have little influence on the curve, so they are used as the ordinary level; the 1st order and 2nd order fitting coefficients represent the slope and parabolic curvature respectively, which have a great influence on the curve, so they are used as the key level; the 3rd order and 4th order fitting coefficients play a regulating role on the curve and also have a great influence on the curve, so they are used as the secondary level. The present invention exponentially amplifies the key level difference, proportionally amplifies the secondary level difference, and conventionally seeks differences for the ordinary level, so as to achieve hierarchical comparison and efficiently and accurately quantify the standard degree of the user's initial consonant pronunciation.

[0162] The initial consonant judgment threshold is a threshold set for the non-standard value of the initial consonant of the word read aloud by the user. The present invention does not limit the value of the initial consonant judgment threshold, which is set by the user according to the correction requirements of the initial consonant syllable pronunciation.

[0163] S5. Prompt the user based on the non-standard syllables detected by S3 and S4, and broadcast the corresponding standard Mandarin syllable pronunciation and spelling method to the user to train and correct the user's Mandarin.

[0164] In summary, the present invention targets the user's pronunciation, performs frequency domain analysis, and based on the different characteristics of initial consonants and finals, accurately captures and meticulously identifies the user's specific syllables, effectively points out the user's pronunciation and spelling problems, and makes the user's Mandarin training and correction more efficient and accurate, which is of great significance to improving people's language ability and communication level.

[0165] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for training and correcting Mandarin in speech processing, characterized in that: The following steps are involved: S1, recording the user's voice when reading the training text, and obtaining the amplitude-frequency feature sequence of each word after segmentation, analog-to-digital conversion, energy normalization and frequency domain analysis; S2, analyzing the frequency of each amplitude peak in each read-aloud character sound amplitude-frequency feature sequence, constructing the corresponding read-aloud character sound vowel feature vector, and analyzing the envelope of each read-aloud character sound amplitude-frequency feature sequence, constructing the corresponding read-aloud character sound initial consonant feature vector; S3, comparing the frequency deviation and trend difference of the final feature vector of each pronounced word with the standard Mandarin pronunciation, and detecting the user's non-standard final syllables; S4, comparing the initial consonant feature vectors of each pronounced character with the standard Mandarin character pronunciation in a graded manner, and detecting the user's non-standard initial consonant syllables; S5. Prompt the user based on the non-standard syllables detected by S3 and S4, and broadcast the corresponding standard Mandarin syllable pronunciation and spelling method to the user to train and correct the user's Mandarin.

2. The method for correcting Mandarin speech processing according to claim 1, characterized in that: The S1 comprises the following sub-steps: S11, recording the user's voice when reading the training text at a uniform speed, obtaining a reading voice electric signal, and dividing it into a plurality of reading word sound electric signals according to the number of words in the training text, wherein the time length of each reading word sound electric signal is: ， Among them, τ is the time length of reading the electric signal of the word sound, T is the time length of reading the electric signal of the voice, and L is the number of words in the training text; S12, converting each read-aloud word sound electrical signal from an analog signal into a digital signal sequence, and performing energy normalization processing according to the following formula to obtain each read-aloud word sound time domain feature sequence: ， Among them, s i (k) is the value of the kth element of the time-domain feature sequence of the i-th pronunciation, is the value of the kth element of the digital signal sequence of the i-th spoken word before normalization, and M is the number of sampling points of the electrical signal of the spoken word during analog-to-digital conversion; S13, calculating the amplitude-frequency feature sequence corresponding to the time-domain feature sequence of each pronounced word.

3. The method for correcting Mandarin speech processing according to claim 2, characterized in that: The S13 comprises the following sub-steps: S131, according to the time domain feature sequence of each pronounced word, through fast Fourier transform FFT, calculate each pronounced word complex frequency domain sequence; S132, by the following formula, according to each read word sound complex frequency domain sequence, calculate each read word sound amplitude frequency sequence: ， in, is the amplitude value of the nth element of the amplitude frequency sequence of the i-th word read aloud, r i (n) is the real part of the nth element of the complex frequency domain sequence of the i-th pronounced word, j i (n) is the imaginary part of the nth element of the complex frequency domain sequence of the i-th pronounced word; S133, by using the following formula, a nonlinear transformation is performed on each read word sound amplitude frequency sequence to enhance recognition, and each read word sound amplitude frequency feature sequence is obtained: ， Among them, p i (n) is the amplitude value of the nth element of the i-th read word frequency characteristic sequence, γ is the gain coefficient, log 10 is the logarithmic function with base 10, b ias is the bias coefficient.

4. The method for mandarin training and correction of speech processing according to claim 1, characterized in that: In S2, the process of analyzing the frequencies of the amplitude peaks in each read-aloud character sound amplitude-frequency feature sequence and constructing the corresponding read-aloud character sound final feature vector includes the following steps: A1. Find the first N elements with the highest amplitude value from high to low in the first M / 2 elements in the amplitude-frequency feature sequence of the read-aloud sound, and record their sequence numbers in the amplitude-frequency feature sequence of the read-aloud sound, where M is the number of sampling points of the read-aloud sound electrical signal during analog-to-digital conversion, and N is a positive integer; A2, according to the sequence number of each element in the read word sound amplitude frequency feature sequence recorded in A1, the frequency of the corresponding element is calculated by the following formula: ， Among them, f x is the frequency of the xth highest amplitude element in the amplitude-frequency feature sequence of the read word, f s is the sampling frequency during analog-to-digital conversion, n x is the sequence number of the element with the xth highest amplitude value in the amplitude-frequency feature sequence of the read-aloud sound, where x is a positive integer less than or equal to N; A3. Concatenate the frequencies of the N elements in descending order of the corresponding amplitude values ​​to form a final feature vector.

5. The method for speech processing in Mandarin training and correction according to claim 1, characterized in that: In S2, the process of analyzing the envelope of each read-aloud sound amplitude-frequency feature sequence and constructing the corresponding read-aloud sound initial consonant feature vector is: The numbers of the elements in the amplitude-frequency feature sequence of the read-out characters are used as independent variables, and the amplitude value is used as the dependent variable. The least squares method is used to perform polynomial fitting to obtain the envelope function and fitting coefficients of each order expressed as follows. The fitting coefficients of each order are concatenated into the initial consonant feature vector, regardless of the order: ， in, is the value of the envelope function of the amplitude-frequency feature sequence of the read-out word when the independent variable is n, n is the element number of the amplitude-frequency feature sequence of the read-out word, α m is the mth order fitting coefficient, n m It is the operation of raising n to the power of m, α0 is the 0th order fitting coefficient, R is the maximum power of the polynomial fitting, and m is the order number of the fitting coefficient.

6. The method for speech processing in Mandarin training and correction according to claim 1, characterized in that: In S3, the method of comparing the frequency deviation and trend difference of the final feature vector of each pronounced pronunciation with the standard Mandarin pronunciation includes the following steps: B1. Perform vector subtraction on the final feature vectors of the user's pronunciation and the standard Mandarin pronunciation to obtain a frequency deviation vector; B2. Calculate the enhancement coefficient sequence according to the frequency offset vector; B3, processing the frequency offset vector by the enhancement coefficient sequence to obtain the frequency difference value; B4, according to the final feature vectors of the user's pronunciation and the standard Mandarin pronunciation, respectively calculate the trend vectors of the user's pronunciation and the standard Mandarin pronunciation; B5. Obtaining a trend difference value according to the trend vectors of the pronunciation of the word read aloud by the user and the pronunciation of the word in standard Mandarin; B6, performing weighted summation on the frequency difference value and the trend difference value to obtain the non-standard value of the finals of the word read aloud by the user; B7. If the non-standard value of the final of the word pronounced by the user is greater than the final decision threshold, it is determined that the pronunciation of the word pronounced by the user is not standard and the final is a problem syllable.

7. The method for speech processing and correction of Mandarin according to claim 6, characterized in that: The expression used in B2 is: ， Among them, ξ(x) is the value of the xth element of the enhancement coefficient sequence, is the value of the xth element of the frequency deviation vector, where x is a positive integer less than or equal to the length of the final feature vector; The expression used in B3 is: ， Among them, d istc,f is the frequency difference value, and |∙| is the absolute value operation.

8. The method for speech processing in Mandarin training and correction according to claim 7, characterized in that: The expression used in B4 is: ， ， Among them, κ(1) is the value of the first element of the trend vector, κ(x)| x>1 is the value of the xth element of the trend vector when x is greater than 1, f(x) is the value of the xth element of the final feature vector, and f(x-1) is the value of the x-1th element of the final feature vector; The expression used in B5 is: ， Among them, d istc,td is the trend gap value, κ user (x) is the xth element value of the pronunciation trend vector of the word read aloud by the user, κ stand (x) is the x-th element value of the standard Mandarin pronunciation trend vector, and |∙| is the absolute value operation.

9. The method for speech processing in Mandarin training and correction according to claim 5, characterized in that: In S4, the method of comparing the initial consonant feature vectors of each pronounced pronunciation with the standard Mandarin pronunciation in a graded manner includes the following steps: C1. The initial consonant feature vectors of the user's pronunciation and the standard Mandarin pronunciation are classified according to the following classification strategy, and the ordinary initial consonant feature vectors, the key initial consonant feature vectors and the secondary initial consonant feature vectors of the user's pronunciation and the standard Mandarin pronunciation are obtained respectively: The 0th order fitting coefficient and the 5th order fitting coefficients are combined from small to large order to form the common initial consonant feature vector; The first-order fitting coefficients and the second-order fitting coefficients are arranged in ascending order to form a key-level initial consonant feature vector; The third-order fitting coefficient and the fourth-order fitting coefficient are arranged in ascending order to form a secondary initial consonant feature vector; C2, according to the common initial consonant feature vector, the key initial consonant feature vector and the secondary initial consonant feature vector of the user's pronunciation and the standard Mandarin pronunciation, respectively calculate the user's pronunciation pronunciation initial consonant non-standard value third component, the user's pronunciation pronunciation initial consonant non-standard value first component and the user's pronunciation pronunciation initial consonant non-standard value second component; C3, adding the first to third components of the non-standard values ​​of the initial consonants of the characters read aloud by the user to obtain the non-standard values ​​of the initial consonants of the characters read aloud by the user; C4. If the non-standard value of the initial consonant of the word read aloud by the user is greater than the initial consonant judgment threshold, it is judged that the pronunciation of the word read aloud by the user is not standard and the initial consonant is a problem syllable.

10. The method for speech processing and correction of Mandarin according to claim 9, characterized in that: In C2, the third component of the non-standard value of the initial consonant of the user's pronunciation is calculated according to the common initial consonant feature vectors of the user's pronunciation and the standard Mandarin pronunciation through the following formula: ， Among them, ς3 is the third component of the non-standard value of the initial consonant of the word read aloud by the user, R is the maximum power of the polynomial fitting, and A 3,user (y3) is the y3th element value of the feature vector of the common initial consonant of the word read aloud by the user, A 3,stand (y3) is the y3th element value of the feature vector of the standard Mandarin initial consonant of the ordinary level, y3 is the sequence number, and |∙| is the absolute value operation; The first component of the non-standard value of the initial consonant of the user's pronunciation is calculated according to the key-level initial consonant feature vectors of the user's pronunciation and the standard Mandarin pronunciation through the following formula: ， Among them, ς1 is the first component of the non-standard value of the initial consonant of the word read aloud by the user, A 1,user (y1) is the y1th element value of the feature vector of the key initial consonant of the word read aloud by the user, A 1,stand (y1) is the y1th element value of the feature vector of the key-level initial consonant of the standard Mandarin pronunciation, y1 is the sequence number, and e is a natural constant; The second component of the non-standard value of the initial consonant of the user's pronunciation is calculated according to the secondary initial consonant feature vectors of the user's pronunciation and the standard Mandarin pronunciation through the following formula: ， Among them, ς2 is the second component of the non-standard value of the initial consonant of the word read aloud by the user, A 2,user (y2) is the y2th element value of the secondary initial consonant feature vector of the word read aloud by the user, A 2,stand (y2) is the y2th element value of the feature vector of the secondary initial consonant of the standard Mandarin pronunciation, and y2 is the sequence number.

Images