Pronunciation correction method, device and electronic equipment

Abstract

The present invention relates to the field of audio processing technology, and specifically to a pronunciation correction method, device and electronic device, the method comprising obtaining the GOP score value of each phoneme in the audio to be tested and obtaining a division threshold; comparing the GOP score value of each phoneme with the division threshold to determine the pronunciation interval to which each phoneme belongs; converting the GOP score value of each phoneme into a pronunciation similarity according to the pronunciation interval to which each phoneme belongs, the pronunciation similarity being used to indicate the degree of similarity between the phoneme and a standard phoneme; and correcting the pronunciation of the phoneme based on the pronunciation similarity. Converting the GOP score value in different pronunciation intervals can ensure the accuracy of the converted pronunciation similarity, and then using the pronunciation similarity to correct the pronunciation of the phoneme, thereby achieving targeted correction of the pronunciation of the phoneme and improving the efficiency of pronunciation correction.

Description

Technical Field

[0001] The present invention relates to the technical field of audio processing, and in particular to a pronunciation correction method, device and electronic equipment. Background Art

[0002] Before the popularization of computer technology, oral assessment was basically done manually. Due to the limitations of manual assessment, professionals can only give a broad evaluation of the results of oral assessment, and learners cannot know their specific pronunciation problems through such evaluation.

[0003] With the development of computer technology, it has become possible to use tools to replace manual methods for oral evaluation. Among them, the mainstream evaluation method is based on the GOP (Goodness of pronunciation) algorithm or its variants. This method models pronunciation based on speech recognition technology, and then scores the pronunciation of each phoneme. The general phoneme scoring method is: set a GOP score threshold, use the GOP algorithm to calculate the GOP score of the target phoneme, and compare the size relationship between the GOP score and the threshold to determine whether the user's pronunciation is accurate.

[0004] However, the above evaluation method based on the GOP algorithm can only give whether the pronunciation of the tested phoneme is correct or not, but it is difficult to characterize the similarity between the tested phoneme and the standard phoneme. Therefore, it is difficult to perform pronunciation correction in a targeted manner, resulting in low efficiency of pronunciation correction. Summary of the invention

[0005] In view of this, the embodiments of the present invention provide a pronunciation correction method, device and electronic device to solve the problem of low efficiency of pronunciation correction.

[0006] According to a first aspect, an embodiment of the present invention provides a pronunciation correction method, comprising:

[0007] Obtain the GOP score value of each phoneme in the audio to be tested and obtain the division threshold;

[0008] Compare the GOP score of each phoneme with the division threshold to determine the pronunciation interval to which each phoneme belongs;

[0009] According to the pronunciation interval to which each phoneme belongs, the GOP score value of each phoneme is converted into a pronunciation similarity, wherein the pronunciation similarity is used to indicate the similarity between the phoneme and a standard phoneme;

[0010] The pronunciation of the phoneme is corrected based on the pronunciation similarity.

[0011] The pronunciation correction method provided by the embodiment of the present invention determines the pronunciation interval to which each phoneme belongs by comparing the GOP score value of each phoneme with the division threshold, and then converts the GOP score value into the pronunciation similarity within the pronunciation interval; that is, the GOP score value is converted in different pronunciation intervals to ensure the accuracy of the converted pronunciation similarity, and the pronunciation of the phoneme is corrected by using the pronunciation similarity, so that the pronunciation of the phoneme can be corrected in a targeted manner, thereby improving the efficiency of pronunciation correction.

[0012] In combination with the first aspect, in a first implementation of the first aspect, obtaining the segmentation threshold includes:

[0013] Acquire a pronunciation corpus, wherein the pronunciation corpus includes standard pronunciation audio and corresponding annotated text;

[0014] Calculating a first GOP score value of each phoneme in the pronunciation corpus using the standard pronunciation audio and the corresponding annotated text;

[0015] Obtaining a preset text, and replacing the marked text with the preset text;

[0016] Calculate the second GOP score of each phoneme by using the standard pronunciation audio and the preset text;

[0017] The split threshold is determined based on the statistical value of the first GOP score value and the statistical value of the second GOP score value.

[0018] The pronunciation correction method provided by the embodiment of the present invention uses the standard pronunciation audio and the annotated text to calculate the first GOP score value, which belongs to the category of accurate pronunciation, and uses the preset text to replace the annotated text to calculate the second GOP score value, which belongs to the category of inaccurate pronunciation. The statistical values ​​of the two are then used to determine the division threshold, thereby ensuring the accuracy of the determination of the division threshold.

[0019] In combination with the first implementation of the first aspect, in the second implementation of the first aspect, the segmentation threshold includes a first segmentation threshold and a second segmentation threshold, and determining the segmentation threshold based on the statistical value of the first GOP score value and the statistical value of the second GOP score value includes:

[0020] Using the first GOP score value of each phoneme, calculate the first GOP score mean and the first GOP score mean square error;

[0021] Performing linear processing on the first GOP score mean and the second GOP score mean square error to obtain the first division threshold;

[0022] Using the second GOP score values ​​of each phoneme, calculating the second GOP score mean and the second GOP score mean square error;

[0023] Linear processing is performed on the second GOP score mean and the second GOP score mean square error to obtain the second division threshold.

[0024] The pronunciation correction method provided by the embodiment of the present invention uses the score mean and the score mean square error to perform linear calculation to obtain the division threshold, which can obtain the division threshold under the premise of simplifying the calculation, reducing the amount of data processing and improving the efficiency of pronunciation correction.

[0025] In combination with the second implementation of the first aspect, in the third implementation of the first aspect, the first segmentation threshold and the second segmentation threshold are calculated by the following formula:

[0026] m=μ1+α1σ1+β1,

[0027] n=μ2+α2σ2+β2,

[0028] Among them, m is the first division threshold, μ1 is the first GOP score mean, σ1 is the first score mean square error, n is the second division threshold, μ2 is the second GOP score mean, σ2 is the second score mean square error, α1, β1, α2 and β2 are all preset empirical constants.

[0029] In combination with the first aspect, or any one of the first to third implementations of the first aspect, in a fourth implementation of the first aspect, the division threshold includes a first division threshold and a second division threshold, and the converting the GOP score value of each phoneme into a pronunciation similarity according to the pronunciation interval to which each phoneme belongs includes:

[0030] When the GOP score value of the phoneme is greater than the first division threshold and less than or equal to zero, performing cosine calculation using the GOP score value of the phoneme to determine the pronunciation similarity of the phoneme;

[0031] When the GOP score value of the phoneme is less than the first division threshold and greater than or equal to the second division threshold, performing linear calculation using the GOP score value of the phoneme to determine the pronunciation similarity of the phoneme;

[0032] When the GOP score value of the phoneme is less than the second division threshold, the GOP score value of the phoneme is used to perform index calculation to determine the pronunciation similarity of the phoneme.

[0033] The pronunciation correction method provided in the embodiment of the present invention uses the cosine function to reduce the disturbance of small-range changes in GOP scores on phoneme score evaluation, thereby avoiding the phenomenon that the phoneme evaluation score changes greatly and is distorted when the pronunciation to be tested is close to the standard pronunciation; at the same time, such a design can also avoid the need to design mapping functions separately for different phonemes, thereby improving the simplicity of pronunciation correction.

[0034] In combination with the fourth implementation of the first aspect, in the fifth implementation of the first aspect, the pronunciation similarity is calculated using the following formula:

[0035]

[0036] Among them, p is the phoneme, m is the first division threshold, n is the second division threshold, and a, b, c, d and e are all constants.

[0037] In combination with the first aspect, in a sixth implementation of the first aspect, the correcting the pronunciation of the phoneme based on the pronunciation similarity includes:

[0038] A correction measure is determined based on the pronunciation similarity to correct the pronunciation of the phoneme.

[0039] The pronunciation correction method provided by the embodiment of the present invention provides different correction measures for different pronunciation similarities, thereby achieving targeted correction of pronunciation and ensuring the efficiency of pronunciation correction.

[0040] According to a second aspect, an embodiment of the present invention further provides a pronunciation correction device, comprising:

[0041] A threshold division module is used to obtain the GOP score value of each phoneme in the audio to be tested and obtain the division threshold;

[0042] An interval matching module, used for comparing the GOP score value of each phoneme with the division threshold to determine the pronunciation interval to which each phoneme belongs;

[0043] A score conversion module, used for converting the GOP score value of each phoneme into a pronunciation similarity according to the pronunciation interval to which each phoneme belongs, wherein the pronunciation similarity is used to indicate the similarity between the phoneme and a standard phoneme;

[0044] The pronunciation correction module is used to correct the pronunciation of the phoneme based on the pronunciation similarity.

[0045] The pronunciation correction device provided by the embodiment of the present invention determines the pronunciation interval to which each phoneme belongs by comparing the GOP score value of each phoneme with the division threshold, and then converts the GOP score value into the pronunciation similarity within the pronunciation interval; that is, the GOP score value is converted in different pronunciation intervals to ensure the accuracy of the converted pronunciation similarity, and the pronunciation of the phoneme is corrected by using the pronunciation similarity, so that the pronunciation of the phoneme can be corrected in a targeted manner, thereby improving the efficiency of pronunciation correction.

[0046] According to the third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the pronunciation correction method described in the first aspect or any one of the embodiments of the first aspect by executing the computer instructions.

[0047] According to a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the pronunciation correction method described in the first aspect or any one of the embodiments of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 is a flow chart of a pronunciation correction method according to an embodiment of the present invention;

[0050] Figure 2 is a flow chart of a pronunciation correction method according to an embodiment of the present invention;

[0051] Figure 3 is a flow chart of a pronunciation correction method according to an embodiment of the present invention;

[0052] Figure 4 is a structural block diagram of a pronunciation correction device according to an embodiment of the present invention;

[0053] Figure 5 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0055] According to an embodiment of the present invention, an embodiment of a pronunciation correction method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0056] In this embodiment, a pronunciation correction method is provided, which can be used in electronic devices such as computers, mobile phones, tablet computers, etc. Figure 1 is a flow chart of a method for correcting pronunciation according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0057] S11, obtaining the GOP score value of each phoneme in the audio to be tested and obtaining the division threshold.

[0058] The GOP score value of each phoneme in the audio to be tested may be calculated by the electronic device, or may be acquired by the electronic device from the outside, and no limitation is imposed on it herein.

[0059] Here, taking the calculation of the GOP score of each phoneme by an electronic device as an example, the electronic device obtains the audio to be tested, first extracts the acoustic features of the audio to be tested, and then calculates the GOP score of each phoneme based on the extracted acoustic features.

[0060] Specifically, the electronic device can first perform frame windowing, pre-emphasis and other operations on the audio to be tested, and then obtain the acoustic features MFCC (Mel Spectrum Cepstrum) and iverse (speaker features) of the audio to be tested through corresponding calculation formulas.

[0061] The electronic device can calculate the log-likelihood value of each phoneme through the acoustic features obtained in the previous step, and then obtain the mapping relationship of word-phoneme alignment through the forced alignment module of the acoustic model. Furthermore, based on the log-likelihood value of the phoneme and the word-phoneme alignment relationship of the audio frame, the GOP score of each phoneme in the audio to be tested can be obtained through the GOP calculation formula.

[0062] The GOP calculation formula is as follows:

[0063]

[0064] Where LLP represents the logarithmic posterior probability of the phoneme, p represents the phoneme to be tested, and q represents the standard phoneme.

[0065] It should be noted that the above process is only an example of calculating the GOP score value, and other methods can also be used to calculate the GOP score value, which is not limited here.

[0066] The division threshold can be artificially given or obtained through statistical analysis of historical data. The division threshold is used to divide the GOP score value into results. For example, by setting two division thresholds, three intervals can be obtained, namely, standard pronunciation, irregular pronunciation, and non-standard pronunciation; or, multiple division thresholds are set. Among them, the more division thresholds are set, the finer the division of the GOP score value, the more accurate the similarity obtained, and the finer the division, the larger the corresponding data processing amount. Therefore, the actual division process needs to be combined with the actual situation.

[0067] This step will be described in detail below.

[0068] S12, comparing the GOP score of each phoneme with the division threshold, and determining the pronunciation interval to which each phoneme belongs.

[0069] After the electronic device obtains the GOP score of each phoneme in S11, it compares the GOP score with the division threshold to determine the pronunciation interval to which each phoneme belongs. The pronunciation interval is determined in order to convert the GOP score into similarity later, and different pronunciation intervals use different conversion methods.

[0070] S13, converting the GOP score of each phoneme into pronunciation similarity according to the pronunciation interval to which each phoneme belongs.

[0071] The pronunciation similarity is used to indicate the similarity between a phoneme and a standard phoneme.

[0072] As mentioned above, the conversion method used by the electronic device is different for different pronunciation intervals. For example, when close to the standard pronunciation, the conversion method used should try to avoid distortion and avoid large changes in the converted similarity; when far from the standard pronunciation, the conversion method used can be to make the similarity change larger to better characterize the pronunciation error. For example, for phonemes close to the standard pronunciation, a function with a smaller change rate can be used for conversion; for phonemes far from the standard pronunciation, a function with a larger change rate can be used for conversion.

[0073] The electronic device may convert the GOP score value into pronunciation similarity after determining the pronunciation intervals of all phonemes in the audio to be tested; or may convert the pronunciation similarity after determining the pronunciation intervals of one or more phonemes without waiting until the pronunciation intervals of all phonemes are determined.

[0074] This step will be described in detail below.

[0075] S14, correcting the pronunciation of the phonemes based on the pronunciation similarity.

[0076] After determining the similarity of each phoneme, the electronic device can provide a corrective measure for each phoneme, and the corrective measure can be pre-stored in the electronic device. The electronic device uses the calculated pronunciation similarity to search in the corrective measure library to determine the corrective measure corresponding to the similarity, etc.

[0077] Different correction measures are given according to different pronunciation similarities, so as to correct the pronunciation in a targeted manner and ensure the efficiency of pronunciation correction.

[0078] The pronunciation correction method provided in this embodiment determines the pronunciation interval to which each phoneme belongs by comparing the GOP score value of each phoneme with the division threshold, and then converts the GOP score value into pronunciation similarity within the pronunciation interval; that is, the GOP score value is converted in different pronunciation intervals to ensure the accuracy of the converted pronunciation similarity, and the pronunciation of the phoneme is corrected by using the pronunciation similarity, so that the pronunciation of the phoneme can be corrected in a targeted manner, thereby improving the efficiency of pronunciation correction.

[0079] In this embodiment, a pronunciation correction method is provided, which can be used in electronic devices such as computers, mobile phones, tablet computers, etc. Figure 2 is a flow chart of a method for correcting pronunciation according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0080] S21, obtaining the GOP score value of each phoneme in the audio to be tested and obtaining the division threshold.

[0081] Specifically, the above S21 may include:

[0082] S211, obtaining the GOP score value of each phoneme in the audio to be tested.

[0083] For more information on how to obtain the GOP score, see Figure 1 S11 of the illustrated embodiment will not be described in detail here.

[0084] Further, the following S212-S216 are used to describe the process of obtaining the partition threshold. In this embodiment, the partition threshold is obtained by performing statistical analysis on data by the electronic device as an example for description.

[0085] S212, obtaining a pronunciation corpus.

[0086] The pronunciation corpus includes standard pronunciation audio and corresponding annotated text.

[0087] The pronunciation corpus contains several standard pronunciation audios and the annotated texts corresponding to the standard pronunciation audios. The annotated texts are not limited in any way and can be set accordingly according to the actual situation.

[0088] S213, using the standard pronunciation audio and the corresponding annotated text, calculate the first GOP score value of each phoneme in the pronunciation corpus.

[0089] The electronic device can calculate the first GOP score for each phoneme in the pronunciation corpus using the GOP score calculation method described in S11. The first GOP score calculated here can be considered as the GOP score of the standard phoneme.

[0090] S214, obtaining a preset text, and replacing the marked text with the preset text.

[0091] The annotated text mentioned above corresponds one-to-one to the standard pronunciation audio. The preset text obtained here should contain all the phonemes in the phoneme set of the measured language. It can be a human-given text spliced ​​from the phonemes in the phoneme set, or it can be a text randomly generated by an electronic device. There is no corresponding relationship between this text and the standard pronunciation audio.

[0092] After acquiring the preset text, the electronic device replaces the annotated text corresponding to the standard pronunciation audio with the preset text, that is, establishes a corresponding relationship between the standard pronunciation audio and the preset text.

[0093] S215, calculating the second GOP score value of each phoneme using the standard pronunciation audio and the preset text.

[0094] The electronic device can also use the GOP score calculation method described in S11 to calculate the second GOP score of each phoneme for the standard pronunciation audio and the preset text. The second GOP score calculated here can be considered as the GOP score of the non-standard phoneme.

[0095] Through the above processing, the electronic device can obtain a first GOP score value belonging to a standard phoneme and a second GOP score value where a non-standard phoneme may exist. Subsequently, by performing statistical analysis on these two GOP score values, the entire interval can be divided.

[0096] S216: Determine a partition threshold based on the statistical value of the first GOP score value and the statistical value of the second GOP score value.

[0097] Specifically, the GOP score interval of the phoneme is (-∞,0], and the value of the GOP score reflects the similarity between the phoneme to be tested and the standard phoneme to a certain extent. The closer the GOP score is to 0, the closer the pronunciation of the phoneme to be tested is to the standard phoneme. Therefore, the electronic device can perform statistical analysis on the first GOP score value of each phoneme and the second GOP score value of each phoneme to determine the threshold of the standard phoneme and the non-standard phoneme. That is, two division thresholds are obtained, namely the first division threshold m and the second division threshold n.

[0098] Two thresholds are divided on the GOP score interval (-∞,0]. The one close to 0 is called the first threshold m, and the one close to -∞ is called the second threshold n. The interval (m,0] has a small range, and the statistical mean of the standard pronunciation GOP scores of different phonemes fluctuates to a certain extent, but the fluctuation range is small and basically concentrated in this interval; the interval (-∞,n] has a large range, and the incorrect pronunciations of different phonemes are basically concentrated in this interval; the remaining (n,m] interval has a limited range, and some irregular pronunciations are basically concentrated in this area, and the GOP scores of phonemes with different pronunciation qualities show a certain linear law.

[0099] In some optional implementations of this embodiment, the above S216 may include:

[0100] (1) Using the first GOP score value of each phoneme, calculate the first GOP score mean and the first GOP score mean square error.

[0101] (2) Linearly process the mean value of the first GOP score and the mean square error of the second GOP score to obtain a first division threshold.

[0102] Specifically, the first division threshold m can be expressed by the following formula:

[0103] m=μ1+α1σ1+β1,

[0104] Wherein, m is the first division threshold, μ1 is the first GOP score mean, σ1 is the first score mean square error, α1 and β1 are preset empirical constants. Optionally, α1=-1, β1=0.

[0105] (3) Using the second GOP score value of each phoneme, calculate the second GOP score mean and the second GOP score mean square error.

[0106] (4) Perform linear processing on the second GOP score mean and the second GOP score mean square error to obtain a second division threshold.

[0107] Specifically, the second division threshold n can be expressed by the following formula:

[0108] n=μ2+α2σ2+β2,

[0109] Wherein, n is the second division threshold, μ2 is the second GOP score mean, σ2 is the second score mean square error, α2 and β2 are both preset empirical constants. Optionally, α2=-1, β2=0.

[0110] The division threshold is obtained by linear calculation using the score mean and the score mean square error. The division threshold can be obtained under the premise of simplifying the calculation, reducing the amount of data processing and improving the efficiency of pronunciation correction.

[0111] S22, comparing the GOP score of each phoneme with the division threshold, and determining the pronunciation interval to which each phoneme belongs.

[0112] After calculating the first and second division thresholds, the electronic device divides the entire interval into three intervals. The electronic device then compares the GOP score of each phoneme with the first and second division thresholds to determine the interval to which the GOP score of each phoneme belongs.

[0113] S23, converting the GOP score of each phoneme into pronunciation similarity according to the pronunciation interval to which each phoneme belongs.

[0114] The pronunciation similarity is used to indicate the similarity between a phoneme and a standard phoneme.

[0115] As described above, after the electronic device calculates the first division threshold and the second division threshold, it is equivalent to dividing the entire interval into three intervals. Accordingly, a three-segment piecewise function can be used to act on these three intervals respectively, and the GOP scores of the phonemes belonging to each interval are mapped to the phoneme score space, and the corresponding pronunciation similarity can be calculated.

[0116] For details, please see Figure 1 S13 of the illustrated embodiment will not be described in detail here.

[0117] S24, correcting the pronunciation of the phonemes based on the pronunciation similarity.

[0118] For details, please see Figure 1S14 of the illustrated embodiment will not be described in detail here.

[0119] In the pronunciation correction method provided in this embodiment, the first GOP score value calculated by using the standard pronunciation audio and the annotated text belongs to the category of accurate pronunciation, and the second GOP score value calculated by replacing the annotated text with the preset text belongs to the category of inaccurate pronunciation. The statistical values ​​of the two are then used to determine the division threshold, thereby ensuring the accuracy of the determination of the division threshold.

[0120] In this embodiment, a pronunciation correction method is provided, which can be used in electronic devices such as computers, mobile phones, tablet computers, etc. Figure 3 is a flow chart of a method for correcting pronunciation according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0121] S31, obtaining the GOP score value of each phoneme in the audio to be tested and obtaining the division threshold.

[0122] For details, please see Figure 2 S21 of the illustrated embodiment will not be described in detail here.

[0123] S32, comparing the GOP score of each phoneme with the division threshold, and determining the pronunciation interval to which each phoneme belongs.

[0124] For details, please see Figure 2 S22 of the illustrated embodiment will not be described in detail here.

[0125] S33, converting the GOP score of each phoneme into pronunciation similarity according to the pronunciation interval to which each phoneme belongs.

[0126] The pronunciation similarity is used to indicate the similarity between a phoneme and a standard phoneme.

[0127] Specifically, the above S33 may include:

[0128] S331: When the GOP score of the phoneme is greater than the first division threshold and less than or equal to zero, a cosine calculation is performed using the GOP score of the phoneme to determine the pronunciation similarity of the phoneme.

[0129] The cosine calculation may be performed by taking the GOP score value as a parameter of the cosine function to obtain the result of the cosine calculation. The result of the cosine calculation may be directly determined as the pronunciation similarity of the phoneme; other parameters may also be combined for calculation on this basis, and so on.

[0130] S332: When the GOP score of the phoneme is less than the first division threshold and greater than or equal to the second division threshold, a linear calculation is performed using the GOP score of the phoneme to determine the pronunciation similarity of the phoneme.

[0131] The linear calculation may be to use the GOP score value as a variable of the linear function to directly calculate the corresponding pronunciation similarity.

[0132] S333: When the GOP score of the phoneme is less than the second division threshold, the GOP score of the phoneme is used to perform an index calculation to determine the pronunciation similarity of the phoneme.

[0133] The index calculation may be to use the GOP score value as a variable of the index function to calculate the corresponding Sanskrit similarity.

[0134] Specifically, the pronunciation similarity is calculated using the following formula:

[0135]

[0136] Among them, p is the phoneme, m is the first division threshold, n is the second division threshold, and a, b, c, d and e are all constants.

[0137] Optionally, specific values ​​of a, b, c, d and e are obtained by calculation.

[0138] In some optional implementations of this embodiment, the specific values ​​of a, b, c, d and e can be calculated in the following way.

[0139] The first dividing threshold m is used to divide the GOP score interval of phonemes with standard pronunciation and accented pronunciation, and the second dividing threshold n is used to divide the GOP score interval of phonemes with accented pronunciation and incorrect pronunciation; GOP(p) is the GOP score of the phoneme p to be tested obtained by the GOP formula.

[0140] In actual application scenarios, the range of phoneme scores and the criteria for judging phoneme scores need to be determined in advance. For example, the phoneme scores are mapped to the probability space of [0, 1.0]. According to people's daily cognitive habits, the phonemes with phoneme scores below 0.5 are set as incorrect pronunciations; the phonemes with phoneme scores above 0.9 are set as standard pronunciations, and the phonemes with phoneme scores in the interval (0.5, 0.9) are set as accented pronunciations. In this way, the phoneme scores converted by the formula can more intuitively reflect the quality of the phoneme pronunciation. Let GOP(p) be 0, m and n respectively, and substitute them into the above formula Score(p):

[0141]

[0142] The above formula can be used to calculate the values ​​of a, b, c, d, and e; the obtained a, b, c, d, e, m, and n can be substituted into Score(p) for score conversion.

[0143] S34, correcting the pronunciation of the phonemes based on the pronunciation similarity.

[0144] For details, please see Figure 2 S24 of the illustrated embodiment will not be described in detail here.

[0145] The pronunciation correction method provided in this embodiment uses the cosine function to reduce the disturbance of small-range changes in GOP scores on phoneme score evaluation, thereby avoiding the phenomenon that the phoneme evaluation score changes greatly and is distorted when the pronunciation to be tested is close to the standard pronunciation; at the same time, such a design can also avoid the need to design mapping functions separately for different phonemes, thereby improving the simplicity of pronunciation correction.

[0146] In the present embodiment, a pronunciation correction device is also provided, and the device is used to implement the above-mentioned embodiment and the preferred embodiment, and the description thereof will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented with software, the implementation of hardware, or a combination of software and hardware is also possible and contemplated.

[0147] This embodiment provides a pronunciation correction device, such as Figure 4 As shown, including:

[0148] A threshold division module 41 is used to obtain the GOP score value of each phoneme in the audio to be tested and obtain a division threshold;

[0149] An interval matching module 42, used to compare the GOP score value of each phoneme with the division threshold to determine the pronunciation interval to which each phoneme belongs;

[0150] A score conversion module 43, used for converting the GOP score value of each phoneme into a pronunciation similarity according to the pronunciation interval to which each phoneme belongs, wherein the pronunciation similarity is used to indicate the similarity between the phoneme and the standard phoneme;

[0151] The pronunciation correction module 44 is used to correct the pronunciation of the phoneme based on the pronunciation similarity.

[0152] The pronunciation correction device provided in this embodiment determines the pronunciation interval to which each phoneme belongs by comparing the GOP score value of each phoneme with the division threshold, and then converts the GOP score value into pronunciation similarity within the pronunciation interval; that is, the GOP score value is converted in different pronunciation intervals to ensure the accuracy of the converted pronunciation similarity, and the pronunciation of the phoneme is corrected by using the pronunciation similarity, so that the pronunciation of the phoneme can be corrected in a targeted manner, thereby improving the efficiency of pronunciation correction.

[0153] The pronunciation correction device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and a memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0154] The further functional description of each of the above modules is the same as that of the above corresponding embodiments and will not be repeated here.

[0155] The embodiment of the present invention also provides an electronic device having the above Figure 4 The pronunciation correction device shown.

[0156] See also Figure 5 , Figure 5 is a schematic diagram of the structure of an electronic device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the electronic device may include: at least one processor 51, such as a CPU (Central Processing Unit), at least one communication interface 53, a memory 54, and at least one communication bus 52. The communication bus 52 is used to realize the connection and communication between these components. The communication interface 53 may include a display screen (Display) and a keyboard (Keyboard), and the optional communication interface 53 may also include a standard wired interface and a wireless interface. The memory 54 may be a high-speed RAM memory (Random Access Memory) or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 54 may optionally be at least one storage device located away from the aforementioned processor 51. The processor 51 may be combined with Figure 4 In the described device, the memory 54 stores an application program, and the processor 51 calls the program code stored in the memory 54 to execute any of the above method steps.

[0157] The communication bus 52 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The communication bus 52 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0158] Among them, the memory 54 may include a volatile memory (English: volatile memory), such as a random access memory (English: random-access memory, abbreviated: RAM); the memory may also include a non-volatile memory (English: non-volatile memory), such as a flash memory (English: flash memory), a hard disk drive (English: hard disk drive, abbreviated: HDD) or a solid-state drive (English: solid-state drive, abbreviated: SSD); the memory 54 may also include a combination of the above types of memory.

[0159] The processor 51 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and a NP.

[0160] The processor 51 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0161] Optionally, the memory 54 is also used to store program instructions. The processor 51 can call the program instructions to implement the present application Figures 1 to 3 The pronunciation correction method shown in the embodiment.

[0162] The embodiment of the present invention also provides a non-transitory computer storage medium, which stores computer executable instructions, and the computer executable instructions can execute the pronunciation correction method in any of the above method embodiments. Wherein, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.

[0163] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A pronunciation correction method, characterized in that: include: Obtaining a GOP score value of each phoneme in the audio to be tested and obtaining a division threshold, wherein the division threshold includes a first division threshold and a second division threshold; Compare the GOP score of each phoneme with the division threshold to determine the pronunciation interval to which each phoneme belongs; According to the pronunciation interval to which each phoneme belongs, the GOP score value of each phoneme is converted into a pronunciation similarity, wherein the pronunciation similarity is used to indicate the similarity between the phoneme and a standard phoneme; Correcting the pronunciation of the phoneme based on the pronunciation similarity; The converting of the GOP score of each phoneme into pronunciation similarity according to the pronunciation interval to which each phoneme belongs includes: When the GOP score value of the phoneme is greater than the first division threshold and less than or equal to zero, performing cosine calculation using the GOP score value of the phoneme to determine the pronunciation similarity of the phoneme; When the GOP score value of the phoneme is less than the first division threshold and greater than or equal to the second division threshold, performing linear calculation using the GOP score value of the phoneme to determine the pronunciation similarity of the phoneme; When the GOP score value of the phoneme is less than the second division threshold, the GOP score value of the phoneme is used to perform index calculation to determine the pronunciation similarity of the phoneme.

2. The method according to claim 1, characterized in that The obtaining of the partition threshold comprises: Acquire a pronunciation corpus, wherein the pronunciation corpus includes standard pronunciation audio and corresponding annotated text; Calculating a first GOP score value of each phoneme in the pronunciation corpus using the standard pronunciation audio and the corresponding annotated text; Obtaining a preset text, and replacing the marked text with the preset text; Calculate the second GOP score of each phoneme by using the standard pronunciation audio and the preset text; The split threshold is determined based on the statistical value of the first GOP score value and the statistical value of the second GOP score value.

3. The method according to claim 2, characterized in that The partition threshold includes a first partition threshold and a second partition threshold, and the determining the partition threshold based on the statistical value of the first GOP score value and the statistical value of the second GOP score value includes: Using the first GOP score value of each phoneme, calculate the first GOP score mean and the first GOP score mean square error; Performing linear processing on the first GOP score mean and the second GOP score mean square error to obtain the first division threshold; Using the second GOP score values ​​of each phoneme, calculating the second GOP score mean and the second GOP score mean square error; Linear processing is performed on the second GOP score mean and the second GOP score mean square error to obtain the second division threshold.

4. The method according to claim 3, characterized in that The first division threshold and the second division threshold are calculated using the following formula: m=μ1+α1σ1+β1 n=μ2+α2σ2+β2 Among them, m is the first division threshold, μ1 is the first GOP score mean, σ1 is the first score mean square error, n is the second division threshold, μ2 is the second GOP score mean, σ2 is the second score mean square error, α1, β1, α2 and β2 are all preset empirical constants.

5. The method according to claim 1, characterized in that The pronunciation similarity is calculated using the following formula: Among them, p is the phoneme, m is the first division threshold, n is the second division threshold, and a, b, c, d and e are all constants.

6. The method according to claim 1, characterized in that The correcting the pronunciation of the phoneme based on the pronunciation similarity comprises: A correction measure is determined based on the pronunciation similarity to correct the pronunciation of the phoneme.

7. A pronunciation correction device, characterized in that: include: A threshold division module, used to obtain the GOP score value of each phoneme in the audio to be tested and obtain a division threshold, wherein the division threshold includes a first division threshold and a second division threshold; An interval matching module, used for comparing the GOP score value of each phoneme with the division threshold to determine the pronunciation interval to which each phoneme belongs; A score conversion module, used for converting the GOP score value of each phoneme into a pronunciation similarity according to the pronunciation interval to which each phoneme belongs, wherein the pronunciation similarity is used to indicate the similarity between the phoneme and a standard phoneme; A pronunciation correction module, used for correcting the pronunciation of the phoneme based on the pronunciation similarity; The score conversion module is used to convert the GOP score value of each phoneme into pronunciation similarity according to the pronunciation interval to which each phoneme belongs, including: When the GOP score value of the phoneme is greater than the first division threshold and less than or equal to zero, performing cosine calculation using the GOP score value of the phoneme to determine the pronunciation similarity of the phoneme; When the GOP score value of the phoneme is less than the first division threshold and greater than or equal to the second division threshold, performing linear calculation using the GOP score value of the phoneme to determine the pronunciation similarity of the phoneme; When the GOP score value of the phoneme is less than the second division threshold, the GOP score value of the phoneme is used to perform index calculation to determine the pronunciation similarity of the phoneme.

8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the pronunciation correction method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the pronunciation correction method according to any one of claims 1 to 6.

Images