Synthetic speech quality evaluation method and device, electronic equipment and storage medium

Abstract

The invention provides a synthetic speech quality evaluation method and device, electronic equipment and a storage medium. The synthetic speech quality evaluation method comprises the following steps: acquiring at least one synthetic speech and respective original text; for each of the at least one synthetic speech, determining a phoneme sequence according to an original text corresponding to the synthetic speech; performing forced alignment on the acoustic features of the synthetic speech and the text features of the phoneme sequence to determine the starting and ending moments of each phoneme of the phoneme sequence; determining a first evaluation score of the synthetic speech according to the starting and ending moments of each phoneme; and evaluating the quality of the synthetic speech according to the first evaluation score. According to the scheme, the quality of the synthesized speech can be evaluated more accurately and objectively.

Description

Technical Field

[0001] The present invention relates to the field of speech signal processing technology, and in particular to a method for evaluating the quality of synthesized speech, a device for evaluating the quality of synthesized speech, an electronic device, a storage medium and a computer program product. Background Art

[0002] With the rapid development of speech synthesis models, the quality evaluation and ranking of synthesized speech have become increasingly important. However, the quality of synthesized speech generated by current speech synthesis models is still unstable, and the quality of synthesized speech affects the effectiveness of its use.

[0003] When evaluating the quality of synthesized speech, it is usually done by humans subjectively. The result of quality evaluation depends on the subjective consciousness of the evaluator, which is easily affected by various factors and cannot stably and accurately evaluate the quality of synthesized speech. As a result, users cannot accurately determine the quality of synthesized speech, which will affect the user experience when using poor quality synthesized speech. Summary of the invention

[0004] The present invention has been made in view of the above-mentioned problems.

[0005] According to a first aspect of the present invention, a method for evaluating the quality of synthesized speech is provided. The method comprises: obtaining at least one synthesized speech and respective original texts; for each of the at least one synthesized speech, determining a phoneme sequence according to the original text corresponding to the synthesized speech; forcibly aligning the acoustic features of the synthesized speech with the text features of the phoneme sequence to determine the start and end time of each phoneme in the phoneme sequence; determining a first evaluation score of the synthesized speech according to the start and end time of each phoneme; and evaluating the quality of the synthesized speech according to the first evaluation score.

[0006] Exemplarily, determining a first evaluation score of the synthesized speech based on the start and end times of each phoneme includes: determining the duration of each phoneme and the duration of each silent segment in the synthesized speech based on the start and end times of each phoneme; and calculating the first evaluation score based on the duration of each phoneme and the duration of each silent segment.

[0007] Exemplarily, determining the first evaluation score based on the duration of each phoneme and the duration of each silence segment includes: determining a first deduction result based on the number of phonemes whose duration exceeds a first duration threshold and a first weight coefficient; determining a second deduction result based on the number of silence segments whose duration exceeds a second duration threshold and a second weight coefficient; and calculating the first evaluation score based on the first deduction result and the second deduction result.

[0008] Exemplarily, determining the first evaluation score according to the duration of each phoneme and the duration of each silence segment further includes: determining the first duration threshold according to the average duration of each phoneme.

[0009] Exemplarily, evaluating the quality of the synthesized speech based on the first evaluation score includes: performing speech recognition on the synthesized speech to obtain a recognized text corresponding to the synthesized speech; comparing the original text and the recognized text to determine missing characters, wrong characters, and extra characters in the recognized text relative to the original text; calculating a second evaluation score based on the missing characters, wrong characters, and extra characters; and evaluating the quality of the synthesized speech based on the first evaluation score and the second evaluation score.

[0010] Exemplarily, the calculating the second evaluation score based on the missing characters, the wrong characters and the multiple characters includes: determining a third deduction result based on the number of the missing characters and a third weight coefficient; determining a fourth deduction result based on the number of the wrong characters and a fourth weight coefficient; determining a fifth deduction result based on the number of the multiple characters and a fifth weight coefficient; determining the second evaluation score based on the third deduction result, the fourth deduction result and the fifth deduction result, wherein the third weight coefficient and the fifth weight coefficient are greater than the fourth weight coefficient.

[0011] Exemplarily, evaluating the quality of the synthesized speech based on the first evaluation score and the second evaluation score includes: determining a first evaluation result based on the first evaluation score and a sixth weight coefficient; determining a second evaluation result based on the second evaluation score and a seventh weight coefficient; determining the quality of the synthesized speech based on the first evaluation result and the second evaluation result, wherein the sixth weight coefficient is smaller than the seventh weight coefficient.

[0012] Exemplarily, the method comprises: determining a mean opinion score of the synthesized speech, wherein the evaluating the quality of the synthesized speech is also based on the mean opinion score.

[0013] Exemplarily, the method further includes: after evaluating the quality of each synthesized speech, generating and outputting an audio list according to the quality of each synthesized speech and each synthesized speech, wherein in the audio list, each synthesized speech is arranged in order according to the quality of each synthesized speech.

[0014] According to a second aspect of the present invention, there is also provided a device for evaluating and recognizing quality of synthesized speech, comprising:

[0015] The receiving module is used to obtain at least one synthesized speech and respective original texts.

[0016] The phoneme recognition module is used to determine a phoneme sequence for each of the at least one synthesized speech according to an original text corresponding to the synthesized speech.

[0017] A forced alignment module, used for forcibly aligning the acoustic features of the synthesized speech with the text features of the phoneme sequence to determine the start and end time of each phoneme in the phoneme sequence;

[0018] A first evaluation module, used for determining a first evaluation score of the synthesized speech according to the start and end time of each phoneme;

[0019] The second evaluation module is used to evaluate the quality of the synthesized speech according to the first evaluation score.

[0020] According to a third aspect of the present invention, there is also provided an electronic device, comprising: a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used to execute the above-mentioned synthetic speech quality assessment method when the processor is running.

[0021] According to a fourth aspect of the present invention, a storage medium is further provided, on which program instructions are stored, and the program instructions are used to execute the above-mentioned synthetic speech quality assessment method when running.

[0022] According to a fifth aspect of the present invention, there is further provided a computer program product, comprising computer program instructions, wherein the computer program instructions are used to execute the above-mentioned synthetic speech quality assessment method when running.

[0023] In the above technical solution, at least one synthesized speech and the respective original text are obtained, and then for each of the at least one synthesized speech, a phoneme sequence is determined according to the original text corresponding to the synthesized speech, and then the acoustic features of the synthesized speech and the text features of the phoneme sequence are forced to align to determine the start and end time of each phoneme in the phoneme sequence, and then the first evaluation score of the synthesized speech is determined according to the start and end time of each phoneme, and finally the quality of the synthesized speech is evaluated according to the first evaluation score. In this way, the quality of the synthesized speech can be evaluated more accurately.

[0024] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and other purposes, features and advantages of the present invention will become more apparent by describing the embodiments of the present invention in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0026] Figure 1 A schematic flow chart of a method for evaluating the quality of synthesized speech according to an embodiment of the present invention is shown;

[0027] Figure 2 A schematic flow chart of determining a first evaluation score of the synthesized speech according to the start and end time of each phoneme according to an embodiment of the present invention is shown;

[0028] Figure 3 A schematic flow chart of determining a first evaluation score according to the duration of each phoneme and the duration of each silence segment according to an embodiment of the present invention is shown;

[0029] Figure 4 A schematic flow chart of evaluating the quality of the synthesized speech according to a first evaluation score according to an embodiment of the present invention is shown;

[0030] Figure 5 A schematic flow chart of calculating a second evaluation score based on missing characters, wrong characters and extra characters according to one embodiment of the present invention is shown;

[0031] Figure 6 A schematic flow chart of evaluating the quality of the synthesized speech according to a first evaluation score and a second evaluation score according to an embodiment of the present invention is shown;

[0032] Figure 7 A schematic block diagram of a device for evaluating the quality of synthesized speech according to an embodiment of the present invention is shown;

[0033] Figure 8 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical scheme and advantages of the present invention more obvious, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the protection scope of the present invention.

[0035] In order to at least partially solve the above problem, a method for evaluating the quality of synthesized speech is proposed. After obtaining at least one synthesized speech and the respective original texts, the method determines a phoneme sequence for each of the at least one synthesized speech according to the original text corresponding to the synthesized speech, forcibly aligns the acoustic features of the synthesized speech with the text features of the phoneme sequence to determine the start and end time of each phoneme in the phoneme sequence, determines a first evaluation score of the synthesized speech according to the start and end time of each phoneme, and evaluates the quality of the synthesized speech according to the first evaluation score.

[0036] Figure 1 FIG. 4 is a schematic flow chart of a method for evaluating the quality of synthesized speech according to an embodiment of the present invention. Figure 1 As shown, the method for evaluating the quality of synthesized speech may include steps S110 to S140.

[0037] In step S110, at least one synthesized speech and respective original texts are obtained.

[0038] Synthesized speech can be a speech signal spliced ​​together from multiple audios. Synthesized speech can also establish a mathematical model to generate a speech signal. Synthesized speech can also use a speech signal generated by a speech synthesis model.

[0039] The synthesized speech may be a real-time synthesized speech or a pre-generated synthesized speech. The synthesized speech may be an original synthesized speech or a synthesized speech after a processing operation, and the processing operation may include filtering, denoising, etc. For example, the speech synthesis model may include a rule-based TTS model, an SPSS model, FastSpeech, VITS, etc.

[0040] When the synthesized speech is a speech signal generated by using a speech synthesis model, the text input into the speech synthesis model can be used as the original text of the synthesized speech. When the synthesized speech is a speech signal generated by splicing multiple audios or establishing a mathematical model, the original text can be determined based on the original text corresponding to the audio before splicing or the original text corresponding to the mathematical model.

[0041] In step S120, for each of the at least one synthesized speech, a phoneme sequence is determined according to the original text corresponding to the synthesized speech.

[0042] For example, the original text may be converted into a corresponding phoneme sequence based on a phoneme dictionary.

[0043] For example, a statistical model may be used to analyze the mapping relationship between the original text and the corresponding phonemes, thereby obtaining a phoneme sequence.

[0044] For example, a deep learning model can be used to analyze the original text to obtain a phoneme sequence.

[0045] In step S130, the acoustic features of the synthesized speech and the text features of the phoneme sequence are forcibly aligned to determine the start and end time of each phoneme in the phoneme sequence.

[0046] For example, the dynamic time warping (DTW) algorithm can be used to align the acoustic features of the synthesized speech and the text features of the phoneme sequence.

[0047] For example, based on the attention mechanism, it is possible to dynamically select which parts of the acoustic features of the synthesized speech are associated with those parts of the text features of the phoneme sequence, so that the acoustic features of the synthesized speech and the text features of the phoneme sequence are aligned.

[0048] For example, a Hidden Markov Model (HMM) can be used to capture the time dependency of the phoneme sequence and the dynamic changes of the phonetic segment voice information, so as to align the acoustic features of the synthesized speech with the text features of the phoneme sequence. For example, Viterbi decoding can be used to find the most likely phoneme sequence based on the acoustic features of the synthesized speech, so as to align it with the text features of the phoneme sequence.

[0049] For example, the step S120 and the step S130 may be performed using the Montreal Forced Aligner (MFA).

[0050] In step S140, a first evaluation score of the synthesized speech is determined according to the start and end times of each phoneme.

[0051] According to the start and end time of each phoneme, the duration corresponding to each phoneme can be determined, and further the valid information of the synthesized speech can be determined. According to the start and end time of each phoneme, the time period in which the valid information contained in the synthesized speech is located can be determined. For example, when the start and end times of adjacent phonemes are not continuous, it means that there is invalid information in the synthesized speech, and points can be deducted according to the start and end time of each phoneme. For example, when a real person speaks, the pronunciation of each word is usually similar, so the first evaluation score can also be determined according to the duration of each phoneme. For example, when a real person speaks, the pronunciation interval of each word is usually similar, so the first evaluation score can be determined according to the time interval between the start and end times of two adjacent phonemes.

[0052] In step S150, the quality of the synthesized speech is evaluated according to the first evaluation score.

[0053] The quality of the synthesized speech may be evaluated based on only the first evaluation score. The final evaluation result may also be determined in combination with other data related to the synthesized speech to evaluate the quality of the synthesized speech. For example, the quality of the synthesized speech may also be evaluated in combination with data such as the recognized text of the synthesized speech, the average opinion score, and the real speech of a real person reading the original text.

[0054] For example, an evaluation score may be determined based on the frequency spectrum difference between the real speech of a real person reading the original text and the synthesized speech, and then combined with the first evaluation score, the quality of the synthesized speech may be evaluated.

[0055] For example, the recognized text of the synthesized speech may be compared with the original text, and an evaluation score may be determined according to the difference between the two texts. Then, the quality of the synthesized speech may be evaluated in combination with the first evaluation score.

[0056] Exemplarily, a mean opinion score of the synthesized speech may also be determined, wherein the quality of the synthesized speech is also evaluated based on the mean opinion score. The mean opinion score usually ranges from 1 point to 5 points.

[0057] It can be understood that the above steps can be used to evaluate the quality of each synthesized speech.

[0058] In the above technical solution, at least one synthesized speech and the respective original text are obtained, and then for each of the at least one synthesized speech, a phoneme sequence is determined according to the original text corresponding to the synthesized speech, and then the acoustic features of the synthesized speech and the text features of the phoneme sequence are forced to align to determine the start and end time of each phoneme in the phoneme sequence, and then the first evaluation score of the synthesized speech is determined according to the start and end time of each phoneme, and finally the quality of the synthesized speech is evaluated according to the first evaluation score. In this way, the quality of the synthesized speech can be evaluated more accurately and objectively.

[0059] Figure 2 FIG. 4 is a schematic flow chart showing a method of determining a first evaluation score of a synthesized speech according to the start and end time of each phoneme according to an embodiment of the present invention. Figure 2 As shown, the above step S140 may include steps S210 to S220.

[0060] In step S210, the duration of each phoneme and the duration of each silent segment in the synthesized speech are determined according to the start and end times of each phoneme.

[0061] It can be understood that since there is pronunciation information in the non-silent segment, there is a corresponding phoneme after alignment. However, there is no pronunciation information in the silent segment, so there is no corresponding phoneme. The duration of each phoneme can be obtained by subtracting the start time from the end time of each phoneme. Each silent segment can be determined based on the time period between the end time of the previous phoneme and the start time of the next phoneme in the adjacent phonemes, thereby determining the duration of each silent segment.

[0062] For example, the step S210 may be performed using the Montreal Forced Aligner (MFA).

[0063] In step S220, a first evaluation score is calculated according to the duration of each phoneme and the duration of each silence segment.

[0064] For example, the first evaluation score can be calculated according to the first difference between the duration of each phoneme and the first preset duration. It can be understood that when the first difference is larger, the duration of the phoneme is more abnormal, and the first evaluation score indicates that the quality of the synthesized speech is worse.

[0065] For example, the first evaluation score can be calculated according to the difference between the duration of each silent segment and the second preset duration. It can be understood that when the second difference is larger, the duration of the silent segment is more abnormal, and the first evaluation score indicates that the quality of the synthesized speech is worse.

[0066] For example, a duration threshold may be set for the duration of the phoneme and the duration of the silence segment, and the number of phonemes and silence segments whose duration exceeds the duration threshold is counted. The greater the number, the more abnormal the duration of the silence segment and the duration of the phoneme are, and the first evaluation score indicates that the quality of the synthesized speech is worse.

[0067] In the above technical solution, the duration of each phoneme and the duration of each silent segment in the synthesized speech are determined according to the start and end time of each phoneme, and the first evaluation score is calculated according to the duration of each phoneme and the duration of each silent segment. In this way, the first evaluation score can be objectively determined in combination with the actual situation of the synthesized speech.

[0068] Figure 3 FIG. 4 is a schematic flow chart of determining a first evaluation score according to the duration of each phoneme and the duration of each silence segment according to an embodiment of the present invention. Figure 3 As shown, the above step S220 may include steps S310 to S330.

[0069] In step S310, a first deduction result is determined according to the number of phonemes whose duration exceeds a first duration threshold and a first weight coefficient.

[0070] The first duration threshold may be a fixed value, or may be determined according to the duration of each phoneme.

[0071] Exemplarily, the first duration threshold is determined according to the average duration of each phoneme.

[0072] The average duration represents the overall state of the duration of the phonemes in the audio segment, and the first duration threshold determined according to the average duration can better measure the duration of each phoneme.

[0073] For example, the average duration may be directly used as the second duration threshold.

[0074] For example, the second duration threshold may be determined by directly combining the average duration and other relevant information of the duration of the phoneme. Other relevant information may include the variance, standard deviation, median, etc. of the duration of the phoneme.

[0075] The first weight coefficient may be a pre-set coefficient. The more phonemes whose duration exceeds the first duration threshold, the more deductions may be made in the second deduction result.

[0076] In step S320, a second deduction result is determined according to the number of silence segments whose duration exceeds the second duration threshold and the second weight coefficient.

[0077] The duration of each silence segment can be compared with the second duration threshold to determine the number of silence segments whose duration is longer than the second duration threshold. Silence segments whose duration is longer than the second duration threshold are usually silence segments with more invalid information, and these silence segments are usually abnormal, so the more silence segments whose duration exceeds the second duration threshold, the more deductions will be in the second deduction result.

[0078] In step S330, a first evaluation score is calculated according to the first deduction result and the second deduction result.

[0079] The first weight coefficient may be the same as the second weight coefficient, or may be different from the second weight coefficient. For example, the first deduction result and the second deduction result may be directly added to determine the added result as the first evaluation score.

[0080] For example, the first evaluation score Q1 may be calculated according to the following formula 1:

[0081] Q1=q1*k1+q2*k2 Formula 1

[0082] Among them, q1 represents the number of phonemes whose duration exceeds the first duration threshold, q2 represents the number of silent segments whose duration exceeds the second duration threshold, k1 represents the first weight coefficient, and k2 represents the second weight coefficient.

[0083] In the above technical solution, a first deduction result is determined according to the number of phonemes whose duration exceeds the first duration threshold and the first weight coefficient, a second deduction result is determined according to the number of silent segments whose duration exceeds the second duration threshold and the second weight coefficient, and a first evaluation score is calculated according to the first deduction result and the second deduction result. Determining the first evaluation score in this way can more accurately represent the quality of the synthesized speech.

[0084] Figure 4 FIG. 4 is a schematic flow chart of evaluating the quality of the synthesized speech according to a first evaluation score according to an embodiment of the present invention. Figure 4 As shown, the above step S150 may include steps S410 to S440.

[0085] In step S410, speech recognition is performed on the synthesized speech to obtain a recognition text corresponding to the synthesized speech.

[0086] A speech recognition model or a speech recognition algorithm may be used to identify a text sequence corresponding to the synthesized speech. For example, an automatic speech recognition model may be used to identify a text sequence corresponding to the synthesized speech.

[0087] In step S420, the original text and the recognized text are compared to determine missing characters, wrong characters and extra characters in the recognized text relative to the original text.

[0088] The original text and the recognized text can be compared word by word, and the missing characters, wrong characters and extra characters of the original text can be recorded. For example, the original text and the recognized text can be compared using a text matching algorithm to determine the missing characters, wrong characters and extra characters of the original text.

[0089] In step S430, a second evaluation score is calculated based on the missing characters, wrong characters and extra characters.

[0090] For example, the second evaluation score may be calculated based on the number of missing characters, wrong characters, and extra characters.

[0091] For example, the second evaluation score may be calculated according to the characters corresponding to the missing characters, wrong characters and extra characters in the original text.

[0092] For example, corresponding weight coefficients may be determined according to the importance of missing characters, wrong characters and multiple characters, and the second evaluation score may be calculated in combination with these weight coefficients.

[0093] In step S440, the quality of the synthesized speech is evaluated according to the first evaluation score and the second evaluation score.

[0094] For example, the first evaluation score and the second evaluation score may be directly added to determine a final evaluation result of the quality of the synthesized speech.

[0095] Exemplarily, the weight coefficients corresponding to the first evaluation score and the second evaluation score can also be determined first, and then the final evaluation result of the quality of the synthesized speech can be determined according to these weight coefficients.

[0096] Exemplarily, the first evaluation score and the second evaluation score can also be divided, multiplied, etc. for further operations to determine the final evaluation result of the quality of the synthesized speech.

[0097] In the above technical solution, speech recognition is performed on the synthesized speech to obtain the recognition text corresponding to the synthesized speech, the original text and the recognition text are compared to determine the missing words, misspelled words and extra words in the recognition text relative to the original text, and the second evaluation score is calculated according to the missing words, misspelled words and extra words. The quality of the synthesized speech is evaluated according to the first evaluation score and the second evaluation score. In this way, the evaluation scores obtained by combining multiple methods can be used to more accurately evaluate the quality of the synthesized speech.

[0098] Figure 5 FIG. shows a schematic flowchart of calculating the second evaluation score according to the missing words, misspelled words and extra words according to an embodiment of the present invention. As Figure 5 shown, the above step S430 may include steps S510 to S540.

[0099] In step S510, according to the number of missing words and the third weight coefficient, the third deduction result is determined.

[0100] The more the number of missing words, the greater the difference between the recognition text of the synthesized speech and the original text. Therefore, the deduction in the third deduction result can be more.

[0101] Exemplarily, for the original text "I'm full" and the recognition text "I ate", the number of missing words is 1, and the third deduction result can be determined based on the number of missing words.

[0102] In step S520, according to the number of misspelled words and the fourth weight coefficient, the fourth deduction result is determined.

[0103] The more the number of misspelled words, the greater the difference between the recognition text of the synthesized speech and the original text. Therefore, the deduction in the fourth deduction result can be more.

[0104] Exemplarily, for the original text "I'm full" and the recognition text "I ate it wrong", the number of misspelled words is 2, and the fourth deduction result can be determined based on the number of misspelled words.

[0105] In step S530, according to the number of extra words and the fifth weight coefficient, the fifth deduction result is determined.

[0106] The greater the number of extra words, the greater the difference between the recognized text of the synthesized speech and the original text, and thus the greater the deduction points in the fifth deduction point result.

[0107] For example, for the original text "I have finished my meal" and the recognized text "I have just finished my meal", the number of extra characters is 1, and the fifth deduction result can be determined based on the number of the extra characters.

[0108] The third weight coefficient, the fourth weight coefficient and the fifth weight coefficient may be the same or different.

[0109] In step S540, a second evaluation score is determined according to the third deduction result, the fourth deduction result and the fifth deduction result, wherein the third weight coefficient and the fifth weight coefficient are greater than the fourth weight coefficient.

[0110] Wrong characters in the recognized text may also be caused by the performance of the speech recognition model and speech recognition algorithm, so the third weight coefficient and the fifth weight coefficient should be greater than the fourth weight coefficient. The number of missing characters and extra characters can better reflect the quality of the recognized text, and thus reflect the quality of the synthesized speech.

[0111] For example, the third deduction result, the fourth deduction result, and the fifth deduction result may be directly added together to determine the added result as the second evaluation score.

[0112] For example, the second evaluation score Q2 may be determined according to the following formula 2:

[0113] Q2=q3*k3+ q4*k4+ q5*k5 Formula 2

[0114] Among them, q3 represents the number of missing characters, q4 represents the number of wrong characters, q5 represents the number of extra characters, k3 represents the third weight coefficient, k4 represents the fourth weight coefficient, and k5 represents the fifth weight coefficient.

[0115] In the above technical solution, the third deduction result is determined according to the number of missing characters and the third weight coefficient, the fourth deduction result is determined according to the number of wrong characters and the fourth weight coefficient, the fifth deduction result is determined according to the number of extra characters and the fifth weight coefficient, and the second evaluation score is determined according to the third deduction result, the fourth deduction result and the fifth deduction result, wherein the third weight coefficient and the fifth weight coefficient are greater than the fourth weight coefficient. In this way, a second evaluation score that more accurately represents the quality of the synthesized speech can be determined.

[0116] Figure 6 FIG. 4 is a schematic flow chart showing a method of evaluating the quality of the synthesized speech according to a first evaluation score and a second evaluation score according to an embodiment of the present invention. Figure 6 As shown, the above step S440 may include steps S610 to S630.

[0117] In step S610, a first evaluation result is determined according to the first evaluation score and the sixth weight coefficient.

[0118] The sixth weight coefficient may be a weight coefficient used to determine the first evaluation score, such as the first weight coefficient and the second weight coefficient, or may be an additionally determined weight coefficient.

[0119] When the sixth weight coefficient is an additionally determined weight coefficient, the product of the first evaluation score and the sixth weight coefficient may be used as the first evaluation result. When the sixth weight coefficient is a weight coefficient used to determine the first evaluation score, the first evaluation score may be directly used as the first evaluation result.

[0120] In step S620, a second evaluation result is determined according to the second evaluation score and the seventh weight coefficient.

[0121] Similar to determining the first evaluation result using the first evaluation score and the sixth weight coefficient, determining the second evaluation result according to the second evaluation score and the seventh weight coefficient will not be described in detail herein.

[0122] In step S630, the quality of the synthesized speech is determined according to the first evaluation result and the second evaluation result, wherein the sixth weight coefficient is less than the seventh weight coefficient.

[0123] The first evaluation result is determined by the original text and the synthesized speech, and the second evaluation result is determined by the original text and the recognized text of the synthesized speech. The original text and the recognized text of the synthesized speech can better reflect the difference between the synthesized speech and the real speech, so the sixth weight coefficient can be set to be less than the seventh weight coefficient.

[0124] For example, in addition to the first evaluation result and the second evaluation result, the quality of the synthesized speech may also be determined according to other scoring results, such as a mean opinion score, a spectrum comparison result between the synthesized speech and the real speech, and the like.

[0125] For example, the following formula 3 can be obtained according to the above formula 1 and formula 2 to determine the final evaluation result Q0:

[0126] Q0= MOS-(q1*k1+q2*k2+ q3*k3+ q4*k4+ q5*k5) Formula 3

[0127] Among them, MOS represents the mean opinion score of the synthesized speech, the first evaluation score includes q1 and q2, the second evaluation score includes q3, q4, and q5, the sixth weight coefficient includes k1 and k2, and the seventh weight coefficient includes k3, k4, and k5.

[0128] The final evaluation result can be used as the quality of the synthesized speech.

[0129] In the above technical solution, the first evaluation result is determined according to the first evaluation score and the sixth weight coefficient, the second evaluation result is determined according to the second evaluation score and the seventh weight coefficient, and the quality of the synthesized speech is determined according to the first evaluation result and the second evaluation result, wherein the sixth weight coefficient is less than the seventh weight coefficient. In this way, the influence of different evaluation results on the quality of the synthesized speech can be accurately balanced.

[0130] Exemplarily, the above-mentioned method for evaluating the quality of synthesized speech may further include step S170:

[0131] After the quality of each synthesized speech is evaluated, an audio list is generated and output according to the quality of each synthesized speech and each synthesized speech, wherein in the audio list, each synthesized speech is arranged in order according to the quality of each synthesized speech.

[0132] The synthesized speech can be arranged in the audio list according to the quality of the synthesized speech, in the order from good to bad or from bad to good. After obtaining the audio list, the audio list can be output to a specified location. For example, the audio list can be output to a user interface so that the user can intuitively determine the quality of the synthesized speech.

[0133] Figure 7 FIG. 4 is a schematic block diagram of a device for evaluating the quality of synthesized speech according to an embodiment of the present invention. Figure 7 As shown, the quality assessment apparatus for synthesized speech includes a receiving module 710 , a phoneme recognition module 720 , a forced alignment module 730 , a first assessment module 740 , and a second assessment module 750 .

[0134] The receiving module 710 is used to obtain at least one synthesized speech and respective original texts.

[0135] The phoneme recognition module 720 is used to determine a phoneme sequence for each of the at least one synthesized speech according to the original text corresponding to the synthesized speech.

[0136] The forced alignment module 730 is used to force alignment of the acoustic features of the synthesized speech and the text features of the phoneme sequence to determine the start and end time of each phoneme in the phoneme sequence.

[0137] The first evaluation module 740 is used to determine a first evaluation score of the synthesized speech according to the start and end times of each phoneme.

[0138] The second evaluation module 750 is used to evaluate the quality of the synthesized speech according to the first evaluation score.

[0139] Exemplarily, the first evaluation module 740 may include a duration determination submodule and an evaluation determination submodule. The duration determination submodule is used to determine the duration of each phoneme and the duration of each silent segment in the synthesized speech according to the start and end time of each phoneme. The evaluation determination submodule is used to calculate the first evaluation score according to the duration of each phoneme and the duration of each silent segment.

[0140] Exemplarily, the evaluation determination submodule may include a first deduction submodule, a second deduction submodule and a first scoring submodule. The first deduction submodule is used to determine a first deduction result according to the number of phonemes whose duration exceeds the first duration threshold and a first weight coefficient. The second deduction submodule is used to determine a second deduction result according to the number of silent segments whose duration exceeds the second duration threshold and a second weight coefficient. The first scoring submodule is used to calculate a first evaluation score according to the first deduction result and the second deduction result.

[0141] Exemplarily, the evaluation and determination submodule may further include a threshold calculation module. The threshold calculation module is used to determine a first duration threshold according to the average duration of each phoneme.

[0142] Exemplarily, the second evaluation module 750 may include a speech recognition submodule, a text comparison submodule, a second scoring submodule and a scoring evaluation submodule. The speech recognition submodule is used to perform speech recognition on the synthesized speech to obtain a recognized text corresponding to the synthesized speech. The text comparison submodule is used to compare the original text and the recognized text to determine the missing characters, wrong characters and extra characters in the recognized text relative to the original text. The second scoring submodule is used to calculate a second evaluation score based on the missing characters, wrong characters and extra characters. The scoring evaluation submodule is used to evaluate the quality of the synthesized speech based on the first evaluation score and the second evaluation score.

[0143] Exemplarily, the second scoring module may include a third deduction submodule, a fourth deduction submodule, a fifth deduction submodule and a third scoring submodule. The third deduction submodule is used to determine a third deduction result according to the number of missing characters and a third weight coefficient. The fourth deduction submodule is used to determine a fourth deduction result according to the number of wrong characters and a fourth weight coefficient. The fifth deduction submodule is used to determine a fifth deduction result according to the number of multiple characters and a fifth weight coefficient. The third scoring submodule is used to determine a second evaluation score according to the third deduction result, the fourth deduction result and the fifth deduction result, wherein the third weight coefficient and the fifth weight coefficient are greater than the fourth weight coefficient.

[0144] Exemplarily, the scoring and evaluation submodule may include a third scoring submodule, a fourth scoring submodule and a comprehensive evaluation submodule. The third scoring submodule is used to determine a first evaluation result according to the first evaluation score and the sixth weight coefficient. The fourth scoring submodule is used to determine a second evaluation result according to the second evaluation score and the seventh weight coefficient. The comprehensive evaluation submodule is used to determine the quality of the synthesized speech according to the first evaluation result and the second evaluation result, wherein the sixth weight coefficient is less than the seventh weight coefficient.

[0145] Exemplarily, the quality assessment device for synthesized speech may further include a mean opinion score determination submodule. The mean opinion score determination submodule is used to determine a mean opinion score of the synthesized speech, wherein the quality of the synthesized speech is also assessed based on the mean opinion score.

[0146] Exemplarily, the quality assessment device for synthesized speech may further include a list generation submodule. The list generation submodule is used to generate and output an audio list according to the quality of each synthesized speech and each synthesized speech after assessing the quality of each synthesized speech, wherein in the audio list, each synthesized speech is arranged in the order of the quality of each synthesized speech.

[0147] According to another aspect of the present invention, an electronic device is provided. Figure 8 FIG. 1 shows a schematic block diagram of an electronic device according to an embodiment of the present invention. Figure 8 As shown, the electronic device includes a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used by the processor to execute the quality assessment method of synthesized speech as described above when the processor is running.

[0148] In addition, according to another aspect of the present invention, a storage medium is also provided, on which program instructions are stored, and when the program instructions are executed by a computer or a processor, the computer or the processor executes the corresponding steps of the quality assessment method of the above-mentioned synthesized speech according to the embodiment of the present invention, and is used to implement the corresponding modules in the quality assessment device of the above-mentioned synthesized speech according to the embodiment of the present invention. The storage medium may include, for example, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0149] According to another aspect of the present invention, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions are used to execute the above-mentioned synthetic speech quality assessment method when running.

[0150] A person skilled in the art can understand the specific implementation and beneficial effects of the above-mentioned synthetic speech quality assessment device, electronic device, storage medium and computer program product by reading the above-mentioned detailed description of the synthetic speech quality assessment method, and for the sake of brevity, they will not be repeated here.

[0151] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present invention thereto. Various changes and modifications may be made therein by one of ordinary skill in the art without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as required by the appended claims.

[0152] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0153] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0154] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0155] Similarly, it should be understood that in order to streamline the present invention and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present invention should not be interpreted as reflecting the following intention: the claimed invention requires more features than the features explicitly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present invention.

[0156] It will be understood by those skilled in the art that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0157] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0158] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some modules in the quality assessment device for synthesized speech according to an embodiment of the present invention. The present invention can also be implemented as a device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0159] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0160] The above is only a specific embodiment of the present invention or an explanation of a specific embodiment. The protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for evaluating the quality of synthesized speech, characterized in that: The method comprises: Obtaining at least one synthesized speech and respective original text; For each of the at least one synthesized speech, Determine a phoneme sequence according to the original text corresponding to the synthesized speech; Forcibly aligning the acoustic features of the synthesized speech with the text features of the phoneme sequence to determine the start and end times of each phoneme in the phoneme sequence; Determining a first evaluation score of the synthesized speech according to the start and end times of each phoneme; The quality of the synthesized speech is evaluated according to the first evaluation score.

2. The method according to claim 1, characterized in that Determining a first evaluation score of the synthesized speech according to the start and end times of each phoneme includes: Determine the duration of each phoneme and the duration of each silent segment in the synthesized speech according to the start and end times of each phoneme; The first evaluation score is calculated according to the duration of each phoneme and the duration of each silence segment.

3. The method according to claim 2, characterized in that The determining the first evaluation score according to the duration of each phoneme and the duration of each silent segment includes: Determining a first deduction result according to the number of phonemes whose duration exceeds a first duration threshold and a first weight coefficient; Determine a second deduction result according to the number of silence segments whose duration exceeds the second duration threshold and the second weight coefficient; The first evaluation score is calculated according to the first deduction result and the second deduction result.

4. The method according to claim 3, characterized in that The determining the first evaluation score according to the duration of each phoneme and the duration of each silent segment further includes: The first duration threshold is determined according to the average duration of each phoneme.

5. The method according to any one of claims 1 to 4, characterized in that: The step of evaluating the quality of the synthesized speech according to the first evaluation score comprises: Performing speech recognition on the synthesized speech to obtain a recognition text corresponding to the synthesized speech; Comparing the original text with the recognized text to determine missing characters, wrong characters, and extra characters in the recognized text relative to the original text; Calculating a second evaluation score based on the missing characters, the wrong characters, and the extra characters; The quality of the synthesized speech is evaluated according to the first evaluation score and the second evaluation score.

6. The method according to claim 5, characterized in that The step of calculating the second evaluation score according to the missing characters, the wrong characters and the extra characters comprises: Determining a third deduction result according to the number of missing characters and a third weight coefficient; Determining a fourth point deduction result according to the number of wrong characters and a fourth weight coefficient; Determining a fifth deduction result according to the number of the multiple words and a fifth weight coefficient; The second evaluation score is determined according to the third deduction result, the fourth deduction result and the fifth deduction result, wherein the third weight coefficient and the fifth weight coefficient are greater than the fourth weight coefficient.

7. A device for evaluating the quality of synthesized speech, characterized in that: include: The receiving module is used to obtain at least one synthesized speech and respective original texts. The phoneme recognition module is used to determine a phoneme sequence for each of the at least one synthesized speech according to the original text corresponding to the synthesized speech. A forced alignment module, used for forcibly aligning the acoustic features of the synthesized speech with the text features of the phoneme sequence to determine the start and end time of each phoneme in the phoneme sequence; A first evaluation module, used for determining a first evaluation score of the synthesized speech according to the start and end time of each phoneme; The second evaluation module is used to evaluate the quality of the synthesized speech according to the first evaluation score.

8. An electronic device comprising a processor and a memory, characterized in that: The memory stores computer program instructions, which are used by the processor to execute the method for evaluating the quality of synthesized speech according to any one of claims 1 to 6 when executed.

9. A storage medium having program instructions stored thereon, characterized in that: The program instructions are used to execute the quality assessment method of synthesized speech as claimed in any one of claims 1 to 6 when running.

10. A computer program product comprising computer program instructions, characterized in that The computer program instructions are used to execute the quality assessment method of synthesized speech as claimed in any one of claims 1 to 6 when running.

Images