Sound output method and device, readable medium and electronic equipment
By acquiring and marking speech file datasets, using machine learning and F0 curve analysis, the speech generation model is trained to output speech with specified emotions, solving the problem of inability to express text emotions in the prior art, and improving user experience and voice anthropomorphism degree.
Patent Information
- Application Number
- CN202411133193.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-11-26
AI Technical Summary
After the prior art text information is converted into pronunciation, it is impossible to effectively express the emotions that the text wants to express, resulting in poor user experience.
By obtaining and marking training speech file data sets, including basic emotional speech and emotional speech files, the duration and frequency difference of phonemes are obtained using machine learning methods, and combined with F0 curve analysis, the speech generation model is trained to output speech with specified emotions.
It achieves the output results of sounds with different emotions when inputting the same text, improves the user experience and enhances the anthropomorphism of the voice.
Smart Images

Figure CN119028318A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a sound output method, device, readable medium and electronic equipment. Background Art
[0002] Speech synthesis, also known as text-to-speech technology, can convert any text information into standard and fluent speech in real time, which is equivalent to installing an artificial mouth on the machine. It involves multiple disciplines such as acoustics, linguistics, digital signal processing, computer science, etc., and is a cutting-edge technology in the field of Chinese information processing.
[0003] In the prior art, after text information is converted into voice information, it is pronounced with one tone and timbre from beginning to end. The pronunciation of the converted voice information is stereotyped and cannot express the emotion that the text wants to express, resulting in a poor user experience. Summary of the invention
[0004] The object of the present invention is to provide a sound output method, device, readable medium and electronic device to solve the problems raised in the above background technology.
[0005] To achieve the above object, a technical solution adopted by the present invention is: a sound output method, comprising the following steps:
[0006] S1: Acquire all training speech file data sets, wherein the all training speech file data sets include a basic emotion speech file data set and multiple emotion speech file data sets, generate a speech concatenation model, and mark the basic emotion speech file data set and the multiple emotion speech file data sets respectively to distinguish them;
[0007] S2: Obtain the duration and frequency of each phoneme between any adjacent phoneme combination in the basic emotional speech file dataset and / or the emotional speech file dataset by machine learning methods;
[0008] S3: The difference in duration and frequency of each phoneme between any two data sets and any adjacent phoneme combination between each emotional speech file data set and the basic emotional speech file data set is obtained through F0 curve analysis;
[0009] S4: Specify the output emotion, input text, and output speech with the specified emotion.
[0010] Preferably, the training and generating steps of the sound output method adopt any one of the following methods:
[0011] Method 1:
[0012] S11: training the overall speech generation model, inputting the basic emotional speech file dataset and multiple emotional speech file datasets respectively marked and distinguished, and obtaining the duration and frequency of each phoneme between any adjacent phoneme combinations of the basic emotional speech file dataset and each emotional speech file dataset;
[0013] S12: performing F0 curve analysis on the duration and frequency of the phonemes between any adjacent phoneme combinations obtained in S11, and obtaining the difference in duration and frequency of each phoneme between any adjacent phoneme combinations obtained by comparing each emotional speech file with the basic emotional speech in a total speech generation model;
[0014] S13: Specify the output emotion and input the text into the overall speech generation model, so as to output the speech with the specified emotion expression.
[0015] Method 2:
[0016] S21: training a basic emotional speech generation model 1, wherein the basic emotional speech generation model 1 can obtain the duration and frequency of each phoneme between any adjacent phoneme combinations in the basic emotional sound;
[0017] S22: training a plurality of emotion speech generation models 1, each of which can obtain the duration and frequency of each phoneme between any adjacent phoneme combination in an emotion sound;
[0018] S23: performing F0 curve analysis on the duration and frequency of phonemes between any adjacent phoneme combinations in the basic emotional speech generation model 1 and the multiple emotional speech generation models 1, and obtaining the difference in duration and frequency of each phoneme between any adjacent phoneme combinations between the two models;
[0019] S24: Connecting the input end of each emotional speech generation model 1 to the output end of the basic emotional speech generation model 1 to obtain a speech generation and voice conversion model;
[0020] S25: Input text and specify the output emotion into the speech generation and transliteration model, that is, output a speech with the specified emotion expression.
[0021] Method 3:
[0022] S31: training a speech generation sub-model, inputting a basic emotional speech file dataset and a single emotional speech file dataset that are respectively marked and distinguished, and obtaining the duration and frequency of each phoneme between any adjacent phoneme combinations of the basic emotional speech file dataset and the single emotional speech file dataset;
[0023] S32: Repeat step S31 to obtain multiple speech generation sub-models of the duration and frequency of each phoneme between any adjacent phoneme combinations for the basic emotional speech file dataset and each single emotional speech file dataset;
[0024] S33: performing F0 curve analysis on the duration and frequency of phonemes between any adjacent phoneme combinations in each speech generation sub-model, and obtaining the difference in duration and frequency of each phoneme between any adjacent phoneme combinations obtained by comparing the emotional speech file with the basic emotional speech in each speech generation sub-model;
[0025] S34: Specify an emotion and input text into the speech generation sub-model of the emotion, so as to output a speech expressing the specified emotion.
[0026] Preferably, the method further comprises the following steps:
[0027] S01: Input basic emotion speech file dataset or emotional speech file dataset;
[0028] S02: using a natural language processing system capable of processing a standard phoneme set to obtain the time length of each phoneme in the voice file, and at the same time, under the condition that there is a gap time length before and after each phoneme, obtaining the gap time length before and after each phoneme, wherein the gap time length is a phoneme extension sound or a blank sound;
[0029] S03: Obtain a collection of gap durations in all training speech files, and use a statistical method to obtain a collection of phoneme extension sounds or blank sounds with a certain confidence level, wherein the confidence level is at least 80%;
[0030] S04: In step S03, manually verify whether the phoneme extension sound or the blank sound is true and correct;
[0031] S05: Input the data obtained in S04 into the deep learning model to learn the time length and frequency of each phoneme in any adjacent phonemes, the phoneme extension sound or blank sound before and after each phoneme, and the duration of the phoneme extension sound or blank sound.
[0032] Preferably, the gap duration also includes emotion and habit features, and the emotion and habit features are phoneme-level features of a non-standard phoneme set;
[0033] In step S03, it is also necessary to obtain a phoneme set of emotion and habit features with a confidence level of at least 80%; in step S04, it is also necessary to manually verify whether the emotion and habit features are true and correct;
[0034] Before step S05, it is also necessary to add the phoneme set of emotion and habit characteristics to the standard phoneme set to obtain a phoneme collection, and input the phoneme collection into a natural language processing system that can process the phoneme collection to obtain the time length of each phoneme of the voice file.
[0035] Preferably, in step S03, the features of the phonemes and the extended sounds of the phonemes in the gap duration of each emotion and habit feature need to be unified, and the steps of the feature unification are as follows:
[0036] SA1: Convert the extended phonemes and gap durations of each emotion and habit feature into spectra respectively;
[0037] SA2: Arrange and group all the extended phonemes or gap durations of each emotion and habit feature according to their similarity;
[0038] SA3: Determine the condition of feature unification, and determine the sound spectrum of the emotional and habitual characteristics determined under the condition or the sound spectrum of the extended sound of the phoneme in the gap duration based on the condition of feature unification.
[0039] Preferably, in step SA2, it is necessary to balance the materials in the group, and in step S05, it is necessary to balance the time length and frequency of each phoneme, the phoneme extension sound and the blank sound before and after the phoneme.
[0040] Preferably, before step S01, a segmentation condition is set, and segmentation is performed when the segmentation condition exists for each training voice file, wherein the segmentation condition is the set duration of the blank sound.
[0041] The present invention also provides a speech processing device, comprising:
[0042] Input module, used to input text files and selected emotions;
[0043] A processing module, used for converting the text file into a phoneme-level file with the selected emotion, and determining whether to insert a phoneme extension sound, a blank sound or an emotion and habit feature before and after each phoneme, and performing processing;
[0044] Generate module, used to merge and output processed files.
[0045] The present invention also provides a computer-readable storage medium on which a program is stored. When the program is run, the computer executes the steps in the speech generation method.
[0046] The present invention also provides an electronic device, comprising:
[0047] a memory configured to store a computer program;
[0048] The processor is configured to make the computer execute the steps in the speech generation method when the program is executed.
[0049] Beneficial effects of the present invention: This solution trains the speech generation model and obtains the frequency difference between the two models through the analysis of the F0 curve, thereby determining the performance difference of each phoneme under different emotions, so that when the same text is input, sound output results with different emotions can be obtained, thereby improving the user experience;
[0050] Before synthesizing speech based on phoneme-level features, it is determined whether a phoneme extension sound or blank sound or emotional and habitual features need to be inserted before and after each phoneme, which can effectively ensure the degree of anthropomorphism of the generated speech and improve the user experience.
[0051] Among them, by balancing the various materials in the group in step SA2, it can be effectively ensured that emotional and habitual characteristics or extended sounds can be trained and learned. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 It is a flow chart of the implementation principle of the sound output method in the present invention;
[0053] Figure 2 It is a flowchart of the first method in the training and generation steps of the sound output method of the present invention;
[0054] Figure 3 It is a flowchart of the second method in the training and generation steps of the sound output method of the present invention;
[0055] Figure 4 It is a flowchart of method three in the training and generation steps of the sound output method in the present invention. DETAILED DESCRIPTION
[0056] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.
[0057] Example:
[0058] A sound output method, device, readable medium and electronic device include a sound output method, a sound output device, a computer-readable storage medium and an electronic device.
[0059] For details, see Figure 1 , a sound output method, comprising the following steps:
[0060] The following steps are involved:
[0061] S1: Acquire all training speech file data sets, wherein the all training speech file data sets include a basic emotion speech file data set and multiple emotion speech file data sets, generate a speech concatenation model, and mark the basic emotion speech file data set and the multiple emotion speech file data sets respectively to distinguish them;
[0062] S2: Obtain the duration and frequency of each phoneme between any adjacent phoneme combination in the basic emotional speech file dataset and / or the emotional speech file dataset through machine learning methods (in this paragraph, "and / or" means processing two file datasets simultaneously in one model or processing two file datasets separately in different models);
[0063] S3: The difference in duration and frequency of each phoneme between any two data sets and any adjacent phoneme combination between each emotional speech file data set and the basic emotional speech file data set is obtained through F0 curve analysis;
[0064] S4: Specify the output emotion, input text, and output speech with the specified emotion.
[0065] Specifically, the training and generation steps of the sound output method adopt any one of the following methods:
[0066] Method 1:
[0067] S11: training the overall speech generation model, inputting the basic emotional speech file dataset and multiple emotional speech file datasets respectively marked and distinguished, and obtaining the duration and frequency of each phoneme between any adjacent phoneme combinations of the basic emotional speech file dataset and each emotional speech file dataset;
[0068] S12: performing F0 curve analysis on the duration and frequency of the phonemes between any adjacent phoneme combinations obtained in S11, and obtaining the difference in duration and frequency of each phoneme between any adjacent phoneme combinations obtained by comparing each emotional speech file with the basic emotional speech in a total speech generation model;
[0069] S13: Specify the output emotion and input the text into the overall speech generation model, so as to output the speech with the specified emotion expression.
[0070] Method 2:
[0071] S21: training a basic emotional speech generation model 1, wherein the basic emotional speech generation model 1 can obtain the duration and frequency of each phoneme between any adjacent phoneme combinations in the basic emotional sound;
[0072] S22: training a plurality of emotion speech generation models 1, each of which can obtain the duration and frequency of each phoneme between any adjacent phoneme combination in an emotion sound;
[0073] S23: performing F0 curve analysis on the duration and frequency of phonemes between any adjacent phoneme combinations in the basic emotional speech generation model 1 and the multiple emotional speech generation models 1, and obtaining the difference in duration and frequency of each phoneme between any adjacent phoneme combinations between the two models;
[0074] S24: Connecting the input end of each emotional speech generation model 1 to the output end of the basic emotional speech generation model 1 to obtain a speech generation and voice conversion model;
[0075] S25: Input text and specify the output emotion into the speech generation and transliteration model, that is, output a speech with the specified emotion expression.
[0076] Method 3:
[0077] S31: training a speech generation sub-model, inputting a basic emotional speech file dataset and a single emotional speech file dataset that are respectively marked and distinguished, and obtaining the duration and frequency of each phoneme between any adjacent phoneme combinations of the basic emotional speech file dataset and the single emotional speech file dataset;
[0078] S32: Repeat step S31 to obtain multiple speech generation sub-models of the duration and frequency of each phoneme between any adjacent phoneme combinations for the basic emotional speech file dataset and each single emotional speech file dataset;
[0079] S33: performing F0 curve analysis on the duration and frequency of phonemes between any adjacent phoneme combinations in each speech generation sub-model, and obtaining the difference in duration and frequency of each phoneme between any adjacent phoneme combinations obtained by comparing the emotional speech file with the basic emotional speech in each speech generation sub-model;
[0080] S34: Specify an emotion and input text into the speech generation sub-model of the emotion, so as to output a speech expressing the specified emotion.
[0081] Among them, in the above method 1, it is necessary to collect training voice files and perform emotion classification. For example, 1,000 voice files can be used as training files, of which 800 are basic emotional voice files that do not contain special emotions, 80 of which are voice files that include all happy emotions, 60 of which are voice files with sad emotions, and the remaining 60 are voice files with angry emotions. It should be noted that some voice files in the above 4 files need to be retained and not participate in training, so as to be used as verification sets;
[0082] In the above method 2, compared with method 1, its advantage is that it can greatly reduce the training materials. For example, at this time, the basic emotional voice files that do not contain special emotions that need to be trained can be 100, the voice files of happy emotions are 10, the voice files of sad emotions are 10, the voice files of angry emotions are 10, and similarly, some of the voice files of the above four files need to be retained and not participate in the training, so as to be used as a verification set;
[0083] In the above method 3, the advantage over method 1 is that the emotional characteristics expressed by the model trained by method 3 are more distinct. Similarly, in this method, some voice files need to be retained and not involved in training, so as to be used as a verification set;
[0084] The hyperparameters are adjusted based on the verification results of the verification set to optimize the training results. This step is a common operation in deep learning and will not be described in detail here.
[0085] Among them, the training or generation process of the above three methods includes the following specific refinement or optimization steps:
[0086] S01: Input the basic emotion speech file data set or the emotional speech file data set. Before this step, the segmentation condition needs to be set. Each training speech file is segmented when the segmentation condition exists. The segmentation condition is the set blank sound duration. For example, segmentation is performed when the blank sound of adjacent phonemes is greater than 1 second, and two speech files are obtained that do not include the blank sound. The preferred segmentation effect is to cut off the blank sound, which can reduce the total duration of the training material and improve the training efficiency.
[0087] S02: using a natural language processing system capable of processing a standard phoneme set to obtain the time length of each phoneme in the voice file, and at the same time, under the condition that there is a gap time length before and after each phoneme, obtaining the gap time length before and after each phoneme, wherein the gap time length is a phoneme extension sound or a blank sound, in this step, the whisper framework can be used, and under the standard whisper approach, the phonemes and time lengths constituting each training file and the gap time lengths between adjacent phonemes can be obtained;
[0088] S03: Obtain a collection of gap durations in all training speech files, and use a statistical method to obtain a collection of phoneme extension sounds or blank sounds with a certain confidence level, wherein the confidence level is at least 80%;
[0089] S04: In step S03, manually verify whether the phoneme extension sound or the blank sound is true and correct;
[0090] S05: Input the data obtained in S04 into a deep learning model for learning to obtain the time length and frequency of each phoneme in any adjacent phonemes, the phoneme extension sound or blank sound before and after each phoneme, and the duration of the phoneme extension sound or blank sound. The deep learning model used in this step can be a Transformer neural network model. The steps of vectorizing the data and inputting it into the deep learning model are routine operations for those skilled in the art, and will not be elaborated on here.
[0091] The gap duration also includes emotion and habit features, which are phoneme-level features of a non-standard phoneme set. In this embodiment, the emotion features include sound characteristics related to emotion, such as popping and whimpering, and the habit features include oral habits with language characteristics, etc.
[0092] In step S03, it is also necessary to obtain a phoneme set of sentiment and habit features with a confidence level of at least 80%;
[0093] In step S04, it is also necessary to manually verify whether the emotion and habit characteristics are true and correct;
[0094] Before step S05, it is also necessary to add the phoneme set of emotional and habitual characteristics to the standard phoneme set to obtain a phoneme collection, and input the phoneme collection into a natural language processing system that can process the phoneme collection to obtain the time length of each phoneme in the voice file. The natural language processing system in this step can also be a whisper framework.
[0095] In step S03, it is necessary to unify the features of the phonemes and the extended sounds of the phonemes in the gap duration of each emotion and habit feature, and the steps of unifying the features are as follows:
[0096] SA1: Convert the phonemes of each emotion and habit feature and the phoneme extension sounds in the gap duration into spectrograms respectively;
[0097] SA2: Arrange and group all the extended phonemes or phoneme sounds in the gap duration of each emotion and habit feature according to their similarity. The grouping condition can be to group the spectra according to a certain value of similarity, for example, to group spectra with an overall similarity of more than 95% into one group, 85%-95% into one group, etc.; or to group a fixed number of spectra according to similarity, for example, 10 into one group; or to group spectra with a certain value of similarity determined by taking a certain segment of each spectra, for example, to select the spectra within 40% of the central axis of the spectra for determination, or to select the spectra with the same time axis on both sides of the central axis of the spectra for similarity determination, etc.;
[0098] SA3: Determine the condition for feature unification, and determine the sound spectrum of the emotion and habit characteristics or the sound spectrum of the extended phoneme in the gap duration determined under the condition based on the condition for feature unification. Specifically, you can choose to select the consistent segments within each group of the sound spectrum within a fixed time axis as the condition for feature unification, or select 95% of each spectrum in each group for example, for balancing, etc. as the condition for feature unification. In actual use, when it is necessary to insert the sound spectrum of emotion and habit characteristics or the sound spectrum of the extended phoneme in the gap duration two, just insert the above-mentioned unified feature spectrum.
[0099] Among them, in step SA2, it is necessary to balance the materials in the group. In step S05, it is necessary to balance the time length and frequency of each phoneme, the phoneme extension sound and the blank sound before and after the phoneme. In this embodiment, the diffusion model can be used for balancing.
[0100] In addition, the speech output device provided by the present invention comprises:
[0101] Input module, used to input text files;
[0102] Input module, used to input text files and selected emotions;
[0103] A processing module, used for converting the text file into a phoneme-level file with the selected emotion, and determining whether to insert a phoneme extension sound, a blank sound or an emotion and habit feature before and after each phoneme, and performing processing;
[0104] Generate module, used to merge and output processed files;
[0105] Generate module, used to merge and output processed text files.
[0106] The computer-readable storage medium provided by the present invention stores a program thereon, and when the program is executed, the computer executes the above-mentioned speech generation method.
[0107] The electronic device provided by the present invention comprises:
[0108] a memory configured to store a computer program;
[0109] The processor is configured to enable the computer to execute the above-mentioned speech generation method when the program is running.
[0110] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A sound output method, characterized in that: Includes the following step: S1: Obtain all training speech file data sets, wherein the all training speech file data sets include a basic emotion speech file data set and multiple emotion speech file data sets, each emotion speech file data set contains only one emotion, and mark the basic emotion speech file data set and the multiple emotion speech file data sets respectively to distinguish them; S2: Obtain the duration and frequency of each phoneme between any adjacent phoneme combination in the basic emotional speech file dataset and / or the emotional speech file dataset by machine learning methods; S3: The difference in duration and frequency of each phoneme between any two data sets and any adjacent phoneme combination between each emotional speech file data set and the basic emotional speech file data set is obtained through F0 curve analysis; S4: Specify the output emotion, input text, and output speech with the specified emotion.
2. A sound output method according to claim 1, characterized in that: The training and generation steps of the sound output method adopt any one of the following methods: Method 1: S11: training the overall speech generation model, inputting the basic emotional speech file dataset and multiple emotional speech file datasets respectively marked and distinguished, and obtaining the duration and frequency of each phoneme between any adjacent phoneme combinations of the basic emotional speech file dataset and each emotional speech file dataset; S12: performing F0 curve analysis on the duration and frequency of the phonemes between any adjacent phoneme combinations obtained in S11, and obtaining the difference in duration and frequency of each phoneme between any adjacent phoneme combinations obtained by comparing each emotional speech file with the basic emotional speech in a total speech generation model; S13: Specify the output emotion and input the text into the overall speech generation model, so as to output the speech with the specified emotion expression. Method 2: S21: training a basic emotional speech generation model 1, wherein the basic emotional speech generation model 1 can obtain the duration and frequency of each phoneme between any adjacent phoneme combinations in the basic emotional sound; S22: training a plurality of emotion speech generation models 1, each of which can obtain the duration and frequency of each phoneme between any adjacent phoneme combination in an emotion sound; S23: performing F0 curve analysis on the duration and frequency of phonemes between any adjacent phoneme combinations in the basic emotional speech generation model 1 and the multiple emotional speech generation models 1, and obtaining the difference in duration and frequency of each phoneme between any adjacent phoneme combinations between the two models; S24: Connecting the input end of each emotional speech generation model 1 to the output end of the basic emotional speech generation model 1 to obtain a speech generation and voice conversion model; S25: Input text and specify the output emotion into the speech generation and transliteration model, that is, output a speech with the specified emotion expression. Method 3: S31: training a speech generation sub-model, inputting a basic emotional speech file dataset and a single emotional speech file dataset that are respectively marked and distinguished, and obtaining the duration and frequency of each phoneme between any adjacent phoneme combinations of the basic emotional speech file dataset and the single emotional speech file dataset; S32: Repeat step S31 to obtain multiple speech generation sub-models of the duration and frequency of each phoneme between any adjacent phoneme combinations for the basic emotional speech file dataset and each single emotional speech file dataset; S33: performing F0 curve analysis on the duration and frequency of phonemes between any adjacent phoneme combinations in each speech generation sub-model, and obtaining the difference in duration and frequency of each phoneme between any adjacent phoneme combinations obtained by comparing the emotional speech file with the basic emotional speech in each speech generation sub-model; S34: Specify an emotion and input text into the speech generation sub-model of the emotion, so as to output a speech expressing the specified emotion.
3. A sound output method according to any one of claim 2, characterized in that: The following steps are also included: S01: Input basic emotion speech file dataset or emotional speech file dataset; S02: using a natural language processing system capable of processing a standard phoneme set to obtain the time length of each phoneme in the voice file, and at the same time, under the condition that there is a gap time length before and after each phoneme, obtaining the gap time length before and after each phoneme, wherein the gap time length includes at least one of a phoneme extension sound or a blank sound; S03: Obtain a collection of gap durations in all training speech files, and use a statistical method to obtain a collection of phoneme extension sounds or blank sounds with a certain confidence level, wherein the confidence level is at least 80%; S04: In step S03, manually verify whether the phoneme extension sound or the blank sound is true and correct; S05: Input the data obtained in S04 into the deep learning model to learn the time length and frequency of each phoneme in any adjacent phonemes, the phoneme extension sound or blank sound before and after each phoneme, and the duration of the phoneme extension sound or blank sound.
4. A sound output method according to claim 3, characterized in that: The gap duration also includes emotion and habit features, and the emotion and habit features are phoneme-level features of a non-standard phoneme set; In step S03, it is also necessary to obtain a phoneme set of sentiment and habit features with a confidence level of at least 80%; In step S04, it is also necessary to manually verify whether the emotion and habit characteristics are true and correct; Before step S05, it is also necessary to add the phoneme set of emotion and habit characteristics to the standard phoneme set to obtain a phoneme collection, and input the phoneme collection into a natural language processing system that can process the phoneme collection to obtain the time length of each phoneme of the voice file.
5. A sound output method according to claim 4, characterized in that: In step S03, it is necessary to unify the features of the phonemes and phoneme extension sounds in the gap duration of each emotion and habit feature, and the steps of unifying the features are as follows: SA1: Convert the phonemes of each emotion and habit feature and the phoneme extension sounds in the gap duration into spectrograms respectively; SA2: Arrange and group all the extended phonemes or phoneme durations of each emotion and habit feature according to their similarity; SA3: Determine the conditions for feature unification, and determine the spectrum of the emotional and habitual characteristics determined under the conditions or the spectrum of the extended sound of the phoneme in the gap duration based on the conditions for feature unification.
6. A speech generation method according to claim 4, characterized in that: In step SA2, it is necessary to balance each material in the group, and in step S05, it is necessary to balance the time length and frequency of each phoneme, and the phoneme extension sound and blank sound before and after the phoneme.
7. A speech generation method according to claim 3, characterized in that: Before step S01, segmentation conditions need to be set. Each training voice file is segmented when the segmentation conditions exist. The segmentation condition is the set duration of blank sounds.
8. A sound output device, which generates speech according to the method described in claims 1-7, characterized in that: include: Input module, used to input text files and selected emotions; A processing module, used for converting the text file into a phoneme-level file with the selected emotion, and determining whether to insert a phoneme extension sound, a blank sound or an emotion and habit feature before and after each phoneme, and performing processing; Generate module, used to merge and output processed files.
9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed, the computer is enabled to execute the method according to any one of claims 1 to 7.
10. An electronic device, characterized in that: include: a memory configured to store a computer program; The processor is configured to make the computer execute the method according to any one of claims 1 to 7 when the program is executed.
