A system and method for arbitrary emotional speech conversion based on mutual information minimization

Abstract

The present invention discloses a system and method for converting arbitrary emotion into speech based on mutual information minimization. The system comprises a training module, which comprises: an emotion encoder for extracting an emotion embedding representation in a speech; a non-emotion encoder for extracting an emotion-independent representation in a speech; a decoder for reconstructing converted acoustic features from the emotion encoder and the emotion embedding representation and the emotion-independent representation generated by the non-emotion encoder; a variational approximation network for obtaining the mean and variance of the approximate distribution of the emotion embedding representation, and then calculating the mutual information with the emotion-independent representation; a training control module for continuously iterating and updating the parameters of the emotion encoder and the non-emotion encoder, and stopping the training when the mutual information calculated by the variational approximation network is the smallest. The present invention has the advantages of making the separation of emotion-related information and emotion-independent information more in-depth and thorough, and making the result of emotion conversion more accurate.

Description

Technical Field

[0001] The present invention relates to the technical fields of deep learning and speech signal processing, and more specifically to a system and method for converting arbitrary emotion into speech based on mutual information minimization. Background Art

[0002] Emotional voice conversion is a type of voice conversion. The emotional voice conversion system changes the emotional information in the voice, that is, the task of emotional voice conversion is to convert the emotion of the source voice into the emotion of the target voice without changing the sentence language information.

[0003] In the past, there have been many deep learning methods used in emotional speech conversion tasks, such as deep neural network (DNN), deep belief network (DBN), deep bidirectional long short-term memory network (DBLSTM), and sequence-to-sequence emotional speech conversion models. However, these models all use parallel speech data and require additional time alignment procedures, which greatly limits their application scope.

[0004] In order to overcome this problem, methods based on non-parallel data have been proposed, such as cycle-consistent generative adversarial networks, autoencoders, and variational autoencoder Wasserstein adversarial networks. For example, the paper "Zhong Xinzi, Liao Wenjian. Research on speech emotion recognition method based on autoencoder [J]. Electronic Design Engineering, 2020, 28(6): 69-72" discloses a method based on autoencoder combined with adversarial training and introduces an improved adversarial loss function in adversarial training for feature dimensionality reduction. Although these models overcome the limitation of parallel corpus, they can only achieve the conversion of a pair or a group of specific emotions.

[0005] Models that can complete invisible emotion conversion are also called any-to-any conversion models. For any-to-any emotion conversion models, the idea of ​​feature separation is more suitable for emotional speech conversion with few samples such as invisible emotions. In the field of speaker speech conversion, researchers often use feature separation methods to separate speaker information and text content information, so that the model can learn different speech representations and replace the speaker representation with the target speech, thereby achieving speaker conversion. There are three commonly used feature separation methods: dimensionality reduction sampling, vector quantization (VQ), and instance normalization. Since speaker information and emotional information are both paralinguistic information in speech, this model also has the potential to be applied to emotional speech conversion tasks. However, in some models, the degree of separation between different features is low, and the non-emotional information of the source speech still retains a part of the emotional representation content. When it is fused with the emotional information of the target emotional speech for emotional conversion, the conversion result contains part of the emotional information of the source speech, resulting in inaccurate emotional conversion results, which is a major technical deficiency. Summary of the invention

[0006] The technical problem to be solved by the present invention is that the degree of separation between different features in any emotional speech conversion method in the prior art is low, resulting in inaccurate results of emotional conversion.

[0007] The present invention solves the above technical problems through the following technical means: an arbitrary emotion speech conversion system based on mutual information minimization is applied to the training stage, including a training module, and the training module includes:

[0008] The emotion encoder is used to extract the emotion embedding representation in a sentence;

[0009] A non-emotional encoder, used to extract emotion-free representations of a sentence;

[0010] A decoder for reconstructing the transformed acoustic features from the emotion encoder and the emotion embedding representation and emotion-independent representation generated by the non-emotion encoder;

[0011] Variational approximation network, used to obtain the mean and variance of the approximate distribution of sentiment embedding representation, and then calculate the mutual information with emotion-independent representation;

[0012] The training control module is used to continuously and iteratively update the parameters of the emotion encoder and the non-emotion encoder, and the training is stopped when the mutual information calculated by the variational approximation network is minimized.

[0013] In order to solve the problem of incomplete separation of emotion-related information and emotion-irrelevant information, the present invention proposes an emotion speech conversion system based on mutual information minimization, which continuously iterates and updates the parameters of the emotion encoder and the non-emotion encoder, and stops training when the mutual information calculated by the variational approximation network is minimized. By minimizing the upper bound of the mutual information, the similarity between emotion-related information and emotion-irrelevant information can be reduced, thereby making the separation of the two information more in-depth and thorough, and the result of emotion conversion is more accurate.

[0014] Furthermore, in the training stage, four emotions of two males and two females in the multi-speaker emotional speech dataset ESD are selected, namely: happiness, sadness, neutrality, and anger, and 330 audio sentences are extracted for each emotion of each speaker as a training set.

[0015] Furthermore, each speaker is trained with neutral-to-happy, neutral-to-sad, and neutral-to-angry transitions during the training phase.

[0016] The present invention also provides an arbitrary emotional speech conversion system based on mutual information minimization, which is applied to the conversion stage and includes a conversion module, wherein the conversion module includes:

[0017] A first vocoder, for extracting acoustic features from the source speech and inputting them into a non-emotional encoder trained to minimize mutual information;

[0018] A second vocoder, used to extract acoustic features from the target speech and input them into an emotion encoder trained by minimizing mutual information;

[0019] A decoder for fusing the emotion-free representation of the source speech output by the non-emotional encoder and the emotion-embedded representation of the target speech output by the emotion encoder;

[0020] The third vocoder is used to receive the fusion result output by the decoder and output the converted speech.

[0021] Furthermore, the conversion stage selects four emotions of two males and two females in the multi-speaker emotional speech dataset ESD, namely: happiness, sadness, neutrality, and anger, and extracts 20 audio sentences for each emotion of each speaker as part of the test set; the four main emotions of four English speakers in the emotional speech dataset JL-Corpus: neutrality, happiness, anger, and sadness, each emotion has a total of 15 sentences recorded in parallel corpus, a total of 220 sentences, as another part of the test set; 30 sentences of parallel data containing the four basic emotions in the German performance corpus EMO-DB are selected as another part of the test set.

[0022] Furthermore, the first vocoder, the second vocoder and the third vocoder are WORLD vocoders.

[0023] Furthermore, the first vocoder extracts 24-dimensional Mel-frequency cepstral coefficients and fundamental frequencies in the source speech to form a 25-dimensional input vector and inputs the vector into a non-emotional encoder trained to minimize mutual information.

[0024] Furthermore, the second vocoder extracts 24-dimensional Mel-frequency cepstral coefficients and fundamental frequencies in the target speech to form a 25-dimensional input vector and inputs the vector into an emotion encoder trained by minimizing mutual information.

[0025] The present invention also provides an arbitrary emotional speech conversion system based on mutual information minimization, which is applied to the training stage and the conversion stage, and includes the training module and the conversion module described above.

[0026] The present invention also provides an arbitrary emotional speech conversion method based on mutual information minimization, the method comprising:

[0027] 1) Select multiple emotional speech data sets and divide them into training sets and test sets;

[0028] 2) Acoustic feature extraction;

[0029] 3) Training phase: The extracted acoustic features are input into the emotion encoder and the non-emotion encoder. The decoder reconstructs the converted acoustic features from the emotion encoder and the emotion embedding representation and the emotion-independent representation generated by the non-emotion encoder. The variational approximation network obtains the mean and variance of the approximate distribution of the emotion embedding representation, and then calculates the mutual information with the emotion-independent representation. The parameters of the emotion encoder and the non-emotion encoder are continuously updated iteratively. The training is stopped when the mutual information calculated by the variational approximation network is the smallest.

[0030] 4) Conversion stage: A source speech and a target speech are selected and input into the non-emotional encoder and the emotional encoder respectively after acoustic feature extraction. The decoder fuses the emotion-independent representation of the source speech output by the non-emotional encoder and the emotion-embedded representation of the target speech output by the emotional encoder.

[0031] The advantages of the present invention are:

[0032] (1) In order to solve the problem of incomplete separation of emotion-related information and emotion-irrelevant information, the present invention proposes an emotion speech conversion system based on mutual information minimization, which continuously iterates and updates the parameters of the emotion encoder and the non-emotion encoder, and stops training when the mutual information calculated by the variational approximation network is minimized. By minimizing the upper bound of the mutual information, the similarity between emotion-related information and emotion-irrelevant information can be reduced, thereby making the separation of the two information more in-depth and thorough, and the result of emotion conversion is more accurate.

[0033] (2) The present invention uses model training to completely separate emotion-related information from emotion-irrelevant information, thereby reducing the degree of information entanglement and being able to achieve the task of any emotion speech conversion. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A training framework diagram of an arbitrary emotion speech conversion system based on mutual information minimization provided in Example 1 of the present invention;

[0035] Figure 2 A conversion process framework diagram of an arbitrary emotional speech conversion system based on mutual information minimization provided by Embodiment 2 of the present invention;

[0036] Figure 3 The subjective opinion scoring interface provided by the present invention;

[0037] Figure 4 It is a statistical chart of subjective opinion score results;

[0038] Figure 5 The comparison selection scoring interface provided by the present invention;

[0039] Figure 6 Select the score result statistical graph for comparison. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Example 1

[0042] In the present invention, embodiments 1 to 4 all require the use of a corpus, and there are three corpora used.

[0043] First, the multi-language, multi-speaker emotional speech dataset ESD released by Nanyang Technological University, Singapore in 2020. The data of native English speakers were used, 2 males and 2 females, whose numbers in the ESD dataset are: 0011, 0012, 0017 and 0018 respectively. Four emotions in the dataset were selected, namely: 1) happy, 2) sad, 3) neutral, 4) angry. For each emotion of each speaker, there are 350 audio sentences, of which 330 sentences are used as training sets, and the remaining 20 sentences belong to the test set.

[0044] Second, JL-Corpus is an emotional speech dataset consisting of 4 English speakers and 10 emotions, which is commonly used in speech synthesis tasks of social robots. This dataset is selected by the present invention only as a test set to verify the robustness of the model. Like ESD, it uses the four main emotions in the dataset: 1) neutral, 2) happy, 3) angry, and 4) sad. Each emotion has 15 sentences recorded in parallel corpus. The test set contains: 4 (speakers) × 4 (main emotions) × 15 (sentences) = 240 sentences.

[0045] Third, the EMO-DB dataset is a German performance corpus recorded by the Technical University of Berlin. The dataset consists of 10 German speakers, 5 male speakers and 5 female speakers, all of whom are actors. After screening, there are 30 sentences of parallel data containing all 4 basic emotions, all of which are used as test sets.

[0046] The above training set is only used in the training phase, while the test set should be used in the conversion phase.

[0047] like Figure 1 As shown in the figure, E θ and D are the non-emotional encoder and decoder respectively; E e is an emotion encoder; C is an emotion embedding representation extracted from a certain sentence; Z is an information representation unrelated to emotion, and Q is a variational approximation network. Embodiment 1 of the present invention provides an arbitrary emotion speech conversion system based on mutual information minimization, which is applied to the training stage and includes a training module, wherein the training module includes:

[0048] Emotion encoder is used to extract the emotion embedding representation in a sentence; the emotion encoder is composed of a reference encoder and a global emotion embedding GEEs module. Global emotion embeddings (GEEs) are composed of a set of emotion token embeddings and multi-head attention modules.

[0049] The non-emotional encoder is used to extract emotion-independent representations from a sentence of speech; the speech information uses the data in the above training set.

[0050] A decoder for reconstructing the transformed acoustic features from the emotion encoder and the emotion embedding representation and emotion-independent representation generated by the non-emotion encoder;

[0051] Variational approximation network, used to obtain the mean and variance of the approximate distribution of sentiment embedding representation, and then calculate the mutual information with emotion-independent representation;

[0052] The training control module is used to continuously and iteratively update the parameters of the emotion encoder and the non-emotion encoder, and the training is stopped when the mutual information calculated by the variational approximation network is minimized.

[0053] It should be noted that for each speaker, training is performed on neutral to happy (N2H), neutral to sad (N2S), and neutral to angry (N2A), and the accuracy of model training is improved through multiple emotion conversions.

[0054] The purpose of training the model is to reduce the correlation between emotion-related representations and emotion-irrelevant representations, that is, the smaller the mutual information value between the two, the better. The specific method is to estimate the upper bound of the mutual information. When the upper bound estimate is smaller, it means that the similarity between the two variables is small, the emotion-related information and emotion-irrelevant information have a low degree of cross-mixing and mutual dependence, and the feature separation is more thorough. After a certain number of training steps, a model with a smaller loss function value can be obtained, which means that the training stage is completed.

[0055] Through the above technical scheme, the present invention aims at the problem of incomplete separation of emotion-related information and emotion-irrelevant information, and proposes an emotion speech conversion system based on mutual information minimization. The parameters of the emotion encoder and the non-emotion encoder are continuously updated iteratively, and the training is stopped when the mutual information calculated by the variational approximation network is minimized. By minimizing the upper bound of the mutual information, the similarity between emotion-related information and emotion-irrelevant information can be reduced, thereby making the separation of the two information more in-depth and thorough, and the result of emotion conversion is more accurate.

[0056] Example 2

[0057] like Figure 2 As shown, the difference between Embodiment 2 of the present invention and Embodiment 1 is that: the present invention further provides an arbitrary emotional speech conversion system based on mutual information minimization, which is applied to the conversion stage and includes a conversion module, and the conversion module includes:

[0058] A first vocoder, for extracting acoustic features from the source speech and inputting them into a non-emotional encoder trained to minimize mutual information;

[0059] The second vocoder is used to extract the acoustic features in the target speech and input them into the emotion encoder trained by minimizing the mutual information; in the present invention, the first to third vocoders used for feature extraction and waveform synthesis are all WORLD vocoders, and the WORLD vocoder needs to extract three acoustic features: fundamental frequency F0, spectrum envelope (spectrum envelope) also called spectrum parameter (Spectrum Parameter, SP), non-periodic signal parameter (aperiodic parameter, AP). The DIO algorithm is used to extract the fundamental frequency F0. The WORLD vocoder uses CheapTrick to perform spectral analysis to obtain the spectrum envelope. The D4C algorithm is used in the WORLD vocoder to extract AP. Among them, AP is only used in the conversion stage and does not participate in the training in the training stage. The WORLD vocoder belongs to the prior art and will not be described here. In actual application, the test data set is input into the WORLD vocoder. The WORLD vocoder extracts the 24-dimensional Mel-Cepstral coefficients MECP and fundamental frequency F0, connects them into a 25-dimensional input vector as input features, and inputs the input features into the emotion encoder and non-emotion encoder respectively.

[0060] A decoder for fusing the emotion-free representation of the source speech output by the non-emotional encoder and the emotion-embedded representation of the target speech output by the emotion encoder;

[0061] The third vocoder is used to receive the fusion result output by the decoder and output the converted speech.

[0062] Example 3

[0063] like Figure 1 and Figure 2 As shown, Example 3 of the present invention provides an arbitrary emotion speech conversion system based on mutual information minimization, which is applied to the training stage and the conversion stage, including the training module described in Example 1 and the conversion module described in Example 2. In actual applications, training is performed first, and speech conversion is performed after training to complete any emotion speech conversion task.

[0064] Example 4

[0065] like Figure 1 and Figure 2 As shown, the difference between Embodiment 4 of the present invention and Embodiment 3 is that the voice conversion method of the system is explained in detail based on the system provided in Embodiment 3, and the method includes:

[0066] 1) Select multiple emotional speech data sets and divide them into training sets and test sets; for the specific data set selection method, refer to the relevant introduction in Example 1.

[0067] 2) Acoustic feature extraction: As described in Example 2, the WORLD vocoder is used to extract acoustic features, and after the extraction, the information is input into the emotion encoder and the non-emotion encoder.

[0068] 3) Training phase: The extracted acoustic features are input into the emotion encoder and the non-emotion encoder. The decoder reconstructs the converted acoustic features from the emotion embedding representation and the emotion-independent representation generated by the emotion encoder and the non-emotion encoder. The variational approximation network obtains the mean and variance of the approximate distribution of the emotion embedding representation, and then calculates the mutual information with the emotion-independent representation. The parameters of the emotion encoder and the non-emotion encoder are continuously updated iteratively. The training is stopped when the mutual information calculated by the variational approximation network is minimized. By minimizing the upper bound of the mutual information, the similarity between emotion-related information and emotion-independent information can be reduced, thereby making the separation of the two information more in-depth and thorough. In this way, in the subsequent conversion process, the emotion representation content will not be retained in the non-emotional information of the source speech. When it is fused with the emotion information of the target emotion speech for emotion conversion, the conversion result only contains the non-emotional information of the source speech and the emotion information of the target speech, so that the result of the emotion conversion is accurate, and it is possible to arbitrarily select a source emotion speech and a target emotion speech, keep the non-emotional information of the source speech unchanged, and extract the emotion information from the target speech.

[0069] 4) Conversion stage: A source speech and a target speech are selected and input into the non-emotional encoder and the emotional encoder respectively after acoustic feature extraction. The decoder fuses the emotion-independent representation of the source speech output by the non-emotional encoder and the emotion-embedded representation of the target speech output by the emotional encoder to obtain the converted acoustic features. The converted acoustic features are output into converted speech through the third vocoder, so that a source emotional speech and a target emotional speech can be arbitrarily selected, the non-emotional information of the source speech is retained unchanged, and the emotional information is extracted from the target speech.

[0070] The two encoder models of the present invention are generative models constructed based on variational autoencoders. First, emotion-related information is extracted through an emotion encoder based on global emotion embeddings (GEEs), and emotion-irrelevant representations are extracted through a non-emotion encoder with activation guidance (AG) as an information bottleneck. Then, in the decoder, the emotion information is embedded in the converted speech through an adaptive instance normalization layer (AdaIN), and finally reconstructed into a waveform through a vocoder.

[0071] In order to verify the effect of the present invention, an experiment was conducted, and the experimental contents are described in detail below:

[0072] The present invention firstly adopts an objective evaluation to measure the degree of feature separation of the model. A simple emotion classification network is used to judge the degree of separation of the emotion-independent representation Z. If the classification accuracy obtained by inputting Z into the emotion classifier is lower, it means that the information contains less emotion information, which proves that the model has a higher degree of separation of emotion-related information and emotion-independent information. The classifier is composed of three layers of 256 hidden nodes of Conv1d one-dimensional convolution layer and a fully connected layer, which is a commonly used setting. The input of the classifier is the emotion-independent representation Z extracted by the model, and the output is the classification accuracy. The data used for comparison is all the data of four speakers and four emotions selected in the data set ESD. Comparing the model with mutual information minimization (that is, the model of the present invention) with the model without adding this method, the classification accuracy is reduced by 3%, which means that the degree of feature separation is deepened by 3%, proving the effectiveness of the system and method in the degree of feature separation.

[0073] The experimental results are also evaluated subjectively, including subjective opinion score (MOS) and comparative choice score, which measure the naturalness and emotional similarity of the converted speech.

[0074] The subjective opinion score of the present invention is 1-5 points, 1 represents the worst, 5 represents the best, and the higher the score, the better the naturalness of the speech. Figure 3 A subjective evaluation scoring interface is designed, such as Figure 4 FIG. 4 is a comparison of the subjective opinion scores of the conversion speech quality of the model of the present invention and the benchmark model. Figure 4 In the three bar charts of N2H, N2S and N2A, from left to right are the results of two benchmark models and the model of the present invention. For example, in the N2H case, 3.29 and 3.05 are the scores of the two benchmark models, and 4.67 is the score of the model of the present invention. The results show that the model proposed by the present invention scores higher than the benchmark models, and the speech quality generated by the model of the present invention is slightly better than the two benchmark models. This shows that the system-converted speech of the present invention has good sound quality and naturalness, and has effective speech emotion conversion capability.

[0075] The method for comparing and selecting scores of the present invention adopts Figure 5 The scoring interface is designed to give the test takers a reference audio and two conversion audios. The test takers are asked to choose an option based on the question "Please choose which of the two audio'semotion you think is more like the reference audio?" and the three options "audio A", "audio B", and "no difference". Figure 6The XAB scoring results of the model and VAWGAN-EVC are shown, including neutral to happy (N2H), neutral to sad (N2S), and neutral to angry (N2A). For the first two, the model-converted speech proposed by the present invention is more similar to the reference speech. Anger is visible to the baseline model, but it is an invisible emotion in the model of the present invention, so the high proportion of "No Difffference" in N2A is caused, which shows that compared with VAWGAN-EVC, although anger is an invisible emotion, the model still has the conversion ability that can match it. Therefore, in this experiment, both visible emotion conversion and invisible emotion conversion are better than the baseline model. These experimental results show that the model of the present invention also has a good performance in emotional similarity, and can even achieve the conversion of any emotion.

[0076] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An arbitrary emotional speech conversion system based on mutual information minimization, characterized in that: Applied in the training phase, including a training module, the training module includes: The emotion encoder is used to extract the emotion embedding representation in a sentence; A non-emotional encoder, used to extract emotion-free representations of a sentence; A decoder for reconstructing the transformed acoustic features from the emotion encoder and the emotion embedding representation and emotion-independent representation generated by the non-emotion encoder; Variational approximation network, used to obtain the mean and variance of the approximate distribution of sentiment embedding representation, and then calculate the mutual information with emotion-independent representation; The training control module is used to continuously and iteratively update the parameters of the emotion encoder and the non-emotion encoder, and the training is stopped when the mutual information calculated by the variational approximation network is minimized.

2. The arbitrary emotion speech conversion system based on mutual information minimization according to claim 1 is characterized in that: In the training stage, four emotions of two males and two females in the multi-speaker emotional speech dataset ESD are selected, namely: happiness, sadness, neutrality, and anger. 330 audios are extracted for each emotion of each speaker as a training set.

3. The arbitrary emotion speech conversion system based on mutual information minimization according to claim 2 is characterized in that: During the training phase, each speaker was trained on neutral-to-happy, neutral-to-sad, and neutral-to-angry transitions.

4. An arbitrary emotional speech conversion system based on mutual information minimization, characterized in that: Applied in the conversion stage, including a conversion module, the conversion module includes: A first vocoder, for extracting acoustic features from the source speech and inputting them into a non-emotional encoder trained to minimize mutual information; The second vocoder is used to extract acoustic features from the target speech and input them into the emotion encoder trained by mutual information minimization; mutual information minimization training refers to continuously iteratively updating the parameters of the emotion encoder and the non-emotion encoder, calculating the mutual information between the emotion embedding representation and the emotion-independent representation, and stopping the training when the calculated mutual information is the smallest, thereby obtaining the emotion encoder trained by mutual information minimization and the non-emotion encoder trained by mutual information minimization; A decoder for fusing the emotion-free representation of the source speech output by the non-emotional encoder and the emotion-embedded representation of the target speech output by the emotion encoder; The third vocoder is used to receive the fusion result output by the decoder and output the converted speech.

5. The arbitrary emotion speech conversion system based on mutual information minimization according to claim 4 is characterized in that: The conversion stage selects four emotions of two males and two females from the multi-speaker emotional speech dataset ESD, namely: happy, sad, neutral, and angry, and extracts 20 audio sentences for each emotion of each speaker as part of the test set. The four main emotions of four English speakers from the emotional speech dataset JL-Corpus: neutral, happy, angry, and sad, each emotion has a total of 15 sentences recorded in parallel corpus, a total of 220 sentences, as another part of the test set; 30 sentences of parallel data containing the four basic emotions in the German performance corpus EMO-DB are selected as another part of the test set.

6. The arbitrary emotion speech conversion system based on mutual information minimization according to claim 4 is characterized in that: The first vocoder, the second vocoder and the third vocoder are WORLD vocoders.

7. The arbitrary emotion speech conversion system based on mutual information minimization according to claim 4 is characterized in that: The first vocoder extracts 24-dimensional Mel-frequency cepstral coefficients and fundamental frequencies from the source speech to form a 25-dimensional input vector and inputs the vector into a non-emotional encoder trained by minimizing mutual information.

8. The arbitrary emotion speech conversion system based on mutual information minimization according to claim 4 is characterized in that: The second vocoder extracts 24-dimensional Mel-frequency cepstral coefficients and fundamental frequencies in the target speech to form a 25-dimensional input vector and inputs the vector into an emotion encoder trained by minimizing mutual information.

9. An arbitrary emotional speech conversion system based on mutual information minimization, characterized in that: Applicable to the training stage and the conversion stage, comprising the training module described in any one of claims 1-3 and the conversion module described in any one of claims 4-8.

10. A method for converting arbitrary emotional speech based on mutual information minimization, characterized in that: The method comprises: 1) Select multiple emotional speech data sets and divide them into training sets and test sets; 2) Acoustic feature extraction; 3) Training phase: The extracted acoustic features are input into the emotion encoder and the non-emotion encoder. The decoder reconstructs the converted acoustic features from the emotion encoder and the emotion embedding representation and the emotion-independent representation generated by the non-emotion encoder. The variational approximation network obtains the mean and variance of the approximate distribution of the emotion embedding representation, and then calculates the mutual information with the emotion-independent representation. The parameters of the emotion encoder and the non-emotion encoder are continuously updated iteratively. The training is stopped when the mutual information calculated by the variational approximation network is the smallest. 4) Conversion stage: A source speech and a target speech are selected and input into the non-emotional encoder and the emotional encoder respectively after acoustic feature extraction. The decoder fuses the emotion-independent representation of the source speech output by the non-emotional encoder and the emotion-embedded representation of the target speech output by the emotional encoder.

Images