Voice editing method and device based on controllable emotion intensity

Through the speech editing method based on the emotional intensity perception neural network model, the intonation and rhythm are dynamically adjusted, and the problems of insufficient emotional expression and lack of naturalness in the existing technology are solved, and the authentic and diverse expression of emotional audio is achieved.

CN118942489APending Publication Date: 2024-11-12SOUTHEAST UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411174686.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Existing emotional voice editing technologies are difficult to achieve real and diverse emotional expressions, especially in terms of emotional intensity control, and the degree of emotional intensity cannot be adjusted according to specific needs, which limits the expressiveness and diversity of generated audio.

Method used

Using a method based on the emotional intensity perception neural network model, emotional audio data is generated by obtaining original sample data, extracting intonation and rhythm characteristics, and combining emotional categories and intensity numerical dynamic adjustment of the intonation and rhythm changes of the sentence. The model includes a posterior network, intensity mapper, prior network and decoder, using a speaker personality holder, emotion evaluation unit, text encoder, and emotion rendering unit, through density local outlier factor algorithm and minimum-maximum value Scaling for emotional intensity processing.

Benefits of technology

It achieves more accurate dynamic adjustment of the tone and rhythm, improves the richness and naturalness of emotional expression, and can flexibly adjust the intensity of emotions in the generated audio according to specific needs, achieving more diverse emotional expressions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942489A_ABST
    Figure CN118942489A_ABST
Patent Text Reader

Abstract

The invention discloses a voice editing method and device based on controllable emotion intensity, and relates to the technical field of voice signal processing, and the method comprises the steps: obtaining original sample data which comprises audio data, text data and emotion category labels; converting the text data into an audio sequence, and converting the audio data into a Mel spectrogram; the Mel spectrogram is input into a posterior network of an emotion intensity perception neural network model, intonation features are extracted and obtained, and the intonation features comprise fundamental frequency information of a speaker and numerical values of titer, wakeup degree and dominance degree; inputting the numerical values of the audio sequence, the titer, the awakening degree and the dominating degree into a prior network of an emotion intensity perception neural network model to obtain content and rhythm features; and inputting the content, the rhythm features and the intonation features into a decoder of a pre-established emotional intensity perception neural network model, outputting to obtain potential intermediate features, and generating emotional audio data according to the potential intermediate features.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of speech signal processing, and in particular to a speech editing method and device based on controllable emotion intensity. Background Art

[0002] Emotional speech editing aims to improve the naturalness and emotional expression of speech by adjusting and converting the emotional elements in speech. This technology is of great significance in enhancing the human-computer interaction experience, improving the realism of virtual characters, and improving the emotional expressiveness of auditory entertainment products. However, achieving realistic and diverse emotional expressions is a very challenging task.

[0003] At present, emotional speech editing technology is mainly based on the autoencoder framework, especially the sequence-to-sequence (seq2seq) model. This type of model achieves emotional conversion by learning the mapping relationship between input speech and target emotional speech. However, these models often face problems such as insufficient emotional expression and lack of naturalness in practical applications. Especially in terms of emotional intensity control, the existing technology has obvious limitations. Most emotional speech editing systems can only achieve conversion of fixed emotional categories and lack fine control over emotional intensity. For example, when expressing anger, it is difficult to adjust the intensity of anger according to specific needs, which limits the expressiveness and diversity of the generated audio. Summary of the invention

[0004] In order to solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide a speech editing method and device based on controllable emotion intensity, which dynamically adjusts the intonation and rhythm changes of the sentence according to the emotion category and intensity value, so that the edited speech is more realistic and natural.

[0005] In a first aspect, the purpose of the present invention can be achieved by the following technical solution: a speech editing method based on controllable emotion intensity, the method comprising the following steps:

[0006] Acquire original sample data, wherein the original sample data includes audio data, text data and emotion category labels; convert the text data into an audio sequence, and convert the audio data into a mel-spectrogram;

[0007] The mel-spectrogram is input into a posterior network of a pre-established emotion intensity perception neural network model to extract intonation features, wherein the intonation features include speaker fundamental frequency information and values ​​of valence, arousal, and dominance; the emotion intensity perception neural network model includes a posterior network, an intensity mapper, a priori network, and a decoder;

[0008] The audio sequence and the values ​​of valence, arousal and dominance are input into the prior network of the pre-established emotion intensity perception neural network model, and the content and rhythm features are output;

[0009] The content, rhythm features and intonation features are input into the decoder of a pre-established emotion intensity perception neural network model, and potential intermediate features are output, and emotional audio data is generated based on the potential intermediate features.

[0010] In combination with the first aspect, in certain implementations of the first aspect, the method also includes: the posterior network of the pre-established emotion intensity perception neural network model includes a speaker personality retainer and an emotion evaluation unit, the speaker personality retainer is used to extract the speaker fundamental frequency information from the mel-spectrogram, and the emotion evaluation unit is used to extract the valence, arousal, and dominance values ​​contained in the mel-spectrogram.

[0011] In combination with the first aspect, in certain implementations of the first aspect, the method also includes: the speaker personality preserver includes a plurality of one-dimensional convolution modules connected in sequence and a fully connected module, wherein each convolution module includes three operations of convolution, normalization, and nonlinear activation function in sequence, and each fully connected module includes 1 fully connected operation.

[0012] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: during the training process, the intensity mapper generates an emotion intensity pseudo-label based on the valence, arousal, and dominance; during the reasoning process, the corresponding valence, arousal, and dominance are generated by specifying the target emotion category and intensity value.

[0013] In combination with the first aspect, in some implementations of the first aspect, the method further includes: a calculation process of the intensity mapper:

[0014] Obtain the valence, arousal, and dominance values ​​extracted by the emotion assessment unit:

[0015] e ki =(v, a, d)

[0016] Among them, e ki represents the i-th coordinate / corpus data of the k-th emotion category, v represents the valence value, a represents the arousal value, and d represents the dominance value;

[0017] The density-based local outlier factor algorithm LOF is used to calculate and remove outliers in the valence, arousal, and dominance values ​​of each emotion category:

[0018] e′ ki =(v′, a′, d′)=e ki -M where M=LOF(e n )

[0019] Among them, M represents the valence, arousal, and dominance values ​​of the center point of the neutral corpus cluster, e n represents neutral corpus, v′ represents valence value, a′ represents arousal value, d′ represents dominance value, e′ ki Represents the i-th coordinate / corpus data of the k-th sentiment category after correction;

[0020] The unnormalized sentiment intensity value r is calculated according to the following formula:

[0021]

[0022] According to the principle of Min-Max Scaling, the emotional intensity value of each emotional category is normalized and mapped to the interval [0,1] to obtain the final emotional intensity value r norm :

[0023]

[0024] Among them, r min Represents the minimum value of the emotional intensity value under the same emotional category, r max Indicates the maximum value of the sentiment intensity under the same sentiment category.

[0025] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the prior network of the pre-established emotion intensity perception neural network model includes:

[0026] A text encoder, used for extracting linguistic features from the audio sequence, comprising a plurality of Transformer modules connected in sequence and a linear mapping module;

[0027] The emotion rendering unit is used to process the linguistic features into the content and rhythm features according to the valence, arousal and dominance, and includes a plurality of two-dimensional convolution modules and a linear mapping module connected in sequence.

[0028] In combination with the first aspect, in certain implementations of the first aspect, the method also includes: the content and rhythm features include a pronunciation mean and a pronunciation variance of a Gaussian distribution, the intonation features include a rhythm mean and a rhythm variance of a Gaussian distribution, and the potential intermediate features include an intermediate feature mean and an intermediate feature variance of a Gaussian distribution.

[0029] In combination with the first aspect, in certain implementations of the first aspect, the method also includes: the decoder includes a plurality of transposed convolution operations and a multi-receptive field fusion module connected in sequence, wherein the multi-receptive field fusion module includes a one-dimensional convolution, a nonlinear activation function and a residual structure connected in sequence.

[0030] In a second aspect, in order to achieve the above-mentioned object, the present invention discloses a speech editing device based on controllable emotion intensity, comprising:

[0031] A data conversion module is used to obtain original sample data, wherein the original sample data includes audio data, text data and emotion category labels; convert the text data into an audio sequence, and convert the audio data into a mel-spectrogram;

[0032] A posterior extraction module is used to input the mel-spectrogram into the posterior network of the pre-established emotion intensity perception neural network model to extract intonation features, wherein the intonation features include speaker fundamental frequency information and valence, arousal and dominance values; the emotion intensity perception neural network model includes a posterior network, an intensity mapper, a priori network and a decoder;

[0033] A priori extraction module is used to input the audio sequence and the values ​​of valence, arousal and dominance into the prior network of the pre-established emotion intensity perception neural network model, and output the content and rhythm features;

[0034] The feature decoding module is used to input content, rhythm features and intonation features into the decoder of a pre-established emotion intensity perception neural network model, output potential intermediate features, and generate emotional audio data based on the potential intermediate features.

[0035] In another aspect of the present invention, in order to achieve the above-mentioned purpose, a terminal device is disclosed, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the memory stores a computer program capable of running on the processor, and when the processor loads and executes the computer program, a speech editing method based on controllable emotional intensity as described above is adopted.

[0036] Beneficial effects of the present invention:

[0037] The present invention can more accurately and dynamically adjust the intonation and rhythm, making the emotional expression richer while maintaining the fluency and naturalness of the speech. In addition, with the help of the intensity mapper, the present invention can flexibly adjust the intensity of the emotions contained in the generated audio according to specific needs, thereby achieving more diverse emotional expressions. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1It is a schematic flow chart of the method of the present invention;

[0040] Figure 2 A flow chart of a voice editing method based on controllable emotion intensity provided by an embodiment of the present invention;

[0041] Figure 3 A structural diagram of a speech editing device based on controllable emotion intensity provided by an embodiment of the present invention;

[0042] Figure 4 A structural diagram of a voice editing device based on controllable emotion intensity provided by an embodiment of the present invention;

[0043] Figure 5 Mel-spectrogram of angry emotion-category speech synthesized by the present invention;

[0044] Figure 6 A mel-spectrogram of speech of the happy emotion category synthesized by the present invention;

[0045] Figure 7 Mel-spectrogram of sad emotion speech synthesized by the present invention;

[0046] Figure 8 A mel-spectrogram of speech of the surprise emotion category synthesized by the present invention;

[0047] Fig. 9 The pitch and energy trajectory diagram corresponding to the mel spectrogram of the synthesized speech of the present invention;

[0048] Fig.10 Synthesize speech mel spectrograms with different emotional intensities of the same text content for the present invention;

[0049] Fig.11 It is a schematic diagram of the structure of the device of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Embodiment 1:

[0052] The following is an introduction to the relevant terms involved in the embodiments of the present application:

[0053] Mel-spectrogram: As a widely used tool in speech signal processing and audio analysis, it is designed based on the characteristics of the human auditory system. It is a two-dimensional representation extracted from the audio signal. It can show how the frequency components of the audio change over time and visualize the human ear's perception of different frequencies in speech.

[0054] like Figure 1 As shown, a speech editing method based on controllable emotion intensity comprises the following steps:

[0055] S101: Acquire original sample data, wherein the original sample data includes audio data, text data, and emotion category labels; convert the text data into an audio sequence, and convert the audio data into a mel-spectrogram;

[0056] The steps of converting audio data into a Mel spectrogram specifically include: performing pre-processing operations such as pre-emphasis, framing, and windowing on the denoised speech signal; performing short-time discrete Fourier transform on the preprocessed speech signal to obtain a spectrum; taking the square of the complex spectrum of each frame to obtain a power spectrum; passing the power spectrum through a group of Mel filters, each filter being equally spaced on the Mel scale; taking the logarithm of the output value of the Mel filter group to obtain a Mel spectrum. In this embodiment, framing, windowing, short-time discrete Fourier transform, and filter group all use the Librosa open source speech signal processing library, the filter window function window length is set to 1024, the sampling rate is set to 16kHz, and the inter-frame overlap rate is 25%.

[0057] S102: inputting the mel-spectrogram into a posterior network of a pre-established emotion intensity perception neural network model to extract intonation features, wherein the intonation features include speaker fundamental frequency information and values ​​of valence, arousal, and dominance; the emotion intensity perception neural network model includes a posterior network, an intensity mapper, a priori network, and a decoder;

[0058] The posterior network of the pre-established emotion intensity perception neural network model includes a speaker personality retainer and an emotion evaluation unit. The speaker personality retainer is used to extract the speaker's fundamental frequency information from the mel spectrogram, and the emotion evaluation unit is used to extract the valence, arousal, and dominance values ​​contained in the mel spectrogram according to the mel spectrogram, wherein the valence represents the positive and negative tendency of the emotion, the arousal represents the degree of emotional excitation, and the dominance represents the speaker's control over the current state. The speaker personality retainer includes a plurality of convolution modules connected in sequence and a fully connected module, wherein each convolution module includes three operations of 1×1 one-dimensional convolution, normalization, and nonlinear activation function (RELU) in sequence, and each fully connected module includes 1 fully connected operation. The emotion evaluation unit includes a connected valence-arousal-dominance identifier and a linear mapping module, the target mapping dimension is 192 dimensions, and the valence-arousal-dominance identifier is a HuBERT model that has been pre-trained on multiple emotional speech databases and has certain generalization and robustness. The intonation features include a prosodic mean and a prosodic variance of a Gaussian distribution.

[0059] During the training process, the intensity mapper generates an emotion intensity pseudo label according to the valence, arousal, and dominance; during the inference process, the corresponding valence, arousal, and dominance are generated by specifying the target emotion category and intensity value.

[0060] The calculation process of the intensity mapper:

[0061] Obtain the valence, arousal, and dominance values ​​extracted by the emotion assessment unit:

[0062] e ki =(v,a,d)

[0063] Among them, e ki represents the i-th coordinate / corpus data of the k-th emotion category, v represents the valence value, a represents the arousal value, and d represents the dominance value;

[0064] The density-based local outlier factor algorithm LOF is used to calculate and remove outliers in the valence, arousal, and dominance values ​​of each emotion category:

[0065] e′ ki =(v′, a′, d′)=e ki -M where M=LOF(e n )

[0066] Among them, M represents the valence, arousal, and dominance values ​​of the center point of the neutral corpus cluster, e n represents neutral corpus, v' represents valence value, a' represents arousal value, d' represents dominance value, e' kiRepresents the i-th coordinate / corpus data of the k-th sentiment category after correction;

[0067] The unnormalized sentiment intensity value r is calculated according to the following formula:

[0068]

[0069] According to the principle of Min-Max Scaling, the emotional intensity value of each emotional category is normalized and mapped to the interval [0,1] to obtain the final emotional intensity value r norm :

[0070]

[0071] Among them, r min Represents the minimum value of the emotional intensity value under the same emotional category, r max Indicates the maximum value of the sentiment intensity under the same sentiment category.

[0072] S103: inputting the audio sequence and the values ​​of valence, arousal and dominance into a priori network of a pre-established emotion intensity perception neural network model, and outputting content and rhythm features;

[0073] The prior network models the content information contained in the audio sequence through a text encoder, generates rhythm features corresponding to the content information according to the valence, arousal, and dominance through an emotional rendering unit, and adaptively predicts the change in speech duration. The text encoder includes a plurality of sequentially connected Transformer modules and a linear mapping module, and the emotional rendering unit includes a plurality of sequentially connected convolution modules and a linear mapping module. The content and rhythm features include the pronunciation mean and pronunciation variance of the Gaussian distribution.

[0074] S104: Input the content, rhythm features and intonation features into the decoder of the pre-established emotion intensity perception neural network model, output the potential intermediate features, and generate emotional audio data according to the potential intermediate features.

[0075] The decoder matches the content with the rhythm features and intonation features, synthesizes potential intermediate features, and generates emotional audio data based on the potential intermediate features. The potential intermediate features include the intermediate feature mean and the intermediate feature variance of the Gaussian distribution. The decoder includes a plurality of transposed convolution operations and a multi-receptive field fusion module connected in sequence, wherein the multi-receptive field fusion module includes a deeply separable one-dimensional convolution, a nonlinear activation function (Leaky ReLU) and a residual structure. The speech detector is specifically a fully connected layer that can identify whether the audio is artificial speech. The emotion classifier is specifically a fully connected layer that classifies the audio emotion.

[0076] Specifically, the present invention is further described below by way of embodiments:

[0077] In order to more intuitively demonstrate the effect of the present invention in generating speech of different emotion categories, we have performed mel-spectrogram visualization processing on speech of emotion categories such as anger, happiness, sadness and surprise generated under the same text content, such as Figures 5 to 8 As shown. In these mel-spectrograms, the horizontal axis represents time, the vertical axis represents frequency, and the color represents the speech energy intensity of the frequency at that time point. It can be seen that when the emotion category changes, the harmonic structure of the generated speech remains consistent as a whole, indicating that the text content is well preserved and has a high degree of content naturalness. At the same time, the energy intensity at different frequencies shows significant differences, indicating that the modeling of emotional information is accurate and delicate, and the generated speech has natural rhythmic changes. In addition, these mel-spectrograms are clear and distinct, indicating that the audio quality of the generated speech is high and has high intelligibility. In order to demonstrate the effect of the present invention in controlling emotion intensity, we visualized the pitch and energy trajectory of the vocal part of the test audio, as shown Fig. 9 As shown in Figure 2, it can be seen that as the intensity value increases, the pitch fluctuation range expands and the energy peak value increases accordingly. In addition, the speech mel spectrograms generated under different emotional intensities for the same text content are visualized, as shown in Figure 2. Fig.10 It can be seen that as the intensity value increases, the acoustic changes become more obvious, accompanied by more short pauses, which shows that the present invention can adaptively adjust the emotional state in the speech based on the controllable emotional intensity, thereby achieving better results in intonation and rhythm generation.

[0078] Embodiment 2: In a second aspect, in order to achieve the above-mentioned purpose, the present invention discloses a voice editing device based on controllable emotion intensity, comprising:

[0079] The data conversion module 11 is used to obtain original sample data, wherein the original sample data includes audio data, text data and emotion category labels; convert the text data into an audio sequence, and convert the audio data into a mel-spectrogram;

[0080] The posterior extraction module 12 is used to input the mel-spectrogram into the posterior network of the pre-established emotion intensity perception neural network model to extract intonation features, wherein the intonation features include the speaker's fundamental frequency information and the values ​​of valence, arousal and dominance; the emotion intensity perception neural network model includes a posterior network, an intensity mapper, a priori network and a decoder;

[0081] A priori extraction module 13 is used to input the audio sequence and the values ​​of valence, arousal and dominance into the priori network of the pre-established emotion intensity perception neural network model, and output the content and rhythm features;

[0082] The feature decoding module 14 is used to input the content, rhythm features and intonation features into the decoder of the pre-established emotion intensity perception neural network model, output the potential intermediate features, and generate emotional audio data according to the potential intermediate features.

[0083] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors, and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is used to implement one or more instructions, specifically for loading and executing one or more instructions in a computer storage medium to implement the above method.

[0084] It needs to be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to execute the above method. The storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0085] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0086] The above shows and describes the basic principles, main features and advantages of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present disclosure. Without departing from the spirit and scope of the present disclosure, the present disclosure may have various changes and improvements, and these changes and improvements fall within the scope of the present disclosure to be protected.

Claims

1. A speech editing method based on controllable emotion intensity, It is characterized in that The method comprises the following steps: Acquire original sample data, wherein the original sample data includes audio data, text data and emotion category labels; convert the text data into an audio sequence, and convert the audio data into a mel-spectrogram; The mel-spectrogram is input into a posterior network of a pre-established emotion intensity perception neural network model to extract intonation features, wherein the intonation features include speaker fundamental frequency information and values ​​of valence, arousal, and dominance; the emotion intensity perception neural network model includes a posterior network, an intensity mapper, a priori network, and a decoder; The audio sequence and the values ​​of valence, arousal and dominance are input into the prior network of the pre-established emotion intensity perception neural network model, and the content and rhythm features are output; The content, rhythm features and intonation features are input into the decoder of a pre-established emotion intensity perception neural network model, and potential intermediate features are output, and emotional audio data is generated based on the potential intermediate features.

2. A speech editing method based on controllable emotion intensity according to claim 1, It is characterized in that The posterior network of the pre-established emotion intensity perception neural network model includes a speaker personality retainer and an emotion evaluation unit. The speaker personality retainer is used to extract the speaker fundamental frequency information from the Mel-spectrogram, and the emotion evaluation unit is used to extract the valence, arousal, and dominance values ​​contained in the Mel-spectrogram.

3. A speech editing method based on controllable emotion intensity according to claim 2, It is characterized in that The speaker personality retainer includes a plurality of one-dimensional convolution modules connected in sequence and a fully connected module, wherein each convolution module includes three operations of convolution, normalization, and nonlinear activation function in sequence, and each fully connected module includes one fully connected operation.

4. A speech editing method based on controllable emotion intensity according to claim 1, It is characterized in that During the training process, the intensity mapper generates an emotion intensity pseudo label according to the valence, arousal, and dominance; during the inference process, the corresponding valence, arousal, and dominance are generated by specifying the target emotion category and intensity value.

5. A speech editing method based on controllable emotion intensity according to claim 4, It is characterized in that The calculation process of the intensity mapper: Obtain the valence, arousal, and dominance values ​​extracted by the emotion assessment unit: e ki =(v,a,d) Among them, e ki represents the i-th coordinate / corpus data of the k-th emotion category, v represents the valence value, a represents the arousal value, and d represents the dominance value; The density-based local outlier factor algorithm LOF is used to calculate and remove outliers in the valence, arousal, and dominance values ​​of each emotion category: e′ ki =(v′,a′,d′)=e ki -M where M=LOF(e n ) Among them, M represents the valence, arousal, and dominance values ​​of the center point of the neutral corpus cluster, e n represents neutral corpus, v′ represents valence value, a′ represents arousal value, d′ represents dominance value, e′ ki Represents the i-th coordinate / corpus data of the k-th sentiment category after correction; The unnormalized sentiment intensity value r is calculated according to the following formula: According to the principle of Min-Max Scaling, the emotional intensity value of each emotional category is normalized and mapped to the interval [0, 1] to obtain the final emotional intensity value r norm : Among them, r min Represents the minimum value of the emotional intensity value under the same emotional category, r max Indicates the maximum value of the sentiment intensity under the same sentiment category.

6. A speech editing method based on controllable emotion intensity according to claim 1, It is characterized in that The prior network of the pre-established emotion intensity perception neural network model includes: A text encoder, used for extracting linguistic features from the audio sequence, comprising a plurality of Transformer modules connected in sequence and a linear mapping module; The emotion rendering unit is used to process the linguistic features into the content and rhythm features according to the valence, arousal and dominance, and includes a plurality of two-dimensional convolution modules and a linear mapping module connected in sequence.

7. A speech editing method based on controllable emotion intensity according to claim 1, It is characterized in that The content and rhythm features include a pronunciation mean and a pronunciation variance of a Gaussian distribution, the intonation features include a rhythm mean and a rhythm variance of a Gaussian distribution, and the potential intermediate features include an intermediate feature mean and an intermediate feature variance of a Gaussian distribution.

8. A speech editing method based on controllable emotion intensity according to claim 1, It is characterized in that The decoder includes a plurality of transposed convolution operations and a multi-receptive field fusion module connected in sequence, wherein the multi-receptive field fusion module includes a one-dimensional convolution, a nonlinear activation function and a residual structure connected in sequence.

9. A voice editing device based on controllable emotional intensity, It is characterized in that include: A data conversion module is used to obtain original sample data, wherein the original sample data includes audio data, text data and emotion category labels; convert the text data into an audio sequence, and convert the audio data into a mel-spectrogram; A posterior extraction module is used to input the mel-spectrogram into the posterior network of the pre-established emotion intensity perception neural network model to extract intonation features, wherein the intonation features include speaker fundamental frequency information and valence, arousal and dominance values; the emotion intensity perception neural network model includes a posterior network, an intensity mapper, a priori network and a decoder; A priori extraction module is used to input the audio sequence and the values ​​of valence, arousal and dominance into the prior network of the pre-established emotion intensity perception neural network model, and output the content and rhythm features; The feature decoding module is used to input content, rhythm features and intonation features into the decoder of a pre-established emotion intensity perception neural network model, output potential intermediate features, and generate emotional audio data based on the potential intermediate features.

10. A terminal device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, It is characterized in that The memory stores a computer program that can be run on the processor. When the processor loads and executes the computer program, a speech editing method based on controllable emotion intensity described in any one of claims 1 to 8 is adopted.