Voice conversion method, voice conversion device, electronic device, and storage medium

Abstract

The embodiments of the present application provide a speech conversion method, a speech conversion device, an electronic device, and a storage medium, which belong to the field of artificial intelligence technology. The speech conversion method includes: obtaining an initial speech; extracting a filter group energy feature of the initial speech; performing feature extraction on the filter group energy feature to obtain a speech style feature corresponding to the initial speech; performing feature extraction on the initial speech to obtain a speech pitch feature corresponding to the initial speech; performing feature synthesis on the speech pitch feature to obtain a first intermediate speech; performing feature encoding on the first intermediate speech to obtain a text feature corresponding to the first intermediate speech; performing feature fusion on the speech style feature, the speech pitch feature, and the text feature to obtain a mixed feature; performing feature decoding on the mixed feature to obtain a target speech. The generated target speech is related to speaker characteristics such as the speech style feature and the speech pitch feature, and can provide personalized customized speech to enhance the effect of speech conversion.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a speech conversion method, a speech conversion device, an electronic device, and a storage medium. Background Art

[0002] In the related art, videos are dubbed by manual or machine synthesis. However, the manual method not only has certain professional requirements for dubbing technology, but also has low dubbing efficiency. The voice obtained by the machine synthesis method is machine sound, the voice conversion effect is poor, and it is impossible to provide personalized customized voice. Summary of the Invention

[0003] The main purpose of the embodiments of the present application is to provide a voice conversion method, a voice conversion device, an electronic device, and a storage medium that can enhance the effect of voice conversion.

[0004] To achieve the above objectives, a first aspect of an embodiment of the present application provides a voice conversion method, comprising:

[0005] Get the initial voice;

[0006] Extracting filter bank energy features of the initial speech;

[0007] Extracting the energy features of the filter bank to obtain a speech style feature corresponding to the initial speech;

[0008] Extracting features of the initial speech to obtain speech tone features corresponding to the initial speech;

[0009] Performing feature synthesis on the speech tone features to obtain a first intermediate speech;

[0010] performing feature encoding on the first intermediate speech to obtain text features corresponding to the first intermediate speech;

[0011] Fusing the speech style feature, the speech tone feature, and the text feature to obtain a hybrid feature;

[0012] The mixed features are subjected to feature decoding to obtain target speech.

[0013] In some embodiments, extracting the filter bank energy feature of the initial speech includes:

[0014] performing pre-emphasis processing on the initial speech to obtain a second intermediate speech;

[0015] performing frame processing on the second intermediate speech to obtain a third intermediate speech;

[0016] performing windowing processing on the third intermediate speech to obtain a fourth intermediate speech;

[0017] performing a discrete Fourier transform on the fourth intermediate speech to obtain a frequency spectrum corresponding to the fourth intermediate speech;

[0018] Obtaining a power spectrum corresponding to the fourth intermediate speech according to the frequency spectrum;

[0019] Mel filtering is performed on the power spectrum according to a preset Mel filter bank to obtain the filter bank energy feature.

[0020] In some embodiments, extracting the filter bank energy features to obtain the speech style features corresponding to the initial speech includes:

[0021] Inputting the filter bank energy features into the neural network model, performing feature extraction on the filter bank energy features, and obtaining a plurality of first activation features output by a target hidden layer of the neural network model;

[0022] Performing regularization processing on the first activation feature to obtain a second activation feature;

[0023] Accumulating and calculating the plurality of second activation features to obtain a third activation feature;

[0024] An average value of the third activation feature is calculated to obtain an average activation feature, and the average activation feature is used as the speech style feature.

[0025] In some embodiments, extracting features from the initial speech to obtain speech tone features corresponding to the initial speech includes:

[0026] Inputting the initial speech into a vocoder, performing feature extraction on the initial speech, and obtaining fundamental frequency, spectrum envelope, and non-periodic parameters corresponding to the initial speech;

[0027] The speech pitch feature is obtained according to the fundamental frequency, the spectrum envelope and the non-periodic parameter.

[0028] In some embodiments, performing feature synthesis on the speech tone features to obtain the first intermediate speech includes:

[0029] Calculating an average value of the fundamental frequencies to obtain an average fundamental frequency;

[0030] Perform feature synthesis on the average fundamental frequency, the spectrum envelope, and the non-periodic parameters to obtain a first intermediate speech.

[0031] In some embodiments, the performing feature encoding on the first intermediate speech to obtain text features corresponding to the first intermediate speech includes:

[0032] inputting the first intermediate speech into an encoder, performing feature encoding on the first intermediate speech, and obtaining speech coding features corresponding to the first intermediate speech;

[0033] The speech coding feature is subtracted from the speech style feature to obtain a text feature corresponding to the first intermediate speech.

[0034] In some embodiments, the encoder includes a convolutional layer and a bidirectional long short-term memory layer, and inputting the first intermediate speech into the encoder, performing feature encoding on the first intermediate speech, and obtaining speech coding features corresponding to the first intermediate speech includes:

[0035] Inputting the first intermediate speech into an encoder, and performing feature extraction on the first intermediate speech through the convolution layer to obtain speech convolution features;

[0036] The speech convolution feature is extracted through the bidirectional long short-term memory layer to obtain the speech coding feature.

[0037] A second aspect of the embodiments of the present application provides a speech conversion device, comprising:

[0038] An acquisition module, used to acquire initial speech;

[0039] A first feature extraction module, configured to extract filter bank energy features of the initial speech;

[0040] A second feature extraction module is used to extract the energy features of the filter bank to obtain the speech style features corresponding to the initial speech;

[0041] a third feature extraction module, configured to extract features from the initial speech to obtain speech tone features corresponding to the initial speech;

[0042] a feature synthesis module, configured to perform feature synthesis on the speech tone feature to obtain a first intermediate speech;

[0043] a feature encoding module, configured to perform feature encoding on the first intermediate speech to obtain text features corresponding to the first intermediate speech;

[0044] A feature fusion module, configured to fuse the speech style feature, the speech tone feature, and the text feature to obtain a mixed feature;

[0045] The feature decoding module is used to perform feature decoding on the mixed features to obtain the target speech.

[0046] A third aspect of the embodiments of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a program, and when the program is executed by the processor, the processor is used to execute the speech conversion method as described in any one of the embodiments of the first aspect of the present application.

[0047] The fourth aspect of the embodiments of the present application proposes a storage medium, which is a computer-readable storage medium. The storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the speech conversion method as described in any one of the embodiments of the first aspect of the present application.

[0048] The speech conversion method, speech conversion device, electronic device, and storage medium proposed in the embodiments of the present application obtain an initial speech, extract the filter bank energy characteristics of the initial speech, perform feature extraction on the filter bank energy characteristics to obtain a speech style characteristic corresponding to the initial speech, perform feature extraction on the initial speech to obtain a speech pitch characteristic corresponding to the initial speech, perform feature synthesis on the speech pitch characteristics to obtain a first intermediate speech, perform feature encoding on the first intermediate speech to obtain a text characteristic corresponding to the first intermediate speech, perform feature fusion on the speech style characteristics, speech pitch characteristics, and text characteristics to obtain a mixed characteristic, and perform feature decoding on the mixed characteristic to obtain a target speech. In the embodiments of the present application, the speech style characteristics are used to characterize the speaker's timbre information, the speech pitch characteristics are used to characterize the speaker's pitch information, and the text characteristics are used to characterize the text content information of the initial speech. Speaker characteristics are obtained based on the timbre information and pitch information. The speaker characteristics and text characteristics are fused and feature decoded to automatically generate a target speech. The generated speech is related to the speaker characteristics, and can provide personalized customized speech to enhance the effect of speech conversion. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a first flow chart of the voice conversion method provided by an embodiment of the present application;

[0050] Figure 2 yes Figure 1 Flowchart of step S120 in FIG.

[0051] Figure 3 yes Figure 1 Flowchart of step S130 in FIG.

[0052] Figure 4 yes Figure 1 Flowchart of step S140 in FIG.

[0053] Figure 5 yes Figure 1 Flowchart of step S150 in FIG.

[0054] Figure 6 yes Figure 1 Flowchart of step S160 in FIG.

[0055] Figure 7 yes Figure 6 Flowchart of step S610 in FIG.

[0056] Figure 8 This is a second flow chart of the voice conversion method provided in an embodiment of the present application;

[0057] Figure 9 This is a block diagram of the module structure of the voice conversion device provided in an embodiment of the present application;

[0058] Figure 10 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0060] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0062] In addition, the described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present disclosure.

[0063] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0064] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0065] First, let’s analyze some of the terms used in this application:

[0066] Artificial Intelligence (AI) is a new technical discipline that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. A branch of computer science, AI seeks to understand the essence of intelligence and create new intelligent machines that can respond in a manner similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thinking. It also encompasses theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0067] Filter bank energy feature (Fbank): a frequency domain feature based on cepstrum extraction.

[0068] Deep Neural Network (DNN): A neural network with multiple hidden layers that can fit almost any function and has strong nonlinear fitting capabilities.

[0069] Activation functions are crucial for artificial neural network models to learn and understand complex and nonlinear functions. They introduce nonlinearity into the network. In neurons, inputs are weighted, summed, and then applied to a function called the activation function. Activation functions are introduced to increase the nonlinearity of neural network models.

[0070] Encoder: Encoding converts the input sequence into a fixed-length vector; Decoder converts the previously generated fixed vector into an output sequence. The input sequence can be text, speech, image, or video; the output sequence can be text or image.

[0071] In the related art, videos are dubbed by manual or machine synthesis. However, the manual method not only has certain professional requirements for dubbing technology, but also has low dubbing efficiency. The voice obtained by the machine synthesis method is machine sound, the voice conversion effect is poor, and it is impossible to provide personalized customized voice.

[0072] Based on this, the embodiments of the present application propose a speech conversion method, a speech conversion device, an electronic device, and a storage medium. By decoupling the speech style features, speech pitch features, and text features, the speaker's timbre information is expressed according to the speech style features, the speaker's pitch information is expressed according to the speech pitch features, and the text content information is expressed according to the text features. The speaker's characteristics can be determined through the timbre information and the pitch information, and the speaker characteristics and text features are feature fused and feature decoded, so that the generated target speech is related to the speaker characteristics, and personalized customized speech can be provided to enhance the effect of speech conversion.

[0073] The speech conversion method, speech conversion device, electronic device, and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the speech conversion method in the embodiments of the present application is described.

[0074] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0075] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0076] The speech conversion method provided in the embodiment of the present application relates to the field of artificial intelligence. The speech conversion method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, or a smart watch, etc.; the server side can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the speech conversion method, etc., but is not limited to the above forms.

[0077] Embodiments of the present application can be used in numerous general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0078] Reference Figure 1 According to the first aspect of the embodiment of the present application, the voice conversion method includes but is not limited to steps S110 to S180.

[0079] Step S110, obtaining initial speech;

[0080] Step S120, extracting the filter bank energy features of the initial speech;

[0081] Step S130, performing feature extraction on the filter bank energy feature to obtain the speech style feature corresponding to the initial speech;

[0082] Step S140, extracting features of the initial speech to obtain speech tone features corresponding to the initial speech;

[0083] Step S150, performing feature synthesis on the speech tone features to obtain a first intermediate speech;

[0084] Step S160, performing feature encoding on the first intermediate speech to obtain text features corresponding to the first intermediate speech;

[0085] Step S170, fusing the speech style feature, the speech tone feature, and the text feature to obtain a hybrid feature;

[0086] Step S180: Decode the mixed features to obtain the target speech.

[0087] In step S110 of some embodiments, an initial speech is acquired, where the initial speech is a speaker's speech.

[0088] In step S120 of some embodiments, the speaker's speech is divided into frames, and the filter bank energy feature of each frame is extracted, where the filter bank energy feature is the FBank feature.

[0089] In step S130 of some embodiments, in order to obtain the speaker's timbre information, feature extraction is performed on the FBank features to obtain the speaker's voice style features, wherein the voice style features are used to characterize the speaker's timbre information.

[0090] In step S140 of some embodiments, the speech pitch is related to the identity of the speaker. In order to make the target speech generated by speech conversion closer to the speaker's speech, feature extraction is performed on the initial speech to obtain speech pitch features, wherein the speech pitch features are used to represent the speaker's pitch information.

[0091] In step S150 of some embodiments, to smooth the fundamental frequency characteristics of the initial speech, the speech pitch characteristics are synthesized to obtain a first intermediate speech, where the first intermediate speech is the smoothed initial speech. Compared to the initial speech, the text content and timbre information of the first intermediate speech remain unchanged, while the fundamental frequency information is smoothly eliminated.

[0092] In step S160 of some embodiments, in order to obtain the text content information of the first intermediate speech, feature encoding is performed on the first intermediate speech to obtain text features corresponding to the first intermediate speech, wherein the text features are used to represent the text content information of the first intermediate speech.

[0093] In step S170 of some embodiments, if the speech style feature is represented as y1, the speech tone feature is represented as y2, the text feature is represented as y3, and the mixed feature is represented as y, the speech style feature, the speech tone feature and the text feature are fused based on the adder to obtain the mixed feature, that is, y=y1+y2+y3.

[0094] In step S180 of some embodiments, since the mixed features include speaker features such as voice style features and voice pitch features, the mixed features are feature decoded to obtain the target voice, so that the target voice generated by the voice conversion is closer to the sound made by the speaker, thereby enhancing the effect of the voice conversion.

[0095] The speech conversion method of the embodiment of the present application obtains an initial speech, extracts the filter group energy characteristics of the initial speech, performs feature extraction on the filter group energy characteristics to obtain a speech style characteristic corresponding to the initial speech, performs feature extraction on the initial speech to obtain a speech pitch characteristic corresponding to the initial speech, performs feature synthesis on the speech pitch characteristic to obtain a first intermediate speech, performs feature encoding on the first intermediate speech to obtain a text characteristic corresponding to the first intermediate speech, fuses the speech style characteristics, speech pitch characteristics and text characteristics to obtain a mixed characteristic, and performs feature decoding on the mixed characteristic to obtain a target speech. In the embodiment of the present application, the speech style characteristics are used to characterize the speaker's timbre information, the speech pitch characteristics are used to characterize the speaker's pitch information, and the text characteristics are used to characterize the text content information of the initial speech. Speaker characteristics are obtained based on the timbre information and pitch information. The speaker characteristics and text characteristics are feature fused and feature decoded to automatically generate a target speech. The generated speech is related to the speaker characteristics, and can provide personalized customized speech to enhance the effect of speech conversion.

[0096] In some embodiments, as Figure 2 As shown, step S120 specifically includes but is not limited to steps S210 to S260.

[0097] Step S210, performing pre-emphasis processing on the initial speech to obtain a second intermediate speech;

[0098] Step S220, performing frame processing on the second intermediate speech to obtain a third intermediate speech;

[0099] Step S230, performing windowing processing on the third intermediate speech to obtain a fourth intermediate speech;

[0100] Step S240, performing a discrete Fourier transform on the fourth intermediate speech to obtain a frequency spectrum corresponding to the fourth intermediate speech;

[0101] Step S250, obtaining a power spectrum corresponding to the fourth intermediate speech according to the frequency spectrum;

[0102] Step S260 , performing Mel filtering on the power spectrum according to a preset Mel filter bank to obtain a filter bank energy feature.

[0103] In step S210 of some embodiments, in order to enhance the high frequency portion of the initial speech, a first-order high-pass filter is used to perform pre-emphasis processing on the initial speech to obtain a second intermediate speech.

[0104] In step S220 of some embodiments, the frame length and sampling rate are determined, half of the frame length value is used as the frame shift, and the second intermediate speech is framed according to the frame shift, frame length and sampling rate to obtain the third intermediate speech, wherein the third intermediate speech is a speech frame obtained by framing the second intermediate speech.

[0105] In step S230 of some embodiments, the third intermediate speech is multiplied by a window function to obtain a fourth intermediate speech, where the window function may be a rectangular window function, a Hamming window function, a Hanning window function, or the like.

[0106] In step S250 of some embodiments, the power spectrum may be obtained by taking the absolute value of the spectrum or by squaring the spectrum.

[0107] In steps S210 to S260 of some embodiments, the initial speech is pre-emphasized using a first-order high-pass filter to obtain a second intermediate speech, the frame length and sampling rate are determined, half of the frame length is used as a frame shift, the second intermediate speech is framed according to the frame length, frame shift, and sampling rate to obtain a third intermediate speech, a window function is determined, the window function is multiplied by the third intermediate speech to perform windowing processing on the third intermediate speech to obtain a fourth intermediate speech, the fourth intermediate speech is discrete Fourier transformed to obtain a spectrum, the absolute value of the spectrum or the square of the spectrum is used as a power spectrum, the power spectrum is Mel-filtered using a Mel filter group to obtain an energy parameter, the energy parameter is logarithmically transformed to obtain a filter group energy feature, so that the speech style feature can be subsequently obtained based on the filter group energy feature, thereby obtaining the speaker's timbre information.

[0108] In some embodiments, as Figure 3 As shown, step S130 specifically includes but is not limited to steps S310 to S340.

[0109] Step S310: Inputting the filter bank energy features into the neural network model, performing feature extraction on the filter bank energy features, and obtaining a plurality of first activation features output by the target hidden layer of the neural network model;

[0110] Step S320, performing regularization processing on the first activation feature to obtain a second activation feature;

[0111] Step S330, performing cumulative calculation on the plurality of second activation features to obtain a third activation feature;

[0112] Step S340 , calculating the average value of the third activation feature to obtain an average activation feature, and using the average activation feature as the speech style feature.

[0113] In step S310 of some embodiments, the filter bank energy features corresponding to all speech frames are input into a deep neural network model. Feature extraction of the filter bank energy features is performed based on the deep neural network model, and the activation value output by the last hidden layer of the deep neural network model is used as the first activation feature, wherein the hidden layer is a fully connected maxout hidden layer, and the activation function of the hidden layer is the maxout function. The deep neural network model includes multiple hidden layers. When performing a feedforward propagation calculation, if the current hidden layer includes k nodes, the features x1, x2, x3 input to the first node, the weights w1, w2, w3 corresponding to each feature, and the biases b1, b2, b3 are obtained, and the biases, features, and weights are weighted to obtain the activation value output by the first node. If the activation value is represented by x, the activation value calculated based on the biases, features, and weights is x = w1x1 + w2x2 + w3x3 + b1 + b2 + b3. By analogy, k activation values ​​output by k nodes are obtained, and the maximum value is selected from the k activation values ​​as the input feature of a node in the next hidden layer, until the activation value output by each node of the last hidden layer can be obtained. The multiple activation values ​​output by the last hidden layer are used as the first activation features of a speech frame. Each speech frame has its own corresponding first activation feature, thereby obtaining multiple first activation features of all speech frames.

[0114] In step S320 of some embodiments, in order to prevent overfitting, L2 regularization is performed on the first activation feature to update the weights of the hidden layer nodes, and weighted calculation is performed on the updated weights, biases and input features to obtain the second activation feature corresponding to the speech frame.

[0115] In step S330 of some embodiments, the second activation features of all speech frames are accumulated and calculated to obtain a third activation feature.

[0116] In step S340 of some embodiments, the frame number of all speech frames is obtained, an average activation feature is obtained according to the ratio of the third activation feature to the frame number, and the average activation feature is used as the speech style feature.

[0117] In steps S310 to S340 of some embodiments, the filter group energy features of each speech frame are input into the DNN, feature extraction is performed on the filter group energy features based on the DNN, the activation value output by the last hidden layer of the DNN is used as the first activation feature of the speech frame, the first activation feature is L2 regularized to obtain the second activation feature, the second activation feature of each speech frame is accumulated to obtain the third activation feature, the number of frames of all speech frames is obtained, the ratio of the third activation feature to the frame number is used as the average activation feature, and the speech style feature is obtained based on the average activation feature to obtain the speaker's timbre information.

[0118] In some embodiments, as Figure 4 As shown, step S140 specifically includes but is not limited to steps S410 to S420.

[0119] Step S410: Input the initial speech into the vocoder, perform feature extraction on the initial speech, and obtain the fundamental frequency, spectrum envelope, and non-periodic parameters corresponding to the initial speech;

[0120] Step S420: obtaining speech pitch features according to the fundamental frequency, spectrum envelope and non-periodic parameters.

[0121] In step S410 of some embodiments, the vocoder includes a vocoder analyzer and a vocoder synthesizer. The vocoder analyzer includes three modules: a fundamental frequency extraction module, a spectrum envelope extraction module, and an aperiodic parameter extraction module. All speech frames of the initial speech are input to the vocoder analyzer. The fundamental frequency extraction module performs feature extraction on the speech frames to obtain the fundamental frequency corresponding to the speech frames. The fundamental frequency and speech frames are input to the spectrum envelope extraction module to obtain the spectrum envelope corresponding to the speech frames. The fundamental frequency, speech frames, and spectrum envelope are input to the aperiodic parameter extraction module to obtain the aperiodic parameters corresponding to the speech frames.

[0122] When extracting the fundamental frequency, the initial speech is low-pass filtered using low-pass filters with different cutoff frequencies to obtain multiple filtered signals. The peak interval, trough interval, upward zero-crossing interval, and downward zero-crossing interval of the filtered signals within a preset interval are calculated. If the peak interval, trough interval, upward zero-crossing interval, and downward zero-crossing interval are all equal, then the filtered signal within the preset interval is the fundamental frequency. The frequency of the fundamental frequency is used as the candidate fundamental frequency, and the confidence of the candidate fundamental frequency is calculated. The candidate fundamental frequency with the highest confidence is used as the target fundamental frequency. It should be noted that the embodiment of the present application uses standard deviation to measure confidence. The smaller the standard deviation, the higher the confidence.

[0123] The spectrum envelope can be obtained by connecting the highest amplitude points of different frequencies through a smooth curve. When extracting the spectrum envelope, the first window function is selected as the Hanning window, the inverse of the target fundamental frequency is used as the fundamental frequency period, the first window length of the first window function is determined according to the fundamental frequency period, and the power spectrum of the speech frame under the Hanning window with a first window length of three fundamental frequency periods is calculated. The second window function is selected as the rectangular window, and the second window length of the second window function is determined according to the fundamental frequency period. The power spectrum is filtered using the rectangular window with the second window length to obtain a smoothed power spectrum, and the smoothed power spectrum is enhanced in the inverse frequency domain to obtain the spectrum envelope.

[0124] It should be noted that if the pitch period is expressed as T0, the second window length is

[0125] When extracting non-periodic parameters, determine the third window function, obtain the fundamental frequency period according to the inverse of the fundamental frequency, use the two fundamental frequency periods as the third window length of the third window function, perform logarithmic transformation on the spectrum envelope to obtain the logarithmic spectrum envelope, perform inverse Fourier transformation on the logarithmic spectrum envelope to obtain the inverse function, construct the minimum phase function according to the inverse function, perform Fourier transformation on the minimum phase function to obtain the minimum phase spectrum, and use the ratio of the third window function of the third window length to the minimum phase spectrum as the non-periodic parameter.

[0126] In step S420 of some embodiments, the fundamental frequency, spectrum envelope and non-periodic parameters are used as speech pitch features.

[0127] In steps S410 to S420 of some embodiments, in order to obtain speech pitch features and facilitate real-time speech conversion, all speech frames of the initial speech are input into the world vocoder, and feature extraction is performed on the speech frames based on the world vocoder to obtain the fundamental frequency, spectrum envelope and non-periodic parameters corresponding to the speech frames. The fundamental frequency, spectrum envelope and non-periodic parameters of all speech frames are used as the speech pitch features of the initial speech. The entire feature extraction process is based on the world vocoder and does not involve the training of the neural network, which can improve the speed of speech pitch feature extraction.

[0128] In some embodiments, as Figure 5 As shown, step S150 specifically includes but is not limited to steps S510 to S520.

[0129] Step S510, calculating the average value of the fundamental frequency to obtain the average fundamental frequency;

[0130] Step S520 , performing feature synthesis on the average fundamental frequency, the spectrum envelope, and the non-periodic parameters to obtain a first intermediate speech.

[0131] In step S510 of some embodiments, the fundamental frequencies of all speech frames are added together to obtain a fundamental frequency sum, and the average fundamental frequency is obtained by dividing the fundamental frequency sum by the total number of speech frames.

[0132] In step S520 of some embodiments, feature synthesis is performed on the average fundamental frequency, the spectrum envelope, and the non-periodic parameters according to a synthesis algorithm of a vocoder to obtain a first intermediate speech.

[0133] In steps S510 to S520 of some embodiments, in order to smooth the fundamental frequency in the initial speech, the average fundamental frequency is used to replace the fundamental frequency, and the average fundamental frequency, spectrum envelope and non-periodic parameters are feature synthesized based on the world vocoder to obtain the first intermediate speech.

[0134] In some embodiments, as Figure 6 As shown, step S160 specifically includes but is not limited to steps S610 to S620.

[0135] Step S610: Input the first intermediate speech into an encoder, perform feature encoding on the first intermediate speech, and obtain speech coding features corresponding to the first intermediate speech;

[0136] Step S620: Subtract the speech coding feature from the speech style feature to obtain the text feature corresponding to the first intermediate speech.

[0137] In step S610 of some embodiments, in order to achieve decoupling of fundamental frequency, voice style features and text features and enhance the effect of voice conversion, the fundamental frequency of the initial voice is first smoothed to obtain a first intermediate voice, so that compared with the initial voice, the timbre information and phoneme information of the first intermediate voice remain unchanged, while the fundamental frequency information disappears smoothly. Then, the first intermediate voice is input into the encoder, and the first intermediate voice is feature-encoded based on the encoder to obtain voice coding features, wherein the voice coding features are continuous vectors including phoneme information and speaker timbre information, wherein the phoneme information is the text content information of the initial voice.

[0138] In step S620 of some embodiments, in order to separate the phoneme information and the speaker's timbre information in the speech coding features, the speech coding features are subtracted from the speech style features representing the speaker's timbre information to obtain the phoneme information of the first intermediate speech, i.e., the text features, where the text features are discrete vectors.

[0139] In steps S610 to S620 of some embodiments, in order to obtain the text content information of the initial audio, the first intermediate speech is input into the encoder, feature encoding is performed on the first intermediate speech to obtain speech coding features corresponding to the first intermediate speech, and the speech coding features are subtracted from the speech style features to obtain text features corresponding to the first intermediate speech.

[0140] In some embodiments, as Figure 7 As shown, the encoder includes a convolutional layer and a bidirectional long short-term memory layer, and step S610 specifically includes but is not limited to steps S710 to S720.

[0141] Step S710: Input the first intermediate speech into an encoder, and perform feature extraction on the first intermediate speech through a convolutional layer to obtain speech convolution features;

[0142] Step S720: extract the speech convolution features through a bidirectional long short-term memory layer to obtain speech coding features.

[0143] In step S710 of some embodiments, the encoder includes three convolution layers, and the first convolution layer performs feature extraction on the first intermediate speech to obtain the first speech convolution feature, the second convolution layer performs feature extraction on the first speech convolution feature to obtain the second speech convolution feature, and the third convolution layer performs feature extraction on the second speech convolution feature to obtain the third speech convolution feature.

[0144] In step S720 of some embodiments, the encoder includes two layers of bidirectional long short-term memory layers, and feature extraction is performed on the third speech convolution feature according to the first bidirectional long short-term memory layer to obtain a first speech coding feature, and feature extraction is performed on the first speech coding feature according to the second bidirectional long short-term memory layer to obtain a second speech coding feature.

[0145] It should be noted that the decoder is an inverse function of the encoder parameters. The decoder structure is opposite to the encoder parameters, but the structure is similar. The decoder also includes three convolutional layers and two bidirectional long short-term memory layers.

[0146] Specifically, the second speech coding feature is subtracted from the speech style feature to obtain the text feature corresponding to the first intermediate speech, the speech style feature, the speech pitch feature and the text feature are fused to obtain a mixed feature, the mixed feature is input into the decoder, and the mixed feature is feature decoded based on the decoder to obtain the target speech.

[0147] Reference Figure 8 Another embodiment of the present application proposes a voice conversion method for video dubbing, including but not limited to steps S810 to S870.

[0148] Step S810, obtaining initial speech;

[0149] Step S820: performing a first feature extraction on the initial speech to obtain a filter bank energy feature, and performing a second feature extraction on the filter bank energy feature based on a deep neural network to obtain a d-vector vector;

[0150] Step S830, performing a third feature extraction on the initial speech based on the world vocoder to obtain the fundamental frequency, spectrum envelope and non-periodic parameters;

[0151] Step S840, calculating the average value of the fundamental frequency, and performing feature synthesis on the average value, the spectrum envelope, and the non-periodic parameters based on the world vocoder to obtain synthesized speech;

[0152] Step S850, performing feature encoding on the synthesized speech based on the encoder to obtain text features corresponding to the synthesized speech;

[0153] Step S860: performing feature fusion on the d-vector, fundamental frequency, and text features based on an adder to obtain a mixed feature;

[0154] Step S870: Decode the mixed features based on the decoder to obtain the target speech.

[0155] It should be noted that the fundamental frequency is also called the pitch parameter, which is related to the speaker's identity and can be used to characterize the speaker's voice tone information. By introducing the pitch parameter, the target speech can be closer to the sound of the speaker.

[0156] It should be further explained that, if the target speech is represented as x' and the initial speech is represented as x, the loss function for the speech conversion task in this embodiment of the application is defined as: L = ||xx'||1. The loss function of the speech conversion model is calculated based on the target speech to obtain a loss value. This loss value is used as the backpropagation amount to adjust the model parameters of the deep neural network, encoder, and decoder to train the deep neural network, encoder, and decoder to obtain the speech conversion model.

[0157] By executing steps S810 to S870, a variety of personalized customized voices can be generated by simply obtaining the lines recorded by the video producer as the initial voice.

[0158] The present application also provides a voice conversion device, such as Figure 9 As shown, the above-mentioned speech conversion method can be implemented. The device includes an acquisition module 910, a first feature extraction module 920, a second feature extraction module 930, a third feature extraction module 940, a feature synthesis module 950, a feature encoding module 960, a feature fusion module 970, and a feature decoding module 980. The acquisition module 910 is used to acquire the initial speech; the first feature extraction module 920 is used to extract the filter bank energy feature of the initial speech; the second feature extraction module 930 is used to perform feature extraction on the filter bank energy feature to obtain the speech style feature corresponding to the initial speech; the third feature extraction module 940 is used to perform feature extraction on the initial speech to obtain the speech pitch feature corresponding to the initial speech; the feature synthesis module 950 is used to perform feature synthesis on the speech pitch feature to obtain a first intermediate speech; the feature encoding module 960 is used to perform feature encoding on the first intermediate speech to obtain the text feature corresponding to the first intermediate speech; the feature fusion module 970 is used to perform feature fusion on the speech style feature, the speech pitch feature, and the text feature to obtain a mixed feature; and the feature decoding module 980 is used to perform feature decoding on the mixed feature to obtain the target speech.

[0159] The speech conversion device of the embodiment of the present application is used to execute the speech conversion method in the above embodiment. Its specific processing process is the same as the speech conversion method in the above embodiment, and will not be repeated here.

[0160] The speech conversion device of the embodiment of the present application obtains an initial speech, extracts the filter bank energy characteristics of the initial speech, performs feature extraction on the filter bank energy characteristics to obtain a speech style characteristic corresponding to the initial speech, performs feature extraction on the initial speech to obtain a speech pitch characteristic corresponding to the initial speech, performs feature synthesis on the speech pitch characteristics to obtain a first intermediate speech, performs feature encoding on the first intermediate speech to obtain a text characteristic corresponding to the first intermediate speech, fuses the speech style characteristics, speech pitch characteristics, and text characteristics to obtain a mixed characteristic, and performs feature decoding on the mixed characteristic to obtain a target speech. In the embodiment of the present application, the speech style characteristics are used to characterize the speaker's timbre information, the speech pitch characteristics are used to characterize the speaker's pitch information, and the text characteristics are used to characterize the text content information of the initial speech. Speaker characteristics are obtained based on the timbre information and pitch information. The speaker characteristics and text characteristics are feature fused and feature decoded to automatically generate a target speech. The generated speech is related to the speaker characteristics, and can provide personalized customized speech to enhance the effect of speech conversion.

[0161] An embodiment of the present application further provides an electronic device, including:

[0162] at least one processor, and

[0163] a memory communicatively connected to at least one processor; wherein,

[0164] The memory stores instructions, which are executed by at least one processor so that when the at least one processor executes the instructions, a speech conversion method as described in any one of the embodiments of the first aspect of the present application is implemented.

[0165] The electronic device provided in the embodiment of the present application obtains an initial speech, extracts the filter group energy characteristics of the initial speech, performs feature extraction on the filter group energy characteristics to obtain a speech style characteristic corresponding to the initial speech, performs feature extraction on the initial speech to obtain a speech pitch characteristic corresponding to the initial speech, performs feature synthesis on the speech pitch characteristic to obtain a first intermediate speech, performs feature encoding on the first intermediate speech to obtain a text characteristic corresponding to the first intermediate speech, fuses the speech style characteristics, speech pitch characteristics and text characteristics to obtain a mixed characteristic, and performs feature decoding on the mixed characteristic to obtain a target speech. In the embodiment of the present application, the speech style characteristics are used to characterize the speaker's timbre information, the speech pitch characteristics are used to characterize the speaker's pitch information, and the text characteristics are used to characterize the text content information of the initial speech. Speaker characteristics are obtained based on the timbre information and pitch information. The speaker characteristics and text characteristics are feature fused and feature decoded to automatically generate a target speech. The generated speech is related to the speaker characteristics, and can provide personalized customized speech to enhance the effect of speech conversion.

[0166] The following combination Figure 10 The hardware structure of the electronic device is described in detail. The electronic device includes: a processor 1010 , a memory 1020 , an input / output interface 1030 , a communication interface 1040 , and a bus 1050 .

[0167] The processor 1010 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0168] The memory 1020 can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device, or RAM (Random Access Memory). The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called by the processor 1010 to execute the speech conversion method of the embodiments of this application.

[0169] Input / output interface 1030, used to implement information input and output;

[0170] Communication interface 1040, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.); and

[0171] bus 1050 , which transmits information between various components of the device (e.g., processor 1010 , memory 1020 , input / output interface 1030 , and communication interface 1040 );

[0172] The processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 are connected to each other in communication within the device via a bus 1050 .

[0173] An embodiment of the present application further provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the speech conversion method of the embodiment of the present application.

[0174] The storage medium provided in the embodiment of the present application obtains an initial speech, extracts the filter group energy characteristics of the initial speech, performs feature extraction on the filter group energy characteristics to obtain a speech style characteristic corresponding to the initial speech, performs feature extraction on the initial speech to obtain a speech pitch characteristic corresponding to the initial speech, performs feature synthesis on the speech pitch characteristic to obtain a first intermediate speech, performs feature encoding on the first intermediate speech to obtain a text characteristic corresponding to the first intermediate speech, fuses the speech style characteristics, speech pitch characteristics and text characteristics to obtain a mixed characteristic, and performs feature decoding on the mixed characteristic to obtain a target speech. In the embodiment of the present application, the speech style characteristics are used to characterize the speaker's timbre information, the speech pitch characteristics are used to characterize the speaker's pitch information, and the text characteristics are used to characterize the text content information of the initial speech. Speaker characteristics are obtained based on the timbre information and pitch information. The speaker characteristics and text characteristics are feature fused and feature decoded to automatically generate a target speech. The generated speech is related to the speaker characteristics, and can provide personalized customized speech to enhance the effect of speech conversion.

[0175] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0176] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0177] It will be understood by those skilled in the art that Figures 1 to 8 The technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figures, or a combination of certain steps, or different steps.

[0178] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0179] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0180] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0181] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0182] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0183] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0184] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0185] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes multiple instructions for enabling an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0186] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A voice conversion method, characterized in that: The method comprises: Get the initial voice; Extracting filter bank energy features of the initial speech; Extracting the energy features of the filter bank to obtain a speech style feature corresponding to the initial speech; Extracting features of the initial speech to obtain speech tone features corresponding to the initial speech; Performing feature synthesis on the speech tone features to obtain a first intermediate speech; performing feature encoding on the first intermediate speech to obtain text features corresponding to the first intermediate speech; Fusing the speech style feature, the speech tone feature, and the text feature to obtain a hybrid feature; Performing feature decoding on the mixed features to obtain target speech; The extracting features of the initial speech to obtain speech tone features corresponding to the initial speech includes: Inputting the initial speech into a vocoder, performing feature extraction on the initial speech to obtain a fundamental frequency, a spectrum envelope, and aperiodic parameters corresponding to the initial speech; and obtaining the speech pitch feature based on the fundamental frequency, the spectrum envelope, and the aperiodic parameters; The step of synthesizing the speech tone features to obtain a first intermediate speech includes: An average value of the fundamental frequency is calculated to obtain an average fundamental frequency; and feature synthesis is performed on the average fundamental frequency, the spectrum envelope, and the non-periodic parameter to obtain a first intermediate speech.

2. The voice conversion method according to claim 1, wherein: The step of extracting the filter bank energy feature of the initial speech comprises: performing pre-emphasis processing on the initial speech to obtain a second intermediate speech; performing frame processing on the second intermediate speech to obtain a third intermediate speech; performing windowing processing on the third intermediate speech to obtain a fourth intermediate speech; performing a discrete Fourier transform on the fourth intermediate speech to obtain a frequency spectrum corresponding to the fourth intermediate speech; Obtaining a power spectrum corresponding to the fourth intermediate speech according to the frequency spectrum; Mel filtering is performed on the power spectrum according to a preset Mel filter bank to obtain the filter bank energy feature.

3. The voice conversion method according to claim 1, wherein: The extracting the energy features of the filter bank to obtain the speech style features corresponding to the initial speech includes: Inputting the filter bank energy features into a neural network model, performing feature extraction on the filter bank energy features, and obtaining a plurality of first activation features output by a target hidden layer of the neural network model; Performing regularization processing on the first activation feature to obtain a second activation feature; Accumulating and calculating the plurality of second activation features to obtain a third activation feature; An average value of the third activation feature is calculated to obtain an average activation feature, and the average activation feature is used as the speech style feature.

4. The voice conversion method according to any one of claims 1 to 3, characterized in that: The performing feature encoding on the first intermediate speech to obtain text features corresponding to the first intermediate speech includes: inputting the first intermediate speech into an encoder, performing feature encoding on the first intermediate speech, and obtaining speech coding features corresponding to the first intermediate speech; The speech coding feature is subtracted from the speech style feature to obtain a text feature corresponding to the first intermediate speech.

5. The voice conversion method according to claim 4, characterized in that: The encoder includes a convolutional layer and a bidirectional long short-term memory layer. Inputting the first intermediate speech into the encoder, performing feature encoding on the first intermediate speech, and obtaining speech coding features corresponding to the first intermediate speech include: Inputting the first intermediate speech into an encoder, and performing feature extraction on the first intermediate speech through the convolution layer to obtain speech convolution features; The speech convolution feature is extracted through the bidirectional long short-term memory layer to obtain the speech coding feature.

6. A voice conversion device, characterized in that The device comprises: An acquisition module, used to acquire initial speech; A first feature extraction module, configured to extract filter bank energy features of the initial speech; A second feature extraction module is used to extract the energy features of the filter bank to obtain the speech style features corresponding to the initial speech; A third feature extraction module is used to extract features of the initial speech to obtain speech tone features corresponding to the initial speech; a feature synthesis module, configured to perform feature synthesis on the speech tone feature to obtain a first intermediate speech; a feature encoding module, configured to perform feature encoding on the first intermediate speech to obtain text features corresponding to the first intermediate speech; A feature fusion module, configured to fuse the speech style feature, the speech tone feature, and the text feature to obtain a mixed feature; A feature decoding module, configured to perform feature decoding on the mixed features to obtain a target speech; The voice conversion device is also used for: Inputting the initial speech into a vocoder, performing feature extraction on the initial speech to obtain a fundamental frequency, a spectrum envelope, and aperiodic parameters corresponding to the initial speech; and obtaining the speech pitch feature based on the fundamental frequency, the spectrum envelope, and the aperiodic parameters; An average value of the fundamental frequency is calculated to obtain an average fundamental frequency; and feature synthesis is performed on the average fundamental frequency, the spectrum envelope, and the non-periodic parameter to obtain a first intermediate speech.

7. An electronic device, characterized in that include: at least one memory; at least one processor; at least one program; The program is stored in the memory, and the processor executes at least one of the programs to implement: The speech conversion method according to any one of claims 1 to 5.

8. Storage medium, wherein the storage medium is a computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to cause a computer to execute: The speech conversion method according to any one of claims 1 to 5.

Images