A voice conversion method, device, equipment and medium

Abstract

The present application discloses a speech conversion method, device, equipment and medium, including: when the speech to be converted is greater than a preset length threshold, the speech to be converted is divided into multiple speech segments; each of the multiple speech segments is used as a target audio segment in turn; the target speech segment is input into a target automatic speech recognition model to obtain semantic features of the target speech segment; wherein the semantic features include speech posterior graph features and classification features; the semantic features and target voiceprint features are input into a target acoustic model to obtain Mel features; wherein the target voiceprint features are voiceprint features corresponding to the target timbre; the Mel features are converted into audio to obtain converted audio corresponding to the target speech segment, and the converted audio is output. The speech conversion delay can be reduced, thereby improving the user experience.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a speech conversion method, device, equipment and medium. Background Art

[0002] As virtual social products are widely used, voice conversion is an indispensable technical function to protect user privacy and improve user experience. At present, in the existing voice conversion technology, voice conversion has a high delay. When a user finishes a sentence, it usually takes about two to three seconds to receive the converted sound. The user faces the problem of long waiting time, resulting in a poor user experience. Summary of the invention

[0003] In view of this, the purpose of this application is to provide a voice conversion method, device, equipment and medium, which can reduce the voice conversion delay and thus improve the user experience. The specific scheme is as follows:

[0004] In a first aspect, the present application discloses a voice conversion method, comprising:

[0005] When the length of the speech to be converted is greater than a preset length threshold, the speech to be converted is divided into multiple speech segments;

[0006] sequentially taking each of the plurality of speech segments as a target audio segment;

[0007] Inputting the target speech segment into a target automatic speech recognition model to obtain semantic features of the target speech segment; wherein the semantic features include speech posterior graph features and classification features;

[0008] Inputting the semantic features and the target voiceprint features into the target acoustic model to obtain Mel features; wherein the target voiceprint features are voiceprint features corresponding to the target timbre;

[0009] The Mel features are converted into audio to obtain converted audio corresponding to the target speech segment, and the converted audio is output.

[0010] Optionally, the target acoustic model includes an encoder and a decoder; wherein the window size of the encoder is determined based on the size of the target audio segment.

[0011] Optionally, the training process of the target acoustic model includes:

[0012] Acquire a first training sample set; wherein the first training sample set includes first speech samples of different durations;

[0013] Selecting a first speech sample from the first training sample set and inputting the first speech sample into the target automatic speech recognition model to obtain speech posterior graph features and classification features of the first speech sample;

[0014] Inputting the speech posterior graph features and classification features of the first speech sample and the voiceprint features of the target speech sample into the initial acoustic model for training, and during the training process, the window size of the encoder is determined based on the length of the selected first speech sample;

[0015] When the first training stop condition is met, the current acoustic model is determined as the target acoustic model.

[0016] Optionally, obtaining a first training sample set includes:

[0017] Randomly select a preset number of samples from the original training sample set;

[0018] Use environmental noise to perform noise processing on the selected samples to obtain noise-added samples;

[0019] A sample set consisting of all the noise-added samples and the original training sample set is determined as a first training sample set.

[0020] Optionally, the initial acoustic model further includes an embedding layer; accordingly, inputting the speech posterior graph features and classification features of the first speech sample and the voiceprint features of the target speech sample into the initial acoustic model for training, including:

[0021] Outputting a feature matrix corresponding to the voiceprint features of the target speech sample using the embedding layer; the target speech sample includes speech samples corresponding to multiple speakers;

[0022] Inputting the feature matrix and the speech posterior graph features and classification features of the first speech sample into an encoder to obtain encoding features;

[0023] The encoded features are input into the decoder to obtain Mel features.

[0024] Optionally, the decoder includes multiple lightweight convolution units.

[0025] Optionally, the training process of the target automatic speech recognition model includes:

[0026] Acquire multiple second training samples from the second training data set to obtain a training sample group; wherein each of the second training samples is a single human voice sample;

[0027] Inputting the training sample group into an initial automatic speech recognition model for training;

[0028] When the second training stop condition is met, the current automatic speech recognition model is determined as the target automatic speech recognition model.

[0029] In a second aspect, the present application discloses a speech conversion device, comprising:

[0030] A speech segmentation module, used for segmenting the speech to be converted into multiple speech segments when the speech to be converted is longer than a preset length threshold;

[0031] a target segment determination module, configured to sequentially use each of the plurality of speech segments as a target audio segment;

[0032] A semantic feature extraction module, used for inputting the target speech segment into a target automatic speech recognition model to obtain semantic features of the target speech segment; wherein the semantic features include speech posterior graph features and classification features;

[0033] A Mel feature extraction module, used for inputting the semantic feature and the target voiceprint feature into the target acoustic model to obtain the Mel feature; wherein the target voiceprint feature is the voiceprint feature corresponding to the target timbre;

[0034] An audio conversion module, used for converting the Mel features into audio to obtain converted audio corresponding to the target speech segment;

[0035] An audio output module is used to output the converted audio.

[0036] In a third aspect, the present application discloses an electronic device, including a memory and a processor, wherein:

[0037] The memory is used to store the computer program;

[0038] The processor is used to execute the computer program to implement the aforementioned voice conversion method.

[0039] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, wherein the computer program implements the aforementioned speech conversion method when executed by a processor.

[0040] It can be seen that, when the speech to be converted is greater than the preset length threshold, the present application divides the speech to be converted into multiple speech segments; takes each of the multiple speech segments as the target audio segment in turn; inputs the target speech segment into the target automatic speech recognition model to obtain the semantic features of the target speech segment; wherein the semantic features include speech posterior graph features and classification features; inputs the semantic features and the target voiceprint features into the target acoustic model to obtain Mel features; wherein the target voiceprint features are voiceprint features corresponding to the target timbre; converts the Mel features into audio to obtain the converted audio corresponding to the target speech segment, and outputs the converted audio. That is, when the speech to be converted is greater than the preset length threshold, the present application divides the speech to be converted into multiple speech segments; inputs each speech segment into the target automatic speech recognition model in turn to obtain the semantic features, and then inputs the semantic features and the target voiceprint features into the target acoustic model to obtain the Mel features, which are then converted into audio and output. In this way, the processing is divided into segments, and a speech segment is processed to output the converted speech corresponding to a speech segment. Moreover, the conversion process utilizes the target automatic speech recognition model to obtain speech posterior graph features and classification features, and then outputs the speech posterior graph features and classification features and the target voiceprint features to the acoustic model to obtain Mel features, which are then converted into audio. In this way, rich semantic features are obtained in the conversion process, which can improve the accuracy of the converted speech.

[0041] The beneficial effect of the present application is that it can reduce the voice conversion delay while ensuring the accuracy of the converted voice, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0043] Figure 1 A flow chart of a speech conversion method disclosed in an embodiment of the present application;

[0044] Figure 2 A schematic diagram of voice conversion disclosed in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of an acoustic model audio processing disclosed in an embodiment of the present application;

[0046] Figure 4 A schematic diagram of the structure of a speech conversion device disclosed in an embodiment of the present application;

[0047] Figure 5 A structural diagram of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] At present, in the existing voice conversion technology, voice conversion has a high delay. When a user finishes speaking a sentence, it usually takes about two to three seconds to receive the converted sound. The user faces the problem of long waiting time, resulting in a poor user experience. To this end, this application provides a voice conversion solution that can reduce the voice conversion delay and thus improve the user experience.

[0050] See also Figure 1 As shown, the embodiment of the present application discloses a voice conversion method, including:

[0051] Step S11: When the length of the speech to be converted is greater than a preset length threshold, the speech to be converted is divided into a plurality of speech segments.

[0052] It is understandable that the voice to be converted is the recorded user voice, and a preset length threshold can be set according to actual needs. For example, the user voice can be windowed and packetized with a window size of 16 frames. One packet is a voice segment, which is output according to the window size. After each packet of data is processed, the corresponding processing result (i.e., the converted voice) is output, and the entire process forms a streaming output. When the converted voice is less than or equal to the preset length threshold, it can be directly input into the target automatic speech recognition model, and then the Mel feature is obtained through the target acoustic model, and then converted into audio output.

[0053] Step S12: taking each of the multiple voice segments in turn as a target audio segment.

[0054] Step S13: inputting the target speech segment into a target automatic speech recognition model to obtain semantic features of the target speech segment; wherein the semantic features include speech posterior graph features and classification features.

[0055] Wherein, the training process of the target automatic speech recognition model includes: obtaining multiple second training samples from the second training data set to obtain a training sample group; wherein each of the second training samples is a single human voice sample; inputting the training sample group into the initial automatic speech recognition model for training; when the second training stop condition is met, the current automatic speech recognition model is determined as the target automatic speech recognition model. In addition, the multiple second training samples can be speech samples corresponding to different human voices, so as to meet the real-time voice change requirements in a multi-person real-time streaming dialogue scenario. Wherein, the size of the group, i.e., the batch size, can be 32 or 64, and the group of training samples is composed of multi-speaker human voices, i.e., multi-voice speech. The classification feature is the feature output by the softmax layer, and the softmax layer is a layer determined based on the softmax function (i.e., the classification function). It can be understood that in the embodiment of the present application, each input to the target automatic speech recognition model is a speech segment, so that the converted audio corresponding to the speech segment can be obtained one by one.

[0056] Step S14: inputting the semantic features and the target voiceprint features into the target acoustic model to obtain Mel features; wherein the target voiceprint features are voiceprint features corresponding to the target timbre.

[0057] In an embodiment of the present application, the target acoustic model includes an encoder and a decoder; wherein the window size of the encoder is determined based on the size of the target audio segment. The decoder includes a plurality of lightweight convolution units. The encoder includes a plurality of encoding units, the encoding unit is a conformer unit, the conformer is a variant of the transformer (i.e., a transformer), and is composed of CNN (i.e., Convolutional Neural Networks) + transformer. The window size of the encoder is the window size of the encoding unit.

[0058] The training process of the acoustic model includes: obtaining a first training sample set; wherein the first training sample set includes first speech samples of different durations; selecting a first speech sample from the first training sample set and inputting it into the target automatic speech recognition model to obtain the speech posterior graph features and classification features of the first speech sample; inputting the speech posterior graph features, classification features of the first speech sample and the voiceprint features of the target speech sample into the initial acoustic model for training, and, during the training process, the window size of the encoder is determined based on the length of the selected first speech sample; when the first training stop condition is met, the current acoustic model is determined as the target acoustic model. The window size can be set dynamically, for example, it can be set to 10 frames, or 20 frames, or 30 frames; when the input audio clip is relatively long, such as more than 3 seconds, the window size is set to be relatively large, otherwise the window size is reduced.

[0059] The method of obtaining the first training sample set includes: randomly selecting a preset number of samples from the original training sample set; performing noise processing on the selected samples using environmental noise to obtain noised samples; and determining a sample set consisting of all noised samples and the original training sample set as the first training sample set. In this way, the robustness of the model can be improved.

[0060] Furthermore, the initial acoustic model also includes an embedding layer; accordingly, the speech posterior graph features, classification features of the first speech sample and the voiceprint features of the target speech sample are input into the initial acoustic model for training, including: using the embedding layer to output a feature matrix corresponding to the voiceprint features of the target speech sample; the target speech sample includes speech samples corresponding to multiple speakers; the feature matrix and the speech posterior graph features and classification features of the first speech sample are input into an encoder to obtain encoded features; and the encoded features are input into a decoder to obtain Mel features.

[0061] In the embodiment of the present application, the target timbre is the timbre selected by the user. The embodiment of the present application can provide multiple timbres for the user to choose from.

[0062] Step S15: convert the Mel features into audio to obtain converted audio corresponding to the target speech segment, and output the converted audio.

[0063] The embodiment of the present application can utilize a vocoder to convert Mel features into audio. Furthermore, the embodiment of the present application can output the converted audio of each audio segment as soon as the converted audio of the audio segment is obtained, without waiting for the converted audio corresponding to other audio segments. In other embodiments, the converted audio corresponding to a preset number of audio segments can be obtained and then output according to the actual scenario, wherein the preset number is less than the number of audio segments after segmentation. For example, if the user's voice is segmented into 100 audio segments, the converted audio corresponding to 10 audio segments can be obtained, and then the converted audio corresponding to the 10 audio segments can be grouped and output. In this way, the delay of the first packet data can be reduced and the result can be output in a streaming manner, thereby solving the problem of long waiting time for users.

[0064] It can be seen that, when the speech to be converted is greater than the preset length threshold, the present application divides the speech to be converted into multiple speech segments; takes each of the multiple speech segments as the target audio segment in turn; inputs the target speech segment into the target automatic speech recognition model to obtain the semantic features of the target speech segment; wherein the semantic features include speech posterior graph features and classification features; inputs the semantic features and the target voiceprint features into the target acoustic model to obtain Mel features; wherein the target voiceprint features are voiceprint features corresponding to the target timbre; converts the Mel features into audio to obtain the converted audio corresponding to the target speech segment, and outputs the converted audio. That is, when the speech to be converted is greater than the preset length threshold, the present application divides the speech to be converted into multiple speech segments; inputs each speech segment into the target automatic speech recognition model in turn to obtain the semantic features, and then inputs the semantic features and the target voiceprint features into the target acoustic model to obtain the Mel features, which are then converted into audio and output. In this way, the processing is divided into segments, and a speech segment is processed to output the converted speech corresponding to a speech segment. Moreover, the conversion process uses the target automatic speech recognition model to obtain speech posterior graph features and classification features, and then outputs the speech posterior graph features and classification features and the target voiceprint features to the acoustic model to obtain Mel features, which are then converted into audio. In this way, rich semantic features are obtained in the conversion process, which can improve the accuracy of the converted speech. The beneficial effect of the present application is that it can reduce the speech conversion delay while ensuring the accuracy of the converted speech, thereby improving the user experience.

[0065] For further information, see Figure 2As shown, the embodiment of the present application discloses a speech conversion schematic diagram. After the user confirms the target timbre, the recording begins. The ASR (Automatic Speech Recognition) model extracts the corresponding ppg (Phonetic Posteriorgram) features and softmax features (i.e., classification features) from the user's recorded voice, which represents the semantic information in the voice; the acoustic model uses the ppg features and softmax features as well as the voiceprint features corresponding to the target timbre as input, and outputs the mel (i.e., Mel) acoustic features corresponding to the target timbre. The mel features are then passed through the vocoder to obtain the audio after voice change, i.e., the target speaker's voice. Among them, the embodiment of the present application can train the ASR model with multi-voice speech data, which is responsible for extracting ppg features and softmax features as input for the subsequent acoustic model. The acoustic model consists of an encoder and a decoder. The encoder uses a conformer as a basic unit and is responsible for encoding ppg and softmax features; the decoder uses LightWeight Conv (i.e., lightweight convolution) as a decoding unit and uses the output of the encoder as input to be responsible for decoding the Mel features. Vocoder, using HiFi-GAN (Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis) vocoder.

[0066] Among them, for the training of the acoustic model, the training data can be expanded by adding noise to enhance the robustness of the acoustic model. The specific method is to randomly select half of the original data, use specific environmental noise (such as street environmental noise, supermarket environment, theater environment, etc.) to add noise to the original data according to different signal-to-noise ratios for data expansion, and then use the ASR model to extract the corresponding ppg and softmax features. And use the voiceprint recognition model to extract the xvector features (i.e., voiceprint features) of the target speaker's speech segment as the speaker representation, and introduce the voiceprint features of multiple speakers into the acoustic model to construct a speaker feature matrix. This representation can be used to control the target timbre of the voice change. In addition, the embodiment of the present application introduces a dynamically selected window size in the Conformer module of the acoustic model encoder to make the model adapt to audio input segments of different lengths, so that the model can adapt to "short-term segment" input, simulating the application scenario of real-time synthesis during actual use. The time series modeling unit LSTM (Long short-term memory) in the acoustic model decoder is changed to a more effective convolution, namely lightweight convolution, which improves the model's ability to connect real-time sound clips and improves the inference speed. Figure 3 As shown, Figure 3 A schematic diagram of an acoustic model audio processing provided in an embodiment of the present application. In this way, when there is only short-term segment training data, a model with real-time conversion capability is trained to achieve sound conversion in real-time scenarios. When there are short-term segment training data and long-term segment training data, a model compatible with long-term and short-term speech is obtained. Among them, the short time is less than the specified time length, and the long time is greater than or equal to the specified time length.

[0067] See also Figure 4 As shown, the embodiment of the present application discloses a speech conversion device, including:

[0068] A speech segmentation module 11 is used to segment the speech to be converted into multiple speech segments when the speech to be converted is longer than a preset length threshold;

[0069] a target segment determination module 12, configured to sequentially use each of the plurality of speech segments as a target audio segment;

[0070] A semantic feature extraction module 13 is used to input the target speech segment into a target automatic speech recognition model to obtain semantic features of the target speech segment; wherein the semantic features include speech posterior graph features and classification features;

[0071] The Mel feature extraction module 14 is used to input the semantic features and the target voiceprint features into the target acoustic model to obtain the Mel features; wherein the target voiceprint features are the voiceprint features corresponding to the target timbre;

[0072] An audio conversion module 15, configured to convert the Mel features into audio to obtain converted audio corresponding to the target speech segment;

[0073] The audio output module 16 is used to output the converted audio.

[0074] It can be seen that in the embodiment of the present application, when the speech to be converted is greater than the preset length threshold, the speech to be converted is divided into multiple speech segments; each of the multiple speech segments is sequentially used as a target audio segment; the target speech segment is input into the target automatic speech recognition model to obtain the semantic features of the target speech segment; wherein the semantic features include speech posterior graph features and classification features; the semantic features and the target voiceprint features are input into the target acoustic model to obtain Mel features; wherein the target voiceprint features are voiceprint features corresponding to the target timbre; the Mel features are converted into audio to obtain the converted audio corresponding to the target speech segment, and the converted audio is output. That is, in the embodiment of the present application, when the speech to be converted is greater than the preset length threshold, the speech to be converted is divided into multiple speech segments; each speech segment is sequentially input into the target automatic speech recognition model to obtain semantic features, and then the semantic features and the target voiceprint features are input into the target acoustic model to obtain Mel features, which are then converted into audio and output. In this way, the segmented processing is performed, and a speech segment is processed to output the converted speech corresponding to a speech segment. Moreover, the conversion process uses the target automatic speech recognition model to obtain speech posterior graph features and classification features, and then outputs the speech posterior graph features and classification features and the target voiceprint features to the acoustic model to obtain Mel features, which are then converted into audio. In this way, rich semantic features are obtained in the conversion process, which can improve the accuracy of the converted speech. The beneficial effect of the present application is that it can reduce the speech conversion delay while ensuring the accuracy of the converted speech, thereby improving the user experience.

[0075] In an embodiment of the present application, the target acoustic model includes an encoder and a decoder, wherein the window size of the encoder is determined based on the size of the target audio segment, and the decoder includes a plurality of lightweight convolution units.

[0076] In addition, the training process of the target acoustic model includes: obtaining a first training sample set; wherein the first training sample set includes first speech samples of different durations; selecting a first speech sample from the first training sample set, and inputting it into the target automatic speech recognition model to obtain speech posterior graph features and classification features of the first speech sample; inputting the speech posterior graph features, classification features of the first speech sample and the voiceprint features of the target speech sample into the initial acoustic model for training, and, during the training process, the window size of the encoder is determined based on the length of the selected first speech sample; when the first training stop condition is met, the current acoustic model is determined as the target acoustic model.

[0077] Among them, the obtaining of the first training sample set includes: randomly selecting a preset number of samples from the original training sample set; using environmental noise to perform noise processing on the selected samples to obtain noisy samples; and determining a sample set consisting of all noisy samples and the original training sample set as the first training sample set.

[0078] Furthermore, the initial acoustic model also includes an embedding layer; accordingly, the speech posterior graph features, classification features of the first speech sample and the voiceprint features of the target speech sample are input into the initial acoustic model for training, including: using the embedding layer to output a feature matrix corresponding to the voiceprint features of the target speech sample; the target speech sample includes speech samples corresponding to multiple speakers; the feature matrix and the speech posterior graph features and classification features of the first speech sample are input into an encoder to obtain encoded features; and the encoded features are input into a decoder to obtain Mel features.

[0079] In addition, the training process of the target automatic speech recognition model includes: obtaining multiple second training samples from a second training data set to obtain a training sample group; wherein each of the second training samples is a single human voice sample; inputting the training sample group into the initial automatic speech recognition model for training; when the second training stop condition is met, determining the current automatic speech recognition model as the target automatic speech recognition model.

[0080] See also Figure 5 As shown, an embodiment of the present application discloses an electronic device 20, including a processor 21 and a memory 22; wherein the memory 22 is used to store a computer program; the processor 21 is used to execute the computer program, the speech conversion method disclosed in the above embodiment.

[0081] For the specific process of the above-mentioned voice conversion method, reference may be made to the corresponding contents disclosed in the above-mentioned embodiments, which will not be described in detail here.

[0082] Furthermore, the memory 22 as a carrier for storing resources may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the storage method may be temporary storage or permanent storage.

[0083] In addition, the electronic device 20 also includes a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26; wherein the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input / output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0084] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the speech conversion method disclosed in the aforementioned embodiment.

[0085] For the specific process of the above-mentioned voice conversion method, reference may be made to the corresponding contents disclosed in the above-mentioned embodiments, which will not be described in detail here.

[0086] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0087] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0088] The above is a detailed introduction to a speech conversion method, device, equipment and medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A speech conversion method, characterized in that: include: When the length of the speech to be converted is greater than a preset length threshold, the speech to be converted is divided into multiple speech segments; sequentially taking each of the plurality of speech segments as a target speech segment; Inputting the target speech segment into a target automatic speech recognition model to obtain semantic features of the target speech segment; wherein the semantic features include speech posterior graph features and classification features; the classification features are features output by a softmax layer, and the softmax layer is a layer determined based on a softmax function; Inputting the semantic features and the target voiceprint features into the target acoustic model to obtain Mel features; wherein the target voiceprint features are voiceprint features corresponding to the target timbre; Convert the Mel features into audio to obtain converted audio corresponding to the target speech segment, and output the converted audio; The training process of the target acoustic model includes: Acquire a first training sample set; wherein the first training sample set includes first speech samples of different durations; Selecting a first speech sample from the first training sample set and inputting the first speech sample into the target automatic speech recognition model to obtain speech posterior graph features and classification features of the first speech sample; Inputting the speech posterior graph features and classification features of the first speech sample and the voiceprint features of the target speech sample into the initial acoustic model for training, and during the training process, the window size of the encoder is determined based on the length of the selected first speech sample; When the first training stop condition is met, the current acoustic model is determined as the target acoustic model.

2. The voice conversion method according to claim 1, characterized in that: The target acoustic model includes an encoder and a decoder; wherein, when the semantic features and the target voiceprint features are input into the target acoustic model, the window size of the encoder is determined based on the size of the target speech segment.

3. The voice conversion method according to claim 2, characterized in that: The obtaining of the first training sample set comprises: Randomly select a preset number of samples from the original training sample set; Use environmental noise to perform noise processing on the selected samples to obtain noise-added samples; A sample set consisting of all the noise-added samples and the original training sample set is determined as a first training sample set.

4. The voice conversion method according to claim 2, characterized in that: The initial acoustic model further includes an embedding layer; accordingly, the speech posterior graph features and classification features of the first speech sample and the voiceprint features of the target speech sample are input into the initial acoustic model for training, including: Outputting a feature matrix corresponding to the voiceprint feature of a target speech sample using the embedding layer; the target speech sample includes speech samples corresponding to multiple speakers; Inputting the feature matrix and the speech posterior graph features and classification features of the first speech sample into an encoder to obtain encoding features; The encoded features are input into the decoder to obtain Mel features.

5. The voice conversion method according to claim 2, characterized in that: The decoder includes a plurality of lightweight convolutional units.

6. The method for voice conversion according to any one of claims 1 to 5, characterized in that: The training process of the target automatic speech recognition model includes: Acquire multiple second training samples from the second training data set to obtain a training sample group; wherein each of the second training samples is a single human voice sample; Inputting the training sample group into an initial automatic speech recognition model for training; When the second training stop condition is met, the current automatic speech recognition model is determined as the target automatic speech recognition model.

7. A speech conversion device, characterized in that: include: A speech segmentation module, used for segmenting the speech to be converted into a plurality of speech segments when the speech to be converted is longer than a preset length threshold; A target segment determination module, used to sequentially take each of the multiple speech segments as a target speech segment; A semantic feature extraction module, used for inputting the target speech segment into a target automatic speech recognition model to obtain semantic features of the target speech segment; wherein the semantic features include speech posterior graph features and classification features; the classification features are features output by a softmax layer, and the softmax layer is a layer determined based on a softmax function; A Mel feature extraction module, used for inputting the semantic feature and the target voiceprint feature into the target acoustic model to obtain the Mel feature; wherein the target voiceprint feature is the voiceprint feature corresponding to the target timbre; An audio conversion module, used for converting the Mel features into audio to obtain converted audio corresponding to the target speech segment; An audio output module, used for outputting the converted audio; The training process of the target acoustic model includes: Acquire a first training sample set; wherein the first training sample set includes first speech samples of different durations; Selecting a first speech sample from the first training sample set and inputting the first speech sample into the target automatic speech recognition model to obtain speech posterior graph features and classification features of the first speech sample; Inputting the speech posterior graph features and classification features of the first speech sample and the voiceprint features of the target speech sample into the initial acoustic model for training, and during the training process, the window size of the encoder is determined based on the length of the selected first speech sample; When the first training stop condition is met, the current acoustic model is determined as the target acoustic model.

8. An electronic device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store the computer program; The processor is used to execute the computer program to implement the speech conversion method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the speech conversion method according to any one of claims 1 to 6 is implemented.

Images