Voice conversion method, voice conversion apparatus, electronic device, and storage medium

Abstract

The application provides a speech conversion method, a speech conversion device, an electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: obtaining original speech data of a target speaker; performing segmentation processing on the original speech data to obtain first speech data and second speech data; performing encoding processing on the first speech data and the second speech data through a vector quantization coding network of a speech conversion model to obtain a first text vector, a first speech feature vector, a second text vector and a second speech feature vector; the first speech feature vector and the second speech feature vector are used for representing speech characteristics of the target speaker; performing splicing processing on the first text vector, the first speech feature vector, the second text vector and the second speech feature vector to obtain a target speech vector; and performing decoding processing on the target speech vector through a decoding network of the speech conversion model to obtain target speech data. The application can improve the speech conversion effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a speech conversion method, a speech conversion device, an electronic device and a storage medium. BACKGROUND

[0002] Speech conversion generally refers to replacing a speaker with another speaker without changing the content information of the speech. Common speech conversion methods often cannot well represent the actual speech content and the style characteristics of the speaker when performing speech conversion, resulting in poor speech conversion effect. Therefore, how to improve the speech conversion effect has become a technical problem to be solved. SUMMARY

[0003] The main purpose of the embodiments of the present application is to provide a speech conversion method, a speech conversion device, an electronic device and a storage medium, which aims to improve the speech conversion effect.

[0004] To achieve the above-mentioned purpose, a first aspect of the embodiments of the present application provides a speech conversion method, which comprises:

[0005] obtaining original speech data of a target speaker;

[0006] segmenting the original speech data to obtain first speech data and second speech data;

[0007] encoding the first speech data through a vector quantization encoding network of a preset speech conversion model to obtain a first text vector and a first speech feature vector; wherein the first speech feature vector is used to represent the speech characteristics of the target speaker;

[0008] encoding the second speech data through the vector quantization encoding network to obtain a second text vector and a second speech feature vector; wherein the second speech feature vector is used to represent the speech characteristics of the target speaker;

[0009] splicing the first text vector, the first speech feature vector, the second text vector and the second speech feature vector to obtain a target speech vector;

[0010] decoding the target speech vector through a decoding network of the speech conversion model to obtain target speech data.

[0011] In some embodiments, the vector quantization encoding network comprises a first encoder and a first vector quantization encoder, and the step of encoding the first speech data through the vector quantization encoding network of the preset speech conversion model to obtain the first text vector and the first speech feature vector comprises:

[0012] encoding processing on the first speech data by the first encoder to obtain a first encoding vector;

[0013] encoding processing on the first encoding vector by the first vector quantization encoder to obtain the first text vector;

[0014] difference processing on the first encoding vector and the first text vector to obtain the first speech feature vector.

[0015] In some embodiments, the vector quantization encoding network comprises a second encoder, a second vector quantization encoder, and the step of encoding processing on the second speech data by the vector quantization encoding network to obtain a second text vector and a second speech feature vector comprises:

[0016] encoding processing on the second speech data by the second encoder to obtain a second encoding vector;

[0017] encoding processing on the second encoding vector by the second vector quantization encoder to obtain the second text vector;

[0018] difference processing on the second encoding vector and the second text vector to obtain the second speech feature vector.

[0019] In some embodiments, the step of concatenating processing on the first text vector, the first speech feature vector, the second text vector, and the second speech feature vector to obtain a target speech vector comprises:

[0020] concatenating processing on the first text vector and the second text vector to obtain a target text vector;

[0021] mean calculation on the first speech feature vector and the second speech feature vector to obtain a target speech feature vector;

[0022] concatenating processing on the target text vector and the target speech feature vector to obtain the target speech vector.

[0023] In some embodiments, the step of decoding processing on the target speech vector by a decoding network of the speech conversion model to obtain target speech data comprises:

[0024] decoding processing on the target speech vector by a decoder of the decoding network to obtain a target speech hidden state vector;

[0025] down-sampling processing on the target speech hidden state vector to obtain the target speech data.

[0026] In some embodiments, the step of segmenting the original speech data to obtain the first speech data and the second speech data comprises:

[0027] performing a spectrum calculation on the original speech data to obtain an initial spectrum graph;

[0028] performing a filtering process on the initial spectrum graph to obtain initial speech data;

[0029] segmenting the initial speech data according to a preset speech length to obtain the first speech data and the second speech data.

[0030] In some embodiments, before the step of encoding the first speech data by the vector quantization encoding network of the preset speech conversion model to obtain the first text vector and the first speech feature vector, the method further comprises pre-training the speech conversion model, specifically comprising:

[0031] obtaining sample speech data, wherein the sample speech data comprises first sample speech data and second sample speech data;

[0032] inputting the sample speech data into the speech conversion model, wherein the speech conversion model comprises a vector quantization encoding network and a decoding network;

[0033] encoding the first sample speech data by the vector quantization encoding network to obtain a first sample text vector and a first sample speech feature vector, and encoding the second sample speech data by the vector quantization encoding network to obtain a second sample text vector and a second sample speech feature vector;

[0034] performing a splicing process on the first sample text vector, the first sample speech feature vector, the second sample text vector, and the second sample speech feature vector to obtain a sample speech vector;

[0035] performing a reconstruction loss calculation on the sample speech vector by a loss function of the decoding network to obtain a reconstruction loss value;

[0036] performing a similarity calculation on the first sample speech feature vector and the second sample speech feature vector by a preset similarity algorithm to obtain a speech feature similarity value;

[0037] adjusting parameters of the speech conversion model according to the speech feature similarity value and the reconstruction loss value to optimize the speech conversion model.

[0038] To achieve the above object, a second aspect of the embodiment of the present application proposes a speech conversion device, which comprises:

[0039] The data acquisition module is configured to acquire original speech data of a target speaker.

[0040] The segmentation module is configured to perform segmentation processing on the original speech data to obtain first speech data and second speech data.

[0041] The first encoding module is configured to perform encoding processing on the first speech data by using a vector quantization encoding network of a preset speech conversion model to obtain a first text vector and a first speech feature vector, wherein the first speech feature vector is used to represent speech characteristics of the target speaker.

[0042] The second encoding module is configured to perform encoding processing on the second speech data by using the vector quantization encoding network to obtain a second text vector and a second speech feature vector, wherein the second speech feature vector is used to represent speech characteristics of the target speaker.

[0043] The splicing module is configured to perform splicing processing on the first text vector, the first speech feature vector, the second text vector, and the second speech feature vector to obtain a target speech vector.

[0044] The decoding module is configured to perform decoding processing on the target speech vector by using a decoding network of the speech conversion model to obtain target speech data.

[0045] To achieve the above object, a third aspect of embodiments of the present application provides an electronic device, which comprises a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program is executed by the processor to realize the method of the first aspect.

[0046] To achieve the above object, a fourth aspect of embodiments of the present application provides a storage medium, which is a computer readable storage medium, for computer readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to realize the method of the first aspect.

[0047] The speech conversion method, device, electronic device, and storage medium proposed in this application acquire the original speech data of the target speaker; segment the original speech data to obtain first speech data and second speech data, which effectively increases the total amount of speech data while ensuring that the segmented speech data conforms to the same context and has good correlation between speech data. Further, the first speech data and second speech data are encoded separately through a vector quantization encoding network of a preset speech conversion model to obtain a first text vector, a first speech feature vector, a second text vector, and a second speech feature vector. The first and second speech feature vectors are used to characterize the speech features of the target speaker. This method strengthens the decoupling of text features and speech features, improving the accuracy of the generated text information and speech feature information. Finally, the first text vector, first speech feature vector, second text vector, and second speech feature vector are concatenated to obtain the target speech vector, which is then decoded through a decoding network of the speech conversion model to obtain the target speech data that meets the requirements. This target speech data is then used as the content of the speech conversion. The speech conversion model of this application embodiment can more reasonably decouple the text features and speech features of the original speech data, thereby improving the speech conversion effect. This makes the speech conversion method of this application embodiment able to meet the needs of dubbing for short animated videos, and can also be well applied to various personalized speech synthesis tasks, with good applicability. Attached Figure Description

[0048] Figure 1 This is a flowchart of the speech conversion method provided in the embodiments of this application;

[0049] Figure 2 yes Figure 1 The flowchart of step S102 in the document;

[0050] Figure 3 This is another flowchart of the speech conversion method provided in the embodiments of this application;

[0051] Figure 4 yes Figure 1 The flowchart of step S103 in the process;

[0052] Figure 5 yes Figure 1 The flowchart of step S104 in the process;

[0053] Figure 6 yes Figure 1 The flowchart of step S105 in the process;

[0054] Figure 7 yes Figure 1 The flowchart of step S106 in the process;

[0055] Figure 8 Fig. 1 is a structural schematic diagram of a voice conversion device provided by an embodiment of the present application;

[0056] Figure 9 Fig. 2 is a hardware structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the objects, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0058] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", and the like in the specification and claims and the above-described drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0060] First, the terms involved in the present application are analyzed:

[0061] Artificial intelligence (AI): is a new technical science that studies, develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence; artificial intelligence is a branch of computer science, and artificial intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. The research in this field includes robots, language recognition, image recognition, natural language processing and expert systems, etc. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is also the theory, method, technology and application system of using digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, to perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0062] Natural Language Processing (NLP): NLP uses computers to process, understand, and utilize human language (such as Chinese and English). NLP is a branch of artificial intelligence and an interdisciplinary field of computer science and linguistics, often referred to as computational linguistics. NLP includes syntactic analysis, semantic analysis, and discourse understanding. It is commonly used in machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, intent recognition, information extraction and filtering, text classification and clustering, sentiment analysis, and opinion mining. It involves data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computation.

[0063] Information extraction is a text processing technique that extracts factual information such as entities, relationships, and events from natural language text and outputs it as structured data. Information extraction is a technique for extracting specific information from text data. Text data is composed of specific units, such as sentences, paragraphs, and chapters. Text information is composed of smaller, specific units, such as characters, words, phrases, sentences, paragraphs, or combinations of these units. Extracting noun phrases, names of people, and place names from text data is an example of text information extraction. Of course, text information extraction techniques can extract information of various types.

[0064] Encoder: Transforms an input sequence into a fixed-length vector.

[0065] Decoding: This involves transforming a previously generated fixed vector into an output sequence; the input sequence can be text, speech, image, or video; the output sequence can be text or image.

[0066] Fourier transform: Represents a function that satisfies certain conditions as a linear combination of trigonometric functions (sine and / or cosine functions) or their integrals. In different research fields, the Fourier transform has various variants, such as the continuous Fourier transform and the discrete Fourier transform.

[0067] Mel-Frequency Cipstal Coefficients (MFCC): A set of cepstrum from a segment of a music signal is obtained to represent the music signal, and the Mel-Frequency Cipstal Coefficients are derived from the cepstrum. Unlike the general cepstrum, the biggest feature of the Mel-Frequency Cipstal Coefficients is that the frequency bands on the Mel-Frequency Cipstal Coefficients are uniformly distributed on the Mel scale, that is, compared with the general linear cepstrum representation method, such frequency bands are closer to the nonlinear human audio system. For example, in the audio compression technology, the Mel-Frequency Cipstal Coefficients are often used for processing.

[0068] Vector Quantization (VQ): A method of clustering original continuous data into discrete data by a method similar to clustering, so as to reduce the amount of data to be stored, and thus achieve the purpose of data compression.

[0069] Voice Conversion (VC) usually refers to changing a speaker into another speaker without changing the content information of the speech. The common voice conversion method often cannot well represent the actual speech content and the style characteristics of the speaker when performing voice conversion, thereby resulting in poor voice conversion effect. Therefore, how to improve the voice conversion effect has become a technical problem to be solved.

[0070] Based on this, the embodiments of the present application provide a voice conversion method, a voice conversion device, an electronic equipment and a storage medium, aiming to improve the voice conversion effect.

[0071] The voice conversion method, the voice conversion device, the electronic equipment and the storage medium provided by the embodiments of the present application are specifically explained by the following embodiments. First, the voice conversion method in the embodiments of the present application is described.

[0072] The embodiments of the present application can acquire and process related data based on artificial intelligence technology. Artificial Intelligence (AI) is a theory, method, technology and application system for using digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain optimal results.

[0073] The artificial intelligence basic technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. The artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric identification technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0074] The voice conversion method provided by the embodiments of the present application relates to the technical field of artificial intelligence. The voice conversion method provided by the embodiments of the present application can be applied to a terminal, can also be applied to a server end, and can also be software running in the terminal or the server end. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.; the server end can be configured as an independent physical server, can also be configured as a server cluster or a distributed system composed of multiple physical servers, can also be configured as a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform; and the software can be an application for implementing the voice conversion method, etc., but is not limited to the above forms.

[0075] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0076] Figure 1 The voice conversion method provided by the embodiments of the present application is an optional flowchart, Figure 1 The method in the flowchart can include but is not limited to steps S101 to S106.

[0077] Step S101, obtaining original voice data of a target speaker;

[0078] Step S102, performing segmentation processing on the original voice data to obtain first voice data and second voice data;

[0079] Step S103, the first speech data is encoded by a preset vector quantization coding network of the speech conversion model to obtain a first text vector and a first speech feature vector; wherein the first speech feature vector is used to represent the speech characteristics of the target speaker;

[0080] Step S104, the second speech data is encoded by the vector quantization coding network to obtain a second text vector and a second speech feature vector; wherein the second speech feature vector is used to represent the speech characteristics of the target speaker;

[0081] Step S105, the first text vector, the first speech feature vector, the second text vector and the second speech feature vector are spliced to obtain a target speech vector;

[0082] Step S106, the target speech vector is decoded by a decoding network of the speech conversion model to obtain target speech data.

[0083] The steps S101 to S106 shown in the embodiments of the present application, by obtaining the original speech data of the target speaker; the original speech data is segmented to obtain the first speech data and the second speech data, which can effectively increase the total amount of speech data, and can make the segmented speech data consistent with the same context, and can better associate the speech data. Further, the first speech data and the second speech data are encoded by a preset vector quantization coding network of the speech conversion model to obtain a first text vector, a first speech feature vector, a second text vector and a second speech feature vector; wherein the first speech feature vector and the second speech feature vector are used to represent the speech characteristics of the target speaker, which can strengthen the feature decoupling of the text feature and the speech feature, and improve the accuracy of the generated text information and the speech feature information. Finally, the first text vector, the first speech feature vector, the second text vector and the second speech feature vector are spliced to obtain a target speech vector, and the target speech vector is decoded by a decoding network of the speech conversion model to obtain target speech data that meets the requirements, and the target speech data is used as the content of speech conversion. The speech conversion model of the embodiments of the present application can more reasonably decouple the text features and speech features of the original speech data, thereby improving the speech conversion effect, so that the speech conversion method of the embodiments of the present application can meet the needs of animation short video dubbing, and can also be well applied to various personalized speech synthesis tasks, and has good applicability.

[0084] In step S101 of some embodiments, the original speech data of the target speaker can be obtained by writing a web crawler and setting up a data source to crawl data targetedly. The original speech data of the target speaker can also be obtained by other ways, which are not limited. The data source can be various types of network platforms, social media, or some specific audio databases, etc. The original speech data can be music materials, speeches, reports, chat conversations, etc. of the target speaker. The target speech data can also be obtained by other ways, which are not limited.

[0085] It should be noted that in each specific embodiment of the present application, when relevant processing needs to be performed according to user information, user behavior data, user historical data, and user location information, etc. related to the identity or characteristics of the user, the user's permission or consent will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards of the country and region. In addition, when the present application needs to obtain sensitive personal information of the user, the separate permission or separate consent of the user will be obtained through a pop-up window or a jump to a confirmation page, etc. After obtaining the separate permission or separate consent of the user, the necessary user-related data for the normal operation of the present application is obtained.

[0086] Please refer to Figure 2 In some embodiments, step S102 can include but is not limited to steps S201 to S203:

[0087] Step S201, performing spectrogram calculation on the original speech data to obtain an initial frequency spectrum graph;

[0088] Step S202, performing filtering processing on the initial frequency spectrum graph to obtain initial speech data;

[0089] Step S203, performing segmentation processing on the initial speech data according to a preset speech length to obtain first speech data and second speech data.

[0090] In step S201 of some embodiments, the original speech data X is calculated by short-time Fourier transform to obtain an initial frequency spectrum graph. Specifically, the original speech data is subjected to signal framing and windowing processing to obtain multiple speech segments, each speech segment is subjected to short-time Fourier transform to convert the time domain features of the speech segment into frequency domain features, and finally, the frequency domain features of each speech segment are stacked in the time dimension to obtain the initial frequency spectrum graph.

[0091] In step S202 of some embodiments, the initial frequency spectrum is filtered by a preset mel cepstrum filter bank, the initial frequency spectrum is subjected to logarithmic operation to obtain an initial log spectrum, and the initial log spectrum is subjected to inverse Fourier transform processing to obtain initial voice data, wherein the initial voice data is a mel cepstrum graph, and the initial voice data can be expressed in a matrix form.

[0092] In step S203 of some embodiments, the preset voice length can be determined according to the length dimension of the matrix, the initial voice data is evenly cut according to the length dimension of the matrix corresponding to the initial voice data, the first half of the length dimension of the matrix is taken as the first voice data x1, and the second half of the length dimension of the matrix is taken as the second voice data x2.

[0093] Through the above steps S201 to S203, a segment of voice data of a target speaker can be segmented to obtain two voice data, the total amount of voice data can be effectively increased, the segmented voice data can be made to be in the same context, the correlation between the voice data can be better, and thus the voice conversion effect is improved.

[0094] Please refer to Figure 3 Before step S103 of some embodiments, the voice conversion method further includes pre-training a voice conversion model, which can specifically include but is not limited to steps S301 to S307:

[0095] Step S301, obtaining sample voice data, wherein the sample voice data includes first sample voice data and second sample voice data;

[0096] Step S302, inputting the sample voice data into a voice conversion model, wherein the voice conversion model includes a vector quantization encoding network and a decoding network;

[0097] Step S303, encoding the first sample voice data by the vector quantization encoding network to obtain a first sample text vector and a first sample voice feature vector, and encoding the second sample voice data by the vector quantization encoding network to obtain a second sample text vector and a second sample voice feature vector;

[0098] Step S304, splicing the first sample text vector, the first sample voice feature vector, the second sample text vector, and the second sample voice feature vector to obtain a sample voice vector;

[0099] Step S305, calculating a reconstruction loss of the sample voice vector by a loss function of the decoding network to obtain a reconstruction loss value;

[0100] Step S306, similarity calculation is performed on the first sample speech feature vector and the second sample speech feature vector by a preset similarity algorithm to obtain a speech feature similarity value;

[0101] Step S307, parameter adjustment is performed on the speech conversion model according to the speech feature similarity value and the reconstruction loss value to optimize the speech conversion model.

[0102] In step S301 of some embodiments, sample speech data can be obtained by writing a web crawler and setting a data source to crawl data targetedly. Sample speech data can also be obtained by other ways, which are not limited. The data source can be various types of network platforms, social media or some specific audio databases, and the sample speech data can be music materials, speeches, chat conversations and the like from different speakers. The sample speech data includes first sample speech data and second sample speech data, which are obtained by dividing sample speech data in the same context.

[0103] In step S302 of some embodiments, the sample speech data is input into the speech conversion model, which includes a vector quantization encoding network and a decoding network. The vector quantization encoding network includes a first encoder, a first vector quantization encoder, a second encoder and a second vector quantization encoder. The first encoder and the second encoder can be BERT encoders, and the first vector quantization encoder and the first vector quantization encoder are vector encoders. The first encoder and the second encoder are mainly used to map input data from semantic space to vector space. The first vector quantization encoder and the second vector quantization encoder are mainly used to cluster continuous input data into discrete data in a manner similar to clustering, so as to reduce the amount of data to be stored and achieve the purpose of data compression. In addition, the first vector quantization encoder and the second vector quantization encoder are also used to save the most important information and separate the phoneme information and the speaker's voice characteristic information contained in the speech by vector quantization. The decoding network includes a decoder, which is mainly used for reconstruction processing of input speech data, so that the speech content of the input speech data is closer to the speech content of the original speech data.

[0104] In step S303 of some embodiments, assuming that the input sample voice data is Y, the sample voice data Y includes a first sample voice data Y1 and a second sample voice data Y2, the first sample voice data Y1 is encoded by the vector quantization coding network to obtain a first sample text vector C1 and a first sample voice feature vector S1, and the second sample voice data Y2 is encoded by the vector quantization coding network to obtain a second sample text vector C2 and a second sample voice feature vector S2. The first sample text vector C1 and the second sample text vector C2 can represent the text content of the sample voice, and the first sample voice feature vector S1 and the second sample voice feature vector S2 can represent the voice characteristics of the speaker of the sample voice.

[0105] In step S304 of some embodiments, when the first sample text vector C1, the first sample voice feature vector S1, the second sample text vector C2 and the second sample voice feature vector S2 are concatenated to obtain the sample voice vector z, the first sample voice feature vector S1 and the second sample voice feature vector S2 are first averaged to obtain a sample voice feature vector S=(S1+S2) / 2, and then the first sample text vector C1 and the second sample text vector C2 and the sample voice feature vector S are added to obtain the sample voice vector z, z=C1+C2+S.

[0106] In step S305 of some embodiments, the reconstruction loss of the sample voice vector is calculated by the loss function of the decoding network to obtain a reconstruction loss value L recon , wherein L recon =‖Y-y‖1.

[0107] In step S306 of some embodiments, the preset similarity algorithm can be a cosine similarity algorithm, and the similarity between the first sample voice feature vector S1 and the second sample voice feature vector S2 is calculated by the cosine similarity algorithm to obtain a voice feature similarity value k.

[0108] In step S307 of some embodiments, when the voice conversion model is adjusted according to the voice feature similarity value and the reconstruction loss value, the model parameters of the voice conversion model are adjusted so that the voice feature similarity k is maximized and tends to 1, and the reconstruction loss value is as small as possible and tends to the preset loss threshold, thereby realizing the training of the voice conversion model.

[0109] The above steps S301 to S307 constrain the training of the model from two aspects of reconstruction loss and similarity value calculation, so that the training of the model is more reasonable, the training effect of the model can be effectively improved, the model performance of the voice conversion model can be improved, and the voice conversion effect can be improved.

[0110] Please refer to Figure 4In some embodiments, the vector quantization coding network comprises a first encoder, a first vector quantization encoder, and step S103 can comprise but is not limited to steps S401-S403:

[0111] In step S401, the first speech data is encoded by the first encoder to obtain a first encoding vector.

[0112] In step S402, the first encoding vector is encoded by the first vector quantization encoder to obtain a first text vector.

[0113] In step S403, the first encoding vector and the first text vector are subtracted to obtain a first speech feature vector.

[0114] In step S401 of some embodiments, the first speech data x1 is encoded by the first encoder to map the first speech data from a semantic space to a vector space, thereby obtaining a first encoding vector m1, wherein the first encoding vector m1 is a continuous vector.

[0115] In step S402 of some embodiments, the first encoding vector m1 is encoded by the first vector quantization encoder to cluster the first encoding vector m1 into discrete data, thereby reducing the amount of data to be stored and achieving the purpose of data compression. In addition, the first vector quantization encoder separates the phoneme information and the speaker's voice characteristic information contained in the first encoding vector m1 by vector quantization to obtain a first text vector c1, wherein the first text vector c1 is a discrete vector, and the information representation of the first text vector c1 can be obtained from a pre-set codebook. The first text vector c1 can be used to represent the content of the first speech data x1, which is the main information in the first speech data x1 and is the part that can be transcribed into text. It should be noted that in English and many other languages, the basic unit of content is a phoneme, and each phoneme has a specific formant pattern.

[0116] In step S403 of some embodiments, since the expected difference between the continuous vector and the discrete vector of the first speech data can be used to reflect the timbre information of the target speaker, the first encoding vector m1 and the first text vector c1 can be subtracted to obtain a first speech feature vector s1, wherein s1 = m1-c1, and the first speech feature vector can be used to represent the voice characteristics of the target speaker, which in some embodiments can be timbre information, pitch information, etc. Timbre information is the voice characteristic of the target speaker and is related to the identity of the speaker. Timbre is considered a feature of sound, reflected by formant frequencies, and is a resonant frequency component in the vocal tract. In a frequency spectrum, formants are shown as components that highlight the frequency spectrum envelope.

[0117] The steps S401 to S403 can decouple the text features and the speech features of the first speech data through the vector quantization coding network, thereby improving the speech conversion effect.

[0118] Please refer to Figure 5 In some embodiments, the vector quantization coding network comprises a second encoder, a second vector quantization encoder, and the step S104 can comprise but is not limited to steps S501 to S503.

[0119] In step S501, the second speech data is encoded by the second encoder to obtain a second encoding vector.

[0120] In step S502, the second encoding vector is encoded by the second vector quantization encoder to obtain a second text vector.

[0121] In step S503, the second encoding vector and the second text vector are subtracted to obtain a second speech feature vector.

[0122] In step S501 of some embodiments, the second speech data x2 is encoded by the second encoder to map the second speech data from a semantic space to a vector space, thereby obtaining a second encoding vector m2. The second encoding vector m2 is a continuous vector.

[0123] In step S502 of some embodiments, the second encoding vector m2 is encoded by the second vector quantization encoder to cluster the second encoding vector m2 into discrete data, thereby reducing the amount of data to be stored and achieving the purpose of data compression. In addition, the second vector quantization encoder separates the phoneme information and the speaker's speech characteristic information in the second encoding vector m2 by vector quantization to obtain a second text vector c2. The second text vector c2 is a discrete vector, and the information of the second text vector c2 can be obtained from a preset codebook. The second text vector can be used to represent the content of the second speech data x2, which is the main information in the second speech data x2 and can be transcribed into text.

[0124] In step S503 of some embodiments, since the expected difference between the continuous vector and the discrete vector of the second speech data can be used to reflect the timbre information of the target speaker, the second encoding vector m2 and the second text vector c2 can be subtracted to obtain a second speech feature vector s2, where s1=m2-c2. The second speech feature vector can be used to represent the speech characteristics of the target speaker, which are mainly timbre information.

[0125] The steps S501 to S503 can decouple the text features and the speech features of the second speech data through the vector quantization coding network, thereby improving the speech conversion effect.

[0126] Referring to Figure 6 In some embodiments, the step S105 can include but is not limited to steps S601 to S603.

[0127] In the step S601, the first text vector and the second text vector are concatenated to obtain a target text vector.

[0128] In the step S602, the first speech feature vector and the second speech feature vector are averaged to obtain a target speech feature vector.

[0129] In the step S603, the target text vector and the target speech feature vector are concatenated to obtain a target speech vector.

[0130] In the step S601 of some embodiments, when the first text vector and the second text vector are concatenated, the first text vector c1 and the second text vector c2 can be added to obtain a target text vector c3, where the target text vector c3 can be represented as c3=c1+c2.

[0131] In the step S602 of some embodiments, when the first speech feature vector and the second speech feature vector are averaged, the first speech feature vector s1 and the second speech feature vector s2 can be added first, and then the average of the sum of the vectors is obtained to obtain a target speech feature vector s3, where the target speech feature vector s3 can be represented as s3=(s1+s2) / 2.

[0132] In the step S603 of some embodiments, when the target text vector and the target speech feature vector are concatenated, the target text vector c3 and the target speech feature vector s3 can be added to obtain a target speech vector n1, where the target speech vector n1 can be represented as n1=c3+s3.

[0133] Referring to Figure 7 In some embodiments, the step S106 can include but is not limited to steps S701 to S702.

[0134] In the step S701, the target speech vector is decoded by a decoder of the decoding network to obtain a target speech hidden state vector.

[0135] In the step S702, the target speech hidden state vector is down-sampled to obtain target speech data.

[0136] In step S701 of some embodiments, the decoding network comprises a decoder and a down-sampling unit, the step of the convolution layer of the decoder can be 1, and the input feature and the output feature of the decoder are of the same size. The semantic content information of the target speech vector is obtained by decoding the target speech vector through the decoder of the decoding network, and the target speech hidden state vector is obtained.

[0137] In step S702 of some embodiments, the target speech hidden state vector is down-sampled by the down-sampling unit to obtain target speech data, wherein the speech content of the target speech data is consistent with the content of the original speech data, but the speech features can have certain differences.

[0138] The speech conversion method of the embodiments of the present application can obtain the original speech data of the target speaker, and can effectively increase the total amount of speech data while ensuring that the segmented speech data is of the same context and has good relevance between the speech data. Further, the first speech data and the second speech data are encoded by the vector quantization encoding network of the preset speech conversion model to obtain a first text vector, a first speech feature vector, a second text vector, and a second speech feature vector. The first speech feature vector and the second speech feature vector are used to represent the speech characteristics of the target speaker, which can strengthen the feature decoupling of the text features and the speech features, and improve the accuracy of the generated text information and speech feature information. Finally, the first text vector, the first speech feature vector, the second text vector, and the second speech feature vector are spliced to obtain a target speech vector, and the target speech vector is decoded by the decoding network of the speech conversion model to obtain target speech data that meets the requirements. The target speech data is used as the content of the speech conversion. Further, since the speech conversion model of the embodiments of the present application can more reasonably decouple the text features and the speech features of the original speech data, it can effectively improve the speech conversion effect, so that the speech conversion method of the embodiments of the present application can meet the needs of animation short video dubbing, such as dubbing of self-made creative videos of self-media, dubbing of self-made animations of animation enthusiasts, dubbing of film and television works, etc. At the same time, the speech conversion method of the embodiments of the present application can be well applied to various personalized speech synthesis tasks, which is conducive to promoting the continuous innovation and development of speech technology in the field of artificial intelligence, and has broad market prospects.

[0139] Please refer to Figure 8 The embodiments of the present application also provide a speech conversion device, which can implement the speech conversion method described above. The device comprises:

[0140] The data acquisition module 801 is configured to obtain the original speech data of the target speaker.

[0141] The segmentation module 802 is configured to perform segmentation processing on the original speech data to obtain first speech data and second speech data.

[0142] The first encoding module 803 is configured to perform encoding processing on the first speech data by using a vector quantization encoding network of a preset speech conversion model to obtain a first text vector and a first speech feature vector; the first speech feature vector is used to represent speech characteristics of a target speaker.

[0143] The second encoding module 804 is configured to perform encoding processing on the second speech data by using the vector quantization encoding network to obtain a second text vector and a second speech feature vector; the second speech feature vector is used to represent speech characteristics of the target speaker.

[0144] The splicing module 805 is configured to perform splicing processing on the first text vector, the first speech feature vector, the second text vector, and the second speech feature vector to obtain a target speech vector.

[0145] The decoding module 806 is configured to perform decoding processing on the target speech vector by using a decoding network of the speech conversion model to obtain target speech data.

[0146] In some embodiments, the segmentation module 802 includes:

[0147] The acoustic spectrum calculation unit is configured to perform acoustic spectrum calculation on the original speech data to obtain an initial frequency spectrum graph.

[0148] The filtering unit is configured to perform filtering processing on the initial frequency spectrum graph to obtain initial speech data.

[0149] The segmentation unit is configured to perform segmentation processing on the initial speech data according to a preset speech length to obtain the first speech data and the second speech data.

[0150] In some embodiments, the speech conversion apparatus includes a training module, and the training module specifically includes:

[0151] The acquisition unit is configured to acquire sample speech data, wherein the sample speech data includes first sample speech data and second sample speech data.

[0152] The input unit is configured to input the sample speech data into a speech conversion model, wherein the speech conversion model includes a vector quantization encoding network and a decoding network.

[0153] The processing unit is configured to perform encoding processing on the first sample speech data by using the vector quantization encoding network to obtain a first sample text vector and a first sample speech feature vector, and perform encoding processing on the second sample speech data by using the vector quantization encoding network to obtain a second sample text vector and a second sample speech feature vector.

[0154] The splicing unit is configured to splice the first sample text vector, the first sample speech feature vector, the second sample text vector, and the second sample speech feature vector to obtain a sample speech vector.

[0155] The loss calculation unit is configured to calculate a reconstruction loss of the sample speech vector by using a loss function of the decoding network to obtain a reconstruction loss value.

[0156] The similarity calculation unit is configured to calculate a similarity between the first sample speech feature vector and the second sample speech feature vector by using a preset similarity algorithm to obtain a speech feature similarity value.

[0157] The optimization unit is configured to adjust parameters of the speech conversion model according to the speech feature similarity value and the reconstruction loss value to optimize the speech conversion model.

[0158] In some embodiments, the vector quantization encoding network includes a first encoder and a first vector quantization encoder, and the first encoding module 803 includes:

[0159] The first encoding unit is configured to encode the first speech data by using the first encoder to obtain a first encoding vector.

[0160] The second encoding unit is configured to encode the first encoding vector by using the first vector quantization encoder to obtain a first text vector.

[0161] The first difference calculation unit is configured to calculate a difference between the first encoding vector and the first text vector to obtain a first speech feature vector.

[0162] In some embodiments, the vector quantization encoding network includes a second encoder and a second vector quantization encoder, and the second encoding module 804 includes:

[0163] The third encoding unit is configured to encode the second speech data by using the second encoder to obtain a second encoding vector.

[0164] The fourth encoding unit is configured to encode the second encoding vector by using the second vector quantization encoder to obtain a second text vector.

[0165] The second difference calculation unit is configured to calculate a difference between the second encoding vector and the second text vector to obtain a second speech feature vector.

[0166] In some embodiments, the splicing module 805 includes:

[0167] The first splicing unit is configured to splice the first text vector and the second text vector to obtain a target text vector.

[0168] a mean calculation unit, configured to perform mean calculation on the first speech feature vector and the second speech feature vector to obtain a target speech feature vector;

[0169] a second concatenation unit, configured to perform concatenation processing on the target text vector and the target speech feature vector to obtain a target speech vector.

[0170] In some embodiments, the decoding module 806 includes:

[0171] a decoding unit, configured to perform decoding processing on the target speech vector by a decoder of the decoding network to obtain a target speech latent state vector;

[0172] a down-sampling unit, configured to perform down-sampling processing on the target speech latent state vector to obtain target speech data.

[0173] The specific implementation of the speech conversion apparatus is basically the same as the specific embodiments of the speech conversion method described above, and will not be repeated here.

[0174] The embodiments of the present application also provide an electronic device, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing the connection communication between the processor and the memory, and the program is executed by the processor to realize the speech conversion method described above. The electronic device can be any smart terminal including a tablet computer, a vehicle-mounted computer, etc.

[0175] Please refer to Figure 9 , Figure 9 a hardware structure of an electronic device of another embodiment is shown, which includes:

[0176] The processor 901 can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., and is used to execute related programs to realize the technical solutions provided by the embodiments of the present application;

[0177] The memory 902 can be implemented in the form of a ROM (ReadOnly Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 902 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are saved in the memory 902 and are called and executed by the processor 901 to realize the speech conversion method of the embodiments of the present application;

[0178] The input / output interface 903 is configured to realize information input and output.

[0179] The communication interface 904 is configured to realize communication interaction between the device and other devices. The communication can be realized in a wired manner (for example, a USB, a network cable, and the like) or in a wireless manner (for example, a mobile network, WIFI, Bluetooth, and the like).

[0180] The bus 905 is configured to transmit information between various components (for example, the processor 901, the memory 902, the input / output interface 903, and the communication interface 904) of the device.

[0181] The processor 901, the memory 902, the input / output interface 903, and the communication interface 904 are connected to each other through the bus 905 to realize communication connection between the device.

[0182] The embodiment of the present application further provides a storage medium, which is a computer readable storage medium, is used for computer readable storage, and stores one or more programs. The one or more programs can be executed by one or more processors to realize the voice conversion method.

[0183] The memory is a non-transitory computer readable storage medium, and can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can further include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0184] The voice conversion method, the voice conversion device, the electronic equipment and the storage medium provided by the embodiments of the present application can obtain the original voice data of a target speaker, perform segmentation processing on the original voice data to obtain first voice data and second voice data, effectively increase the total amount of voice data, make the segmented voice data consistent with the same context, and better associate the voice data. Further, the first voice data and the second voice data are respectively encoded by a vector quantization coding network of a preset voice conversion model to obtain a first text vector, a first voice feature vector, a second text vector and a second voice feature vector; wherein the first voice feature vector and the second voice feature vector are used to represent the voice characteristics of the target speaker, which can strengthen the feature decoupling of the text feature and the voice feature, and improve the accuracy of the generated text information and voice feature information. Finally, the first text vector, the first voice feature vector, the second text vector and the second voice feature vector are spliced to obtain a target voice vector, and the target voice vector is decoded by a decoding network of the voice conversion model to obtain the target voice data meeting the requirements, and the target voice data is used as the content of voice conversion. Further, since the voice conversion model of the embodiments of the present application can more reasonably decouple the text feature and the voice feature of the original voice data, the voice conversion effect can be effectively improved, so that the voice conversion method of the embodiments of the present application can meet the needs of animation short video dubbing, such as dubbing of self-made creative videos of self-media, dubbing of self-made animations of animation lovers, dubbing of film and television works, etc.; at the same time, the voice conversion method of the embodiments of the present application can be better applied to various personalized voice synthesis tasks, which is conducive to promoting the continuous innovation and development of voice technology in the field of artificial intelligence, and has broad market prospects.

[0185] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0186] Those skilled in the art can understand that, Figures 1-7 The technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps, or combine certain steps, or different steps.

[0187] The apparatus embodiments described above are merely exemplary, and the units described as separate units can or can not be physically separate, i.e., can be located in one place, or can be distributed over multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment.

[0188] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the functional modules / units in the system and the device can be implemented as software, firmware, hardware and appropriate combinations thereof.

[0189] The terms "first", "second", "third", "fourth" and the like in the description of the application and in the claims of the foregoing drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so construed can be interchanged, such that the embodiments of the application described herein can be carried out in other than the order discussed herein without departing from the scope of the application. Further, the terms "comprise" and "comprising" and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, product or apparatus that comprises a list of steps or units does not necessarily comprise only those steps or units but can include other not expressly listed steps or units.

[0190] It should be understood that in this application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the association relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents that the associated objects before and after are in an "or" relationship. "At least one of the following" or the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0191] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the above units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0192] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they can be located in one place, or can be distributed to a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0193] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0194] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that makes a contribution or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0195] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, but this does not limit the scope of the embodiments of the present application. Any modification, equivalent replacement and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A voice conversion method characterized by, The method comprises: obtaining original speech data of a target speaker; performing segmentation processing on the original speech data to obtain first speech data and second speech data; performing encoding processing on the first speech data through a vector quantization encoding network of a preset speech conversion model to obtain a first text vector and a first speech feature vector; wherein the first speech feature vector is used to represent the speech characteristics of the target speaker; performing encoding processing on the second speech data through the vector quantization encoding network to obtain a second text vector and a second speech feature vector; wherein the second speech feature vector is used to represent the speech characteristics of the target speaker; performing concatenation processing on the first text vector, the first speech feature vector, the second text vector and the second speech feature vector to obtain a target speech vector; performing decoding processing on the target speech vector through a decoding network of the speech conversion model to obtain target speech data; The speech conversion model comprises a vector quantization encoding network and a decoding network; the speech conversion model is trained in the following manner: obtaining sample speech data, the sample speech data comprising first sample speech data and second sample speech data; performing encoding processing on the first sample speech data through the vector quantization encoding network to obtain a first sample text vector and a first sample speech feature vector, and performing encoding processing on the second sample speech data through the vector quantization encoding network to obtain a second sample text vector and a second sample speech feature vector; averaging the first sample speech feature vector and the second sample speech feature vector to obtain a sample speech feature vector, and performing vector addition on the first sample text vector, the sample speech feature vector and the second sample text vector to obtain a sample speech vector; performing reconstruction loss calculation on the sample speech vector through a loss function of the decoding network to obtain a reconstruction loss value; performing similarity calculation on the first sample speech feature vector and the second sample speech feature vector through a preset similarity algorithm to obtain a speech feature similarity value; adjusting parameters of the speech conversion model according to the speech feature similarity value and the reconstruction loss value to optimize the speech conversion model.

2. The voice conversion method of claim 1, wherein, The vector quantization encoding network comprises a first encoder and a first vector quantization encoder, and the step of performing encoding processing on the first speech data through a vector quantization encoding network of a preset speech conversion model to obtain a first text vector and a first speech feature vector comprises: performing encoding processing on the first speech data through the first encoder to obtain a first encoding vector; performing encoding processing on the first encoding vector through the first vector quantization encoder to obtain the first text vector; performing difference processing on the first encoding vector and the first text vector to obtain the first speech feature vector.

3. The voice conversion method of claim 1, wherein, The vector quantization coding network comprises a second encoder and a second vector quantization encoder. The second voice data is encoded by the second encoder to obtain a second encoding vector. The second encoding vector is encoded by the second vector quantization encoder to obtain the second text vector. The second encoding vector and the second text vector are subtracted to obtain the second voice feature vector.

4. The voice conversion method of claim 1, wherein, The first text vector and the second text vector are spliced to obtain a target text vector. The first voice feature vector and the second voice feature vector are averaged to obtain a target voice feature vector. The target text vector and the target voice feature vector are spliced to obtain the target voice vector. The target voice vector is decoded by a decoder of the decoding network of the voice conversion model to obtain target voice data.

5. The voice conversion method of claim 1, wherein, The target voice vector is decoded by a decoder of the decoding network to obtain a target voice hidden state vector. The target voice hidden state vector is down-sampled to obtain the target voice data. The original voice data is divided into first voice data and second voice data.

6. The voice conversion method of claim 1, wherein, The original voice data is spectrally calculated to obtain an initial frequency spectrum. The initial frequency spectrum is filtered to obtain initial voice data. The initial voice data is divided according to a preset voice length to obtain the first voice data and the second voice data. The device comprises:

7. A voice conversion apparatus characterized by comprising: a data acquisition module configured to acquire original voice data of a target speaker; a division module configured to divide the original voice data to obtain first voice data and second voice data; a first encoding module configured to encode the first voice data by a vector quantization coding network of a preset voice conversion model to obtain a first text vector and a first voice feature vector, wherein the first voice feature vector is used to represent voice characteristics of the target speaker; a second encoding module configured to encode the second voice data by the vector quantization coding network to obtain a second text vector and a second voice feature vector, wherein the second voice feature vector is used to represent voice characteristics of the target speaker; a splicing module configured to splice the first text vector, the first voice feature vector, the second text vector, and the second voice feature vector to obtain a target voice vector; a decoding module configured to decode the target voice vector by a decoding network of the voice conversion model to obtain target voice data; and ​ The voice conversion model comprises a vector quantization coding network and a decoding network; the voice conversion model is trained by the following manner: Obtaining sample voice data, the sample voice data comprising first sample voice data and second sample voice data; Encoding the first sample voice data through the vector quantization coding network to obtain a first sample text vector and a first sample voice feature vector, and encoding the second sample voice data through the vector quantization coding network to obtain a second sample text vector and a second sample voice feature vector; Obtaining a sample voice feature vector by averaging the first sample voice feature vector and the second sample voice feature vector, and obtaining a sample voice vector by vector addition of the first sample text vector, the sample voice feature vector and the second sample text vector; Calculating a reconstruction loss value of the sample voice vector through a loss function of the decoding network; Calculating a voice feature similarity value by similarity calculation of the first sample voice feature vector and the second sample voice feature vector through a preset similarity algorithm; Adjusting parameters of the voice conversion model according to the voice feature similarity value and the reconstruction loss value to optimize the voice conversion model.

8. An electronic device, comprising: The electronic device comprises a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection communication between the processor and the memory, and the program, when executed by the processor, realizes the steps of the voice conversion method according to any one of claims 1 to 6.

9. A storage medium, the storage medium being a computer-readable storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, and the one or more programs are executable by one or more processors to realize the steps of the voice conversion method according to any one of claims 1 to 6.

Images