Cross-language voice conversion method, readable storage medium and equipment

Abstract

The invention discloses a cross-language voice conversion method, a readable storage medium and equipment, and belongs to the technical field of voice signal processing, and the method comprises the steps: obtaining a first acoustic feature of a source voice and a second acoustic feature of a target voice; inputting the first acoustic feature and the second acoustic feature into a trained voice conversion model to obtain reconstructed acoustic features of the source voice and the target voice; and carrying out conversion processing on the reconstructed acoustic features to obtain synthetic speech obtained by cross-language conversion of the source speech and the target speech. The first acoustic feature of the source voice and the second acoustic feature of the target voice are reconstructed into the reconstructed acoustic features through the trained voice conversion model, and potential information of the acoustic features can be fully utilized through decoupling of the acoustic features; the quality of synthetic speech through reconstruction acoustic feature conversion processing is improved, and the parameter amount and the calculation amount of the model are reduced.

Description

Technical Field

[0001] The invention relates to a cross-language speech conversion method, a readable storage medium and a device, and belongs to the technical field of speech signal processing. Background Art

[0002] Speech conversion is a technology that converts the speaker identity features of the source speech into the speaker identity features of the target speech while ensuring that the content features of the source speech remain unchanged. Cross-language speech conversion is a research branch in the field of speech conversion. Cross-language speech conversion breaks the limitation of the same language, and the language and text of the source and target speech are different. Since the speech to be converted in cross-language speech conversion comes from different languages, and different languages ​​have great differences in pronunciation, intonation, and phonemes, the research on cross-language speech conversion faces great challenges.

[0003] In 2021, Liu et al. used a two-level nested U-structure to achieve cross-language speech conversion (Liu F, Wang H, Peng R, et al. U 2-VC: one-shot voice conversion using two-level nested U-structure[J]. EURASIP Journal on Audio, Speech, and Music Processing, 2021,2021(1): 1-15.). The two-level nested U-structure consists of multiple residual U-shaped modules. Compared with the standard U-structure, the two-level nested U-structure increases the number of parameters and calculations. In 2022, Lu and Wang et al. proposed a method based on A cross-language speech conversion method (Lu H, Wang D, Wu X, et al. Disentangled speech representation learning for one-shot cross-lingual voice conversion using ß-vae[C] / / 2022IEEE Spoken Language Technology Workshop (SLT). IEEE, 2023: 814-821.) uses the KL divergence between content representation and acoustic features, speaker information representation and acoustic features as the upper bound of the mutual information between each representation and acoustic features. At the same time, the KL divergence term is introduced and Two weight parameters are used to control the amount of data information that can be captured by content representation and speaker information representation. However, this method requires an additional speaker encoder to extract speaker features, which results in a large number of model parameters and computational complexity. In the same year, Dai proposed a cross-language speech conversion method based on activation guidance and inner convolution (Dai Shaoliang. Research on cross-language speech conversion based on activation guidance and inner convolution [D]. Nanjing University of Posts and Telecommunications, 2022. DOI: 10.27251 / d.cnki.gnjdc.2022.001228.). This method adopts a U-connected encoder-decoder structure and uses the activation guidance function as a soft bottleneck to extract content representations of different languages, thereby realizing cross-language speech conversion between any speakers. This method only requires one encoder to decouple the input acoustic features into content features and speaker features, and introduces inner convolution to reduce the number of parameters and computational complexity of the model, but the quality of the converted speech is still not satisfactory. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a cross-language speech conversion method, a readable storage medium and a device, which reduce the number of parameters and the amount of calculation of the model and improve the quality of the converted speech, and solve the problem that the current cross-language speech conversion method adopts a model that cannot well balance the number of parameters and the amount of calculation, and the conversion effect is not ideal.

[0005] In order to achieve the above purpose / solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0006] The present invention provides a method for cross-language speech conversion, comprising:

[0007] Acquiring a first acoustic feature of a source speech and a second acoustic feature of a target speech;

[0008] Inputting the first acoustic feature and the second acoustic feature into a trained speech conversion model to obtain reconstructed acoustic features of the source speech and the target speech; wherein the speech conversion model includes an encoder and a decoder;

[0009] The input of the encoder is the first acoustic feature and the second acoustic feature, and the output is the speaker information representation and the content representation;

[0010] The input of the decoder is speaker information representation and content representation, and the output is reconstructed acoustic features;

[0011] The reconstructed acoustic features are converted to obtain a synthesized speech that is a cross-language conversion of the source speech and the target speech.

[0012] Further, the encoder includes a first one-dimensional convolutional layer, a plurality of groups of encoding processing layers, a second one-dimensional convolutional layer and an activation guidance function layer connected in sequence, wherein the encoding processing layer includes a first bundled block convolutional layer, a first batch of normalization layers, a first LeakyReLU activation function layer, a third one-dimensional convolutional layer and a first instance normalization layer connected in sequence;

[0013] The decoder includes a second instance normalization layer, a first adaptive instance normalization layer, a fourth one-dimensional convolution layer, a second LeakyReLU activation function layer, multiple groups of decoding processing layers, a gated recurrent unit layer and a fully connected layer connected in sequence, wherein the decoding processing layer includes a decoding convolution module, a generating convolution module and a second adaptive instance normalization layer connected in sequence;

[0014] The generation convolution module has the same structure as the decoding convolution module, and the decoding convolution module includes a second bundling block convolution layer, a second batch normalization layer, a third LeakyReLU activation function layer and a fifth one-dimensional convolution layer connected in sequence.

[0015] Furthermore, the training of the speech conversion model includes:

[0016] Construct training corpus;

[0017] Perform feature extraction on the training corpus to obtain sample acoustic features;

[0018] Input the sample acoustic features into the target encoder to obtain the sample speaker information representation and sample content representation;

[0019] Input the sample speaker information representation and sample content representation into the decoder to obtain the sample reconstructed acoustic features;

[0020] Construct an objective function with the optimization goal of minimizing the loss between the sample reconstructed acoustic features and the sample acoustic features, and jointly train the encoder and decoder until the objective function meets the preset goal;

[0021] Get the trained speech conversion model.

[0022] Furthermore, the objective function is constructed with minimizing the loss between the sample reconstructed acoustic features and the sample acoustic features as the optimization goal, and the encoder and the decoder are jointly trained until the objective function meets the preset goal, including:

[0023] The objective function is:

[0024] ;

[0025] in: Indicates the parameters corresponding to the encoder, Indicates the parameters corresponding to the decoder, represents the reconstruction loss function, represents the weight of the reconstruction loss function, Represents the Loss loss function;

[0026] The reconstruction loss function is:

[0027] ;

[0028] Where: X represents the acoustic features of the encoder input, represents the reconstructed acoustic features output by the decoder, represents the L1 norm, Represents the expected loss between the reconstructed acoustic features of the decoder and the input acoustic features of the encoder.

[0029] Furthermore, the step of inputting the sample acoustic features into the target encoder to obtain the sample speaker information representation and the sample content representation includes:

[0030] According to the acoustic features of the sample, a first feature vector is obtained;

[0031] Normalizing the first eigenvector to obtain information representation of the sample speaker;

[0032] Normalizing the first feature vector using the sample speaker information representation to obtain a second feature vector;

[0033] The second eigenvector is filtered to obtain a representation of the sample content.

[0034] Furthermore, filtering the second feature vector to obtain the sample content representation includes:

[0035] ;

[0036] ;

[0037] ;

[0038] ;

[0039] Where: C represents the sample content representation, represents the second eigenvector, represents the first eigenvector; Indicates that given a batch n and the number of channels c, along the c direction The i-th frame size, L represents the frame length;

[0040] The sample speaker information representation includes and , Indicates that given a batch n and the number of channels c, calculation is done along the c direction The mean of Indicates that given a batch n and the number of channels c, calculation is done along the c direction The standard deviation of Represents a constant, used to prevent the standard deviation from being 0;

[0041] represents a hyperparameter, with a value of 0.1;

[0042] express function, represents instance normalization, exp( ) represents an exponential function with a natural number e as its base.

[0043] Furthermore, the step of inputting the sample speaker information representation and the sample content representation into a decoder to obtain the sample reconstructed acoustic features includes:

[0044] Extract the representation of the sample content to obtain the latent representation;

[0045] The sample speaker information representation and potential representation are normalized to obtain the sample reconstructed acoustic features.

[0046] Furthermore, the normalization of the speaker information representation and the potential representation of the sample to obtain the reconstructed acoustic features of the sample includes:

[0047] ;

[0048] in: represents the acoustic features of sample reconstruction, Represents the potential representation, the sample speaker information representation includes and , Indicates that given a batch n and the number of channels c, calculation is done along the c direction The mean of represents the first eigenvector;

[0049] represents instance normalization, stands for Adaptive Instance Normalization.

[0050] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the cross-language speech conversion method as described above is implemented.

[0051] The present invention also provides a computer device, comprising:

[0052] A memory for storing instructions;

[0053] The processor is used to execute the instructions so that the device implements the cross-language speech conversion method as described above.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] 1. The present invention reconstructs the first acoustic feature of the source speech and the second acoustic feature of the target speech into a reconstructed acoustic feature through a trained speech conversion model. By decoupling the acoustic features, the potential information of the acoustic features can be fully utilized, the quality of the synthesized speech processed by the reconstructed acoustic feature conversion is improved, and the number of parameters and the amount of calculation of the model are reduced, thereby solving the problem that the current cross-language speech conversion method adopts a model that cannot well balance the number of parameters and the amount of calculation, and the conversion effect is not ideal.

[0056] 2. The encoding processing layer of the encoder of the present invention uses bundled block convolution to replace the original standard convolution. The bundled block convolution is based on grouped convolution and shares the same thinner filter group among all grouped sub-features. Since a single filter group acts on all input grouped sub-features to produce multiple responses, the bundled block convolution can not only aggregate global information across channels of the input features and better model cross-channel dependencies, thereby improving the expressiveness of the features, but also greatly reduce the number of parameters and the amount of computation of the model by sharing the same filter group on all input grouped sub-features. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a flow chart of a cross-language speech conversion method provided by an embodiment of the present invention;

[0058] Figure 2 Schematic diagram of the conversion principle of a cross-language speech conversion method provided by an embodiment of the present invention;

[0059] Figure 3 It is a flow chart of speech conversion model training of a cross-language speech conversion method provided by an embodiment of the present invention;

[0060] Figure 4 It is a schematic diagram of the training conversion principle of a speech conversion model of a cross-language speech conversion method provided by an embodiment of the present invention;

[0061] Figure 5 Schematic diagram of the encoder structure of the speech conversion model in an embodiment of the present invention;

[0062] Figure 6 Schematic diagram of the decoder structure of the speech conversion model in the embodiment of the present invention. DETAILED DESCRIPTION

[0063] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0064] In the description of the present invention, it is to be understood that the terms "first", "second", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", etc. may explicitly or implicitly include one or more of the feature. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0065] Embodiment 1

[0066] like Figure 1 As shown, a cross-language speech conversion method includes:

[0067] Obtain the first acoustic feature of the source speech and the second acoustic feature of the target speech; in this example, the source speech and the target speech are from the AISHELL-3 Chinese corpus and the CSTR VCTK English corpus, specifically:

[0068] In the Chinese and English corpora, 8 speakers were randomly selected from the AISHELL-3 Chinese corpus. Each speaker had about 200 voices. Ten voices were randomly selected from the 200 voices, and the duration of each voice varied from 2 to 10 seconds. 8 speakers were selected from the CSTR VCTK English corpus. Each speaker had about 200 voices. Ten voices were randomly selected from the 200 voices, and the duration of each voice varied from 2 to 8 seconds.

[0069] The selected Chinese and English speech are combined into the corpus to be converted, the logarithmic Mel spectrum feature of the source speech in the corpus to be converted is extracted, represented as the first acoustic feature X1, and the logarithmic Mel spectrum feature of the target speech in the corpus to be converted is extracted, represented as the second acoustic feature X2;

[0070] like Figure 2 , Figure 5 and Figure 6 As shown, the first acoustic feature X1 and the second acoustic feature X2 are input into the trained speech conversion model, wherein the speech conversion model includes an encoder and a decoder, specifically:

[0071] The input of the encoder is the first acoustic feature and the second acoustic feature, and the output is the speaker information representation and the content representation; the encoder includes a first one-dimensional convolution layer, 6 groups of encoding processing layers, a second one-dimensional convolution layer and an activation guidance function layer connected in sequence. It should be noted that Figure 5A represents the encoding processing layer, wherein the encoding processing layer includes the first bundled block convolution layer, the first normalization layer, the first LeakyReLU activation function layer, the third one-dimensional convolution layer and the first instance normalization layer connected in sequence; the bundled block convolution performs standard convolution operations by sharing the same set of filters between the channels of all input sub-features, which can not only aggregate global information across channels of the input features and better model cross-channel dependencies, thereby improving the expressiveness of the features, but also greatly reducing the number of parameters and the amount of calculation of the model;

[0072] The input of the decoder is the speaker information representation and content representation, and the output is the reconstructed acoustic features;

[0073] The decoder includes a second instance normalization layer, a first adaptive instance normalization layer, a fourth-dimensional convolution layer, a second LeakyReLU activation function layer, 6 groups of decoding processing layers, a gated recurrent unit layer and a fully connected layer. It should be noted that: Figure 6 B represents a decoding processing layer, wherein the decoding processing layer includes a decoding convolution module, a generating convolution module and a second adaptive instance normalization layer connected in sequence;

[0074] The structure of the generating convolution module is the same as that of the decoding convolution module. The decoding convolution module includes the second bundled block convolution layer, the second batch normalization layer, the third LeakyReLU activation function layer and the fifth one-dimensional convolution layer connected in sequence. Since the structure of the generating convolution module is the same as that of the decoding convolution module, Figure 6 The structure of a set of decoding processing layers is shown in the form of a decoding convolution module × 2;

[0075] Obtain the content representation C1 of the source speech and the speaker information representation S2 of the target speech;

[0076] The content representation C1 and the speaker information representation S2 are used as the input of the decoder to obtain the reconstructed transformed logarithmic Mel spectrum feature D (C1, S2), which is represented as the reconstructed acoustic feature X 1→2 ;

[0077] Using the Griffin-Lim vocoder, the reconstructed acoustic feature X 1→2 Convert it into speech waves for output, thus obtaining synthesized speech with cross-language speech conversion.

[0078] like Figure 3 As shown, the training of the speech conversion model includes:

[0079] Construct a training corpus. In this example, the corpus in the training corpus comes from the AISHELL-3 Chinese corpus and the CSTR VCTK English corpus. Specifically:

[0080] 40 speakers were selected from the AISHELL-3 Chinese corpus, each with about 200 voices, each with a duration of 2 to 10 seconds; 40 speakers were selected from the CSTR VCTK English corpus, each with about 200 voices, each with a duration of 2 to 8 seconds; the selected Chinese and English voices were combined to construct a training corpus;

[0081] like Figure 4 As shown, feature extraction is performed on the training corpus to obtain sample acoustic features, specifically:

[0082] The sampling rate of the training corpus is set to 24kHz, the number of fast Fourier transform points is set to 2048, the window length is set to 1200, and the window moving length is set to 300. 128 frames are randomly selected from the logarithmic Mel spectrum features of each training speech and concatenated to obtain the logarithmic Mel spectrum feature fragments, which are represented as sample acoustic features X;

[0083] Input the sample acoustic feature X into the target encoder to obtain the sample speaker information representation and sample content representation. Specifically:

[0084] According to the sample acoustic feature X, a first feature vector is obtained, wherein the sample acoustic feature X is subjected to the first one-dimensional convolution layer to obtain a preliminary feature vector Z, and the preliminary feature vector Z is processed by the bundled block convolution layer, the batch normalization layer, the LeakyReLU activation function layer, and the one-dimensional convolution layer of the encoding processing layer to obtain the first feature vector;

[0085] Preliminary feature vector , as the input feature of the bundled block convolutional layer, where represents the set of real numbers, C in represents the number of channels of the input feature, H1 represents the height of the input feature, and W1 represents the width of the input feature; the output features of the bundled block convolutional layer , where C out Represents the number of channels of the output feature, H2 represents the height of the output feature, and W2 represents the width of the output feature;

[0086] The bundled block convolution operation does not involve changes in the height and width of the feature, so the height and width of the output feature are consistent with the input feature vector: H1= H2, W1= W2;

[0087] In the bundled block convolution, the preliminary feature vector Z as the input feature is divided into G groups of sub-features by channel, and the number of input channels of each group of sub-features is C in / G, the sub-features after division can be expressed as , the output feature V is composed of G groups of output sub-features concatenated from the channel dimension, so the number of channels corresponding to each group of output sub-features is C out / G, since the number of channels of each filter is equal to the number of input channels of the input feature, and the number of filters in each group needs to be equal to the number of channels of each group of output features, each filter can be expressed as ,in represents the i-th filter, and K represents the size of the filter;

[0088] Then the bundled convolution operation can be expressed as:

[0089]

[0090] represents the standard convolution operation, Represents feature concatenation operation;

[0091] The number of convolution parameters is equal to the number of filters Number of filter channels The filter is high The width of the filter, the amount of convolution calculation is equal to the number of filters Number of filter channels The filter is high Filter width The input feature height The width of the input feature. Since the bundled block convolution applies the filter group defined on a set of input sub-features to all input sub-features, the number of parameters of the bundled block convolution is expressed as: ; The initial feature vector Z as the input feature is convolved by the bundled block to obtain the output feature V. The computational complexity of the bundled block convolution in this process is expressed as: ;

[0092] In summary, compared with the number of parameters and computation required to perform standard convolution operations on input features, the number of parameters and computation required for bundled block convolution operations are reduced to the original .

[0093] Normalize the first eigenvector to obtain the sample speaker information representation S;

[0094] Normalizing the first feature vector using the sample speaker information representation to obtain a second feature vector;

[0095] The second feature vector is filtered to obtain the sample content representation, where: ;

[0096] ;

[0097] ;

[0098] ;

[0099] C represents the sample content representation, represents the second eigenvector, represents the first eigenvector; Indicates that given a batch n and the number of channels c, along the c direction The i-th frame size, L represents the frame length;

[0100] The sample speaker information representation S includes and , Indicates that given a batch n and the number of channels c, calculation is done along the c direction The mean of Indicates that given a batch n and the number of channels c, calculation is done along the c direction The standard deviation of Represents a constant, used to prevent the standard deviation from being 0; represents a hyperparameter, with a value of 0.1; express function, represents instance normalization, exp( ) represents an exponential function with a natural number e as its base.

[0101] Input the sample speaker information representation S and sample content representation C into the decoder to obtain the sample reconstructed acoustic features, where:

[0102] Extract the representation of the sample content representation C to obtain the latent representation;

[0103] Normalize the sample speaker information representation S and potential representation to obtain the sample reconstructed acoustic features: ;

[0104] represents the acoustic features of sample reconstruction, Represents the potential representation, the sample speaker information representation includes and , Indicates that given a batch n and the number of channels c, calculation is done along the c direction The mean of represents the first eigenvector; represents instance normalization, stands for Adaptive Instance Normalization.

[0105] Construct an objective function with the optimization goal of minimizing the loss between the sample reconstructed acoustic features and the sample acoustic features, and jointly train the encoder and decoder until the objective function meets the preset goal. Specifically:

[0106] The objective function is:

[0107]

[0108] in: Indicates the parameters corresponding to the encoder, Indicates the parameters corresponding to the decoder, represents the reconstruction loss function, represents the weight of the reconstruction loss function, Represents the Loss loss function;

[0109] The reconstruction loss function is:

[0110] ;

[0111] Where: X represents the acoustic features of the encoder input, represents the reconstructed acoustic features output by the decoder, represents the L1 norm, represents the expected loss between the reconstructed acoustic features of the decoder and the input acoustic features of the encoder;

[0112] Get the trained speech conversion model.

[0113] Embodiment 2

[0114] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the following method:

[0115] Acquiring a first acoustic feature of a source speech and a second acoustic feature of a target speech;

[0116] Inputting the first acoustic feature and the second acoustic feature into a trained speech conversion model to obtain reconstructed acoustic features of the source speech and the target speech; wherein the speech conversion model includes an encoder and a decoder;

[0117] The input of the encoder is acoustic features, and the output is speaker information representation and content representation;

[0118] The input of the decoder is speaker information representation and content representation, and the output is reconstructed acoustic features;

[0119] The reconstructed acoustic features are converted to obtain a synthesized speech that is a cross-language conversion of the source speech and the target speech.

[0120] Embodiment 3

[0121] A computer device comprising:

[0122] A memory for storing instructions;

[0123] The processor is configured to execute the instructions so that the device performs the steps of implementing the following method:

[0124] Acquiring a first acoustic feature of a source speech and a second acoustic feature of a target speech;

[0125] Inputting the first acoustic feature and the second acoustic feature into a trained speech conversion model to obtain reconstructed acoustic features of the source speech and the target speech; wherein the speech conversion model includes an encoder and a decoder;

[0126] The input of the encoder is acoustic features, and the output is speaker information representation and content representation;

[0127] The input of the decoder is speaker information representation and content representation, and the output is reconstructed acoustic features;

[0128] The reconstructed acoustic features are converted to obtain a synthesized speech that is a cross-language conversion of the source speech and the target speech.

[0129] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0130] The present application is described with reference to the flowcharts of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0131] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A function specified in a process or multiple processes.

[0132] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 The steps of a specified function in a process or multiple processes.

[0133] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A cross-language speech conversion method, characterized in that: include: Acquiring a first acoustic feature of a source speech and a second acoustic feature of a target speech; Inputting the first acoustic feature and the second acoustic feature into the trained speech conversion model to obtain reconstructed acoustic features of the source speech and the target speech; The speech conversion model includes an encoder and a decoder; the encoder input is the first acoustic feature and the second acoustic feature, and the output is the speaker information representation and the content representation; the decoder input is the speaker information representation and the content representation, and the output is the reconstructed acoustic feature; The reconstructed acoustic features are converted to obtain a synthesized speech that is a cross-language conversion of the source speech and the target speech.

2. The cross-language speech conversion method according to claim 1, characterized in that: The encoder includes a first one-dimensional convolutional layer, a plurality of groups of encoding processing layers, a second one-dimensional convolutional layer and an activation guidance function layer connected in sequence, wherein the encoding processing layer includes a first bundled block convolutional layer, a first batch of normalization layers, a first LeakyReLU activation function layer, a third one-dimensional convolutional layer and a first instance normalization layer connected in sequence; The decoder includes a second instance normalization layer, a first adaptive instance normalization layer, a fourth one-dimensional convolution layer, a second LeakyReLU activation function layer, multiple groups of decoding processing layers, a gated recurrent unit layer and a fully connected layer connected in sequence, wherein the decoding processing layer includes a decoding convolution module, a generating convolution module and a second adaptive instance normalization layer connected in sequence; The generation convolution module has the same structure as the decoding convolution module, and the decoding convolution module includes a second bundling block convolution layer, a second batch normalization layer, a third LeakyReLU activation function layer and a fifth one-dimensional convolution layer connected in sequence.

3. The cross-language speech conversion method according to claim 1, characterized in that: The training of the speech conversion model includes: Construct training corpus; Perform feature extraction on the training corpus to obtain sample acoustic features; Input the sample acoustic features into the target encoder to obtain the sample speaker information representation and sample content representation; Input the sample speaker information representation and sample content representation into the decoder to obtain the sample reconstructed acoustic features; Construct an objective function with the optimization goal of minimizing the loss between the sample reconstructed acoustic features and the sample acoustic features, and jointly train the encoder and decoder until the objective function meets the preset goal; Get the trained speech conversion model.

4. The cross-language speech conversion method according to claim 3, characterized in that: The objective function is constructed with minimizing the loss between the sample reconstructed acoustic features and the sample acoustic features as the optimization goal, and the encoder and the decoder are jointly trained until the objective function meets the preset goal, including: The objective function is: ； in: Indicates the parameters corresponding to the encoder, Indicates the parameters corresponding to the decoder, represents the reconstruction loss function, represents the weight of the reconstruction loss function, Represents the Loss loss function; The reconstruction loss function is: ； Where: X represents the acoustic features of the encoder input, represents the reconstructed acoustic features output by the decoder, represents the L1 norm, Represents the expected loss between the reconstructed acoustic features of the decoder and the input acoustic features of the encoder.

5. The cross-language speech conversion method according to claim 3, characterized in that: The step of inputting the sample acoustic features into the target encoder to obtain the sample speaker information representation and the sample content representation includes: According to the acoustic features of the sample, a first feature vector is obtained; Normalizing the first eigenvector to obtain information representation of the sample speaker; Normalizing the first feature vector using the sample speaker information representation to obtain a second feature vector; The second eigenvector is filtered to obtain a representation of the sample content.

6. The cross-language speech conversion method according to claim 5, characterized in that: The filtering process of the second feature vector to obtain the sample content representation includes: ； ； ； ； Where: C represents the sample content representation, represents the second eigenvector, represents the first eigenvector; Indicates that given a batch n and the number of channels c, along the c direction The i-th frame size, L represents the frame length; The sample speaker information representation includes and , Indicates that given a batch n and the number of channels c, calculation is done along the c direction The mean of Indicates that given a batch n and the number of channels c, calculation is done along the c direction The standard deviation of Represents a constant, used to prevent the standard deviation from being 0; represents a hyperparameter, with a value of 0.1; express function, represents instance normalization, exp( ) represents an exponential function with a natural number e as its base.

7. The cross-language speech conversion method according to claim 5, characterized in that: The step of inputting the sample speaker information representation and the sample content representation into the decoder to obtain the sample reconstructed acoustic features includes: Extract the representation of the sample content to obtain the latent representation; The sample speaker information representation and potential representation are normalized to obtain the sample reconstructed acoustic features.

8. The cross-language speech conversion method according to claim 7, characterized in that: The normalization of the speaker information representation and potential representation of the sample to obtain the reconstructed acoustic features of the sample includes: ； in: represents the acoustic features of sample reconstruction, Represents the potential representation, the sample speaker information representation includes and , Indicates that given a batch n and the number of channels c, calculation is done along the c direction The mean of represents the first eigenvector; represents instance normalization, stands for Adaptive Instance Normalization.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the cross-language speech conversion method according to any one of claims 1 to 8 is implemented.

10. A computer device, characterized in that: include: A memory for storing instructions; A processor is used to execute the instruction so that the device implements the cross-language speech conversion method according to any one of claims 1 to 8.

Images