Voice synthesizer, voice synthesis method, and computer program therefor

Abstract

The present invention relates to a voice synthesizer (10) comprising: - a projection module (12) configured to project, via a first unidirectional recurrent neural network, input parameters characterizing each frame of an input sound signal comprising a human voice, in a latent space having a parameterizable dimension; - a harmonic signal generator (14) configured to generate, via a second neural network, a signal representative of the harmonics of said input sound signal; - a voice synthesizer (16) configured to synthesize, via a third unidirectional recurrent neural network, an encoded voice associated with said human voice from: - the projection of the projected input parameters into said latent space having a parameterizable dimension, and - the signal representative of the harmonics of the input sound signal comprising the human voice and provided by the generator.

Description

TITLE: Voice synthesizer, voice synthesis method, and computer program therefor The present invention relates to a voice synthesizer having an architecture according to a neural source-filter model.The present invention also relates to a speech synthesis method implemented by said voice synthesizer.The invention also relates to a computer program comprising software instructions which, when implemented by said voice synthesizer, implement such a method. The invention relates to the field of vocoding, speech synthesis and voice transmission.Hereinafter, according to the present invention, “vocoding” and, by extension, the corresponding device “voice synthesizer” are associated with the contraction of the English words Voice Coding and Voice coder respectively, in other words, voice coding by an electronic device for processing the sound signal, substantially corresponding to a method according to which, from a decomposition into important properties (e.g. main spectral components) of an input voice or of another sound, an associated synthetic voice is synthesized at the output.In recent years, vocoding solutions comprising artificial intelligence AI capabilities have been proposed. However, such solutions are generally very demanding in terms of input parameters, requiring for example at the input a number on the order of eighty Mel frequencies to provide speech synthesis. In addition, such AI solutions are generally not designed to allow real-time implementation, favoring operation suited to uses of the text-to-speech type (from the English text-to-speech) or voice assistants. Furthermore, the number of parameters generally associated with the implementation of such AI solutions, on the order of ten million parameters, reduces the possibilities of implementations in embedded hardware with strong memory and computing capacity constraints.In conventional low-bitrate digital voice synthesizers, the input parameters are often limited in number and compressed so as to use the least possible bitrate, however such a reduction in the number of input parameters used by the voice synthesizer is generally accompanied by a degradation of the quality of the speech synthesis. Thus, conventional low-bitrate solutions generally suffer from a rather significant deterioration of the synthesized voice, these conventional low-bitrate solutions being unable to compensate for the lack of information with a priori information, as is the case for the aforementioned AI solutions.For example, for these conventional low-bitrate solutions, reducing the number of input frequencies causes a deterioration of the high-frequency components, and without a compensation mechanism, conventional low-bitrate voice synthesizers often provide a very robotic voice.In summary, currently, the reduction in the number of input parameters is therefore systematically accompanied by a reduction in the quality of speech synthesis, and conventional low-bitrate voice synthesizers cannot compensate for the loss of information linked to the reduction in the number of parameters, whereas AI voice synthesizers compensate for the loss of information but are often characterized by heavy numerical computations, as well as by a significant number of parameters implemented, these properties being inherent in the fact that AI voice synthesizers learn an a priori distribution to compensate for areas without information.The object of the invention is therefore to provide a voice synthesizer suitable for synthesizing a human voice of sufficient quality for voice communication while using a very small number of input parameters.For this purpose, the invention relates to a voice synthesizer having an architecture according to a neural source-filter model comprising at least:- a projection module configured to project, via a first unidirectional recurrent neural network, input parameters, characterizing each frame of an input sound signal comprising a human voice, into a latent space of parameterizable dimension;- a harmonic signal generator configured to generate, via a second neural network, a signal representative of the harmonics of said input sound signal comprising said human voice;- a voice synthesizer configured to synthesize, via a third unidirectional recurrent neural network, an encoded voice associated with said human voice from:- the projection of said input parameters projected into said latent space of parameterizable dimension provided by said projection module, and- said signal representative of the harmonics of the input sound signal comprising said human voice and provided by said generator.Such a voice synthesizer is advantageous because, unlike neural source-filter architectures (from the English source-filter) of the literature, the use of unidirectional recurrent neural networks is compatible with real-time use, that is to say with a latency of less than 50 ms.Furthermore, the voice synthesizer according to the present invention is configured to provide a frugal speech synthesis solution, because it is based on a data-efficient architecture. Indeed, the source-filter model is admittedly a conventional model for decomposing the voice into source and filter component(s), however the use of this model, boosted by “modern” unidirectional recurrent neural networks (i.e. suitable for restoring causality) makes it possible to offer a framework conducive to voice modeling. Such “artificial intelligence boosting” results in better reconstruction of the voice with lower memory and computation cost.According to other advantageous aspects of the invention, the voice synthesizer comprises one or more of the following features, taken in isolation or according to all technically possible combinations:- said signal representative of the harmonics of said input sound signal comprising said human voice is a weighted sum of the signal corresponding to the pitch of said human voice and of its harmonics, said weighting having been learned beforehand by said second neural network;- white noise is added to said weighted sum;- said input parameters characterizing said input sound signal comprising said human voice are frequency parameters corresponding to one of the elements belonging to the group comprising at least:- a Fourier spectrum of said input sound signal comprising said human voice;- a cepstrum defined by Mel-frequency cepstral coefficients calculated by a discrete cosine transform applied to the power spectrum of said input sound signal comprising said human voice;- the constant-Q transform of said input sound signal comprising said human voice;- the synthesizer further comprises an adversarial training module comprising two discriminators configured to be trained simultaneously with said voice synthesizer during its supervised learning phase, according to a zero-sum game, said discriminators being trained to differentiate real signals from those suitable for being synthesized by said voice synthesizer, said voice synthesizer being trained to deceive said two discriminators;- one of said two discriminators is sensitive to low-frequency information and the other of said two discriminators is sensitive to high-frequency information;- the synthesizer further comprises:- a reconstruction module configured to reconstruct missing frames of said synthesized encoded voice from frames preceding or following said missing frames, and / or- a band extension module configured to reconstruct a signal comprising said synthesized encoded voice with a sampling frequency higher than that of said input sound signal.- at least one of said neural networks of said voice synthesizer is also configured to extract the vocal biomarkers of the speaker having generated said input sound signal.The invention also relates to a speech synthesis method implemented by such a speech synthesizer having an architecture according to a neural source-filter model, said method comprising the following steps:- projection, via a first unidirectional recurrent neural network of said speech synthesizer, of input parameters, characterizing each frame of an input sound signal comprising a human voice, into a latent space of configurable dimension;- generation, via a second neural network of said speech synthesizer, of a signal representative of the harmonics of said input sound signal comprising said human voice;- speech synthesis, via a third unidirectional recurrent neural network, of an encoded voice associated with said human voice from:- the projection of said projected input parameters into said latent space of configurable dimension provided by said projection module, and- said signal representative of the harmonics of the input sound signal comprising said human voice and provided by said generator.The invention also relates to a computer program comprising software instructions which, when executed by a speech synthesizer, implement a speech synthesis method, as defined above. The invention will become clearer upon reading the following description, given solely by way of a non-limiting example, and made with reference to the drawings in which:[Fig. 1] figure 1 is a general schematic representation of a speech synthesizer according to the present invention;[Fig. 2] figure 2 illustrates the use of the speech synthesizer of figure 1;[Fig. 3] figure 3 illustrates an example of an architecture of the speech synthesizer of figure 1;[Fig. 4] figure 4 illustrates the implementation of an optional module of the speech synthesizer of figure 1 during its training phase; points of interest of said geographical areaFig. 5] figure 5 is a flowchart of a speech synthesis method implemented by the speech synthesizer of figure 1 according to an embodiment. In the remainder of the description, the expression “substantially equal to” is understood as meaning an equality relationship to within plus or minus 10%, that is to say with a variation of no more than 10%, more preferably still as an equality relationship to within plus or minus 5%, that is to say with a variation of no more than 5%.Moreover, subsequently, it is considered that a neural network comprises an ordered succession of neural layers, each of which takes its inputs from the outputs of the preceding layer.More precisely, each layer comprises neurons taking their inputs from the outputs of the neurons of the preceding layer, or from the input variables for the first layer.As a variant, more complex neural network structures may be contemplated with a layer which may be connected to a layer more distant than the immediately preceding layer.Each neuron is also associated with an operation, that is to say a type of processing, to be performed by said neuron within the corresponding processing layer.Each layer is connected to the other layers by a plurality of synapses. A synaptic weight is associated with each synapse, and each synapse forms a connection between two neurons. It is often a real number, which takes positive as well as negative values. In some cases, the synaptic weight is a complex number.Each neuron is capable of performing a weighted sum of the value(s) received from the neurons of the preceding layer, each value then being multiplied by the respective synaptic weight of each synapse, or connection, between said neuron and the neurons of the preceding layer, then applying an activation function, typically a non-linear function, to said weighted sum, and delivering at the output of said neuron, in particular to the neurons of the following layer which are connected thereto, the value resulting from the application of the activation function. The activation function makes it possible to introduce a non-linearity into the processing performed by each neuron. The sigmoid function, the hyperbolic tangent function, and the Heaviside function are examples of an activation function.As an optional addition, each neuron is also capable of additionally applying a multiplicative factor, also called a bias, to the output of the activation function, and the value delivered at the output of said neuron is then the product of the bias value and the value resulting from the activation function.A fully connected neural layer is a layer in which the neurons of said layer are each connected to all the neurons of the preceding layer.Such a type of layer is more often referred to by the English term “fully connected”, and is sometimes designated by the name “dense layer”. The electronic sound signal processing device 10, also subsequently called a speech synthesizer, is illustrated in figure 1. As can be seen in this figure 1, the speech synthesizer 10 comprises a projection module 12 configured to project, via a first unidirectional recurrent neural network RN1, input parameters, characterizing each frame of an input sound signal comprising a human voice, into a latent space (from the English embedding) of configurable dimension.Such a projection module 12 is also called an encoder, and is therefore capable of “embedding” the input parameters, characterizing each frame of an input sound signal comprising a human voice, into a latent space (i.e. a vector space) of configurable dimension, for example of a size equal to 128 (i.e. capable of comprising 128 components). The neural network RN1, previously trained as will be seen subsequently, handles the non-linear projection of said input parameters into the latent space potentially containing more information than the input parameters since temporal relationships may be incorporated therein owing to the recurrent and unidirectional nature of the neural network RN1.The input parameters capable of being handled by said projection module 12 are parameters which characterize said input sound signal comprising said human voice and correspond in particular to frequency parameters corresponding to one of the elements belonging to the group comprising at least:- a Fourier spectrum of said input sound signal comprising said human voice;- a cepstrum defined by Mel-frequency cepstral coefficients calculated by a discrete cosine transform applied to the power spectrum of said input sound signal comprising said human voice;- the constant-Q transform CQT (from the English constant-Q transform) of said input sound signal comprising said human voice- etc.As illustrated by figure 1, the speech synthesizer 10 further comprises a harmonic signal generator 14 configured to generate, via a second neural network RN2, a signal representative of the harmonics of said input sound signal comprising said human voice.As an optional addition, such a harmonic signal generator 14 is configured to generate said signal representative of the harmonics of said input sound signal comprising said human voice which is a weighted sum of the signal corresponding to the pitch of said human voice and its harmonics, said weighting having been previously learned by said second neural network.According to a variant of this optional addition, white noise is added to said weighted sum.In other words, said harmonic signal generator 14 is configured to produce a signal composed of a sinusoidal signal at the pitch of the voice and its harmonics. The output signal of this generator 14 is a weighted sum of the pitch of the voice and its harmonics. The weighting is learned by the second neural network RN2 during a prior training phase. White noise is added to this harmonic signal to serve as a basis for the stochastic components of the voice. The unvoiced frames or frames not containing voice are composed only of white noise without a harmonic component.According to a voice synthesizer architecture according to the known “source-filter” model, the output generated by the harmonic signal generator 14 plays the role of the “source” part of this “source-filter” architecture.As illustrated by Figure 1, the voice synthesizer 10 further comprises a voice synthesizer 16 configured to synthesize, via a third unidirectional recurrent neural network RN3, an encoded voice associated with said human voice from:- the projection of said projected input parameters in said latent space of parameterizable dimension provided by said projection module 12, and- said signal representative of the harmonics of the input sound signal comprising said human voice and provided by said generator 14.Such a voice synthesizer 16 is also called a decoder, and like the preceding elements 12 and 14 of the voice synthesizer 10 is also based on a recurrent neural network architecture. Such a voice synthesizer 16 therefore has as inputs the output of the projection model 12 and the output of the harmonic signal generator 14. The projection model 12 and the harmonic signal generator 14 are suitable for operating in parallel or successively according to any order, provided that the output of the projection model 12 and the output of the harmonic signal generator 14 associated with the same frame of the input sound signal are processed simultaneously by said voice synthesizer 16.According to a voice synthesizer architecture according to the known “source-filter” model, the voice synthesizer 16 plays the role of the “filter” part of this “source-filter” architecture.Thus, unlike the neural source-filter architectures according to the current state of the art, the use of unidirectional recurrent neural networks proposed according to the present invention makes it possible to operate in real time.For example, the voice synthesizer 10 according to the present invention is suitable for operating with an input sound signal decomposed beforehand into frames of 40ms with a 40ms anticipation buffer (in English look ahead), essentially due to the extraction of the spectral parameters at the input of the projection module 12 (i.e. of the encoder).As an optional addition, as represented by dotted lines, said voice synthesizer 10 further comprises an adversarial training module 18 comprising two discriminators D1 and D2 configured to be trained simultaneously with said voice synthesizer 10 during its supervised learning phase, according to a zero-sum game, said discriminators D1 and D2 being trained to differentiate real signals from those suitable for being synthesized by said voice synthesizer 10, said voice synthesizer 10 being trained to deceive said two discriminators D1 and D2.It should be noted that once the learning phase has been carried out, such an adversarial training module 18 is no longer used during the inference phase using the voice synthesizer 10 thus previously trained adversarially.According to an advantageous variant of this optional addition, one of said two discriminators is sensitive to low-frequency information and the other of said two discriminators is sensitive to high-frequency information.In other words, the two discriminators D1 and D2 are trained at the same time as the voice synthesizer 10 to differentiate real signals from signals synthesized by the voice synthesizer 10 by creating a zero-sum game in which the discriminators D1 and D2 attempt to differentiate the signals created by the voice synthesizer 10 from the original signals and in which the voice synthesizer 10 is trained to deceive the two discriminators D1 and D2.Such a zero-sum game makes it possible to balance the voice synthesizer resulting from said adversarial training, and the use of two discriminators, sensitive to low- and high-frequency information, makes it possible for the voice synthesizer 10 to correct its defects over extended frequency ranges, and is particularly effective for removing the metallic artifacts from the voice synthesized via the voice synthesizer 10.For example, the two discriminators correspond to the discriminators D1 and D2 described in the article by J. Kong et al. entitled “HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis” NeurIPS 2020, but applied in another context, without combined implementation of a voice synthesizer architecture according to a source-filter model, and of the zero-sum game according to the present invention in which said discriminators D1 and D2 are trained to differentiate real signals from those suitable for being synthesized by said voice synthesizer 10, said voice synthesizer 10 being trained to deceive said discriminators D1 and D2.The adversarial training implemented via said optional adversarial training module 18 is described further below in relation to Figure 4.Such an optional adversarial training module 18 therefore makes it possible to improve the synthesis quality.As an optional addition, said voice synthesizer 10 further comprises a reconstruction module 20 configured to reconstruct (in English inpainting) missing frames of said synthesized encoded voice from the frames preceding or following said missing frames, and / or a bandwidth extension module 22 configured to reconstruct a signal comprising said synthesized encoded voice with a sampling frequency higher than that of said input sound signal.The optional reconstruction module 20 makes it possible to improve the overall transmission of the signal synthesized via said voice synthesizer 10 by compensating for the micro-cuts that would occur without its implementation due to missing frame(s), while the optional bandwidth extension module 22 makes it possible to increase the output sampling frequency.As an optional addition, at least one of said neural networks of said voice synthesizer 10, for example the first unidirectional recurrent neural network RN1 of the projection module 12 or else the second neural network RN2, is also configured to extract the vocal biomarkers of the speaker having generated said input sound signal, after having been previously trained for this purpose in a supervised manner.In a non-represented variant, the voice synthesizer 10 is also suitable for optionally comprising another neural network dedicated to extracting the vocal biomarkers of the speaker having generated said input sound signal after having been previously trained for this purpose in a supervised manner.In the example of Figure 1, the voice synthesizer 10 comprises an information processing unit 26 formed, for example, by a memory 28 and a processor 30 associated with the memory 28.In the example of Figure 1, the projection module 12, the generator 14 and the voice synthesizer 16, as well as, as an optional addition, the adversarial training module 18, the reconstruction module 20 and the bandwidth extension module 22, are each produced in the form of software, or a software block, executable by the processor 30. The memory 28 of the voice synthesizer 10 is then suitable for storing projection software, harmonic signal generation software, and voice synthesis software, as well as, as an optional addition, adversarial training software, reconstruction software and bandwidth extension software. The processor is then suitable for executing each of the software among the projection software, the harmonic signal generation software, and the voice synthesis software, as well as, as an optional addition, the adversarial training software, the reconstruction software, the bandwidth extension software.In a non-represented variant, the projection module 12, the generator 14 and the voice synthesizer 16, as well as, as an optional addition, the adversarial training module 18, the reconstruction module 20 and the bandwidth extension module 22, are each produced in the form of a programmable logic component, such as an FPGA (in English Field Programmable Gate Array), or else an integrated circuit, such as an ASIC (in English Application Specific Integrated Circuit).When the voice synthesizer 10 is produced in the form of one or more software programs, that is to say in the form of a computer program, also called a computer program product, it is further suitable for being recorded on a medium, not represented, readable by a computer. The computer-readable medium is for example a medium suitable for storing electronic instructions and for being coupled to a bus of a computer system. By way of example, the readable medium is an optical disc, a magneto-optical disc, a ROM memory, a RAM memory, any type of non-volatile memory (for example FLASH or NVRAM) or a magnetic card. A computer program comprising software instructions is then stored on the readable medium.Figure 2 illustrates the use 32 of the voice synthesizer of Figure 1. According to this use, a speaker 34 produces a sound signal 36 comprising his own voice. Such a sound signal 36 is previously decomposed by means of a known decomposition tool into input parameters 38 characterizing it.These input parameters 38 are in particular frequency parameters corresponding to one of the elements belonging to the group comprising at least:- a Fourier spectrum of said input sound signal comprising said human voice;- a cepstrum defined by Mel-frequency cepstral coefficients calculated by a discrete cosine transform applied to the power spectrum of said input sound signal comprising said human voice;- the constant-Q transform of said input sound signal comprising said human voice.These input parameters 38 are provided at the input of the projection module 40 (corresponding to the projection module 12 of Figure 2) which comprises the first unidirectional recurrent neural network RN1 configured for projecting said input parameters into a latent space of parameterizable dimension.In parallel, said input sound signal 36 comprising said human voice is supplied as input to harmonic signal generator 42 (corresponding to generator 14 of Figure 1), said generator 42 supplying at its output a signal 44 representative of the harmonics of said input sound signal comprising said human voice.Said signal 44 representative of the harmonics of said input sound signal comprising said human voice as well as the result supplied by projection module 40 are associated with the same frame of input sound signal 36 and simultaneously supplied as input to voice synthesizer 46 configured to synthesize, via a third unidirectional recurrent neural network RN3, an encoded voice 48 associated with said human voice of input sound signal 36.Figure 3 illustrates an example 50 of an architecture of the voice synthesizer of Figure 1. According to this example 50, the voice synthesizer receives as input sound signal 52 comprising a human voice.As illustrated by the architecture example 50, projection module 12 optionally first comprises a module 54 suitable for extracting the parameters that characterize said input sound signal 52 comprising said human voice and correspond in particular to frequency parameters.According to example 50 of Figure 3, these parameters correspond to the cepstrum defined by Mel frequency cepstral coefficients MFCC (from the English Mel-Frequency Cepstral Coefficients) calculated by a discrete cosine transform applied to the power spectrum of said input sound signal 52 comprising said human voice.Projection module 12 further comprises the first unidirectional recurrent neural network RN1 configured to project these MFCC coefficients into a latent space of parameterizable dimension, for example of dimension 128 (i.e. capable of comprising 128 components).As illustrated by example 50 of Figure 3, the first unidirectional recurrent neural network RN1 successively comprises a normalization layer LN (from the English Layer-norm), followed by three dense layers, a gated recurrent unit GRU (from the English Gate Recurrent Unit) and a dense layer, so as to supply at its output the projection Z of said MFCC parameters in the latent space of parameterizable dimension.As a variant of the GRU unit, a long short-term memory layer of the LSTM type (from the English Long Short Term Memory) or, even more conventionally, a recurrent RNN layer (from the English Recurrent Neural Network) is used.It should be noted that according to another implementation option, not represented, projection module 12 does not comprise module 54 suitable for extracting the parameters that characterize said input sound signal 52 but directly receives these parameters, determined outside the voice synthesizer, as input to unidirectional recurrent neural network RN1.Moreover, as illustrated by architecture example 50, harmonic signal generator 14 optionally first comprises a module 56 suitable for extracting the pitch of input sound signal 52.Harmonic signal generator 14 further comprises a second neural network RN2 comprising in particular a Sin Gen layer, configured to generate a sinusoidal signal at the pitch of the voice, followed by a dense layer so that the signal produced by generator 14 is a signal composed of a sinusoidal signal at the pitch of the voice and its harmonics, this signal corresponding to a weighted sum E of the pitch of the voice and its harmonics. The weighting is learned by the second neural network RN2 during a prior training phase. The output E generated by harmonic signal generator 14 plays the role of the “source” part of this “source-filter” architecture.Projection model 12 and harmonic signal generator 14 are suitable for operating in parallel or successively according to any order, provided that output Z of projection model 12 and output E of harmonic signal generator 14 associated with the same frame of input sound signal are supplied as input and processed simultaneously by said voice synthesizer 16.From said inputs Z and E, voice synthesizer 16, via its third unidirectional recurrent neural network RN3, is configured to synthesize an encoded voice associated with said human voice of said input signal 52.More precisely, voice synthesizer 16 plays the role of the “filter” part of this “source-filter” architecture of the voice synthesizer according to the present invention. According to example 50 of Figure 3, the third unidirectional recurrent neural network RN3 of voice synthesizer 16 successively comprises a first dense layer corresponding in particular to a conventional perceptron and having only input E as its input.This first dense layer of the third unidirectional recurrent neural network RN3 is followed by two successive filtering units F1 and F2 (from the English Filter Unit). The first filtering unit F1 receives two inputs, namely both output Z of projection model 12 and the output of the first dense layer, whereas the second filtering unit F2, placed after the first filtering unit F1, also receives two inputs, namely both output Z of projection model 12 and the output of the first filtering unit F1.These two filtering units F1 and F2 have an identical structure as illustrated in zoom 57 inside said filtering unit structure.More precisely, each filtering unit F1 or F2 first comprises a concatenation layer suitable for concatenating the two received inputs represented, on the one hand, by the letter “y” (corresponding to output S of the first dense layer of the third unidirectional recurrent neural network RN3 for the first filtering unit F1 or to output S11 of the first filtering unit F1 for the second filtering unit F2), and, on the other hand, by the letter “Z” corresponding to output Z of projection model 12.At the output of said concatenation layer, each filtering unit F1 or F2 comprises a gated recurrent unit GRU (from the English Gate Recurrent Unit).As a variant of the GRU unit, a long short-term memory layer of the LSTM type (from the English Long Short Term Memory) or, even more conventionally, a recurrent RNN layer (from the English Recurrent Neural Network) is used.The GRU unit of each filtering unit F1 or F2 provides two outputs denoted in zoom 57 S1 and S2 each as input of a dense layer, the input represented by the letter “y”, as specified above, being added as illustrated within zoom 57 of Figure 3 to output S1 of one of these two dense layers.For the first filtering unit F1, outputs S1 and S2 indicated in zoom 57 correspond respectively to outputs S11 and S12, whereas for the second filtering unit F2, outputs S1 and S2 indicated in zoom 57 correspond respectively to outputs S21 and S22.Finally, as illustrated by example 50 of Figure 3, output S21 of the second filtering unit F2 is supplied as input to a last dense layer of said third unidirectional recurrent neural network RN3.Optionally, as illustrated by Figure 3, the last dense layer of said third unidirectional recurrent neural network RN3 also receives a second input equal to the sum of: output S12 of the first filtering unit F1 to which output S22 of the second filtering unit F2 is added, to which is further added a random white noise represented by the letter a in particular to serve as a basis for the stochastic components of the voice.Output 58 of the last dense layer of said third unidirectional recurrent neural network RN3 corresponds to the encoded voice (i.e. the synthesized voice) supplied by voice synthesizer 16.Figure 4 illustrates the principle of adversarial training optionally implemented according to the present invention.More precisely, in Figure 4, adversarial training module 60 comprises two discriminators (not represented in Figure 4) configured to be trained simultaneously with said voice synthesizer 10 during its supervised learning phase, according to a zero-sum game, said discriminators being trained to differentiate real signals 36 generated by a human speaker 34 from those 48 suitable for being synthesized by said voice synthesizer 10, said voice synthesizer 10 being trained to deceive said two discriminators.For example, the two discriminators correspond to discriminators D1 and D2 described in the article by J. Kong et al. entitled “HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis” NeurIPS 2020, but applied in another context without combined implementation of a voice synthesizer architecture according to a source-filter model, and of the zero-sum game according to the present invention where said discriminators D1 and D2 are trained to differentiate real signals from those suitable for being synthesized by said voice synthesizer 10, said voice synthesizer 10 being trained to deceive said discriminators D1 and D2.As indicated previously, advantageously one of said two discriminators is sensitive to low-frequency information and the other of said two discriminators is sensitive to high-frequency information, the use of two discriminators, sensitive to low- and high-frequency information, allows the speech synthesizer 10 to correct its defects over extended frequency ranges, and is particularly effective for removing metallic artifacts from the voice synthesized via the speech synthesizer 10.More precisely, each discriminator of the adversarial training module 60 is configured to provide a prediction PR when the incoming signal is indeed real 36 or a prediction PS when the incoming signal has been synthesized by the speech synthesizer 10 and the prediction result is subsequently used in a loss function of the speech synthesizer during training. The speech discriminators and synthesizer are trained jointly (i.e. joint training).The loss function of the speech synthesizer is the average, over the different frames of the sample, of (1- PS)2 (PS being equal to one if a discriminator “says” that the signal is real and zero otherwise).The loss function of each discriminator is the sum of the averages of:(1- PR)2 and of PS2 (PS and PR being equal to one if the discriminator predicts that the signal is real and zero otherwise).The loss function taking into account this prediction during the supervised training phase of the adversarial training module, comprising the two discriminators, and jointly of the speech synthesizer 10 is advantageously adaptive. In particular, during training, a temporal average of the spectrograms is performed, followed by a moving average over the frequencies (i.e. second axis of the spectrogram) in order to smooth the frequency distribution of the signal used for each iteration of the supervised training.Then, this smoothed spectrum x is transformed into v by means of the following activation function: v = 1,3 - sigmoid (log10(amplitude(x))) in order to calculate frequency weighting weights, in the loss function using the spectrogram, this loss function using the spectrogram corresponding to the sum, for each time / frequency value, of the amplitude of the spectrogram of the L1 norm (absolute value) of the difference between the predicted spectrogram and the reference spectrogram.Such processing allows adaptation, at each training step, to the signal actually used and therefore allows the training to focus on the relevant frequencies of the frame in question (i.e. the current packet of sound data (from the English batch) in order to have training resulting in a higher-quality model.A speech synthesis method 70 implemented via said speech synthesizer 10 will now be explained with reference to Figure 5 presenting a flowchart of the steps of this method 70.Generally, said method 70 comprises, during its inference phase 72, implemented by a trained speech synthesizer V_E having an architecture according to a neural source-filter model, a step 74 of projection P, via a first unidirectional recurrent neural network of said speech synthesizer, of input parameters, characterizing each frame of an input sound signal S_E comprising a human voice, into a latent space of configurable dimension.Said method 70 also comprises a step 76 of generation G, via a second neural network of said trained speech synthesizer V_E, of a signal representative of the harmonics of said input sound signal S_E comprising said human voice.The steps of projection 74 and generation 76 of the harmonic signal are suitable for being implemented in parallel as illustrated according to Figure 5, or, in a manner not shown, successively according to any order, provided that the output of the projection step 74 and the output of the harmonic signal generation step 76 associated with the same frame of the input sound signal S_E are processed simultaneously during a subsequent step 78 of speech synthesis SYNTH as such via a third unidirectional recurrent neural network, of an encoded voice associated with said human voice of the input sound signal S_E.Optionally, as represented in dashed lines, said inference phase 72 further comprises a step 80 of reconstruction R_T_M of the missing frames of said synthesized encoded voice from the frames preceding or following said missing frames, and / or a step 82 of band extension E_B for reconstructing a signal comprising said synthesized encoded voice with a sampling frequency higher than that of said input sound signal S_E, and / or a step 84 of BIO extraction of the vocal biomarkers of the speaker having generated said sound signal via at least one of said neural networks of said speech synthesizer.As an optional supplement, when the speech synthesizer used has not been previously trained, method 70 also comprises beforehand a training phase 88 (i.e. a training phase of each element of the speech synthesizer).According to an optional aspect of this optional supplement, said training phase 88 comprises a step 90 of adversarial training E_A via said previously described adversarial training module.Those skilled in the art will understand that the invention is not limited to the embodiments described, nor to the particular examples of the description, the embodiments and variants mentioned above being suitable for being combined with one another to generate new embodiments of the invention.The present invention thus makes it possible, as experimentally verified, by virtue of the specific structure of the speech synthesizer according to the present invention, to provide an improvement over current vocoding for equivalent input parameters.In particular, the source-filter-type neural architecture of the speech synthesizer according to the present invention makes it possible to have a “light” solution (i.e. frugal) in computational and memory footprint compared with current AI solutions, and opens up the possibility of implementing other “functionalities” such as, for example, reconstruction by Inpainting, band extension (from the English bandwidth extension) (this involves an addition of information by prediction) or else “voice conversion” (i.e. transition from one voice to another voice (a priori associated with existing speakers) or else “voice transformation) (i.e. transition from one voice to another voice (which may be, more generally, that of a virtual speaker))By delegating speech synthesis to AI approaches with recurrent neural networks at each stage of the source-filter model rather than to solutions designed by “experts” (i.e. solutions resulting from modeling), the solution according to the present invention makes it possible in particular to compensate for a potential loss of information linked to the reduction in the number of input parameters, in particular in the context of a low-bit-rate application.The choice of a “biased” source-filter architecture (i.e. an architecture of neural networks that has been “biased” or rather specialized toward speech synthesis by adding a modeling of speech by a sum of harmonics) toward speech synthesis makes it possible to reduce the digital footprint, in terms of memory and computation, of the implemented neural networks.Finally, optionally, the practical difficulty of training the neural networks in such an open task with few input parameters for a large number of possible temporal signals is advantageously mitigated by the use of adversarial training in which two discriminators are trained to recognize the signals generated by the speech synthesizer according to the present invention or jointly said speech synthesizer is trained to deceive the discriminators.Thus, the speech synthesizer and the corresponding speech synthesis method are suitable for being integrated within the framework of digital radiocommunication applications, voice over IP (from the English Internet Protocol) VoIP, audio compression, etc.  

Claims

 1. Speech synthesizer (10) having an architecture according to a neural source-filter model characterized in that it comprises at least:- a projection module (12) configured to project, via a first unidirectional recurrent neural network, input parameters, characterizing each frame of an input sound signal comprising a human voice, into a latent space of configurable dimension;- a harmonic signal generator (14) configured to generate, via a second neural network, a signal representative of the harmonics of said input sound signal comprising said human voice;- a speech synthesizer (16) configured to synthesize, via a third unidirectional recurrent neural network, an encoded voice associated with said human voice from:- the projection of said input parameters projected into said latent space of configurable dimension provided by said projection module, and- said signal representative of the harmonics of the input sound signal comprising said human voice and provided by said generator;said signal representative of the harmonics of said input sound signal comprising said human voice being a weighted sum of the signal corresponding to the pitch of said human voice and of its harmonics, said weighting being learned beforehand by said second neural network.

2. Speech synthesizer (10) according to claim 1, wherein white noise is added to said weighted sum.

3. Speech synthesizer (10) according to any one of the preceding claims, wherein said input parameters characterizing said input sound signal comprising said human voice are frequency parameters corresponding to one of the elements belonging to the group comprising at least:- a Fourier spectrum of said input sound signal comprising said human voice;- a cepstrum defined by Mel-frequency cepstral coefficients calculated by a discrete cosine transform applied to the power spectrum of said input sound signal comprising said human voice;- the constant-Q transform of said input sound signal comprising said human voice.

4. Vocal synthesizer (10) according to any one of the preceding claims, further comprising an adversarial training module (18) comprising two discriminators configured to be trained simultaneously with said vocal synthesizer during its supervised learning phase, according to a zero-sum game, said discriminators being trained to differentiate real signals from those suitable for being synthesized by said vocal synthesizer, said vocal synthesizer being trained to fool said two discriminators.

5. Vocal synthesizer (10) according to claim 4, wherein one of said two discriminators is sensitive to low-frequency information and the other of said two discriminators is sensitive to high-frequency information.

6. Vocal synthesizer (10) according to any one of the preceding claims, further comprising:- a reconstruction module (20) configured to reconstruct missing frames of said synthesized encoded voice from frames preceding or following said missing frames, and / or- a band extension module (22) configured to reconstruct a signal comprising said synthesized encoded voice with a sampling frequency higher than that of said input sound signal.

7. Vocal synthesizer (10) according to any one of the preceding claims, wherein at least one of said neural networks of said vocal synthesizer is also configured to extract the vocal biomarkers of the speaker having generated said input sound signal.

8. Method (70) for speech synthesis implemented by a vocal synthesizer having an architecture according to a neural source-filter model according to any one of the preceding claims, said method comprising the following steps:- projecting (74), via a first unidirectional recurrent neural network of said vocal synthesizer, input parameters, characterizing each frame of an input sound signal comprising a human voice, into a latent space of parameterizable dimension;- generating (76), via a second neural network of said vocal synthesizer, a signal representative of the harmonics of said input sound signal comprising said human voice;- speech synthesizing (78), via a third unidirectional recurrent neural network, an encoded voice associated with said human voice from:- the projection of said input parameters projected into said latent space of parameterizable dimension provided by said projection module, and- said signal representative of the harmonics of the input sound signal comprising said human voice and provided by said generator;said signal representative of the harmonics of said input sound signal comprising said human voice being a weighted sum of the signal corresponding to the pitch of said human voice and its harmonics, said weighting having been previously learned by said second neural network.

9. Program comprising software instructions which, when executed by said vocal synthesizer, implement a speech synthesis method according to claim 8. 

Images