Device and method for improving voice synthesis produced, by means of a predetermined codec, by an existing vocoder, and associated computer program

Abstract

Device and method for improving the voice synthesis produced, by means of a predetermined codec, by an existing vocoder, and associated computer program The present invention relates to a device for improving the voice synthesis of an existing vocoder, comprising: - a module for receiving the parameters characterizing the voice synthesis capable of being produced by said existing vocoder; - at least one voice synthesizer with artificial intelligence (10) comprising: - a projection module (12), via a first neural network, of said parameters in a latent space of parameterizable dimension; - a harmonic signal generator (14) generating, via a second neural network, a signal representative of the harmonics associated with said parameters; - a voice synthesis module (16), via a third neural network, from the projection, and said signal representative of the harmonics;said voice synthesizer being trained during a supervised learning phase. Figure: Figure 1;

Description

Title of the invention: Device and method for improving the voice synthesis produced, by means of a predetermined code, by an existing vocoder, and associated computer program

[0001] The present invention relates to a device for improving the voice synthesis produced, by means of a predetermined codec, by an existing vocoder.

[0002] The present invention also relates to a method for improving the voice synthesis produced, by means of a predetermined codec, by an existing vocoder.

[0003] The invention also relates to a computer program comprising software instructions which, when implemented by a computer, implement such a method.

[0004] The invention relates to the field of speech coding also called "vocoding", voice synthesis and voice transmission.

[0005] Subsequently, according to the present invention "vocoding" and by extension the corresponding device "vocoder" is associated with the contraction of the English words Voice Coding respectively Voice coder, in other words voice coding by an electronic device for processing the sound signal, corresponding substantially to a method according to which, from a decomposition into important properties (eg main spectral components) of an input voice or another sound, synthesizes an associated synthetic voice at the output.

[0006] Digital communications offer a significant number of benefits compared to analog communications. In addition to being of better quality, they are often more robust to interfering noise during transmission. However, one of the major problems linked to the very nature of digital communications is the limitation of the flow rate. Indeed, the higher the flow rate, the more it is possible to send a large amount of information allowing a higher quality of the voice. In addition, digital transmissions suffer from a "binarization" of the received data. Indeed, either the transmission of the digital frames is correct and the communication is unchanged, or it is incorrect (i.e. missing frames) and all the information contained in these missing frames at reception is lost.

[0007] Some transmission contexts impose constrained bit rates and the use of codecs (or coder-decoder from the English coder-decoder) whose sound quality is limited. In these contexts, the codecs used are generally standardized such as the MELP codec (from the English Mixed-excitation linearprediction) or the improved MELPe codec (from the English enhanced Mixed-excitation linearprediction) associated with the more often at a rate, for example MELP 1200 corresponding to a MELP codec with a rate of 1200 bits / s.

[0008] Codecs are generally optimized to operate according to precise modeling (for example LPC (from the English Linear Predictive Coder, MELP) for the extraction of parameters from the speech signal.

[0009] The resulting parametric model can be used to synthesize a speech signal. This is called a parametric coder or vocoder (to clarify that the model applies to the speech signal).

[0010] The quantization / dequantization step allows transmission over a constrained channel by restricting the flow rate. There are therefore two sources of degradation: the approximation of the model, and the quantization of the model parameters.

[0011] The different MELP bit rates correspond to different operating modes. It is the quantization scheme that is optimized for a given model (LPC, MELP, ...) and a target bit rate. If the bit streams are obviously different, the information is of the same type (associated with the MELP parameters).

[0012] A bitstream associated with the MELPe 1200 codec works with the MELPe 1200 vocoder and cannot work with a MELPe 2400 vocoder.

[0013] Indeed, a modification of the quantification scheme implies a loss of interoperability: a MELP 1200 is not interoperable with a MELP 2400. Furthermore, with these conventional low-speed solutions, a loss of frame transmission is accompanied by a cutoff during decoding of the communication.

[0014] The aim of the invention is then to improve the sound quality and to extend the robustness of low bit rate or even very low bit rate codecs used in certain transmission contexts which impose constrained bit rates, such as for example the aforementioned MELPe 1200 codec while remaining interoperable with existing vocoders.

[0015] To this end, the invention relates to a device for improving the voice synthesis produced, by means of a predetermined codec, by an existing vocoder, said device comprising:

[0016] - a module for receiving the parameters characterizing each frame of the synthesis vocal suitable for production by said existing vocoder;

[0017] - at least one voice synthesizer with artificial intelligence having a architecture according to a neural source-filter model, said voice synthesizer with artificial intelligence comprising at least:

[0018] - a projection module configured to project, via a first neural network monodirectional recurrent, said parameters, or parameters derived from said parameters, in a latent space of parameterizable dimension;

[0019] - a harmonic signal generator configured to generate, via a second network of neurons, a signal representative of the harmonics of said voice synthesis associated with said parameters received via said reception module;

[0020] - a voice synthesis module configured to synthesize, via a third network of unidirectional recurrent neurons, an encoded voice associated with said human voice from:

[0021] - the projection of said projected parameters into said latent space of dimension configurable provided by said projection module, and

[0022] - said signal representative of the harmonics of the voice synthesis associated with said parameters received via said receiving module and provided by said generator;

[0023] said voice synthesizer with artificial intelligence being previously trained during a supervised learning phase using training voice syntheses produced by means of said predetermined codec.

[0024] Note that by “improvement” we mean a better perception of the quality of the transmitted voice but also better robustness to transmission interruptions (i.e. frame losses).

[0025] The improvement device according to the present invention is advantageous, because it proposes to use a voice synthesizer with artificial intelligence to improve the voice synthesis of an already existing vocoder.

[0026] The voice synthesizer with artificial intelligence AI of the improvement device according to the present invention is advantageous because unlike the neural source-filter architectures of the literature, the use of monodirectional recurrent neural networks is compatible with real-time use, i.e. with a latency of less than 50ms.

[0027] Furthermore, the voice synthesizer with artificial intelligence according to the present invention is configured to provide a frugal voice synthesis solution, because it is based on an architecture that requires little data. Indeed, the source-filter model is certainly a classic model for decomposing the voice into source and filter components, however the use of this model, boosted with “modern” monodirectional recurrent neural networks (i.e. capable of restoring causality) makes it possible to offer a framework conducive to voice modeling. Such “artificial intelligence boosting” leads to better voice reconstruction with lower memory and computation costs.

[0028] The voice synthesizer with artificial intelligence AI according to the present invention has the advantage of optimally learning the exploitation of the input data, namely the parameters provided by an existing vocoder, to accomplish its task, i.e. the restitution of the voice communication. To do this, said voice synthesizer with artificial intelligence of the proposed improvement device is specialized in a mode of operation after a so-called supervised learning phase where said voice synthesizer with artificial intelligence AI is trained from a binary train of a low-bit rate codec. Such prior training then allows, during the inference phase, a significant improvement in the quality of the result originally provided by the low-bit rate codec.

[0029] More precisely, the neural network does not use the binary train directly here but the parameters of the model previously reconstructed from the binary train (dequantization step) or even other parameters derived from the parameters of the model (replacing or complementing them).

[0030] According to other advantageous aspects of the invention, the improvement device comprises one or more of the following characteristics, taken individually or in all technically possible combinations:

[0031] - the reception module comprises a tool for identifying said predetermined code by analyzing the voice synthesis produced by said existing vocoder, said identification tool being further configured to select, from a plurality of voice synthesizers with artificial intelligence previously trained and associated respectively with distinct codecs, said voice synthesizer with artificial intelligence corresponding to said identified predetermined codec;

[0032] - the receiving module comprises an interpolation tool configured to interpolate linearly said parameters and provide as input to said voice synthesizer with artificial intelligence interpolated parameters compatible with said voice synthesis module;

[0033] - said signal generated by the generator and representative of the harmonics of said voice synthesis associated with said parameters received via said reception module is a weighted sum of the signal corresponding to the pitch associated with said parameters received and its harmonics, said weighting being learned beforehand by said second neural network;

[0034] - white noise is added to said weighted sum;

[0035] - the voice synthesizer with artificial intelligence further comprises a module adversarial training comprising two discriminators configured to be trained simultaneously with said voice synthesizer with artificial intelligence during its supervised learning phase, according to a zero-sum game, said discriminators being trained to differentiate real signals from those suitable for being synthesized by said voice synthesizer with artificial intelligence, said voice synthesizer with artificial intelligence being trained to fool said two discriminators;

[0036] - one of said two discriminators is sensitive to low frequency information and the other of said two discriminators is sensitive to high frequency information;

[0037] - the voice synthesizer with artificial intelligence further comprises:

[0038] - a reconstruction module configured to reconstruct missing frames of said encoded voice synthesized from frames before or after said missing frames, and / or

[0039] - a band extension module configured to reconstruct a signal comprising said encoded voice synthesized with a sampling frequency higher than that of said voice synthesis associated with said parameters received via said reception module.

[0040] The invention also relates to a method for improving the voice synthesis produced, by means of a predetermined codec, by an existing vocoder, said method being implemented by an improvement device as previously described, said method comprising:

[0041] - reception of the parameters characterizing each frame of the voice synthesis suitable for production by said existing vocoder;

[0042] and the following steps implemented during an inference phase by a voice synthesizer with artificial intelligence of said improvement device, said voice synthesizer with artificial intelligence being previously trained during a supervised learning phase using training parameters characterizing each frame of voice syntheses capable of being produced by means of said predetermined codec:

[0043] - projection, via a first monodirectional recurrent neural network of said voice synthesizer with artificial intelligence, parameters received via said reception module or parameters derived from said parameters, in a latent space of configurable dimension;

[0044] - generation, via a second neural network of said voice synthesizer with artificial intelligence, of a signal representative of the harmonics of the voice synthesis associated with said parameters received via said reception module;

[0045] - voice synthesis, via a third monodirectional recurrent neural network, of an encoded voice associated with said human voice from:

[0046] - the projection of said projected parameters into said latent space of dimension configurable provided by said projection module, and

[0047] - said signal representative of the harmonics of the voice synthesis associated with said parameters received via said receiving module and provided by the generator of the voice synthesizer with artificial intelligence of said improvement device.

[0048] The invention also relates to a computer program comprising software instructions which, when executed by a computer, implement a voice synthesis method, as defined above.

[0049] The invention will appear more clearly on reading the description which follows, given solely by way of non-limiting example, and made with reference to the drawings in which:

[0050] [Fig-1] [Fig.l] is a general schematic representation of a device of improving the voice synthesis produced, by means of a predetermined codec, by an existing vocoder according to the present invention;

[0051] [Fig.2] [Fig.2] illustrates the use of the improvement device of [Fig.l];

[0052] [Fig.3] [Fig.3] illustrates an example of the architecture of the device for improving the [Fig.l];

[0053] [Fig.4] [Fig.4] illustrates the implementation of an optional module of the improvement device of [Fig.l] during its training phase;

[0054] [Fig.5] [Fig.5] is a flowchart of a method for improving the voice synthesis produced, by means of a predetermined codec, by an existing vocoder implemented by the improvement device of [Fig.l] according to one embodiment.

[0055] In the remainder of the description, the expression “substantially equal to” is understood as a relationship of equality to plus or minus 10%, that is to say with a variation of at most 10%, more preferably as a relationship of equality to plus or minus 5%, that is to say with a variation of at most 5%.

[0056] Furthermore, subsequently, we consider that a neural network comprises an ordered succession of layers of neurons, each of which takes its inputs from the outputs of the previous layer.

[0057] More precisely, each layer comprises neurons taking their inputs from the outputs of the neurons of the previous layer, or from the input variables for the first layer.

[0058] Alternatively, more complex neural network structures can be envisaged with a layer that can be connected to a layer further away than the immediately preceding layer.

[0059] Each neuron is also associated with an operation, i.e. a type of processing, to be carried out by said neuron within the corresponding processing layer.

[0060] Each layer is connected to the other layers by a plurality of synapses. A synaptic weight is associated with each synapse, and each synapse forms a connection between two neurons. It is often a real number, which takes both positive and negative values. In some cases, the synaptic weight is a complex number.

[0061] Each neuron is capable of performing a weighted sum of the value(s) received from the neurons of the previous layer, each value then being multiplied by the respective synaptic weight of each synapse, or link, between said neuron and the neurons of the previous layer, then applying an activation function, typically a non-linear function, to said weighted sum, and to deliver at the output of said neuron, in particular to the neurons of the following layer which are connected to it, the value resulting from the application of the activation function. The activation function makes it possible to introduce a non-linearity in the processing carried out by each neuron. The sigmoid function, the hyperbolic tangent function, the Heaviside function are examples of activation functions.

[0062] As an optional addition, each neuron is also capable of applying, in addition, a multiplicative factor, also called bias, to the output of the activation function, and the value delivered at the output of said neuron is then the product of the bias value and the value from the activation function.

[0063] A fully connected layer of neurons is a layer in which the neurons of said layer are each connected to all the neurons of the previous layer.

[0064] Such a type of layer is more often referred to as “fully connected” and sometimes referred to as “dense layer”.

[0065] The electronic device for improving the voice synthesis produced, by means of a predetermined code, by an existing vocoder is illustrated in [Fig.l].

[0066] Such an existing vocoder conventionally produces a parametric model of the speech signal with, on the one hand, an analysis module implementing the extraction of the parameters and, on the other hand, a synthesis module implementing the reconstruction of the signal from the extracted parameters.

[0067] By coded, we mean the combination of a coder and a decoder (i.e. coder + decoder), the coder comprising for its part an analysis module and a quantification module of the parameters resulting from the analysis, the decoder being capable of carrying out the dequantization (i.e. the restitution of the parameters from the binary train) then the voice synthesis as such.

[0068] The electronic device for improving the voice synthesis is located at the output of the dequantization module of the existing vocoder (i.e. conventional such as a MELP 1200 vocoder), i.e. after the dequantization module of the codec decoder conventionally configured to dequantize (i.e. restore the parameters from the binary train).

[0069] In other words, the electronic device for improving voice synthesis according to the present invention uses as input the parameters Sv resulting from the dequantization of the existing vocoder and / or the parameters derived from the latter, and replaces the pre-existing synthesis module of the existing vocoder. Thus, the synthesis model of the existing vocoder is no longer used but replaced by the voice synthesizer 10 described below which uses as input the parameters resulting from the dequantization of the existing vocoder 200 and / or the parameters derived from the latter.

[0070] Such an electronic device for improving voice synthesis firstly comprises a module for receiving the parameters characterizing each frame of the voice synthesis capable of being produced by said existing vocoder.

[0071] An example of the architecture of this reception module 110 is notably described below in relation to [Fig.3].

[0072] Optionally, the reception module 110 comprises a tool for identifying said predetermined codec, the identification tool being configured to determine said predetermined codec by analyzing the parameters characterizing each frame of the voice synthesis capable of being produced by said existing vocoder.

[0073] In addition, said optional identification tool is further configured to select, from a plurality, not shown, of voice synthesizers with artificial intelligence previously trained and associated respectively with distinct codecs, said voice synthesizer with artificial intelligence 10 corresponding to said identified predetermined codec.

[0074] More precisely, the reception module 110 is configured at a given instant to operate with a speech coder (there is a negotiation between the transmitter and the receiver to use a common coder). In reception, the information is therefore available and the voice synthesizer with artificial intelligence previously trained and adapted can be selected or configured (for example same network topology but different weights for a different quantization scheme but an identical parametric model) from this information.

[0075] In other words, such an identification tool makes it possible to adapt the voice synthesizer with artificial intelligence to be used subsequently, according to the received bitstream that it used to identify the codec used by the existing vocoder whose voice synthesis is to be improved according to the present invention. Thus, the identification tool is capable of implementing a phase of identifying the codec used by the existing vocoder to then switch to the voice synthesizer with artificial intelligence 10 adapted to the received parameters Sv, in particular when the improvement device previously comprises a plurality of voice synthesizers with artificial intelligence previously trained and associated respectively with distinct codecs.

[0076] Thus, said selected voice synthesizer with artificial intelligence 10 has been previously trained during a supervised learning phase using training voice syntheses produced by means of said identified predetermined codec. In other words, supervised training adapted to the use case is implemented by exploiting the parameters encoded by the codec of the existing vocoder whose voice synthesis is to be improved.

[0077] According to another variant, the reception module 110 does not include an identification tool, and the voice synthesizer with artificial intelligence 10 is directly adapted to the existing vocoder, namely that it has been previously trained with training voice syntheses produced by means of the predetermined codec of the existing vocoder, for example by means of a MELPe 1200 binary train making it possible to guarantee the condition of interoperability with an existing MELPe 1200 vocoder. Once trained, the voice synthesizer with artificial intelligence 10 therefore receives, via its reception module 110, as input the current decoded parameters supplied by the decoder of the existing MELPe 1200 vocoder.

[0078] As can be seen in this [Fig.l], the voice synthesizer with artificial intelligence 10 comprises a projection module 12 configured to project, via a first monodirectional recurrent neural network RNb, parameters characterizing each frame of said voice synthesis suitable for being produced by the existing vocoder, received via said reception module, into a latent space (from the English embedding) of configurable dimension.

[0079] The input parameters of the monodirectional recurrent network RNi can be the parameters of the coder used in the existing vocoder or can be replaced or supplemented by parameters obtained by transformation of the initial parameters.

[0080] For example, for the MELP, the parameters characterizing the spectral envelope of the speech signal are the LSF parameters (Line Spectral Frequencies) directly associated with the LPC model. These parameters can be used as input to the monodirectional recurrent network RNi or replaced by other parameters such as: the cepstral coefficients (LPCC) or log-spectral coefficients obtained by DCT (Discrete Cosine Transform) of the cepstral coefficients. These transformations can make it possible to exploit orthogonalization properties or better resolution by increasing the number of parameters (for example, ten LSF coefficients, eighteen LPCC coefficients, sixty-four spectral coefficients, etc.).

[0081] The LPCC coefficients are obtained by transforming the linear prediction coefficients (LPC model). The transition relationships between the LSF coefficients, the linear prediction coefficients and the LPCC coefficients are established and known in the field of speech processing.

[0082] Such a projection module 12 is also called an encoder of the voice synthesizer with artificial intelligence 10, and is therefore capable of “immersing” the parameters characterizing each frame of the voice synthesis capable of being produced by said existing vocoder and received via said reception module 110 in a latent space (i.e. a vector space) of configurable dimension, for example of size equal to 128 (i.e. capable of comprising 128 components). The RNi neural network, previously trained as will be seen later, supports the non-linear projection of said parameters in the latent space potentially containing more information than the parameters since temporal relationships can be incorporated into it thanks to the recurrent and unidirectional nature of the RNb neural network

[0083] The parameters specific to being supported by said projection module 12 are parameters characterizing each frame of the voice synthesis specific to being produced by said existing vocoder and received via said reception module 110. The nature of these parameters is imposed by the existing vocoder.

[0084] As illustrated by [Fig.l], the voice synthesizer with artificial intelligence 10 further comprises a harmonic signal generator 14 configured to generate, via a second neural network RN2, a signal representative of the harmonics of said voice synthesis associated with said parameters received via said reception module 110.

[0085] As an optional addition, such a harmonic signal generator 14 is configured to generate said signal representative of the harmonics of said voice synthesis associated with said parameters received via said reception module 110 which is a weighted sum of the signal corresponding to the pitch associated with said voice synthesis itself associated with said parameters received and its harmonics, said weighting being learned beforehand by said second neural network.

[0086] According to a variant of this optional complement, white noise is added to said weighted sum.

[0087] In other words, said harmonic signal generator 14 is configured to produce a signal composed of a sinusoidal signal at the pitch associated with said voice synthesis associated with said received parameters and its harmonics. The output signal of this generator 14 is a weighted sum of the pitch associated with said voice synthesis associated with said received parameters and its harmonics. The weighting is learned by the second neural network RN2 during a prior training phase. White noise is added to this harmonic signal to serve as a basis for the stochastic components of the voice. The unvoiced frames or frames not containing voices are composed only of white noise without a harmonic component.

[0088] According to a voice synthesizer architecture according to the known “source-filter” model, the output generated by the harmonic signal generator 14 plays the role of the “source” part of this “source-filter” architecture.

[0089] As illustrated by [Fig.l], the voice synthesizer with artificial intelligence 10 further comprises a voice synthesis module 16 configured to synthesize, via a third monodirectional recurrent neural network RN3, an encoded voice (associated with the human voice of the speaker having produced a sound signal at the input of the existing vocoder) from:

[0090] - the projection of said projected parameters into said latent space of dimension configurable provided by said projection module 12, and

[0091] - said signal representative of the harmonics of said associated voice synthesis to said parameters received via said reception module 110 and supplied by said generator 14.

[0092] Such a voice synthesis module 16 is also called a decoder of the voice synthesizer with artificial intelligence 10, and like the previous elements 12 and 14 of the voice synthesizer with artificial intelligence 10 is also based on a recurrent neural network architecture. Such a voice synthesis module 16 therefore has as inputs the output of the projection model 12 and the output of the harmonic signal generator 14. The projection model 12 and the harmonic signal generator 14 are capable of operating in parallel or successively in any order, provided that the output of the projection model 12 and the output of the harmonic signal generator 14 associated with the same frame of said voice synthesis associated with said parameters received via said reception module 110 are processed simultaneously by said voice synthesis module 16.

[0093] According to a voice synthesizer architecture with artificial intelligence 10 according to the known “source-filter” model, the voice synthesis module 16 plays the role of the “filter” part of this “source-filter” architecture.

[0094] Thus, unlike the neural source-filter architectures according to the current state of the art, the use of monodirectional recurrent neural networks proposed according to the present invention makes it possible to operate in real time.

[0095] For example, the voice synthesizer with artificial intelligence 10 according to the present invention is capable of operating with said voice synthesis associated with said parameters received via said reception module 110 previously broken down into 40ms frames with a look ahead buffer zone of 40ms, essentially due to the extraction of the parameters at the input of the projection module 12 (i.e. the encoder).

[0096] As an optional addition, as shown in dotted lines, said voice synthesizer with artificial intelligence 10 further comprises an adversarial training module 18 comprising two discriminators Di and D2 configured to be trained simultaneously with said voice synthesizer with artificial intelligence 10 during its supervised learning phase, according to a zero-sum game, said discriminators Di and D2 being trained to differentiate the real signals from those suitable for being synthesized by said voice synthesizer with artificial intelligence 10, said voice synthesizer with artificial intelligence 10 being trained to fool said two discriminators Di and D2.

[0097] It should be noted that once the learning phase has been carried out, such an adversarial training module 18 is no longer used during the inference phase using the voice synthesizer with artificial intelligence 10 thus previously trained in an adversarial manner.

[0098] According to an advantageous variant of this optional complement, one of said two discriminators is sensitive to low frequency information and the other of said two discriminators is sensitive to high frequency information.

[0099] In other words, the two discriminators Di and D2 are trained at the same time as the speech synthesizer with artificial intelligence 10 to differentiate the real signals from the signals synthesized by the speech synthesizer with artificial intelligence 10 by creating a zero-sum game where the discriminators Di and D2 try to differentiate the signals created by the speech synthesizer with artificial intelligence 10 from the original signals and where the speech synthesizer with artificial intelligence 10 is trained to fool the two discriminators Di and D2.

[0100] Such a zero-sum game allows balancing the voice synthesizer with artificial intelligence 10 resulting from said adversarial training, and the use of two discriminators, sensitive to low and high frequency information, allows the voice synthesizer with artificial intelligence 10 to correct its defects over extended frequency ranges, and is particularly effective in removing metallic artifacts from the voice synthesized via the voice synthesizer with artificial intelligence 10.

[0101] For example, the two discriminators correspond to the discriminators Di and D2 described in the article by J. Kong et al entitled “HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis” NeurIPS 2020, but applied in another context, without combined implementation of a speech synthesizer architecture according to a source-filter model and the zero-sum game according to the present invention where said discriminators Di and D2 are trained to differentiate real signals from those suitable for being synthesized by said speech synthesizer with artificial intelligence 10, said speech synthesizer with artificial intelligence 10 being trained to fool said discriminators Di and D2.

[0102] The adversary training implemented via said optional adversary training module 18 is further described below in relation to [Fig.4].

[0103] Such an optional adversary training module 18 therefore makes it possible to improve the quality of synthesis.

[0104] As an optional addition, said voice synthesizer with artificial intelligence 10 further comprises a reconstruction module 20 configured to reconstruct (inpainting) missing frames of said encoded voice synthesized from frames before or after said missing frames, and / or a band extension module 22 configured to reconstruct a signal comprising said synthesized encoded voice with a sampling frequency higher than that of said voice synthesis associated with said parameters received via said reception module 110.

[0105] The optional reconstruction module 20 makes it possible to improve the overall transmission of the synthesized signal via said voice synthesizer with artificial intelligence 10 by compensating for micro-cuts which would occur without its implementation due to missing frame(s), while the optional band extension module 22 makes it possible to increase the output sampling frequency, for example to predict the missing information (between 4 and 8 kHz for example for a change from a sampling frequency of 8 to 16 kHz).

[0106] As an optional addition, at least one of said neural networks of said voice synthesizer with artificial intelligence 10, for example the first monodirectional recurrent neural network RNi of the projection module 12 or the second neural network RN2, is also configured to extract the vocal biomarkers of the original speaker having generated said voice synthesis associated with said parameters received via said reception module 110, after having been previously trained for this purpose in a supervised manner.

[0107] In a variant not shown, the voice synthesizer with artificial intelligence 10 is also capable of optionally comprising another neural network dedicated to the extraction of vocal biomarkers from the speaker having generated said voice synthesis associated with said parameters received via said reception module 110 after having been previously trained for this purpose in a supervised manner.

[0108] In the example of [Fig.l], the voice synthesizer with artificial intelligence 10 comprises an information processing unit 26 formed for example of a memory 28 and a processor 30 associated with the memory 28.

[0109] In the example of [Fig.l], the projection module 12, the generator 14 and the voice synthesis module 16, as well as optionally the adversary training module 18, the reconstitution module 20 and the band extension module 22 are each produced in the form of software, or a software brick, executable by the processor 30. The memory 28 of the voice synthesizer with artificial intelligence 10 is then capable of storing projection software, harmonic signal generation software, and voice synthesis software, as well as optionally adversary training software, reconstitution software, and band extension software. The processor is then capable of executing each of the software among the projection software, the harmonic signal generation software, and the voice synthesis software, as well as optionally the adversary training software, reconstruction software, band extension software.

[0110] In a variant not shown, the projection module 12, the generator 14 and the voice synthesis module 16, as well as, as an optional addition, the adversary training module 18, the reconstitution module 20 and the band extension module 22 are each produced in the form of a programmable logic component, such as an FPGA (Field Programmable Gate Array), or an integrated circuit, such as an ASIC (Application Specific Integrated Circuit).

[0111] When the voice synthesizer with artificial intelligence 10 is produced in the form of one or more software programs, that is to say in the form of a computer program, also called a computer program product, it is furthermore capable of being recorded on a medium, not shown, readable by a computer. The computer-readable medium is for example a medium capable of storing electronic instructions and of being coupled to a bus of a computer system. For example, the readable medium is an optical disk, a magneto-optical disk, a ROM memory, a RAM memory, any type of non-volatile memory (for example FLASH or NVRAM) or a magnetic card. A computer program comprising software instructions is then stored on the readable medium.

[0112] [Fig. 2] illustrates the use 32 of the voice synthesizer with artificial intelligence 10 of [Fig. 1]. According to this use, a speaker 34 first produces a sound signal 36 comprising his own human voice. Such a sound signal 36 is then transmitted as input to the existing vocoder.

[0113] The existing vocoder is configured to extract from the sound signal 36 the parameters conforming to the codec that it uses, implement a quantization and a dequantization providing the decoded parameters 39 as input to the voice synthesis improvement device.

[0114] The nature of these parameters 39 is imposed by the existing vocoder. These parameters 39 are provided as input to the projection module 40 (corresponding to the projection module 12 of [Fig.2]) which comprises the first monodirectional recurrent neural network RNi configured for said parameters in a latent space of parameterizable dimension.

[0115] In parallel, the decoded parameters 39 received via said reception module 110 are provided as input to the harmonic signal generator 42 (corresponding to the generator 14 of [Fig.l]), said generator 42 providing as output a signal 44 representative of the harmonics of said voice synthesis associated with said parameters received via said reception module 110.

[0116] Said signal 44 representative of the harmonics of said voice synthesis associated with said parameters received via said reception module 110 as well as the result provided by the projection module 40 are associated with the same frame of said voice synthesis associated with said parameters received via said reception module 110 and supplied simultaneously as input to the voice synthesizer 46 configured to synthesize, via a third monodirectional recurrent neural network RN3, an encoded voice 48 associated with the decoded parameters 39 received via said reception module 110 of the voice synthesis improvement device.

[0117] [Fig. 3] illustrates an example of architecture of the speech synthesis improvement device 100. According to this example, the speech synthesis improvement device 100 receives as input parameters Sv characterizing each frame of the speech synthesis suitable for being produced by an existing vocoder, for which its own speech synthesizer with artificial intelligence 10 has been previously trained.

[0118] For example, the existing vocoder is of type MELPe 1200 (or of type MELPe 2400, etc.) and provides a MELPe 1200 (respectively MELPe 2400) bitstream. The voice synthesizer with artificial intelligence 10 has been previously trained with acoustic parameters derived from this type of MELPe 1200 (respectively MELPe 2400) bitstream in order to guarantee the condition of interoperability with the existing vocoder 200 of type MELPe 1200 (respectively MELPe 2400). The MELP 2400 uses a quantization scheme on a 54-bit frame and duration 22.5 ms. The MELP 1200 uses a quantization scheme on a “super-frame” composed of three consecutive frames = 3 x 22.5 ms = 67.5 ms. The quantization scheme allows to exploit the information redundancy and to reduce the bit rate to 81 bits / superframe.

[0119] The reception module 110 therefore receives as input parameters Sv comprising, according to one example, the decoded parameters of the MELPe 1200.

[0120] As an optional addition, the reception module 110 comprises an interpolation tool configured to linearly interpolate the received parameters and provide as input to said voice synthesizer with artificial intelligence 10 interpolated parameters compatible with said voice synthesis module 16.

[0121] In other words, if the existing vocoder uses a frame duration that is unsuitable for the voice synthesizer with artificial intelligence 10 of the improvement device 100, the parameters from the existing vocoder are interpolated to be synchronous with the voice synthesis module 16 of the AI ​​voice synthesizer 10. Optionally, a change of parameter space is also applied.

[0122] It is thus possible to temporally interpolate the input parameters of the voice synthesizer with artificial intelligence 10 to adapt the operating rate of the network (often to increase the rate of synthesis: for example from 22.5 ms to 10 ms). The interpolation can be carried out linearly (for example at the level of the LSF coefficients which have good properties interpolation, and which can be used via successive transformations from LSF to LPCC, then from LPCC to spectral coefficients, the transition relationships between the LSF coefficients, the linear prediction coefficients and the LPCC coefficients being established and known in the field of speech processing.

[0123] Thus, optionally interpolated, the decoded parameters of the existing vocoder are extracted, namely for example the variables corresponding in particular to the linear prediction cepstral coefficients LPCC (from the English linearprediction cepstral coefficients), to the dynamic linear prediction cepstral coefficients DLPCC (from the English dynamic linearprediction cepstral coefficients), to the gain and to the voicing.

[0124] According to the example architecture of [Fig.3], these parameters are preprocessed upstream of the voice synthesizer with artificial intelligence 10.

[0125] More precisely, the variables corresponding to the linear prediction cepstral coefficients LPCC (from the English linearprediction cepstral coefficients), and to the dynamic linear prediction cepstral coefficients DLPCC (from the English dynamic linear prediction cepstral coefficients) are processed by a normalization tool configured to center and normalize them (i.e. normalize them at the scale of the data batch) at the input of the voice synthesizer with artificial intelligence 10. Note that the dynamic linear prediction cepstral coefficients DLPCC are obtained by derivation of the LPCC coefficients generally using the first and second order derivatives.

[0126] The gain is projected via a projection tool into a space of dimension greater than its input dimension, for example of dimension equal to eight. For example, the projection tool comprises a simple perceptron for this purpose.

[0127] The voicing is also projected via a projection tool into a space of a dimension greater than its input dimension. For example, the projection tool includes a simple perceptron for this purpose.

[0128] The reception module 110, as illustrated by the example of [Fig.3] is configured to provide the output 54 which comprises the outputs of the normalization and projection tools as input to the projection module 12 comprising the first monodirectional recurrent neural network RNp

[0129] In addition, the reception module 110 also provides an output, comprising the parameters Sv characterizing each frame of the voice synthesis capable of being produced by an existing vocoder, at the input of the harmonic signal generator 14.

[0130] As illustrated by the architectural example, the projection module 12 comprises the first monodirectional recurrent neural network RNi configured to project the parameters of the output 54 grouping the elements provided by the aforementioned tools into a latent space of parameterizable dimension, for example of dimension 128 (i.e. capable of comprising 128 components).

[0131] As illustrated by the example of [Fig.3], the first monodirectional recurrent neural network RNi successively comprises a normalization layer LN (from the English Layer-norm), followed by three dense layers, a recurrent unit with gate GRU (from the English Gate Recurrent Unit) and a dense layer, so as to provide as output the projection Z of said parameters in the latent space of parameterizable dimension.

[0132] As an alternative to the GRU unit, a layer with long short term memory such as LSTM (Long Short Term Memory) or even more conventionally a recurrent RNN (Recurrent Neural Network) layer is used.

[0133] Furthermore, as illustrated by the example architecture, the harmonic signal generator 14 optionally firstly comprises a module 56 capable of extracting the pitch associated with said parameters Sv.

[0134] The harmonic signal generator 14 further comprises a second neural network RN2 comprising in particular a Sin Gen layer, configured to generate a sinusoidal signal at the pitch associated with said received parameters, followed by a dense layer such that the signal produced by the generator 14 is a signal composed of a sinusoidal signal at the pitch associated with said received parameters and its harmonics, this signal corresponding to a weighted sum E of the pitch and its harmonics. The weighting is learned by the second neural network RN2 during a prior training phase. The output generated E by the harmonic signal generator 14 plays the role of the “source” part of this “source-filter” architecture.

[0135] The projection model 12 and the harmonic signal generator 14 are capable of operating in parallel or successively in any order, provided that the output Z of the projection model 12 and the output E of the harmonic signal generator 14 are provided as input and processed simultaneously by said voice synthesis module 16.

[0136] From said inputs Z and E, the voice synthesis module 16, via its third monodirectional recurrent neural network RN3, is configured to synthesize an encoded voice associated with said parameters Sv.

[0137] More precisely, the voice synthesis module 16 plays the role of the “filter” part of this “source-filter” architecture of the synthesizer according to the present invention. According to the example of [Fig.3], the third monodirectional recurrent neural network RN3 of the voice synthesis module 16 successively comprises a first dense layer corresponding in particular to a conventional perceptron and having only the input E as input.

[0138] This first dense layer of the third monodirectional recurrent neural network RN3 is followed by two filtering units Fi and F2 (from the English Filter Unit) successive. The first filtering unit Fi receives two inputs namely both the output Z of the projection model 12 and the output of the first dense layer, while the second filtering unit F2, placed after the first filtering unit Fi, also receives two inputs namely both the output Z of the projection model 12 and the output of the first filtering unit Fh

[0139] These two filtering units Fi and F2 have an identical structure as illustrated at zoom 57 inside said filtering unit structure.

[0140] More precisely, each filtering unit Fi or F2 firstly comprises a concatenation layer capable of concatenating the two received inputs represented on the one hand by the letter “y” (corresponding to the output S of the first dense layer of the third monodirectional recurrent neural network RN3 for the first filtering unit Fi or to the output Su of the first filtering unit Fi for the second filtering unit F2), and on the other hand by the letter “Z” corresponding to the output Z of the projection model 12.

[0141] At the output of said concatenation layer, each filtering unit Fi or F2 comprises a recurrent gate unit GRU (from the English Gate Recurrent Unit).

[0142] As an alternative to the GRU unit, a layer with long short term memory such as LSTM (Long Short Term Memory) or even more conventionally a recurrent RNN (Recurrent Neural Network) layer is used.

[0143] The GRU unit of each filtering unit Fi or F2 provides two outputs noted in the zoom 57 Si and S2 as input each of a dense layer, the input represented by the letter "y", as specified previously, being added as illustrated within the zoom 57 of [Fig.3] to the output Si of one of these two dense layers.

[0144] For the first filtering unit FB the outputs Si and S2 indicated in zoom 57 correspond respectively to the outputs Si i and Si 2, while for the second filtering unit F2, the outputs Si and S2 indicated in zoom 57 correspond respectively to the outputs S2i and S2 2.

[0145] Finally, as illustrated by the example of [Fig.3], the output S2i of the second filtering unit F2 is provided as input to a last dense layer of said third monodirectional recurrent neural network RN3.

[0146] Optionally, as illustrated by [Fig.3], the last dense layer of said third monodirectional recurrent neural network RN3 also receives a second input equal to the sum of: the output Si 2 of the first filtering unit Fi to which is added the output S2 2 of the second filtering unit F2, to which is further added a random white noise represented by the letter a in particular to serve as a basis for the stochastic components of the voice.

[0147] The output 58 of the last dense layer of said third monodirectional recurrent neural network RN3 corresponds to the encoded channel (i.e. the synthesized voice) provided by the voice synthesis module 16.

[0148] [Fig.4] illustrates the principle of adverse training implemented optionally according to the present invention.

[0149] More precisely, in [Fig. 4] the adversarial training module 60 comprises two discriminators (not shown in [Fig. 4]) configured to be trained simultaneously with said voice synthesizer with artificial intelligence 10 during its supervised learning phase, according to a zero-sum game, said discriminators being trained to differentiate the voice syntheses provided by an existing vocoder from those 48 suitable for being synthesized by said voice synthesizer with artificial intelligence 10 of the improvement device 100, said voice synthesizer with artificial intelligence 10 being trained to fool said two discriminators.

[0150] For example, the two discriminators correspond to the discriminators Di and D2 described in the article by J. Kong et al entitled “HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis” NeurIPS 2020, but applied in another context without combined implementation of a speech synthesizer architecture according to a source-filter model and the zero-sum game according to the present invention where said discriminators Di and D2 are trained to differentiate real signals from those suitable for being synthesized by said speech synthesizer with artificial intelligence 10, said speech synthesizer with artificial intelligence 10 being trained to fool said discriminators Di and D2.

[0151] As indicated previously, advantageously one of said two discriminators is sensitive to low frequency information and the other of said two discriminators is sensitive to high frequency information, the use of two discriminators, sensitive to low and high frequency information, allows the voice synthesizer with artificial intelligence 10 to correct its defects over extended frequency ranges, and is particularly effective for removing metallic artifacts from the voice synthesized via the voice synthesizer with artificial intelligence 10.

[0152] More precisely, each discriminator of the adversarial training module 60 is configured to provide a PR prediction when the incoming signal is real 36 or a Ps prediction when the incoming signal has been synthesized by the voice synthesizer with artificial intelligence 10 and the prediction result is then used in a loss function of the synthesizer during training. The discriminators and synthesizers are trained jointly (i.e. joint learning).

[0153] The loss function of the voice synthesizer is the average, over the different frames of the sample, of the (1- Ps)2 (Ps being equal to one if a discriminator “says” that the signal is real and zero otherwise).

[0154] The loss function of each discriminator is the sum of the averages of:

[0155] (1- PR)2 and Ps2 (Ps and PR being equal to one if the discriminator predicts that the signal is real and zero otherwise).

[0156] The loss function taking into account this prediction during the supervised learning phase of the adversarial training module, comprising the two discriminators, and jointly of the voice synthesizer with artificial intelligence 10 is advantageously adaptive. In particular, during learning, a time average of the spectrograms is carried out, followed by a sliding average on the frequencies (i.e. second axis of the spectrogram) in order to smooth the frequency distribution of the signal used for each iteration of the supervised learning.

[0157] Then, this smoothed spectrum x is transformed into v by means of the following activation function: v = 1.3 - sigmoid (logl0(amplitude(x))) in order to calculate, weighting weights by frequency, in the loss function using the spectrogram, this loss function using the spectrogram corresponding to the sum for each time / frequency value of the amplitude of the spectrogram of the L1 norm (absolute value) of the difference between the predicted spectrogram and the reference spectrogram.

[0158] Such processing allows an adaptation, at each stage of the learning, to the signal actually used and therefore allows the learning to focus on the relevant frequencies of the frame in question (i.e. the current packet of sound data (from the English batch) in order to have a learning resulting in a model of better quality.

[0159] A voice synthesis method 70 implemented via said voice synthesizer with artificial intelligence 10 will now be explained with reference to [Fig.5] presenting a flowchart of the steps of this method 70.

[0160] Generally speaking, said method 70 comprises, during its inference phase 72, a first step of receiving the parameters Sv produced by an existing vocoder.

[0161] Optionally, this reception step comprises a sub-step of identifying the predetermined codec used by the existing vocoder by analyzing the parameters characterizing each frame of the voice synthesis capable of being produced by said existing vocoder, said identification sub-step further comprising the selection, from among a plurality of voice synthesizers with artificial intelligence previously trained and associated respectively with distinct codecs, of the voice synthesizer with artificial intelligence corresponding to said identified predetermined codec.

[0162] Said method 70 further comprises a plurality of following steps implemented by a trained artificial intelligence voice synthesizer V_E of the improvement device 100 having an architecture according to a neural source-filter model.

[0163] Said plurality of following steps comprises a step 74 of projection P, via a first monodirectional recurrent neural network of said voice synthesizer, of the parameters, Sv, into a latent space of parameterizable dimension.

[0164] Said method 70 also comprises a step 76 of generation G, via a second neural network of said trained voice synthesizer V_E, of a signal representative of the harmonics of the voice synthesis associated with said received parameters Sv.

[0165] The steps of projection 74 and generation 76 of the harmonic signal are suitable for being implemented in parallel as illustrated according to [Fig.5], or in a manner not shown, successively in any order, provided that the output of the projection step 74 and the output of the step 76 of generation of the harmonic signal associated with the same frame of the voice synthesis associated with said received parameters S v are processed simultaneously during a subsequent step 78 of voice synthesis SYNTH as such via a third monodirectional recurrent neural network, of an encoded voice associated with said frame of the voice synthesis associated with said received parameters Sv.

[0166] Optionally, as shown in dotted lines, said inference phase 72 further comprises a step 80 of reconstitution R_T_M of the missing frames of said synthesized encoded voice from frames prior or subsequent to said missing frames, and / or a step 82 of band extension E_B to reconstitute a signal comprising said synthesized encoded voice with a sampling frequency higher than that of the voice synthesis associated with said received parameters Sv, and / or a step 84 of extraction BIO of the vocal biomarkers of the speaker having generated said input sound signal of the existing vocoder having provided said parameters Sv via at least one of said neural networks of said voice synthesizer.

[0167] As an optional addition, when the voice synthesizer with artificial intelligence (i.e. AI voice synthesizer) used has not been previously trained, the method 70 also comprises beforehand a learning phase 88 (i.e. a training phase of each element of the AI ​​voice synthesizer).

[0168] More precisely, the learning phase 88 consists of reconstructing the original communication (i.e. without alteration due to the encoding of the parameters) from the encoded (and therefore degraded) parameters. This learning mode can be enriched and allow new use cases.

[0169] A new use case is obtained for example by requesting (i.e. by training to) reconstruct, via the aforementioned optional reconstruction module 20, the frames lost during transmission from a context (i.e. the frames before and after the lost frames), which makes it possible to reconstruct part of the lost communication.

[0170] Another new use case is obtained by requesting (i.e. training) to reconstruct an audio signal with a higher sampling than the original codec via the aforementioned optional band extension module 22, which makes it possible to improve the perception of voice quality and robustness to low-frequency noise. For example, the MELPe 1200 is designed to reconstruct audio signals sampled at 8kHz and the AI ​​speech synthesizer of the speech synthesis enhancement device 100 is trained to reconstruct signals sampled at 16kHz.

[0171] According to an optional aspect of this optional supplement, said learning phase 88 comprises a step 90 of adversary training E_A via said adversary training module previously described.

[0172] Those skilled in the art will understand that the invention is not limited to the embodiments described, nor to the particular examples of the description, the embodiments and variants mentioned above being suitable for being combined with each other to generate new embodiments of the invention.

[0173] The present invention thus makes it possible to “replace” the voice synthesis part of an existing vocoder, and to improve the restitution of the voice communication corresponding to the transmission received.

[0174] From a hardware point of view, the proposed solution is suitable for use in all existing equipment used in code (e.g. portable military or civilian radio, on fixed post, etc.)

[0175] Furthermore, optionally, as mentioned above, the present invention is capable of offering new functionalities compared to existing vocoders by allowing, via the reconstitution module 20, a reconstitution (from the English inpainting) of the missing frames of the voice synthesis which would have been produced by an existing vocoder in order in particular to fill in micro-cuts, and / or again, via the band extension module 22, a band extension without changing the codec of the already existing vocoders, which allows interoperability with radio stations already transmitting in the identified codec.

[0176] This results in an improvement in the quality of voice synthesis at a constant bit rate and the solution according to the present invention is capable of reaching, or even exceeding, the performance of vocoders that are twice as demanding in bit rate as the existing vocoder having provided it with the input parameters characterizing each frame of the voice synthesis capable of being produced by said existing vocoder. The reduction in bit rate is also accompanied by new performance gains related to communication. Indeed, at a lower data rate, the range of the transmission stations is increased. A lower data rate allows, at a constant transmission capacity, to allocate space to other transmissions.

[0177] The present invention further allows, as experimentally verified, due to the specific structure of the voice synthesizer with artificial intelligence of the improvement device according to the present invention, to provide an improvement compared to current vocoding with equivalent input parameters.

[0178] In particular, the source-filter type neural architecture of the voice synthesizer according to the present invention makes it possible to have a "light" solution (i.e. frugal and not very demanding) in computational footprint and memory compared to current AI solutions, integrable within embedded products, and opens up the possibility of implementing other "functionalities" such as, for example, reconstruction by Inpainting, bandwidth extension (this is an addition of information by prediction) or even "voice conversion" (i.e. switching from one voice to another voice (a priori associated with existing speakers) or even "voice transformation" (i.e. switching from one voice to another voice (which may more generally be that of a virtual speaker)).

[0179] By delegating voice synthesis to AI approaches with recurrent neural networks at each stage of the source-filter model rather than to solutions designed by “experts” (i.e. solutions resulting from modeling), the solution according to the present invention makes it possible in particular to compensate for a potential loss of information linked to the reduction in the number of input parameters, particularly in the context of a low-speed application.

[0180] The choice of a “biased” source-filter architecture (i.e. a neural network architecture that has been “biased” or rather specialized towards voice synthesis by adding speech modeling by a sum of harmonics) towards voice synthesis makes it possible to reduce the digital footprint, in terms of memory and calculation, of the implemented neural networks.

[0181] Finally, optionally, the practical difficulty of training neural networks in such an open task with few input parameters for a large number of possible temporal signals, is advantageously mitigated by the use of adversarial training where two discriminators are trained to recognize the signals generated by the voice synthesizer according to the present invention or jointly said voice synthesizer is trained to fool the discriminators.

[0182] Thus, the device and method for improving voice synthesis are suitable for integration into the framework of digital radiocommunication applications, voice over IP (from the English Internet Protocol) VoIP, audio compression, etc.

Claims

Claims

1. Device (100) for improving the voice synthesis produced, by means of a predetermined codec, by an existing vocoder, said device comprising: - a module (110) for receiving the parameters characterizing each frame of the voice synthesis capable of being produced by said existing vocoder; - at least one voice synthesizer with artificial intelligence (10) having an architecture according to a neural source-filter model, said voice synthesizer with artificial intelligence comprising at least: - a projection module (12) configured to project, via a first monodirectional recurrent neural network, said parameters, or parameters derived from said parameters, into a latent space of configurable dimension;- a harmonic signal generator (14) configured to generate, via a second neural network, a signal representative of the harmonics of said voice synthesis associated with said parameters received via said reception module (110); - a voice synthesis module (16) configured to synthesize, via a third monodirectional recurrent neural network, an encoded voice associated with said human voice from: - the projection of said projected parameters in said latent space of configurable dimension provided by said projection module, and - said signal representative of the harmonics of the voice synthesis associated with said parameters received via said reception module (110) and provided by said generator; said voice synthesizer with artificial intelligence (10) being previously trained during a supervised learning phase using training voice syntheses produced by means of said predetermined codec.;

2. Device (100) according to claim 1, wherein the receiving module (110) comprises a tool for identifying said predetermined codec by analyzing the voice synthesis produced by said existing vocoder, said identification tool being further configured to select, from a plurality of voice synthesizers with artificial intelligence previously trained and associated respectively to distinct codecs, said voice synthesizer with artificial intelligence (10) corresponding to said identified predetermined codec.

3. Device (100) according to claim 1 or 2, wherein the receiving module (110) comprises an interpolation tool configured to linearly interpolate said parameters and provide as input to said voice synthesizer with artificial intelligence (10) interpolated parameters compatible with said voice synthesis module (16).

4. Device (100) according to any one of the preceding claims wherein said signal generated by the generator (14) and representative of the harmonics of said voice synthesis associated with said parameters received via said reception module (110) is a weighted sum of the signal corresponding to the pitch associated with said parameters received and its harmonics, said weighting being learned beforehand by said second neural network.

5. A device (100) according to claim 3, wherein white noise is added to said weighted sum.

6. Device (100) according to any one of the preceding claims, wherein the voice synthesizer with artificial intelligence (10) further comprises an adversarial training module (18) comprising two discriminators configured to be trained simultaneously with said voice synthesizer with artificial intelligence during its supervised learning phase, according to a zero-sum game, said discriminators being trained to differentiate real signals from those suitable for being synthesized by said voice synthesizer with artificial intelligence, said voice synthesizer with artificial intelligence being trained to fool said two discriminators.

7. The device (100) of claim 5, wherein one of said two discriminators is responsive to low frequency information and the other of said two discriminators is responsive to high frequency information.

8. Device (100) according to any one of the preceding claims, wherein the voice synthesizer with artificial intelligence (10) further comprises: - a reconstruction module (20) configured to reconstruct missing frames of said synthesized encoded voice from frames before or after said missing frames, and / or

9. - a band extension module (22) configured to reconstruct a signal comprising said synthesized encoded voice with a sampling frequency higher than that of said voice synthesis associated with said parameters received via said reception module (110). A method (70) for improving the speech synthesis produced, by means of a predetermined codec, by an existing vocoder, said method being implemented by an improvement device (100) according to any one of the preceding claims, said method comprising: - receiving the parameters characterizing each frame of the voice synthesis suitable for being produced by said existing vocoder; and the following steps implemented during an inference phase by a voice synthesizer with artificial intelligence (10) of said improvement device (100), said voice synthesizer with artificial intelligence (10) being previously trained during a supervised learning phase (88) using training parameters characterizing each frame of voice syntheses suitable for being produced by means of said predetermined codec: - projection (74), via a first monodirectional recurrent neural network of said voice synthesizer with artificial intelligence, of the parameters received via said reception module (110) or of the parameters derived from said parameters, into a latent space of configurable dimension; - generation (76), via a second neural network of said voice synthesizer with artificial intelligence (10), of a signal representative of the harmonics of the voice synthesis associated with said parameters received via said reception module (110); - voice synthesis (78), via a third monodirectional recurrent neural network, of an encoded voice associated with said human voice from: - the projection of said projected parameters into said latent space of configurable dimension provided by said projection module, and - said signal representative of the harmonics of the voice synthesis associated with said parameters received via said reception module (110) and provided by the generator of the voice synthesizer with artificial intelligence (10) of said improvement device (100).

10. A program comprising software instructions which, when executed by a computer, implement a method for improving speech synthesis according to claim 9.

Images