Device and method for improving speech synthesis produced, using a predetermined codec, by an existing vocoder, and associated computer program

Abstract

Device and method for improving speech synthesis produced, by means of a predetermined codec, by an existing vocoder, and associated computer program. The present invention relates to a device for improving the speech synthesis of an existing vocoder, comprising: - a module for receiving the parameters characterizing the speech synthesis to be produced by said existing vocoder; - at least one speech synthesizer with artificial intelligence (10) comprising: - a projection module (12), via a first neural network, of said parameters in a latent space of parameterizable dimension; - a harmonic signal generator (14) generating, via a second neural network, a signal representative of the harmonics associated with said parameters; - a speech synthesis module (16), via a third neural network, from the projection, and of said signal representative of the harmonics;said speech synthesizer being trained during a supervised learning phase. Figure: Figure 1;

Description

Title of the invention: Device and method for improving speech synthesis produced, using a predetermined code, by an existing vocoder, and associated computer program

[0001] The present invention relates to a device for improving speech synthesis produced, by means of a predetermined code, by an existing vocoder.

[0002] The present invention also relates to a method for improving speech synthesis produced, by means of a predetermined code, by an existing vocoder.

[0003] The invention also relates to a computer program comprising software instructions which, when implemented by a computer, implement such a process.

[0004] The invention relates to the field of speech coding also called "vocoding", speech synthesis and voice transmission.

[0005] Hereafter, according to the present invention, "vocoding" and by extension the corresponding device "vocoder" is associated with the contraction of the English words Voice Coding and Voice coder, in other words, voice coding by an electronic sound signal processing device, corresponding substantially to a process according to which, from a decomposition into important properties (e.g., main spectral components) of an input voice or other sound, synthesizes an associated synthetic voice at the output.

[0006] Digital communications offer a number of significant advantages over analog communications. In addition to being of higher quality, they are often more robust against interference during transmission. However, one of the major problems inherent in the very nature of digital communications is the data rate limitation. Indeed, the higher the data rate, the more information can be sent, resulting in higher voice quality. Furthermore, digital transmissions suffer from a "binarization" of the received data. In fact, either the transmission of the digital frames is correct and the communication remains unchanged, or it is incorrect (i.e., missing frames) and all the information contained in these missing frames is lost at the receiver.

[0007] Certain transmission contexts impose constrained data rates and the use of coded signals (or encoder-decoders) whose sound quality is limited. In these contexts, the coded signals used are generally standardized, such as the MELP (Mixed-excitation linearprediction) coded signal or the enhanced MELPe (enhanced Mixed-excitation linearprediction) coded signal associated with the more often at a rate, for example MELP 1200 corresponding to a coded MELP with a rate of 1200 bits / s.

[0008] The coded ones are generally optimized to work according to a precise model (for example LPC (from the English Linear Predictive Coder, MELP) with regard to the extraction of parameters from the speech signal.

[0009] The resulting parametric model can be used to synthesize a speech signal. This is referred to as a parametric encoder or vocoder (to specify that the model applies to the speech signal).

[0010] The quantization / dequantization step enables transmission over a constrained channel by restricting the data rate. Therefore, there are two sources of degradation: the approximation of the model, and the quantization of the model parameters.

[0011] The different MELP bit rates correspond to different operating modes. It is the quantization scheme that is optimized for a given model (LPC, MELP, etc.) and a target bit rate. While the binary streams are obviously different, the information is of the same type (associated with the MELP parameters).

[0012] A binary train associated with the MELPe 1200 code works with the MELPe 1200 vocoder and cannot work with a MELPe 2400 vocoder.

[0013] Indeed, a modification of the quantization scheme implies a loss of interoperability: a MELP 1200 is not interoperable with a MELP 24OO. Moreover, with these classic low-speed solutions, a loss of frame transmission is accompanied by a break during the decoding of the communication.

[0014] The aim of the invention is therefore to improve the sound quality and extend the robustness of low bitrate or even very low bitrate coded audio used in certain transmission contexts which impose constrained bitrates, such as for example the aforementioned MELPe 1200 coded audio, while remaining interoperable with existing vocoders.

[0015] To this end, the invention relates to a device for improving the speech synthesis produced, by means of a predetermined code, by an existing vocoder, said device comprising:

[0016] - a module for receiving parameters characterizing each frame of the synthesis vocal specific to be produced by said existing vocoder;

[0017] - at least one speech synthesizer with artificial intelligence exhibiting a architecture based on a neural source-filter model, said speech synthesizer with artificial intelligence comprising at least:

[0018] - a projection module configured to project, via a first neural network recurrent unidirectional, said parameters, or parameters derived from said parameters, in a latent space of parameterizable dimension;

[0019] - a harmonic signal generator configured to generate, via a second network of neurons, a signal representative of the harmonics of said speech synthesis associated with said parameters received via said receiving module;

[0020] - a speech synthesis module configured to synthesize, via a third network of unidirectional recurrent neurons, an encoded voice associated with said human voice from:

[0021] - the projection of said projected parameters into said latent space of dimension configurable provided by said projection module, and

[0022] - said signal representing the harmonics of the speech synthesis associated with said parameters received via said receiving module and provided by said generator;

[0023] said voice synthesizer with artificial intelligence being previously trained during a supervised learning phase using training voice syntheses produced by means of said predetermined code.

[0024] Note that by "improvement" we mean a better perception of the quality of the transmitted voice but also better robustness to transmission interruptions (i.e. frame losses).

[0025] The improvement device according to the present invention is advantageous because it proposes to use a speech synthesizer with artificial intelligence to improve the speech synthesis of an already existing vocoder.

[0026] The AI ​​artificial intelligence speech synthesizer of the enhancement device according to the present invention is advantageous because, unlike the neural source-filter architectures of the literature, it uses unidirectional recurrent neural networks compatible with real-time use, i.e. with a latency of less than 50ms.

[0027] Furthermore, the artificial intelligence speech synthesizer according to the present invention is configured to provide a frugal speech synthesis solution, as it is based on a data-efficient architecture. Indeed, while the source-filter model is a classic model for decomposing voice into source and filter components, the use of this model, enhanced with "modern" unidirectional recurrent neural networks (i.e., those capable of restoring causality), provides a suitable framework for voice modeling. Such "AI enhancement" results in better voice reconstruction with lower memory and computational costs.

[0028] The AI-powered speech synthesizer according to the present invention has the advantage of optimally learning how to process input data, namely the parameters provided by an existing vocoder, to perform its task, i.e., the reproduction of speech communication. To this end, said AI-powered speech synthesizer of the proposed enhancement device It specializes in a mode of operation following a so-called supervised learning phase where the AI-powered speech synthesizer is trained using a binary stream of low-bitrate code. This preliminary training then allows, during the inference phase, a significant improvement in the quality of the result originally provided by the low-bitrate code.

[0029] More precisely, the neural network here does not use the binary train directly but the model parameters previously reconstructed from the binary train (dequantization step) or even other parameters derived from the model parameters (as a replacement or in addition to them).

[0030] According to other advantageous aspects of the invention, the improvement device comprises one or more of the following features, taken individually or in all technically possible combinations:

[0031] - the receiving module includes a tool for identifying said predetermined code by analyzing the speech synthesis produced by said existing vocoder, said identification tool being further configured to select, from among a plurality of previously trained artificial intelligence speech synthesizers associated respectively with distinct codes, said artificial intelligence speech synthesizer corresponding to said predetermined identified code;

[0032] - the receiving module includes an interpolation tool configured to interpolate linearly the said parameters and provide as input to said speech synthesizer with artificial intelligence interpolated parameters compatible with said speech synthesis module;

[0033] - said signal generated by the generator and representative of the harmonics of said speech synthesis associated with said parameters received via said receiving module is a weighted sum of the signal corresponding to the pitch associated with said received parameters and its harmonics, said weighting being learned beforehand by said second neural network;

[0034] - white noise is added to said weighted sum;

[0035] - the voice synthesizer with artificial intelligence further comprises a module an opposing training system comprising two discriminators configured to be trained simultaneously with said AI-enabled speech synthesizer during its supervised learning phase, according to a zero-sum game, said discriminators being trained to differentiate real signals from those likely to be synthesized by said AI-enabled speech synthesizer, said AI-enabled speech synthesizer being trained to fool said two discriminators;

[0036] - one of said two discriminators is sensitive to low-frequency information and the other of the two said discriminators is sensitive to high-frequency information;

[0037] - the voice synthesizer with artificial intelligence further comprises:

[0038] - a reconstruction module configured to reconstruct missing frames of said encoded voice synthesized from frames prior to or subsequent to said missing frames, and / or

[0039] - a band extender module configured to reconstruct a signal comprising said encoded voice synthesized with a sampling frequency higher than that of said speech synthesis associated with said parameters received via said receiving module.

[0040] The invention also relates to a method for improving speech synthesis produced, by means of a predetermined code, by an existing vocoder, said method being implemented by an enhancement device as previously described, said method comprising:

[0041] - the reception of the parameters characterizing each frame of the speech synthesis suitable to be produced by said existing vocoder;

[0042] and the following steps implemented during an inference phase by a speech synthesizer with artificial intelligence of said enhancement device, said speech synthesizer with artificial intelligence being previously trained during a supervised learning phase using training parameters characterizing each speech synthesis frame suitable for production by means of said predetermined code:

[0043] - projection, via a first unidirectional recurrent neural network of said speech synthesizer with artificial intelligence, parameters received via said receiving module or parameters derived from said parameters, in a latent space of configurable dimension;

[0044] - generation, via a second neural network of said speech synthesizer with artificial intelligence, of a signal representative of the harmonics of speech synthesis associated with said parameters received via said receiving module;

[0045] - speech synthesis, via a third unidirectional recurrent neural network, of an encoded voice associated with said human voice from:

[0046] - the projection of said projected parameters into said latent space of dimension configurable provided by said projection module, and

[0047] - said signal representing the harmonics of the speech synthesis associated with said parameters received via said receiving module and provided by the voice synthesizer generator with artificial intelligence of said enhancement device.

[0048] The invention also relates to a computer program comprising software instructions which, when executed by a computer, implement a speech synthesis process, as defined above.

[0049] The invention will become clearer upon reading the following description, given solely by way of non-limiting example, and made with reference to the drawings in which:

[0050] [Fig-1] [Fig.1] is a general schematic representation of a device improvement of speech synthesis produced, by means of a predetermined code, by an existing vocoder according to the present invention;

[0051] [Fig.2] [Fig.2] illustrates the use of the enhancement device of [Fig.1];

[0052] [Fig.3] [Fig.3] illustrates an example of the architecture of the device for improving the [Fig.l];

[0053] [Fig.4] [Fig.4] illustrates the implementation of an optional module of the improvement device of [Fig.1] during its training phase;

[0054] [Fig.5] [Fig.5] is a flowchart of a method for improving speech synthesis produced, by means of a predetermined coded, by an existing vocoder implemented by the improvement device of [Fig.1] according to an embodiment.

[0055] In the following description, the expression "approximately equal to" is understood as a relationship of equality plus or minus 10%, that is to say with a variation of at most 10%, preferably also as a relationship of equality plus or minus 5%, that is to say with a variation of at most 5%.

[0056] Furthermore, subsequently, a neural network is considered to comprise an ordered succession of layers of neurons, each of which takes its inputs from the outputs of the previous layer.

[0057] More precisely, each layer comprises neurons taking their inputs from the outputs of the neurons of the previous layer, or from the input variables for the first layer.

[0058] Alternatively, more complex neural network structures can be envisaged with a layer that can be linked to a layer further away than the immediately preceding layer.

[0059] Each neuron is also associated with an operation, that is to say a type of processing, to be carried out by said neuron within the corresponding processing layer.

[0060] Each layer is connected to the other layers by a plurality of synapses. A synaptic weight is associated with each synapse, and each synapse forms a link between two neurons. It is often a real number, which takes on both positive and negative values. In some cases, the synaptic weight is a complex number.

[0061] Each neuron is capable of performing a weighted sum of the value(s) received from the neurons of the previous layer, each value then being multiplied by the respective synaptic weight of each synapse, or link, between said neuron and the neurons of the previous layer, and then applying an activation function, Typically, a non-linear function, weighted to the said sum, delivers at the output of said neuron, particularly to the neurons of the next layer connected to it, the value resulting from the application of the activation function. The activation function introduces non-linearity into the processing performed by each neuron. The sigmoid function, the hyperbolic tangent function, and the Heaviside function are examples of activation functions.

[0062] As an optional complement, each neuron is also capable of applying, in addition, a multiplicative factor, also called bias, to the output of the activation function, and the value delivered at the output of said neuron is then the product of the bias value and the value from the activation function.

[0063] A fully connected layer of neurons is a layer in which the neurons of said layer are each connected to all the neurons of the preceding layer.

[0064] Such a type of layer is more often referred to by the English term "fully connected", and sometimes designated by the name "dense layer".

[0065] The electronic device for improving speech synthesis produced, by means of a predetermined code, by an existing vocoder is illustrated in [Fig.1].

[0066] Such an existing vocoder classically produces a parametric model of the speech signal with on the one hand an analysis module implementing the extraction of parameters and on the other hand a synthesis module implementing the reconstruction of the signal from the extracted parameters.

[0067] By coded, we mean the combination of an encoder and a decoder (i.e. encoder + decoder), the encoder comprising for its part an analysis module and a module for quantifying the parameters from the analysis, the decoder being suitable for performing the dequantization (i.e. the restitution of the parameters from the binary stream) and then the speech synthesis as such.

[0068] The electronic speech synthesis enhancement device is located at the output of the existing vocoder dequantization module (i.e., a classic one such as a MELP 1200 vocoder), that is, after the dequantization module of the classically coded decoder configured to dequantize (i.e., to restore the parameters from the binary stream).

[0069] In other words, the electronic speech synthesis enhancement device according to the present invention uses as input the Sv parameters resulting from the dequantization of the existing vocoder and / or parameters derived from them, and replaces the pre-existing synthesis module of the existing vocoder. Thus, the synthesis model of the existing vocoder is no longer used but replaced by the speech synthesizer 10 described below, which uses as input the parameters resulting from the dequantization of the existing vocoder 200 and / or parameters derived from them.

[0070] Such an electronic speech synthesis enhancement device includes first of all a module for receiving the parameters characterizing each frame of the speech synthesis specific to be produced by said existing vocoder.

[0071] An example of the architecture of this receiving module 110 is described in particular later in relation to [Fig.3].

[0072] Optionally, the receiver module 110 includes a tool for identifying said predetermined code, the identification tool being configured to determine said predetermined code by analyzing the parameters characterizing each frame of the speech synthesis specific to be produced by said existing vocoder.

[0073] Furthermore, said optional identification tool is further configured to select, from among a plurality, not shown, of previously trained artificial intelligence speech synthesizers associated respectively with distinct codes, said artificial intelligence speech synthesizer 10 corresponding to said predetermined identified code.

[0074] More specifically, the receiver module 110 is configured at a given time to operate with a speech encoder (there is a negotiation between the transmitter and the receiver to use a common encoder). Upon reception, the information is therefore available, and the previously trained and adapted speech synthesizer with artificial intelligence can be selected or configured (for example, same network topology but different weights for a different quantization scheme but an identical parametric model) based on this information.

[0075] In other words, such an identification tool makes it possible to adapt the speech synthesizer with artificial intelligence to be used subsequently, according to the received binary stream that it used to identify the coded string used by the existing vocoder whose speech synthesis is to be improved according to the present invention. Thus, the identification tool is capable of implementing an identification phase of the coded string used by the existing vocoder in order to then switch to the speech synthesizer with artificial intelligence 10 adapted to the received Sv parameters, particularly when the improvement device previously comprises a plurality of speech synthesizers with artificial intelligence that have been previously trained and respectively associated with distinct coded strings.

[0076] Thus, the selected artificial intelligence speech synthesizer 10 was previously trained during a supervised learning phase using training speech syntheses produced by means of the identified predetermined code. In other words, supervised training adapted to the use case is implemented by exploiting the parameters encoded by the code of the existing vocoder whose speech synthesis is to be improved.

[0077] According to another variant, the receiving module 110 does not include an identification tool, and the speech synthesizer with artificial intelligence 10 is directly adapted to the existing vocoder, namely that it has been previously trained with training speech syntheses produced using the predetermined code of the existing vocoder, for example using a MELPe 1200 binary train to guarantee the condition of interoperability with an existing MELPe 1200 vocoder. Once trained, the speech synthesizer with artificial intelligence 10 therefore receives, via its receiver module 110, as input the current decoded parameters provided by the decoder of the existing MELPe 1200 vocoder.

[0078] As can be seen in this [Fig.1], the speech synthesizer with artificial intelligence 10 includes a projection module 12 configured to project, via a first unidirectional recurrent neural network RNb, parameters characterizing each frame of said speech synthesis to be produced by the existing vocoder, received via said receiving module, into a latent space (from the English embedding) of parameterizable dimension.

[0079] The input parameters of the unidirectional recurrent network RNi can be the parameters of the encoder used from the existing vocoder or be replaced or supplemented by parameters obtained by transformation of the initial parameters.

[0080] For example, for MELP, the parameters characterizing the spectral envelope of the speech signal are the LSF (Line Spectral Frequencies) parameters directly associated with the LPC model. These parameters can be used as input to the unidirectional recurrent network RNi or replaced by other parameters such as: linear prediction ceptral coefficients (LPCC) or log-spectral coefficients obtained by DCT (Discrete Cosine Transform) of the cepstral coefficients. These transformations can allow the exploitation of orthogonalization properties or improved resolution by increasing the number of parameters (for example, ten LSF coefficients, eighteen LPCC coefficients, sixty-four spectral coefficients, etc.).

[0081] The LPCC coefficients are obtained by transforming the linear prediction coefficients (LPC model). The relationships between the LSF coefficients, the linear prediction coefficients, and the LPCC coefficients are established and known in the field of speech processing.

[0082] Such a projection module 12 is also called the speech synthesizer encoder with artificial intelligence 10, and is therefore capable of "embedding" the parameters characterizing each frame of the speech synthesis to be produced by said existing vocoder and received via said receiving module 110 into a latent space (i.e., a vector space) of parameterizable dimension, for example, of size equal to 128 (i.e., capable of comprising 128 components). The neural network RNi, previously trained as will be seen later, handles the nonlinear projection of these parameters in the latent space potentially containing more information than the parameters themselves, since temporal relationships can be incorporated there thanks to the recurrent and unidirectional nature of the RNb neural network

[0083] The parameters to be taken into account by said projection module 12 are parameters characterizing each frame of the speech synthesis to be produced by said existing vocoder and received via said receiving module 110. The nature of these parameters is imposed by the existing vocoder.

[0084] As illustrated by [Fig.1], the speech synthesizer with artificial intelligence 10 further includes a harmonic signal generator 14 configured to generate, via a second neural network RN2, a signal representative of the harmonics of said speech synthesis associated with said parameters received via said receiver module 110.

[0085] As an optional complement, such a harmonic signal generator 14 is configured to generate said signal representing the harmonics of said speech synthesis associated with said parameters received via said receiving module 110 which is a weighted sum of the signal corresponding to the pitch associated with said speech synthesis itself associated with said received parameters and its harmonics, said weighting being learned beforehand by said second neural network.

[0086] According to a variant of this optional complement, white noise is added to said weighted sum.

[0087] In other words, said harmonic signal generator 14 is configured to produce a signal composed of a sinusoidal signal at the pitch associated with said speech synthesis associated with said received parameters and its harmonics. The output signal of this generator 14 is a weighted sum of the pitch associated with said speech synthesis associated with said received parameters and its harmonics. The weighting is learned by the second neural network RN2 during a preliminary training phase. White noise is added to this harmonic signal to serve as the basis for the stochastic components of the voice. Unvoiced frames or frames not containing voice consist only of white noise without a harmonic component.

[0088] According to a speech synthesizer architecture based on the known "source-filter" model, the output generated by the harmonic signal generator 14 plays the role of the "source" part of this "source-filter" architecture.

[0089] As illustrated by [Fig. 1], the speech synthesizer with artificial intelligence 10 further comprises a speech synthesis module 16 configured to synthesize, via a third unidirectional recurrent neural network RN3, an encoded voice (associated with the human voice of the speaker who produced a sound signal as input to the existing vocoder) from:

[0090] - the projection of said projected parameters into said latent space of dimension configurable provided by said projection module 12, and

[0091] - said signal representing the harmonics of said associated speech synthesis to the said parameters received via said receiving module 110 and supplied by said generator 14.

[0092] Such a speech synthesis module 16 is also called a speech synthesizer decoder with artificial intelligence 10, and like the preceding elements 12 and 14 of the speech synthesizer with artificial intelligence 10, it is also based on a recurrent neural network architecture. Such a speech synthesis module 16 therefore has as inputs the output of the projection model 12 and the output of the harmonic signal generator 14. The projection model 12 and the harmonic signal generator 14 are capable of operating in parallel or successively in any order, provided that the output of the projection model 12 and the output of the harmonic signal generator 14, associated with the same frame of said speech synthesis and associated with said parameters received via said receiver module 110, are processed simultaneously by said speech synthesis module 16.

[0093] According to a speech synthesizer architecture with artificial intelligence 10 according to the known "source-filter" model, the speech synthesis module 16 plays the role of the "filter" part of this "source-filter" architecture.

[0094] Thus, unlike neural source-filter architectures according to the current state of the art, the use of unidirectional recurrent neural networks proposed according to the present invention allows to operate in real time.

[0095] For example, the voice synthesizer with artificial intelligence 10 according to the present invention is suitable for working with said voice synthesis associated with said parameters received via said receiving module 110 previously decomposed into 40ms frames with a look-ahead buffer of 40ms, essentially due to the extraction of input parameters from the projection module 12 (i.e., the encoder).

[0096] As an optional complement, as shown in dotted lines, said speech synthesizer with artificial intelligence 10 further includes an adversarial training module 18 comprising two discriminators Di and D2 configured to be trained simultaneously with said speech synthesizer with artificial intelligence 10 during its supervised learning phase, according to a zero-sum game, said discriminators Di and D2 being trained to differentiate real signals from those likely to be synthesized by said speech synthesizer with artificial intelligence 10, said speech synthesizer with artificial intelligence 10 being trained to fool said two discriminators Di and D2.

[0097] It should be noted that once the learning phase has been completed, such an adversarial training module 18 is no longer used during the inference phase using the speech synthesizer with artificial intelligence 10 thus previously trained in an adversarial manner.

[0098] According to an advantageous variant of this optional complement, one of said two discriminators is sensitive to low frequency information and the other of said two discriminators is sensitive to high frequency information.

[0099] In other words, the two discriminators Di and D2 are trained at the same time as the speech synthesizer with artificial intelligence 10 to differentiate the real signals from the signals synthesized by the speech synthesizer with artificial intelligence 10 by creating a zero-sum game where the discriminators Di and D2 try to differentiate the signals created by the speech synthesizer with artificial intelligence 10 from the original signals and where the speech synthesizer with artificial intelligence 10 is trained to fool the two discriminators Di and D2.

[0100] Such a zero-sum game makes it possible to balance the AI-powered speech synthesizer 10 resulting from said adversarial training, and the use of two discriminators, sensitive to low and high frequency information, allows the AI-powered speech synthesizer 10 to correct its defects over extended frequency ranges, and is particularly effective at removing metallic artifacts from the voice synthesized via the AI-powered speech synthesizer 10.

[0101] For example, the two discriminators correspond to the discriminators Di and D2 described in the article by J.Kong et al entitled “HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis” NeurIPS 2020, but applied in another context, without combined implementation of a speech synthesizer architecture according to a source-filter model and the zero-sum game according to the present invention where said discriminators Di and D2 are trained to differentiate real signals from those suitable for being synthesized by said speech synthesizer with artificial intelligence 10, said speech synthesizer with artificial intelligence 10 being trained to fool said discriminators Di and D2.

[0102] The opponent training implemented via said optional opponent training module 18 is further described later in relation to [Fig.4].

[0103] Such an optional adversary training module 18 therefore makes it possible to improve the quality of synthesis.

[0104] As an optional addition, said speech synthesizer with artificial intelligence 10 further comprises a reconstruction module 20 configured to reconstruct (from the English "inpainting") missing frames of said encoded voice synthesized from frames prior to or subsequent to said missing frames, and / or a band extension module 22 configured to reconstruct a signal including said encoded synthesized voice with a sampling frequency higher than that of said speech synthesis associated with said parameters received via said receiving module 110.

[0105] The optional reconstruction module 20 makes it possible to improve the overall transmission of the synthesized signal via said speech synthesizer with artificial intelligence 10 by compensating for micro-interruptions which would occur without its implementation due to missing frame(s), while the optional band extension module 22 makes it possible to increase the output sampling frequency, for example to predict missing information (between 4 and 8 kHz for example for a change from a sampling frequency of 8 to 16 kHz).

[0106] As an optional complement, at least one of said neural networks of said speech synthesizer with artificial intelligence 10, for example the first unidirectional recurrent neural network RNi of the projection module 12 or the second neural network RN2, is also configured to extract the vocal biomarkers of the original speaker who generated said speech synthesis associated with said parameters received via said reception module 110, after having been previously trained for this purpose in a supervised manner.

[0107] In an alternative not shown, the speech synthesizer with artificial intelligence 10 is also capable of optionally including another neural network dedicated to the extraction of vocal biomarkers of the speaker who generated said speech synthesis associated with said parameters received via said reception module 110 after having been previously trained for this purpose in a supervised manner.

[0108] In the example of [Fig.1], the speech synthesizer with artificial intelligence 10 includes an information processing unit 26 formed for example of a memory 28 and a processor 30 associated with the memory 28.

[0109] In the example of [Fig. 1], the projection module 12, the generator 14, and the speech synthesis module 16, as well as the optional adversary training module 18, the reconstruction module 20, and the band extension module 22, are each implemented as software, or a software component, executable by the processor 30. The memory 28 of the speech synthesizer with artificial intelligence 10 is then capable of storing projection software, harmonic signal generation software, and speech synthesis software, as well as the optional adversary training software, reconstruction software, and band extension software. The processor is then capable of executing each of the following software components: projection software, harmonic signal generation software, and speech synthesis software, as well as the optional component Opponent training software, replay software, band extension software.

[0110] In an alternative not shown, the projection module 12, the generator 14 and the speech synthesis module 16, as well as optionally the opponent training module 18, the reconstruction module 20 and the band extension module 22 are each made in the form of a programmable logic component, such as an FPGA (Field Programmable Gate Array), or an integrated circuit, such as an ASIC (Application Specified Integrated Circuit).

[0111] When the speech synthesizer with artificial intelligence 10 is implemented in the form of one or more software programs, i.e., in the form of a computer program, also called a computer program product, it is further capable of being stored on a computer-readable medium, not shown. The computer-readable medium is, for example, a medium capable of storing electronic instructions and being connected to a bus of a computer system. By way of example, the readable medium is an optical disc, a magneto-optical disc, ROM, RAM, any type of non-volatile memory (e.g., FLASH or NVRAM), or a magnetic card. A computer program comprising software instructions is then stored on the readable medium.

[0112] Figure 2 illustrates the use 32 of the speech synthesizer with artificial intelligence 10 of Figure 1. According to this use, a speaker 34 first produces a sound signal 36 comprising their own human voice. Such a sound signal 36 is then fed into the input of the existing vocoder.

[0113] The existing vocoder is configured to extract from the sound signal 36 the parameters conforming to the coded it uses, implement a quantization and a dequantization providing the decoded parameters 39 as input to the speech synthesis enhancement device.

[0114] The nature of these parameters 39 is imposed by the existing vocoder. These parameters 39 are provided as input to the projection module 40 (corresponding to the projection module 12 of [Fig.2]) which includes the first unidirectional recurrent neural network RNi configured for said parameters in a latent space of parameterizable dimension.

[0115] In parallel, the decoded parameters 39 received via said receiving module 110 are provided as input to the harmonic signal generator 42 (corresponding to generator 14 of [Fig.1]), said generator 42 providing as output a signal 44 representative of the harmonics of said speech synthesis associated with said parameters received via said receiving module 110.

[0116] Said signal 44 representing the harmonics of said speech synthesis associated with said parameters received via said receiving module 110 and the result provided by the projection module 40 are associated with the same frame of said speech synthesis associated with said parameters received via said receiving module 110 and provided simultaneously as input to the speech synthesizer 46 configured to synthesize, via a third unidirectional recurrent neural network RN3, an encoded voice 48 associated with the decoded parameters 39 received via said receiving module 110 of the speech synthesis enhancement device.

[0117] Figure 3 illustrates an example of the architecture of the speech synthesis enhancement device 100. According to this example, the speech synthesis enhancement device 100 receives as input parameters Sv characterizing each frame of the speech synthesis specific to be produced by an existing vocoder, for which its own speech synthesizer with artificial intelligence 10 has been previously trained.

[0118] For example, the existing vocoder is of the MELPe 1200 type (or of the MELPe 2400 type, etc.) and provides a MELPe 1200 (respectively MELPe 2400) binary stream. The speech synthesizer with artificial intelligence 10 was previously trained with acoustic parameters derived from this type of MELPe 1200 (respectively MELPe 2400) binary stream in order to guarantee interoperability with the existing vocoder 200 of the MELPe 1200 (respectively MELPe 2400) type. The MELP 2400 uses a quantization scheme on a 54-bit frame with a duration of 22.5 ms. The MELP 1200 uses a quantization scheme on a "super-frame" composed of three consecutive frames = 3 x 22.5 ms = 67.5 ms. The quantization scheme allows us to exploit information redundancy and reduce the data rate to 81 bits / superframe.

[0119] The receiving module 110 therefore receives as input parameters Sv including, according to an example, the decoded parameters of the MELPe 1200.

[0120] As an optional complement, the receiver module 110 includes an interpolation tool configured to linearly interpolate the received parameters and provide input to said speech synthesizer with artificial intelligence 10 of interpolated parameters compatible with said speech synthesis module 16.

[0121] In other words, if the existing vocoder uses a frame duration unsuitable for the AI ​​speech synthesizer 10 of the enhancement device 100, the parameters from the existing vocoder are interpolated to be synchronous with the speech synthesis module 16 of the AI ​​speech synthesizer 10. Optionally, a change of parametric space is also applied.

[0122] It is thus possible to temporally interpolate the input parameters of the speech synthesizer with artificial intelligence 10 to adapt the operating rate of the network (often to increase the synthesis rate: for example, from 22.5 ms to 10 ms). The interpolation can be performed linearly (for example, at the level of the LSF coefficients, which have good properties). interpolation, and which can be used via successive transformations from LSF to LPCC, then from LPCC to spectral coefficients, the transition relationships between LSF coefficients, linear prediction coefficients and LPCC coefficients being established and known in the field of speech processing.

[0123] Thus, optionally interpolated, the decoded parameters of the existing vocoder are extracted, namely for example the variables corresponding in particular to the linear prediction cepstral coefficients LPCC (from the English linearprediction cepstral coefficients), to the dynamic linear prediction cepstral coefficients DLPCC (from the English dynamic linearprediction cepstral coefficients), to the gain and to the voicing.

[0124] According to the architectural example in [Fig.3], these parameters are pre-processed upstream of the speech synthesizer with artificial intelligence 10.

[0125] More specifically, the variables corresponding to the linear prediction cepstral coefficients LPCC (from the English linear prediction cepstral coefficients), and to the dynamic linear prediction cepstral coefficients DLPCC (from the English dynamic linear prediction cepstral coefficients) are processed by a normalization tool configured to center and normalize them (i.e. normalize them on the scale of the batch of data (from the English batch)) in input to the speech synthesizer with artificial intelligence 10. Note that the dynamic linear prediction cepstral coefficients DLPCC are obtained by derivation of the LPCC coefficients using generally first and second order derivatives.

[0126] The gain is projected via a projection tool into a space of dimension higher than its input dimension, for example, of dimension eight. For example, the projection tool includes a simple perceptron for this purpose.

[0127] The voicing is also projected via a projection tool into a space of higher dimension than its input dimension. For example, the projection tool includes a simple perceptron for this purpose.

[0128] The receiving module 110, as illustrated by the example in [Fig. 3], is configured to provide output 54, which includes the outputs of the normalization and projection tools into the input of the projection module 12, comprising the first unidirectional recurrent neural network RNp

[0129] In addition, the receiver module 110 also provides an output, including the Sv parameters characterizing each frame of speech synthesis specific to be produced by an existing vocoder, as input to the harmonic signal generator 14.

[0130] As illustrated by the architectural example, the projection module 12 includes the first unidirectional recurrent neural network RNi configured to project the parameters of the output 54 grouping the elements provided by the aforementioned tools into a latent space of parameterizable dimension, for example of dimension 128 (i.e. able to include 128 components).

[0131] As illustrated by the example in [Fig.3], the first unidirectional recurrent neural network RNi comprises successively a normalization layer LN (from the English Layer-norm), followed by three dense layers, a gate recurrent unit GRU (from the English Gate Recurrent Unit) and a dense layer, so as to provide as output the projection Z of said parameters in the latent space of parameterizable dimension.

[0132] As a variant to the GRU unit, a layer with long short-term memory type LSTM (Long Short Terni Memory) or, more classically, a recurrent layer RNN (Recurrent Neural Network) is used.

[0133] Furthermore, as illustrated by the architectural example, the harmonic signal generator 14 optionally includes first of all a module 56 suitable for extracting the pitch associated with said parameters Sv.

[0134] The harmonic signal generator 14 further comprises a second neural network RN2 including, in particular, a Sin Gen layer configured to generate a sinusoidal signal at the pitch associated with said received parameters, followed by a dense layer such that the signal produced by the generator 14 is a signal composed of a sinusoidal signal at the pitch associated with said received parameters and its harmonics, this signal corresponding to a weighted sum E of the pitch and its harmonics. The weighting is learned by the second neural network RN2 during a prior training phase. The output generated E by the harmonic signal generator 14 acts as the "source" part of this "source-filter" architecture.

[0135] The projection model 12 and the harmonic signal generator 14 are suitable for operating in parallel or successively in any order, provided that the output Z of the projection model 12 and the output E of the harmonic signal generator 14 are supplied as input and processed simultaneously by said speech synthesis module 16.

[0136] From said inputs Z and E, the speech synthesis module 16, via its third unidirectional recurrent neural network RN3, is configured to synthesize an encoded voice associated with said parameters Sv.

[0137] More specifically, the speech synthesis module 16 plays the role of the "filter" part of this "source-filter" architecture of the synthesizer according to the present invention. According to the example in [Fig. 3], the third unidirectional recurrent neural network RN3 of the speech synthesis module 16 comprises successively a first dense layer corresponding in particular to a classical perceptron and having only one input, the input E.

[0138] This first dense layer of the third unidirectional recurrent neural network RN3 is followed by two filtering units Fi and F2 (from the English Filter successive units. The first filtering unit Fi receives two inputs, namely both the Z output of the projection model 12 and the output of the first dense layer, while the second filtering unit F2, placed after the first filtering unit Fi, also receives two inputs, namely both the Z output of the projection model 12 and the output of the first filtering unit Fh

[0139] These two filtering units Fi and F2 have an identical structure as illustrated at zoom 57 inside said filtering unit structure.

[0140] More specifically, each filtering unit Fi or F2 includes first of all a concatenation layer suitable for concatenating the two received inputs represented on the one hand by the letter "y" (corresponding to the output S of the first dense layer of the third unidirectional recurrent neural network RN3 for the first filtering unit Fi or to the output Su of the first filtering unit Fi for the second filtering unit F2), and on the other hand by the letter "Z" corresponding to the output Z of the projection model 12.

[0141] At the output of said concatenation layer, each filtering unit Fi or F2 includes a gate recurrent unit (GRU).

[0142] As a variant to the GRU unit, a layer with long short-term memory type LSTM (Long Short Terni Memory) or, more classically, a recurrent layer RNN (Recurrent Neural Network) is used.

[0143] The GRU unit of each filtering unit Fi or F2 provides two outputs noted in zoom 57 Si and S2 at the input of each of a dense layer, the input represented by the letter "y", as specified previously, being added as illustrated within zoom 57 of [Fig.3] to the output Si of one of these two dense layers.

[0144] For the first filtering unit FB, the outputs Si and S2 shown in zoom 57 correspond respectively to the outputs Si i and Si 2, while for the second filtering unit F2, the outputs Si and S2 shown in zoom 57 correspond respectively to the outputs S2i and S2 2.

[0145] Finally, as illustrated by the example in [Fig.3], the output S2i of the second filtering unit F2 is provided as input to a final dense layer of said third unidirectional recurrent neural network RN3.

[0146] Optionally, as illustrated by [Fig.3], the last dense layer of said third unidirectional recurrent neural network RN3 also receives a second input equal to the sum of: the output Si 2 of the first filtering unit Fi to which is added the output S2 2 of the second filtering unit F2, to which is further added random white noise represented by the letter a in particular to serve as a basis for the stochastic components of the voice.

[0147] The output 58 of the last dense layer of said third unidirectional recurrent neural network RN3 corresponds to the encoded channel (i.e. the synthesized voice) provided by the speech synthesis module 16.

[0148] Figure 4 illustrates the principle of adverse training optionally implemented according to the present invention.

[0149] More specifically, in [Fig.4] the adversarial training module 60 includes two discriminators (not shown in [Fig.4]) configured to be trained simultaneously with said speech synthesizer with artificial intelligence 10 during its supervised learning phase, according to a zero-sum game, said discriminators being trained to differentiate speech syntheses provided by an existing vocoder from those 48 suitable for being synthesized by said speech synthesizer with artificial intelligence 10 of the enhancement device 100, said speech synthesizer with artificial intelligence 10 being trained to fool said two discriminators.

[0150] For example, the two discriminators correspond to the discriminators Di and D2 described in the article by J.Kong et al entitled “HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis” NeurIPS 2020, but applied in another context without combined implementation of a speech synthesizer architecture according to a source-filter model and the zero-sum game according to the present invention where said discriminators Di and D2 are trained to differentiate the real signals from those suitable for being synthesized by said speech synthesizer with artificial intelligence 10, said speech synthesizer with artificial intelligence 10 being trained to fool said discriminators Di and D2.

[0151] As stated previously, advantageously one of said two discriminators is sensitive to low frequency information and the other of said two discriminators is sensitive to high frequency information, the use of two discriminators, sensitive to low and high frequency information, allows the speech synthesizer with artificial intelligence 10 to correct its defects over extended frequency ranges, and is particularly effective in removing metallic artifacts from the voice synthesized via the speech synthesizer with artificial intelligence 10.

[0152] More specifically, each discriminator in the adversarial training module 60 is configured to provide a PR prediction when the incoming signal is real 36 or a Ps prediction when the incoming signal has been synthesized by the speech synthesizer with artificial intelligence 10, and the prediction result is then used in a loss function of the synthesizer during training. The discriminators and synthesizers are trained jointly (i.e., joint learning).

[0153] The loss function of the speech synthesizer is the average, over the different frames of the sample, of the (1- Ps)2 (Ps being one if a discriminator "says" that the signal is real and zero otherwise).

[0154] The loss function of each discriminator is the sum of the averages of:

[0155] (1- PR)2 and Ps2 (Ps and PR equaling one if the discriminator predicts that the signal is real and zero otherwise).

[0156] The loss function taking into account this prediction during the supervised learning phase of the adversarial training module, comprising the two discriminators, and jointly of the speech synthesizer with artificial intelligence 10, is advantageously adaptive. In particular, during training, a time average of the spectrograms is performed, followed by a moving average over the frequencies (i.e., second axis of the spectrogram) in order to smooth the frequency distribution of the signal used for each iteration of the supervised learning.

[0157] Next, this smoothed spectrum x is transformed into v by means of the following activation function: v = 1.3 - sigmoid (logl0(amplitude(x))) in order to calculate, frequency weighting weights, in the loss function using the spectrogram, this loss function using the spectrogram corresponding to the sum for each time / frequency value of the amplitude of the spectrogram of the norm L1 (absolute value) of the difference between the predicted spectrogram and the reference spectrogram.

[0158] Such processing allows adaptation, at each stage of learning, to the signal actually used and therefore allows learning to focus on the relevant frequencies of the frame in question (i.e. the current packet of sound data (from the English batch) in order to have learning resulting in a better quality model.

[0159] A speech synthesis process 70 implemented via said speech synthesizer with artificial intelligence 10 will now be explained with reference to [Fig.5] showing a flowchart of the steps of this process 70.

[0160] In general, said process 70 includes, during its inference phase 72, a first step of receiving the parameters Sv produced by an existing vocoder.

[0161] Optionally, this reception step includes a substep of identifying the predetermined code used by the existing vocoder by analyzing the parameters characterizing each frame of the speech synthesis specific to be produced by said existing vocoder, said identification substep further including the selection, from among a plurality of previously trained artificial intelligence speech synthesizers associated respectively with distinct codees, of the artificial intelligence speech synthesizer corresponding to said identified predetermined code.

[0162] Said method 70 further comprises a plurality of subsequent steps implemented by a voice synthesizer with trained artificial intelligence V_E of the enhancement device 100 having an architecture according to a neural source-filter model.

[0163] Said plurality of subsequent steps includes a projection step 74 P, via a first unidirectional recurrent neural network of said speech synthesizer, of the parameters, Sv, in a latent space of parameterizable dimension.

[0164] Said method 70 also includes a generation step 76 of generating G, via a second neural network of said trained speech synthesizer V_E, a signal representative of the harmonics of the speech synthesis associated with said received parameters Sv.

[0165] The projection 74 and harmonic signal generation 76 steps are suitable for implementation in parallel as illustrated in [Fig.5], or in a manner not shown, successively in any order, provided that the output of the projection 74 step and the output of the harmonic signal generation 76 step associated with the same speech synthesis frame associated with said received parameters Sv are processed simultaneously in a subsequent speech synthesis 78 step SYNTH as such via a third unidirectional recurrent neural network, of an encoded voice associated with said speech synthesis associated with said received parameters Sv.

[0166] Optionally, as shown in dotted lines, said inference phase 72 further includes a step 80 of R_T_M reconstruction of the missing frames of said encoded synthesized voice from the frames prior to or subsequent to said missing frames, and / or a step 82 of E_B band extension to reconstruct a signal comprising said encoded synthesized voice with a sampling frequency higher than that of the speech synthesis associated with said received parameters Sv, and / or a step 84 of BIO extraction of the voice biomarkers of the speaker who generated said input sound signal of the existing vocoder that provided said parameters Sv via at least one of said neural networks of said speech synthesizer.

[0167] As an optional complement, when the speech synthesizer with artificial intelligence (i.e. AI speech synthesizer) used has not been previously trained, the method 70 also includes a prior learning phase 88 (i.e. a training phase for each element of the AI ​​speech synthesizer).

[0168] More precisely, learning phase 88 consists of reconstructing the original communication (i.e., without alteration due to parameter encoding) from the encoded (and therefore degraded) parameters. This learning method can be enhanced and enable new use cases.

[0169] A new use case is obtained for example by requesting (i.e. training to) reconstruct, via the aforementioned optional reconstruction module 20, the frames lost during transmission from a context (i.e. the frames before and after the lost frames), which makes it possible to reconstruct part of the lost communication.

[0170] Another new use case is achieved by requesting (i.e., training) the reconstruction of an audio signal with a higher sampling rate than the original encoded signal via the aforementioned optional bandwidth extender module 22, which makes it possible to improve the perceived voice quality and robustness to low-frequency noise. For example, the MELPe 1200 is designed to reconstruct audio signals sampled at 8 kHz, and the AI ​​speech synthesizer of the speech synthesis enhancement device 100 is trained to reconstruct signals sampled at 16 kHz.

[0171] According to an optional aspect of this optional supplement, said learning phase 88 includes an adversarial training step 90 E_A via said adversarial training module previously described.

[0172] A person skilled in the art will understand that the invention is not limited to the embodiments described, nor to the particular examples of the description, the embodiments and variants mentioned above being capable of being combined with each other to generate new embodiments of the invention.

[0173] The present invention thus makes it possible to "replace" the speech synthesis part of an existing vocoder, and to improve the reproduction of the speech communication corresponding to the received transmission.

[0174] From a hardware perspective, the proposed solution is suitable for use with all existing coded equipment (e.g., portable military or civilian radio, fixed station, etc.)

[0175] In addition, optionally, as mentioned above, the present invention is capable of offering new functionalities compared to existing vocoders by allowing, via the reconstruction module 20, a reconstruction (from the English inpainting) of the missing frames of the speech synthesis which would have been produced by an existing vocoder in order in particular to fill micro-interruptions, and / or, via the band extension module 22, a band extension without changing the code of already existing vocoders, which allows interoperability with radio stations already transmitting in the identified code.

[0176] This results in an improvement in speech synthesis quality at a constant bitrate, and the solution according to the present invention is capable of achieving, or even exceeding, the performance of vocoders with twice the bitrate of the existing vocoder that provided it with the input parameters characterizing each frame of the speech synthesis to be produced by said existing vocoder. The bitrate reduction This is also accompanied by new performance gains related to communication. Indeed, at a lower data rate, the range of the transmitting stations is increased. A lower data rate allows, with a constant transmission capacity, for more bandwidth to be allocated to other transmissions.

[0177] The present invention further enables, as experimentally verified, by virtue of the specific structure of the voice synthesizer with artificial intelligence of the enhancement device according to the present invention, to provide an improvement over current vocoding with equivalent input parameters.

[0178] In particular, the source-filter type neural architecture of the speech synthesizer according to the present invention makes it possible to have a "lightweight" (i.e. frugal and low-power) solution in terms of computational footprint and memory compared to current AI solutions, integrable within embedded products, and opens up the possibility of implementing other "functionalities" such as reconstruction by Inpainting, bandwidth extension (this is an addition of information by prediction) or "voice conversion" (i.e., passing from one voice to another voice (a priori associated with existing speakers) or "voice transformation" (i.e., passing from one voice to another voice (which can more generally be that of a virtual speaker)).

[0179] By delegating speech synthesis to AI approaches with recurrent neural networks at each stage of the source-filter model rather than to solutions designed by "experts" (i.e. solutions from modeling), the solution according to the present invention makes it possible in particular to compensate for a potential loss of information linked to the reduction in the number of input parameters, particularly in the context of a low-bandwidth application.

[0180] The choice of a "biased" source-filter architecture (i.e., a neural network architecture that has been "biased" or rather specialized towards speech synthesis by adding a modeling of speech by a sum of harmonics) towards speech synthesis makes it possible to reduce the digital footprint, in terms of memory and computation, of the implemented neural networks.

[0181] Finally, optionally, the practical difficulty of training neural networks in such an open task with few input parameters for a large number of possible temporal signals is advantageously mitigated by the use of adversarial training where two discriminators are trained to recognize the signals generated by the speech synthesizer according to the present invention or jointly said speech synthesizer is trained to fool the discriminators.

[0182] Thus, the device and method for improving speech synthesis are suitable for integration into digital radiocommunication applications, voice over IP (from the English Internet Protocol) VoIP, audio compression, etc.

Claims

Demands

1. Device (100) for improving speech synthesis produced, by means of a predetermined code, by an existing vocoder, said device comprising: - a receiving module (110) for the parameters characterizing each frame of the speech synthesis specific to be produced by said existing vocoder; - at least one speech synthesizer with artificial intelligence (10) having an architecture according to a neural source-filter model, said speech synthesizer with artificial intelligence comprising at least: - a projection module (12) configured to project, via a first unidirectional recurrent neural network, said parameters, or parameters derived from said parameters, into a latent space of parameterizable dimension;- a harmonic signal generator (14) configured to generate, via a second neural network, a signal representative of the harmonics of said speech synthesis associated with said parameters received via said receiver module (110); - a speech synthesis module (16) configured to synthesize, via a third unidirectional recurrent neural network, an encoded voice associated with said human voice from: - the projection of said projected parameters into said latent space of configurable dimension provided by said projection module, and - said signal representative of the harmonics of the speech synthesis associated with said parameters received via said receiver module (110) and provided by said generator; said speech synthesizer with artificial intelligence (10) being previously trained during a supervised learning phase using training speech syntheses produced by means of said predetermined encoding.

2. Device (100) according to claim 1, wherein the receiving module (110) comprises a tool for identifying said predetermined code by analyzing the speech synthesis produced by said existing vocoder, said identification tool being further configured to select, from among a plurality of previously trained and associated artificial intelligence speech synthesizers respectively to distinct codes, said voice synthesizer with artificial intelligence (10) corresponding to said predetermined code identified.

3. Device (100) according to claim 1 or 2, wherein the receiving module (110) includes an interpolation tool configured to linearly interpolate said parameters and provide input to said speech synthesizer with artificial intelligence (10) interpolated parameters compatible with said speech synthesis module (16).

4. Device (100) according to any one of the preceding claims wherein said signal generated by the generator (14) and representative of the harmonics of said speech synthesis associated with said parameters received via said receiving module (110) is a weighted sum of the signal corresponding to the pitch associated with said received parameters and its harmonics, said weighting being learned beforehand by said second neural network.

5. Device (100) according to claim 3, wherein white noise is added to said weighted sum.

6. Device (100) according to any one of the preceding claims, wherein the AI-powered speech synthesizer (10) further comprises an adversarial training module (18) comprising two discriminators configured to be trained simultaneously with said AI-powered speech synthesizer during its supervised learning phase, according to a zero-sum game, said discriminators being trained to differentiate real signals from those likely to be synthesized by said AI-powered speech synthesizer, said AI-powered speech synthesizer being trained to fool said two discriminators.

7. Device (100) according to claim 5, wherein one of said two discriminators is sensitive to low frequency information and the other of said two discriminators is sensitive to high frequency information.

8. Device (100) according to any one of the preceding claims, wherein the speech synthesizer with artificial intelligence (10) further comprises: - a reconstruction module (20) configured to reconstruct missing frames of said synthesized encoded voice from frames prior to or following said missing frames, and / or

9. - a band extension module (22) configured to reconstruct a signal comprising said encoded synthesized voice with a sampling frequency higher than that of said speech synthesis associated with said parameters received via said receiving module (110). A method (70) for improving speech synthesis produced, by means of a predetermined code, by an existing vocoder, said method being implemented by an improvement device (100) according to any one of the preceding claims, said method comprising: - the reception of the parameters characterizing each frame of the speech synthesis specific to be produced by said existing vocoder; and the following steps implemented during an inference phase by a speech synthesizer with artificial intelligence (10) of said improvement device (100), said speech synthesizer with artificial intelligence (10) being previously trained during a supervised learning phase (88) using training parameters characterizing each frame of speech synthesis specific to be produced by means of said predetermined code: - projection (74), via a first unidirectional recurrent neural network of said speech synthesizer with artificial intelligence, of the parameters received via said reception module (110) or parameters derived from said parameters, into a latent space of parameterizable dimension; - generation (76), via a second neural network of said speech synthesizer with artificial intelligence (10), of a signal representative of the harmonics of the speech synthesis associated with said parameters received via said reception module (110); - speech synthesis (78), via a third unidirectional recurrent neural network, of an encoded voice associated with said human voice from: - the projection of said projected parameters into said latent space of parameterizable dimension provided by said projection module, and - said signal representing the harmonics of speech synthesis associated with said parameters received via said receiving module (110) and provided by the speech synthesizer generator with artificial intelligence (10) of said enhancement device (100).

10. A program comprising software instructions which, when executed by a computer, implement a speech synthesis enhancement method according to claim 9.

Images