A speech synthesis method and device

Abstract

The present application provides a speech synthesis method and device, which belongs to the field of speech synthesis technology. The embodiment of the present application generates a relative position coding vector having a strong correlation with the absolute position coding vector by performing relative position coding on the basis of the absolute position coding vector, so that the relative position coding is no longer limited by the sequence length of the absolute position coding, thereby being able to meet the needs of ultra-long speech synthesis. At the same time, without changing the original acoustic model structure, a highly scalable and easy-to-converge acoustic model can be obtained through simple fine-tuning, which not only ensures the accuracy of ultra-long speech synthesis, but also does not need to retrain the acoustic model, thereby saving data resources and reducing migration costs.

Description

Technical Field

[0001] The present application relates to the field of speech synthesis technology, and in particular to a speech synthesis method and device. Background Art

[0002] Text to Speech (TTS): The input is text information, and the output is the speech of the target speaker. It is generally divided into three modules: front-end, acoustic model and vocoder. The front-end is used to convert text information into phoneme sequences, the acoustic model is used to convert phoneme sequences into acoustic features, and the vocoder is used to convert acoustic features into the final complete speech.

[0003] In traditional end-to-end speech synthesis solutions, text data with a large difference in length may cause the acoustic model to converge slowly and produce poor synthesis results. Therefore, during the text data preparation stage, the length of the text data is usually limited to short and medium sentences. If the acoustic model trained with short and medium sentences is used to directly synthesize long text, problems such as missing words and ambiguity may occur, affecting the effect of speech synthesis. At the same time, when resources are limited, the amount of very long text data and its corresponding acoustic features is usually small, which limits the training effect of the acoustic model. Summary of the invention

[0004] The present application provides a speech synthesis method and device with strong scalability and easy convergence, which can realize the synthesis of ultra-long speech under limited data resources.

[0005] In order to solve the above problems, this application adopts the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a speech synthesis method, the method comprising:

[0007] Obtain the phoneme sequence corresponding to the text to be processed;

[0008] Inputting the phoneme sequence into a pre-trained acoustic model to obtain corresponding acoustic features;

[0009] Processing the acoustic features to obtain a speech synthesis result;

[0010] The process of inputting the phoneme sequence into a pre-trained acoustic model to obtain corresponding acoustic features comprises the following steps:

[0011] Performing absolute position encoding on the first n phonemes in the phoneme sequence to generate corresponding n absolute position encoding vectors;

[0012] For each remaining phoneme in the phoneme sequence, relative position encoding is performed according to the n absolute position encoding vectors to generate a corresponding relative position encoding vector;

[0013] The encoding result of the phoneme sequence is processed to obtain corresponding acoustic features.

[0014] In an embodiment of the present application, for each remaining phoneme in the phoneme sequence, relative position encoding is performed according to the n absolute position encoding vectors to generate a corresponding relative position encoding vector, including:

[0015] Generate n position coding basis vectors according to the n absolute position coding vectors;

[0016] For each remaining phoneme in the phoneme sequence, two position encoding basis vectors among the n position encoding basis vectors are linearly transformed to generate a corresponding relative position encoding vector.

[0017] In an embodiment of the present application, for each remaining phoneme in the phoneme sequence, two position coding basis vectors among the n position coding basis vectors are linearly transformed to generate a corresponding relative position coding vector, including:

[0018] For the mth phoneme in the phoneme sequence, generate the corresponding relative position encoding vector according to the following formula:

[0019] q m =αb i +(1-α)b j ;

[0020] i=m / / n+1;

[0021] j = m%n;

[0022] Among them: α is a hyperparameter; n is the maximum serial number value of the absolute position encoding vector; m is the serial number value corresponding to the relative position encoding vector whose serial number value is greater than n; i and j are the serial numbers obtained by taking the integer and modulo of m over n respectively; q m is the relative position encoding vector corresponding to the mth phoneme; b i , b j are the i-th position encoding basis vector and the j-th corresponding position encoding basis vector respectively.

[0023] In an embodiment of the present application, n position coding basis vectors are generated according to the n absolute position coding vectors, including:

[0024] Generate the i-th position encoding basis vector according to the following formula:

[0025]

[0026] Among them: α is a hyperparameter; p1 is the absolute position encoding vector corresponding to the first phoneme in the phoneme sequence; p iis the absolute position encoding vector corresponding to the ith phoneme in the phoneme sequence; b i Encode the basis vector for the i-th position.

[0027] In an embodiment of the present application, the method further includes:

[0028] Obtaining a basic acoustic model for absolute position encoding of a phoneme sequence whose length is not greater than n;

[0029] Obtain a phoneme sequence sample with a length greater than n corresponding to the sample text;

[0030] The phoneme sequence samples are input into the basic acoustic model for training to obtain the pre-trained acoustic model.

[0031] In a second aspect, based on the same inventive concept, an embodiment of the present application provides a speech synthesis device, the device comprising:

[0032] A phoneme sequence acquisition module is used to obtain the phoneme sequence corresponding to the text to be processed;

[0033] An acoustic feature acquisition module, used for inputting the phoneme sequence into a pre-trained acoustic model to obtain corresponding acoustic features;

[0034] A synthesis module, used for processing the acoustic features to obtain a speech synthesis result;

[0035] Among them, the acoustic feature acquisition module includes:

[0036] A first generating submodule is used to perform absolute position encoding on the first n phonemes in the phoneme sequence to generate corresponding n absolute position encoding vectors;

[0037] A second generating submodule performs relative position encoding on each remaining phoneme in the phoneme sequence according to the n absolute position encoding vectors to generate a corresponding relative position encoding vector;

[0038] The processing submodule is used to process the encoding result of the phoneme sequence to obtain corresponding acoustic features.

[0039] In one embodiment of the present application, the second generation submodule includes:

[0040] A basis vector generation submodule, used to generate n position coding basis vectors according to the n absolute position coding vectors;

[0041] The transformation submodule is used to perform a linear transformation on two position coding basis vectors among the n position coding basis vectors for each remaining phoneme in the phoneme sequence to generate a corresponding relative position coding vector.

[0042] In an embodiment of the present application, the transformation submodule includes:

[0043] The first transformation submodule is used to generate a corresponding relative position encoding vector for the mth phoneme in the phoneme sequence according to the following formula:

[0044] q m =αb i +(1-α)b j ;

[0045] i=m / / n+1;

[0046] j = m%n;

[0047] Among them: α is a hyperparameter; n is the maximum serial number value of the absolute position encoding vector; m is the serial number value corresponding to the relative position encoding vector whose serial number value is greater than n; i and j are the serial numbers obtained by taking the integer and modulo of m over n respectively; q m is the relative position encoding vector corresponding to the mth phoneme; b i , b j are the i-th position encoding basis vector and the j-th corresponding position encoding basis vector respectively.

[0048] In one embodiment of the present application, the basis vector generation submodule includes:

[0049] The second transformation submodule is used to generate the i-th position coding basis vector according to the following formula:

[0050]

[0051] Among them: α is a hyperparameter; p1 is the absolute position encoding vector corresponding to the first phoneme in the phoneme sequence; p i is the absolute position encoding vector corresponding to the ith phoneme in the phoneme sequence; b i Encode the basis vector for the i-th position.

[0052] In one embodiment of the present application, the device further includes:

[0053] A first acquisition module is used to obtain a basic acoustic model for absolute position encoding of a phoneme sequence whose length is not greater than n;

[0054] The second acquisition module is used to obtain a phoneme sequence sample with a length greater than n corresponding to the sample text;

[0055] The training module is used to input the phoneme sequence samples into the basic acoustic model for training to obtain the pre-trained acoustic model.

[0056] Compared with the prior art, this application has the following advantages:

[0057] A speech synthesis method and device provided in an embodiment of the present application performs relative position encoding on the basis of an absolute position encoding vector to generate a relative position encoding vector that is strongly correlated with the absolute position encoding vector, so that the relative position encoding is no longer limited by the sequence length of the absolute position encoding, thereby being able to meet the needs of ultra-long speech synthesis. At the same time, without changing the original acoustic model structure, a highly scalable and easy-to-converge acoustic model can be obtained through simple fine-tuning, which not only ensures the accuracy of ultra-long speech synthesis, but also does not need to retrain the acoustic model, thereby saving data resources and reducing migration costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 is a flowchart of a speech synthesis method in an embodiment of the present application;

[0060] Figure 2 is a flowchart of the specific steps of step S102 in the embodiment of the present application;

[0061] Figure 3 is a functional module diagram of a speech synthesis device in an embodiment of the present application;

[0062] Figure 4 It is a schematic diagram of functional submodules of an acoustic feature acquisition module in a speech synthesis device in an embodiment of the present application.

[0063] Figure numerals: 200 - speech synthesis device; 201 - phoneme sequence acquisition module; 202 - acoustic feature acquisition module; 203 - synthesis module; 2021 - first generation submodule; 2022 - second generation submodule; 2023 - processing submodule. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] It should be noted that in the present embodiment, in a conventional end-to-end speech synthesis solution, the position coding part usually adopts absolute position coding, and its formula is:

[0066]

[0067]

[0068] Where: p is the position number of the absolute position code, d pos is the dimension of the absolute position encoding vector, PE 2i (p) is the value of the even-numbered elements of the absolute position encoding vector, PE 2i+1 (p) is the value of the odd-numbered elements of the absolute position encoding vector.

[0069] From formula (1) and formula (2), we can see that the absolute position encoding method is actually mapping the position with serial number p to a d pos The position vector of the dimension changes with the change of the sequence number p, and the value of the absolute position coding vector also changes accordingly. The position sequence number p is the only factor that affects the absolute position coding vector. With absolute position coding, each absolute position coding vector is trained by the acoustic model itself, and the previous and next positions are not associated, so it is impossible to infer the absolute position coding vectors of the remaining positions. Therefore, there is a limit on the sequence length of absolute position coding. It is precisely based on this property of absolute position coding that if a medium-short length speech training set is used in the acoustic model training stage, then when synthesizing long speech, because the acoustic model has not been trained with a sufficient amount of long speech and is limited by the sequence length of absolute position coding, the acoustic model cannot effectively recognize and convert the data in the longer speech that exceeds the sequence length of the absolute position coding, resulting in the final synthesized speech having problems such as missing words and ambiguity. If the long speech is directly added to the speech training set, the model will be difficult to converge due to the large difference in data length, and the synthesis effect will be poor.

[0070] In response to the defects existing in the existing acoustic models when using absolute position coding, the embodiment of the present application proposes a speech synthesis method, which aims to obtain an acoustic model with strong scalability and easy convergence through simple fine-tuning without changing the original acoustic model structure. The acoustic model is no longer limited by the sequence length of the absolute position coding, and can achieve accurate synthesis of ultra-long speech with limited data resources.

[0071] Reference Figure 1 , showing that an embodiment of the present application proposes a speech synthesis method, which may include the following steps:

[0072] Step S101, obtaining a phoneme sequence corresponding to the text to be processed.

[0073] In this embodiment, a corresponding pinyin sequence is obtained according to the text to be processed, and corresponding phonemes are obtained according to the pinyin sequence, wherein the phonemes include initial consonants and finals. By encoding and sorting the phonemes, the phoneme sequence corresponding to the text to be processed can be obtained.

[0074] Step S102: input the phoneme sequence into a pre-trained acoustic model to obtain corresponding acoustic features.

[0075] In this embodiment, as a preferred solution, the pre-trained acoustic model adopts the FastSpeech model, wherein the Fast Speech model includes an encoder module, a duration prediction module and a decoder module.

[0076] Specifically, the phoneme sequence is first self-encoded through the encoder module to obtain the encoder output. The duration prediction module receives the encoder output, predicts the duration corresponding to the phoneme sequence, and expands it according to the duration of each phoneme to obtain the phoneme matrix. The decoder module receives the phoneme matrix for self-decoding to obtain the corresponding acoustic features.

[0077] It should be noted that in the present embodiment, in the traditional end-to-end speech synthesis scheme, the encoder module and the decoder module usually adopt the absolute position coding scheme. The present embodiment aims to optimize the absolute position coding scheme of the encoder module and the decoder module so that the encoder module and the decoder module are no longer limited by the sequence length of the absolute position coding when processing ultra-long speech, thereby meeting the synthesis requirements of ultra-long speech.

[0078] Step S103: Process the acoustic features to obtain a speech synthesis result.

[0079] In this embodiment, the acoustic features may be sent to a vocoder for synthesis to obtain a speech signal.

[0080] Specifically, Figure 2 The specific step flow chart of step S102 in the embodiment of the present application, step S102 may include the following steps:

[0081] Step S102-1, performing absolute position encoding on the first n phonemes in the phoneme sequence to generate corresponding n absolute position encoding vectors.

[0082] In this embodiment, the absolute position encoding vectors are denoted as p1, p2, p3...p n , where n is the maximum serial number value of the absolute position code, that is, in this embodiment, for the phonemes with serial numbers less than or equal to n in the phoneme sequence, the absolute position code is used for encoding to obtain the relative position code vectors q1, q2, q3..q n ,in,

[0083] q1=p1,q2=p2,....,q n =p n (3);

[0084] In this embodiment, the acoustic model does not need to be retrained, but the pre-trained acoustic model can continue to be used, and the newly generated corresponding n absolute position encoding vectors are consistent with the old absolute position encoding vectors and are therefore compatible with the pre-trained acoustic model.

[0085] Step S102-2: For each remaining phoneme in the phoneme sequence, relative position encoding is performed according to the n absolute position encoding vectors to generate a corresponding relative position encoding vector.

[0086] It should be noted that in this embodiment, each remaining phoneme in the phoneme sequence is a phoneme with a serial number value greater than n in the phoneme sequence. For these phonemes, relative position coding is performed based on absolute position coding to generate a relative position coding vector q n The relative position encoding vector q with strong correlation m , where m>n, so that the acoustic model can successfully identify the relative position encoding vector q corresponding to this part of the phonemes m , to avoid problems such as missing words and ambiguity in the final synthesized speech.

[0087] Step S102-3: Process the encoding result of the phoneme sequence to obtain corresponding acoustic features.

[0088] In a feasible implementation manner, the specific steps of step S102-2 may further include:

[0089] Step S102-2-1, generating n position coding basis vectors according to the n absolute position coding vectors.

[0090] In this embodiment, for the obtained n absolute position encoding vectors p1, p2, p3...p n , each absolute position encoding vector is decomposed into n position encoding basis vectors b1, b2, b3...bn, that is, on the basis of the original n encoding positions, n 2 Encoded position.

[0091] Step S102-2-2, for each remaining phoneme in the phoneme sequence, performing linear transformation on two position coding basis vectors among the n position coding basis vectors to generate a corresponding relative position coding vector, the specific step may also include:

[0092] In this implementation, for the mth phoneme in the phoneme sequence, the corresponding relative position encoding vector is generated according to the following formula:

[0093] q m =αb i +(1-α)b j (4);

[0094] i=m / / n+1 (5);

[0095] j = m%n (6);

[0096] Among them: α is a hyperparameter; n is the maximum serial number value of the absolute position encoding vector; m is the serial number value corresponding to the relative position encoding vector whose serial number value is greater than n; i and j are the serial numbers obtained by taking the integer and modulo of m over n respectively; q m is the relative position encoding vector corresponding to the mth phoneme; b i , b j are the i-th position encoding basis vector and the j-th corresponding position encoding basis vector respectively.

[0097] According to formula (3), formula (4), formula (5) and formula (6), we can deduce the following by recursive induction:

[0098]

[0099] Among them: α is a hyperparameter; p1 is the absolute position encoding vector corresponding to the first phoneme in the phoneme sequence; p i is the absolute position encoding vector corresponding to the ith phoneme in the phoneme sequence; b i Encode the basis vector for the i-th position.

[0100] It can be seen from formula (7) that the position encoding basis vector b i Still composed of absolute position encoding vectors p1, p2, p3...p n But now it can express n 2 The code of the position, and this n 2 The first n positions of the encoding positions are compatible with the pre-trained acoustic model. Therefore, when generating n position encoding basis vectors based on the n absolute position encoding vectors, the i-th position encoding basis vector can be generated according to formula (7).

[0101] In a feasible implementation, the maximum serial number value n of the absolute position encoding vector of the acoustic model is 512, that is, the absolute position encoding vector is recorded as p1, p2, p3...p 512When using this acoustic model to encode a very long speech with a maximum sequence number of 1024, the phoneme sequence is first split into absolute position encoding vectors q1, q2, q3..q with sequence numbers less than or equal to 512. 512 and the relative position encoding vector q whose serial number is greater than 512 513 ,q 514 ,q 515 ..q 1024 ;

[0102] For q1,q2,q3..q 512 The phoneme sequence uses the corresponding absolute position encoding vector, that is, q1 = p1, q2 = p2, ..., q 512 =p 512 ;

[0103] For q 513 ,q 514 ,q 515 ..q 1024 The phoneme sequence is encoded based on the absolute position q1,q2,q3..q 512 , relative position coding is performed to generate a relative position coding vector that is strongly associated with the absolute position coding. Specifically, the relative position coding vector q 600 For example, according to formula (5) and formula (6), where m is 600 and n is 512, i is 2 and j is 98 respectively. Then according to formula (4), we can get:

[0104] q 600 =αb2+(1-α)b 98 (8);

[0105] According to formula (7), we can get:

[0106]

[0107]

[0108] According to formula (8), formula (9) and formula (10), according to the known absolute position codes p1, p2, p 98 Then we can get the relative position encoding vector q 600 .

[0109] Therefore, the phoneme sequence can be split into absolute position encoding vectors q1, q2, q3..q with sequence values ​​less than or equal to 512 512 and the relative position encoding vector q whose serial number is greater than 512 513 ,q 514 ,q 515 ..q 1024 .

[0110] That is, in this embodiment, in the scenario of synthesizing ultra-long speech, that is, when the maximum sequence number value of the phoneme sequence length is greater than 512, the absolute position encoding vectors q1, q2, q3..q 512 , the model does not need to be trained from scratch, but can continue to use the pre-trained acoustic model. For the relative position encoding vector q of the phoneme sequence whose serial number value is greater than 512 513 ,q 514 ,q 515 ..q 1024 , each relative position encoding vector is still obtained through linear transformation of the previous absolute position encoding vector. Therefore, the new relative position encoding vector has a strong correlation with the old absolute position encoding vector, so the acoustic model can accurately identify the new relative position encoding vector, that is, through simple fine-tuning, a new acoustic model with strong scalability and easy convergence can be obtained.

[0111] In a feasible implementation, the speech synthesis method may further include the following steps:

[0112] Step S104: obtaining a basic acoustic model for absolute position encoding of a phoneme sequence whose length is not greater than n.

[0113] Step S105, obtaining a phoneme sequence sample with a length greater than n corresponding to the sample text.

[0114] Step S106: input the phoneme sequence sample into the basic acoustic model for training to obtain the pre-trained acoustic model.

[0115] In this embodiment, when performing initial training of the acoustic model, a medium or short-length speech training set corresponding to a phoneme sequence with a length not greater than n can be selected and absolute position encoding can be performed to obtain a basic acoustic model. On this basis, only a small amount of sample text corresponding to a phoneme sequence sample with a length greater than n is needed for retraining to obtain a pre-trained acoustic model.

[0116] In this embodiment, the phoneme sequence is split into two parts according to the maximum sequence number value n of the absolute position coding vector and encoded separately. For the phonemes in the phoneme sequence whose sequence number values ​​are greater than n, relative position coding is performed on the basis of the absolute position coding vector to generate a relative position coding vector that has a strong correlation with the absolute position coding vector. Without changing the original acoustic model structure, a highly scalable and easy-to-converge acoustic model can be obtained through simple fine-tuning. The acoustic model is no longer limited to the sequence length of the absolute position coding, and can achieve accurate synthesis of ultra-long speech under limited data resources.

[0117] Based on the same inventive concept, Figure 3 , Figure 3 : is a functional module diagram of a speech synthesis device 200 in an embodiment of the present application, and the speech synthesis device 200 at least includes:

[0118] A phoneme sequence obtaining module 201 is used to obtain a phoneme sequence corresponding to the text to be processed;

[0119] An acoustic feature acquisition module 202, used for inputting the phoneme sequence into a pre-trained acoustic model to obtain corresponding acoustic features;

[0120] A synthesis module 203 is used to process the acoustic features to obtain a speech synthesis result;

[0121] Among them, refer to Figure 4 , the acoustic feature acquisition module 202 at least includes:

[0122] The first generating submodule 2021 is used to perform absolute position encoding on the first n phonemes in the phoneme sequence to generate corresponding n absolute position encoding vectors;

[0123] The second generating submodule 2022 performs relative position encoding on each remaining phoneme in the phoneme sequence according to the n absolute position encoding vectors to generate a corresponding relative position encoding vector;

[0124] The processing submodule 2023 is used to process the encoding result of the phoneme sequence to obtain corresponding acoustic features.

[0125] In one embodiment of the present application, the second generation submodule 2022 includes:

[0126] A basis vector generation submodule, used to generate n position coding basis vectors according to the n absolute position coding vectors;

[0127] The transformation submodule is used to perform a linear transformation on two position coding basis vectors among the n position coding basis vectors for each remaining phoneme in the phoneme sequence to generate a corresponding relative position coding vector.

[0128] In an embodiment of the present application, the transformation submodule includes:

[0129] The first transformation submodule is used to generate a corresponding relative position encoding vector for the mth phoneme in the phoneme sequence according to the following formula:

[0130] q m =αb i +(1-α)b j ;

[0131] i=m / / n+1;

[0132] j = m%n;

[0133] Among them: α is a hyperparameter; n is the maximum serial number value of the absolute position encoding vector; m is the serial number value corresponding to the relative position encoding vector whose serial number value is greater than n; i and j are the serial numbers obtained by taking the integer and modulo of m over n respectively; q m is the relative position encoding vector corresponding to the mth phoneme; b i , b j are the i-th position encoding basis vector and the j-th corresponding position encoding basis vector respectively.

[0134] In one embodiment of the present application, the basis vector generation submodule includes:

[0135] The second transformation submodule is used to generate the i-th position coding basis vector according to the following formula:

[0136]

[0137] Among them: α is a hyperparameter; p1 is the absolute position encoding vector corresponding to the first phoneme in the phoneme sequence; p i is the absolute position encoding vector corresponding to the ith phoneme in the phoneme sequence; b i Encode the basis vector for the i-th position.

[0138] In one embodiment of the present application, the device further includes:

[0139] A first acquisition module is used to obtain a basic acoustic model for absolute position encoding of a phoneme sequence whose length is not greater than n;

[0140] The second acquisition module is used to obtain a phoneme sequence sample with a length greater than n corresponding to the sample text;

[0141] The training module is used to input the phoneme sequence samples into the basic acoustic model for training to obtain the pre-trained acoustic model.

[0142] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0143] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0144] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0146] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0147] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0148] The above is a detailed introduction to a speech synthesis method and device provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A speech synthesis method, characterized in that: The method comprises: Obtain the phoneme sequence corresponding to the text to be processed; Inputting the phoneme sequence into a pre-trained acoustic model to obtain corresponding acoustic features; Processing the acoustic features to obtain a speech synthesis result; The process of inputting the phoneme sequence into a pre-trained acoustic model to obtain corresponding acoustic features comprises the following steps: Performing absolute position encoding on the first n phonemes in the phoneme sequence to generate corresponding n absolute position encoding vectors; For each remaining phoneme in the phoneme sequence, relative position encoding is performed according to the n absolute position encoding vectors to generate a corresponding relative position encoding vector; The encoding result of the phoneme sequence is processed to obtain corresponding acoustic features.

2. The method according to claim 1, characterized in that For each remaining phoneme in the phoneme sequence, relative position encoding is performed according to the n absolute position encoding vectors to generate a corresponding relative position encoding vector, including: Generate n position coding basis vectors according to the n absolute position coding vectors; For each remaining phoneme in the phoneme sequence, two position encoding basis vectors among the n position encoding basis vectors are linearly transformed to generate a corresponding relative position encoding vector.

3. The method according to claim 2, characterized in that For each remaining phoneme in the phoneme sequence, two position coding basis vectors among the n position coding basis vectors are linearly transformed to generate a corresponding relative position coding vector, including: For the mth phoneme in the phoneme sequence, generate the corresponding relative position encoding vector according to the following formula: q m =αb i +(1-α)b j ; i=m / / n+1; j = m%n; Among them: α is a hyperparameter; n is the maximum serial number value of the absolute position encoding vector; m is the serial number value corresponding to the relative position encoding vector whose serial number value is greater than n; i and j are the serial numbers obtained by taking the integer and modulo of m over n respectively; q m is the relative position encoding vector corresponding to the mth phoneme; b i , b j are the i-th position encoding basis vector and the j-th corresponding position encoding basis vector respectively.

4. The method according to claim 3, characterized in that According to the n absolute position encoding vectors, n position encoding basis vectors are generated, including: Generate the i-th position encoding basis vector according to the following formula: Among them: α is a hyperparameter; p1 is the absolute position encoding vector corresponding to the first phoneme in the phoneme sequence; p i is the absolute position encoding vector corresponding to the ith phoneme in the phoneme sequence; b i Encode the basis vector for the i-th position.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Obtaining a basic acoustic model for absolute position encoding of a phoneme sequence whose length is not greater than n; Obtain a phoneme sequence sample with a length greater than n corresponding to the sample text; The phoneme sequence samples are input into the basic acoustic model for training to obtain the pre-trained acoustic model.

6. A speech synthesis device, characterized in that: The device comprises: A phoneme sequence acquisition module is used to obtain the phoneme sequence corresponding to the text to be processed; An acoustic feature acquisition module, used for inputting the phoneme sequence into a pre-trained acoustic model to obtain corresponding acoustic features; A synthesis module, used for processing the acoustic features to obtain a speech synthesis result; Among them, the acoustic feature acquisition module includes: A first generating submodule is used to perform absolute position encoding on the first n phonemes in the phoneme sequence to generate corresponding n absolute position encoding vectors; A second generating submodule performs relative position encoding on each remaining phoneme in the phoneme sequence according to the n absolute position encoding vectors to generate a corresponding relative position encoding vector; The processing submodule is used to process the encoding result of the phoneme sequence to obtain corresponding acoustic features.

7. The device according to claim 6, characterized in that The second generation submodule includes: A basis vector generation submodule, used to generate n position coding basis vectors according to the n absolute position coding vectors; The transformation submodule is used to perform a linear transformation on two position coding basis vectors among the n position coding basis vectors for each remaining phoneme in the phoneme sequence to generate a corresponding relative position coding vector.

8. The device according to claim 7, characterized in that The transformation submodule comprises: The first transformation submodule is used to generate a corresponding relative position encoding vector for the mth phoneme in the phoneme sequence according to the following formula: q m =αb i +(1-α)b j ; i=m / / n+1; j = m%n; Among them: α is a hyperparameter; n is the maximum serial number value of the absolute position encoding vector; m is the serial number value corresponding to the relative position encoding vector whose serial number value is greater than n; i and j are the serial numbers obtained by taking the integer and modulo of m over n respectively; q m is the relative position encoding vector corresponding to the mth phoneme; b i , b j are the i-th position encoding basis vector and the j-th corresponding position encoding basis vector respectively.

9. The device according to claim 7, characterized in that The basis vector generation submodule comprises: The second transformation submodule is used to generate the i-th position coding basis vector according to the following formula: Among them: α is a hyperparameter; p1 is the absolute position encoding vector corresponding to the first phoneme in the phoneme sequence; p i is the absolute position encoding vector corresponding to the ith phoneme in the phoneme sequence; b i Encode the basis vector for the i-th position.

10. The device according to any one of claims 6 to 9, characterized in that: The device also includes: A first acquisition module is used to obtain a basic acoustic model for absolute position encoding of a phoneme sequence whose length is not greater than n; The second acquisition module is used to obtain a phoneme sequence sample with a length greater than n corresponding to the sample text; The training module is used to input the phoneme sequence samples into the basic acoustic model for training to obtain the pre-trained acoustic model.

Images