Speech synthesis method and device, storage medium and electronic equipment

Abstract

The invention relates to a speech synthesis method and device, a storage medium and electronic equipment. Obtaining to-be-processed text data; inputting the to-be-processed text data into a pre-trained low-frequency band acoustic model to obtain a low-frequency spectrum; inputting the low-frequency spectrum into a pre-trained high-frequency decoder to obtain synthetic speech; wherein the pre-training process of the low-frequency-band acoustic model is completed by utilizing to-be-processed text data and pre-acquired low-frequency voice data, and the pre-training process of the high-frequency decoder is completed by utilizing the to-be-processed text data, pre-acquired high-frequency voice data and the pre-trained low-frequency-band acoustic model. The low-frequency voice data is determined through a first preset sampling rate, the high-frequency voice data is determined through a second preset sampling rate, and the first preset sampling rate is smaller than the second preset sampling rate. Compared with the prior art, the method has the advantages that speech synthesis can be performed by using a small amount of high-sampling-rate data, and the speech synthesis quality is ensured.

Description

Technical Field

[0001] The present invention relates to the field of speech synthesis technology, and in particular to a speech synthesis method, device, storage medium and electronic equipment. Background Art

[0002] Text To Speech (TTS) technology converts computer-generated or externally input text information into understandable and fluent Chinese spoken language (or other language speech) output, and belongs to the category of Speech Synthesis.

[0003] In the speech synthesis system, a trained acoustic model is used to map text or language features to acoustic features (such as spectrum), and then a decoder is used to directly convert the spectrum into speech to achieve speech synthesis. Among them, the training of the acoustic model plays an important role in high-quality speech synthesis. During training, low sampling rate data (such as 16k sampling rate data) or high sampling rate data (such as 32kHz, 48kHz sampling rate data) can be used. Low sampling rate data is large in volume and easy to obtain. There are a large number of open source speech data sets, but high sampling rate data is high-quality, clean data is less and difficult to obtain.

[0004] In the prior art, when training acoustic models, frequency bands are generally not distinguished, and data with different sampling rates are trained separately. It is difficult to train with a small amount of high sampling rate data. In addition, low-frequency output is achieved through downsampling, which easily loses effective high-frequency information. Therefore, not distinguishing frequency bands and training data with different sampling rates separately, and achieving low-frequency output through downsampling, will eventually reduce the quality of speech synthesis. Summary of the invention

[0005] The purpose of the present invention is to provide a speech synthesis method, device, storage medium and electronic device in order to overcome the defects of the above-mentioned prior art. Through the scheme of multi-subband fusion and two-stage training, a small amount of high sampling rate data can be used for speech synthesis and the quality of speech synthesis can be guaranteed.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] According to a first aspect of the present invention, a speech synthesis method is provided, comprising the following steps: acquiring text data to be processed; inputting the text data to be processed into a pre-trained low-frequency band acoustic model to obtain a low-frequency spectrum; inputting the low-frequency spectrum into a pre-trained high-frequency decoder to obtain synthesized speech; wherein the pre-training process of the low-frequency band acoustic model is completed using the text data to be processed and pre-acquired low-frequency speech data, and the pre-training process of the high-frequency decoder is completed using the text data to be processed, pre-acquired high-frequency speech data and the pre-trained low-frequency band acoustic model, the low-frequency speech data is determined by a first preset sampling rate, and the high-frequency speech data is determined by a second preset sampling rate, and the first preset sampling rate is less than the second preset sampling rate.

[0008] As a preferred technical solution, the low-frequency band acoustic model is an autoregressive low-frequency band acoustic model.

[0009] As a preferred technical solution, the low-frequency band acoustic model includes a text encoder, an attention module and a low-frequency decoder connected in sequence; the pre-training process of the low-frequency band acoustic model specifically includes: obtaining the alignment data of the text data to be processed and the low-frequency speech data; extracting text features from the text data to be processed, and extracting a first target spectrum from the low-frequency speech data, the first target spectrum being used to supervise the first loss change during the training process; the text features are input into the text encoder to obtain semantic features; the semantic features are input into the attention module to obtain an intermediate representation; the intermediate representation is input into the low-frequency decoder to obtain a low-frequency spectrum; when the first loss change meets a first preset condition, a pre-trained low-frequency band acoustic model is obtained.

[0010] As an optimal technical solution, it specifically includes: extracting a second target spectrum from the high-frequency speech data, the second target spectrum is used to supervise the second loss change in the training process; the text feature is input into the pre-trained low-frequency band acoustic model to obtain the current low-frequency spectrum; the current low-frequency spectrum and duration information are input into the high-frequency decoder to obtain the high-frequency spectrum, and the duration information is obtained using the attention module; merging the low-frequency spectrum and the high-frequency spectrum to obtain complete speech data; when the second loss change meets the second preset condition, a pre-trained high-frequency decoder is obtained.

[0011] As a preferred technical solution, the low-frequency band acoustic model is a non-autoregressive low-frequency band acoustic model.

[0012] As an optimal technical solution, the low-frequency band acoustic model includes a text encoder, a duration predictor and a low-frequency decoder connected in sequence; the pre-training process of the low-frequency band acoustic model specifically includes: obtaining the alignment data of the text data to be processed and the low-frequency speech data; extracting text features from the text data to be processed, and extracting a first target spectrum from the low-frequency speech data, the first target spectrum being used to supervise the first loss change in the training process; the text features are input into the text encoder to obtain semantic features; the semantic features are input into the duration predictor to obtain the duration relationship between the semantic features and the low-frequency speech data; the semantic features and the duration relationship are input into the low-frequency decoder to obtain a low-frequency spectrum; when the first loss change meets the first preset condition, a pre-trained low-frequency band acoustic model is obtained.

[0013] As an optimal technical solution, the pre-training process of the high-frequency decoder specifically includes: extracting a second target spectrum from the high-frequency speech data, the second target spectrum is used to supervise the second loss change in the training process; the text features are input into the pre-trained low-frequency band acoustic model to obtain the current low-frequency spectrum; the current low-frequency spectrum and duration information are input into the high-frequency decoder to obtain the high-frequency spectrum, and the duration information is obtained using the duration predictor; the low-frequency spectrum and the high-frequency spectrum are merged to obtain complete speech data; when the second loss change meets the second preset condition, a pre-trained high-frequency decoder is obtained.

[0014] According to a second aspect of the present invention, there is provided a speech synthesis device, comprising: a data acquisition module for acquiring text data to be processed; a first pre-training module for pre-training a preset low-frequency band acoustic model using the text data to be processed and pre-acquired low-frequency speech data to obtain a pre-trained low-frequency band acoustic model; a low-frequency spectrum acquisition module for inputting the text data to be processed into the pre-trained low-frequency band acoustic model to obtain a low-frequency spectrum; a second pre-training module for pre-training a preset high-frequency decoder using the text data to be processed, pre-acquired high-frequency speech data and the pre-trained low-frequency band acoustic model to obtain a pre-trained high-frequency decoder; and a synthesized speech acquisition module for inputting the low-frequency spectrum into the pre-trained high-frequency decoder to obtain synthesized speech; wherein the low-frequency speech data is determined by a first preset sampling rate, and the high-frequency speech data is determined by a second preset sampling rate, and the first preset sampling rate is less than the second preset sampling rate.

[0015] According to a third aspect of the present invention, there is provided a computer-readable storage medium, the computer-readable storage medium comprising a stored program, wherein the method is executed when the program is run.

[0016] According to a fourth aspect of the present invention, there is provided an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the method according to the computer program.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] 1. The training process of the acoustic model in the present invention is a two-stage training for distinguishing frequency bands. In the first stage of training, a low-frequency band acoustic model is trained using a large amount of low-sampling rate data. After obtaining a pre-trained low-frequency band acoustic model, in the second stage of training, the high-frequency decoder is trained using the pre-trained low-frequency band acoustic model and a small amount of high-sampling rate data. The low-frequency spectrum can be migrated to the high frequency through the trained low-frequency band acoustic model, thereby realizing the joint training of low-sampling rate data and high-sampling rate data, reducing the demand for high-sampling rate data for acoustic model training, and fully capturing important frequency band features while reducing the overall training cost, outputting a high-sampling rate spectrum, and ensuring the quality of speech synthesis;

[0019] 2. When the present invention is used for inference in actual applications, the low-frequency band acoustic model and the high-frequency decoder are separated. The output sampling rate is optional during the entire speech synthesis process. The user can choose to return low-sampling rate audio or high-sampling rate audio. This implementation is accomplished through a neural network, which is not prone to loss of important information and is not a simple downsampling or upsampling. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of the two-stage training process of the acoustic model provided by the present invention;

[0021] Figure 2 A flowchart of a speech synthesis method provided in Embodiment 1 of the present invention;

[0022] Figure 3 A schematic diagram of the pre-training process of the low-frequency band acoustic model in the two-stage training provided in Example 1 of the present invention;

[0023] Figure 4 A schematic diagram of a low-frequency band and a high-frequency band joint training process in a two-stage training provided in Embodiment 1 of the present invention;

[0024] Figure 5 A schematic diagram of the structure of a speech synthesis device provided in Embodiment 1 of the present invention;

[0025] Wherein: 501, data acquisition module; 502, first pre-training module; 503, low-frequency spectrum acquisition module; 504, second pre-training module; 505, synthetic speech acquisition module;

[0026] Figure 6A schematic diagram of the pre-training process of the low-frequency band acoustic model in the two-stage training provided in Example 2 of the present invention;

[0027] Figure 7 This is a schematic diagram of the joint training process of low-frequency band and high-frequency band in the two-stage training provided in Example 2 of the present invention. DETAILED DESCRIPTION

[0028] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0029] Embodiment 1:

[0030] In the prior art, when training acoustic models, frequency bands are generally not distinguished, and data with different sampling rates are trained separately. On the one hand, due to the difference in the contribution of different frequency bands to speech information, indiscriminate training can easily lead to the acoustic model failing to fully capture important frequency band features. On the other hand, low-sampling rate data is large in volume and easy to obtain, and there are a large number of open source speech data sets, but high-sampling rate data is high-quality, clean data is less and difficult to obtain, and indiscriminate training cannot be performed with a small amount of high-sampling rate data. In addition, downsampling is used to achieve low-frequency output, which easily loses effective high-frequency information. Therefore, not distinguishing frequency bands and training data with different sampling rates separately, and achieving low-frequency output through downsampling, will ultimately reduce the quality of speech synthesis.

[0031] To address this problem, the present invention proposes an acoustic model training method of multi-subband fusion and two-stage training. The acoustic model trained by this method is used for speech synthesis, which can save training costs while ensuring the high quality of the synthesized speech, and the sampling rate of speech synthesis is selectable.

[0032] Multi-subband fusion means: first predicting the low-frequency spectrum (eg, 0-16k spectrum), and then using the low-frequency spectrum and duration information to predict the high-frequency spectrum (eg, 16-48k spectrum).

[0033] like Figure 1 As shown, two-stage training means:

[0034] In the first stage of training, the low-frequency band acoustic model is trained using the pre-acquired text data to be processed and a large amount of low-frequency speech data to obtain a pre-trained low-frequency band acoustic model. The pre-trained low-frequency band acoustic model is used to predict the low-frequency spectrum (e.g., 0-16k spectrum). The model can better complete the low-frequency information learning;

[0035] The second stage of training is the joint training of low-frequency band and high-frequency band. Specifically, the high-frequency decoder is trained using the pre-trained low-frequency band acoustic model, the text data to be processed and a small amount of pre-acquired high-frequency speech data. The high-frequency decoder is used to predict the high-frequency spectrum (for example, 16-48k spectrum).

[0036] like Figure 2 As shown, this embodiment provides a speech synthesis method, which obtains the text data to be processed, inputs the text data to be processed into a pre-trained low-frequency band acoustic model to obtain a low-frequency spectrum, and then inputs the low-frequency spectrum into a pre-trained high-frequency decoder to obtain synthesized speech. Among them, the pre-training process of the low-frequency band acoustic model is completed using the text data to be processed and a large amount of pre-acquired low-frequency speech data, and the pre-training process of the high-frequency decoder is completed using the text data to be processed, the pre-acquired high-frequency speech data and the pre-trained low-frequency band acoustic model. The low-frequency speech data is determined by a first preset sampling rate, and the high-frequency speech data is determined by a second preset sampling rate, and the first preset sampling rate is less than the second preset sampling rate. Optionally, the first preset sampling rate is 0-16k, and the second preset sampling rate is 16-48k. In some other embodiments, the setting of the sampling rate can be determined according to actual conditions.

[0037] One of the two-stage training methods is:

[0038] Autoregressive low-frequency band acoustic model + non-autoregressive high-frequency band acoustic model. Specifically, in this training method, the low-frequency band acoustic model is an autoregressive low-frequency band acoustic model. After the pre-trained low-frequency band acoustic model is spliced ​​with the high-frequency decoder, a non-autoregressive high-frequency band acoustic model can be obtained. Compared with using a non-autoregressive low-frequency band acoustic model to predict high frequencies, an autoregressive low-frequency band acoustic model can obtain higher quality.

[0039] Alternatively, if Figure 3 As shown, the low-frequency band acoustic model includes a text encoder, an attention module, and a low-frequency decoder connected in sequence. The process of pre-training the low-frequency band acoustic model using a large amount of low-frequency speech data, i.e., 0-16k speech data, specifically includes:

[0040] Step S301, obtaining alignment data of the text data to be processed and the 0-16k voice data.

[0041] Step S302, extracting text features from the text data to be processed, and extracting a first target spectrum from the 0-16k speech data, the first target spectrum being used to supervise the first loss change during the training process.

[0042] Step S303: Input text features into a text encoder to obtain semantic features. For example, input text tokens and output semantic tokens. Optionally, the text encoder uses a pre-trained model hubert for clustering, or uses a vq model.

[0043] Step S304: input the semantic features into the attention module to obtain an intermediate representation. For example, input a semantic token and output a codec token. Optionally, the attention module uses GPT or llama.

[0044] Step S305, the intermediate representation is input into a low frequency decoder to obtain a 0-16k spectrum, i.e., a predicted low frequency spectrum. For example, the input is a codec token, and the output is a 0-16k spectrum. Optionally, the low frequency decoder uses a codec decoder.

[0045] Step S306, iterative training, when the first loss change meets the first preset condition, a pre-trained low-frequency band acoustic model is obtained. The first loss change can consider the losses of each module of the text encoder, attention module and low-frequency decoder respectively, and each module optimizes the difference between the target output determined based on the target spectrum and the actual output. The first preset condition is to consider the optimization goal to be achieved by each module respectively; the first loss change can also comprehensively consider the overall loss, that is, perform end-to-end optimization, and minimize the overall difference between the final predicted low-frequency spectrum and the target spectrum as the optimization goal. The first preset condition is to comprehensively consider the optimization goal to be achieved from the text feature to the predicted low-frequency spectrum as a whole process.

[0046] After obtaining the pre-trained low-frequency band acoustic model, such as Figure 4 As shown, the pre-training process of the high-frequency decoder may specifically include:

[0047] Step S401, extracting a second target spectrum from the 16-48k speech data, where the second target spectrum is used to supervise the change of the second loss during the training process;

[0048] Step S402: input the text features into the pre-trained low-frequency band acoustic model to obtain the current 0-16k spectrum; optionally, fine-tune the pre-trained low-frequency band acoustic model so that the output low-frequency spectrum is within the range of 0-16k;

[0049] Step S403, the current 0-16k spectrum and duration information are input into the high frequency decoder to obtain the 16-48k spectrum, and the duration information is obtained using the attention module;

[0050] Step S404, merging the 0-16k spectrum and the 16-48k spectrum to obtain complete speech data (0-48k);

[0051] Step S405, iterative training, when the second loss change meets the second preset condition, a pre-trained high frequency decoder is obtained. Optionally, the second loss change comprehensively considers the overall loss, and the second preset condition comprehensively considers the optimization goal to be achieved in the overall process of predicting the high frequency spectrum from the text feature.

[0052] Furthermore, this embodiment also provides a speech synthesis device. Figure 5 One of the structural block diagrams is shown, and the device includes: a data acquisition module 501, which is used to acquire text data to be processed; a first pre-training module 502, which is used to pre-train a preset low-frequency band acoustic model using the text data to be processed and the pre-acquired low-frequency voice data to obtain a pre-trained low-frequency band acoustic model; a low-frequency spectrum acquisition module 503, which is used to input text features into the pre-trained low-frequency band acoustic model to obtain a low-frequency spectrum; a second pre-training module 504, which is used to pre-train a preset high-frequency decoder using the text data to be processed, the pre-acquired high-frequency voice data and the pre-trained low-frequency band acoustic model to obtain a pre-trained high-frequency decoder; and a synthetic speech acquisition module 505, which is used to input the low-frequency spectrum into the pre-trained high-frequency decoder to obtain a synthetic speech; wherein the low-frequency voice data is determined by a first preset sampling rate, and the high-frequency voice data is determined by a second preset sampling rate, and the first preset sampling rate is less than the second preset sampling rate. The specific operation process of each module of the device is the same as the execution process of the aforementioned method, and will not be repeated here.

[0053] Furthermore, this embodiment also provides a computer-readable storage medium, which includes a stored program, wherein the program executes the various steps of the aforementioned method when it is run, and the specific running process is the same as the execution process of the aforementioned method, which will not be repeated here.

[0054] Furthermore, this embodiment also provides an electronic device, including a memory and a processor, characterized in that a computer program is stored in the memory, and the processor is configured to execute each step of the aforementioned method through the computer program. The specific execution process is the same as the execution process of the aforementioned method, and will not be repeated here.

[0055] Embodiment 2:

[0056] This embodiment provides another speech synthesis method, which is basically the same as the method in Embodiment 1, and the difference between the two is that the specific training method used in the two-stage training is different. In this embodiment, another training method of the two-stage training is as follows:

[0057] Non-regressive low-frequency band acoustic model + non-autoregressive high-frequency band acoustic model. Specifically, in this training method, the low-frequency band acoustic model is a non-regressive low-frequency band acoustic model. After the pre-trained low-frequency band acoustic model is spliced ​​with the high-frequency decoder, a non-autoregressive high-frequency band acoustic model can be obtained.

[0058] Alternatively, if Figure 6 As shown, the low-frequency band acoustic model includes a text encoder, a duration predictor, and a low-frequency decoder connected in sequence. The process of pre-training the low-frequency band acoustic model using a large amount of low-frequency speech data, i.e., 0-16k speech data, specifically includes:

[0059] Step 601, obtaining alignment data of the text data to be processed and the 0-16k voice data.

[0060] Step 602, extracting text features from the text data to be processed, and extracting a first target spectrum from the 0-16k speech data, the first target spectrum being used to supervise the first loss change during the training process.

[0061] Step 603: Input the text features into the text encoder to obtain semantic features. For example, input the text token and output the semantic token. Optionally, the text encoder uses the pre-trained model Hubert for clustering, or uses the VQ model.

[0062] Step 604: The semantic features are input into the duration predictor to obtain the duration relationship between the semantic features and the 0-16k speech data. This part is generally used in speech synthesis systems with structures similar to VITs.

[0063] Step 605: The semantic features and the duration relationship are input into a low-frequency decoder to obtain a 0-16k spectrum, ie, a predicted low-frequency spectrum.

[0064] Step 606, iterative training, when the first loss change meets the first preset condition, a pre-trained low-frequency band acoustic model is obtained. The first loss change can consider the loss of the two modules of the text encoder and the low-frequency decoder respectively, and each module optimizes the difference between the target output determined based on the target spectrum and the actual output. The first preset condition is to consider the optimization goal to be achieved by each module respectively; the first loss change can also comprehensively consider the overall loss, that is, perform end-to-end optimization, and minimize the overall difference between the final predicted low-frequency spectrum and the target spectrum as the optimization goal. The first preset condition is to comprehensively consider the optimization goal to be achieved from the overall process of text features-prediction spectrum.

[0065] After obtaining the pre-trained low-frequency band acoustic model, such as Figure 7 As shown, the pre-training process of the high-frequency decoder may specifically include:

[0066] Step S701, extracting a second target spectrum from the 16-48k speech data, where the second target spectrum is used to supervise the change of the second loss during the training process;

[0067] Step S702: input the text feature into the pre-trained low-frequency band acoustic model to obtain the current 0-16k spectrum; optionally, fine-tune the pre-trained low-frequency band acoustic model so that the output low-frequency spectrum is within the range of 0-16k;

[0068] Step S703, the current 0-16k spectrum and duration information are input into a high frequency decoder to obtain a high frequency spectrum, and the duration information is obtained using a duration predictor;

[0069] Step S704, merging the 0-16k spectrum and the 16-48k spectrum to obtain complete speech data (0-48k);

[0070] Step S705, iterative training, when the second loss change meets the second preset condition, a pre-trained high frequency decoder is obtained. Optionally, the second loss change comprehensively considers the overall loss, and the second preset condition comprehensively considers the optimization goal to be achieved in the overall process of predicting the high frequency spectrum from the text feature.

[0071] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A speech synthesis method, characterized in that: The following steps are involved: Get the text data to be processed; Inputting the text data to be processed into a pre-trained low-frequency band acoustic model to obtain a low-frequency spectrum; Inputting the low-frequency spectrum into a pre-trained high-frequency decoder to obtain synthesized speech; Among them, the pre-training process of the low-frequency band acoustic model is completed using the text data to be processed and the pre-acquired low-frequency speech data, and the pre-training process of the high-frequency decoder is completed using the text data to be processed, the pre-acquired high-frequency speech data and the pre-trained low-frequency band acoustic model, the low-frequency speech data is determined by a first preset sampling rate, and the high-frequency speech data is determined by a second preset sampling rate, and the first preset sampling rate is less than the second preset sampling rate.

2. The speech synthesis method according to claim 1, characterized in that: The low-frequency band acoustic model is an autoregressive low-frequency band acoustic model.

3. The speech synthesis method according to claim 2, characterized in that: The low-frequency band acoustic model includes a text encoder, an attention module, and a low-frequency decoder connected in sequence; The pre-training process of the low-frequency band acoustic model specifically includes: Acquire alignment data of the text data to be processed and the low-frequency speech data; Extracting text features from the text data to be processed, and extracting a first target spectrum from the low-frequency speech data, wherein the first target spectrum is used to supervise a first loss change during a training process; The text features are input into the text encoder to obtain semantic features; The semantic features are input into the attention module to obtain an intermediate representation; The intermediate representation is input into the low-frequency decoder to obtain a low-frequency spectrum; When the first loss change meets a first preset condition, a pre-trained low-frequency band acoustic model is obtained.

4. The speech synthesis method according to claim 3, characterized in that: The pre-training process of the high-frequency decoder specifically includes: Extracting a second target spectrum from the high-frequency speech data, wherein the second target spectrum is used to supervise a second loss change during training; The text features are input into the pre-trained low-frequency band acoustic model to obtain a current low-frequency spectrum; The current low-frequency spectrum and duration information are input into the high-frequency decoder to obtain the high-frequency spectrum, and the duration information is obtained by using the attention module; Combining the low-frequency spectrum with the high-frequency spectrum to obtain complete speech data; When the second loss change meets the second preset condition, a pre-trained high-frequency decoder is obtained.

5. The speech synthesis method according to claim 1, characterized in that: The low-frequency band acoustic model is a non-autoregressive low-frequency band acoustic model.

6. The speech synthesis method according to claim 5, characterized in that: The low-frequency band acoustic model includes a text encoder, a duration predictor and a low-frequency decoder connected in sequence; The pre-training process of the low-frequency band acoustic model specifically includes: Acquire alignment data of the text data to be processed and the low-frequency speech data; Extracting text features from the text data to be processed, and extracting a first target spectrum from the low-frequency speech data, wherein the first target spectrum is used to supervise a first loss change during a training process; The text features are input into the text encoder to obtain semantic features; The semantic feature is input into the duration predictor to obtain the duration relationship between the semantic feature and the low-frequency speech data; The semantic feature and the duration relationship are input into the low-frequency decoder to obtain a low-frequency spectrum; When the first loss change meets a first preset condition, a pre-trained low-frequency band acoustic model is obtained.

7. The speech synthesis method according to claim 6, characterized in that: The pre-training process of the high-frequency decoder specifically includes: Extracting a second target spectrum from the high-frequency speech data, wherein the second target spectrum is used to supervise a second loss change during training; The text features are input into the pre-trained low-frequency band acoustic model to obtain a current low-frequency spectrum; The current low-frequency spectrum and duration information are input into the high-frequency decoder to obtain the high-frequency spectrum, and the duration information is obtained by using the duration predictor; Combining the low-frequency spectrum with the high-frequency spectrum to obtain complete speech data; When the second loss change meets the second preset condition, a pre-trained high-frequency decoder is obtained.

8. A speech synthesis device, characterized in that: include: A data acquisition module, used to acquire text data to be processed; A first pre-training module, used to pre-train a preset low-frequency band acoustic model using the text data to be processed and the pre-acquired low-frequency speech data to obtain a pre-trained low-frequency band acoustic model; A low-frequency spectrum acquisition module, used for inputting the text data to be processed into the pre-trained low-frequency band acoustic model to obtain a low-frequency spectrum; A second pre-training module is used to pre-train a preset high-frequency decoder using the text data to be processed, the pre-acquired high-frequency speech data and the pre-trained low-frequency band acoustic model to obtain a pre-trained high-frequency decoder; and A synthesized speech acquisition module, used for inputting the low-frequency spectrum into the pre-trained high-frequency decoder to obtain synthesized speech; The low-frequency voice data is determined by a first preset sampling rate, and the high-frequency voice data is determined by a second preset sampling rate, and the first preset sampling rate is smaller than the second preset sampling rate.

9. A computer-readable storage medium, the computer-readable storage medium comprising a stored program, wherein: When the program is executed, the method described in any one of claims 1 to 7 is executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images