Method and device for speech synthesis

Abstract

The present disclosure provides a method and a device for speech synthesis, wherein the method includes obtaining a target text in a speech synthesis request; performing preset processing on the target text to obtain to-be-synthesized input information, wherein the to-be- synthesized input information includes information obtained by splicing the target text and historical information, or encoded combination information obtained by parsing text encoding of the target text and combining fixed speech encoding and / or model output speech encoding; and inputting the to-be-synthesized input information into a pre-trained speech synthesis model, so that the speech synthesis model outputs target synthesized speech corresponding to the target text in combination with text-associated content of the to-be-synthesized input information. In this way, it ensures that the input speech synthesis model information contains text-associated content, and also reduces the delay between the target text input and the target synthesized speech output, which helps improve the accuracy and efficiency of speech synthesis.

Description

METHOD AND DEVICE FOR SPEECH SYNTHESISTECHNICAL FIELD

[0001] The present disclosure relates to the technology field of speech processing, and particularly to a method and a device for speech synthesis.BACKGROUND ART

[0002] With the development of science and technology, speech synthesis technology has received increasing attention. By utilizing speech synthesis technology and speech recognition technology, human-computer interaction can be realized, or text can be converted into speech. Specifically, the conversion between text and synthesized speech can be processed by a speech synthesis model.

[0003] In the prior art, speech synthesis models generally perform speech synthesis unit by unit of a sentence or a paragraph. Specifically, by inputting a speech text, the corresponding speech information is obtained. This processing approach completely isolates the contextual relationship between the text contents, making the speech synthesis overly rigid. The synthesized speech lacks emotional expression from the context, resulting in a poor speech synthesis experience.

[0004] Furthermore, in order to address the issue of discontinuous contextual information between output speech, speech synthesis can be initiated after receiving the entire text content. This preserves the contextual relationship between the text contents and inputs it into the speech synthesis model. However, this approach increases the waiting time for receiving the synthesized speech, and when the model synthesizes subsequent speech, the time from when the model receives the contextual information is often quite far away. Therefore, the previously obtained contextual information has limited effect on the synthesized later speech, and the synthesized speech still lacks a complete and rich emotional expression from the context. Due to the fact that each word during speech synthesis has a certain pronunciation duration and there is a time gap between different words, the time for synthesizing the speech is often longer than the time for receiving the text, which leads to low speech synthesis efficiency.SUMMARY

[0005] In view of this, the objective of the present disclosure is to provide a method and a device for speech synthesis that, after obtaining the target text in the speech synthesis request, immediately begins preset processing of the target text. During processing, historical informationis spliced to obtain a combination processing that includes text-associated content or encoding level. This ensures that the input speech synthesis model information contains text-associated content, and also reduces the delay between the target text input and the target synthesized speech output, which helps improve the accuracy and efficiency of speech synthesis.

[0006] In a first aspect, the embodiments of the present disclosure provide a method for speech synthesis, wherein the method for speech synthesis includes:

[0007] obtaining a target text in a speech synthesis request;

[0008] performing preset processing on the target text to obtain to-be-synthesized input information, wherein the to-be-synthesized input information includes information obtained by splicing the target text and historical information, or encoded combination information obtained by parsing text encoding of the target text and combining fixed speech encoding and / or model output speech encoding; and

[0009] inputting the to-be-synthesized input information into a pre-trained speech synthesis model, so that the speech synthesis model outputs target synthesized speech corresponding to the target text in combination with text-associated content of the to-be-synthesized input information.

[0010] In a second aspect, the embodiments of the present disclosure provide a device for speech synthesis, wherein the device for speech synthesis includes:

[0011] a text acquisition module, configured for obtaining a target text in a speech synthesis request;

[0012] an input information synthesis module, configured for performing preset processing on the target text to obtain to-be-synthesized input information, wherein the to-be-synthesized input information includes information obtained by splicing the target text and historical information, or encoded combination information obtained by parsing text encoding of the target text and combining fixed speech encoding and / or model output speech encoding; and

[0013] a synthesized speech output module, configured for inputting the to-be-synthesized input information into a pre-trained speech synthesis model, so that the speech synthesis model outputs target synthesized speech corresponding to the target text in combination with text-associated content of the to-be-synthesized input information.

[0014] In a third aspect, the embodiments of the present disclosure provide further provides an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device operates, the processor communicates with the storage medium via the bus, and theprocessor executes the machine-readable instructions to perform the steps of the method for speech synthesis as described in the first aspect.

[0015] In a fourth aspect, the embodiments of the present disclosure provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by the processor, it performs the steps of the method for speech synthesis as described in the first aspect.

[0016] A method and a device for speech synthesis, and an electronic device are provided in the embodiments of the present disclosure, wherein the method includes obtaining a target text in a speech synthesis request; performing preset processing on the target text to obtain to-be-synthesized input information, wherein the to-be-synthesized input information includes information obtained by splicing the target text and historical information, or encoded combination information obtained by parsing text encoding of the target text and combining fixed speech encoding and / or model output speech encoding; and inputting the to-be-synthesized input information into a pre-trained speech synthesis model, so that the speech synthesis model outputs target synthesized speech corresponding to the target text in combination with text-associated content of the to-be-synthesized input information. In this way, after obtaining the target text in the speech synthesis request, the preset processing of the target text immediately begins. During processing, historical information is spliced to obtain a combination processing that includes text-associated content or encoding level. This ensures that the input speech synthesis model information contains text-associated content, and also reduces the delay between the target text input and the target synthesized speech output, which helps improve the accuracy and efficiency of speech synthesis.|0017| In order to make the above objectives, features, and advantages of the present disclosure more obvious and easier to understand, the following better embodiments, together with the attached drawings, are described in detail as follows.BRIEF DESCRIPTION OF DRAWINGS

[0018] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings used in the embodiments. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore they should not be regarded as a limitation on the scope. Those ordinary skilled in the art can also obtain other related drawings based on these drawings without inventive effort.

[0019] FIG. 1 is a schematic diagram of a flow chart of a target synthesized speech output in the prior art;

[0020] FIG. 2 is another schematic diagram of a flow chart of a target synthesized speech output in the prior art;

[0021] FIG. 3 is a schematic diagram of a speech synthesis model training process in the prior art;

[0022] FIG. 4 is another schematic diagram of a speech synthesis model training process in the prior art;

[0023] FIG. 5 is a flowchart of a method for speech synthesis provided by the embodiment of the present disclosure;

[0024] FIG. 6 is a schematic diagram of a target synthesized speech output process provided in the embodiment of the present disclosure;

[0025] FIG. 7 is another schematic diagram of a target synthesized speech output process provided in the embodiment of the present disclosure;

[0026] FIG. 8 is a schematic diagram of a speech synthesis model training process provided in the embodiment of the present disclosure;

[0027] FIG. 9 is another schematic diagram of a speech synthesis model training process provided in the embodiment of the present disclosure;

[0028] FIG. 10 is a structural schematic diagram of a device for speech synthesis provided in the embodiment of the present disclosure; and

[0029] FIG. 11 is a schematic diagram of the structure of an electronic equipment provided by the embodiment of the present disclosure.DETAILED DESCRIPTION OF EMBODIMENTS

[0030] In order to make the objective, technical solutions, and advantages of the embodiments of the present disclosure clearer, the following description will provide a clear and comprehensive explanation of the technical solutions in the embodiments of the present disclosure with reference to the drawings in the embodiments of the present disclosure. Clearly, the described embodiments are part of the embodiments of the present disclosure and not the entire embodiments. The components of embodiments of the present disclosure which are generally described and illustrated in the drawings herein can be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for which protection is claimed but merely represents selected embodiments of thepresent disclosure. Based on the embodiments in the present disclosure, every other embodiment obtained of those of skill in the art of without making inventive efforts is within the scope of protection of the present disclosure.

[0031] First, the application scenarios to which the present disclosure can be applied are introduced. The present disclosure can be applied in the technology field of speech processing.

[0032] With the development of science and technology, speech synthesis technology has received increasing attention. By utilizing the speech synthesis technology and speech recognition technology, human-computer interaction can be realized, or text can be converted into speech. Specifically, the conversion between text and synthesized speech can be processed by a speech synthesis model.

[0033] With reference to FIG. 1 to FIG. 4, the existing speech synthesis models generally synthesize speech on the basis of a sentence or a paragraph, where speech 1, speech 2, and speech i are fitting data, and (speech 1), (speech 2), (speech i) are the predicted output generated by the model based on the input text content, input text 1, text 2, text i; and input fitting data content, speech 1, speech 2, speech i. The loss value is calculated based on (speech 1), (speech 2), (speech i), and speech 1, speech 2, speech i, and the model parameters are optimized to complete the model training.

[0034] In the model training process, the generation of the predicted output (speech 1), (speech 2), and (speech i) is as follows Based on all the input data text 1, text 2, text i, the first token of the predicted output (speech 1), (speech 2), (speech i) is predicted, and then, based on all the input data text 1, text 2, text i, and the first token of the fitting data speech 1, speech 2, speech i, the second token of the predicted output (speech 1), (speech 2), (speech i) is predicted. This process continues until all tokens of the predicted output (speech 1), (speech 2), (speech i) generated by all tokens of the input text 1, text 2, text i and the fitting data speech 1, speech 2, speech i are obtained.

[0035] In the model inference process, the generation of output content speech 1, speech 2, and speech i is as follow s. Based on all the input data text 1, text 2, text i, the first token of the output speech 1, speech 2, speech i is output. Then, based on all the input data text 1, text 2, text i, and the first token of the output speech 1, speech 2, speech i, the second token of output speech 1, speech 2, speech i is output. This process continues until all tokens of the output content speech 1, speech 2, and speech i are generated based on the input text 1, text 2, and text i. The input data text 1, text 2, text i, speech 1, speech 2, speech i, and (speech 1), (speech 2), (speech i) are all input to and output from the model in the form of tokens (encoding vectors).

[0036] In the prior art 1, since speech synthesis does not require waiting for the full text content to be received before starting the synthesis process, it can reduce the waiting time for receiving synthesized speech to some extent. However, the prior art 1 completely disconnects the contextual relationship between the text content, making the speech synthesis mechanical and lacking emotional expression from the context of the synthesized speech. For example, if the text is passionate in one part, the tone will likely remain passionate in the subsequent part. This continuity is absent in the prior art 1, leading to a poor speech synthesis experience.

[0037] In prior art 2, since speech synthesis in prior art 2 requires waiting for the full text content to be received before starting the synthesis process, this increases the waiting time for receiving synthesized speech. Although the prior art 2 can retain the contextual relationship between the text content and input it into the large model, the time between receiving the contextual information and synthesizing the subsequent speech is often long. Therefore, the previously obtained contextual information has limited impact on the synthesized later speech, and the synthesized speech still lacks a complete and rich emotional expression from the context. Particularly, when the input text is lengthy, due to the fact that each word during speech synthesis has a certain pronunciation duration and there is a time gap between different words, the time for synthesizing the speech is often longer than the time for receiving the text. For example, it takes 4 s to receive 100 characters of text, but synthesizing the speech for those 100 characters will take 30 seconds. The longer the waiting time, the poorer the emotional expression in the context, and the lower the efficiency of speech synthesis.

[0038] Based on this, the embodiment of the present disclosure provides a method for speech synthesis to improve both the accuracy and efficiency of speech synthesis.[00391 Referring to FIG. 5, a method for speech synthesis is provided in the embodiments of the present disclosure, including the following steps.

[0040] S501 : obtaining a target text in a speech synthesis request.

[0041] S502: performing preset processing on the target text to obtain to-be-synthesized input information, wherein the to-be-synthesized input information includes information obtained by splicing the target text and historical information, or encoded combination information obtained by parsing text encoding of the target text and combining fixed speech encoding and / or model output speech encoding.

[0042] S503: inputting the to-be-synthesized input information into a pre-trained speech synthesis model, so that the speech synthesis model outputs target synthesized speechcorresponding to the target text in combination with text-associated content of the to-be-synthesized input information.

[0043] In step S501, streaming text content refers to the text content being displayed sequentially, word by word, or sentence by sentence, typically used in scenarios such as dialogue or reading aloud. After receiving the text content, there is a demand to convert the text into speech and play the text content in audio form. For example, for an article or a piece of text, the user may want to convert the article or the piece of text into speech for reading aloud.

[0044] Furthermore, when a speech synthesis request is received, it is necessary to determine the target text that needs to be converted into speech from the speech request, where the target text could be an article, a paragraph, a sentence, a word, or even a character, etc.

[0045] In the embodiment of the present disclosure, to improve the efficiency of text-to-speech synthesis and reduce the speech synthesis time, the speech synthesis process can start as soon as the target text is received via a streaming manner, without waiting for the entire speech data to be received. This approach enhances the efficiency of speech synthesis.

[0046] Further, in order for the process of inputting the target text into the speech synthesis model, where the speech synthesis model can accurately and quickly synthesize the target text to generate a more accurate and refined target synthesized speech before the obtained target text is input into the speech synthesis model, the target text can be processed in advance, thus obtaining the to-be-synthesized input information that the speech synthesis model can recognize

[0047] In the step S502, the to-be- synthesized input information, obtained by performing preset processing on the target text, can include information obtained by splicing the target text and historical information, or encoded combination information obtained by parsing text encoding of the target text and combining fixed speech encoding and / or model output speech encoding. The following will explain different preset processing methods.

[0048] In the first aspect, if the to-be-synthesized input information includes information obtained by splicing the target text and historical information, the step of performing preset processing on the target text to obtain to-be-synthesized input information includes the following steps

[0049] al: detecting whether the target text has historical information.

[0050] a2: using, if the target text does not have historical information, the target text as the to-be-synthesized input information.

[0051] a3: splicing, if the target text has historical information, the historical information with the target text according to an information category to obtain the to-be-synthesized input information.

[0052] In one possible embodiment, the historical information represents the contextual information of the current target text. When performing preset processing on the target text, adding the contextual information associated with the target text can more accurately determine the contextual logic and emotion of the target synthesized speech corresponding to the target text. This helps improve the logical consistency, fluency, and emotional continuity of the synthesized speech corresponding to the target text.

[0053] Specifically, the historical information is determined through following steps.

[0054] bl: ascertaining the historical information based on contextual information of the target text comprised in the speech synthesis request; and / or

[0055] b2: determining other speech synthesis requests containing text content associated with the target text, and determining other historical information comprised in other speech synthesis requests as the historical information.

[0056] In one possible embodiment, the historical information corresponding to the target text can be the contextual information associated with the target text in the same speech synthesis request. For example, if the current speech synthesis request is to convert an article into target synthesized speech, and the entire article contains three paragraphs of text, with the first paragraph of text already synthesized, and the current target text corresponds to the second paragraph of text of the article, then the historical information of the target text can include the first paragraph of text and / or the target synthesized speech generated from the first paragraph of text.

[0057] In another possible embodiment, the historical information corresponding to the target text can also include historical information from other speech synthesis requests that are related to the target text.

[0058] The text content associated with the target text comprises at least one of following:

[0059] text content associated with a text logic of the target text, and text content associated with a speech synthesis scenario information of the target text.

[0060] In one possible embodiment, the current target text obtained can be text information that contains logical dialogue content. Therefore, the text content logically associated with the current target text and / or the target text associated with the speech synthesis scenario information can be determined as the historical information of the current target text.

[0061] For example, The current target text is a paragraph in the dialogue information, which is the question answer sentence "Today is Monday", and the text content associated with the target text can be the logically related question content “What day is today?”; or if the currently synthesized speech is the quotations of character A, then the target text is some sentences in the quotations of character A, and the historical information associated with the target text can be other contents in the quotations of character A, or narration contents.

[0062] Furthermore, when determining the to-be-synthesized input information based on the target text, it is necessary to determine whether the current target text information has historical information. If the target text is the first synthesized text, it is determined that the current target text does not have historical information. In this case, the target text can be directly determined as the to-be-synthesized input information. If it is determined that the current target text has historical information, the historical information needs to be spliced with the target text to obtain the to-be-synthesized input information.

[0063] When splicing the historical information with the target text, the historical information can be spliced with the target text in the form of prompt information to obtain the to-be-synthesized input information.

[0064] Specifically, the step of splicing the historical information with the target text according to an information category to obtain the to-be-synthesized input information includes the following step.

[0065] cl: splicing the historical information with the target text in a form of prompt information according to the information category to obtain the to-be-synthesized input information.

[0066] In one possible embodiment, when splicing the historical information with the target text, the entire historical information can be spliced with the target text in the form of prompt information to obtain the to-be-synthesized input information.

[0067] In one possible embodiment, the historical information can include historical input text information and historical speech synthesis information. When splicing the historical information with the target text, it is necessary to splice with the target text according to the information category of the historical information, so as to obtain the to-be-synthesized input information

[0068] Specifically, the step of splicing the historical information with the target text according to an information category to obtain the to-be-synthesized input information includes the following step.

[0069] dl: splicing at least one piece of historical input text information comprised in the historical information with the target text to obtain spliced text information.

[0070] d2: splicing at least one piece of historical speech synthesis information comprised in the historical information to obtain spliced speech information.

[0071] d3: using a combination of the spliced text information and / or the target text with the spliced speech information as the to-be-synthesized input information.

[0072] In one possible embodiment, when splicing the historical information with the target text, the splicing needs to be performed according to the category of the historical information. That is, the historical input text information is spliced with the target text to obtain spliced text information. Then, multiple historical speech synthesis information is spliced to obtain spliced speech information. Finally, the spliced speech information is used as the to-be-synthesized input information, or the target text and the spliced speech information are used as the to-be-synthesized input information, or the combination of the spliced text information and spliced speech information is used as the to-be-synthesized input information.

[0073] For example, referring to FIG. 6, the speech synthesis request includes text 1, text 2, and text 3. After the preset processing of text 1, since text 1 does not have historical information, text 1 itself is determined as the to-be-synthesized input information, and after processing by the speech synthesis model, synthesized speech 1 is obtained. When processing text 2, the historical information of text 2 includes text 1 and synthesized speech 1. Therefore, text 1 is spliced with text 2 to obtain text 1+text 2, and synthesized speech 1 is added. Therefore, the to-be-synthesized input information corresponding to text 2 is (text 1+text 2, synthesized speech 1). The (text 1+text 2, synthesized speech 1) is input into the speech synthesis model for processing to obtain synthesized speech 2. When processing text 3, the historical information of text 2 includes text 1, synthesized speech 1, text 2, and synthesized speech 2. Therefore, text 1, text 2, and text 3 are spliced to obtain text 1+text 2+text 3, and synthesized speech 1 is spliced with synthesized speech 2 to obtain synthesized speech 1+synthesized speech 2. Therefore, the to-be-synthesized input information corresponding to text 3 is (text 1+text 2+text 3, synthesized speech 1+synthesized speech 2). The (text 1+text 2+text 3, synthesized speech 1+synthesized speech 2) is input into the speech synthesis model for processing to obtain synthesized speech 3.

[0074] Further, the obtained to-be-synthesized input information is input into a pre-trained speech synthesis model. The speech synthesis model can combine the contextual information contained in the spliced historical information within the input data to output the target synthesized speech corresponding to the target text, ensuring the fluency, logical consistency, and emotional continuity of the context of the output target synthesized speech.

[0075] In the second aspect, when the to-be-synthesized input information includes encoded combination information obtained by parsing text encoding of the target text and combining fixed speech encoding and / or model output speech encoding, the step of performing preset processing on the target text to obtain to-be-synthesized input information includes the following steps

[0076] el : parsing the target text to obtain multiple text encodings corresponding to the target text.

[0077] e2: combining, for the text encoding located at a first position among the multiple text encodings, the text encoding with fixed speech encoding to obtain the to-be-synthesized input information.

[0078] e3: combining, for a text encoding among the multiple text encodings other than the text encoding located at the first position, the text encoding with a model output speech encoding obtained by inputting a text encoding preceding the current text encoding into the speech synthesis model to obtain the to-be-synthesized input information.

[0079] e4: combining, after an input of the multiple text encodings is completed, preset placeholder encoding with the model output speech encoding obtained by the speech synthesis model to obtain the to-be-synthesized input information.

[0080] In the embodiment of the present disclosure, after receiving the target text, the target text can be processed by word segmentation. One or more text encodings can represent a minimal word segmentation unit, thereby obtaining multiple text encodings of the target text. For different text encodings, the corresponding to-be-synthesized input information needs to be determined according to the order of the text encodings.|0081| In one possible implementation, when it is determined that the text encoding to be input is the text encoding located at a first position in the sequence, the text encoding can be combined with a fixed speech encoding to obtain the to-be-synthesized input information.

[0082] The fixed speech encoding can be a start encoding, a placeholder encoding, or other special encodings.

[0083] In one possible implementation, when it is determined that the text encoding to be input is a text encoding other than the text encoding located at the first position, the text encoding can be combined with the model output speech encoding obtained by inputting the previous text encoding into the speech synthesis model, to obtain the to-be-synthesized input information.

[0084] For example, for the first text encoding 1 located at the first position of text encodings, the corresponding to-be-synthesized input information is text encoding 1 + start encoding orplaceholder encoding. After inputting the to-be-synthesized input information (text encoding 1 + start encoding or placeholder encoding) into the speech synthesis model, synthesized speech encoding 1 is obtained. For the text encoding 2 located at a second position of text encodings, the corresponding to-be-synthesized input information is text encoding 2 + synthesized speech encoding 1. After inputting the to-be-synthesized input information (text encoding 2 + synthesized speech encoding 1) into the speech synthesis model, synthesized speech encoding 2 is obtained.

[0085] In another possible implementation, after inputting multiple text encodings, if the synthesized speech output processed by the speech model has not yet been completed, a preset placeholder encoding can be combined with the model output speech encoding obtained from the speech synthesis model to obtain the to-be-synthesized input information.

[0086] Further, after determining the to-be-synthesized input information, the to-be-synthesized input information can be input into the speech synthesis model. The speech synthesis model, based on the to-be-synthesized input information, generates the target synthesized speech

[0087] Specifically, the step of inputting the to-be-synthesized input information into a pretrained speech synthesis model, so that the speech synthesis model outputs target synthesized speech corresponding to the target text in combination with text-associated content of the to-be-synthesized input information includes the following steps.

[0088] fl : combining, for the text encoding located at the first position among the multiple text encodings, the fixed speech encoding and inputting it into the speech synthesis model to output a corresponding first-position audio encoding.

[0089] f2: sequentially combining, for text encodings among the multiple text encodings other than the text encoding located at the first position, each text encoding with the speech encoding output by the speech synthesis model before the current text encoding, inputting them into the speech synthesis model, and outputting a corresponding audio encoding, until the speech synthesis model outputs an ending encoding; and obtaining the target synthesized speech based on the multiple audio encodings output by the speech synthesis model.

[0090] Tn the embodiment of the present disclosure, when receiving text encodings through a streaming reception method, for a first position text encoding, the text encoding can be combined with a fixed speech encoding and then input into the speech synthesis model to output the first-position audio encoding, where the fixed speech encoding can include a start encoding, a placeholder encoding, or a special encoding. For text encodings among the multiple text encodings other than the text encoding located at the first position, each text encoding can besequentially combined with the speech encoding output by the speech synthesis model before the current text encoding, and then input into the speech synthesis model to output a corresponding audio encoding. Until the speech synthesis model outputs the encoding, it is determined that the current target text input is completed, and then the multiple audio encodings obtained are combined in sequence to obtain the target synthesized speech.

[0091] For example, as shown in FIG. 7, text encoding 1 with placeholder 1, text encoding 2 with placeholder 2, and text encoding 3 with placeholder 3 are input into the speech synthesis model, resulting in audio encoding 1, audio encoding 2, and audio encoding 3, which are combined to obtain the target synthesized speech.

[0092] In one possible embodiment, in order to ensure that the target synthesized speech better meets the synthesis requirements of the current speech synthesis scenario, the to-be-synthesized input information can be input into the speech synthesis model simultaneously with the speech synthesis scenario, so that the speech synthesis model, when combining the contextual information, also combines the speech synthesis scenario information to output a target synthesized speech that better matches the current speech scenario. This further improves the accuracy and relevance of the target speech synthesis.

[0093] Specifically, the speech synthesis request further comprises speech synthesis scenario information; and the step of inputting the to-be-synthesized input information into a pre-trained speech synthesis model, so that the speech synthesis model outputs target synthesized speech corresponding to the target text in combination with text-associated content of the to-be-synthesized input information includes the following steps.

[0094] gl: inputting the to-be-synthesized input information and the speech synthesis scenario information into a pre-trained speech synthesis model, so that the speech synthesis model outputs target synthesized speech corresponding to the target text and conforming to a speech synthesis scenario by combining the text-associated content of the to-be-synthesized input information and the speech synthesis scenario information.

[0095] In one possible embodiment, the speech synthesis scenario information can represent the specific scenario of the current speech synthesis, such as the role information, timbre information, or emotional information of the target speech to be output currently.

[0096] In another possible embodiment, the speech synthesis scenario information can include one or more control encodings and the number of control encodings can be set according to the control requirements of the speech scenario. The speech synthesis scenario information can bespliced with the to-be-synthesized input information in a form of prompts or a form of control encoding.

[0097] For example, it can be spliced at the beginning of the to-be-synthesized input information. This way, before the model receives the to-be-synthesized input information, the specific scenario and requirements of the speech synthesis are provided in advance, ensuring that the model correspondingly outputs the target synthesized speech, and the corresponding relationship between the target text data and the target synthesized speech information output from the large model is stronger, thus improving the speech synthesis effect of the model.|0098| Specifically, for the same target text, if the corresponding speech synthesis scenario information differs, it can output different target synthesized speech information. For example, if the target text is “It’s a sunny day,” and the speech synthesis scenario information is the text content to be read by character A, then after combining the speech synthesis scenario information with the to-be-synthesized input information and inputting it into the speech synthesis model, it will output a target synthesized speech corresponding to the timbre of character A. If the speech synthesis scenario information is the text content to be read by character B, a different target synthesized speech will be output, where the generated target synthesized speech corresponds to the timbre of character B.

[0099] In one possible embodiment, the speech synthesis scenario information is a control encoding, set before each target text. When the to-be-synthesized input information and the speech synthesis scenario information are input into the pre-trained speech synthesis model, the speech synthesis model can first receive the speech synthesis scenario information, thereby determining what role, timbre, or emotion to use to synthesize the target synthesized speech corresponding to the target text.

[0100] In one possible embodiment, in order to ensure the accuracy of the output from the speech synthesis model, the training process of the speech synthesis model is also of great importance. The following will describe the training process of the speech synthesis model.

[0101] Specifically, the speech synthesis model is trained through the following steps.

[0102] hl : obtaining multiple sample text data and multiple sample speech data, and constructing multiple pieces of sample input information based on the multiple sample text data and the multiple sample speech data.

[0103] h2: inputting each piece of sample input information into a pre-constructed large model, and determining multiple pieces of output synthesized information output by the large model.

[0104] h3: comparing audio information in each piece of output synthesized information with the corresponding sample speech data, calculating performance data of the large model, and adjusting model parameters of the large model based on the performance data; and determining that, when the performance data reaches a preset value, a training of the large model is complete, so as to obtain the speech synthesis model.

[0105] In the embodiment of the present disclosure, multiple pieces of sample input information can be constructed based on the acquired multiple sample text data and sample speech data, and then the pre-constructed large model can be trained by inputting the constructed multiple pieces of sample input information.

[0106] There are different ways to construct sample input information, which will be explained separately below.

[0107] Specifically, the step of constructing multiple pieces of sample input information based on the multiple sample text data and the multiple sample speech data comprises the following steps

[0108] il: obtaining multiple pieces of sample historical information corresponding to the multiple sample text data and the multiple sample speech data.

[0109] i2: splicing each piece of sample historical information with each sample text data and / or each sample speech data in the form of prompt information according to the information category, and constructing multiple pieces of sample input information.

[0110] In the embodiment of the present disclosure, similarly, the sample history information can be contextual information associated with the sample text data in the same speech synthesis request as the sample text data, or the sample historical information can be historical information of text content and / or speech content in other speech synthesis requests associated with the sample text data.

[0111] In one possible embodiment, when splicing the sample historical information with the sample text data, it is also necessary to splice according to the classification of information in the sample historical information. Specifically, the sample historical information includes sample input text information and sample speech information. When splicing the sample text data and sample speech data, the sample text data can be spliced with the sample input text information, and the sample speech data can be spliced with the sample speech information, thereby obtaining the sample input information.

[0112] Further, in the step of inputting sample input information into a pre-constructed large model, audio information in output synthesized information output by the large model iscompared with the sample speech data in the sample input information, performance data of the large model is calculated, and model parameters of the large mode is adjusted based on the performance data. When the performance data reaches a preset value, it is determined that a training of the large model is complete, thus obtaining the speech synthesis model.

[0113] The performance data of the large model can be the loss function of the large model, such as the mean squared error loss function, mean absolute error loss, cross-entropy loss function, etc.

[0114] In one possible embodiment, when it is determined that the loss function of the large model is less than a preset threshold, it is determined that the performance data of the large model has reached the preset value, and the training of the large model is complete, thus obtaining the speech synthesis model.

[0115] For example, referring to FIG. 8, for sample text data 1 and sample speech data 1 input into the large model, after processing by the speech synthesis model, synthesized speech 1 is obtained. The synthesized speech 1 is compared with the sample speech data 1, and the performance data of the large model is calculated. For sample text data 2, the sample text data 2 is spliced with sample text data 1, thus obtaining sample text data 1+sample text data 2. The sample speech data 2 is spliced with sample speech data 1, thus obtaining sample speech data 1+sample speech data 2. The data (sample text data 1+sample text data 2, sample speech data 1+sample speech data 2) is input into the large model, processed by the speech synthesis model to obtain synthesized speech 2, and the synthesized speech 2 is compared with the sample speech data 2, and the performance data of the large model is calculated. For sample text data 3, the sample text data 2 is spliced with sample text data 1 and sample text data 2, thus obtaining sample text data 1+sample text data 2+sample text data 3. The sample speech data 3 is spliced with sample speech data 1 and sample speech data 2, thus obtaining sample speech data 1+sample speech data 2+sample speech data 3. The data (sample text data 1+sample text data 2+sample text data 3, sample speech data 1+sample speech data 2+sample speech data 3) is input into the large model, processed by the speech synthesis model to obtain synthesized speech 3, and the synthesized speech 3 is compared with the sample speech data 3. The performance data of the large model is calculated, and based on the performance data of each large model, the model parameters of the large model are adjusted. When the performance data reaches the preset value, the training of the large model is complete, and the speech synthesis model is obtained.

[0116] In another possible implementation, the step of constructing multiple pieces of sample input information based on the multiple sample text data and the multiple sample speech data comprises the following steps.

[0117] jl: performing, for each sample text data, word segmentation to obtain multiple sample text segmentation units corresponding to the sample text data, and determining at least one sample text encoding corresponding to each sample text segmentation unit.

[0118] j2: discretizing, for each sample speech data, the sample speech data according to a preset frame rate to obtain multiple discrete frame sample speech data, and determining a sample speech encoding corresponding to each discrete frame speech data.

[0119] j3: aligning and combining the obtained sample text encodings of each sample text data with the sample speech encodings of each sample speech data to obtain the aligned and combined sample text encodings and sample speech encodings.

[0120] j4: splicing and / or summing the aligned sample text encodings and sample speech encodings to construct multiple pieces of sample input information.

[0121] In the embodiment of the present disclosure, for each sample text data, word segmentation is performed to obtain multiple sample text segmentation units corresponding to the sample text data. When performing word segmentation on each sample text data, the minimum segmentation unit can be determined based on the specific text data included in the sample text data, to perform word segmentation on the sample text data. One or more encoding vectors are used to represent a minimum segmentation unit, resulting in at least one sample text encoding.

[0122] For example, the minimum segmentation unit can be a single character, a phrase, or a radical or stroke of a character.

[0123] In one possible embodiment, after segmenting the sample text data, at least one layer of sample text encoding can be obtained by segmenting the sample text data.

[0124] For example, under normal circumstances, an encoding vector represents a single character or phrase. When the model encounters rare characters that it does not recognize, the character needs to be split into two or more radicals or even strokes for individual encoding. This results in multiple encoding vectors representing a single character. In this case, multiple encodings of a character can be taken as one layer, or multiple encodings of a character can be taken as multiple layers. Other characters or phrases without multilayer encoding information can be replaced with placeholders, thus generating multilayer sample text encodings for the sample text data.

[0125] Further, for each sample speech data, the sample speech data can be discretized according to a preset frame rate to obtain multiple discrete frame sample speech data, and a sample speech encoding corresponding to each discrete frame speech data is determined.

[0126] Specifically, the step of determining a sample speech encoding corresponding to each discrete frame speech data comprises the following steps.

[0127] kl: determining, for each discrete frame speech data, a first standard speech encoding corresponding to the discrete frame speech data, and determining a first-layer sample speech encoding based on the first standard speech encoding.|00128| k2: determining, for each discrete frame speech data, except for the first standard speech encoding, a current standard speech encoding corresponding to a target residual based on the target residual between an actual sample speech encoding corresponding to the discrete frame speech data and a previous standard speech encoding; and determining a current-layer sample speech encoding based on the current standard speech encoding.

[0129] k3: when the target residual between the obtained standard speech encoding and the actual sample speech encoding is less than a preset residual threshold, or when the obtained standard speech encoding reaches a preset number, determining the obtained multi-layer standard speech encoding.

[0130] k4: determining the sample speech encoding corresponding to each discrete frame speech data based on the multi-layer standard speech encoding.

[0131] In one possible embodiment, for each sample text data corresponding to the sample speech data, the sample speech data is discretely encoded into at least one layer of sample speech encoding.|00132| For example, the sample speech data is discretized into 10 frames per second, with each frame represented by one or more encoding vectors in one layer or multiple layers.

[0133] Furthermore, for each frame of the sample speech data, by comparing and querying a preset first standard audio codebook, a first standard audio encoding most similar to the frame of sample speech data is obtained. The first layer of audio encoding for the sample speech data is then generated based on the first standard audio encoding. The target residual between the actual encoding of the frame and the first standard audio encoding is calculated. A preset second standard audio codebook is compared and queried according to the target residual to obtain a second standard audio encoding most similar to the target residual of the frame of sample speech data. A second layer of audio encoding for the sample speech data of the frame is then generated based on the second standard audio encoding. This process continues, and when the targetresidual between the standard speech encoding and the actual sample speech encoding is determined to be smaller than a preset residual threshold, or when the obtained standard speech encodings reach a preset number, the multi-layer standard speech encoding is determined. The sample speech encoding corresponding to each discrete frame speech data is then obtained based on the multi-layer standard speech encoding.

[0134] The preset residual threshold and the preset number of standard speech encodings can be set according to the specific encoding requirements and are not specifically limited herein.

[0135] Further, after obtaining the sample text encoding and sample speech encoding, the sample text encoding and sample speech encoding are aligned to obtain the aligned and combined sample text encodings and sample speech encodings.

[0136] Specifically, the step of aligning and combining the obtained sample text encodings of each sample text data with the sample text encodings of each sample speech data to obtain the aligned and combined sample text encodings and sample speech encodings comprises the following steps.

[0137] 11: sequentially aligning the sample text encodings in the sample text data with the sample speech encodings in the sample speech data in a corresponding order according to a text encoding order, to obtain the aligned and combined sample text encodings and sample speech encodings, wherein

[0138] for positions with differences in a number of encodings between the sample text encodings and the sample speech encodings, the sample speech encodings are aligned with the sample text encodings by preset placeholder encodings.

[0139] In one possible embodiment, alignment can refer to bitwise alignment, for example, aligning the first encoding of the sample text encoding with the first encoding of the sample speech encoding, and then proceeding to align the subsequent encodings. Since the sample speech encoding is generally several to tens of times larger than the sample text encoding, the missing portions of the sample text encoding during the alignment process can be replaced with placeholder encodings or special encodings.

[0140] Tn another possible embodiment, alignment can also refer to staggered alignment. The following will describe the alignment method for staggered alignment.

[0141] Specifically, in the step of sequentially aligning the sample text encodings in the sample text data with the sample speech encodings in the sample speech data in a corresponding order according to a text encoding order, the alignment method is staggered alignment, includes the following steps.

[0142] ml: aligning a first encoding of the sample text encoding with the fixed speech encoding of the sample speech encoding, wherein the fixed speech encoding can be a start encoding, a placeholder encoding, or a special encoding.

[0143] m2: aligning a second encoding of the sample text encoding with the first encoding of the sample speech encoding, until all encodings are aligned.

[0144] Similarly, for positions with differences in a number of encodings between the sample text encodings and the sample speech encodings, the sample speech encodings are aligned with the sample text encodings by preset placeholder encodings.|00145| In one possible embodiment, the alignment method for the sample text encoding and sample speech data can be staggered alignment, which includes, for example, aligning a first encoding of the sample text encoding with the fixed speech encoding of the sample speech encoding, wherein the fixed speech encoding can be a start encoding, a placeholder encoding, or a special encoding. For example, the first encoding of the sample text encoding is aligned with the first encoding of the sample speech encoding, and the same alignment follows, and so on. Since the sample speech encoding is generally several to tens of times larger than the sample text encoding, the missing portions of the sample text encoding during the alignment process can be replaced with placeholder encodings or special encodings.

[0146] Further, the step includes obtaining the aligned and combined sample text encodings and sample speech encodings, and splicing and / or summing the aligned sample text encodings and sample speech encodings to construct multiple pieces of sample input information.

[0147] Specifically, the sample text encoding and the sample speech encoding, when aligned and combined, generate at least two layers of encoding (one layer of sample text encoding and one layer of sample speech encoding). The sample input information is generated by splicing and / or adding the layers and is input into the large model for training.

[0148] For example, if the sample text encoding is a 1-layer 256-dimensional encoding vector, and the sample speech encoding is a 1-layer 256-dimensional encoding vector, splicing them will generate a 1-layer 512-dimensional input encoding vector. If they are added, a 1-layer 256-dimensional input encoding vector will be generated; or multiple layers of sample text encoding and / or sample speech encoding can be first added together to generate a 1-layer multidimensional vector, which is then spliced to generate a 1-layer input encoding vector with the dimension equal to the sum of the two encodings as the sample input information; or adding them together generates a 1-layer input encoding vector with the same dimension as the two encodings as the sample input information.

[0149] This training enables the large model to simultaneously acquire text information and speech information, allowing the large model to receive the input target text encoding and output the target synthesized speech encoding corresponding to the target text encoding. Thus, the trained speech synthesis model achieves higher processing efficiency.

[0150] Further, after obtaining multiple pieces of sample input information, the large model can be trained based on the multiple pieces of sample input information to obtain the speech synthesis model.

[0151] Specifically, the step of comparing audio information in each piece of output synthesized information with the corresponding sample speech data, calculating performance data of the large model, adjusting model parameters of the large model based on the performance data; and determining that, when the performance data reaches a preset value, a training of the large model is complete, so as to obtain the speech synthesis model, includes the following steps.

[0152] nl: calculating the performance data corresponding to the sample input information, for each piece of sample input information, based on the sample speech encoding in the sample input information and the corresponding output speech encoding in the output synthesized information output by the large model.

[0153] n2: adjusting the model parameters of the large model based on the performance data corresponding to each piece of sample input information; and determining that, when the performance data reaches the preset value, the training of the large model is complete, so as to obtain the speech synthesis model.

[0154] In the embodiment of the present disclosure, the generated sample input information is input into the large model for training, i.e., input the first input encoding (including the first sample text encoding of the sample text data and the fixed speech encoding of the sample speech data), and the model predicts and outputs the first output synthesis information. Then, the second input encoding (including the second sample text encoding of the sample text data and the first sample speech encoding of the sample speech data) is input, and the model predicts and outputs the second output synthesis information. Under normal circumstances, the output synthesis information output by the large model is consistent with the input sample input information in terms of the information content contained. That is, both contain text encoding content and speech encoding content, and the two can be combined and / or added together for output. This continues until all predicted output synthesis information is output.

[0155] Further, the output speech encoding is separated from the predicted output synthesis information and compared with the sample speech encoding in the sample input informationThe performance data of the model is calculated, and the parameters of the large model are optimized and adjusted to complete the training of the large model. After the large model training is completed, the speech synthesis model obtained can output only the speech encoding using the same method,

[0156] wherein the performance data of the large model can be the loss function

[0157] For example, referring to FIG. 9, the training process includes inputting the sample text encoding 1, the sample speech encoding 1 and placeholder 1; the sample text encoding 2, the sample speech encoding 2 and placeholder 2; and the sample text encoding 3, the sample speech encoding 3 and placeholder 3 into the large model to synchronously obtain the output synthesis information 1 (including output speech encoding 1), output synthesis information 2 (including output speech encoding 2), and output synthesis information 3 (including output speech encoding 3); comparing the output speech encoding 1 with the sample speech encoding 1, and calculating the performance data of the large model; comparing the output speech encoding 2 with the sample speech encoding 2, and calculating the performance data of the large model; comparing the output speech encoding 3 with the sample speech encoding 3, and calculating the performance data of the large model; adjusting the model parameters of the large model based on the performance data of each large model; and determining that, when the performance data reaches the preset value, the training of the large model is complete, so as to obtain the speech synthesis model. The placeholders 1 / 2 / 3 herein can be placeholder encoding or other special encoding.

[0158] In one possible embodiment, in order to ensure that the target synthesized speech better meets the synthesis when training the large model, the sample speech scenario information corresponding to the sample text data is also combined to train the large model. Therefore, the speech synthesis model, when combining the contextual information, also combines the speech synthesis scenario information to output a target synthesized speech that better matches the current speech scenario. This further improves the accuracy and relevance of the speech synthesis of the large model.

[0159] Specifically, the method for speech synthesis further comprises the following steps.

[0160] ol: determining sample speech scenario information corresponding to each sample text data.

[0161] o2: splicing the sample speech scenario information with the acquired initial text data in a form of a prompt word or a form of control encoding, to obtain the sample text data.

[0162] In one possible embodiment, the sample speech scenario information corresponding to the sample text data comprises at least one of the following:

[0163] role information corresponding to the sample text data, role timbre information corresponding to the sample text data, and speech emotion information corresponding to the sample text data.

[0164] In another possible embodiment, the sample speech scenario information can include one or more control encodings and the number of control encodings can be set according to the control requirements of the speech scenario. The sample speech scenario information can be spliced with the acquired initial text data in a form of a prompt word or a form of control encoding to obtain the sample text data. For example, it can be spliced at the beginning of the initial text data. This way, before the model receives the sample text data, the specific scenario and requirements of the speech synthesis are provided in advance, ensuring that the model correspondingly outputs the synthesized speech, and the corresponding relationship between the sample text data and the speech information in the output synthesized information output from the large model is stronger, thus improving the model training effect.

[0165] The method for speech synthesis is provided in the embodiments of the present disclosure, including obtaining a target text in a speech synthesis request; performing preset processing on the target text to obtain to-be-synthesized input information, wherein the to-be-synthesized input information includes information obtained by splicing the target text and historical information, or encoded combination information obtained by parsing text encoding of the target text and combining fixed speech encoding and / or model output speech encoding; and inputting the to-be-synthesized input information into a pre-trained speech synthesis model, so that the speech synthesis model outputs target synthesized speech corresponding to the target text in combination with text-associated content of the to-be-synthesized input information. In this way, after obtaining the target text in the speech synthesis request, the preset processing of the target text immediately begins. During processing, historical information is spliced to obtain a combination processing that includes text-associated content or encoding level. This ensures that the input speech synthesis model information contains text-associated content, and also reduces the delay between the target text input and the target synthesized speech output, which helps improve the accuracy and efficiency of speech synthesis.

[0166] Based on the same inventive concept, the embodiment of the present disclosure also provides a device for speech synthesis corresponding to the method for speech synthesis. Since the principle of the device in the embodiment of the present disclosure for solving the problem issimilar to the method for speech synthesis mentioned above in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repetitive parts will not be reiterated.

[0167] Referring to FIG. 10, the device for speech synthesis 1000 comprises:

[0168] a text acquisition module 1010, configured for obtaining a target text in a speech synthesis request,

[0169] an input information synthesis module 1020, configured for performing preset processing on the target text to obtain to-be-synthesized input information, wherein the to-be-synthesized input information includes information obtained by splicing the target text and historical information, or encoded combination information obtained by parsing text encoding of the target text and combining fixed speech encoding and / or model output speech encoding; and

[0170] a synthesized speech output module 1030, configured for inputting the to-be-synthesized input information into a pre-trained speech synthesis model, so that the speech synthesis model outputs target synthesized speech corresponding to the target text in combination with text-associated content of the to-be-synthesized input information.

[0171] A device for speech synthesis is provided in the embodiments of the present disclosure, which is configured for obtaining a target text in a speech synthesis request; performing preset processing on the target text to obtain to-be-synthesized input information, wherein the to-be-synthesized input information includes information obtained by splicing the target text and historical information, or encoded combination information obtained by parsing text encoding of the target text and combining fixed speech encoding and / or model output speech encoding; and inputting the to-be-synthesized input information into a pre-trained speech synthesis model, so that the speech synthesis model outputs target synthesized speech corresponding to the target text in combination with text-associated content of the to-be-synthesized input information. In this way, after obtaining the target text in the speech synthesis request, the preset processing of the target text immediately begins. During processing, historical information is spliced to obtain a combination processing that includes text-associated content or encoding level. This ensures that the input speech synthesis model information contains text-associated content, and also reduces the delay between the target text input and the target synthesized speech output, which helps improve the accuracy and efficiency of speech synthesis.

[0172] Referring FIG. 11, the electronic device 1100 comprises a processor 1110, a memory 1120, and a bus 1130, wherein

[0173] the memory 1120 stores machine-readable instructions that are executed by the processor 1110, the processor 1110 communicates with the memory 1120 via the bus 1130 when the electronic device 1100 is in operation, and the machine-readable instructions perform the steps of the method for speech synthesis as FIG. 5 described above when run by the processor 1110 Specific implementations can be found in the method embodiments and will not be repeated here.

[0174] The embodiment of the present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium is provided with a computer program stored thereon; the computer program is able to perform the steps of the method for speech synthesis as in the method embodiment shown in FIG. 5 above; specific implementations can be found in the method embodiments and will not be repeated here.

[0175] It will be clear to those skilled in the field that, for the convenience and brevity of the description, the specific working processes of the systems, devices, and units described above can be referred to as the corresponding processes in the preceding method embodiments and will not be repeated here.

[0176] In the several embodiments provided in the present disclosure, it should be understood that the systems, devices, and methods disclosed, can be implemented in other ways. The abovedescribed embodiments of the device are merely schematic. For example, the division of the units described, which is only a logical functional division, can be divided in another way when actually implemented; and for another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not implemented. On another point, the mutual coupling, direct coupling, or communication connection shown or discussed herein can be an indirect coupling or communication connection through communication interfaces, devices, or units, which can be electrical, mechanical, or other forms.

[0177] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, meaning they can be located in one place or distributed across multiple network units. Some or all of the units can be selected as needed to achieve the objectives of the embodiments of the present disclosure.

[0178] Further, each functional unit in each embodiment of the present disclosure can be integrated into a single processing unit, each unit can be physically present separately, or two or more units can be integrated into a unit.

[0179] Said functionality, when implemented as a software functional unit and sold or used as a stand-alone product, may be stored in a processor-executable, non-volatile, computer-readablestorage medium. Based on this understanding, the technical solution of the present disclosure can essentially be embodied in the form of a software product, which contributes to or includes parts of the prior art. The software product is stored in a storage medium and includes multiple instructions for causing a computer device (which can be a personal computer, server, network device, etc.) to execute all or some of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage media include various media that can store program code, such as USB drives, external hard drives, read-only memory (ROM), random access memory (RAM), disks, or optical discs.|00180| Finally, it should be noted that the above-described embodiments are only specific embodiments of the present disclosure to illustrate the technical solutions of the present disclosure, and not to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that any person skilled in the art can still make modifications or easily envisage variations to the technical solutions described in the aforementioned embodiments within the technical scope disclosed by the present disclosure; or some technical features can be equivalently substituted. These modifications, changes, or substitutions do not depart from the essence of the technical solutions of the embodiments of the present disclosure and its scope. All these should be encompassed within the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be stated to be subject to the scope of protection of the claims.

Claims

CLAIMS1. A method for speech synthesis, wherein the method for speech synthesis comprises:obtaining a target text in a speech synthesis request;performing preset processing on the target text to obtain to-be-synthesized input information, wherein the to-be-synthesized input information comprises information obtained by splicing the target text and historical information, or encoded combination information obtained by parsing text encoding of the target text and combining fixed speech encoding and / or model output speech encoding; andinputting the to-be-synthesized input information into a pre-trained speech synthesis model, so that the speech synthesis model outputs target synthesized speech corresponding to the target text in combination with text-associated content of the to- be-synthesized input information.

2. The method for speech synthesis according to claim 1, wherein the step of performing preset processing on the target text to obtain to-be-synthesized input information comprises:parsing the target text to obtain multiple text encodings corresponding to the target text;combining, for a text encoding located at a first position among the multiple text encodings, the text encoding with fixed speech encoding to obtain the to-be- synthesized input information;combining, for a text encoding among the multiple text encodings other than the text encoding located at the first position, the text encoding with a model output speech encoding obtained by inputting a text encoding preceding the text encoding into the speech synthesis model to obtain the to-be-synthesized input information; combining, after an input of the multiple text encodings is completed, preset placeholder encoding with the model output speech encoding obtained by the speech synthesis model to obtain the to-be-synthesized input information.

3. The method for speech synthesis according to claim 1, wherein the step of performing preset processing on the target text to obtain to-be-synthesized input information further comprises:detecting whether the target text has historical information;using, when the target text does not have the historical information, the target text as the to-be-synthesized input information; andsplicing, when the target text has the historical information, the historical information with the target text according to an information category to obtain the to-be- synthesized input information.

4. The method for speech synthesis according to claim 3, wherein the step of splicing the historical information with the target text according to an information category to obtain the to-be- synthesized input information comprises:splicing the historical information with the target text in a form of prompt information according to the information category to obtain the to-be-synthesized input information.

5. The method for speech synthesis according to claim 3, wherein the historical information comprises historical input text information and historical speech synthesis information; and the step of splicing the historical information with the target text according to an information category to obtain the to-be-synthesized input information comprises:splicing at least one piece of historical input text information comprised in the historical information with the target text to obtain spliced text information; splicing at least one piece of historical speech synthesis information comprised in the historical information to obtain spliced speech information; andusing a combination of the spliced text information and / or the target text with the spliced speech information as the to-be-synthesized input information.

6. The method for speech synthesis according to claim 1, wherein the historical information is determined through following steps:ascertaining the historical information based on contextual information of the target text comprised in the speech synthesis request; and / or,determining other speech synthesis requests containing text content associated with the target text, and determining other historical information comprised in other speech synthesis requests as the historical information7. The method for speech synthesis according to claim 6, wherein the text content associated with the target text comprises at least one of following:text content associated with a text logic of the target text, and text content associated with a speech synthesis scenario information of the target text.

8. The method for speech synthesis according to claim 2, wherein the step of inputting the to-be-synthesized input information into a pre-trained speech synthesis model, so that the speech synthesis model outputs target synthesized speech corresponding to the target text in combination with text-associated content of the to-be-synthesized input information comprises:combining, for the text encoding located at the first position among the multiple text encodings, the fixed speech encoding and inputting the combination into the speech synthesis model to output a corresponding first-position audio encoding; and sequentially combining, for text encodings among the multiple text encodings other than the text encoding located at the first position, each text encoding with a speech encoding output by the speech synthesis model before the text encoding, inputting them into the speech synthesis model, and outputting a corresponding audio encoding, until the speech synthesis model outputs an ending encoding; and obtaining the target synthesized speech based on multiple audio encodings output by the speech synthesis model.

9. The method for speech synthesis according to claim 1, wherein the speech synthesis request further comprises speech synthesis scenario information; and the step of inputting the to-be-synthesized input information into a pre-trained speech synthesis model, so that the speech synthesis model outputs target synthesized speech corresponding to the target text in combination with text-associated content of the to- be-synthesized input information comprises:inputting the to-be-synthesized input information and the speech synthesis scenario information into the pre-trained speech synthesis model, so that the speech synthesis model outputs target synthesized speech corresponding to the target text and conforming to a speech synthesis scenario by combining the text-associated content of the to-be-synthesized input information and the speech synthesis scenario information10. The method for speech synthesis according to claim 9, wherein the speech synthesis scenario information comprises one or more control encodings set before the to-be- synthesized input information; when the to-be-synthesized input information and the speech synthesis scenario information are input into the pre-trained speech synthesis model, the speech synthesis model first receives the speech synthesis scenario information.

11. The method for speech synthesis according to claim 1, wherein the speech synthesis model is trained through following steps:obtaining multiple sample text data and multiple sample speech data, and constructing multiple pieces of sample input information based on the multiple sample text data and the multiple sample speech data;inputting each piece of sample input information into a pre-constructed large model, determining multiple pieces of output synthesized information output by the large model; andcomparing audio information in each piece of output synthesized information with corresponding sample speech data, calculating performance data of the large model, and adjusting model parameters of the large model based on the performance data; and determining that, when the performance data reaches a preset value, a training of the large model is complete, so as to obtain the speech synthesis model.

12. The method for speech synthesis according to claim 11, wherein the step of constructing multiple pieces of sample input information based on the multiple sample text data and the multiple sample speech data comprises:obtaining multiple pieces of sample historical information corresponding to the multiple sample text data and the multiple sample speech data, andsplicing each piece of sample historical information with each sample text data and / or each sample speech data in the form of prompt information according to the information category, and constructing the multiple pieces of sample input information.

13. The method for speech synthesis according to claim 11, wherein the step of constructing multiple pieces of sample input information based on the multiple sample text data and the multiple sample speech data comprises:performing, for each sample text data, word segmentation to obtain multiple sample text segmentation units corresponding to the sample text data, and determining at least one sample text encoding corresponding to each sample text segmentation unit; discretizing, for each sample speech data, the sample speech data according to a preset frame rate to obtain multiple discrete frame sample speech data, and determining a sample speech encoding corresponding to each discrete frame speech data; aligning and combining the obtained sample text encodings of each sample text data with the sample speech encodings of each sample speech data to obtain the aligned and combined sample text encodings and sample speech encodings; andsplicing and / or summing the aligned sample text encodings and sample speech encodings to construct the multiple pieces of sample input information14. The method for speech synthesis according to claim 13, wherein the step of determining a sample speech encoding corresponding to each discrete frame speech data comprises:determining, for each discrete frame speech data, a first standard speech encoding corresponding to the discrete frame speech data, and determining a first-layer sample speech encoding based on the first standard speech encoding;determining, for each discrete frame speech data, except for the first standard speech encoding, a current standard speech encoding corresponding to a target residual based on the target residual between an actual sample speech encoding corresponding to the discrete frame speech data and a previous standard speech encoding; and determining a current-layer sample speech encoding based on the current standard speech encoding; determining an obtained multi-layer standard speech encoding, when the target residual between the obtained standard speech encoding and the actual sample speech encoding is less than a preset residual threshold, or when the obtained standard speech encoding reaches a preset number; anddetermining the sample speech encoding corresponding to each discrete frame speech data based on the multi-layer standard speech encoding.

15. The method for speech synthesis according to claim 13, wherein the step of aligning and combining the obtained sample text encodings of each sample text data with the sample text encodings of each sample speech data to obtain the aligned and combined sample text encodings and sample speech encodings comprises:sequentially aligning the sample text encodings in the sample text data with the sample speech encodings in the sample speech data in a corresponding order according to a text encoding order, to obtain the aligned and combined sample text encodings and sample speech encodings, whereinfor positions with differences in the number of encodings between the sample text encodings and the sample speech encodings, the sample speech encodings are aligned with the sample text encodings by preset placeholder encodings.

16. The method for speech synthesis according to claim 15, wherein in the step of sequentially aligning the sample text encodings in the sample text data with the sample speech encodings in the sample speech data in a corresponding order according to a text encoding order, the alignment method is staggered alignment, comprising:aligning a first encoding of the sample text encodings with the fixed speech encoding of the sample speech encodings, wherein the fixed speech encoding can be a start encoding, a placeholder encoding, or a special encoding; andaligning a second encoding of the sample text encodings with a first encoding of the sample speech encodings, until all encodings are aligned, whereinfor positions with differences in the number of encodings between the sample text encodings and the sample speech encodings, the sample speech encodings are aligned with the sample text encodings by the preset placeholder encodings.

17. The method for speech synthesis according to claim 13, wherein the step of comparing audio information in each piece of output synthesized information with corresponding sample speech data, calculating performance data of the large model, and adjusting model parameters of the large model based on the performance data; and determining that, when the performance data reaches a preset value, a training of the large model is complete, so as to obtain the speech synthesis model, comprises:calculating the performance data corresponding to the sample input information, for each piece of sample input information, based on the sample speech encoding in the sample input information and a corresponding output speech encoding in the output synthesized information output by the large model; andadjusting the model parameters of the large model based on the performance data corresponding to each piece of sample input information; and determining that, when the performance data reaches the preset value, the training of the large model is complete, so as to obtain the speech synthesis model.

18. The method for speech synthesis according to claim 11, wherein the method for speech synthesis further comprises:determining sample speech scenario information corresponding to each sample text data; andsplicing the sample speech scenario information with acquired initial text data in a form of a prompt word or a form of control encoding, to obtain the sample text data.

19. The method for speech synthesis according to claim 18, wherein the sample speech scenario information corresponding to the sample text data comprises at least one of following:role information corresponding to the sample text data, role timbre information corresponding to the sample text data, and speech emotion information corresponding to the sample text data.

20. A device for speech synthesis, wherein the device for speech synthesis comprises: a text acquisition module, configured for obtaining a target text in a speech synthesis request;an input information synthesis module, configured for performing preset processing on the target text to obtain to-be-synthesized input information, wherein the to-be- synthesized input information comprises information obtained by splicing the target text and historical information, or encoded combination information obtained by parsing text encoding of the target text and combining fixed speech encoding and / or model output speech encoding; anda synthesized speech output module, configured for inputting the to-be-synthesized input information into a pre-trained speech synthesis model, so that the speech synthesis model outputs target synthesized speech corresponding to the target text in combination with text-associated content of the to-be-synthesized input information.

Images