Speech synthesis method and device, equipment and storage medium

Abstract

The invention discloses a speech synthesis method and device, equipment and a storage medium, belongs to the technical field of speech synthesis, and is used for reducing the waiting time of a user in a speech synthesis process. The method comprises the following steps: acquiring a target text for speech synthesis, and segmenting the target text by adopting a plurality of preset short sentence segmentation modes to obtain a plurality of first short sentence character strings; based on the character length of each first short sentence, determining a first length and a first consumed time corresponding to each first short sentence through a preset corresponding relation table; based on the first length corresponding to each first short sentence, determining a first duration after each first short sentence generates voice, so as to determine waiting time of a user when streaming voice synthesis is carried out on each first short sentence character string according to the first duration and first consumed time corresponding to each first short sentence; and determining the first short sentence character string of which the waiting time is less than a preset threshold value as a target short sentence character string, and performing streaming speech synthesis based on the target short sentence character string.

Description

Technical Field

[0001] The present application belongs to the field of speech synthesis technology, and specifically relates to a speech synthesis method, apparatus, device and storage medium. Background Art

[0002] Existing speech synthesis methods generally use a text-to-speech (TTS) model based on an encoder-decoder architecture. These models take a text segment as input and generate a sequence of speech vectors as output. The speed of speech synthesis depends on factors such as the length of the input text, the number of TTS model parameters, and the hardware environment in which the model runs. Given a certain number of TTS model parameters and hardware environment, the longer the input text, the slower the model's speech calculation speed. In real-time or near-real-time scenarios (such as digital human voice question-and-answer scenarios), shortening user waiting time is crucial to improve the user experience.

[0003] Therefore, a speech synthesis method is needed that can reduce the waiting time of a user during the speech synthesis process. Summary of the Invention

[0004] The embodiment of the present application provides a speech synthesis method, which can reduce the waiting time of users during the speech synthesis process.

[0005] In a first aspect, an embodiment of the present application provides a speech synthesis method, the method comprising: obtaining a target text for speech synthesis, and segmenting the target text using a plurality of preset short sentence segmentation methods to obtain a plurality of first short sentence strings, each of the first short sentence strings including a plurality of first short sentences; determining a first length and a first time corresponding to each first short sentence based on the character length of each first short sentence through a preset correspondence table, wherein the correspondence table pre-stores the first lengths and the first time corresponding to short sentences of different character lengths, the first length being used to characterize the length of a vector generated when synthesizing a short sentence with the same character length as the first short sentence, and the first time being used to characterize the inference time required when synthesizing a short sentence with the same character length as the first short sentence; determining a first duration after speech generation for each first short sentence based on the first length corresponding to each first short sentence, so as to determine a user's waiting time when performing streaming speech synthesis on each first short sentence string based on the first duration and the first time corresponding to each first short sentence; determining the first short sentence string whose waiting time is less than a preset threshold as a target short sentence string, and performing streaming speech synthesis based on the target short sentence string.

[0006] In a second aspect, an embodiment of the present application provides a speech synthesis device, which includes: a first acquisition module for acquiring a target text for speech synthesis, and segmenting the target text using a plurality of preset short sentence segmentation methods to obtain a plurality of first short sentence character strings, each of which includes a plurality of first short sentences; a first determination module for determining a first length and a first time corresponding to each first short sentence based on the character length of each first short sentence through a preset correspondence table, wherein the correspondence table pre-stores the first length and the first time corresponding to short sentences of different character lengths, and the first length is used to characterize the synthesis and The first time consumption is used to characterize the inference time required for synthesizing a short sentence with the same character length as the first short sentence; the second determination module is used to determine the first duration of each first short sentence after generating speech based on the first length corresponding to each first short sentence, so as to determine the user's waiting time when performing streaming speech synthesis on each first short sentence string according to the first time length corresponding to each first short sentence; the first synthesis module is used to determine the first short sentence string whose waiting time is less than a preset threshold as a target short sentence string, and perform streaming speech synthesis based on the target short sentence string.

[0007] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0010] In a sixth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the method described in the first aspect.

[0011] In an embodiment of the present application, a target text for speech synthesis is obtained and segmented using a plurality of preset short sentence segmentation methods to obtain a plurality of first short sentence character strings, each of which includes a plurality of first short sentences. Based on the character length of each first short sentence, a first length and a first time corresponding to each first short sentence are determined using a preset correspondence table, wherein the correspondence table pre-stores the first lengths and first time corresponding to sentences of different character lengths, the first length being used to represent the length of a vector generated when synthesizing a short sentence of the same character length as the first short sentence, and the first time being used to represent the inference time required when synthesizing a short sentence of the same character length as the first short sentence. Based on the first length corresponding to each first short sentence, a first duration after speech generation is performed for each first short sentence is determined, so as to determine a user waiting time when performing streaming speech synthesis on each first short sentence character string based on the first duration and the first time corresponding to each first short sentence. The first short sentence character string having a waiting time less than a preset threshold is determined as a target short sentence character string, and streaming speech synthesis is performed based on the target short sentence character string, thereby reducing the user waiting time during the speech synthesis process. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a flow chart of a speech synthesis method provided in an embodiment of the present application; Figure 2 1 is a flow chart of a second speech synthesis method provided in an embodiment of the present application; Figure 3 1 is a flow chart of a third speech synthesis method provided in an embodiment of the present application; Figure 4 1 is a flow chart of a fourth speech synthesis method provided in an embodiment of the present application; Figure 5 This is a structural diagram of a speech synthesis device provided in an embodiment of the present application; Figure 6 This is a structural diagram of a speech synthesis device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0014] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0015] The speech synthesis method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0016] Figure 1 A speech synthesis method provided by an embodiment of the present invention is shown. The method can be performed by an electronic device, which may include: a server and / or a terminal device, where the terminal device may be, for example, an in-vehicle terminal or a mobile phone terminal. In other words, the method can be performed by software or hardware installed in the speech synthesis device, and the method includes the following steps: Step 102: Obtain a target text for speech synthesis, and segment the target text using a plurality of preset short sentence segmentation methods to obtain a plurality of first short sentence character strings.

[0017] Each of the first short sentence character strings includes multiple first short sentences.

[0018] The speech synthesis method described in this application can be performed by a speech synthesis system or speech synthesis software. The embodiments of this application use a speech synthesis system as an example. The speech synthesis system obtains a target text for speech synthesis and segments the target text into short sentences using a plurality of preset short sentence segmentation methods. The segmentation results corresponding to the various short sentence segmentation methods are obtained, namely, a plurality of first short sentence character strings, where the target text is the text to be synthesized into speech, and each first short sentence character string includes a plurality of first short sentences.

[0019] The speech synthesis system can directly determine the input text entered by the user as the target text, or it can first perform a preprocessing operation on the input text entered by the user and determine the preprocessing result as the target text, wherein the preprocessing operation can be a normalization process. When the input text is normalized, the special characters in the user input text are mainly processed, including character deletion, character replacement and other operations, deleting symbols such as line breaks, spaces, braces, square brackets, parentheses, and replacing punctuation marks that are inconsistent between Chinese and English, or between full-width and half-width characters. By normalizing the user input text, a standardized and easy-to-process text format can be generated to facilitate its processing in the next step.

[0020] There are multiple preset sentence segmentation methods, which can be based on punctuation marks or based on the number of characters. When segmenting sentences based on punctuation marks, the sentences can be segmented based on commas and periods, or based on periods only, or selectively segmented based on punctuation marks according to the preset punctuation segmentation method. When segmenting sentences based on the number of characters, the target input can be uniformly segmented into sentences of five characters, fifteen characters, or other numbers of characters. A preset number can also be randomly selected and the sentences can be segmented based on the randomly selected number of characters.

[0021] Because multiple sentence segmentation methods are preset, the speech synthesis system can segment the target text in various ways. The speech synthesis system segments the target text according to each sentence segmentation method, thereby obtaining segmentation results corresponding to each sentence segmentation method, namely, first sentence character strings. Specifically, each first sentence character string includes multiple first sentences, each of which is a text segment of the target text. Furthermore, if the multiple first sentences in each first sentence character string are merged, the complete target text can still be obtained.

[0022] Step 104: Based on the character length of each of the first short sentences, determine the first length and first time consumption corresponding to each of the first short sentences through a preset correspondence table.

[0023] Among them, the correspondence table pre-stores the first length and the first time corresponding to short sentences of different character lengths. The first length is used to represent the vector length generated when synthesizing a short sentence with the same character length as the first short sentence, and the first time is used to represent the inference time required when synthesizing a short sentence with the same character length as the first short sentence.

[0024] After determining multiple first short sentence character strings and multiple first short sentences included in each first short sentence character string, the speech synthesis system determines the first length and first time corresponding to each first short sentence from a preset correspondence table based on the character length of each first short sentence, wherein the preset correspondence table pre-stores the first lengths and first time corresponding to short sentences of various character lengths, so that the speech synthesis system can determine the first length and first time corresponding to each first short sentence based on the character length.

[0025] The first length corresponding to the first short sentence is used to represent the length of the vector generated when the speech synthesis model synthesizes a short sentence of the same character length as the first short sentence; the first time corresponding to the first short sentence is used to represent the model inference time required when the speech synthesis model synthesizes a short sentence of the same character length as the first short sentence. For example, when the first short sentence is two characters long, the speech synthesis system determines the vector length generated and the model inference time required when the preset speech synthesis model synthesizes a short sentence of two characters from a preset correspondence table.

[0026] That is to say, the preset correspondence table pre-stores the vector lengths and the required model inference time generated when the speech synthesis model synthesizes short sentences of various different character lengths. In this way, after the speech synthesis system obtains the character length of each first short sentence, it can determine the vector length and the required model inference time generated when generating speech based on the first short sentence according to the character length of the first short sentence, that is, determine the first length and first time corresponding to each first short sentence according to the first length and first time corresponding to the preset multiple character lengths.

[0027] Specifically, the speech synthesis model can obtain the i-th first short sentence S in each first short sentence string i and the character length L of the first phrase i , and then determine the first length V corresponding to each first short sentence i The first time taken is T i , and V i and T i As the generated S i The estimated length of the speech vector generated and the time taken to infer the model.

[0028] Step 106: Determine the first duration of each first short sentence after generating speech based on the first length corresponding to each first short sentence, so as to determine the user's waiting time when performing streaming speech synthesis on each first short sentence character string according to the first duration and the first time consumption corresponding to each first short sentence.

[0029] After determining the first length and first duration corresponding to each first short sentence, the speech synthesis system determines the speech duration (i.e., the first duration) after the first short sentence is generated based on the first length of each first short sentence. Furthermore, based on the first durations and first durations of the multiple first short sentences included in each first short sentence string, the speech synthesis system determines the user's waiting time when performing streaming speech synthesis on each first short sentence string. The waiting time is the time during which no speech is played during the streaming speech synthesis process. For example, when speech synthesis is performed based on the first short sentence in the first short sentence string, no speech is played. The time (the first duration) required for speech synthesis of the first short sentence in the first short sentence string is the user's waiting time. If the speech playback duration of the first first short sentence in the first short sentence string is two seconds, but the time (the first duration) required for speech synthesis of the second first short sentence is three seconds, then after the speech of the first first short sentence is played, the speech of the second first short sentence cannot be played immediately, and another one-second wait is required. This one-second wait is the user's waiting time.

[0030] When determining the first duration based on the first length, the speech synthesis system can determine the first duration based on the first length, the audio sampling rate, and the speech playback speed multiple, wherein the audio sampling rate and the speech playback speed are values ​​selected by the user during the streaming speech generation process, and both can be used as constants. Then, the first duration corresponding to the i-th first phrase in each first character string can be calculated according to formula C i =V i / (sr*vs) OK. When determining the user's waiting time when streaming speech synthesis is performed on each first short sentence string based on the first duration and first time consumption corresponding to each first short sentence, the speech synthesis system can determine the user waiting sub-time of each next short sentence based on the first duration of each first short sentence and the first time consumption of the next short sentence, and then determine the waiting time corresponding to the first short sentence string based on the user waiting sub-time corresponding to each first short sentence in the first short sentence string.

[0031] Step 108: Determine the first short sentence character string whose waiting time is less than a preset threshold as a target short sentence character string, and perform streaming speech synthesis based on the target short sentence character string.

[0032] After determining the waiting time corresponding to each first short sentence character string, the speech synthesis system determines the first short sentence character string whose waiting time is less than a preset threshold, and performs streaming speech synthesis based on the first short sentence character string whose waiting time is less than the preset threshold, wherein the preset threshold is a pre-set value.

[0033] Specifically, when determining the target short sentence string, the speech synthesis system can determine a first short sentence string whose waiting time is less than a preset threshold as the target short sentence string. When there are multiple first short sentence strings whose waiting time is less than the preset threshold, the speech synthesis system can determine the first short sentence string with the shortest waiting time among the multiple first short sentence strings as the target short sentence string.

[0034] The speech synthesis method provided by an embodiment of the present invention obtains a target text for speech synthesis and segments the target text using multiple preset short sentence segmentation methods to obtain multiple first short sentence character strings, each of which includes multiple first short sentences. Based on the character length of each first short sentence, a first length and a first time consumption corresponding to each first short sentence are determined using a preset correspondence table. The correspondence table pre-stores the first lengths and first time consumptions corresponding to short sentences of different character lengths. The first length is used to represent the length of a vector generated when synthesizing a short sentence with the same character length as the first short sentence, and the first time consumption is used to represent the inference time consumption required when synthesizing a short sentence with the same character length as the first short sentence. Based on the first length corresponding to each first short sentence, a first duration after speech generation for each first short sentence is determined, so as to determine a user waiting time when performing streaming speech synthesis on each first short sentence character string based on the first duration and first time consumption corresponding to each first short sentence. The first short sentence character string whose waiting time is less than a preset threshold is determined as a target short sentence character string, and streaming speech synthesis is performed based on the target short sentence character string, thereby reducing the user waiting time during the speech synthesis process.

[0035] In one implementation, the target text is segmented using a plurality of preset short sentence segmentation methods to obtain a plurality of first short sentence character strings (step 102 ), and steps A1-A4 may be performed: Step A1: Segment the target text into short sentences based on punctuation marks in the target text to obtain a second short sentence character string.

[0036] When the speech synthesis system segments the target text into short sentences based on the punctuation marks in the target text, each punctuation mark can be regarded as a segmentation point, thereby performing segmentation and obtaining a second short sentence character string. Step A2: Merge the second short sentence and the second adjacent short sentence in the second short sentence string to obtain a third short sentence string.

[0037] The second short sentence is a short sentence in which the character length of the second short sentence character string is less than a first threshold, and the second adjacent short sentence is a short sentence adjacent to the second short sentence.

[0038] The speech synthesis system obtains the second short sentence and the second adjacent short sentence in the second short sentence character string, wherein the second short sentence is a short sentence whose character length is less than a first threshold, the first threshold can be 5 or 10, and the second adjacent short sentence is a short sentence adjacent to the second short sentence, which can be a left-adjacent short sentence or a right-adjacent short sentence.

[0039] After obtaining the second short sentence and the second adjacent short sentence, the speech synthesis system merges the second short sentence and the second adjacent short sentence. In other words, the speech synthesis system merges the short sentence in the second short sentence string whose character length is less than the first threshold with its left adjacent sentence or right adjacent short sentence to obtain a merged third short sentence string.

[0040] Step A3: Acquire multiple third short sentences in the third short sentence character string and third adjacent short sentences corresponding to each of the third short sentences.

[0041] The third short sentence is any short sentence in the third short sentence character string, and the third adjacent short sentence is a short sentence adjacent to the third short sentence.

[0042] The speech synthesis system determines multiple third sentences in the third sentence string and third adjacent sentences adjacent to each third sentence, where a third sentence is any sentence in the third sentence string, and the sentence adjacent to each third sentence can be a left adjacent sentence or a right adjacent sentence. In other words, the speech synthesis system can determine all sentences in the third sentence string as third sentences and determine a third adjacent sentence corresponding to each third sentence.

[0043] Step A4: Based on a preset short sentence merging strategy, selectively merge the plurality of third short sentences in the third short sentence character string and the third adjacent short sentences corresponding to each third short sentence to obtain a plurality of first short sentence character strings.

[0044] After determining the third short sentence and the third adjacent short sentence, the speech synthesis system selectively merges the third short sentence and the third adjacent short sentence corresponding to each third short sentence based on a preset multiple short sentence merging strategy, thereby obtaining multiple first short sentence character strings.

[0045] Specifically, the preset short sentence merging strategy can be to merge short sentences whose character length is less than a threshold, or to randomly determine whether to merge (such as determining whether to merge each short sentence with a probability of 50%), and when merging, the third short sentence can be chosen to be merged with the third adjacent short sentence to its left, or the third short sentence can be chosen to be merged with the third adjacent short sentence to its right.

[0046] Furthermore, the preset correspondence table stores the first length and first time corresponding to character lengths within a certain range (for example, character lengths of 10 to 100). Therefore, when the target text is segmented into short sentences to determine multiple first short sentence character strings, or the third short sentence character string is segmented into short sentences to determine multiple first short sentence character strings, the speech synthesis system needs to ensure that the character length of each first short sentence obtained after segmentation is less than the maximum character length threshold and greater than the minimum character length threshold, for example, to ensure that the character length of each first short sentence is less than 100 and greater than 10, and thus to ensure that each short sentence can find the corresponding vector length and model inference time for generating speech in the correspondence table.

[0047] In one implementation, after merging the second short sentence and the second adjacent short sentence in the second short sentence string to obtain a third short sentence string (step A2), step B may be further performed: Step B: Merge multiple short sentences in the third short sentence string in multiple merging modes to obtain multiple first short sentence strings.

[0048] The merging manner is used to represent a manner in which multiple short sentences can be merged.

[0049] After determining the third short sentence string, the speech synthesis system can also merge the short sentences in the third short sentence string in multiple merging modes to obtain multiple first short sentence strings, where the merging modes are modes that can merge multiple short sentences.

[0050] Specifically, the speech synthesis system can determine all possible ways to merge the third short sentences in the third short sentence string, and traverse all possible ways to merge, and obtain the first short sentence string obtained by merging based on each possible way. For example, when there are n third short sentences in the third short sentence string, there are n-1 gaps, each of which can be spliced ​​or not spliced. Therefore, there are n-1 gaps to be arranged and combined, so it can be determined that there are 2 n-1 The speech synthesis system searches through various segmentation strategies to determine the first short sentence string obtained by executing each combined segmentation strategy. To reduce traversal time, n is guaranteed to be ≤ 20 in a single traversal. This ensures that the total number of segmentation strategies traversed and searched is in the millions, allowing the traversal calculation to be completed almost imperceptibly. When n is greater than 20, the third short sentence string can be segmented again to meet the segmentation conditions for convenient operation.

[0051] Figure 2 This is a flow chart of a second speech synthesis method provided in an embodiment of this specification. Figure 2 As shown, the schematic diagram includes: Step 202: Acquire target text for speech synthesis.

[0052] Step 204: Segment the target text into short sentences based on punctuation marks in the target text to obtain a second short sentence character string.

[0053] Step 206: Merge the second short sentence and the second adjacent short sentence in the second short sentence string to obtain a third short sentence string.

[0054] The second short sentence is a short sentence in which the character length of the second short sentence character string is less than a first threshold, and the second adjacent short sentence is a short sentence adjacent to the second short sentence.

[0055] Step 208: Merge multiple short sentences in the third short sentence string in multiple merging modes to obtain multiple first short sentence strings.

[0056] The merging manner is used to represent a manner in which multiple short sentences can be merged, and each of the first short sentence character strings includes multiple first short sentences.

[0057] Step 210: Based on the character length of each of the first short sentences, determine the first length and first time consumption corresponding to each of the first short sentences through a preset correspondence table.

[0058] Among them, the correspondence table pre-stores the first length and the first time corresponding to short sentences of different character lengths. The first length is used to represent the vector length generated when synthesizing a short sentence with the same character length as the first short sentence, and the first time is used to represent the inference time required when synthesizing a short sentence with the same character length as the first short sentence.

[0059] Step 212: Determine the first duration of each first short sentence after generating speech based on the first length corresponding to each first short sentence, so as to determine the user's waiting time when performing streaming speech synthesis on each first short sentence character string based on the first duration and the first time consumption corresponding to each first short sentence.

[0060] Step 214: Determine the first short sentence character string whose waiting time is less than a preset threshold as a target short sentence character string, and perform streaming speech synthesis based on the target short sentence character string.

[0061] In the embodiments of the specification, by first segmenting the target text based on punctuation marks, then determining multiple merging methods based on the segmentation results, and then traversing each merging method, and finally determining the target short sentence string based on the first short sentence string corresponding to each merging method, the target short sentence string with the shortest user waiting time can be accurately and quickly obtained, thereby reducing the user's waiting time during the streaming speech synthesis process.

[0062] In one implementation, before determining the first length and first time consumption corresponding to each first short sentence using a preset correspondence table based on the character length of each first short sentence (step 104), steps C1-C4 may be further performed: Step C1: Obtain multiple sample short sentence sets.

[0063] The character lengths of the sentences in different sample sentence sets are different, and each sample sentence set includes multiple sample sentences.

[0064] Before determining the first length and first time consumption corresponding to each first short sentence based on the preset correspondence table, the correspondence table needs to be set in advance. To determine the correspondence table, the speech synthesis system first needs to obtain multiple sample short sentence sets, where the short sentences included in each sample short sentence set have different character lengths. For example, the first sample short sentence set includes short sentences with a character length of five characters, and the second sample short sentence set includes short sentences with a character length of six characters. Each sample short sentence set includes multiple sample short sentences, such as the first sample short sentence set includes multiple short sentences with a character length of five characters.

[0065] Specifically, the speech synthesis system can obtain a large text string (for example, the character length is greater than 1000 characters), initialize the position index to 0, and then start from the position index to traverse the text string with a length of 1 to 100, and determine multiple sample short sentence sets based on the character length of the truncated text.

[0066] Step C2: Determine the average sample length of the plurality of sample sentences in each of the sample sentence sets.

[0067] The sample average length is the average vector length generated when multiple sample short sentences are synthesized into speech.

[0068] After obtaining multiple sets of sample sentences, the speech synthesis system determines the average length of the vectors generated when synthesizing speech from multiple sample sentences in each set. In other words, the speech synthesis model sequentially uses sample sentences of the same length from the sample sentence set as input to the text-to-speech (TTS) model, obtains the length of the speech vectors generated for each sample sentence, and then determines the average length of the speech vectors for the multiple sample sentences in the sample sentence set, which is the sample average length.

[0069] Step C3: Determine the average sample time consumption of the plurality of sample sentences in each sample sentence set.

[0070] The average sample time is the average inference time required to synthesize multiple sample sentences into speech.

[0071] After obtaining multiple sample sentence sets, the speech synthesis system can also determine the average model inference time required to synthesize speech for multiple sample sentences in each sample sentence set. In other words, the speech synthesis model uses sample sentences of the same character length in the sample sentence set as input to the speech synthesis model (TTS), obtains the inference time required for model inference on each sample sentence, and then determines the average model inference time required for multiple sample sentences in the sample sentence set, which is the sample average inference time.

[0072] Step C4: Determine the correspondence table based on the average sample length, the average sample time and the character length of the short sentences in the sample short sentence set, so as to determine the first length and the first time corresponding to the first short sentence based on the character length of the first short sentence.

[0073] After determining the average sample length and average sample time corresponding to each set of sample short sentences, the speech synthesis system determines a correspondence table based on the character length of the short sentences in each set of sample short sentences, that is, determines the correspondence between each character length and the average sample length and the average sample time, and generates a correspondence table based on the correspondence.

[0074] Specifically, after the speech synthesis system traverses and extracts text strings with a length of 1 to 100, the speech synthesis system can also start from the initialization index 0, and traverse the short sentence strings with a length of 1 to 100 in sequence, and use each short sentence of character length as the input of the TTS model to obtain the length of the generated speech vector and the time consumption of model inference and save them. Then, the position index is moved backward in a sliding window manner, and the speech vector lengths and model inference times corresponding to short sentences of different lengths are collected in a loop until the position index exceeds the long text range. Finally, the average value of the speech vector length and model de-inference time corresponding to the short sentences with a character length of 1 to 100 is calculated, and a corresponding relationship table is generated based on the determined average value.

[0075] Figure 3 This is a flow chart of a third speech synthesis method provided in an embodiment of this specification. Figure 3 As shown, the schematic diagram includes: Step 302: Acquire multiple sample short sentence sets.

[0076] The character lengths of the sentences in different sample sentence sets are different, and each sample sentence set includes multiple sample sentences.

[0077] Step 304: Determine the average sample length of the plurality of sample sentences in each of the sample sentence sets.

[0078] The sample average length is the average vector length generated when multiple sample short sentences are synthesized into speech.

[0079] Step 306: Determine the average sample time consumption of the plurality of sample sentences in each of the sample sentence sets.

[0080] The average sample time is the average inference time required to synthesize multiple sample sentences into speech.

[0081] Step 308: Determine the corresponding relationship table based on the average sample length, the average sample time consumption, and the character length of the short sentences in the sample short sentence set.

[0082] Step 310: Obtain a target text for speech synthesis, and segment the target text using a plurality of preset short sentence segmentation methods to obtain a plurality of first short sentence character strings.

[0083] Each of the first short sentence character strings includes multiple first short sentences.

[0084] Step 312: Based on the character length of each of the first short sentences, determine the first length and first time consumption corresponding to each of the first short sentences through a preset correspondence table.

[0085] Among them, the correspondence table pre-stores the first length and the first time corresponding to short sentences of different character lengths. The first length is used to represent the vector length generated when synthesizing a short sentence with the same character length as the first short sentence, and the first time is used to represent the inference time required when synthesizing a short sentence with the same character length as the first short sentence.

[0086] Step 314: Determine the first duration of each first short sentence after generating speech based on the first length corresponding to each first short sentence, so as to determine the user's waiting time when performing streaming speech synthesis on each first short sentence character string according to the first duration and the first time consumption corresponding to each first short sentence.

[0087] Step 316: Determine the first short sentence character string whose waiting time is less than a preset threshold as a target short sentence character string, and perform streaming speech synthesis based on the target short sentence character string.

[0088] In the embodiments of the specification, by pre-acquiring multiple sample short sentence sets and determining a correspondence table based on the sample short sentence sets, after obtaining multiple first short sentences, the corresponding first length and first duration can be determined according to the character length of the first short sentence, and then the user waiting time corresponding to each first short sentence character string can be quickly determined.

[0089] In one implementation, the determining of the first duration of each first short sentence after speech generation based on the first length corresponding to each first short sentence, so as to determine the user's waiting time when performing streaming speech synthesis on each first short sentence character string according to the first duration corresponding to each first short sentence (step 106), may include performing steps D1-D3: Step D1: Obtain the first duration corresponding to the fourth short sentence and the first time consumption corresponding to the fifth short sentence in the first short sentence string.

[0090] The fourth short sentence is any short sentence in the first short sentence character string, and the fifth short sentence is a short sentence adjacent to and after the fourth short sentence.

[0091] The speech synthesis system obtains any short sentence in the first short sentence string as the fourth short sentence, and determines its corresponding first duration, determines a short sentence adjacent to the fourth short sentence but located after the fourth short sentence as the fifth short sentence, and determines its corresponding first time consumption.

[0092] That is to say, the speech synthesis system can obtain any two adjacent short sentences in the first short sentence string, and then determine the short sentence in the front adjacent position as the fourth short sentence, and obtain the first duration corresponding to the fourth short sentence, determine the short sentence in the back adjacent position as the fifth short sentence, and obtain the first time consumption corresponding to the fifth short sentence.

[0093] Step D2: determining whether there is a pause in the streaming speech synthesis process based on the first duration corresponding to the fourth short sentence and the first time consumption corresponding to the fifth short sentence.

[0094] After determining the first duration corresponding to the fourth short sentence and the first time consumption corresponding to the fifth short sentence, the speech synthesis system can determine whether the speech corresponding to the fifth short sentence can be played immediately after the speech playback of the fourth short sentence is completed based on the first duration corresponding to the fourth short sentence and the first time consumption corresponding to the fifth short sentence. For example, when the first duration of the fourth short sentence is 2 seconds but the first time consumption of the fifth short sentence is 3 seconds, the speech corresponding to the fifth short sentence cannot be played immediately after the speech playback of the fourth short sentence is completed, and it is necessary to wait again for 1 second, so it can be determined that there is a pause; when the first duration of the fourth short sentence is 2 seconds but the first time consumption of the fifth short sentence is 1 second, the speech corresponding to the fifth short sentence can be played immediately after the speech playback of the fourth short sentence is completed, and there is no need to wait, so it can be determined that there is no pause.

[0095] The first short sentence string includes multiple fourth short sentences, and therefore also includes multiple fifth short sentences corresponding to the fourth short sentences. The speech synthesis system determines whether there is a pause in the entire first short sentence string based on the first duration of multiple (or all) fourth short sentences in the first short sentence string and the first duration of the fifth short sentence corresponding to each fourth short sentence. If there is no pause in multiple (or all) fourth short sentences and fifth short sentences, then it is determined that there is no pause in the streaming speech synthesis process based on the first short sentence string. If there is a pause in any of the fourth short sentences and the fifth short sentences, then it is determined that there is a pause in the streaming speech synthesis process based on the first short sentence string.

[0096] Specifically, the speech synthesis system can also determine whether there is a pause in the process of streaming speech synthesis based on the playback revenue time of the fourth short sentence and the playback revenue time of the fourth short sentence, wherein the playback revenue time is used to characterize the maximum time consumed by the fifth short sentence model inference. In other words, the speech synthesis system can determine whether the playback revenue time corresponding to the fourth short sentence is greater than the first time consumed corresponding to the fifth short sentence, thereby determining that there is no pause when the playback revenue time corresponding to the fourth short sentence is greater than the first time consumed corresponding to the fifth short sentence, and determining that there is a pause when the playback revenue time corresponding to the fourth short sentence is less than the first time consumed corresponding to the fifth short sentence. The playback revenue time determination formula for the fourth short sentence can be: G i =C1 (when i=1); G i =G i-1 -T i +C i (When i>1), C1 is the first duration corresponding to the first short sentence in the first short sentence string, G i-1 is the playback revenue time of the i-1th short sentence, T i is the first time consumption of the i-th short sentence, C i is the first duration corresponding to the i-th short sentence.

[0097] Step D3: If not, determining the user's waiting time when each of the first short sentence character strings is subjected to streaming speech synthesis.

[0098] After determining that there is no pause in the first short sentence string, the speech synthesis system further determines a user waiting time when performing streaming speech synthesis on the first short sentence string. Specifically, when there is no pause in the first short sentence string, the user waiting time is the first time taken for the first short sentence in the first short sentence string.

[0099] That is, the speech synthesis system will filter out the first short sentence strings that have pauses during the synthesis process, and only determine the first short sentence strings without pauses. When traversing each first short sentence string, the speech synthesis system can determine whether the current strategy is the first first short sentence string without pauses. If so, it saves and updates the user's optimal waiting time; if it has been updated before, it compares the current optimal waiting time with the saved optimal waiting time, determines to update the optimal strategy, and then continues to process the next first short sentence string until all first short sentence strings are traversed, obtains the optimal waiting time, and performs streaming speech synthesis based on the first short sentence string corresponding to the optimal waiting time.

[0100] In one implementation, steps E1-E2 may also be performed: Step E1: Obtain the second length and second time consumption corresponding to each of the first short sentences in the target character string.

[0101] Among them, the second length is the length of the speech vector generated when synthesizing the first short sentence, and the second time consumption is the inference time consumption required when synthesizing the first short sentence.

[0102] When the speech synthesis system performs streaming speech synthesis based on the target character string, the speech synthesis system obtains the second length and second time corresponding to the first short sentence in the target character string, wherein the second length is the length of the speech vector generated by the speech synthesis model when performing speech synthesis based on the first short sentence, and the second time is the model inference time required for the speech synthesis model to perform speech synthesis based on the first short sentence, that is, the speech synthesis system obtains the length of the speech vector and the model inference time generated when each first short sentence is actually synthesized.

[0103] That is, under the current TTS model and operating environment, the speech synthesis model uses each short sentence as the input of the TTS model, performs model inference, and adds the speech vector output by the model to the streaming speech data. During the TTS model inference process for each short sentence, the speech synthesis model records the actual time taken T r And the length of the final generated speech vector V r .

[0104] Step E2: updating the correspondence table based on the second length and the second time consumption.

[0105] After obtaining the second length and the second time, the speech synthesis system updates the first length and the first time corresponding to different characters in the preset correspondence table based on the second length and the second time, and obtains the updated correspondence table, and then determines the next target short sentence character string according to the updated correspondence table.

[0106] Specifically, the speech synthesis model can update the first length and first time based on the second length, the second time, and a preset update weight. This can be done according to the formulas Vi = (1-a)Vi + aVr; Ti = (1-a)Ti + aTr, where Vi and Ti are the first length and first time corresponding to the character length, respectively, pre-set in the correspondence table. a represents the weight ratio of the actual value updated once in the subsequent estimation process. This can be determined based on the difference between the current operating environment and the historical data collection. If the current environment has changed significantly from the historical environment, the value of a can be increased to quickly adapt to the current environment. Conversely, if the current environment has not changed much from the historical environment, the value of a can be decreased. The value range of a is 0≤a≤1.

[0107] Figure 4 This is a flow chart of a fourth speech synthesis method provided in an embodiment of this specification. Figure 4 As shown, the schematic diagram includes: Step 402: Obtain a target text for speech synthesis, and segment the target text using a plurality of preset short sentence segmentation methods to obtain a plurality of first short sentence character strings.

[0108] Each of the first short sentence character strings includes multiple first short sentences.

[0109] Step 404: Based on the character length of each of the first short sentences, determine the first length and first time consumption corresponding to each of the first short sentences through a preset correspondence table.

[0110] Among them, the correspondence table pre-stores the first length and the first time corresponding to short sentences of different character lengths. The first length is used to represent the vector length generated when synthesizing a short sentence with the same character length as the first short sentence, and the first time is used to represent the inference time required when synthesizing a short sentence with the same character length as the first short sentence.

[0111] Step 406: Obtain the first duration corresponding to the fourth short sentence and the first time consumption corresponding to the fifth short sentence in the first short sentence string.

[0112] The fourth short sentence is any short sentence in the first short sentence character string, and the fifth short sentence is a short sentence adjacent to and after the fourth short sentence.

[0113] Step 408: Determine whether there is a pause in the streaming speech synthesis process based on the first duration corresponding to the fourth short sentence and the first time consumption corresponding to the fifth short sentence.

[0114] Step 410: If not, determining the user's waiting time when each of the first short sentence strings is subjected to streaming speech synthesis.

[0115] Step 412: Determine the first short sentence character string whose waiting time is less than a preset threshold as a target short sentence character string, and perform streaming speech synthesis based on the target short sentence character string.

[0116] Step 414: Obtain the second length and second time consumption corresponding to each of the first short sentences in the target character string.

[0117] Among them, the second length is the length of the speech vector generated when synthesizing the first short sentence, and the second time consumption is the inference time consumption required when synthesizing the first short sentence.

[0118] Step 416: Update the correspondence table based on the second length and the second time consumption.

[0119] In the embodiments of the specification, by determining whether there is a pause during the streaming speech synthesis process of each first short sentence character string, and determining the target short sentence character string among multiple first short sentence character strings without pauses, it is possible to ensure that the generated speech is continuous and uninterrupted. By obtaining the second length and second time corresponding to the first short sentence, and updating the preset correspondence table based on the second length and second time, it is possible to increase the accuracy of the determined user waiting time and the accuracy of the determined target short sentence character string.

[0120] It should be noted that the speech synthesis method provided in the embodiments of the present application can be executed by a speech synthesis device or a control module in the speech synthesis device for executing the speech synthesis method. In the embodiments of the present application, the speech synthesis device provided in the embodiments of the present application is described by taking the speech synthesis device executing the speech synthesis method as an example.

[0121] Figure 5 FIG is a schematic diagram of the structure of a speech synthesis device according to an embodiment of the present invention. Figure 5 As shown, the speech synthesis apparatus includes: a first acquisition module 502 , a first determination module 504 , a second determination module 506 , and a first synthesis module 508 .

[0122] A first acquisition module 502 is configured to acquire a target text for speech synthesis and segment the target text using a plurality of preset short sentence segmentation methods to obtain a plurality of first short sentence character strings, each of which includes a plurality of first short sentences; A first determining module 504 is configured to determine, based on the character length of each of the first short sentences, a first length and a first time consumption corresponding to each of the first short sentences using a preset correspondence table, wherein the correspondence table pre-stores the first lengths and first time consumptions corresponding to short sentences of different character lengths, wherein the first length is used to represent the length of a vector generated when synthesizing a short sentence of the same character length as the first short sentence, and the first time consumption is used to represent the inference time consumption required to synthesize a short sentence of the same character length as the first short sentence; A second determining module 506 is configured to determine, based on the first length corresponding to each of the first short sentences, a first duration after each of the first short sentences is generated into speech, so as to determine a user waiting time when performing streaming speech synthesis on each of the first short sentence strings according to the first duration corresponding to each of the first short sentences; The first synthesis module 508 is configured to determine the first short sentence character string whose waiting time is less than a preset threshold as a target short sentence character string, and perform streaming speech synthesis based on the target short sentence character string.

[0123] The speech synthesis device in the embodiments of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, the mobile electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), while the non-mobile electronic device can be a server, network attached storage (NAS), personal computer (PC), television, ATM, or self-service machine, etc., without specific limitations in the embodiments of the present application.

[0124] The speech synthesis device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0125] The speech synthesis device provided in the embodiment of the present application can achieve Figures 1 to 4 To avoid repetition, the various processes implemented in the method embodiment will not be described again here.

[0126] Based on the same technical concept, an embodiment of the present application further provides an electronic device, which is used to execute the above-mentioned speech synthesis method. Figure 6The following is a schematic diagram of the structure of an electronic device for implementing various embodiments of the present application. Electronic devices may vary significantly due to different configurations or performance, and may include a processor 602, a communications interface 604, a memory 606, and a communication bus 608. The processor 602, the communications interface 604, and the memory 606 communicate with each other via the communication bus 608. The processor 602 may call a computer program stored in the memory 606 and executable on the processor 602 to perform the following steps: Acquiring a target text for speech synthesis, and segmenting the target text using a plurality of preset short sentence segmentation methods to obtain a plurality of first short sentence character strings, each of which includes a plurality of first short sentences; Based on the character length of each of the first short sentences, determining a first length and a first time consumption corresponding to each of the first short sentences using a preset correspondence table, wherein the correspondence table pre-stores the first lengths and first time consumptions corresponding to short sentences of different character lengths, wherein the first length is used to represent the length of a vector generated when synthesizing a short sentence of the same character length as the first short sentence, and the first time consumption is used to represent the inference time consumption required to synthesize a short sentence of the same character length as the first short sentence; determining, based on the first length corresponding to each of the first short sentences, a first duration after speech generation for each of the first short sentences, and determining, based on the first duration and the first time consumption corresponding to each of the first short sentences, a user's waiting time when performing streaming speech synthesis on each of the first short sentence character strings; The first short sentence character string whose waiting time is less than a preset threshold is determined as a target short sentence character string, and streaming speech synthesis is performed based on the target short sentence character string.

[0127] In one implementation, the target text is segmented using a plurality of preset short sentence segmentation methods to obtain a plurality of first short sentence character strings, including: Segmenting the target text into short sentences based on punctuation marks in the target text to obtain a second short sentence character string; merging the second short sentence and a second adjacent short sentence in the second short sentence string to obtain a third short sentence string, wherein the second short sentence is a short sentence in the second short sentence string whose character length is less than a first threshold, and the second adjacent short sentence is a short sentence adjacent to the second short sentence; Acquire multiple third short sentences in the third short sentence string and third adjacent short sentences corresponding to each third short sentence, wherein the third short sentence is any short sentence in the third short sentence string, and the third adjacent short sentences are short sentences adjacent to the third short sentence; Based on a preset short sentence merging strategy, the plurality of third short sentences in the third short sentence character string and the third adjacent short sentences corresponding to each third short sentence are selectively merged to obtain a plurality of first short sentence character strings.

[0128] In one implementation, after merging the second short sentence and the second adjacent short sentence in the second short sentence string to obtain a third short sentence string, the method further includes: The multiple short sentences in the third short sentence character string are merged according to multiple merging modes to obtain multiple first short sentence character strings, where the merging modes are used to represent the ways in which the multiple short sentences can be merged.

[0129] In one implementation, before determining the first length and first time consumption corresponding to each of the first sentences using a preset correspondence table based on the character length of each of the first sentences, the method further includes: Acquire multiple sample short sentence sets, where the short sentences in different sample short sentence sets have different character lengths, and each sample short sentence set includes multiple sample short sentences; Determining an average sample length of a plurality of the sample sentences in each of the sample sentence sets, wherein the average sample length is an average vector length generated when the plurality of the sample sentences are synthesized into speech; Determining an average sample time of a plurality of the sample sentences in each of the sample sentence sets, where the average sample time is an average inference time required to synthesize the plurality of the sample sentences into speech; The correspondence table is determined based on the sample average length, the sample average time consumption and the character length of the short sentences in the sample short sentence set, so as to determine the first length and the first time consumption corresponding to the first short sentence based on the character length of the first short sentence.

[0130] In one implementation, determining, based on the first length corresponding to each first short sentence, a first duration after speech generation for each first short sentence, so as to determine, based on the first duration corresponding to each first short sentence, a user's waiting time when performing streaming speech synthesis on each first short sentence character string, includes: Obtaining the first duration corresponding to a fourth short sentence and the first time consumption corresponding to a fifth short sentence in the first short sentence string, where the fourth short sentence is any short sentence in the first short sentence string, and the fifth short sentence is a short sentence adjacent to and after the fourth short sentence; determining whether there is a pause in the streaming speech synthesis process based on the first duration corresponding to the fourth short sentence and the first time consumption corresponding to the fifth short sentence; If not, the waiting time of the user when performing streaming speech synthesis on each of the first short sentence character strings is determined.

[0131] In one implementation, the method further includes: Obtaining a second length and a second time corresponding to each of the first short sentences in the target character string, where the second length is the length of the speech vector generated when synthesizing the first short sentence, and the second time is the inference time required for synthesizing the first short sentence; The correspondence table is updated based on the second length and the second time consumption.

[0132] The specific execution steps can refer to the various steps of the above-mentioned speech synthesis method embodiment, and can achieve the same technical effect. To avoid repetition, they will not be described here.

[0133] It should be noted that the electronic devices in the embodiments of the present application include: servers, terminals, or other devices other than terminals.

[0134] The above electronic device structure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, the input unit may include a graphics processing unit (GPU) and a microphone, and the display unit may be configured as a display panel in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit includes at least one of a touch panel and other input devices. A touch panel is also called a touch screen. Other input devices may include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick, which will not be detailed here.

[0135] The memory can be used to store software programs and various data. The memory may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory may include volatile memory or non-volatile memory, or the memory may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus random access memory (DRRAM).

[0136] The processor may include one or more processing units; optionally, the processor may integrate an application processor and a modem processor, wherein the application processor primarily handles operations related to the operating system, user interface, and application programs, and the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into the processor.

[0137] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned speech synthesis method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0138] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.

[0139] An embodiment of the present application also provides a computer program product, which, when executed by a processor, implements the various processes of the above-mentioned speech synthesis method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0140] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned speech synthesis method embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0141] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0142] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0143] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of this application.

[0144] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A speech synthesis method, characterized in that: include: Acquiring a target text for speech synthesis, and segmenting the target text using a plurality of preset short sentence segmentation methods to obtain a plurality of first short sentence character strings, each of which includes a plurality of first short sentences; Based on the character length of each of the first short sentences, determining a first length and a first time consumption corresponding to each of the first short sentences using a preset correspondence table, wherein the correspondence table pre-stores the first lengths and first time consumptions corresponding to short sentences of different character lengths, wherein the first length is used to represent the length of a vector generated when synthesizing a short sentence of the same character length as the first short sentence, and the first time consumption is used to represent the inference time consumption required to synthesize a short sentence of the same character length as the first short sentence; determining, based on the first length corresponding to each of the first short sentences, a first duration after speech generation for each of the first short sentences, and determining, based on the first duration and the first time consumption corresponding to each of the first short sentences, a user's waiting time when performing streaming speech synthesis on each of the first short sentence character strings; The first short sentence character string whose waiting time is less than a preset threshold is determined as a target short sentence character string, and streaming speech synthesis is performed based on the target short sentence character string.

2. The method according to claim 1, characterized in that The target text is segmented using a plurality of preset short sentence segmentation methods to obtain a plurality of first short sentence character strings, including: Segmenting the target text into short sentences based on punctuation marks in the target text to obtain a second short sentence character string; merging the second short sentence and a second adjacent short sentence in the second short sentence string to obtain a third short sentence string, wherein the second short sentence is a short sentence in the second short sentence string whose character length is less than a first threshold, and the second adjacent short sentence is a short sentence adjacent to the second short sentence; Acquire multiple third short sentences in the third short sentence string and third adjacent short sentences corresponding to each third short sentence, wherein the third short sentence is any short sentence in the third short sentence string, and the third adjacent short sentences are short sentences adjacent to the third short sentence; Based on a preset short sentence merging strategy, the plurality of third short sentences in the third short sentence character string and the third adjacent short sentences corresponding to each third short sentence are selectively merged to obtain a plurality of first short sentence character strings.

3. The method according to claim 2, characterized in that After merging the second short sentence and the second adjacent short sentence in the second short sentence string to obtain a third short sentence string, the method further includes: The multiple short sentences in the third short sentence character string are merged according to multiple merging modes to obtain multiple first short sentence character strings, where the merging modes are used to represent the ways in which the multiple short sentences can be merged.

4. The method according to claim 1, wherein Before determining the first length and first time consumption corresponding to each of the first sentences using a preset correspondence table based on the character length of each of the first sentences, the method further includes: Acquire multiple sample short sentence sets, where the short sentences in different sample short sentence sets have different character lengths, and each sample short sentence set includes multiple sample short sentences; Determining an average sample length of a plurality of the sample sentences in each of the sample sentence sets, wherein the average sample length is an average vector length generated when the plurality of the sample sentences are synthesized into speech; Determining an average sample time of a plurality of the sample sentences in each of the sample sentence sets, where the average sample time is an average inference time required to synthesize the plurality of the sample sentences into speech; The correspondence table is determined based on the sample average length, the sample average time consumption and the character length of the short sentences in the sample short sentence set, so as to determine the first length and the first time consumption corresponding to the first short sentence based on the character length of the first short sentence.

5. The method according to claim 1, wherein The determining, based on the first length corresponding to each of the first short sentences, a first duration after each of the first short sentences generates speech, so as to determine a user waiting time when performing streaming speech synthesis on each of the first short sentence strings according to the first duration corresponding to each of the first short sentences, includes: Obtaining the first duration corresponding to a fourth short sentence and the first time consumption corresponding to a fifth short sentence in the first short sentence string, where the fourth short sentence is any short sentence in the first short sentence string, and the fifth short sentence is a short sentence adjacent to and after the fourth short sentence; determining whether there is a pause in the streaming speech synthesis process based on the first duration corresponding to the fourth short sentence and the first time consumption corresponding to the fifth short sentence; If not, the waiting time of the user when performing streaming speech synthesis on each of the first short sentence character strings is determined.

6. The method according to claim 1, characterized in that The method further comprises: Obtaining a second length and a second time corresponding to each of the first short sentences in the target character string, where the second length is the length of the speech vector generated when synthesizing the first short sentence, and the second time is the inference time required for synthesizing the first short sentence; The correspondence table is updated based on the second length and the second time consumption.

7. A speech synthesis device, characterized in that: include: a first acquisition module, configured to acquire a target text for speech synthesis and segment the target text using a plurality of preset short sentence segmentation methods to obtain a plurality of first short sentence character strings, each of which includes a plurality of first short sentences; a first determining module, configured to determine, based on the character length of each of the first short sentences, a first length and a first time consumption corresponding to each of the first short sentences using a preset correspondence table, wherein the correspondence table pre-stores the first lengths and first time consumptions corresponding to short sentences of different character lengths, wherein the first length is used to represent the length of a vector generated when synthesizing a short sentence of the same character length as the first short sentence, and the first time consumption is used to represent the inference time consumption required to synthesize a short sentence of the same character length as the first short sentence; a second determining module, configured to determine, based on the first length corresponding to each of the first short sentences, a first duration after each of the first short sentences is generated into speech, so as to determine, based on the first duration corresponding to each of the first short sentences, a user's waiting time when performing streaming speech synthesis on each of the first short sentence strings; The first synthesis module is configured to determine the first short sentence character string whose waiting time is less than a preset threshold as a target short sentence character string, and perform streaming speech synthesis based on the target short sentence character string.

8. A computer device, characterized in that: The device comprises: processor; and A memory arranged to store computer-executable instructions, wherein the executable instructions are configured to be executed by the processor, and the executable instructions include instructions for executing the steps in the speech synthesis method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is used to store computer-executable instructions, and the executable instructions enable a computer to execute the speech synthesis method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the speech synthesis method according to any one of claims 1 to 6 when executed by a processor.

Images