Speech synthesis method and device, electronic equipment and storage medium

Abstract

The invention provides a speech synthesis method and device, electronic equipment and a storage medium. The voice synthesis method comprises the following steps: segmenting an audio signal according to the duration of the audio signal prompted by voice to obtain at least one audio segment; for each audio segment, determining a mute segment and a non-mute segment in the audio segment, and in the mute segments in the audio segment, determining a target mute segment in the audio segment; removing a target mute segment in each audio segment to obtain an effective audio segment; according to the duration of the non-mute segment in each effective audio segment, a voice prompt signal is determined in all the effective audio segments, and the voice prompt signal comprises at least one effective audio segment with the longest duration of the non-mute segment; and inputting a first text and the voice prompt signal into a voice synthesis model so as to output a synthesized voice which conforms to the voice prompt signal and corresponds to the first text. According to the scheme, the speech synthesis quality can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of speech signal processing, and in particular to a speech synthesis method, a speech synthesis device, an electronic device, a storage medium and a computer program product. Background Art

[0002] With the rapid development of speech synthesis models, some speech synthesis models have attracted widespread attention because they can still generate natural speech when there is a lack of specific target speaker data. Speech synthesis models usually rely on voice prompts to generate speech to obtain synthesized natural speech. Speech synthesis models can analyze voice prompts and then generate corresponding natural speech.

[0003] With existing technologies, speech synthesis is performed using voice prompts and speech synthesis models, but the quality of the generated natural speech is difficult to meet user needs. Summary of the invention

[0004] The present invention has been made in view of the above-mentioned problems.

[0005] According to the first aspect of the present invention, a speech synthesis method is provided. The method comprises: according to the duration of the audio signal of the voice prompt, the audio signal is segmented to obtain at least one audio segment; for each audio segment, the silent segment and the non-silent segment in the audio segment are determined, and in the silent segment in the audio segment, the target silent segment in the audio segment is determined; the target silent segment in each audio segment is removed to obtain a valid audio segment; according to the duration of the non-silent segment in each valid audio segment, a voice prompt signal is determined in all valid audio segments, and the voice prompt signal includes at least one valid audio segment with the longest duration of the non-silent segment; a first text and the voice prompt signal are input into a speech synthesis model to output a synthesized speech that conforms to the voice prompt signal and corresponds to the first text.

[0006] Exemplarily, determining the voice prompt signal in all valid audio segments according to the duration of the non-silent segments in each valid audio segment includes: determining a first duration threshold according to the duration of the audio signal; sorting all valid audio segments at least according to the duration of their respective non-silent segments to obtain an audio segment queue; in the audio segment queue, determining n consecutive valid audio segments starting from the first end of the audio segment queue as the voice prompt signal, wherein the sum of the durations of the non-silent segments in the valid audio segments in the voice prompt signal is equal to or greater than the first duration threshold, the sum of the durations of the non-silent segments in n-1 valid audio segments out of the n valid audio segments is less than the first duration threshold, and in the audio segment queue, in the direction from the first end to the second end of the audio queue, there are at least adjacent valid audio segments whose durations change from long to short.

[0007] Exemplarily, the method of sorting all valid audio segments at least according to the duration of their respective non-silent segments to obtain an audio segment queue includes: determining a text sequence corresponding to each valid audio segment; searching for each word in each text sequence in a vocabulary library; for the text sequence corresponding to each valid audio segment, calculating a first ratio between the number of words in the text sequence found in the vocabulary library and the total number of words in the text sequence; sorting the valid audio segments whose first ratio is greater than a ratio threshold from long to short according to the duration of their respective non-silent segments, and arranging the valid audio segments whose first ratio is less than or equal to the ratio threshold in order of the duration of their respective non-silent segments from long to short after the valid audio segments whose first ratio is greater than the ratio threshold.

[0008] Exemplarily, determining the first duration threshold according to the duration of the audio signal and the first ratio includes: determining the first duration threshold according to the following formula:

[0009] T1=ɑ×T

[0010] Wherein, T1 represents the first duration threshold, T represents the duration of the audio signal, ɑ represents the first coefficient, and 0<ɑ≤1.

[0011] Exemplarily, determining a target silent segment in the audio segment from among the silent segments in the audio segment comprises: based on the duration of the silent segments in the audio segment, taking a silent segment having a duration greater than a second duration threshold as a target silent segment in the audio segment.

[0012] Exemplarily, the method of taking a silence segment with a duration greater than a second duration threshold as a target silence segment based on the duration of the silence segment in the audio segment includes: determining an average duration of the silence segments in the audio segment; determining the second duration threshold based on the average duration; and determining a silence segment with a duration greater than the second duration threshold in the audio segment as a target silence segment in the audio segment.

[0013] Exemplarily, determining the second duration threshold based on the average duration includes: determining a standard deviation based on the average duration and the duration of each silent segment in the audio segment; and determining the second duration threshold based on the standard deviation and the average duration.

[0014] Exemplarily, determining the second duration threshold according to the standard deviation and the average duration includes: determining the second duration threshold according to the following formula:

[0015] T2=μ+k*σ

[0016] Wherein, T2 represents the second duration threshold, μ represents the average duration, k represents the second coefficient, and σ represents the standard deviation.

[0017] Exemplarily, for each audio segment, determining a silent segment in the audio segment includes: determining a second text corresponding to the audio segment; determining a phoneme sequence based on the second text; aligning the acoustic features of the audio segment with the text features of the phoneme sequence to determine the start and end times of each phoneme in the phoneme sequence in the audio segment; and determining a time period in the audio segment without a corresponding phoneme based on the start and end times of each phoneme in the audio segment as a silent segment in the audio segment.

[0018] Exemplarily, determining the second text corresponding to the audio segment includes: inputting the audio segment into an automatic speech recognition model to output the second text.

[0019] Exemplarily, the audio signal is divided according to the duration of the audio signal of the voice prompt to obtain at least one audio segment, including: when the duration of the audio signal is greater than a second duration threshold, performing voice activity detection (VAD) on the audio signal to determine a silent segment in the audio signal; using at least a part of the determined moments in the silent segment as dividing points, to divide the audio signal into multiple audio segments, wherein the duration of each audio segment is not less than a third duration threshold.

[0020] According to a second aspect of the present invention, there is also provided a speech synthesis device, comprising:

[0021] A segmentation module, configured to segment the audio signal according to the duration of the audio signal of the voice prompt to obtain at least one audio segment;

[0022] A detection module, configured to determine, for each audio segment, a silent segment and a non-silent segment in the audio segment, and, among the silent segments in the audio segment, determine a target silent segment in the audio segment;

[0023] A filtering module is used to remove the target silent segment in each audio segment to obtain a valid audio segment;

[0024] A selection module, configured to determine a voice prompt signal in all valid audio segments according to the duration of the non-silent segment in each valid audio segment, wherein the voice prompt signal includes at least one valid audio segment with the longest duration of the non-silent segment;

[0025] The synthesis module is used to input the first text and the voice prompt signal into a speech synthesis model to output a synthesized speech that conforms to the voice prompt signal and corresponds to the first text.

[0026] According to a third aspect of the present invention, there is also provided an electronic device, comprising: a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used to execute the above-mentioned speech synthesis method when the processor is running.

[0027] According to a fourth aspect of the present invention, a storage medium is further provided, on which program instructions are stored, and the program instructions are used to execute the above-mentioned speech synthesis method when running.

[0028] According to a fifth aspect of the present invention, there is also provided a computer program product, comprising computer program instructions, wherein the computer program instructions are used to execute the above-mentioned speech synthesis method when running.

[0029] In the above technical scheme, the audio signal is segmented according to the duration of the audio signal of the voice prompt to obtain at least one audio segment. Then for each audio segment, the silent segment and the non-silent segment in the audio segment are determined, and in the silent segment in the audio segment, the target silent segment in the audio segment is determined. Then the target silent segment in each audio segment is removed to obtain a valid audio segment. According to the duration of the non-silent segment in each valid audio segment, at least one valid audio segment with the longest duration of the non-silent segment is determined as a voice prompt signal among all valid audio segments. Finally, the first text and the voice prompt signal are input into a speech synthesis model to output a synthesized speech that conforms to the voice prompt signal and corresponds to the first text. The voice prompt signal with some silent segments removed and the duration shortened contains less invalid information. When the voice prompt signal is used for speech synthesis, the calculation error of the speech model can be reduced to improve the quality of the synthesized speech.

[0030] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and other purposes, features and advantages of the present invention will become more apparent by describing the embodiments of the present invention in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0032] Figure 1 A schematic flow chart of a speech synthesis method according to an embodiment of the present invention is shown;

[0033] Figure 2 A schematic flow chart of determining a voice prompt signal in all valid audio segments according to an embodiment of the present invention is shown;

[0034] Figure 3 A schematic flow chart of sorting all valid audio segments at least according to the duration of their respective non-silent segments to obtain an audio segment queue according to one embodiment of the present invention is shown;

[0035] Figure 4 A schematic flow chart of taking a silence segment whose duration is greater than a second duration threshold as a target silence segment in the audio segment according to an embodiment of the present invention is shown;

[0036] Figure 5 A schematic flow chart of determining a second duration threshold according to an average duration according to an embodiment of the present invention is shown;

[0037] Figure 6 A schematic flow chart of determining a silent segment in each audio segment according to an embodiment of the present invention is shown;

[0038] Figure 7 A schematic flow chart showing a method of dividing an audio signal according to the duration of an audio signal of a voice prompt to obtain at least one audio segment according to an embodiment of the present invention;

[0039] Figure 8 A schematic block diagram of a speech synthesis device according to an embodiment of the present invention is shown;

[0040] Fig. 9 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical scheme and advantages of the present invention more obvious, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the protection scope of the present invention.

[0042] Voice prompts used for speech synthesis are usually long and contain a lot of invalid information, which is not conducive to generating synthesized speech that meets user needs.

[0043] In order to at least partially solve the above problems, a speech synthesis method is proposed. The method divides the audio signal of the voice prompt into at least one audio segment, then removes part of the silent segments in each audio segment to obtain a valid audio segment, and determines the voice prompt signal in all valid audio segments according to the duration of the non-silent segments in each valid audio segment, and finally inputs the determined voice prompt signal and the first text into a speech synthesis model to obtain a synthesized speech that conforms to the voice prompt signal and corresponds to the first text. By removing the target silent segment, some invalid information can be removed, and by dividing into multiple audio segments, the duration of an overly long audio signal can be shortened. Using a voice prompt signal with less invalid information and not too long for speech synthesis can improve the effect of the synthesized speech.

[0044] Figure 1 FIG. 4 shows a schematic flow chart of a speech synthesis method according to an embodiment of the present invention. Figure 1 As shown, the speech recognition method may include steps S110 to S150.

[0045] Step S110: segment the audio signal according to the duration of the audio signal of the voice prompt to obtain at least one audio segment.

[0046] The audio signal of the voice prompt can be an audio signal of the sound emitted by a real person according to the voice prompt text. For example, when the voice prompt text is "Today is Sunday, so you can have a rest", the voice prompt text can be read aloud by a real person, and the pronunciation of the real person when reading the voice prompt text can be collected as an audio signal. It can be understood that for the same voice prompt text, the tone and pitch when different people read aloud may be different, so the audio signal of the voice prompt can also include the tone, pitch and other related voice information of the reader. The duration of the audio signal of the voice prompt can be relatively long, so that it can be divided into multiple audio segments, and the duration of each audio segment is long enough. When the duration of the audio signal of the voice prompt is too short, the audio signal can be directly used as an audio segment.

[0047] For example, the audio signal may be evenly divided into at least one audio segment of the same duration.

[0048] For example, the audio segment where the invalid information in the audio signal is located can be detected first, and the audio signal can be divided into at least one audio segment according to the time node of the audio segment where the invalid information is located. There is usually no valid voice information in the silent segment. For example, the audio signal can be divided into at least one audio segment by determining the time node from the silent segment of the audio signal.

[0049] For example, the audio signal may be segmented using a neural network model to obtain at least one audio segment.

[0050] Step S120: for each audio segment, determine the silent segments and non-silent segments in the audio segment, and among the silent segments in the audio segment, determine the target silent segment in the audio segment.

[0051] For example, the silent segment and the non-silent segment in each audio segment can be determined according to the energy change in the audio segment. The energy value of the silent segment is usually low, and the energy value of the non-silent segment is usually high. For example, a first energy threshold can be set, and the part of the audio segment with energy greater than the first energy threshold is regarded as the non-silent segment, and the part of the audio segment with energy less than the first energy threshold is regarded as the silent segment.

[0052] For example, the silent segment and the non-silent segment in each audio segment may be determined according to the frequency spectrum characteristics in each audio segment.

[0053] For example, each audio segment may be analyzed using a machine learning model or a deep learning model to determine silent segments and non-silent segments in each audio segment.

[0054] For example, the silent segment and the non-silent segment may be distinguished according to the statistical characteristics in each audio segment.

[0055] For example, a duration threshold may be set, and when the duration of a silence segment in the audio segment is greater than the duration threshold, the silence segment may be used as a target silence segment in the audio segment.

[0056] For example, a second energy threshold may be set, and a silence segment with energy lower than the second energy threshold is used as a target silence segment in the audio segment. The second energy threshold may be less than or equal to the first energy threshold.

[0057] Step S130: remove the target silent segment in each audio segment to obtain a valid audio segment.

[0058] It can be understood that for each audio segment after removing the target silent segment, a valid audio segment can be obtained. Although the target silent segment is removed, the target silent segment is only a part of the silent segment, and there may still be some silent segments in the valid audio segment. However, these silent segments usually do not affect the voice information in the valid audio segment too much.

[0059] Step S140, determining a voice prompt signal in all valid audio segments according to the duration of the non-silent segment in each valid audio segment, the voice prompt signal including at least one valid audio segment with the longest duration of the non-silent segment.

[0060] The longer the duration of the non-silent segment is, the more effective voice information is contained in the effective audio segment, which is more conducive to the effect of speech synthesis.

[0061] For example, at least one valid audio segment with the longest duration of the non-silent segment may be directly determined as the voice prompt signal based only on the duration of the non-silent segment.

[0062] For example, at least one valid audio segment with the longest duration of the non-silent segment can be determined as a voice prompt signal under a specified rule. For example, some valid audio segments can be selected as voice prompt signals first, and when the number of these valid audio segments is insufficient to serve as the required voice prompt signals, at least one valid audio segment with the longest duration of the non-silent segment can be determined from other valid audio segments as a voice prompt signal. The voice prompt signal is an audio signal.

[0063] Step S150: input the first text and the voice prompt signal into a speech synthesis model to output a synthesized speech that conforms to the voice prompt signal and corresponds to the first text.

[0064] The determined voice prompt signal can be pre-stored in a designated location. When the first text is input into the speech synthesis model, the speech synthesis model can automatically obtain the desired voice prompt signal and output a synthesized speech that conforms to the voice prompt signal and corresponds to the first text. For example, the speech synthesis model can include a rule-based TTS model, an SPSS model, FastSpeech, VITS, etc.

[0065] In the above technical scheme, the audio signal is segmented according to the duration of the audio signal of the voice prompt to obtain at least one audio segment. Then for each audio segment, the silent segment and the non-silent segment in the audio segment are determined, and in the silent segment in the audio segment, the target silent segment in the audio segment is determined. Then the target silent segment in each audio segment is removed to obtain a valid audio segment. According to the duration of the non-silent segment in each valid audio segment, at least one valid audio segment with the longest duration of the non-silent segment is determined as a voice prompt signal among all valid audio segments. Finally, the first text and the voice prompt signal are input into a speech synthesis model to output a synthesized speech that conforms to the voice prompt signal and corresponds to the first text. The voice prompt signal with some silent segments removed and the duration shortened contains less invalid information. When the voice prompt signal is used for speech synthesis, the calculation error of the speech model can be reduced to improve the quality of the synthesized speech.

[0066] Figure 2 FIG. 4 is a schematic flow chart of determining a voice prompt signal in all valid audio segments according to an embodiment of the present invention. Figure 2 As shown, the above step S140 may include steps S210 to S230.

[0067] In step S210, a first duration threshold is determined according to the duration of the audio signal.

[0068] The first duration threshold can be determined based only on the duration of the audio signal and the expected coefficient. The first duration threshold represents the lower limit of the duration of the expected non-silent segment. The first duration threshold is usually less than or equal to the duration of the audio signal.

[0069] Exemplarily, the first duration threshold may be determined according to the following formula 1:

[0070] T1=ɑ×T Formula 1

[0071] Wherein, T1 represents the first duration threshold, T represents the duration of the audio signal, ɑ represents the first coefficient, and 0<ɑ≤1.

[0072] In step S220, all valid audio segments are sorted at least according to the duration of their respective non-silent segments to obtain an audio segment queue.

[0073] For example, all valid audio segments may be sorted from long to short or from short to long only according to the duration of their respective non-silent segments.

[0074] For example, each valid audio segment can be set with a priority, and valid audio segments with higher priorities are sorted first according to the duration of their respective non-silent segments, and valid audio segments with lower priorities are sorted first according to the duration of their respective non-silent segments. Among them, for valid audio segments of each priority, the arrangement order is the same, and they are all sorted from long to short or from short to long according to the duration of their respective non-silent segments.

[0075] In step S230, in the audio segment queue, n consecutive valid audio segments are determined as voice prompt signals starting from the first end of the audio segment queue, wherein the sum of the durations of the non-silent segments in the valid audio segments in the voice prompt signal is equal to or greater than a first duration threshold, the sum of the durations of the non-silent segments in n-1 valid audio segments out of the n valid audio segments is less than the first duration threshold, and in the audio segment queue, in the direction from the first end to the second end of the audio queue, there are at least adjacent valid audio segments whose durations change from long to short.

[0076] For example, when there is a priority for the valid audio segment, the valid audio segments in the audio segment queue can be sorted only according to the duration of the non-silent segment. In the direction from one end of the audio segment queue to the other end, the duration of the non-silent segment only changes from long to short or from short to long. Therefore, at this time, one end of the audio segment queue where the valid audio segment with the longest duration of the non-silent segment is located can be used as the first end, and the other end of the audio segment queue can be used as the second end. Starting from the first valid audio segment at the first end, the duration of the non-silent segment of each valid audio segment in the direction of the second end of the valid audio segment can be accumulated in sequence until the accumulated duration of the non-silent segment is equal to or greater than the first duration threshold. Then the n valid audio segments with the accumulated duration of the non-silent segment can be used as voice prompt signals. It can be understood that when the duration of the non-silent segment of the nth valid audio segment is accumulated, the accumulated duration of the non-silent segment is equal to or greater than the first duration threshold, so the sum of the duration of the non-silent segment accumulated in the n-1th valid audio segment is less than the first duration threshold.

[0077] For example, when there is a priority for the valid audio segment, the valid audio segments of each priority in the audio segment queue can be sorted according to the duration of the non-silent segment. The higher the priority of the valid audio segment, the closer it is to the first end of the audio segment queue. In the direction from the first end to the second end of the audio queue, the valid audio segments of each priority are sorted from long to short according to the duration of the non-silent segment. Starting from the first audio segment at the first end, the duration of the non-silent segment of each valid audio segment in the direction of the second end of the valid audio segment can be accumulated in sequence until the accumulated duration of the non-silent segment is equal to or greater than the first duration threshold. Among them, the first valid audio segment at the first end is the valid audio segment with the highest priority and the longest duration of the non-silent segment. Then the n valid audio segments with the accumulated duration of the non-silent segment can be used as voice prompt signals. It can be understood that when the duration of the non-silent segment of the nth valid audio segment is accumulated, the accumulated duration of the non-silent segment is equal to or greater than the first duration threshold, so the sum of the duration of the non-silent segment accumulated in the n-1th valid audio segment is less than the first duration threshold.

[0078] In the above technical solution, according to the duration of the audio signal, a first duration threshold is determined, and all valid audio segments are sorted at least according to the duration of their respective non-silent segments to obtain an audio segment queue. In the audio segment queue, n consecutive valid audio segments are determined as voice prompt signals starting from the first end of the audio segment queue, wherein the sum of the durations of the non-silent segments in the valid audio segments in the voice prompt signal is equal to or greater than the first duration threshold, the sum of the durations of the non-silent segments in n-1 valid audio segments among the n valid audio segments is less than the first duration threshold, and in the audio segment queue, in the direction from the first end to the second end of the audio queue, at least the duration of the non-silent segments in the adjacent valid audio segments changes from long to short. In this way, the duration of the non-silent segments in the valid audio segments as voice prompt signals can be as long as possible, and the valid audio segments determined as voice prompt signals can contain as much valid voice information as possible, which can improve the quality of synthesized voice when performing voice synthesis.

[0079] Figure 3 FIG. 1 is a schematic flow chart of sorting all valid audio segments at least according to the duration of their respective non-silent segments to obtain an audio segment queue according to an embodiment of the present invention. Figure 3 As shown, the above step S220 may include steps S310 to S340.

[0080] In step S310, a text sequence corresponding to each valid audio segment is determined.

[0081] A speech recognition model or a speech recognition algorithm may be used to recognize a text sequence corresponding to each valid audio segment.

[0082] In step S320, each word in each text sequence is searched in the vocabulary library.

[0083] The vocabulary library may be a pre-established vocabulary library, which may contain vocabulary of a determined category. The category of the vocabulary may be the professional field corresponding to the vocabulary, such as the medical field, the communication field, the customer service field (customer, service, complaint, consultation, etc.), the social security field (social insurance, pension insurance, pension, social security card, etc.), etc. For example, the text sequence may be divided into at least one vocabulary, and then each vocabulary in each text sequence may be searched in the vocabulary library using a text matching algorithm or a neural network model. It is understandable that when the vocabulary library contains vocabulary of a determined professional field, each vocabulary in each text sequence searched in the vocabulary library is a vocabulary of the professional field.

[0084] In step S330, for each text sequence corresponding to a valid audio segment, a first ratio between the number of words in the text sequence found in the vocabulary library and the total number of words in the text sequence is calculated.

[0085] The larger the first ratio is, the more text information in the text sequence can be matched in the vocabulary library, and the easier it is to determine the category of the vocabulary in the text sequence.

[0086] For example, the first ratio P may be calculated according to the following formula 2:

[0087]

[0088] Among them, N1 represents the number of words found in the vocabulary library in the text sequence corresponding to the valid audio segment, and N2 represents the total number of words in the text sequence.

[0089] In step S340, the valid audio segments whose first ratio is greater than the ratio threshold are sorted from long to short according to the duration of their respective non-silent segments, and the valid audio segments whose first ratio is less than or equal to the ratio threshold are arranged after the valid audio segments whose first ratio is greater than the ratio threshold in the order of their respective non-silent segments from long to short.

[0090] The ratio threshold may be a fixed value determined based on experience, or may be a value determined based on the first ratio corresponding to the text sequence corresponding to each valid audio segment, for example, it may be an average value, a median value, etc.

[0091] When the first ratio of valid audio segments is greater than the ratio threshold, it means that the vocabulary of professional fields in the text sequences corresponding to these valid audio segments is relatively high, and there is more voice information that is easy to analyze in these valid audio segments. When performing speech synthesis, the speech synthesis model can analyze the voice prompt signals determined based on these valid audio segments to determine the professional fields corresponding to the synthesized speech to be generated. The speech synthesis model can generate more accurate synthesized speech by combining the first input text, the voice prompt signal and the professional field corresponding to the voice prompt signal. Therefore, these valid audio segments can be prioritized, so as to preferentially determine the voice prompt signal from these valid audio segments. At this time, the valid audio segments whose first ratio is greater than the ratio threshold are sorted at a position closer to the first end of the audio segment queue, and the valid audio segments whose first ratio is less than or equal to the ratio threshold are sorted at a position closer to the second end of the audio segment queue.

[0092] In the above technical solution, the text sequence corresponding to each valid audio segment is determined, each word in each text sequence is searched in the vocabulary library, and for each text sequence corresponding to the valid audio segment, the first ratio between the number of words in the text sequence that are found in the vocabulary library and the total number of words in the text sequence is calculated, and the valid audio segments whose first ratio is greater than the ratio threshold are sorted from long to short according to the duration of their respective non-silent segments, and the valid audio segments whose first ratio is less than or equal to the ratio threshold are arranged in order from long to short according to the duration of their respective non-silent segments after the valid audio segments whose first ratio is greater than the ratio threshold. In this way, the valid silent segments that are easy to analyze and have longer non-silent periods can be preferentially used as voice prompt signals, which is conducive to improving the efficiency of speech synthesis.

[0093] Exemplarily, the above-mentioned step S120 determines a target silent segment in the audio segment in the silent segment in the audio segment, including step S121: according to the duration of the silent segment in the audio segment, taking a silent segment with a duration greater than a second duration threshold as the target silent segment in the audio segment.

[0094] The second duration threshold may be a predetermined fixed value, or a value determined according to the median or average value of the duration of the silence segment. A silence segment whose duration is greater than the second duration threshold indicates that the silence segment contains a lot of invalid information, and retaining the silence segment is not conducive to speech synthesis. Therefore, the silence segment may be used as a target silence segment and may be removed later.

[0095] In the above technical solution, according to the duration of the silent segment in the audio segment, the silent segment with a duration greater than the second duration threshold is used as the target silent segment in the audio segment. In this way, a longer silent segment can be simply and effectively determined as the target silent segment in the audio segment, so that these silent segments can be removed in subsequent operations.

[0096] Figure 4FIG. 4 is a schematic flow chart showing how to use a silence segment whose duration is greater than a second duration threshold as a target silence segment in the audio segment according to an embodiment of the present invention. Figure 4 As shown, the above step S121 may include steps S410 to S430.

[0097] Step S410, determining the average duration of the silent segments in the audio segment.

[0098] When determining the silent segments in the audio segment, the start time and the end time of each silent segment can be recorded, and then the duration of each silent segment can be determined according to the start time and the end time. Then, the average value is calculated according to the duration of each silent segment to obtain the average duration of the silent segments in the audio segment.

[0099] Step S420: determining a second duration threshold according to the average duration.

[0100] The average duration indicates the overall status of the silence segments in the audio segment.

[0101] For example, the average duration may be directly used as the second duration threshold.

[0102] For example, the second duration threshold may be determined by directly combining the average duration and other relevant information of the silence segment. Other relevant information may include the variance, standard deviation, median, etc. of the duration of the silence segment in the audio segment.

[0103] Step S430: Determine in the audio segment a silence segment whose duration is greater than a second duration threshold as a target silence segment in the audio segment.

[0104] The duration of each silent segment in the audio segment can be compared with the second duration threshold, and the silent segment with a duration greater than the second duration threshold is determined as the target silent segment in the audio segment. The silent segment with a duration greater than the second duration threshold is usually a silent segment with a lot of invalid information, and these silent segments are usually abnormal and need to be removed.

[0105] In the above technical solution, the second duration threshold is determined based on the average duration of the silent segments in the audio segment, and the silent segments with durations greater than the second duration threshold are determined in the audio segment as the target silent segments in the audio segment. The second duration threshold is determined based on the average duration of the silent segments, so that combined with the overall state of the silent segments in the audio segment, abnormal silent segments can be more accurately screened out as target silent segments for removal in subsequent operations.

[0106] Figure 5 FIG. 4 is a schematic flow chart showing a method of determining a second duration threshold according to an average duration according to an embodiment of the present invention. Figure 5 As shown, the above step S420 may include steps S510 to S520.

[0107] In step S510, a standard deviation is determined according to the average duration and the duration of each silent segment in the audio segment.

[0108] The standard deviation can represent the fluctuation of the duration of the silent segment in the audio segment.

[0109] In step S520, a second duration threshold is determined according to the standard deviation and the average duration.

[0110] The coefficients corresponding to the standard deviation and the average duration may be determined, and then the second duration threshold may be determined according to the standard deviation, the average duration and the coefficients corresponding to the two.

[0111] Exemplarily, the second duration threshold may be determined according to the following formula 3:

[0112] T2=μ+k*σ Formula 3

[0113] Wherein, T2 represents the second duration threshold, μ represents the average duration, k represents the second coefficient, and σ represents the standard deviation.

[0114] In the above technical solution of dictionary-based phoneme conversion, the standard deviation is determined according to the average duration and the duration of each silent segment in the audio segment, and the second duration threshold is determined according to the standard deviation and the average duration. In this way, the influence of the average duration and the duration fluctuation on the silent segment can be considered at the same time, and the second duration threshold that is more conducive to screening the target silent segment can be determined.

[0115] Figure 6 FIG. 4 is a schematic flow chart of determining a silent segment in each audio segment according to an embodiment of the present invention. Figure 6 As shown, the above step S420 may include steps S610 to S640.

[0116] In step S610, a second text corresponding to the audio segment is determined.

[0117] For example, a speech recognition algorithm may be used to determine the second text corresponding to the audio segment.

[0118] Exemplarily, the audio segment may be input into an automatic speech recognition model to output a second text.

[0119] In step S620, a phoneme sequence is determined according to the second text.

[0120] For example, the second text may be converted into a phoneme sequence based on a phoneme dictionary.

[0121] For example, a statistical model may be used to analyze the mapping relationship between the second text and the corresponding phonemes, thereby obtaining a phoneme sequence.

[0122] For example, the second text may be analyzed using a deep learning model to obtain a phoneme sequence.

[0123] In step S630, the acoustic features of the audio segment and the text features of the phoneme sequence are aligned to determine the start and end time of each phoneme in the phoneme sequence in the audio segment.

[0124] For example, a dynamic time warping (DTW) algorithm may be used to align the acoustic features of the audio segment and the text features of the phoneme sequence.

[0125] Exemplarily, based on the attention mechanism, which parts of the acoustic features of the audio segment are associated with which parts of the text features of the phoneme sequence can be dynamically selected, so that the acoustic features of the audio segment and the text features of the phoneme sequence are aligned.

[0126] For example, a Hidden Markov Model (HMM) can be used to capture the time dependency of the phoneme sequence and the dynamic changes of the phonetic segment voice information, so as to align the acoustic features of the audio segment with the text features of the phoneme sequence. For example, Viterbi decoding can be used to find the most likely phoneme sequence based on the acoustic features of the audio segment, so as to align it with the text features of the phoneme sequence.

[0127] In step S640, according to the start and end time of each phoneme in the audio segment, a time period in the audio segment without the corresponding phoneme is determined as a silent period in the audio segment.

[0128] It can be understood that since the non-silent segment has pronunciation information, there are corresponding phonemes after alignment, while the silent segment has no pronunciation information and therefore no corresponding phonemes.

[0129] For example, the processing operations from step S620 to step S640 may be performed using a Montreal Forced Aligner (MFA).

[0130] In the above technical solution, the second text corresponding to the audio segment is determined, and then the phoneme sequence is determined. Then, the acoustic features of the audio segment and the text features of the phoneme sequence are aligned to determine the start and end time of each phoneme in the phoneme sequence in the audio segment. Then, according to the start and end time of each phoneme in the audio segment, the time period without the corresponding phoneme in the audio segment is determined as the silent segment in the audio segment. The silent segment in the audio segment can be determined more accurately through the alignment result of the acoustic features of the audio segment and the text features of the phoneme sequence.

[0131] Figure 7FIG. 1 is a schematic flow chart showing a method of dividing an audio signal according to the duration of an audio signal of a voice prompt to obtain at least one audio segment according to an embodiment of the present invention. Figure 7 As shown, the above step S110 may include steps S710 to S720.

[0132] In step S710, when the duration of the audio signal is greater than the second duration threshold, voice activity detection is performed on the audio signal to determine a silent segment in the audio signal.

[0133] When the duration of the audio signal is greater than the second duration threshold, it indicates that the duration of the audio signal is too long, which is not conducive to the calculation of the speech synthesis model. Through voice activity detection, the silent segment in the audio signal can be roughly determined.

[0134] For example, the voice activity detection may be energy-based VAD, that is, the silence segment is determined by detecting energy changes of the audio signal.

[0135] For example, the voice activity detection may be spectrum-based VAD, that is, the silence segment is determined by analyzing the spectrum characteristics of the audio signal.

[0136] For example, the voice activity detection may be a model-based VAD, that is, a machine learning or deep learning model is used to distinguish between speech and non-speech segments. The model may be a Gaussian mixture model, a deep neural network, or the like.

[0137] In step S720, the audio signal is divided into a plurality of audio segments using at least a portion of the determined moments in the silence segment as segmentation points, wherein the duration of each audio segment is not less than a third duration threshold.

[0138] The voice information in the non-silent segment is usually continuous, so the segmentation point should not be determined in the non-silent segment, otherwise it will affect the voice information in the non-silent segment. The segmentation point can be determined in the silent segment to segment the audio signal into multiple audio segments, and the duration of each audio segment is not less than the third duration threshold, so as to avoid the audio segments after segmentation being too short.

[0139] In the above technical solution, when the duration of the audio signal is greater than the second duration threshold, voice activity detection is performed on the audio signal to determine the silent segments in the audio signal, and the audio signal is divided into multiple audio segments using at least a portion of the determined moments in the silent segments as segmentation points, wherein the duration of each audio segment is not less than the third duration threshold. In this way, an overly long audio signal can be divided into audio segments of desired length without affecting the voice information of the non-silent segments.

[0140] Exemplarily, the audio signal of the voice prompt (prompt audio) can be divided into N audio segments to be processed: P_1, P_2, ..., P_N by VAD technology. Then, the pre-trained ASR model is used to perform voice recognition on each audio segment P_i to obtain the recognized text (second text) Text_i. Then, MFA can be used to force alignment of P_i with the text Text_i to obtain the start and end time and duration of each phoneme corresponding to the text Text_i. After that, it can be detected whether there is a silent segment exceeding the second duration threshold in the audio, and each target silent segment exceeding the second duration threshold is removed according to the timestamp information of MFA to obtain a valid audio segment P_i_cutsil from which the target silent segment is removed. Among them, the valid audio segment P_i_cutsil corresponds to the audio segment P_i. According to the result of MFA, the duration of the non-silent segment in each valid audio segment P_i_cutsil is counted. All valid audio segments are sorted in descending order according to the effective duration to obtain a sorted audio list. The first valid audio segment in the audio list can be used as a voice prompt signal. When performing speech synthesis, the speech prompt signal and the first text may be used as input data for speech synthesis, so that the speech synthesis model can output corresponding synthesized speech.

[0141] Figure 8 FIG. 4 is a schematic block diagram of a speech synthesis device according to an embodiment of the present invention. Figure 8 As shown, the speech synthesis device includes a segmentation module 810 , a detection module 820 , a filtering module 830 , a selection module 840 , and a speech synthesis module 850 .

[0142] The segmentation module 810 is used to segment the audio signal according to the duration of the audio signal of the voice prompt to obtain at least one audio segment.

[0143] The detection module 820 is used to determine, for each audio segment, a silent segment and a non-silent segment in the audio segment, and determine, among the silent segments in the audio segment, a target silent segment in the audio segment.

[0144] The filtering module 830 is used to remove the target silent segment in each audio segment to obtain a valid audio segment.

[0145] The selection module 840 is used to determine a voice prompt signal in all valid audio segments according to the duration of the non-silent segment in each valid audio segment, and the voice prompt signal includes at least one valid audio segment with the longest duration of the non-silent segment.

[0146] The synthesis module 850 is used to input the first text and the voice prompt signal into the speech synthesis model to output a synthesized speech that conforms to the voice prompt signal and corresponds to the first text.

[0147] Exemplarily, the selection module 840 includes a first duration threshold calculation submodule, a sorting submodule and a screening submodule. The first duration threshold calculation submodule is used to determine the first duration threshold according to the duration of the audio signal. The sorting submodule is used to sort all valid audio segments at least according to the duration of their respective non-silent segments to obtain an audio segment queue. The screening submodule is used to determine n consecutive valid audio segments as voice prompt signals in the audio segment queue starting from the first end of the audio segment queue, wherein the sum of the durations of the non-silent segments in the valid audio segments in the voice prompt signal is equal to or greater than the first duration threshold, the sum of the durations of the non-silent segments in the n-1 valid audio segments in the n valid audio segments is less than the first duration threshold, and in the audio segment queue, in the direction from the first end to the second end of the audio queue, at least there is a non-silent segment in the adjacent valid audio segments whose duration changes from long to short.

[0148] Exemplarily, the sorting submodule is used for the text recognition submodule, the vocabulary search submodule, the ratio calculation submodule and the valid audio segment sorting submodule. The text recognition submodule is used to determine the text sequence corresponding to each valid audio segment. The vocabulary search submodule is used to search each word in each text sequence in the vocabulary library. The ratio calculation submodule is used to calculate the first ratio between the number of words in the text sequence found in the vocabulary library and the total number of words in the text sequence for the text sequence corresponding to each valid audio segment. The valid audio segment sorting submodule is used to sort the valid audio segments whose first ratio is greater than the ratio threshold from long to short according to the duration of their respective non-silent segments, and to arrange the valid audio segments whose first ratio is less than or equal to the ratio threshold in the order of the duration of their respective non-silent segments from long to short after the valid audio segments whose first ratio is greater than the ratio threshold.

[0149] Exemplarily, the first duration threshold calculation submodule includes a first calculation submodule. The first calculation submodule is used to determine the first duration threshold according to the following formula:

[0150] T1=ɑ×T

[0151] Wherein, T1 represents the first duration threshold, T represents the duration of the audio signal, ɑ represents the first coefficient, and 0<ɑ≤1.

[0152] Exemplarily, the detection module 820 includes a target silence segment determination submodule. The target silence segment determination submodule is used to select a silence segment with a duration greater than a second duration threshold as a target silence segment in the audio segment according to the duration of the silence segment in the audio segment.

[0153] Exemplarily, the target silence segment determination submodule includes an average duration determination submodule, a second duration threshold determination submodule and a target silence segment selection submodule. The average duration determination submodule is used to determine the average duration of the silence segment in the audio segment. The second duration threshold determination submodule is used to determine the second duration threshold according to the average duration. The target silence segment selection submodule is used to determine the silence segment with a duration greater than the second duration threshold in the audio segment as the target silence segment in the audio segment.

[0154] Exemplarily, the second duration threshold determination submodule includes a standard deviation determination submodule and a second duration threshold calculation submodule.

[0155] The standard deviation determination submodule is used to determine the standard deviation according to the average duration and the duration of each silent segment in all audio segments.

[0156] The second duration threshold calculation submodule is used to determine the second duration threshold according to the standard deviation and the average duration.

[0157] Exemplarily, the second duration threshold calculation submodule includes a second calculation submodule. The second calculation submodule is used to determine the second duration threshold according to the following formula:

[0158] T2=μ+k*σ

[0159] Wherein, T2 represents the second duration threshold, μ represents the average duration, k represents the second coefficient, and σ represents the standard deviation.

[0160] Exemplarily, the detection module 820 includes a second text recognition submodule, a phoneme sequence determination submodule, an alignment submodule and a silence segment determination submodule. The second text recognition submodule is used to determine the second text corresponding to the audio segment. The phoneme sequence determination submodule is used to determine the phoneme sequence based on the second text. The alignment submodule is used to align the acoustic features of the audio segment with the text features of the phoneme sequence to determine the start and end times of each phoneme in the phoneme sequence in the audio segment. The silence segment determination submodule is used to determine the time period without the corresponding phoneme in the audio segment according to the start and end times of each phoneme in the audio segment, as a silence segment in the audio segment.

[0161] Exemplarily, the second text recognition submodule includes a speech recognition submodule. The speech recognition submodule is used to input the audio segment into an automatic speech recognition model to output the second text.

[0162] Exemplarily, the segmentation module 810 includes a silence segment detection submodule and a segmentation submodule. The silence segment detection submodule is used to perform voice activity detection on the audio signal when the duration of the audio signal is greater than the second duration threshold to determine the silence segment in the audio signal. The segmentation submodule is used to use the moments in at least a portion of the determined silence segments as segmentation points to segment the audio signal into a plurality of audio segments, wherein the duration of each audio segment is not less than the third duration threshold.

[0163] According to another aspect of the present invention, an electronic device is provided. Fig. 9 FIG. 1 shows a schematic block diagram of an electronic device according to an embodiment of the present invention. Fig. 9 As shown, the electronic device includes a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used by the processor to execute the speech synthesis method as described above when the processor is running.

[0164] In addition, according to another aspect of the present invention, a storage medium is also provided, on which program instructions are stored, and when the program instructions are executed by a computer or a processor, the computer or the processor performs the corresponding steps of the above-mentioned speech synthesis method of the embodiment of the present invention, and is used to implement the corresponding modules in the above-mentioned speech synthesis device according to the embodiment of the present invention. The storage medium may include, for example, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0165] According to another aspect of the present invention, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions are used to execute the above-mentioned speech synthesis method when running.

[0166] A person skilled in the art can understand the specific implementation and beneficial effects of the above-mentioned speech synthesis device, electronic device, storage medium and computer program product by reading the above-mentioned detailed description of the speech synthesis method. For the sake of brevity, they will not be repeated here.

[0167] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present invention thereto. Various changes and modifications may be made therein by one of ordinary skill in the art without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as required by the appended claims.

[0168] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0169] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0170] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0171] Similarly, it should be understood that in order to streamline the present invention and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present invention should not be interpreted as reflecting the following intention: the claimed invention requires more features than the features explicitly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present invention.

[0172] It will be understood by those skilled in the art that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0173] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0174] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some modules in a speech synthesis device according to an embodiment of the present invention. The present invention can also be implemented as a device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0175] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0176] The above is only a specific embodiment of the present invention or an explanation of a specific embodiment. The protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A speech synthesis method, characterized in that: The method comprises: According to the duration of the audio signal of the voice prompt, the audio signal is segmented to obtain at least one audio segment; For each audio segment, determine a silent segment and a non-silent segment in the audio segment, and among the silent segments in the audio segment, determine a target silent segment in the audio segment; Removing the target silent segment in each audio segment to obtain a valid audio segment; Determine a voice prompt signal in all valid audio segments according to the duration of the non-silent segment in each valid audio segment, wherein the voice prompt signal includes at least one valid audio segment with the longest duration of the non-silent segment; The first text and the voice prompt signal are input into a speech synthesis model to output a synthesized speech that conforms to the voice prompt signal and corresponds to the first text.

2. The method according to claim 1, characterized in that The step of determining the voice prompt signal in all valid audio segments according to the duration of the non-silent segment in each valid audio segment includes: Determining a first duration threshold according to the duration of the audio signal; Sort all valid audio segments at least by the duration of their respective non-silent segments to obtain an audio segment queue; In the audio segment queue, n consecutive valid audio segments are determined as the voice prompt signal starting from the first end of the audio segment queue, wherein the sum of the durations of the non-silent segments in the valid audio segments in the voice prompt signal is equal to or greater than a first duration threshold, the sum of the durations of the non-silent segments in n-1 valid audio segments among the n valid audio segments is less than the first duration threshold, and in the audio segment queue, in the direction from the first end to the second end of the audio queue, there are at least adjacent valid audio segments whose durations change from long to short.

3. The method according to claim 2, characterized in that The step of sorting all valid audio segments at least according to the duration of their respective non-silent segments to obtain an audio segment queue includes: Determine the text sequence corresponding to each valid audio segment; Look up each word in each text sequence in the vocabulary; For each text sequence corresponding to a valid audio segment, calculating a first ratio between the number of words in the text sequence found in the vocabulary library and the total number of words in the text sequence; The valid audio segments whose first ratio is greater than the ratio threshold are sorted from long to short according to the duration of their respective non-silent segments, and the valid audio segments whose first ratio is less than or equal to the ratio threshold are arranged after the valid audio segments whose first ratio is greater than the ratio threshold in the order of their respective non-silent segments from long to short.

4. The method according to claim 2, characterized in that: The determining a first duration threshold according to the duration of the audio signal and the first ratio includes: The first duration threshold is determined according to the following formula: T1=ɑ×T Wherein, T1 represents the first duration threshold, T represents the duration of the audio signal, ɑ represents the first coefficient, and 0<ɑ≤1.

5. The method according to claim 1, characterized in that The step of determining a target silent segment in the audio segment in the silent segment in the audio segment includes: According to the duration of the silence segment in the audio segment, a silence segment with a duration greater than a second duration threshold is used as a target silence segment in the audio segment.

6. The method according to claim 5, characterized in that The step of taking a silence segment having a duration greater than a second duration threshold as a target silence segment according to the duration of the silence segment in the audio segment includes: Determine the average duration of silence in the audio segment; Determine the second duration threshold according to the average duration; A silence segment with a duration greater than a second duration threshold is determined in the audio segment as a target silence segment in the audio segment.

7. A speech synthesis device, characterized in that: include: A segmentation module, used to segment the audio signal according to the duration of the audio signal of the voice prompt to obtain at least one audio segment; A detection module, configured to determine, for each audio segment, a silent segment and a non-silent segment in the audio segment, and, among the silent segments in the audio segment, determine a target silent segment in the audio segment; A filtering module is used to remove the target silent segment in each audio segment to obtain a valid audio segment; A selection module, configured to determine a voice prompt signal in all valid audio segments according to the duration of the non-silent segment in each valid audio segment, wherein the voice prompt signal includes at least one valid audio segment with the longest duration of the non-silent segment; The synthesis module is used to input the first text and the voice prompt signal into a speech synthesis model to output a synthesized speech that conforms to the voice prompt signal and corresponds to the first text.

8. An electronic device comprising a processor and a memory, characterized in that: The memory stores computer program instructions, which are used by the processor to execute the speech synthesis method according to any one of claims 1 to 6 when executed.

9. A storage medium having program instructions stored thereon, characterized in that: The program instructions are used to execute the speech synthesis method according to any one of claims 1 to 6 when running.

10. A computer program product comprising computer program instructions, characterized in that The computer program instructions are used to execute the speech synthesis method according to any one of claims 1 to 6 when running.

Images