Speech synthesis method and device, computer equipment and medium

Abstract

The embodiment of the invention is suitable for the technical field of signal processing, and provides a speech synthesis method and device, computer equipment and a medium, and the method comprises the steps: segmenting a text of a to-be-synthesized speech into a plurality of phrases; in the process of performing speech synthesis on the text based on the plurality of phrases, determining a resource occupancy rate of a current system; if the resource occupancy rate is not within a preset range, adjusting pause intervals of the plurality of phrases; and performing speech synthesis on the text based on the adjusted pause interval. According to the method, the resource occupancy rate of the system can be adjusted in the speech synthesis process, so that the stable operation of the system is guaranteed.

Description

Technical Field

[0001] The present application belongs to the field of signal processing technology, and in particular, relates to a speech synthesis method, apparatus, computer equipment and medium. Background Art

[0002] Speech synthesis is a technology that produces artificial speech through mechanical and electronic methods. For example, text can be converted into speech based on speech synthesis technology.

[0003] The speech produced by traditional speech synthesis technology is relatively mechanical. At present, speech synthesis can be performed through neural networks to obtain more natural speech. However, speech synthesis through neural networks consumes more resources such as memory and computing power. When the speech synthesis method based on neural networks is applied to resource-constrained embedded devices, the resource problem becomes more prominent.

[0004] Therefore, it is particularly important to maintain the real-time performance of speech synthesis while minimizing resource usage, especially resource control for multi-channel speech synthesis. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a speech synthesis method, apparatus, computer equipment and medium, which can adjust the resource occupancy of the system during the speech synthesis process, thereby ensuring the smooth operation of the system.

[0006] A first aspect of an embodiment of the present application provides a speech synthesis method, comprising:

[0007] Segment the text to be synthesized into multiple phrases;

[0008] In the process of performing speech synthesis on the text based on the plurality of the phrases, determining the resource occupancy of the current system;

[0009] If the resource occupancy rate is not within a preset range, adjusting the pause intervals of the plurality of phrases;

[0010] Speech synthesis is performed on the text based on the adjusted pause interval.

[0011] A second aspect of an embodiment of the present application provides a speech synthesis device, including:

[0012] A segmentation module, used for segmenting the text of the speech to be synthesized into multiple phrases;

[0013] A determination module, used for determining the resource occupancy rate of the current system in the process of performing speech synthesis on the text based on the plurality of the phrases;

[0014] An adjustment module, configured to adjust the pause intervals of the plurality of phrases if the resource occupancy rate is not within a preset range;

[0015] A synthesis module is used to perform speech synthesis on the text based on the adjusted pause interval.

[0016] A third aspect of an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0017] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect above is implemented.

[0018] A fifth aspect of the embodiments of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device executes the method described in the first aspect.

[0019] Compared with the prior art, the embodiments of the present application have at least the following advantages:

[0020] When the method in the embodiment of the present application is used for speech synthesis, the text of the speech to be synthesized can be divided into multiple phrases; then, based on the multiple phrases, speech synthesis is performed on the text. In the process of speech synthesis of the text based on multiple phrases, the resource occupancy rate of the current system can be determined, so that whether the current system resource allocation is reasonable can be determined based on the resource occupancy rate; if the resource occupancy rate is not within the preset range, it indicates that the resource allocation of the current system is unreasonable. At this time, the pause intervals of the multiple phrases can be adjusted; then, speech synthesis is performed on the text based on the adjusted pause intervals. Based on the adjustment of the pause intervals, the speech synthesis process of the text can be controlled so that the text synthesis process can adapt to the fluctuation of system resources and reduce the auditory jamming of the synthesized speech; by controlling the speech synthesis process of the text, the resource occupancy rate of the system can be adjusted so that the system resource allocation is balanced and the stable operation of the system is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art.

[0022] Figure 1 It is a schematic diagram of the steps of a speech synthesis method provided in an embodiment of the present application;

[0023] Figure 2 It is a schematic diagram of increasing the pause interval provided in an embodiment of the present application;

[0024] Figure 3is a schematic diagram of reducing the pause interval provided by an embodiment of the present application;

[0025] Figure 4 is a schematic diagram of the steps of another speech synthesis method provided in an embodiment of the present application;

[0026] Figure 5 is a schematic diagram of a speech synthesis device provided in an embodiment of the present application;

[0027] Figure 6 It is a schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In the following description, specific details such as specific system structures, technologies, etc. are proposed for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from hindering the description of the present application.

[0029] When using a computer for speech synthesis, in order to fully utilize computing resources, multiple speech synthesis tasks can be executed simultaneously. However, executing multiple speech synthesis tasks simultaneously will require more computing resources. When too many computer resources are occupied in the system, the system may run slowly and speech synthesis tasks may be stuck. When the computer resources occupied in the system are too low, the computer resources in the system cannot be fully utilized.

[0030] Based on this, the present application provides a speech synthesis method that balances the resource usage of the system while ensuring the real-time performance of speech synthesis.

[0031] The technical solution of the present application is described below through specific embodiments.

[0032] Reference Figure 1 , shows a schematic flow chart of a speech synthesis method provided in an embodiment of the present application, which may specifically include the following steps:

[0033] S101, dividing the text of the speech to be synthesized into multiple phrases.

[0034] The execution subject of this embodiment may be a computer device, which may include a personal service terminal, a server, a supercomputer, etc. This embodiment does not limit the specific type of the computer device.

[0035] Exemplarily, the computer device may be a speech synthesis server. The speech synthesis server may receive speech synthesis tasks submitted by various terminals and return speech information to each terminal in real time. Each speech synthesis task may include a text, and the text corresponding to each speech synthesis task may be called a text path. The text for speech synthesis in the computer device may include a text path or multiple text paths.

[0036] When performing speech synthesis, speech synthesis can be performed based on phrases. Therefore, the text of each input speech to be synthesized can be divided into multiple phrases. A phrase refers to multiple words with a large pause. The computer device can divide the text into multiple phrases through semantic segmentation. The computer device can also directly divide the text based on punctuation and other symbols, thereby dividing the text into multiple phrases.

[0037] The computer device may include a program for speech synthesis. After the text is segmented into phrases, the speech synthesis program may be called to perform speech synthesis on the text. When performing speech synthesis, computing resources in the system need to be used, such as occupying a cache and calling a thread.

[0038] S102, in the process of performing speech synthesis on the text based on the plurality of phrases, determining the resource occupancy rate of the current system.

[0039] In the process of performing speech synthesis on the text based on the plurality of phrases, when too many system resources are occupied, it is easy to cause the system to run slowly, resulting in system freezes, thereby causing speech synthesis to pause. Therefore, it is necessary to monitor the occupied system resources.

[0040] Therefore, in the process of speech synthesis of text based on multiple phrases, the resource occupancy of the current system can be determined. The system resource occupancy can be directly obtained through the resource usage in the current system, or it can be indirectly represented by the task execution status in the current system. For example, the resource occupancy can be represented by the threads used, the cache occupied, etc., and can also be reflected by the speech synthesis speed in the current system.

[0041] For example, when using speech synthesis speed to represent resource occupancy, the current average speech synthesis speed of each text path can be calculated, and the average speech synthesis speed can be used to represent the resource occupancy of the current system. The faster the current average speech synthesis speed, the more sufficient the system resources are, that is, the lower the resource occupancy rate is. The slower the current average speech synthesis speed, the less sufficient the system resources are, that is, the higher the resource occupancy rate is.

[0042] If speech synthesis of a text is performed in the current system, the speech synthesis speed of the text can be used to represent the resource occupancy of the current system. The speech synthesis speed can be represented by the ratio of the synthesized speech time length to the synthesis time. For example, the time length of the synthesized speech within the current 1 second can be obtained, and then the speech time length can be divided by 1 second to obtain the current speech synthesis speed, which can represent the resource occupancy. Assuming that the time length of the synthesized speech is tv1 and the synthesis time is ts1, the speech synthesis speed is: SV = tv1 / ts1.

[0043] If speech synthesis of multiple texts is performed in the current system, the average speech synthesis speed of the multiple texts can be used to represent the resource occupancy of the current system. For example, the total length of the synthesized speech time corresponding to each text within the current 1 second can be obtained, and then the total length of the speech time can be divided by 1 second to obtain the current speech synthesis speed, which can represent the resource occupancy. Assume that the time length of the first synthesized speech is tv1, and the synthesis time is ts1; the time length of the second synthesized speech is tv2, and the synthesis time is ts2, then the speech synthesis speed is: SV = (tv1+tv2) / (ts1+ts2).

[0044] It should be noted that when using speech synthesis speed to represent resource occupancy, the speech synthesis speed can be used directly for representation, or a variant value of the speech synthesis speed can be used for representation, for example, the inverse of the speech synthesis speed can be used for representation. In addition, the resource occupancy corresponding to each speech synthesis speed can be pre-calibrated, so that the corresponding resource occupancy can be determined according to the speech synthesis speed.

[0045] S103: If the resource occupancy rate is not within a preset range, adjusting the pause intervals of the plurality of phrases.

[0046] When the resource occupancy rate is within a preset range, it can be indicated that the resources in the current system are balanced, and speech synthesis can continue to be performed according to the current state.

[0047] If the resource occupancy rate is not within the preset range, it can indicate that the resources in the current system are insufficient or the resources in the current system are not fully utilized. At this time, the resource occupancy rate of the current system can be adjusted to balance the resources in the current system.

[0048] The above preset range can be characterized by a first threshold and a second threshold. For example, the first threshold and the second threshold can be the same value. That is, when the resource occupancy rate is equal to the first threshold, it is determined that the resource occupancy rate is within the preset range; when the resource occupancy rate is not equal to the first threshold, it is determined that the resource occupancy rate is not within the preset range.

[0049] In addition, the first threshold may also be greater than the second threshold. When the resource occupancy rate is greater than or equal to the second threshold and less than or equal to the first threshold, it is determined that the resource occupancy rate is within the preset range. When the resource occupancy rate is less than the second threshold or greater than the first threshold, it is determined that the resource occupancy rate is not within the preset range. If the resource occupancy rate is greater than the first threshold, it indicates that the current system resources are insufficient, and the pause interval of the phrase can be increased, thereby reducing the resource occupancy rate. Figure 2 is a schematic diagram of increasing the pause interval provided in an embodiment of the present application. Figure 2 As shown, the steps to increase the pause interval can be as follows:

[0050] S201, determining a first target text from each of the texts.

[0051] The computer device may include multiple texts, each of which may be used to synthesize one voice. When selecting one text from the multiple texts as the first target text, it may be selected randomly or in sequence.

[0052] Exemplarily, each text may have sequence identification information, and according to the sequence identification, one text is selected as the first target text. The sequence identification information may be determined according to the generation time of the speech synthesis task corresponding to the text, or may be determined according to the priority of the speech synthesis task corresponding to the text. For example, the texts may be sorted in order from low to high priority of the speech synthesis task corresponding to the text. In addition, the order may be determined according to the pause intervals between phrases in each text. For example, the computer device may sort the texts in order from small to large pause intervals between phrases corresponding to each text.

[0053] S202: when synthesizing the first target text into speech, increasing pause intervals between multiple phrases in the first target text.

[0054] The first target text can be divided into a plurality of phrases, and speech synthesis can be performed based on the phrases. When a pause interval is added between phrases, it is equivalent to pausing for a period of time after processing a phrase, and then performing speech synthesis of the next phrase. When pausing, the text is equivalent to not performing speech synthesis, which is equivalent to reducing resource occupation. Therefore, in this embodiment, the resource occupancy rate can be reduced by increasing the pause interval.

[0055] S203, updating the resource occupancy rate of the current system.

[0056] After increasing the pause intervals between the multiple phrases in the first target text, the resource occupancy rate of the current system can be adjusted. In order to determine whether the resource occupancy rate of the current system is within a preset range, the resource occupancy rate of the current system can be recalculated. The method for calculating the resource occupancy rate of the current system can refer to the above-mentioned S102, which is not repeated here.

[0057] S204: If the updated resource occupancy rate is greater than the first threshold, determine whether the pause interval currently added in each path of the text is less than a preset interval threshold.

[0058] When the updated resource occupancy rate is within the preset range, it can be represented that the resources in the current system are balanced, and speech synthesis can continue to be performed according to the current state.

[0059] If the updated resource occupancy rate is greater than the first threshold, it can be indicated that the resources in the current system are still insufficient. At this time, the resource occupancy rate of the current system can be further reduced, so that the resources in the current system are balanced.

[0060] In this embodiment, the resource occupancy rate is reduced by increasing the pause interval. However, the pause interval that can be added in the text is limited. When too many pause intervals are added in the text, the voice interval between two phrases may be too large, resulting in a stuttering of the voice. Therefore, an interval threshold can be set. When the pause interval added in each text is less than the interval threshold, it indicates that the resource occupancy rate can continue to be reduced by increasing the adjustment interval threshold. When the pause interval added in each text is greater than the interval threshold, it indicates that the resource occupancy rate can no longer be reduced by increasing the adjustment interval threshold.

[0061] S205: If the pause interval currently added in each path of the text is greater than or equal to the interval threshold, lower the first threshold, and adjust the resource occupancy rate based on the new first threshold.

[0062] When the pause interval added in each text is greater than the interval threshold, it indicates that further increasing the pause interval is likely to cause speech jamming, and therefore the resource occupancy rate cannot be further reduced by increasing the adjustment interval threshold.

[0063] At this time, the first threshold may be lowered, and then based on the lowered first threshold, it is determined whether the resource occupancy needs to be further adjusted.

[0064] S206, if the pause interval currently added in each of the texts is less than the interval threshold, a new first target text is determined from the text, and the target step and the steps after the target step are executed, wherein the target step is to increase the pause intervals between multiple phrases in the first target text when synthesizing the first target text into speech.

[0065] When the pause interval added in each text path is less than the interval threshold, it indicates that the resource occupancy rate can be further reduced by increasing the adjustment interval threshold.

[0066] When increasing the pause interval, you can first select a new first target text with an increased pause interval. When selecting the first target text, you can select the next text in sequence as the target text. When each text is used as the first target text once, you can re-select the second round, and select the first target text from the multiple texts in sequence. Similarly, when the pause interval added in each text is less than the interval threshold, multiple pause intervals can be added to the multiple texts.

[0067] After the new first target text is determined, the resource occupancy rate may be adjusted through steps S202 to S206 until the resource occupancy rate of the current system is within a preset range.

[0068] If the resource occupancy rate of the computer device is less than the second threshold, it indicates that the current system resources are underutilized, and the pause interval of the phrase can be reduced, thereby improving the resource occupancy rate. Figure 3 is a schematic diagram of reducing the pause interval provided by an embodiment of the present application. Figure 3 As shown, the steps to increase the pause interval can be as follows:

[0069] S301, determining a second target text from each of the texts.

[0070] The computer device may include multiple texts, each of which may be used to synthesize one voice. When selecting one text from the multiple texts as the second target text, it may be selected randomly or in sequence.

[0071] Exemplarily, each text may have sequence identification information, and according to the sequence identification, one text is selected as the second target text. The sequence identification information may be determined according to the generation time of the speech synthesis task corresponding to the text. It may also be determined according to the priority of the speech synthesis task corresponding to the text. For example, the computer device may sort the texts according to the priority of the speech synthesis task corresponding to the text from low to high. In addition, the order may also be determined according to the pause intervals between phrases in each text. For example, the computer device may sort the texts according to the pause intervals between phrases corresponding to each text from more to less.

[0072] S302: when synthesizing the second target text into speech, reducing the pause intervals between the plurality of phrases in the second target text.

[0073] The second target text can be divided into a plurality of phrases, and speech synthesis can be performed based on the phrases. When the pause interval between the phrases is reduced, it is equivalent to reducing the pause time between the speech synthesis of two phrases. When there is a pause, the text is equivalent to not performing speech synthesis; reducing the pause interval is equivalent to reducing the time when speech synthesis is not performed, which is equivalent to improving resource occupancy. Therefore, in this embodiment, the resource occupancy rate can be improved by reducing the pause interval.

[0074] S303: Update the resource occupancy rate of the current system.

[0075] After reducing the pause intervals between the multiple phrases in the second target text, the resource occupancy rate of the current system can be adjusted. In order to determine whether the resource occupancy rate of the current system is within a preset range, the resource occupancy rate of the current system can be recalculated. The method for calculating the resource occupancy rate of the current system can refer to the above S102, which is not repeated here.

[0076] S304: If the updated current system resource occupancy rate is less than the second threshold, determine whether there is a pause interval between the phrases of the texts currently being speech synthesized.

[0077] When the updated resource occupancy rate is within the preset range, it can be represented that the resources in the current system are balanced, and speech synthesis can continue to be performed according to the current state.

[0078] If the updated current system resource occupancy rate is less than the second threshold, it indicates that the current system resources are still underutilized. At this time, it can be determined whether there is a pause interval between the phrases of the texts currently being synthesized.

[0079] S305: If there is no pause interval between the phrases of the texts currently being speech synthesized, increase the second threshold, and adjust the resource occupancy rate based on the new second threshold.

[0080] If there is no pause interval between the phrases of the texts currently being synthesized, it means that the resource occupancy rate cannot be increased by reducing the pause interval, which means that the system resources can no longer be used. Therefore, the second threshold value can be increased to update the preset range. Based on the new second threshold value, the resource occupancy rate is continuously adjusted according to steps S101-S104.

[0081] S306, if there are pause intervals between the phrases of the texts currently undergoing speech synthesis, a new second target text is determined from the text, and the target step and the steps after the target step are executed, wherein the target step is to reduce the pause intervals between the multiple phrases of the second target text when synthesizing the first target text into speech.

[0082] If there are pause intervals between the phrases of the texts currently being synthesized, the resource occupancy rate can be improved by further reducing the pause intervals.

[0083] When reducing the pause interval, a new second target text with increased pause interval can be selected first. When selecting the second target text, the next text can be selected in sequence as the target text. When each text is used as the second target text once, a second round of selection can be performed again, and the second target text can be selected from multiple texts in sequence, and so on. Of course, it should be noted that each selected second target text needs to be a text with pause intervals between phrases.

[0084] After the new second target text is determined, the resource occupancy rate may be adjusted through steps S202 to S206 until the resource occupancy rate of the current system is within a preset range.

[0085] In a possible implementation, the computing device may also update the above-mentioned preset range based on the amount of text for speech synthesis in the current system and / or the resource occupancy of the current system.

[0086] Exemplarily, the computer device may determine the initial preset range according to the number of texts for speech synthesis in the current system. For example, when the resource occupancy rate is characterized by speech synthesis speed, assuming that the speech synthesis speed is sv, when there is one text, the first threshold value may be set equal to the second threshold value equal to a; when there are multiple texts, the first threshold value may be set equal to the second threshold value equal to b, and a>b.

[0087] When the resource occupancy rate of the current system is too low, the computer device can also update the preset range. For example, when there is one text, sv>1.4a; when there are multiple texts, sv>1.4b; it can be indicated that the resource occupancy rate of the current system is too low. At this time, the threshold can be increased, for example, a=1.1a, b=1.1b.

[0088] S104: Perform speech synthesis on the text based on the adjusted pause interval.

[0089] After the pause interval is adjusted, each text may be synthesized according to the pause interval of each text.

[0090] In this embodiment, the computer device can adjust the system's current speech synthesis process by adjusting the pause interval of the phrase, thereby adjusting the system's resource occupancy rate, so that the system resources are reasonably utilized, while avoiding speech synthesis jams and ensuring the auditory effect of the synthesized speech.

[0091] Reference Figure 4, shows a schematic diagram of the steps of another speech synthesis method provided in an embodiment of the present application, which may specifically include the following steps:

[0092] S401, dividing the text of the speech to be synthesized into multiple phrases.

[0093] Steps S401 - S402 of this embodiment and steps S101 - S102 of the previous embodiment may refer to each other and are not described in detail here.

[0094] S402: In the process of performing speech synthesis on the text based on the plurality of phrases, determining the resource occupancy rate of the current system.

[0095] S403: If the resource occupancy rate of the current system is less than the second threshold, determine the system cache occupied in the process of synthesizing the text.

[0096] When the resource occupancy rate of the current system is lower than the second threshold, it indicates that the system resources are sufficient, and a new speech synthesis task can be considered to be executed.

[0097] Executing a new speech synthesis task requires occupying the system cache. Therefore, the currently occupied system cache, the corresponding text size, and the length of the synthesized speech can be determined first.

[0098] S404, determining a standby system buffer required for the newly added text of the speech to be synthesized, wherein the newly added text and the text belong to texts of different paths.

[0099] The newly added text may be the text of a new speech synthesis task. The computer device may determine the ratio of the current text length to the occupied buffer in the system, and determine the standby system buffer required for the newly added text. For example, the ratio of the current text length to the occupied buffer may be equal to the ratio of the newly added text length to the standby system buffer. Therefore, the standby system buffer may be determined based on the current text length, the currently occupied buffer, and the newly added text length.

[0100] S405: Calculate the cache sum of the occupied system cache and the standby system cache.

[0101] S406: If the cache sum is less than the cache threshold, speech synthesis is performed on the newly added text.

[0102] If the cache sum is less than the cache threshold, the computer device may perform speech synthesis for the newly added text.

[0103] S407, updating the resource occupancy of the current system.

[0104] Step S407 of this embodiment and step S102 of the previous embodiment may refer to each other and will not be described in detail here.

[0105] S408. If the updated resource occupancy rate is not within the preset range, adjust the pause intervals of multiple phrases.

[0106] S409. Perform speech synthesis on the text based on the adjusted pause intervals.

[0107] The steps of S408 - S409 in this embodiment and S103 - S104 in the previous embodiment can be referred to each other and will not be elaborated here.

[0108] The speech synthesis method proposed in this embodiment can perceive the system resources and synthesis situation in real time, support multi - path speech synthesis as much as possible under limited resources, adapt to the fluctuations of the system, and reduce auditory stuttering. The computer device can perceive the multi - path speech synthesis speed and the resource load of the system in real time, and dynamically adjust the synthesis interval and synthesis control strategy, so as to make reasonable use of system resources.

[0109] To better elaborate the solution of this application, specific examples are used to illustrate the solution of this application below.

[0110] If there is only one piece of text to be synthesized in the current system, synthesize it by phrase. And count the time length tv1 of the synthesized speech, the synthesis time ts1, and the current occupied cache size bf1; assume that the system supports at least one - path real - time speech synthesis; after a new phrase is synthesized, update tv1, ts1, and bf1.

[0111] When there is already one piece of text being synthesized in the current system, if the synthesis speed sv of the current piece of text

[0112]

[0113] It indicates that the current system resources are not enough to support the speech synthesis of the new text, then the synthesis of the new text can start after the speech synthesis of the current text is completed. Here, a is an empirical coefficient used to balance the resource fluctuations of the system, generally less than 1 and can be updated in real time.

[0114] When sv >= a, the cache size bf2 required for the speech synthesis task of the newly added text can be estimated. If bf1 + bf2 < bfm, then perform the speech synthesis of the new text. Here, bfm is the set maximum cache size. At the same time, the computer device can count and continuously update the time length tv2 of the synthesized speech and the synthesis time ts2, and calculate the current average speech synthesis speed And so on, so that n - path speech synthesis can be started in the computer device.

[0115] When there are n texts for speech synthesis in the system, due to possible fluctuations in system resources, the current average speech synthesis speed sv can be calculated, and the speech synthesis can be controlled according to the current average speech synthesis speed.

[0116] If the current average speech synthesis speed sv < b, (b < a), then a pause interval pti can be added between the phrases of the text of the first path first, and then the current average speech synthesis speed sv is updated. After the current average speech synthesis speed sv is updated, if sv >= b, the pause interval is not added anymore. If sv is still less than b, then continue to add the pause interval pti between the phrases of the second path text, and so on until sv >= b. After the pause intervals have been added to all n paths of text, if the current average speech synthesis speed sv is still less than b, then the pause interval pti can be continuously added between the phrases from the first path text to the nth path text until sv > b. The pti can be set according to the actually played speech.

[0117] However, the pause interval that can be added is limited. If the currently added pause intervals for the 1st to nth paths > 3pti, then a and b can be modified simultaneously

[0118] a' = 0.9a

[0119] b' = 0.9b

[0120] If sv > a and there are currently added pause intervals, then from the mth path to the first path, the pause intervals are reduced one by one until the pause interval is 0 or sv <= b.

[0121] If the current sv > 1.4a, then adjust a and b

[0122] a' = 1.1a

[0123] b' = 1.1b

[0124] In this embodiment, the computer device can adjust the speech synthesis process in the computer device in real time based on the speech synthesis speed, so that while making full use of computer resources, the speech synthesis efficiency can be guaranteed. The dynamic resource adjustment based on the speech synthesis speed is simple in method and can ensure the stable operation of the computer device.

[0125] It should be noted that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0126] Referring to Figure 5 , a schematic diagram of a speech synthesis device provided by an embodiment of the present application is shown, which may specifically include a segmentation module 51, a determination module 52, an adjustment module 53, and a synthesis module 54, where:

[0127] A segmentation module 51, used for segmenting the text of the speech to be synthesized into multiple phrases;

[0128] A determination module 52, configured to determine a resource occupancy rate of a current system during speech synthesis of the text based on the plurality of phrases;

[0129] An adjusting module 53, configured to adjust the pause intervals of the plurality of phrases if the resource occupancy rate is not within a preset range;

[0130] The synthesis module 54 is used to perform speech synthesis on the text based on the adjusted pause interval.

[0131] In a possible implementation, the adjustment module 53 includes:

[0132] an interval increasing submodule, configured to increase the pause interval of the phrase if the resource occupancy rate is greater than a first threshold;

[0133] The interval reduction submodule is used to reduce the pause interval of the phrase if the resource occupancy rate is less than a second threshold, and the second threshold is less than or equal to the first threshold.

[0134] In a possible implementation, the text to be synthesized includes multiple texts, and the interval increasing submodule includes:

[0135] A first target text determination unit, used to determine a first target text from each of the texts;

[0136] a pause interval increasing unit, configured to increase the pause intervals between a plurality of phrases in the first target text when synthesizing the first target text into speech;

[0137] A resource occupancy rate updating unit, used for updating the resource occupancy rate of the current system;

[0138] A first judgment unit, configured to determine whether the pause interval currently added in each of the texts is less than a preset interval threshold if the updated resource occupancy rate is greater than the first threshold;

[0139] A first threshold value adjustment unit, configured to reduce the first threshold value if the pause interval currently added in each path of the text is greater than or equal to the interval threshold value, and adjust the resource occupancy rate based on the new first threshold value;

[0140] The pause interval continuing increasing unit is used to determine a new first target text from the text if the pause interval currently added in each path of the text is less than the interval threshold, and execute the target step and the steps after the target step. The target step is to increase the pause interval between multiple phrases in the first target text when synthesizing the first target text into speech.

[0141] In a possible implementation, the text to be synthesized includes multiple texts, and the interval reduction submodule includes:

[0142] A second target text determination unit, used for determining a second target text from each of the texts;

[0143] a pause interval reducing unit, configured to reduce the pause intervals between the plurality of phrases of the second target text when synthesizing the second target text into speech;

[0144] A resource occupancy rate updating unit, used for updating the resource occupancy rate of the current system;

[0145] A second judgment unit is used to determine whether there is a pause interval between the phrases of the texts currently being speech synthesized if the updated current system resource occupancy rate is less than the second threshold;

[0146] a second threshold value adjustment unit, configured to increase the second threshold value if there is no pause interval between the phrases of the texts of the various paths currently undergoing speech synthesis, and adjust the resource occupancy rate based on the new second threshold value;

[0147] The pause interval continuing reduction unit is used to determine a new second target text from the text if there is a pause interval between the phrases of each of the texts currently undergoing speech synthesis, and execute the target step and the steps after the target step. The target step is to reduce the pause intervals between multiple phrases of the second target text when synthesizing the second target text into speech.

[0148] In a possible implementation manner, the device further includes:

[0149] The preset range determination module is used to update the preset range based on the amount of texts for speech synthesis in the current system and / or the resource occupancy rate of the current system.

[0150] In a possible implementation, the determination module 52 includes:

[0151] The synthesis speed calculation submodule is used to calculate the current average speech synthesis speed of the text, and use the average speech synthesis speed to represent the resource occupancy rate of the current system.

[0152] In a possible implementation manner, the device further includes:

[0153] An occupied system cache determining module, configured to determine the occupied system cache in the process of synthesizing the text if the resource occupancy rate of the current system is less than the second threshold;

[0154] A standby system cache determination module, used to determine the standby system cache required for the newly added text of the speech to be synthesized, wherein the newly added text and the text belong to texts of different paths;

[0155] A cache sum calculation module, used for calculating the cache sum of the occupied system cache and the standby system cache;

[0156] A new text synthesis module is added, which is used to perform speech synthesis on the new text if the cache sum is less than the cache threshold.

[0157] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment part.

[0158] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 6 As shown, the computer device 6 of this embodiment includes: at least one processor 60 ( Figure 6 Only one is shown in the figure), a memory 61, and a computer program 62 stored in the memory 61 and executable on the at least one processor 60, wherein the processor 60 implements the steps of any of the above-mentioned method embodiments when executing the computer program 62.

[0159] The computer device 6 may be a desktop computer, a notebook computer, a PDA, a cloud server, or other computer devices. The computer device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will appreciate that Figure 6 It is only an example of the computer device 6 and does not constitute a limitation on the computer device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0160] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0161] In some embodiments, the memory 61 may be an internal storage unit of the computer device 6, such as a hard disk or memory of the computer device 6. In other embodiments, the memory 61 may also be an external storage device of the computer device 6, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 6. Further, the memory 61 may also include both an internal storage unit of the computer device 6 and an external storage device. The memory 61 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0162] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0163] An embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0164] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application is described in detail with reference to the above-mentioned embodiments, a person skilled in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A speech synthesis method, It is characterized in that include: Segment the text to be synthesized into multiple phrases; In the process of performing speech synthesis on the text based on the plurality of the phrases, determining the resource occupancy of the current system; If the resource occupancy rate is not within a preset range, adjusting the pause intervals of the plurality of phrases; Speech synthesis is performed on the text based on the adjusted pause interval.

2. The method according to claim 1, It is characterized in that If the resource occupancy rate is not within a preset range, the pause intervals of the plurality of phrases are adjusted, including: If the resource occupancy rate is greater than a first threshold, increasing the pause interval of the phrase; If the resource occupancy rate is less than a second threshold, the pause interval of the phrase is reduced, and the second threshold is less than or equal to the first threshold.

3. The method according to claim 2, It is characterized in that The text of the speech to be synthesized includes multiple texts, and if the resource occupancy rate is greater than a first threshold, increasing the pause interval of the phrase includes: Determine a first target text from the texts in each channel; When synthesizing the first target text into speech, increasing the pause intervals between multiple phrases in the first target text; Update the resource occupancy of the current system; If the updated resource occupancy rate is greater than the first threshold, determining whether the pause interval currently added in each path of the text is less than a preset interval threshold; If the pause interval currently added in each path of the text is greater than or equal to the interval threshold, lowering the first threshold, and adjusting the resource occupancy rate based on the new first threshold; If the pause interval currently added in each of the texts is less than the interval threshold, a new first target text is determined from the text, and the target step and the steps after the target step are executed. The target step is to increase the pause interval between multiple phrases in the first target text when synthesizing the first target text into speech.

4. The method according to claim 2, It is characterized in that The text of the speech to be synthesized includes multiple texts, and if the resource occupancy rate is less than a second threshold, reducing the pause interval of the phrase includes: Determine a second target text from the texts in each channel; When synthesizing the second target text into speech, reducing the pause intervals between the plurality of phrases of the second target text; Updating the resource occupancy of the current system; If the updated current system resource occupancy rate is less than the second threshold, determining whether there is a pause interval between the phrases of the texts currently being speech synthesized; If there is no pause interval between the phrases of the texts currently undergoing speech synthesis, increasing the second threshold, and adjusting the resource occupancy rate based on the new second threshold; If there are pause intervals between the phrases of the texts currently undergoing speech synthesis, a new second target text is determined from the text, and the target step and the steps after the target step are executed. The target step is to reduce the pause intervals between the multiple phrases of the second target text when synthesizing the second target text into speech.

5. The method according to any one of claims 1 to 4, It is characterized in that Before adjusting the pause interval of the phrase to adjust the current system resource occupancy rate if the current system resource occupancy rate is not within the preset range, the method further includes: The preset range is updated based on the amount of texts to be speech synthesized in the current system and / or the resource occupancy of the current system.

6. The method according to any one of claims 1 to 4, It is characterized in that The determining of the current system resource occupancy rate includes: The current average speech synthesis speed of the text is calculated, and the resource occupancy rate of the current system is represented by the average speech synthesis speed.

7. The method according to any one of claims 2 to 4, It is characterized in that Before adjusting the pause interval of the phrase to adjust the current system resource occupancy rate if the current system resource occupancy rate is not within the preset range, the method further includes: If the resource occupancy rate of the current system is less than the second threshold, determining the system cache occupied in the process of synthesizing the text; Determine a standby system buffer required for a newly added text of the speech to be synthesized, wherein the newly added text and the text belong to a text of a different path; Calculating a cache sum of the occupied system cache and the standby system cache; If the cache sum is less than the cache threshold, speech synthesis is performed on the newly added text.

8. A speech synthesis device, It is characterized in that include: A segmentation module, used for segmenting the text of the speech to be synthesized into multiple phrases; A determination module, used for determining the resource occupancy rate of the current system in the process of performing speech synthesis on the text based on the plurality of the phrases; An adjustment module, configured to adjust the pause intervals of the plurality of phrases if the resource occupancy rate is not within a preset range; A synthesis module is used to perform speech synthesis on the text based on the adjusted pause interval.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images