speech synthesis methods
By adjusting the field splitting granularity, bit depth, and database classification type under different preset conditions through the central control module, the accuracy and stability issues in speech synthesis are resolved, and the accuracy and anti-interference ability of speech synthesis are improved.
Patent Information
- Application Number
- CN202311111292.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-30
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-08-30
AI Technical Summary
In existing speech synthesis technologies, the accuracy of speech synthesis is reduced due to inaccurate adjustment of field splitting granularity, and the anti-interference ability of the speech synthesis process is reduced due to inaccurate adjustment of alignment depth.
The central control module determines the stability of the speech synthesis process and the accuracy of database calls under different preset conditions, and adjusts the field splitting granularity, bit depth and the number of field classification types in the database to improve the accuracy and stability of speech synthesis.
It improves the accuracy and stability of speech synthesis, reduces the impact of inaccurate field splitting granularity and bit depth adjustment, and enhances the anti-interference capability of speech synthesis.
Smart Images

Figure CN116994554B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of speech synthesis technology, and more particularly to a speech synthesis method. Background Technology
[0002] In current technologies, with the development and widespread application of speech synthesis technology, speech synthesis services are increasingly being accepted and used by users. A large proportion of speech synthesis users are bilingual or multilingual users, and speech synthesis is increasingly being applied to multilingual content. This has created a demand for multilingual speech synthesis, with mixed Chinese and English reading being particularly common. Users typically require multilingual speech synthesis to be intelligible, followed by accurate, natural pronunciation and a consistent tone. While current speech synthesis technology has largely solved the intelligibility issue, how to synthesize natural, accurate, and consistent-tone multilingual speech has become a technical challenge in speech synthesis.
[0003] Chinese Patent Publication No. CN105845125B discloses a speech synthesis method, comprising: determining the language type of the text information to be synthesized, wherein the language type includes a first language type and a second language type; determining a first basic model corresponding to the first language type and determining a second basic model corresponding to the second language type, wherein the first basic model includes a first spectral parameter model and a first fundamental frequency parameter module, and the second basic model includes a second spectral parameter model and a second fundamental frequency parameter module; determining a target timbre, and adjusting the first spectral parameter model and the second spectral parameter model according to the target timbre respectively. An adaptive transformation is performed, and the text information of the sentence to be synthesized is trained based on the first and second spectral parameter models after the adaptive transformation to generate corresponding spectral parameters. The text information of the sentence to be synthesized is then trained based on the first and second fundamental frequency parameter modules to generate corresponding fundamental frequency parameters, and the fundamental frequency parameters of the first and second language types are adjusted according to the target timbre. The target speech is synthesized based on the trained spectral parameters of the first and second language types, as well as the adjusted fundamental frequency parameters of the first and second language types. Therefore, the speech synthesis method suffers from problems such as decreased speech synthesis accuracy due to inaccurate adjustment of field splitting granularity and decreased anti-interference capability of the speech synthesis process due to inaccurate adjustment of alignment depth. Summary of the Invention
[0004] To address this, the present invention provides a speech synthesis method to overcome the problems in the prior art where the accuracy of speech synthesis is reduced due to inaccurate adjustment of field splitting granularity and the anti-interference capability of the speech synthesis process is reduced due to inaccurate adjustment of alignment depth.
[0005] To achieve the above objectives, the present invention provides a speech synthesis method, comprising: step S1, an information acquisition terminal acquires text information to be processed and splits the text information to be processed into several fields of different lengths, and retrieves field data corresponding to the several fields of different lengths from a database; step S2, the field data corresponding to the several fields of different lengths are converted into several corresponding audio data, and an audio synthesizer is used to convert the several corresponding audio data into speech; step S3, when the audio synthesizer completes the speech conversion of the several corresponding audio data, a sound velocity meter is used to detect the sound playback speed at several time points, and the variance of the sound playback speed at several time points is calculated, and the central control module... The block determines the stability of the speech synthesis process based on the variance of the sound playback speed at several time points; in step S4, when the central control module determines that the stability of the speech synthesis process is lower than the allowable range, it adjusts the field splitting granularity to the first corresponding granularity, or, when it initially determines that the accuracy of calling database data is lower than the allowable range, it adjusts the bit depth to the corresponding depth based on the overlap of the sound waveform; in step S5, when the central control module completes the initial adjustment of the field splitting granularity, it adjusts the field splitting granularity to the second corresponding granularity based on the proportion of sound stuttering duration, or, based on the average number of times several split fields are filtered, it adjusts the number of field classification types in the database to the corresponding number.
[0006] Furthermore, the central control module determines whether the stability of the speech synthesis process is within an allowable range based on three methods according to the variance of the sound playback speed at several time points.
[0007] The first type of determination method is that the central control module determines that the stability of the speech synthesis process is within the allowable range under the preset first variance condition;
[0008] The second type of determination method is that the central control module determines that the stability of the speech synthesis process is lower than the allowable range under the preset second variance condition, initially determines that the accuracy of database calling is lower than the allowable range, and makes a second determination on whether the accuracy of database calling is lower than the allowable range based on the overlap of the sound waveform.
[0009] The third type of determination method is that the central control module determines that the stability of the speech synthesis process is lower than the allowable range under the preset third variance condition, and adjusts the field splitting granularity to the first corresponding granularity by calculating the difference between the variance of the sound playback speed at several time points and the preset second variance.
[0010] The first preset variance condition is that the variance of the sound playback speed at a certain number of time points is less than or equal to the first preset variance; the second preset variance condition is that the variance of the sound playback speed at a certain number of time points is greater than the first preset variance and less than or equal to the second preset variance; the third preset variance condition is that the variance of the sound playback speed at a certain number of time points is greater than the second preset variance; and the first preset variance is less than the second preset variance.
[0011] Furthermore, the central control module, under the preset third variance condition, determines two types of adjustment methods for the field splitting granularity based on the difference between the variance of the sound playback speed at several time points and the preset second variance.
[0012] The first type of adjustment method is that the central control module adjusts the field splitting granularity to the first granularity using a preset first granularity adjustment coefficient under the condition of a preset first variance difference.
[0013] The second type of adjustment method is that the central control module adjusts the field splitting granularity to the second granularity under the condition of a preset second variance difference;
[0014] The first preset variance difference condition is that the difference between the variance of the sound playback speed at a certain time point and the preset second variance is less than or equal to the preset variance difference; the second preset variance difference condition is that the difference between the variance of the sound playback speed at a certain time point and the preset second variance is greater than the preset variance difference; and the first preset granularity adjustment coefficient is less than the second preset granularity adjustment coefficient.
[0015] Furthermore, the central control module uses two secondary determination methods—based on the overlap of the sound waveforms—to determine whether the accuracy of database retrieval is below the allowable range under the preset second variance condition.
[0016] The first type of accuracy secondary determination method is that the central control module determines the accuracy of database calls within the allowable range under the preset first overlap condition;
[0017] The second type of accuracy secondary determination method is that when the accuracy of the database call is lower than the allowable range under the preset second overlap condition, the central control module adjusts the bit depth to the corresponding depth by calculating the difference between the overlap of the sound waveform and the preset overlap.
[0018] The first preset overlap condition is that the overlap of the sound waveforms is less than or equal to a preset overlap condition; the second preset overlap condition is that the overlap of the sound waveforms is greater than a preset overlap condition.
[0019] Furthermore, the central control module, under the preset second overlap condition, determines two adjustment methods for bit depth based on the difference between the overlap of the sound waveform and the preset overlap, wherein...
[0020] The first type of bit depth adjustment method is that the central control module adjusts the bit depth to the first depth using a preset first bit depth adjustment coefficient under the condition of a preset first overlap difference.
[0021] The second type of bit depth adjustment method is that the central control module adjusts the bit depth to the second depth using a preset second bit depth adjustment coefficient under the condition of a preset second overlap difference.
[0022] The first preset overlap difference condition is that the difference between the overlap of the sound waveform and the preset overlap is less than or equal to the preset overlap difference; the second preset overlap difference condition is that the difference between the overlap of the sound waveform and the preset overlap is greater than the preset overlap difference; and the first preset depth adjustment coefficient is less than the second preset depth adjustment coefficient.
[0023] Furthermore, after the central control module completes the initial adjustment of the field splitting granularity, it determines whether the speech conversion efficiency is within the allowable range based on three methods according to the proportion of sound stuttering duration.
[0024] The first type of efficiency determination method is that the central control module determines that the voice conversion efficiency is within the allowable range under a preset first proportion condition;
[0025] The second type of efficiency determination method is that the central control module determines that the voice conversion efficiency is lower than the allowable range under the preset second proportion condition, and adjusts the field splitting granularity to the second corresponding granularity by calculating the difference between the proportion of sound stuttering duration and the preset first proportion.
[0026] The third type of efficiency determination method is that the central control module determines that the speech conversion efficiency is lower than the allowable range under the preset third proportion condition, initially determines that the classification accuracy of the sound model in the database is lower than the allowable range, and makes a second determination on the classification accuracy of the sound model in the database based on the average number of screenings of several split fields.
[0027] The first preset percentage condition is that the percentage of sound stuttering duration is less than or equal to the first preset percentage; the second preset percentage condition is that the percentage of sound stuttering duration is greater than the first preset percentage and less than or equal to the second preset percentage; the third preset percentage condition is that the percentage of sound stuttering duration is greater than the second preset percentage; and the first preset percentage is less than the second preset percentage.
[0028] Furthermore, the formula for calculating the percentage of pause duration in the sound is as follows:
[0029]
[0030] Where Z represents the percentage of audio pauses, N represents the duration of audio pauses, and K represents the total duration of audio output.
[0031] Furthermore, under the second proportion condition, the central control module determines two types of secondary adjustment methods for the field splitting granularity based on the difference between the proportion of sound stuttering duration and the preset first proportion.
[0032] The first type of granularity secondary adjustment method is that the central control module uses a preset fourth granularity adjustment coefficient to adjust the field splitting granularity to the third granularity under the preset first proportion difference condition.
[0033] The second type of granularity secondary adjustment method is that the central control module uses a preset third granularity adjustment coefficient to adjust the field splitting granularity to the fourth granularity under the preset second proportion difference condition.
[0034] The first preset percentage difference condition is that the difference between the percentage of sound stuttering duration and the preset first percentage is less than or equal to the preset percentage difference; the second preset percentage difference condition is that the difference between the percentage of sound stuttering duration and the preset first percentage is greater than the preset percentage difference; and the third preset granularity adjustment coefficient is less than the fourth preset granularity adjustment coefficient.
[0035] Furthermore, the central control module, under the preset third proportion condition, determines whether the classification accuracy of the sound model in the database is within the allowable range using two secondary judgment methods based on the average number of screenings after several split fields.
[0036] The first type of accuracy secondary determination method is that the central control module determines that the classification accuracy of the sound model in the database is within the allowable range under the condition of a preset first screening number;
[0037] The second type of accuracy determination method is that the central control module determines that the classification accuracy of the sound model in the database is lower than the allowable range under the condition of a preset second number of screenings. It then adjusts the number of field classification types in the database to the corresponding number by calculating the difference between the average number of screenings of several split fields and the preset number of screenings.
[0038] The first preset filtering count condition is that the average number of filtering counts for several split fields is less than or equal to the preset filtering count; the second preset filtering count condition is that the average number of filtering counts for several split fields is greater than the preset filtering count.
[0039] Furthermore, the central control module, under the preset second filtering count condition, determines two adjustment methods for the number of field classification types in the database based on the difference between the average filtering count of several split fields and the preset filtering count.
[0040] The first type of quantity adjustment method is that the central control module adjusts the quantity of field classification types in the database to the first quantity using a preset first quantity adjustment coefficient under the condition of a preset first screening number difference.
[0041] The second type of quantity adjustment method is that the central control module adjusts the quantity of field classification types in the database to the second quantity using a preset second quantity adjustment coefficient under the condition of a preset second screening number difference.
[0042] The first preset filtering frequency difference condition is that the difference between the average filtering frequency of several split fields and the preset filtering frequency is less than or equal to the preset filtering frequency difference; the second preset filtering frequency difference condition is that the difference between the average filtering frequency of several split fields and the preset filtering frequency is greater than the preset filtering frequency difference; and the first preset quantity adjustment coefficient is less than the second preset quantity adjustment coefficient.
[0043] Compared with the prior art, the beneficial effects of the present invention are as follows: the method of the present invention, through setting steps S1-S5, adjusts the field splitting granularity to the first corresponding granularity when the stability of the speech synthesis process is lower than the allowable range, thereby reducing the impact of inaccurate adjustment of the field splitting granularity on the accuracy of speech synthesis; by adjusting the bit depth to the corresponding depth according to the overlap of the sound waveform, it reduces the impact of inaccurate adjustment of the bit depth on the anti-interference ability of the speech synthesis process; by adjusting the field splitting granularity to the second corresponding granularity according to the proportion of sound stuttering duration after the initial adjustment of the field splitting granularity by the central control module, it reduces the impact of inaccurate adjustment of the field splitting granularity on the accuracy of speech synthesis; and by adjusting the number of field classification types in the database to the corresponding number according to the average number of screenings of several split fields, it reduces the impact of inaccurate adjustment of the number of field classification types in the database on the anti-interference ability of the speech synthesis process, thereby improving the accuracy and stability of speech synthesis.
[0044] Furthermore, the method of the present invention uses a central control module to perform a secondary judgment on whether the stability of the speech synthesis process or the accuracy of database calls based on the overlap of sound waveforms is below the allowable range under different preset variance conditions. This avoids the problem of decreased speech synthesis accuracy caused by inaccurate judgment of the stability of the speech synthesis process, and further improves the accuracy of speech synthesis and the stability of the speech synthesis process.
[0045] Furthermore, the method of the present invention adjusts the field splitting granularity to the first corresponding granularity when the stability of the speech synthesis process is lower than the allowable range in the central control module, thereby reducing the impact of inaccurate adjustment of the field splitting granularity on the accuracy of speech synthesis, and further improving the accuracy of speech synthesis and the stability of the speech synthesis process.
[0046] Furthermore, the method of the present invention improves the anti-interference ability of the speech synthesis process by having the central control module perform a secondary judgment on whether the accuracy of database calls is lower than the allowable range under a preset second variance condition, thereby further improving the accuracy and stability of speech synthesis.
[0047] Furthermore, the method of the present invention adjusts the bit depth to the corresponding depth according to the overlap of the sound waveform through the central control module, which reduces the impact of inaccurate bit depth adjustment on the anti-interference ability of the speech synthesis process, and further improves the accuracy of speech synthesis and the stability of the speech synthesis process.
[0048] Furthermore, the method of the present invention uses the central control module to determine whether the speech conversion efficiency is within the allowable range after the initial adjustment of the field splitting granularity. This reduces the decrease in speech conversion efficiency caused by excessively high proportion of sound stuttering time, improves the accuracy of speech synthesis, and further enhances the accuracy and stability of speech synthesis process.
[0049] Furthermore, the method of the present invention, through the central control module, after completing the initial adjustment of the field splitting granularity, adjusts the field splitting granularity to a second corresponding granularity based on the proportion of sound stuttering duration, thereby reducing the impact of inaccurate secondary adjustment of the field splitting granularity on the accuracy of speech synthesis, and further improving the accuracy of speech synthesis and the stability of the speech synthesis process.
[0050] Furthermore, the method of the present invention determines the classification accuracy of the sound models in the database by means of a central control module under a preset third ratio condition, thereby reducing the decline in speech synthesis quality caused by unreasonable average number of screenings of the split fields, improving the anti-interference ability of the speech synthesis process, and further improving the accuracy and stability of the speech synthesis process.
[0051] Furthermore, the method of the present invention adjusts the number of field classification types in the database to a corresponding number based on the average number of screenings of several split fields through a central control module. This reduces the impact of inaccurate adjustment of the number of field classification types in the database on the anti-interference ability of the speech synthesis process, and further improves the accuracy and stability of the speech synthesis process. Attached Figure Description
[0052] Figure 1 This is an overall flowchart of the speech synthesis method according to an embodiment of the present invention;
[0053] Figure 2 This is a detailed flowchart of step S3 of the speech synthesis method according to an embodiment of the present invention;
[0054] Figure 3 This is a flowchart illustrating step S4 of the speech synthesis method according to an embodiment of the present invention.
[0055] Figure 4 This is a flowchart of step S5 of the speech synthesis method in an embodiment of the present invention. Detailed Implementation
[0056] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.
[0057] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.
[0058] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.
[0059] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0060] Please see Figure 1 , Figure 2 , Figure 3 as well as Figure 4 The flowcharts shown are the overall flowchart, the detailed flowchart of step S3, the detailed flowchart of step S4, and the detailed flowchart of step S5 of the speech synthesis method according to an embodiment of the present invention. The present invention provides a speech synthesis method, comprising:
[0061] Step S1: The information acquisition terminal acquires the text information to be processed and splits the text information to be processed into several fields of different lengths, and retrieves the field data corresponding to the several fields of different lengths from the database.
[0062] Step S2: Convert the field data corresponding to the several fields of different lengths into several corresponding audio data, and use an audio synthesizer to convert the several corresponding audio data into speech;
[0063] Step S3: When the audio synthesizer completes the speech conversion of several corresponding audio data, the sound speed meter is used to detect the sound playback speed at several time points and the variance of the sound playback speed at several time points is calculated. The central control module determines the stability of the speech synthesis process based on the variance of the sound playback speed at several time points.
[0064] Step S4: When the central control module determines that the stability of the speech synthesis process is lower than the allowable range, the field splitting granularity is adjusted to the first corresponding granularity; or, when it is initially determined that the accuracy of calling database data is lower than the allowable range, the bit depth is adjusted to the corresponding depth according to the overlap of the sound waveform.
[0065] Step S5: When the central control module completes the initial adjustment of the field splitting granularity, the central control module adjusts the field splitting granularity to the second corresponding granularity according to the proportion of sound stuttering duration, or adjusts the number of field classification types in the database to the corresponding number according to the average number of screenings of several split fields.
[0066] Specifically, the method for calculating the variance of sound playback speed at several points in time is a conventional technique well-known to those skilled in the art, and will not be elaborated here.
[0067] Specifically, the field splitting granularity can be the maximum splitting length of the field in the text information to be processed or the number of field types to be split into; the field types to be split can be adjective fields, noun fields, verb fields or preposition fields.
[0068] Specifically, the formula for calculating the overlap of the sound waveforms is as follows:
[0069]
[0070] Where T is the overlap of the sound waveforms, A is the number of sound waveforms whose amplitude is the same as that of the synthesized sound waveform and the number of sound waveforms with the same amplitude in the same context stored in the database, and S is the total number of sound waveforms with the same context stored in the database.
[0071] Specifically, the formula for calculating the average number of filters for the several split fields is as follows:
[0072]
[0073] Where G is the average number of times the fields are filtered after several splits, Xn is the number of times the nth split field is filtered, n is the total number of split fields, and n is a natural number greater than or equal to 1.
[0074] Specifically, the database stores text data, voice data, and voice synthesis configuration information.
[0075] Specifically, step S3 includes:
[0076] Step S31: When the audio synthesizer completes the speech conversion of several corresponding audio data, it uses a sound speed meter to detect the sound playback speed at several time points and calculates the variance of the sound playback speed at several time points.
[0077] Step S32: The central control module determines the stability of the speech synthesis process based on the variance of the sound playback speed at the aforementioned time points.
[0078] Specifically, step S4 includes:
[0079] Step S41: When the central control module determines that the stability of the speech synthesis process is lower than the allowable range, the field splitting granularity is adjusted to the first corresponding granularity.
[0080] In step S42, when the central control module initially determines that the accuracy of calling the database data is lower than the allowable range, it adjusts the bit depth to the corresponding depth according to the overlap of the sound waveform.
[0081] Specifically, step S5 includes:
[0082] Step S51: When the central control module completes the initial adjustment of the field splitting granularity, the central control module adjusts the field splitting granularity to the second corresponding granularity according to the proportion of sound stuttering duration.
[0083] In step S52, the central control module adjusts the number of field classification types in the database to the corresponding number based on the average number of times the fields are filtered after being split.
[0084] The method of this invention, through steps S1-S5, involves the central control module adjusting the field splitting granularity to the first corresponding granularity when the stability of the speech synthesis process is below the allowable range. This reduces the impact of inaccurate adjustment of the field splitting granularity on the accuracy of speech synthesis. It also adjusts the bit depth to the corresponding depth based on the overlap of the sound waveform, reducing the impact of inaccurate bit depth adjustment on the anti-interference capability of the speech synthesis process. Furthermore, the central control module, after completing the initial adjustment of the field splitting granularity, adjusts the field splitting granularity to the second corresponding granularity based on the proportion of sound stuttering duration, reducing the impact of inaccurate secondary adjustment on the accuracy of speech synthesis. Finally, it adjusts the number of field classification types in the database to the corresponding number based on the average number of times several split fields are filtered, reducing the impact of inaccurate adjustment of the number of field classification types in the database on the anti-interference capability of the speech synthesis process. This ultimately improves both the accuracy and stability of speech synthesis.
[0085] Please continue reading. Figure 2 As shown, the central control module determines whether the stability of the speech synthesis process is within an acceptable range based on the variance of the sound playback speed at several time points using three different methods.
[0086] The first type of determination method is that the central control module determines that the stability of the speech synthesis process is within the allowable range under the preset first variance condition;
[0087] The second type of determination method is that the central control module determines that the stability of the speech synthesis process is lower than the allowable range under the preset second variance condition, initially determines that the accuracy of database calling is lower than the allowable range, and makes a second determination on whether the accuracy of database calling is lower than the allowable range based on the overlap of the sound waveform.
[0088] The third type of determination method is that the central control module determines that the stability of the speech synthesis process is lower than the allowable range under the preset third variance condition, and adjusts the field splitting granularity to the first corresponding granularity by calculating the difference between the variance of the sound playback speed at several time points and the preset second variance.
[0089] The first preset variance condition is that the variance of the sound playback speed at a certain number of time points is less than or equal to the first preset variance; the second preset variance condition is that the variance of the sound playback speed at a certain number of time points is greater than the first preset variance and less than or equal to the second preset variance; the third preset variance condition is that the variance of the sound playback speed at a certain number of time points is greater than the second preset variance; and the first preset variance is less than the second preset variance.
[0090] Specifically, the variance of the sound playback speed at several time points is denoted as Q, the preset first variance is denoted as Q1, the preset second variance is denoted as Q2, and the difference between the variance of the sound playback speed at several time points and the preset second variance is denoted as ΔQ. ΔQ is set to Q-Q2, where Q1 < Q2.
[0091] Furthermore, the method of the present invention uses a central control module to perform a secondary judgment on whether the stability of the speech synthesis process or the accuracy of database calls based on the overlap of sound waveforms is below the allowable range under different preset variance conditions. This avoids the problem of decreased speech synthesis accuracy caused by inaccurate judgment of the stability of the speech synthesis process, and further improves the accuracy of speech synthesis and the stability of the speech synthesis process.
[0092] Please continue reading. Figure 3 As shown, the central control module determines two adjustment methods for the field splitting granularity based on the difference between the variance of the sound playback speed at several time points and the preset second variance under the preset third variance condition.
[0093] The first type of adjustment method is that the central control module adjusts the field splitting granularity to the first granularity using a preset first granularity adjustment coefficient under the condition of a preset first variance difference.
[0094] The second type of adjustment method is that the central control module adjusts the field splitting granularity to the second granularity under the condition of a preset second variance difference;
[0095] The first preset variance difference condition is that the difference between the variance of the sound playback speed at a certain time point and the preset second variance is less than or equal to the preset variance difference; the second preset variance difference condition is that the difference between the variance of the sound playback speed at a certain time point and the preset second variance is greater than the preset variance difference; and the first preset granularity adjustment coefficient is less than the second preset granularity adjustment coefficient.
[0096] Specifically, the preset variance difference is denoted as △Q0, the preset first granularity adjustment coefficient is denoted as α1, the preset second granularity adjustment coefficient is denoted as α2, the field splitting granularity is denoted as V, where 1 < α1 < α2, the adjusted field splitting granularity is denoted as V', and V' is set to V × (1 + αi) / 2, where αi is the preset i-th granularity adjustment coefficient, and i = 1, 2.
[0097] The method described in this invention adjusts the field splitting granularity to the first corresponding granularity when the stability of the speech synthesis process is below the allowable range in the central control module. This reduces the impact of inaccurate adjustment of the field splitting granularity on the accuracy of speech synthesis, and further improves the accuracy of speech synthesis and the stability of the speech synthesis process.
[0098] Please continue reading. Figure 3 As shown, the central control module uses two secondary determination methods to determine whether the accuracy of database retrieval is below the allowable range based on the overlap of sound waveforms under the preset second variance condition.
[0099] The first type of accuracy secondary determination method is that the central control module determines the accuracy of database calls within the allowable range under the preset first overlap condition;
[0100] The second type of accuracy secondary determination method is that when the accuracy of the database call is lower than the allowable range under the preset second overlap condition, the central control module adjusts the bit depth to the corresponding depth by calculating the difference between the overlap of the sound waveform and the preset overlap.
[0101] The first preset overlap condition is that the overlap of the sound waveforms is less than or equal to a preset overlap condition; the second preset overlap condition is that the overlap of the sound waveforms is greater than a preset overlap condition.
[0102] Specifically, the preset overlap is denoted as P1, the overlap of the sound waveform is denoted as P, the difference between the overlap of the sound waveform and the preset overlap is denoted as △P, and △P is set to P-P1.
[0103] The method described in this invention improves the anti-interference capability of the speech synthesis process by having the central control module perform a secondary judgment on whether the accuracy of database calls is below the allowable range under a preset second variance condition. This further enhances the accuracy and stability of speech synthesis.
[0104] Please continue reading. Figure 3 As shown, the central control module determines two adjustment methods for bit depth based on the difference between the overlap of the sound waveform and the preset overlap under the preset second overlap condition.
[0105] The first type of bit depth adjustment method is that the central control module adjusts the bit depth to the first depth using a preset first bit depth adjustment coefficient under the condition of a preset first overlap difference.
[0106] The second type of bit depth adjustment method is that the central control module adjusts the bit depth to the second depth using a preset second bit depth adjustment coefficient under the condition of a preset second overlap difference.
[0107] The first preset overlap difference condition is that the difference between the overlap of the sound waveform and the preset overlap is less than or equal to the preset overlap difference; the second preset overlap difference condition is that the difference between the overlap of the sound waveform and the preset overlap is greater than the preset overlap difference; and the first preset depth adjustment coefficient is less than the second preset depth adjustment coefficient.
[0108] Specifically, the preset overlap difference is denoted as △P0, the preset first depth adjustment coefficient is denoted as β1, the preset second depth adjustment coefficient is denoted as β2, and the depth is denoted as H, where 1 < β1 < β2. The adjusted depth is denoted as H', and H' is set to H × (1 + 2βj) / 3, where βj is the preset j-th depth adjustment coefficient, and j = 1, 2.
[0109] The method described in this invention adjusts the bit depth to the corresponding depth according to the overlap of the sound waveform through the central control module, thereby reducing the impact of inaccurate bit depth adjustment on the anti-interference ability of the speech synthesis process and further improving the accuracy and stability of speech synthesis.
[0110] Please continue reading. Figure 4 As shown, after the central control module completes the initial adjustment of the field splitting granularity, it determines whether the speech conversion efficiency is within the allowable range based on three methods according to the proportion of sound stuttering duration.
[0111] The first type of efficiency determination method is that the central control module determines that the voice conversion efficiency is within the allowable range under a preset first proportion condition;
[0112] The second type of efficiency determination method is that the central control module determines that the voice conversion efficiency is lower than the allowable range under the preset second proportion condition, and adjusts the field splitting granularity to the second corresponding granularity by calculating the difference between the proportion of sound stuttering duration and the preset first proportion.
[0113] The third type of efficiency determination method is that the central control module determines that the speech conversion efficiency is lower than the allowable range under the preset third proportion condition, initially determines that the classification accuracy of the sound model in the database is lower than the allowable range, and makes a second determination on the classification accuracy of the sound model in the database based on the average number of screenings of several split fields.
[0114] The first preset percentage condition is that the percentage of sound stuttering duration is less than or equal to the first preset percentage; the second preset percentage condition is that the percentage of sound stuttering duration is greater than the first preset percentage and less than or equal to the second preset percentage; the third preset percentage condition is that the percentage of sound stuttering duration is greater than the second preset percentage; and the first preset percentage is less than the second preset percentage.
[0115] Specifically, the percentage of sound pause duration is denoted as Y, the first preset percentage is denoted as Y1, the second preset percentage is denoted as Y2, and the difference between the percentage of sound pause duration and the first preset percentage is denoted as △Y. △Y is set to Y-Y1.
[0116] The method described in this invention uses the central control module to determine whether the speech conversion efficiency is within the allowable range after the initial adjustment of the field splitting granularity. This reduces the decrease in speech conversion efficiency caused by excessively high proportion of sound stuttering time, improves the accuracy of speech synthesis, and further enhances the accuracy and stability of speech synthesis process.
[0117] Please continue reading. Figure 4 As shown, the formula for calculating the percentage of pauses in the sound is:
[0118]
[0119] Where Z represents the percentage of audio pauses, N represents the duration of audio pauses, and K represents the total duration of audio output.
[0120] Please continue reading. Figure 4 As shown, under the second proportion condition, the central control module determines two types of secondary adjustment methods for the field splitting granularity based on the difference between the proportion of sound stuttering duration and the preset first proportion.
[0121] The first type of granularity secondary adjustment method is that the central control module uses a preset fourth granularity adjustment coefficient to adjust the field splitting granularity to the third granularity under the preset first proportion difference condition.
[0122] The second type of granularity secondary adjustment method is that the central control module uses a preset third granularity adjustment coefficient to adjust the field splitting granularity to the fourth granularity under the preset second proportion difference condition.
[0123] The first preset percentage difference condition is that the difference between the percentage of sound stuttering duration and the preset first percentage is less than or equal to the preset percentage difference; the second preset percentage difference condition is that the difference between the percentage of sound stuttering duration and the preset first percentage is greater than the preset percentage difference; and the third preset granularity adjustment coefficient is less than the fourth preset granularity adjustment coefficient.
[0124] Specifically, the preset percentage difference is denoted as △Y0, the preset third granularity adjustment coefficient is denoted as α3, and the preset fourth granularity adjustment coefficient is denoted as α4, where 0 < α3 < α4 < 1. The adjusted field splitting granularity is denoted as V”, V” = V' × (1 + αm) / 2, where αm is the preset m-th granularity adjustment coefficient, and m is set to 3 and 4.
[0125] The method described in this invention, through the central control module, after completing the initial adjustment of the field splitting granularity, adjusts the field splitting granularity a second time to a second corresponding granularity based on the proportion of stuttering duration in the sound. This reduces the impact of inaccurate secondary adjustment of the field splitting granularity on the accuracy of speech synthesis, and further improves the accuracy of speech synthesis and the stability of the speech synthesis process.
[0126] Please continue reading. Figure 4 As shown, the central control module uses two secondary judgment methods to determine whether the classification accuracy of the sound model in the database is within the allowable range based on the average number of screenings of several split fields under the preset third proportion condition.
[0127] The first type of accuracy secondary determination method is that the central control module determines that the classification accuracy of the sound model in the database is within the allowable range under the condition of a preset first screening number;
[0128] The second type of accuracy determination method is that the central control module determines that the classification accuracy of the sound model in the database is lower than the allowable range under the condition of a preset second number of screenings. It then adjusts the number of field classification types in the database to the corresponding number by calculating the difference between the average number of screenings of several split fields and the preset number of screenings.
[0129] The first preset filtering count condition is that the average number of filtering counts for several split fields is less than or equal to the preset filtering count; the second preset filtering count condition is that the average number of filtering counts for several split fields is greater than the preset filtering count.
[0130] Specifically, the preset number of filters is denoted as R0, the average number of filters for several split fields is denoted as R, and the difference between the average number of filters for several split fields and the preset number of filters is denoted as △R. △R is set to R-R0.
[0131] The method described in this invention uses a central control module to determine the classification accuracy of sound models in the database under a preset third ratio condition. This reduces the decline in speech synthesis quality caused by unreasonable average number of screenings of the split fields, improves the anti-interference ability of the speech synthesis process, and further enhances the accuracy and stability of speech synthesis.
[0132] Please continue reading. Figure 4 As shown, the central control module determines two adjustment methods for the number of field classification types in the database based on the difference between the average number of screenings of several split fields and the preset number of screenings, under the condition of the preset second screening number.
[0133] The first type of quantity adjustment method is that the central control module adjusts the quantity of field classification types in the database to the first quantity using a preset first quantity adjustment coefficient under the condition of a preset first screening number difference.
[0134] The second type of quantity adjustment method is that the central control module adjusts the quantity of field classification types in the database to the second quantity using a preset second quantity adjustment coefficient under the condition of a preset second screening number difference.
[0135] The first preset filtering frequency difference condition is that the difference between the average filtering frequency of several split fields and the preset filtering frequency is less than or equal to the preset filtering frequency difference; the second preset filtering frequency difference condition is that the difference between the average filtering frequency of several split fields and the preset filtering frequency is greater than the preset filtering frequency difference; and the first preset quantity adjustment coefficient is less than the second preset quantity adjustment coefficient.
[0136] Specifically, the preset difference in the number of screenings is denoted as △R0, the preset first quantity adjustment coefficient is denoted as γ1, the preset second quantity adjustment coefficient is denoted as γ2, the number of field category types in the database is denoted as L, where 1 < γ1 < γ2, the adjusted number of field category types in the database is denoted as L', and L' is set to L × (1 + 3γw) / 4, where γw is the preset w-th quantity adjustment coefficient, and w is set to 1, 2.
[0137] The method described in this invention adjusts the number of field classification types in the database to a corresponding number based on the average number of screenings after several split fields through a central control module. This reduces the impact of inaccurate adjustment of the number of field classification types in the database on the anti-interference ability of the speech synthesis process, and further improves the accuracy and stability of the speech synthesis process.
[0138] Example 1
[0139] In this embodiment 1, the central control module determines the adjustment method for the field splitting granularity based on the difference between the variance of the sound playback speed at several time points and the preset second variance under the preset third variance condition. The preset variance difference is denoted as △Q0, the preset first granularity adjustment coefficient is denoted as α1, the preset second granularity adjustment coefficient is denoted as α2, and the field splitting granularity is denoted as V. Where 1 < α1 < α2, α1 = 1.2, α2 = 1.4, △Q0 = 3, and V = 4 bytes.
[0140] In this embodiment 1, △Q = 5 is obtained. The central control module determines that △Q > △Q0 and uses the preset first granularity adjustment coefficient α1 to adjust the field splitting granularity to the second granularity V'. V' = 4 bytes × (1 + 1.2) / 2 = 5 bytes is calculated.
[0141] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.
[0142] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A speech synthesis method, characterized in that, include: Step S1: The information acquisition terminal acquires the text information to be processed and splits the text information to be processed into several fields of different lengths, and retrieves the field data corresponding to the several fields of different lengths from the database. Step S2: Convert the field data corresponding to the several fields of different lengths into several corresponding audio data, and use an audio synthesizer to convert the several corresponding audio data into speech; Step S3: When the audio synthesizer completes the speech conversion of several corresponding audio data, the sound speed meter is used to detect the sound playback speed at several time points and the variance of the sound playback speed at several time points is calculated. The central control module determines the stability of the speech synthesis process based on the variance of the sound playback speed at several time points. Step S4: When the central control module determines that the stability of the speech synthesis process is lower than the allowable range, the field splitting granularity is adjusted to the first corresponding granularity; or, when it is initially determined that the accuracy of calling database data is lower than the allowable range, the bit depth is adjusted to the corresponding depth according to the overlap of the sound waveform. Step S5: When the central control module completes the initial adjustment of the field splitting granularity, the central control module adjusts the field splitting granularity to the second corresponding granularity according to the proportion of sound stuttering duration, or adjusts the number of field classification types in the database to the corresponding number according to the average number of screenings of several split fields.
2. The speech synthesis method according to claim 1, characterized in that, The central control module determines whether the stability of the speech synthesis process is within an acceptable range based on three methods, including the variance of the sound playback speed at several time points. The first type of determination method is that the central control module determines that the stability of the speech synthesis process is within the allowable range under the preset first variance condition; The second type of determination method is that the central control module determines that the stability of the speech synthesis process is lower than the allowable range under the preset second variance condition, initially determines that the accuracy of database calling is lower than the allowable range, and makes a second determination on whether the accuracy of database calling is lower than the allowable range based on the overlap of the sound waveform. The third type of determination method is that the central control module determines that the stability of the speech synthesis process is lower than the allowable range under the preset third variance condition, and adjusts the field splitting granularity to the first corresponding granularity by calculating the difference between the variance of the sound playback speed at several time points and the preset second variance. The first preset variance condition is that the variance of the sound playback speed at a certain number of time points is less than or equal to the first preset variance; the second preset variance condition is that the variance of the sound playback speed at a certain number of time points is greater than the first preset variance and less than or equal to the second preset variance; the third preset variance condition is that the variance of the sound playback speed at a certain number of time points is greater than the second preset variance; and the first preset variance is less than the second preset variance.
3. The speech synthesis method according to claim 2, characterized in that, The central control module, under the preset third variance condition, determines two types of adjustment methods for the field splitting granularity based on the difference between the variance of the sound playback speed at several time points and the preset second variance. The first type of adjustment method is that the central control module adjusts the field splitting granularity to the first granularity using a preset first granularity adjustment coefficient under the condition of a preset first variance difference. The second type of adjustment method is that the central control module adjusts the field splitting granularity to the second granularity under the condition of a preset second variance difference; The first preset variance difference condition is that the difference between the variance of the sound playback speed at a certain time point and the preset second variance is less than or equal to the preset variance difference; the second preset variance difference condition is that the difference between the variance of the sound playback speed at a certain time point and the preset second variance is greater than the preset variance difference; and the first preset granularity adjustment coefficient is less than the second preset granularity adjustment coefficient.
4. The speech synthesis method according to claim 3, characterized in that, The central control module employs two secondary determination methods based on the overlap of sound waveforms under the preset second variance condition to determine whether the accuracy of database retrieval is below the allowable range. The first type of accuracy secondary determination method is that the central control module determines the accuracy of database calls within the allowable range under the preset first overlap condition; The second type of accuracy secondary determination method is that when the accuracy of the database call is lower than the allowable range under the preset second overlap condition, the central control module adjusts the bit depth to the corresponding depth by calculating the difference between the overlap of the sound waveform and the preset overlap. The first preset overlap condition is that the overlap of the sound waveforms is less than or equal to a preset overlap condition; the second preset overlap condition is that the overlap of the sound waveforms is greater than a preset overlap condition.
5. The speech synthesis method according to claim 4, characterized in that, The central control module determines two adjustment methods for bit depth based on the difference between the overlap of the sound waveform and the preset overlap under the preset second overlap condition. The first type of bit depth adjustment method is that the central control module adjusts the bit depth to the first depth using a preset first bit depth adjustment coefficient under the condition of a preset first overlap difference. The second type of bit depth adjustment method is that the central control module adjusts the bit depth to the second depth using a preset second bit depth adjustment coefficient under the condition of a preset second overlap difference. The first preset overlap difference condition is that the difference between the overlap of the sound waveform and the preset overlap is less than or equal to the preset overlap difference; the second preset overlap difference condition is that the difference between the overlap of the sound waveform and the preset overlap is greater than the preset overlap difference; and the first preset depth adjustment coefficient is less than the second preset depth adjustment coefficient.
6. The speech synthesis method according to claim 5, characterized in that, After the central control module completes the initial adjustment of the field splitting granularity, it determines whether the speech conversion efficiency is within the allowable range based on three methods according to the proportion of sound stuttering duration. The first type of efficiency determination method is that the central control module determines that the voice conversion efficiency is within the allowable range under a preset first proportion condition; The second type of efficiency determination method is that the central control module determines that the voice conversion efficiency is lower than the allowable range under the preset second proportion condition, and adjusts the field splitting granularity to the second corresponding granularity by calculating the difference between the proportion of sound stuttering duration and the preset first proportion. The third type of efficiency determination method is that the central control module determines that the speech conversion efficiency is lower than the allowable range under the preset third proportion condition, initially determines that the classification accuracy of the sound model in the database is lower than the allowable range, and makes a second determination on the classification accuracy of the sound model in the database based on the average number of screenings of several split fields. The first preset percentage condition is that the percentage of sound stuttering duration is less than or equal to the first preset percentage; the second preset percentage condition is that the percentage of sound stuttering duration is greater than the first preset percentage and less than or equal to the second preset percentage; the third preset percentage condition is that the percentage of sound stuttering duration is greater than the second preset percentage; and the first preset percentage is less than the second preset percentage.
7. The speech synthesis method according to claim 6, characterized in that, The formula for calculating the percentage of pauses in the sound is: Where Z represents the percentage of audio pauses, N represents the duration of audio pauses, and K represents the total duration of audio output.
8. The speech synthesis method according to claim 7, characterized in that, Under the second proportion condition, the central control module determines two types of secondary adjustment methods for the field splitting granularity based on the difference between the proportion of sound stuttering duration and the preset first proportion. The first type of granularity secondary adjustment method is that the central control module uses a preset fourth granularity adjustment coefficient to adjust the field splitting granularity to the third granularity under the preset first proportion difference condition. The second type of granularity secondary adjustment method is that the central control module uses a preset third granularity adjustment coefficient to adjust the field splitting granularity to the fourth granularity under the preset second proportion difference condition. The first preset percentage difference condition is that the difference between the percentage of sound stuttering duration and the preset first percentage is less than or equal to the preset percentage difference; the second preset percentage difference condition is that the difference between the percentage of sound stuttering duration and the preset first percentage is greater than the preset percentage difference; and the third preset granularity adjustment coefficient is less than the fourth preset granularity adjustment coefficient.
9. The speech synthesis method according to claim 8, characterized in that, The central control module, under the preset third proportion condition, determines whether the classification accuracy of the sound model in the database is within the allowable range using two secondary judgment methods based on the average number of screenings after several split fields. The first type of accuracy secondary determination method is that the central control module determines that the classification accuracy of the sound model in the database is within the allowable range under the condition of a preset first screening number; The second type of accuracy determination method is that the central control module determines that the classification accuracy of the sound model in the database is lower than the allowable range under the condition of a preset second number of screenings. It then adjusts the number of field classification types in the database to the corresponding number by calculating the difference between the average number of screenings of several split fields and the preset number of screenings. The first preset filtering count condition is that the average number of filtering counts for several split fields is less than or equal to the preset filtering count; the second preset filtering count condition is that the average number of filtering counts for several split fields is greater than the preset filtering count.
10. The speech synthesis method according to claim 9, characterized in that, The central control module, under the preset second filtering count condition, determines two adjustment methods for the number of field classification types in the database based on the difference between the average filtering count of several split fields and the preset filtering count. The first type of quantity adjustment method is that the central control module adjusts the quantity of field classification types in the database to the first quantity using a preset first quantity adjustment coefficient under the condition of a preset first screening number difference. The second type of quantity adjustment method is that the central control module adjusts the quantity of field classification types in the database to the second quantity using a preset second quantity adjustment coefficient under the condition of a preset second screening number difference. The first preset filtering frequency difference condition is that the difference between the average filtering frequency of several split fields and the preset filtering frequency is less than or equal to the preset filtering frequency difference; the second preset filtering frequency difference condition is that the difference between the average filtering frequency of several split fields and the preset filtering frequency is greater than the preset filtering frequency difference; and the first preset quantity adjustment coefficient is less than the second preset quantity adjustment coefficient.
Citation Information
Patent Citations
Speech synthesis methods and speech synthesis devices
CN105845125B
Text-to-audio conversion method and device, computer equipment and storage medium
CN111312207A
Speech synthesis method and device, computer equipment and storage medium
CN113421548A
