A flow-controlled Chinese speech synthesis method and system

Abstract

The present invention discloses a streaming controllable Chinese speech synthesis method and system, firstly, the Chinese text to be processed is marked with the SSML specification to synthesize speech variable parameters, then the marked Chinese text is segmented, and finally the segmented text is spliced, and when the splicing length reaches a preset threshold, the spliced ​​text is returned in a streaming manner according to the synthesized speech variable parameters and the synthesized audio. The Chinese speech synthesis method and system provided by the embodiment of the present invention synthesizes a relatively smooth and natural speech with a very low response time, which can be well applied to human-computer interaction scenarios such as voice assistants, and the user can customize the text synthesis speech parameters, such as pronunciation frequency, speech rhythm and sound effects, etc., to meet customization requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of speech synthesis, and in particular to a flow-controllable Chinese speech synthesis method and system. Background Art

[0002] Speech synthesis is the task of converting input text into standard fluent human speech. Speech synthesis technology provides a way to convert readable text into audible sound, and has been widely used in many fields, such as voice assistants, smart education, smart customer service, etc.

[0003] In scenarios such as voice assistants that require human-computer interaction, the real-time rate of the speech synthesis system is very high, and a very low latency is required to achieve a good product experience. For non-streaming speech synthesis systems, it is difficult to achieve the above real-time rate requirements. The advantage of a streaming speech synthesis system is that it can synthesize and play at the same time, as long as the audio playback time is greater than the time required for audio synthesis.

[0004] One of the most difficult problems in existing Chinese speech synthesis systems is that a module can convert all non-standardized Chinese text into standardized text that can be pronounced correctly, because there are polyphones and different combinations of numbers and symbols. The controllable speech synthesis system we proposed allows users to mark non-standardized text to ensure that the pronunciation of the synthesized speech meets the user's expectations, thereby solving the problem of inaccurate pronunciation. Summary of the invention

[0005] Therefore, in order to overcome the defects of the Chinese speech synthesis system in the prior art, that is, the pronunciation is not standardized and accurate, the present invention provides a flow-controllable Chinese speech synthesis method and system.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a flow-controlled Chinese speech synthesis method, comprising: using SSML standard markings to synthesize speech variable parameters of a Chinese text to be processed;

[0008] Segment the marked Chinese text;

[0009] The segmented texts are spliced ​​together, and when the splicing length reaches a preset threshold, the spliced ​​texts are returned in a streaming manner according to the synthesized audio according to the synthesized speech variable parameters.

[0010] Preferably, the process of segmenting the Chinese text includes:

[0011] The long Chinese text is segmented according to punctuation marks. If the length of the segmented text exceeds the preset threshold, the segmented text is segmented again using a word segmentation tool.

[0012] Preferably, the preset threshold is determined according to the semantic completeness of the concatenated text.

[0013] Preferably, the preset threshold is determined according to a preset streaming audio return time.

[0014] Preferably, the synthesized speech variable parameters include:

[0015] Audio parameters, including: audio rate, volume, and pitch;

[0016] Text pronunciation parameters, including marking different types of numbers in the text as different pronunciation types and polyphonic characters;

[0017] Background music parameters, set different background music for different text types;

[0018] Pause parameter, set different pause times after different texts.

[0019] Preferably, different types of numbers in the text pronunciation parameters are marked with different pronunciation types, including: pronunciation as integers or decimals, pronunciation as numbers, pronunciation in the usual way of telephone numbers, pronunciation as addresses, account names or nicknames, reading the text in the label character by character, reading the text in the label as punctuation marks, pronunciation by date, pronunciation by time, pronunciation by amount, and pronunciation by unit of measurement.

[0020] In a second aspect, an embodiment of the present invention provides a streaming controllable Chinese speech synthesis system, comprising:

[0021] A synthetic speech variable parameter marking module is used to mark the synthetic speech variable parameters of the Chinese text to be processed using the SSML specification;

[0022] A text segmentation module is used to segment the marked Chinese text;

[0023] The synthetic speech streaming return module is used to splice the segmented texts. When the splicing length reaches a preset threshold, the spliced ​​texts are synthesized and returned in a streaming manner according to the synthetic speech variable parameters and the synthetic audio.

[0024] In a third aspect, an embodiment of the present invention provides a computer device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the streaming controllable Chinese speech synthesis method of the first aspect of the embodiment of the present invention.

[0025] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the streaming controllable Chinese speech synthesis method of the first aspect of the embodiment of the present invention.

[0026] The technical solution of the present invention has the following advantages:

[0027] The streaming controllable Chinese speech synthesis method and system provided by the present invention use SSML specifications to mark the synthesized speech variable parameters of the Chinese text to be processed; segment the marked Chinese text; splice the segmented text, and when the splicing length reaches a preset threshold, the spliced ​​text is returned in a streaming manner according to the synthesized speech variable parameters and synthesized audio. The method and system provided by the embodiment of the present invention reasonably segment and splice the text so that the final synthesized speech is relatively smooth, natural, and has a very low response time, and can be well applied to human-computer interaction scenarios such as voice assistants; and users can use SSML to customize text synthesis speech parameters, such as pronunciation frequency, speech rhythm, and sound effects, etc., to meet customization needs, so as to ensure that the pronunciation of the synthesized speech meets the user's expectations, thereby solving the problem of inaccurate pronunciation. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 A flowchart of a specific example of a flow-controlled Chinese speech synthesis method provided in an embodiment of the present invention;

[0030] Figure 2 A module composition diagram of a specific example of a flow-controllable Chinese speech synthesis system provided in an embodiment of the present invention;

[0031] Figure 3 A composition diagram of a specific example of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0034] Example 1

[0035] The embodiment of the present invention provides a flow-controlled Chinese speech synthesis method, such as Figure 1 As shown, including:

[0036] Step S1: Using the SSML standard to mark the Chinese text to be processed into synthetic speech variable parameters.

[0037] SSML (speech synthesis markup language) is an XML-based markup standard that can help generate synthesized speech in web pages and other applications. The basic function of the markup language is to provide a standard method for synthesizing content to control the various parameters of speech synthesis. The format of SSML text is: <speak> The text that needs to call the SSML tag< / speak> .

[0038] In actual applications, users can use SSML to set the text synthesis speech variable parameters to customize the final speech synthesis effect. The synthesis speech variable parameters include the following:

[0039] 1) Adjust audio parameters

[0040] Example of use:<speak rate=11volume=5pitch=2> Who are you?

[0041] Effect: Use the three SSML parameters rate, volume, and pitch to adjust the speech rate, volume, and pitch of the audio.

[0042] 2) Adjust text pronunciation

[0043] Example of use: <speak> This year is<say-as interpret-as=\"digits\"> 2021< / speak>

[0044] Effect: The numbers in the text can be marked with different pronunciation types. The pronunciation types provided in the embodiments of the present invention are: Cardinal, Digit, Telephone, Address, id, Characters, Characters, Date, Time, Currency, Measure. Cardinal is pronounced as an integer or a decimal; Digit is pronounced as a number; Telephone is pronounced in the common way of a telephone number; Address is pronounced as an address; id is applicable to account names, nicknames, etc. Characters reads the text within the tag character by character; Characters reads the text within the tag in the way of punctuation marks; Date is pronounced as a date; Time is pronounced as a time; Currency is pronounced as an amount; Measure is pronounced as a measurement unit.

[0045] 3) Add background music

[0046] Usage example: <speak bgm="url"backgroundMusicVolume="30">The old trees on the shady cliff are shrouded in mist

[0047] Effect: Add the specified background music to the text.

[0048] 4) Add pauses

[0049] Usage example: <speak> Please close your eyes and take a rest<break time="500ms" / > Okay, please open your eyes.< / speak>

[0050] Effect: The above SSML tag represents a 500 - millisecond pause after "under".

[0051] The above parameter settings are only for illustrative purposes and are limited to this. In actual applications, users can make appropriate settings according to actual needs. By marking non - standard texts, it is ensured that the pronunciation of the synthesized voice meets the expectations of the user, thus solving the problem of inaccurate pronunciation.

[0052] Step S2: Segment the marked Chinese text.

[0053] In the embodiments of the present invention, the Chinese long text is segmented according to punctuation marks. When the length of the segmented text exceeds the preset threshold, a word - segmentation tool is used to perform secondary segmentation on the segmented text. When segmenting according to punctuation marks, if the length of the segmented text exceeds the preset threshold, it will affect the Chinese return response time. Therefore, it is necessary to use a word - segmentation tool to perform secondary segmentation on the segmented text. The word - segmentation tool is, for example, based on the dictionary - based word - segmentation algorithm, the statistical machine - learning algorithm, or a combination of the machine - learning algorithm and the dictionary. On the one hand, it can improve the word - segmentation accuracy, and on the other hand, it can improve the domain adaptability.

[0054] Step S3: splice the segmented texts, and when the splicing length reaches a preset threshold, return the spliced ​​texts in a streaming manner according to the synthesized speech variable parameters and the synthesized audio.

[0055] In the embodiment of the present invention, the threshold of the secondary segmentation and the preset threshold of the splicing length can be set to be the same (for example, the length is 10), determined according to the semantic completeness of the text and / or according to the preset time of the streaming audio return. The purpose of segmentation according to the symbol and the word segmentation result is to enable the streaming returned audio to maintain semantic integrity and to be as close to the non-streaming returned audio as possible, that is, the audio synthesized after segmentation is close to the audio without segmentation and synthesis, and the segmented text has no effect on the synthesized audio. By segmenting and splicing long texts, the high delay of synthesizing long texts can be reduced.

[0056] The speech synthesized by the streaming controllable Chinese speech synthesis method provided in the embodiment of the present invention can reasonably segment and splice the text so that the final synthesized speech is relatively smooth, natural, and has a very low response time. It can be well applied to human-computer interaction scenarios such as voice assistants; and the user can use SSML to customize text-synthesized speech parameters, such as pronunciation frequency, speech rhythm, and sound effects, to meet customization needs, so as to ensure that the pronunciation of the synthesized speech meets the user's expectations, thereby solving the problem of inaccurate pronunciation.

[0057] Example 2

[0058] The embodiment of the present invention provides a flow controllable Chinese speech synthesis system, such as Figure 2 As shown, including:

[0059] The synthetic speech variable parameter marking module 1 uses the SSML specification to mark the synthetic speech variable parameters of the Chinese text to be processed; this module executes the method described in step S1 of embodiment 1, which will not be repeated here.

[0060] The text segmentation module 2 is used to segment the marked Chinese text; the module executes the method described in step S2 of embodiment 1, which will not be repeated here.

[0061] The synthesized speech streaming return module 3 is used to splice the segmented text. When the splicing length reaches a preset threshold, the spliced ​​text is returned in a streaming manner according to the synthesized speech variable parameters and the synthesized audio; this module executes the method described in step S3 of Example 1, which will not be repeated here.

[0062] The streaming controllable Chinese speech synthesis system provided by the embodiment of the present invention can reasonably segment and splice the text so that the final synthesized speech is relatively smooth, natural, and has a very low response time. It can be well applied to human-computer interaction scenarios such as voice assistants; and users can use SSML to customize text-to-speech parameters, such as pronunciation frequency, speech rhythm, and sound effects, to meet customization needs, so as to ensure that the pronunciation of the synthesized speech meets the user's expectations, thereby solving the problem of inaccurate pronunciation.

[0063] Example 3

[0064] An embodiment of the present invention provides a computer device, such as Figure 3 As shown, it includes: at least one processor 401, such as a CPU (Central Processing Unit), at least one communication interface 403, a memory 404, and at least one communication bus 402. Among them, the communication bus 402 is used to realize the connection and communication between these components. Among them, the communication interface 403 may include a display screen (Display), a keyboard (Keyboard), and the optional communication interface 403 may also include a standard wired interface and a wireless interface. The memory 404 may be a high-speed RAM memory (Ramdom Access Memory, volatile random access memory) or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 404 may also be at least one storage device located away from the aforementioned processor 401. Among them, the processor 401 can execute the streaming controllable Chinese speech synthesis method of Example 1. A set of program codes are stored in the memory 404, and the processor 401 calls the program code stored in the memory 404 to execute the streaming controllable Chinese speech synthesis method of Example 1.

[0065] The communication bus 402 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The communication bus 402 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 The fact that only one line is used in the diagram does not mean that there is only one bus or only one type of bus.

[0066] Among them, the memory 404 may include a volatile memory (English: volatile memory), such as a random access memory (English: random-access memory, abbreviated: RAM); the memory may also include a non-volatile memory (English: non-volatile memory), such as a flash memory (English: flash memory), a hard disk drive (English: hard disk drive, abbreviated: HDD) or a solid-state drive (English: solid-state drive, abbreviated: SSD); the memory 404 may also include a combination of the above types of memory.

[0067] The processor 401 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and a NP.

[0068] The processor 401 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0069] Optionally, the memory 404 is also used to store program instructions. The processor 401 can call the program instructions to implement the streaming controllable Chinese speech synthesis method in the embodiment 1 of the present application.

[0070] The embodiment of the present invention also provides a computer-readable storage medium, on which computer-executable instructions are stored, and the computer-executable instructions can execute the streaming controllable Chinese speech synthesis method of embodiment 1. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0071] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the scope of protection of the invention.

Claims

1. A flow-controlled Chinese speech synthesis method, characterized in that: include: The Chinese text to be processed is marked with synthetic speech variable parameters using SSML standards; Segment the marked Chinese text, including: segmenting the long Chinese text according to punctuation marks, and if the length of the segmented text exceeds a preset threshold, using a word segmentation tool to perform secondary segmentation on the segmented text; The segmented text is spliced, and when the splicing length reaches a preset threshold, the spliced ​​text is returned in streaming form according to the synthesized speech variable parameters and the synthesized audio, and the preset threshold is determined according to the semantic completeness of the spliced ​​text and / or according to the preset streaming audio return time.

2. The flow-controlled Chinese speech synthesis method according to claim 1, characterized in that: The synthetic speech variable parameters include: Audio parameters, including: audio rate, volume, and pitch; Text pronunciation parameters, including marking different types of numbers in the text as different pronunciation types and polyphonic characters; Background music parameters, set different background music for different text types; Pause parameter, set different pause times after different texts.

3. The flow-controlled Chinese speech synthesis method according to claim 2, characterized in that: Different types of numbers in the text pronunciation parameters are marked with different pronunciation types, including: pronunciation by integer or decimal, pronunciation by number, pronunciation in the common way of telephone number, pronunciation by address, account name or nickname, reading the text in the label one by one, reading the text in the label in the form of punctuation marks, pronunciation by date, pronunciation by time, pronunciation by amount, and pronunciation by unit of measurement.

4. A streaming controllable Chinese speech synthesis system, characterized in that: include: A synthetic speech variable parameter marking module is used to mark the synthetic speech variable parameters of the Chinese text to be processed using the SSML specification; The text segmentation module is used to segment the marked Chinese text, including: segmenting the long Chinese text according to punctuation marks, and if the length of the segmented text exceeds a preset threshold, using a word segmentation tool to perform secondary segmentation on the segmented text; The synthesized speech streaming return module is used to splice the segmented text. When the splicing length reaches a preset threshold, the spliced ​​text is returned in a streaming manner according to the synthesized speech variable parameters and the synthesized audio. The preset threshold is determined according to the semantic completeness of the spliced ​​text and / or according to the preset streaming audio return time.

5. A computer device, characterized in that: include: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the streaming controllable Chinese speech synthesis method described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute any of the streaming controllable Chinese speech synthesis methods described in 1-3.

Images