Method, system and medium for improving the performance of TTS model in processing long text

Abstract

The present application discloses a method, system and medium for improving the performance of a TTS model in processing long texts, mainly relating to the technical field of TTS models, and is used to solve the problems existing in existing TTS models: poor segmentation effect for long or complex texts or high hardware cost requirements. The method includes: segmenting a sample text in a preset sample text library into a number of words, and then constructing a commonly used dictionary set; traversing each word in the text to be converted, and then converting the misspelled words in the text to be converted into correct words based on the similarity between each word and the words in the commonly used dictionary set; converting the text to be converted into a preset standard format; segmenting sentences with a length greater than L into sentences with a length less than or equal to L; determining the predicted emotional parameters of each sentence through an emotional recognition model; the TTS model generates sentence audio files based on the sentences and the predicted emotional parameters corresponding to the sentences, and then merges them into a complete audio file.

Description

Technical Field

[0001] The present application relates to the field of live broadcast auxiliary application technology, and in particular to a method, system and medium for improving the performance of a TTS model in processing long texts. Background Art

[0002] The Text To Speech (TTS) model is a technology that can convert input text into simulated natural human voices. Traditional speech synthesis methods usually take the entire text as input and directly input it into the TTS model for processing. However, for long or complex texts, this method has the following problems: Low processing efficiency of the TTS model: The semantic information of long texts is complex and it takes a long time for the TTS model to process it, resulting in slow speech synthesis and reduced speech synthesis quality.

[0003] Existing text segmentation methods include character segmentation, which uses simple static character data blocks. This method makes sentences incomplete and affects the output effect of the TTS model; recursive character text segmentation, which uses recursive segmentation based on a list of delimiters. This method is still difficult to handle when faced with long sentences or complex texts; specific document segmentation, which uses various segmentation methods for different document types (PDF, Python, Markdown) but has poor universality; and LLM (Large Language Model) segmentation, which uses an experimental text segmentation method similar to a proxy system. This method has high hardware cost requirements and requires the use of a large language model for capacity analysis.

[0004] Therefore, there is an urgent need for a method, system and medium to improve the performance of the TTS model in processing long texts, so as to solve the problems existing in the existing TTS model: poor segmentation effect for long or complex texts or high hardware cost requirements. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present application provides a system, method and medium for improving the performance of the TTS model in processing long texts, so as to solve the problems existing in the existing TTS model: poor segmentation effect for long or complex texts or high hardware cost requirements.

[0006] In a first aspect, the present application provides a method for improving the performance of a TTS model in processing long texts, the method comprising:

[0007] The sample text in the preset sample text library is divided into several words, and then a common dictionary set is constructed; each word of the text to be converted is traversed, and then based on the similarity between each word and the words in the common dictionary set, the wrong characters in the text to be converted are converted into correct words; the text to be converted is converted into a preset standard format; the sentence is segmented according to the punctuation mark and the period, and it is judged whether the length of the segmented sentence is greater than the preset length L; when there are sentences greater than L, the sentence is segmented according to the preset end symbol, and it is judged again whether the length of the segmented sentence is greater than the preset length L; when there are still sentences greater than L, the sentence is segmented according to the preset segmentation adverb, and it is judged again whether the length of the segmented sentence is greater than the preset length L; when there are still sentences greater than L, the sentence is segmented according to the preset segmentation conjunction, and it is judged again whether the length of the segmented sentence is greater than the preset length L; when there are still sentences greater than L, the sentence with a length greater than L is segmented into sentences less than or equal to L by using a syntactic segmentation tool; through the emotion recognition model, the predicted emotion parameters of each sentence are determined; the TTS model generates sentence audio files according to the predicted emotion parameters corresponding to the sentences, and then merges them into a complete audio file.

[0008] Furthermore, each word in the text to be converted is traversed, and then based on the similarity between each word and the words in the commonly used dictionary set, the wrong words in the text to be converted are converted into correct words, specifically including:

[0009] Traverse each word of the text to be converted to determine whether the current word exists in the commonly used dictionary set; when the current word does not exist in the commonly used dictionary set, convert the current word into pinyin; calculate the similarity between the current pinyin and the pinyin of each word in the commonly used dictionary set; when there is a pinyin of a word with a similarity greater than a preset similarity threshold, determine that the word corresponding to the pinyin of the word is the correct word of the current word; when there is no pinyin of a word with a similarity greater than the preset similarity threshold, determine that the current word is the correct word.

[0010] Further, converting the text to be converted into a preset standard format specifically includes:

[0011] Convert several consecutive blank characters in the text to be converted into one blank character; remove the preset text characters in the text to be converted; wherein the preset text characters at least include: the leading and trailing spaces of the text, line breaks, and formatting marks.

[0012] Furthermore, the sentence is segmented according to the preset ending symbol, specifically including:

[0013] Match the paragraph boundaries in the text to be converted according to the regular expression "\n\s*\n|[\。\?\!]+", and then break the sentences at the paragraph boundaries; break the sentences according to the ending symbols: ".", "!", "?", "", ";", ":".

[0014] Furthermore, before the TTS model generates a sentence audio file based on the sentence and the predicted emotion parameter corresponding to the sentence, the method includes:

[0015] After the text to be converted is segmented, the sentences are encoded and sorted according to the segmentation order.

[0016] Merging into a complete audio file specifically includes: encoding and merging the sentence audio files according to the segmentation order to obtain a complete audio file.

[0017] In a second aspect, the present application provides a system for improving the performance of a TTS model in processing long texts, the system comprising:

[0018] The construction module is used to divide the sample text in the preset sample text library into several words, and then build a common dictionary set; the conversion module is used to traverse each word of the text to be converted, and then convert the wrong words in the text to be converted into correct words based on the similarity between each word and the words in the common dictionary set; convert the text to be converted into a preset standard format, and convert the preset markers in the text to be converted into spaces; the segmentation module is used to segment the sentences according to punctuation marks and periods, and determine whether the length of the segmented sentences is greater than the preset length L; when there is a sentence greater than L, segment it according to the preset end symbol, and again determine whether the length of the segmented sentences is greater than the preset Length L; when there are still sentences longer than L, segment them according to the preset segmentation adverbs, and judge whether the length of the segmented sentences is longer than the preset length L; when there are still sentences longer than L, segment them according to the preset segmentation conjunctions, and judge whether the length of the segmented sentences is longer than the preset length L; when there are still sentences longer than L, use the syntactic segmentation tool to segment the sentences longer than L into sentences less than or equal to L in length; a merging module is used to determine the predicted emotional parameters of each sentence through the emotion recognition model; the TTS model generates sentence audio files according to the predicted emotional parameters corresponding to the sentences, and then merges them into a complete audio file.

[0019] Furthermore, the conversion module includes a first conversion unit, which is used to traverse each word of the text to be converted and determine whether the current word exists in a commonly used dictionary set; when the current word does not exist in the commonly used dictionary set, convert the current word into pinyin; calculate the similarity between the current pinyin and the pinyin of each word in the commonly used dictionary set; when there is a pinyin of a word with a similarity greater than a preset similarity threshold, determine that the word corresponding to the pinyin of the word is the correct word of the current word; when there is no pinyin of a word with a similarity greater than a preset similarity threshold, determine that the current word is the correct word.

[0020] Furthermore, the conversion module includes a second conversion unit, which is used to convert several consecutive blank characters in the text to be converted into one blank character; remove preset text characters in the text to be converted; wherein the preset text characters at least include: spaces at the beginning and end of the text, line breaks, and formatting marks.

[0021] Furthermore, the segmentation module includes a sentence segmentation unit, which is used to match paragraph boundaries in the text to be converted according to the regular expression "\n\s*\n|[\。\?\!]+", and then segment at the paragraph boundaries; and segment according to the ending symbols: ".", "!", "?", "", ";", ":".

[0022] In a third aspect, the present application provides a non-volatile computer storage medium having computer instructions stored thereon, which, when executed, implement a method for improving the performance of a TTS model in processing long texts as described above.

[0023] Those skilled in the art can understand that the present application has at least the following beneficial effects:

[0024] This application can correct potential misspellings before segmentation, thus avoiding segmentation errors caused by misspellings. In addition, the longer the text, the longer the conversion time of the TTS model, and the voice quality will also decline. This application achieves effective segmentation of long sentences or complex texts through punctuation, preset ending symbols, spaCy tool library (syntactic segmentation tool), etc. In addition, there is no need to rely on large language models with high hardware cost requirements, which solves the problem of high hardware cost requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Some embodiments of the present disclosure are described below with reference to the accompanying drawings, in which:

[0026] Figure 1 This is a flow chart of a method for improving the performance of a TTS model in processing long texts provided in an embodiment of the present application.

[0027] Figure 2 It is a schematic diagram of the internal structure of a system for improving the performance of a TTS model in processing long texts provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] It should be understood by those skilled in the art that the embodiments described below are only preferred embodiments of the present disclosure, and do not mean that the present disclosure can only be implemented through the preferred embodiments. The preferred embodiments are only used to explain the technical principles of the present disclosure, and are not used to limit the protection scope of the present disclosure. Based on the preferred embodiments provided by the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work should still fall within the protection scope of the present disclosure.

[0029] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0030] The technical solution proposed in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0031] The present application embodiment provides a method for improving the performance of a TTS model in processing long texts, such as Figure 1 As shown, the method provided in the embodiment of the present application mainly includes the following steps:

[0032] Step 110: Segment the sample text in the preset sample text library into a number of words, and then construct a commonly used dictionary set.

[0033] It should be noted that the sample text is an existing text, and those skilled in the art can select a sample text of a corresponding field according to the field to which the text to be converted belongs.

[0034] Step 120, traverse each word in the text to be converted, and then convert the misspelled words in the text to be converted into correct words based on the similarity between each word and the words in the commonly used dictionary set; convert the text to be converted into a preset standard format.

[0035] Here, each word in the text to be converted is traversed, and then based on the similarity between each word and the words in the commonly used dictionary set, the wrong words in the text to be converted are converted into correct words, which can be specifically:

[0036] Traverse each word of the text to be converted to determine whether the current word exists in the commonly used dictionary set; when the current word does not exist in the commonly used dictionary set, convert the current word into pinyin; calculate the similarity between the current pinyin and the pinyin of each word in the commonly used dictionary set; when there is a pinyin of a word with a similarity greater than a preset similarity threshold, determine that the word corresponding to the pinyin of the word is the correct word of the current word; when there is no pinyin of a word with a similarity greater than the preset similarity threshold, determine that the current word is the correct word.

[0037] It should be noted that the preset similarity threshold may be 0.8. The method for calculating the similarity may be any existing solution capable of calculating the pinyin similarity, and this application does not limit this.

[0038] The process of converting the text to be converted into a preset standard format includes:

[0039] Convert several consecutive blank characters in the text to be converted into one blank character; remove the preset text characters in the text to be converted.

[0040] The preset text characters include at least: spaces at the beginning and end of the text, line breaks, and formatting marks.

[0041] Step 130, segment the sentences according to punctuation marks and periods, and determine whether the length of the segmented sentences is greater than the preset length L; when there are sentences longer than L, segment them according to the preset end symbol, and again determine whether the length of the segmented sentences is greater than the preset length L; when there are still sentences longer than L, segment them according to the preset segmentation adverbs, and again determine whether the length of the segmented sentences is greater than the preset length L; when there are still sentences longer than L, segment them according to the preset segmentation conjunctions, and again determine whether the length of the segmented sentences is greater than the preset length L; when there are still sentences longer than L, use the syntactic segmentation tool to segment the sentences longer than L into sentences less than or equal to L in length.

[0042] It should be noted that the preset length L can be any feasible length, and this application does not limit this. When segmenting is performed subsequently, only sentences greater than the preset length are segmented.

[0043] Among them, the sentence is segmented according to the preset ending symbol, specifically including:

[0044] Match the paragraph boundaries in the text to be converted according to the regular expression "\n\s*\n|[\。\?\!]+", and then break the sentences at the paragraph boundaries; break the sentences according to the ending symbols: ".", "!", "?", "", ";", ":".

[0045] The preset sentence-breaking adverbs may be: "then", "however", "therefore", "so", etc., and the sentence is broken before the preset sentence-breaking adverbs. The preset sentence-breaking conjunctions may be "and", "or", "and", etc., and the sentence is broken before the preset sentence-breaking conjunctions.

[0046] After the clause segmentation is completed, the clauses are combined into a list or array in the original order. The processed short sentence list or array is returned.

[0047] Step 140: Determine the predicted emotion parameters of each sentence through the emotion recognition model; the TTS model generates sentence audio files based on the sentences and the predicted emotion parameters corresponding to the sentences, and then merges them into a complete audio file.

[0048] It should be noted that in order to optimize the emotional expression of clause-to-speech conversion, emotional recognition of clauses is added. The emotional recognition model will give "positive", "neutral" and "negative" emotional judgments, and adjust the emotional parameters respectively and send them to the TTS model to make the generated speech more emotionally expressive.

[0049] The merging into a complete audio file specifically includes: encoding and merging the sentence audio files according to the segmentation order to obtain a complete audio file.

[0050] besides, Figure 2 A system for improving the performance of a TTS model in processing long texts is provided in an embodiment of the present application. Figure 2 As shown, the system provided in the embodiment of the present application mainly includes:

[0051] The construction module 210 is used to divide the sample text in the preset sample text library into a number of words, and then construct a commonly used dictionary set.

[0052] The conversion module 220 is used to traverse each word in the text to be converted, and then convert the misspelled words in the text to be converted into correct words based on the similarity between each word and the words in the commonly used dictionary set; convert the text to be converted into a preset standard format, and convert the preset markers in the text to be converted into spaces.

[0053] The conversion module 220 includes a first conversion unit, which is used to traverse each word of the text to be converted and determine whether the current word exists in the commonly used dictionary set; when the current word does not exist in the commonly used dictionary set, convert the current word into pinyin; calculate the similarity between the current pinyin and the pinyin of each word in the commonly used dictionary set; when there is a pinyin of a word with a similarity greater than a preset similarity threshold, determine that the word corresponding to the pinyin of the word is the correct word of the current word; when there is no pinyin of a word with a similarity greater than the preset similarity threshold, determine that the current word is the correct word.

[0054] The conversion module 220 includes a second conversion unit, which is used to convert several consecutive blank characters in the text to be converted into one blank character; remove preset text characters in the text to be converted; wherein the preset text characters at least include: text leading and trailing spaces, line breaks, and formatting marks.

[0055] The segmentation module 230 is used to segment the sentences according to the punctuation marks and periods, and determine whether the length of the segmented sentences is greater than a preset length L; when there are sentences longer than L, segment the sentences according to the preset end symbol, and again determine whether the length of the segmented sentences is greater than the preset length L; when there are still sentences longer than L, segment the sentences according to the preset segmentation adverbs, and again determine whether the length of the segmented sentences is greater than the preset length L; when there are still sentences longer than L, segment the sentences according to the preset segmentation conjunctions, and again determine whether the length of the segmented sentences is greater than the preset length L; when there are still sentences longer than L, use the syntactic segmentation tool to segment the sentences longer than L into sentences less than or equal to L in length;

[0056] The segmentation module 230 includes a sentence segmentation unit,

[0057] It is used to match the paragraph boundaries in the text to be converted according to the regular expression "\n\s*\n|[\。\?\!]+", and then break the sentences at the paragraph boundaries; it breaks the sentences according to the ending symbols: ".", "!", "?", "", ";", ":".

[0058] The merging module 240 is used to determine the predicted emotion parameters of each sentence through the emotion recognition model; the TTS model generates sentence audio files according to the sentences and the predicted emotion parameters corresponding to the sentences, and then merges them into a complete audio file.

[0059] In addition, an embodiment of the present application further provides a non-volatile computer storage medium on which executable instructions are stored. When the executable instructions are executed, a method for improving the performance of a TTS model in processing long texts as described above is implemented.

[0060] So far, the technical solutions of the present disclosure have been described in combination with the above multiple embodiments, but it is easy for those skilled in the art to understand that the protection scope of the present disclosure is not limited to these specific embodiments. Without departing from the technical principles of the present disclosure, those skilled in the art can split and combine the technical solutions in the above-mentioned various embodiments, and can also make equivalent changes or replacements to the relevant technical features. Any changes, equivalent replacements, improvements, etc. made within the technical concept and / or technical principle of the present disclosure will fall within the protection scope of the present disclosure.

Claims

1. A method for improving the performance of a TTS model in processing long texts, characterized in that: The method comprises: Segment the sample text in the preset sample text library into several words, and then construct a commonly used dictionary set; Traversing each word of the text to be converted, and then converting the wrong words in the text to be converted into correct words based on the similarity between each word and the words in the commonly used dictionary set; converting the text to be converted into a preset standard format; Segment according to punctuation marks and periods, and determine whether the length of the segmented sentences is greater than the preset length L; when there are sentences longer than L, segment according to the preset end symbol, and again determine whether the length of the segmented sentences is greater than the preset length L; when there are still sentences longer than L, segment according to the preset segmentation adverbs, and again determine whether the length of the segmented sentences is greater than the preset length L; when there are still sentences longer than L, segment according to the preset segmentation conjunctions, and again determine whether the length of the segmented sentences is greater than the preset length L; when there are still sentences longer than L, use the syntactic segmentation tool to segment the sentences longer than L into sentences less than or equal to L in length; The predicted emotional parameters of each sentence are determined through the emotion recognition model; the TTS model generates sentence audio files based on the sentences and the predicted emotional parameters corresponding to the sentences, and then merges them into a complete audio file.

2. The method for improving the performance of a TTS model in processing long text according to claim 1, characterized in that: Traverse each word in the text to be converted, and then convert the wrong words in the text to be converted into correct words based on the similarity between each word and the words in the common dictionary set, including: Traverse each word of the text to be converted to determine whether the current word exists in the common dictionary set; When the current word does not exist in the commonly used dictionary set, the current word is converted into pinyin; Calculate the similarity between the current pinyin and the pinyin of each word in the commonly used dictionary set; When there is a pinyin of a word with a similarity greater than a preset similarity threshold, determining that the word corresponding to the pinyin of the word is the correct word of the current word; When there is no pinyin of a word with a similarity greater than a preset similarity threshold, the current word is determined to be a correct word.

3. The method for improving the performance of a TTS model in processing long text according to claim 1, characterized in that: Convert the text to be converted into a preset standard format, including: Convert several consecutive blank characters in the text to be converted into one blank character; Remove the preset text characters in the text to be converted; wherein the preset text characters at least include: leading and trailing spaces, line breaks, and formatting marks.

4. The method for improving the performance of a TTS model in processing long text according to claim 1, characterized in that: Sentence segmentation is performed according to the preset ending symbol, including: Match the paragraph boundaries in the text to be converted according to the regular expression "\n\s*\n|[\。\?\!]+", and then break the sentences at the paragraph boundaries; Break sentences according to the ending symbols: ".", "!", "?", "", ";", ":".

5. The method for improving the performance of a TTS model in processing long text according to claim 1, characterized in that: Before the TTS model generates a sentence audio file according to the sentence and the predicted emotion parameter corresponding to the sentence, the method includes: After the segmentation of the text to be converted is completed, the sentences are encoded and sorted according to the segmentation order; Merge into a complete audio file, including: Encode and merge the sentence audio files in the segmentation order to obtain a complete audio file.

6. A system for improving the performance of a TTS model in processing long texts, characterized in that: The system comprises: A construction module is used to segment the sample text in the preset sample text library into a number of words, and then construct a commonly used dictionary set; The conversion module is used to traverse each word of the text to be converted, and then convert the wrong words in the text to be converted into correct words based on the similarity between each word and the words in the commonly used dictionary set; convert the text to be converted into a preset standard format, and convert the preset markers in the text to be converted into spaces; A segmentation module is used to segment sentences according to punctuation marks and periods, and determine whether the length of the segmented sentences is greater than a preset length L; when there are sentences longer than L, segment the sentences according to a preset end symbol, and again determine whether the length of the segmented sentences is greater than the preset length L; when there are still sentences longer than L, segment the sentences according to preset segmentation adverbs, and again determine whether the length of the segmented sentences is greater than the preset length L; when there are still sentences longer than L, segment the sentences according to preset segmentation conjunctions, and again determine whether the length of the segmented sentences is greater than the preset length L; when there are still sentences longer than L, use a syntactic segmentation tool to segment sentences longer than L into sentences less than or equal to L in length; The merging module is used to determine the predicted emotion parameters of each sentence through the emotion recognition model; the TTS model generates sentence audio files based on the sentences and the predicted emotion parameters corresponding to the sentences, and then merges them into a complete audio file.

7. The system for improving the performance of TTS model in processing long text according to claim 6, characterized in that: The conversion module comprises a first conversion unit, Used to traverse each word of the text to be converted and determine whether the current word exists in the common dictionary set; When the current word does not exist in the commonly used dictionary set, the current word is converted into pinyin; Calculate the similarity between the current pinyin and the pinyin of each word in the commonly used dictionary set; When there is a pinyin of a word with a similarity greater than a preset similarity threshold, determining that the word corresponding to the pinyin of the word is the correct word of the current word; When there is no pinyin of a word with a similarity greater than a preset similarity threshold, the current word is determined to be a correct word.

8. The system for improving the performance of TTS model in processing long text according to claim 6, characterized in that: The conversion module includes a second conversion unit, Used to convert several consecutive blank characters in the text to be converted into one blank character; Remove the preset text characters in the text to be converted; wherein the preset text characters at least include: leading and trailing spaces, line breaks, and formatting marks.

9. The system for improving the performance of TTS model in processing long text according to claim 6, characterized in that: The segmentation module includes sentence segmentation units, Used to match paragraph boundaries in the text to be converted according to the regular expression "\n\s*\n|[\。\?\!]+", and then break sentences at paragraph boundaries; Break sentences according to the ending symbols: ".", "!", "?", "", ";", ":".

10. A non-volatile computer storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed, the method for improving the performance of a TTS model in processing long texts as described in any one of claims 1 to 5 is implemented.

Images