Speech synthesis system based on human-computer voice interaction and related device

Abstract

The application belongs to the technical field of speech synthesis, and provides a speech synthesis system based on human-computer voice interaction, a terminal and a computer readable storage medium, in order to solve the problem of poor speech synthesis effect based on human-computer voice interaction in the prior art, the speech recognition module included in the speech synthesis system is used to acquire the voice input by a first user, and the voice is subjected to speech recognition to obtain recognized text and a corresponding pronunciation sequence, and the pronunciation extraction module is used to acquire the pronunciation corresponding to the preset type target text according to the pronunciation sequence in the case that the recognized text contains the preset type target text, the text generation module is used to acquire the target recognized text corresponding to a second user, and the corresponding response text is generated according to the target recognized text, and the speech synthesis module is used to synthesize the response voice corresponding to the response text by using the pronunciation in the case that the response text contains the preset type target text, so that the effect of human-computer voice interaction can be improved.

Description

Technical Field

[0001] This application relates to the field of speech synthesis technology, and in particular to a speech synthesis system, terminal and computer-readable storage medium based on human-computer speech interaction. Background Technology

[0002] Voice is the most convenient and effective means for people to obtain information and communicate. Human-computer voice interaction refers to the process of information transmission between people and devices through natural speech. In recent years, with the popularization of smart hardware, the application of human-computer voice interaction has developed rapidly, and its technology has been widely used in medical, customer service, education, smart home, mobile devices, and automotive scenarios.

[0003] In human-computer voice interaction technology, the user's voice is first converted into text by speech recognition technology, and the information generated by the device needs to be transmitted to the user through speech synthesis and voice broadcast. Speech recognition is the starting point in the human-computer interaction scenario, and speech synthesis is the end point in the human-computer interaction scenario.

[0004] In human-computer voice interaction, the speech synthesis of texts related to names and places has always been one of the difficulties in speech synthesis. Such texts are usually sparse or even non-existent in the training corpus, but users may encounter such entity words during human-computer interaction.

[0005] In traditional technologies, speech synthesis in human-computer voice interaction typically processes this type of text independently. That is, it uses corresponding preset rules to determine the pronunciation of this type of text and then plays the pronunciation as speech to the user. However, since this type of text is usually sparse or even non-existent in the training corpus, there is a discrepancy between the pronunciation of this type of text and the user's actual pronunciation. This results in poor speech synthesis in human-computer voice interaction and reduces the effectiveness of human-computer voice interaction. Summary of the Invention

[0006] This application provides a speech synthesis system, terminal, and computer-readable storage medium based on human-computer speech interaction, which can solve the technical problem of poor speech synthesis effect based on human-computer speech interaction in traditional technology.

[0007] In a first aspect, this application provides a speech synthesis system based on human-computer voice interaction, comprising: a speech recognition module, configured to acquire speech input by a first user and perform speech recognition on the speech to obtain recognized text and a pronunciation sequence corresponding to the recognized text; a pronunciation extraction module, configured to, when the recognized text contains target text of a preset type, acquire the pronunciation corresponding to the target text of the preset type according to the pronunciation sequence; a text generation module, configured to acquire target recognized text corresponding to a second user and generate response text corresponding to the target recognized text according to the target recognized text, wherein the target recognized text describes the text obtained by recognizing the speech input by the second user; and a speech synthesis module, configured to, when the response text contains the target text of the preset type, synthesize response speech corresponding to the response text using the pronunciation.

[0008] Secondly, this application provides a terminal including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to run the speech synthesis system based on human-computer voice interaction.

[0009] Thirdly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to execute the speech synthesis system based on human-computer voice interaction.

[0010] This application provides a speech synthesis system, terminal, and computer-readable storage medium based on human-computer voice interaction. The system extracts preset-type target text and its corresponding pronunciation from the speech input by a first user in a human-computer voice interaction scenario. When speech synthesis involves the preset-type target text, it synthesizes speech based on the pronunciation corresponding to the preset-type target text. This ensures that the pronunciation of the preset-type target text in the first user's input speech is present throughout the entire human-computer voice interaction process, satisfying the speech synthesis needs of different users and making the pronunciation of the preset-type target text in the synthesized speech more realistic. This invention reflects the user's actual pronunciation of a preset type of target text. Compared to traditional technologies that treat speech recognition and speech synthesis as two separate steps, this embodiment combines speech recognition and speech synthesis in human-computer voice interaction scenarios. It incorporates the pronunciation of the preset type of target text involved in the user's input speech from speech recognition into speech synthesis, ensuring that the pronunciation of the preset type of target text in the synthesized response speech is consistent with the corresponding user's actual pronunciation. This solves the problem that the corresponding preset type of target text is usually sparse or even non-existent in the training corpus, making the synthesized speech more accurate, realistic, and vivid, and improving the effect of human-computer voice interaction. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A schematic diagram illustrating the composition of a speech synthesis system based on human-computer voice interaction provided in an embodiment of this application;

[0013] Figure 2 A schematic diagram illustrating the pronunciation generation process of the response text of the speech synthesis system based on human-computer voice interaction provided in this application embodiment;

[0014] Figure 3 Another schematic diagram of a speech synthesis system based on human-computer voice interaction provided in an embodiment of this application;

[0015] Figure 4 A schematic diagram illustrating an example operation of a speech synthesis system based on human-computer voice interaction, provided as an embodiment of this application;

[0016] Figure 5 A schematic block diagram of a terminal provided in an embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0019] This application provides a speech synthesis system based on human-computer voice interaction. The system can be applied to devices based on human-computer voice interaction, including but not limited to terminals such as intelligent robots, smartphones, and smartwatches, or servers such as cloud servers. It can be applied in scenarios including but not limited to medical care, customer service, education, smart homes, mobile devices, and vehicles, to provide users with voice interaction based on human-computer voice interaction.

[0020] To address the technical problem of poor speech synthesis performance based on human-computer voice interaction in traditional technologies, the inventors propose a speech synthesis system based on human-computer voice interaction according to the embodiments of this application. The core idea of ​​the embodiments of this application is as follows: In a human-computer voice interaction scenario, the speech input by a first user is acquired. When performing speech recognition on the speech, a data pair consisting of a preset type target text contained in the speech and its corresponding pronunciation is acquired. If the response text corresponding to the speech of a second user contains the preset type target text, the response speech corresponding to the response text is synthesized by pronunciation. Then, the response speech is played back to the user to perform human-computer voice interaction.

[0021] In this embodiment, the pronunciation of a preset type target text contained in the speech is obtained by using the speech input by a first user. When the response text corresponding to the speech of a second user contains the preset type target text, the speech corresponding to the response text is synthesized by using the pronunciation. Thus, the synthesized speech provided by the speech of the first user is dynamically optimized and fed back to the corresponding user. This enables the preset type target text in the synthesized speech to more realistically reflect the user's actual pronunciation of the preset type target text, thereby solving the problem that the corresponding preset type target text is usually sparse or even non-existent in the training corpus. This makes the synthesized speech more accurate, more realistic, and more vivid, and improves the effect of human-computer voice interaction.

[0022] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined to construct different implementation methods.

[0023] Please see Figure 1 and Figure 2 , Figure 1 This is a schematic diagram illustrating the composition of a speech synthesis system based on human-computer voice interaction provided in an embodiment of this application. Figure 2 This is a schematic diagram illustrating the pronunciation generation process of the response text in a speech synthesis system based on human-computer voice interaction, provided in an embodiment of this application. For example... Figure 1 As shown, the system includes, but is not limited to, the following modules 101 to 104:

[0024] The speech recognition module 101 is used to acquire the speech input by the first user and perform speech recognition on the speech to obtain the recognized text and the pronunciation sequence corresponding to the recognized text.

[0025] For example, in human-computer voice interaction scenarios, the user typically initiates the interaction via voice. In this case, the device interacting with the user via voice can acquire the user's voice input through a microphone or other voice input device, and perform speech recognition to obtain the recognized text and the corresponding pronunciation sequence. Speech recognition, also known as automatic speech recognition technology, or ASR for short, describes the technology of converting human speech into text.

[0026] In the process of speech recognition, the first step is to extract features from the speech input by the first user. Based on the extracted features, and combined with the acoustic model, pronunciation dictionary, and language model, the features extracted from the speech input by the first user are decoded and searched to obtain the recognized text. During this process, the pronunciation sequence corresponding to the recognized text is collected and stored, thus obtaining the recognized text and the pronunciation sequence corresponding to the recognized text. The pronunciation sequence describes the ordered set of pronunciations composed of the pronunciations of the recognized text. The pronunciation sequence includes, but is not limited to, the initial and final sequence containing initials and finals, and the phonetic symbol sequence containing vowels and consonants.

[0027] For example, when the user inputs Chinese speech, and the recognized text corresponding to the first user input speech is "call Xiao Le's phone number", the corresponding initial and final sequence can be "bo1da1xiao3yue4de3dian4hua4hao5ma3", thus obtaining a data pair consisting of "call Xiao Le's phone number" and "bo1da1xiao3yue4de3dian4hua4hao5ma3". When the user inputs English speech, the pronunciation sequence is a phonetic symbol sequence containing vowels and consonants.

[0028] The pronunciation extraction module 102 is used to obtain the pronunciation corresponding to the preset type target text according to the pronunciation sequence when the recognized text contains a preset type target text.

[0029] For example, when the identified text contains a preset type of target text, the pronunciation corresponding to the preset type of target text is obtained according to the pronunciation sequence. The preset type of target text describes the text that is determined as the target according to the business needs of speech synthesis. The preset type of target text can describe text that is relatively sparse or even does not appear in the training corpus. The preset type of target text includes, but is not limited to, entity words. Entity words are also known as named entity recognition. Named entity recognition, abbreviated as NER, is also known as proper name recognition. It refers to the recognition of entities with specific meanings in text, mainly including personal names, place names, proper nouns, organization names, etc.

[0030] To determine whether the identified text contains target text of a preset type, it can be based on content matching of rule templates. This matching is mainly done through regular expression matching to determine whether the identified text contains target text of a preset type. Alternatively, it can be based on the matching method of a preset text recognition model. This method involves training a preset text model using training samples and then using the trained preset text model to determine whether the identified text contains target text of a preset type.

[0031] For example, a content matching method based on rule templates is used to determine whether the identified text contains target text of a preset type. Thus, if the corresponding content is matched, it is determined that the identified text contains target text of a preset type. For instance, the rule template in the rule base contains the rule: "Dial *'s phone number". Furthermore, if the identified text corresponding to the first user's input voice is "Dial Xiao Le's phone number", then "Xiao Le" in the identified text can be identified as target text of a preset type. Based on the above example, the initial consonant of "Xiao Le" is "xiao3yue4", meaning the pronunciation of "Xiao Le" is "xiao3yue4".

[0032] The text generation module 103 is used to obtain the target recognition text corresponding to the second user, and generate the response text corresponding to the target recognition text based on the target recognition text, wherein the target recognition text describes the text obtained by recognizing the speech input by the second user.

[0033] For example, in a human-computer voice interaction scenario, the voice input by the second user is acquired and subjected to speech recognition to obtain the corresponding target recognition text, i.e., the target recognition text corresponding to the second user. Based on the target recognition text, the corresponding response text is generated. Generally, the action after the decision is generated as the response text to be replied to the user based on Natural Language Generation (NLG). Semantic understanding can be performed on the target recognition text, and then the response text corresponding to the target recognition text is generated based on the result of semantic understanding. Alternatively, the response text corresponding to the target recognition text can be generated based on content matching based on rule templates. The response text describes the text that responds to the voice input by the second user.

[0034] It should be noted that the terms "first" and "second" in "first user" and "second user" are merely used to distinguish different users and are not intended to limit the number of users.

[0035] The second user can be one of the following: 1) The second user is the same as the first user. In this case, the target recognition text and the recognition text mentioned above can refer to the same text; 2) The second user is different from the first user. In this case, the target recognition text and the recognition text mentioned above are not the same text, so that the pronunciation corresponding to the preset type target text is effective for all users.

[0036] For example, in the case of generating response text corresponding to target recognition text based on content matching of rule templates, rule templates can be understood as regular expression matching items. When a rule is matched according to the target recognition text, the corresponding action is executed.

[0037] For example, please refer to Table 1 below. As shown in Table 1, the rule base contains templates as shown in Table 1:

[0038] Table 1

[0039] Call * Calling you*

[0040] As shown in Table 1, the asterisk (*) in Table 1 indicates that any text is matched. According to the rule template, when the target text is "call *", the corresponding response text is "calling * for you". As shown in the example above, when the asterisk in Table 1 matches "Xiao Le", that is, when the target text is "call Xiao Le's phone number", the corresponding response text can be "calling Xiao Le for you".

[0041] The speech synthesis module 104 is used to synthesize the response speech corresponding to the response text by using the pronunciation when the response text contains the target text of the preset type.

[0042] For example, the speech synthesis module is used to synthesize response speech corresponding to the response text based on pronunciation when the response text contains target text of a preset type. That is, it uses pronunciation as the corresponding content in the response speech to synthesize the response speech corresponding to the response text. Generally, this is done using speech synthesis (also known as text-to-speech, TTS) to convert the response text into corresponding speech. The response speech describes the synthesized speech that responds to the speech input by the second user in a human-computer voice interaction application scenario. Furthermore, the response speech can also be played back to the second user to perform human-computer voice interaction.

[0043] Furthermore, speech synthesis can include a synthesis front-end and a synthesis back-end. The synthesis front-end and synthesis back-end describe different stages, units, or components of speech synthesis, respectively, and are not intended to limit the description of different devices. The synthesis front-end uses a pronunciation dictionary to convert the input text into pronunciation labels through processes such as text normalization and word segmentation. For example, Chinese text is converted into a sequence of initials and finals. The synthesis back-end generally includes two parts: an acoustic model and a vocoder. The acoustic model is responsible for converting the pronunciation labels into intermediate speech representations (commonly such as Mel spectrum, filterbank features, etc.), and the vocoder is responsible for converting the intermediate speech representations into speech waveform points, thereby obtaining the corresponding response speech.

[0044] Please see Figure 2 ,like Figure 2 As shown in this example, before incorporating the user's voice input, the possible pronunciation descriptions of the determined response text are as follows:

[0045] <Calling>[zheng4zai4]<for>[wei2]<you>[nin2]<calling>[hu2jiao4]<Xiao Le>[xiao3le4];

[0046] Please continue reading. Figure 2 Based on the user's voice input, the pronunciation description of the determined response text is as follows:

[0047] <Calling>[zheng4zai4]<for>[wei2]<you>[nin2]<calling>[hu2jiao4]<Xiao Le>[xiao3yue4];

[0048] It is evident that combining the pronunciation of the response text after the user's input speech can better reflect the user's actual pronunciation of the preset type of target text. Especially when the corresponding preset type of target text is usually sparse or even non-existent in the training corpus, it can make the synthesized speech more accurate, realistic, and vivid, thereby improving the effect of human-computer voice interaction.

[0049] This application embodiment extracts the preset type target text and its corresponding pronunciation contained in the speech input by the first user in a human-computer voice interaction scenario. When the preset type target text is involved in speech synthesis, speech synthesis is performed based on the pronunciation corresponding to the preset type target text. This allows the pronunciation of the preset type target text contained in the first user's speech to be integrated throughout the entire human-computer voice interaction process, satisfying the speech synthesis needs of different users. As a result, the pronunciation of the preset type target text in the synthesized speech more realistically reflects the user's actual pronunciation of the preset type target text. Compared to traditional technologies that treat speech recognition and speech synthesis as two independent steps, this application embodiment combines speech recognition and speech synthesis in a human-computer voice interaction scenario. The pronunciation of the preset type target text involved in the user's speech is brought from speech recognition into speech synthesis, ensuring that the pronunciation of the preset type target text in the synthesized response speech is consistent with the corresponding user's actual pronunciation. This solves the problem that the corresponding preset type target text is usually sparse or even non-existent in the training corpus, making the synthesized speech more accurate, realistic, and vivid, and improving the effect of human-computer voice interaction.

[0050] In one embodiment, the speech synthesis system further includes:

[0051] The pronunciation library module is used to store the pronunciations corresponding to the target text of the preset type.

[0052] For example, a pronunciation library module is set up in a speech synthesis system based on human-computer voice interaction. The pronunciation library module is connected to the pronunciation extraction module and the speech synthesis module. It is used to store the pronunciations corresponding to preset type target texts extracted from the speech input by the first user. It can continuously accumulate the pronunciations corresponding to different preset type target texts, thereby continuously accumulating the pronunciations corresponding to texts that are usually sparse or even non-existent in the training corpus. The pronunciation library can be an entity word pronunciation library, used to store the pronunciations corresponding to the extracted entity words.

[0053] Please see Figure 3 , Figure 3 This is another schematic diagram of a speech synthesis system based on human-computer voice interaction provided in an embodiment of this application. (See diagram below.) Figure 3As shown in this example, the pronunciation library module 105 in the speech synthesis system based on human-computer voice interaction is connected to the pronunciation extraction module 102 to store the pronunciation corresponding to the preset type target text extracted from the speech input by the first user. It is also connected to the speech synthesis module 104 through the pronunciation dictionary module 106, so that the pronunciation in the pronunciation library module 105 is used in the speech synthesis process corresponding to the speech synthesis module 104. This feeds back the user's actual pronunciation of the preset type target text to the speech synthesis, making the pronunciation in the synthesized speech more accurate and realistic, thereby improving the effect of human-computer voice interaction.

[0054] In this embodiment, a corresponding pronunciation library is set up to store the pronunciations corresponding to preset type target texts. Based on the pronunciations in the pronunciation library, speech synthesis is performed, which can further improve the accuracy and authenticity of the pronunciation of different preset type target texts in the synthesized speech, thereby making the synthesized speech more accurate, more realistic, and more vivid, and further improving the effect of human-computer voice interaction.

[0055] In one embodiment, the speech synthesis system further includes:

[0056] The pronunciation dictionary module is used to store mappings from individual words to phonemes;

[0057] The pronunciation filtering module is used to filter out pronunciations that meet preset pronunciation filtering conditions from the pronunciations contained in the pronunciation library module, and add the filtered pronunciations to the pronunciation dictionary module.

[0058] For example, the speech synthesis system further includes a pronunciation dictionary module and a pronunciation filtering module. The pronunciation filtering module is connected to the pronunciation dictionary module and the pronunciation library module, respectively. The pronunciation dictionary module (Lexicon) contains a mapping from individual words to phonemes, used to connect the acoustic model and the language model. For example, in Chinese, the pronunciation dictionary contains the initials and finals corresponding to Chinese words. The pronunciation filtering module is used to filter pronunciations that meet preset pronunciation filtering conditions from the pronunciations contained in the pronunciation library module, and add the filtered pronunciations to the pronunciation dictionary module. This adds the pronunciations of preset type target text extracted from the user's speech to the pronunciation dictionary module, enabling the pronunciations of preset type target text to become general pronunciation candidates for use in different user human-computer voice interaction scenarios, and thus taking effect for all users performing human-computer voice interaction. The preset pronunciation filtering conditions include, but are not limited to, the frequency of use of the pronunciation, the number of times the pronunciation appears, the usage scenario of the pronunciation, and the source of the pronunciation.

[0059] Please see Figure 3 and Figure 4 , Figure 4This is a schematic diagram illustrating an example operation of a speech synthesis system based on human-computer voice interaction, provided as an embodiment of this application. Figure 3 As shown, the speech synthesis system based on human-computer voice interaction includes a pronunciation library module 105 and a pronunciation dictionary module 106. A pronunciation filtering module is set between the pronunciation library module 105 and the pronunciation dictionary module 106. The pronunciation filtering module is connected to both the pronunciation library module 105 and the pronunciation dictionary module 106. The pronunciation filtering module filters out pronunciations that meet preset pronunciation filtering conditions from the pronunciations contained in the pronunciation library module 105 and adds the filtered pronunciations to the pronunciation dictionary module 106. Figure 4 As shown in the example, the user's input voice is processed by speech recognition. The pronunciation of the target text of the preset type is added to the pronunciation library module. Then, pronunciations that meet the preset pronunciation filtering conditions are selected from the pronunciation library module and added to the pronunciation dictionary module. When generating the pronunciation of the response text, the corresponding pronunciation is obtained from the pronunciation dictionary module. As shown in the example above, when the received user voice is "Dial Xiao Le's phone number (bo1da1xiao3yue4de3dian4hua4hao5ma3)," "Xiao Le (xiao3yue4)" is added to the pronunciation library module, and "Xiao Le (xiao3yue4)" is added from the pronunciation library module to the pronunciation dictionary module. When the response text is "Calling Xiao Le for you," "Xiao Le (xiao3yue4)" is obtained from the pronunciation dictionary module, and the corresponding pronunciation of the response text is generated as follows:

[0060] Zheng4zai4wei2nin2hu2jiao4[xiao3yue4];

[0061] Then, the pronunciation "Zheng4zai4wei2nin2hu2jiao4[xiao3yue4]" is synthesized to generate a response voice, which can be broadcast to the customer to achieve human-computer voice interaction.

[0062] In this embodiment, the pronunciation filtering module selects pronunciations that meet preset filtering conditions from the pronunciations contained in the pronunciation library module and adds the selected pronunciations to the pronunciation dictionary module. This enables the pronunciations of preset type target texts to become general pronunciation candidates, which can be used in different human-computer voice interaction scenarios. This makes the synthesized speech of human-computer voice interaction more accurate, more realistic, and more vivid for different users, thereby improving the effect of human-computer voice interaction for different users.

[0063] Furthermore, the pronunciation screening module includes:

[0064] The pronunciation filtering condition unit is used to count the frequency of occurrence of the target pronunciation contained in the pronunciation library module in the pronunciation library module, or to count the number of times the target pronunciation appears in the pronunciation library module;

[0065] The pronunciation adding unit is used to add the pronunciations selected according to the pronunciation filtering condition unit to the pronunciation dictionary module.

[0066] For example, the pronunciation filtering module includes a pronunciation filtering condition unit and a pronunciation adding unit. The pronunciation filtering condition unit is used to count the frequency of occurrence of the target pronunciation contained in the pronunciation library module, or to count the number of times the target pronunciation appears in the pronunciation library module. The pronunciation adding unit adds the pronunciations filtered by the pronunciation filtering condition unit to the pronunciation dictionary module. For example, if the frequency of occurrence is greater than or equal to a preset frequency threshold, the target pronunciation is added to the pronunciation dictionary module; or if the number of occurrences is greater than or equal to a preset number of occurrences threshold, the target pronunciation is added to the pronunciation dictionary module.

[0067] In this embodiment, the pronunciations in the pronunciation library module are filtered by a pronunciation filtering condition unit, and the pronunciations filtered by the pronunciation filtering condition unit are added to the pronunciation dictionary module. This not only enables the filtering of pronunciations added to the pronunciation dictionary module, avoiding redundancy and interference in the pronunciations in the pronunciation dictionary module, and ensuring the applicability and practicality of the pronunciations in the pronunciation dictionary module, but also allows for the dynamic modification of the pronunciation dictionary by statistically analyzing the corresponding preset type target text and its pronunciation contained in the user's input speech. This dynamically optimizes the pronunciation of the corresponding preset type target text involved in speech synthesis, so that the preset type target text in the synthesized speech more realistically reflects the user's actual pronunciation of the preset type target text. This solves the problem that the corresponding preset type target text is usually sparse or even non-existent in the training corpus, making the synthesized speech more accurate, realistic, and vivid, and improving the effect of human-computer speech interaction.

[0068] Furthermore, the pronunciation screening module includes:

[0069] The frequency statistics subunit is used to count the frequency of occurrence of the target pronunciation contained in the pronunciation library module when it appears in the pronunciation library module.

[0070] The frequency statistics subunit is used to count the number of times the target pronunciation appears in the pronunciation library module;

[0071] A pronunciation adding unit, configured to add the target pronunciation to the pronunciation dictionary module when the occurrence frequency is greater than or equal to a preset occurrence frequency threshold, and the occurrence count is greater than or equal to a preset occurrence count threshold.

[0072] Exemplarily, the pronunciation screening module includes a frequency statistics unit, a count statistics unit and a pronunciation adding unit, wherein the proportion statistics unit is configured to count the occurrence frequency corresponding to the occurrence of the target pronunciation contained in the pronunciation library module in the pronunciation library module, the count statistics unit is configured to count the occurrence count corresponding to the occurrence of the target pronunciation in the pronunciation library module, and the pronunciation adding unit is configured to add the target pronunciation to the pronunciation dictionary module when the occurrence frequency is greater than or equal to the preset occurrence frequency threshold and the occurrence count is greater than or equal to the preset occurrence count threshold. Thus, for the case where there are multiple pronunciations for a preset type of target text, for example, multiple pronunciations caused by different scenarios such as pronunciations of different users or pronunciations of polyphonic characters, a more appropriate pronunciation is screened out as a prediction, so as to ensure as much as possible that the pronunciation effect of the corresponding synthesized speech afterwards is relatively accurate, authentic and natural. For example, for "Xiaole" in the above example, where "Le" is a polyphonic character, in the case where "xiao3le4" and "xiao3yue4" exist, [xiao3yue4] is screened out as the pronunciation of "Xiaole" according to the voice input by the user.

[0073] Please refer to Table 2 below. As in the example shown in Table 2, the pronunciations extracted from the pronunciation extraction module are added to the pronunciation library module at least in the form of data pairs, that is, the pronunciations contained in the pronunciation library module are retained at least in the form of "word-pronunciation" data pairs. In the case where the pronunciation library module contains Word 1, Word 2 and Word 3, through statistics, the occurrence frequency of Pronunciation 1 corresponding to Word 1 is n11, the occurrence frequency of Pronunciation 2 corresponding to Word 1 is n12, the occurrence frequency of Pronunciation 3 corresponding to Word 1 is n13, the occurrence frequency of Pronunciation 1 corresponding to Word 2 is n21, and the occurrence frequency of Pronunciation 1 corresponding to Word 3 is n31. Further, it is possible to determine whether to add the corresponding entry to the pronunciation dictionary module according to the occurrence frequency of each word. For the case where Word 1 has multiple pronunciations, it is necessary to formulate corresponding screening rules according to the actual requirements of specific services, so as to ensure that a determined pronunciation can be screened out for subsequent speech synthesis.

[0074] Table 2

[0075]

[0076] In this embodiment, a frequency statistics unit is used to count the frequency of occurrence of the target pronunciation contained in the pronunciation library module, and a frequency statistics unit is used to count the number of times the target pronunciation appears in the pronunciation library module. A pronunciation addition unit adds the target pronunciation to the pronunciation dictionary module when the frequency of occurrence is greater than or equal to a preset frequency threshold and the number of occurrences is greater than or equal to a preset frequency threshold. This allows for stricter screening and filtering of the pronunciations added to the pronunciation dictionary module, further avoiding redundancy and interference, and ensuring greater applicability and practicality. Furthermore, by statistically analyzing the corresponding preset type target text and its pronunciation contained in the user's input speech, the pronunciation dictionary is dynamically modified, thereby dynamically optimizing the pronunciation of the corresponding preset type target text involved in speech synthesis. This allows the preset type target text in the synthesized speech to more realistically reflect the user's actual pronunciation of the preset type target text, thus solving the problem that the corresponding preset type target text is usually sparse or even non-existent in the training corpus. This makes the synthesized speech more accurate, realistic, and vivid, further improving the effect of human-computer voice interaction.

[0077] In one embodiment, the pronunciation extraction module 102 is further configured to acquire speech feature information corresponding to the pronunciation;

[0078] The speech synthesis module 104 is further configured to, when the response text contains the preset type target text, use the speech feature information and the pronunciation to synthesize the response speech corresponding to the response text.

[0079] For example, when the identified text contains target text of a preset type, based on the pronunciation sequence, not only can the pronunciation corresponding to the target text of the preset type be obtained, but also the speech feature information such as speech rate, volume, and pauses corresponding to the pronunciation can be obtained. The speech feature information is used as additional information for pronunciation and then used together to generate synthesized speech, which can make the generated synthesized speech richer and more realistic.

[0080] In this embodiment, the speech feature information corresponding to the pronunciation is obtained through the pronunciation extraction module, and the speech synthesis module, when the response text contains a preset type of target text, uses the speech feature information and pronunciation to synthesize the response speech corresponding to the response text. This makes the synthesized speech more accurate, more realistic, and more vivid, and further improves the effect of human-computer voice interaction.

[0081] It should be noted that the speech synthesis system based on human-computer voice interaction described in the above embodiments can be recombined with the technical features included in different embodiments as needed to obtain a combined implementation scheme, but all of them are within the protection scope claimed in this application.

[0082] Meanwhile, the division and connection methods of the various parts in the above-mentioned speech synthesis system based on human-computer voice interaction are only for illustrative purposes. In other embodiments, the speech synthesis system based on human-computer voice interaction can be divided into different parts as needed, and the various parts in the speech synthesis system based on human-computer voice interaction can be connected in different orders and ways to complete all or part of the functions of the speech synthesis system based on human-computer voice interaction.

[0083] The aforementioned speech synthesis system based on human-computer voice interaction can be implemented as a computer program, which can be used in, for example... Figure 5 It runs on the terminal shown.

[0084] Please see Figure 5 , Figure 5 This is a schematic block diagram of a terminal provided in an embodiment of this application. The terminal 500 can be a smartphone, tablet computer, laptop computer, desktop computer, or other terminal, or it can be a component or part in other devices.

[0085] See Figure 5 The terminal 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504, or it may be a volatile storage medium.

[0086] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, it causes the processor 502 to execute a speech synthesis system based on human-computer voice interaction as described above.

[0087] The processor 502 provides computing and control capabilities to support the operation of the entire terminal 500.

[0088] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a speech synthesis system based on human-computer voice interaction as described above.

[0089] This network interface 505 is used for network communication with other devices. Those skilled in the art will understand that... Figure 5The structure shown is merely a block diagram of a portion of the structure related to the solution of this application and does not constitute a limitation on the terminal 500 to which the solution of this application is applied. A specific terminal 500 may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. For example, in some embodiments, the terminal may only include a memory and a processor. In such embodiments, the structure and function of the memory and processor are different from those shown in the figures. Figure 5 The embodiments shown are consistent and will not be described again here.

[0090] The processor 502 is used to run a computer program 5032 stored in a memory to implement the speech synthesis system based on human-computer voice interaction as described above.

[0091] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0092] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program, which can be stored in a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0093] Therefore, this application also provides a computer-readable storage medium. This computer-readable storage medium can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, storing a computer program that, when executed by a processor, causes the processor to perform the following steps:

[0094] A computer program product, when run on a computer, causes the computer to execute the speech synthesis system based on human-computer speech interaction described in the above embodiments.

[0095] The computer-readable storage medium can be an internal storage unit of the aforementioned device, such as the device's hard drive or memory. The computer-readable storage medium can also be an external storage device of the device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the device.

[0096] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the devices, systems, and modules described above can be referred to the corresponding processes in the foregoing embodiments, and will not be repeated here.

[0097] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk, or any other physical storage medium capable of storing computer programs.

[0098] Those skilled in the art will recognize that the units and algorithms of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and processes of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0099] In the embodiments provided in this application, it should be understood that the disclosed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of each module is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0100] The system in this application embodiment can be rearranged, merged, and reduced in order according to actual needs. The modules in the system of this application embodiment can be merged, divided, and reduced according to actual needs. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0101] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0102] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A speech synthesis system based on human-computer voice interaction, characterized in that, include: The speech recognition module is used to acquire the speech input by the first user, and to perform speech recognition on the speech to obtain the recognized text and the pronunciation sequence corresponding to the recognized text; The pronunciation extraction module is used to obtain the pronunciation corresponding to the preset type target text based on the pronunciation sequence when the recognized text contains a preset type target text; The pronunciation filtering module includes a pronunciation filtering condition unit and a pronunciation adding unit. The pronunciation filtering condition unit is used to count the frequency of occurrence of a target pronunciation contained in the pronunciation library module in the pronunciation library module, or to count the number of times the target pronunciation appears in the pronunciation library module. The pronunciation adding unit is used to add the pronunciations filtered according to the pronunciation filtering condition unit to the pronunciation dictionary module. The text generation module is used to obtain the target recognition text corresponding to the second user, and generate the response text corresponding to the target recognition text based on the target recognition text, wherein the target recognition text describes the text obtained by recognizing the speech input by the second user; A speech synthesis module is used to synthesize the corresponding speech of the response text by using the pronunciation when the response text contains the target text of the preset type.

2. The speech synthesis system based on human-computer voice interaction according to claim 1, characterized in that, The speech synthesis system also includes: The pronunciation library module is used to store the pronunciations corresponding to the preset type of target text.

3. The speech synthesis system based on human-computer voice interaction according to claim 2, characterized in that, The speech synthesis system also includes: The pronunciation dictionary module is used to store the mapping between individual words and phonemes; The pronunciation filtering module is used to filter out pronunciations that meet preset pronunciation filtering conditions from the pronunciations contained in the pronunciation library module, and add the filtered pronunciations to the pronunciation dictionary module.

4. The speech synthesis system based on human-computer voice interaction according to claim 1, characterized in that, The pronunciation filtering module includes: The frequency statistics subunit is used to count the frequency of occurrence of the target pronunciation contained in the pronunciation library module in the pronunciation library module. The frequency statistics subunit is used to count the number of times the target pronunciation appears in the pronunciation library module; The pronunciation addition unit is used to add the target pronunciation to the pronunciation dictionary module when the occurrence frequency is greater than or equal to a preset occurrence frequency threshold and the occurrence count is greater than or equal to a preset occurrence count threshold.

5. The speech synthesis system based on human-computer speech interaction according to claim 1, characterized in that, The pronunciation extraction module is also used to obtain the speech feature information corresponding to the pronunciation; The speech synthesis module is further configured to, when the response text contains the preset type target text, use the speech feature information and the pronunciation to synthesize the response speech corresponding to the response text.

6. The speech synthesis system based on human-computer speech interaction according to claim 1, characterized in that, The pronunciation sequence includes at least one of the following: a sequence of initials and finals containing initials and finals, and a phonetic symbol sequence.

7. The speech synthesis system based on human-computer speech interaction according to any one of claims 1-6, characterized in that, The preset type of target text includes entity words.

8. A terminal, characterized in that, The terminal includes a memory and a processor connected to the memory; the memory is used to store a computer program; the processor is used to run the computer program to run the speech synthesis system based on human-computer voice interaction as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, can implement the speech synthesis system based on human-computer voice interaction as described in any one of claims 1-7.

Images