Voice generation method, virtual human voice generation method and voice generation system
By using the front-end network to recognize language and phoneme tone in live broadcast scenarios, combining features of style coding and back-end encoding, and adjusting pronunciation characteristics with the rhythm style adaptation network, the problem of colloquial text and trademark noun recognition in live broadcast speech synthesis is solved, and the problem of colloquial text and trademark noun recognition in live broadcast speech synthesis is realized. More natural and accurate speech generation.
Patent Information
- Application Number
- CN202411153853.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-12
AI Technical Summary
In live broadcast scenarios, it is difficult to accurately predict and synthesize interjections, paralinguistics and laughter in spoken texts, and the recognition of English trademark nouns is not accurate enough, resulting in the inadequate natural and accurate speech synthesis results.
By receiving text data and style information, obtaining target text information and style features, encoding and adjustment, generating high-dimensional voice features, and adjusting voice features based on style information to generate target voice, including using the front-end network to identify languages, phoneme tones and stresses Rhythm, style encoding network processes style information, back-end encoder merges text and style features, and rhythm style adapts to network to adjust speech features.
It improves the accuracy and nature of speech synthesis, and can better adapt to the characteristics of live broadcast scenes, including accent rhythms and interjection processing, ensuring the coordination between speech and human animation.
Smart Images

Figure CN118942443A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a speech generation method, a virtual human speech generation method and a speech generation system. Background Art
[0002] In current live broadcast scenarios, text-to-speech (TTS) technology is often used to convert text messages into speech so as to convey information to the audience in real time. This technology can convert text content into speech in a natural and smooth way, and has high customizability and adaptability, which can meet the real-time and interactive requirements of live broadcasts.
[0003] Compared with regular voice data, live streaming has significant characteristics. These characteristics involve the diversity of colloquial text, including a large number of interjections, paralanguage, and laughter, which makes it difficult to accurately predict and synthesize the speech. In addition, live streaming often involves various English trademark nouns, and it is difficult to accurately identify and synthesize these specific words, resulting in the speech synthesis results being inaccurate and unnatural.
[0004] Therefore, there is an urgent need for a speech generation method, a virtual human speech generation method and a speech generation system to solve the above problems. Summary of the invention
[0005] In view of the problems existing in the prior art, the present invention provides a speech generation method, a virtual human speech generation method and a speech generation system.
[0006] The present invention provides a speech generation method, comprising: receiving a first input, the first input comprising text data and style information; Acquire target text information corresponding to the text data; Performing a first encoding on the style information to obtain a target style feature; Performing a second encoding on the target text information and the target style feature to obtain a high-dimensional speech feature; The high-dimensional speech features are adjusted according to the style information to obtain target speech features corresponding to the style information, and a target speech is generated according to the target speech features.
[0007] According to a speech generation method provided by the present invention, the step of obtaining target text information corresponding to the text data includes: The text data is input into a front-end network to obtain the target text information output by the front-end network, wherein the front-end network includes any one or more of a first front-end branch sub-network, a second front-end branch sub-network, and a third front-end branch sub-network, wherein: The first front-end branch sub-network is trained based on sample text data marked with word language labels, and the first front-end branch sub-network is used to identify the language of the text data to obtain text word language information corresponding to the text data; The second front-end branch sub-network is trained based on sample text data marked with phoneme tone labels, and the second front-end branch sub-network is used to convert the text data into corresponding pronunciation annotation information, and generate corresponding word phoneme tone information according to the pronunciation annotation information; The third front-end branch sub-network is obtained by training a deep learning network based on sample text data marked with stress and prosody labels, and is used to predict stress and prosody information in the text data.
[0008] According to a speech generation method provided by the present invention, the second front-end branch sub-network is also used to judge the polyphones in the text data, and output the corresponding word phoneme tone information according to the polyphone information in the judgment result; The second front-end branch sub-network is also used to perform regularization processing on the text data before generating the word phoneme tone information to obtain the interjection information in the text data.
[0009] According to a speech generation method provided by the present invention, the style information includes speech style information and scene style information; The first encoding of the style information to obtain a target style feature includes: The speech style information and the scene style information are input into a style encoding network to obtain the target style features output by the style encoding network, wherein the style encoding network is trained based on sample style feature vectors marked with speech style labels and scene style labels.
[0010] According to a speech generation method provided by the present invention, the second encoding of the target text information and the target style combined features to obtain high-dimensional speech features includes: The target text information and the target style merged features are input into a back-end encoder network to obtain the high-dimensional speech features output by the back-end encoder network, wherein the back-end encoder network is trained based on sample high-dimensional speech features corresponding to sample text information and the sample style merged features.
[0011] According to a speech generation method provided by the present invention, the high-dimensional speech feature is adjusted according to the style information to obtain a target speech feature corresponding to the style information, including: performing style correction processing on the high-dimensional speech feature according to the target style feature corresponding to the style information and the stress prosody information to obtain the high-dimensional speech feature after the style correction processing; According to the word phoneme tone information, predict the duration information of each phoneme in the text data; According to the duration information of each phoneme, the high-dimensional speech feature after the style correction processing is up-sampled and adjusted to obtain the target speech feature.
[0012] According to a speech generation method provided by the present invention, the method of upsampling and adjusting the high-dimensional speech features after the style correction processing according to the word phoneme tone information to obtain the target speech features includes: Determine the key speech action of the target character animation corresponding to the style information, and obtain the rhythm duration corresponding to the key speech action; Acquire the voice playing time of the target character animation, and modify the time information of each phoneme in the text data according to the ratio between the rhythm time and the voice playing time to obtain modified time information; According to the modified duration information, up-sampling adjustment is performed on the high-dimensional speech features after the style correction processing to obtain the target speech features.
[0013] According to a speech generation method provided by the present invention, the high-dimensional speech feature is subjected to style correction processing according to the target style feature corresponding to the style information and the stress prosody information to obtain the high-dimensional speech feature after the style correction processing, including: The target style features, the stress prosody information and the high-dimensional speech features are input into a prosody-style adaptation network to obtain high-dimensional speech features after style correction output by the prosody-style adaptation network, wherein the prosody-style adaptation network is trained based on sample style features, sample stress prosody information and corresponding sample high-dimensional speech features.
[0014] According to a speech generation method provided by the present invention, the second front-end branch sub-network is trained by the following steps: Obtaining sample text data marked with phoneme tone tags; Annotating interjections in the sample text data marked with phoneme and tone labels with corresponding interjection labels, thereby constructing a phoneme and tone sample text data set; The neural network model is trained using the training samples in the phoneme tone sample text data set to obtain the second front-end branch sub-network.
[0015] According to a speech generation method provided by the present invention, after generating the target speech according to the target speech feature, the method further includes: receiving a second input, wherein the second input includes the text data and style change information, wherein the style change information includes voice style change information and scene style change information; In response to the second input, the text data and the style change information are input into the style encoding network to obtain a new target style feature, so as to change the style of the target speech through the new target style feature.
[0016] The present invention also provides a method for generating virtual human voice, comprising: receiving a first input, the first input comprising text data and style information; receiving a third input, the third input comprising action information of the virtual person, and association information between a duration corresponding to the action information and the text data; Based on the association information, a target voice is generated according to the text data, the style information and the action information, and the duration of the target voice is determined based on the duration of the action information.
[0017] The present invention also provides a speech generation system, comprising: A receiving module, configured to receive a first input, wherein the first input includes text data and style information; A processing module, used for obtaining target text information of the text data; A first encoding module, used for performing a first encoding on the style information to obtain a target style feature; A second encoding module, used for performing a second encoding on the target text information and the target style feature to obtain a high-dimensional speech feature; The speech generation module is used to adjust the high-dimensional speech features according to the style information, obtain target speech features corresponding to the style information, and generate target speech according to the target speech features.
[0018] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above methods when executing the program.
[0019] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and the computer program implements any of the above-mentioned methods when executed by a processor.
[0020] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above methods.
[0021] The speech generation method, virtual human speech generation method and speech generation system provided by the present invention encode the target style features obtained by encoding the style information with the target text information corresponding to the text data to obtain high-dimensional speech features, and then adjust the high-dimensional speech features according to the style information to obtain the target speech generated by the target speech features, so that the speech generated by the text content is more accurate and more consistent with the real speech. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0023] Figure 1 A flow chart of the speech generation method provided by the present invention; Figure 2 A schematic diagram of the overall flow of the speech generation method provided by the present invention; Figure 3 A schematic diagram of the flow of the virtual human voice generation method provided by the present invention; Figure 4 A schematic diagram of the structure of the speech generation system provided by the present invention; Figure 5 This is a schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION
[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0025] Live speech generation is a technology that inputs live text and outputs live speech in a specified style. It is widely used in broadcasting scenarios. However, when this technology is directly applied to e-commerce live broadcast scenarios, the following problems often occur: First, the accent of live streaming is very different from that of ordinary people (for example, audiobooks, speech videos, etc.), and it is difficult for existing broadcast voice technology to accurately predict it; Second, live broadcasts contain a lot of colloquial texts, including a large number of interjections, paralanguage, and laughter. Existing technologies make it difficult to synthesize such texts with intonations with high naturalness. 3. The live broadcast involves various English trademark terms, and the existing technology has the problem of generalization; Fourth, when the existing TTS technology is combined with human animation, the animation generation is modified according to the audio rhythm, and directly modifying the rate of the action animation may destroy the coordination of the action. Especially in live broadcast scenarios, human animation involves a large number of key actions with fixed rhythms and semantics, such as making a heart shape and enumerating. The rhythm of the voice needs to be adjusted according to the action to ensure the consistency and coordination between the voice and the animation. Therefore, the existing technology has certain limitations in meeting the needs of live broadcast.
[0026] Figure 1 A flow chart of the speech generation method provided by the present invention is shown in FIG. Figure 1 As shown, the present invention provides a speech generation method, comprising: Step 101: Receive a first input, where the first input includes text data and style information.
[0027] In the present invention, the text data is mainly the text content to be generated for speech, and the style information specifies the style of the target speech generated based on the text data, including language style, emotional tendency and sound characteristics, so as to ensure that the generated speech results meet specific requirements.
[0028] Step 102: Acquire target text information corresponding to the text data.
[0029] In the present invention, different languages have different pronunciation rules and speech features, and the language type used by the text data is determined, for example, English, Chinese, and French. At the same time, by analyzing the text, the phoneme and tone information therein can be identified for subsequent accurate speech synthesis or pronunciation, wherein the phoneme is the basic segment that constitutes the smallest speech unit in the language, and the tone is the characteristic of the pitch change in the language. It should be noted that in the present invention, the tone is mainly for the Chinese live broadcast scene, and for English or other languages, only the phoneme information in the text data can be extracted. As for the stress and rhythm in the text data, which are important rhythm and intonation features in language expression, the present invention provides recognition of stress and rhythm in text data, which helps to achieve more natural and fluent speech synthesis, and makes the synthesized speech closer to the expression of human natural language.
[0030] Step 103: Perform a first encoding on the style information to obtain a target style feature.
[0031] In the present invention, specific style types that need to be reflected in speech synthesis, such as formal, relaxed, solemn and humorous, as well as specific scenes, such as business meetings, family gatherings and travel commentaries, are obtained through the user end. These specified style types and scenes constitute style information, which will be used in the subsequent encoding process.
[0032] Furthermore, based on the style information specified by the user, it is converted into target style features through a certain encoding method. For example, the style information is converted into a numerical vector or other form of representation. These features will be used in subsequent speech synthesis or other related tasks to ensure that the generated speech has the specific style and emotional tendency expected by the user, thereby accurately reflecting the style characteristics expected by the user in the speech synthesis process.
[0033] Step 104: performing a second encoding on the target text information and the target style feature to obtain a high-dimensional speech feature.
[0034] In the present invention, the target text information is the target text information extracted from the text data in the above embodiment, including features such as language, phoneme tone, stress rhythm, etc. The target text information serves as the input of speech synthesis and describes the specific content to be converted into speech.
[0035] The target style feature is a feature obtained through encoding in the above embodiment that represents the user-specified speech style category and scene category. The target style feature describes the specific style and emotional tendency that the user expects to be reflected in speech synthesis.
[0036] Furthermore, the target text information and target style features are subjected to a second encoding, and this encoding process converts the text information and style features into high-dimensional vectors or other forms of representation, thereby obtaining high-dimensional speech features representing the target text information and target style features. These features contain rich representations of text content and style information, and can be used in subsequent speech synthesis or other related tasks to generate high-quality speech with the style and emotion expected by the user.
[0037] Step 105: adjusting the high-dimensional speech features according to the style information to obtain target speech features corresponding to the style information, and generating target speech according to the target speech features.
[0038] In the present invention, the high-dimensional speech features are adjusted according to the style information. This adjustment process involves changing the voice, pitch and rhythm of the speech features to match the target style information. After the adjustment, the target speech features corresponding to the style information are obtained. The target speech features contain the information of the speech style and emotional tendency specified by the user, which is the basis for generating the target speech. Finally, the target speech features can be converted into sound signals through the vocoder network to generate the target speech that meets the user's expectations, with high speech quality and accuracy.
[0039] The speech generation method provided by the present invention encodes the target style features obtained by encoding the style information and the target text information corresponding to the text data to obtain high-dimensional speech features, and then adjusts the high-dimensional speech features according to the style information to obtain the target speech generated by the target speech features, so that the speech generated by the text content is more accurate and more consistent with the real speech.
[0040] Based on the above embodiment, the step of obtaining target text information corresponding to the text data includes: The text data is input into a front-end network to obtain the target text information output by the front-end network, wherein the front-end network includes any one or more of a first front-end branch sub-network, a second front-end branch sub-network, and a third front-end branch sub-network, wherein: The first front-end branch sub-network is trained based on sample text data marked with word language labels, and the first front-end branch sub-network is used to identify the language of the text data to obtain text word language information corresponding to the text data; The second front-end branch sub-network is trained based on sample text data marked with phoneme tone labels, and the second front-end branch sub-network is used to convert the text data into corresponding pronunciation annotation information, and generate corresponding word phoneme tone information according to the pronunciation annotation information; The third front-end branch sub-network is obtained by training a deep learning network based on sample text data marked with stress and prosody labels, and is used to predict stress and prosody information in the text data.
[0041] In the present invention, in view of the significant difference between the stress rhythm of existing live broadcast scenarios and ordinary broadcasts, the present invention provides a separate network for live broadcast stress rhythm prediction; at the same time, special decoupling modeling is performed on the tone in the live broadcast to realize various interjections and laughter of the AI anchor; and, the language is decoupled, especially for some professional terms, such as various trademark names, so that the synthesized speech of various trademark names is robust enough.
[0042] Specifically, in the present invention, the front-end network is a neural network model used to process input text data and extract relevant information to obtain a representation of the target text information, wherein the target text information describes the characteristics of the text in terms of language, phoneme tone, stress rhythm, etc. In the present invention, the first front-end branch subnetwork, the second front-end branch subnetwork, and the third front-end branch subnetwork are several branch subnetworks that constitute the entire front-end network, and each subnetwork focuses on extracting a specific type of text information. For example, the first front-end branch subnetwork can be used to extract basic language features of speech, the second front-end branch subnetwork can be used to extract stress and rhythm information, and the third front-end branch subnetwork can be used to extract semantic information. When constructing the front-end network, one or more subnetworks can be selected to process text data according to actual scenario requirements to obtain the corresponding target text information.
[0043] Furthermore, the main function of the first front-end branch subnetwork is to identify the language of the text data and extract the language information corresponding to the text data. In the present invention, the first front-end branch subnetwork is obtained by training using sample text data with marked language information. During the training process, the network model receives data containing words and their corresponding language labels. By learning these sample data, the network model can learn to recognize texts in different languages. After the training is completed, the first front-end branch subnetwork receives the input text data and determines the language to which the text belongs. During the recognition process, it involves analyzing and comparing the characters, vocabulary or grammatical structures in the text to determine the language of the text. By identifying the text data, the first front-end branch subnetwork can output information indicating the language to which the text belongs. This information can be at the word level, that is, the network can determine the language to which each word belongs, or it can be the language information of the entire text.
[0044] Furthermore, the main function of the second front-end branch sub-network is to convert the input text data into corresponding pronunciation annotation information, and generate corresponding word phoneme tone information based on these annotation information. In the present invention, the network model is trained by using sample text data with marked phoneme tone information. During the training process, the network model receives data containing text and its corresponding phoneme tone labels. By learning these sample data, the network model can learn to convert the text into corresponding pronunciation annotation information. After the training is completed, the second front-end branch sub-network can receive the input text data and convert it into corresponding pronunciation annotation information, and based on the converted pronunciation annotation information, the second front-end branch sub-network can generate corresponding word phoneme tone information, which describes the pronunciation of each word in speech, including information on phonemes (speech units) and tones (tone changes), etc., to provide accurate pronunciation information for the subsequent speech synthesis process.
[0045] Further, the main function of the third front-end branch sub-network is to predict the stress and rhythm information in the input text data, that is, to determine which parts of the text should be stressed. In the present invention, the model network is trained by using sample text data with marked stress and rhythm information. During the training process, the network model receives data containing text and its corresponding stress and rhythm labels. By learning these sample data, the network model can understand which parts of the text need to be emphasized or highlighted. After the training is completed, the third front-end branch sub-network can receive the input text data and predict the stress and rhythm information of each part of the text, so as to analyze and compare the words, phrases or sentences in the text, determine which parts need to strengthen the stress or rhythm in the speech synthesis process, and help the speech synthesis process to express the stress and rhythm of the text more naturally and fluently.
[0046] In one embodiment, the accent prosody can be predicted by a deep learning network (i.e., the third front-end branch sub-network), and other information in the target text information can be obtained by the following methods in addition to the above-mentioned corresponding branch sub-networks: For the language ID, phoneme tone, interjection paralanguage and other labels in the text data, a preset database (the database stores the language ID information, the pinyin / phonetic symbols corresponding to the words, etc.) can be called first to convert the text data into the form of pinyin / phonetic symbols + language ID (if the text data is in Chinese, the words in the text data are converted into pinyin, and if the text data is in English, the corresponding words are converted into phonetic symbols). Then, based on the pinyin / phonetic symbols and the language ID, the text data is uniformly converted into the corresponding phoneme sequence. The phoneme is the smallest pronunciation unit in a language and is used to represent the basic component of speech. In the present invention, the pinyin / phonetic symbols can be mapped to the corresponding phonemes through a predefined phoneme table.
[0047] Furthermore, if it is necessary to determine polyphones in text data, a deep learning network can be used to determine polyphones before the phoneme conversion process. The deep learning network can recognize polyphones and annotate the corresponding polyphone information by model training. In one embodiment, if it is necessary to obtain interjection paralanguage tags in text data, a regular processing can be added before converting phonemes. For example, a regular expression that can match interjections can be constructed, and the constructed regular expression can be applied to the text data to identify the part of the text that meets the interjection pattern.
[0048] Based on the above embodiment, the second front-end branch sub-network is trained by the following steps: Obtaining sample text data marked with phoneme tone tags; Annotating interjections in the sample text data marked with phoneme and tone labels with corresponding interjection labels, thereby constructing a phoneme and tone sample text data set; The neural network model is trained using the training samples in the phoneme tone sample text data set to obtain the second front-end branch sub-network.
[0049] In the present invention, sample text data with phoneme and tone labels are first collected and marked, and each word in the sample text data is marked with the corresponding phoneme and its corresponding tone information. Further, interjections are extracted from the sample text data marked with phoneme and tone labels and the corresponding interjection labels are marked, so as to construct a sample text data set containing phoneme and tone labels. Then, the neural network model is trained using the training samples in the constructed phoneme and tone sample text data set, and after the training is completed, a second front-end branch sub-network is obtained to extract information related to phonemes, tones and interjections in the text data.
[0050] Based on the above embodiment, the second front-end branch sub-network is further used to judge the polyphones in the text data, and output the corresponding word phoneme tone information according to the polyphone information in the judgment result; The second front-end branch sub-network is also used to perform regularization processing on the text data before generating the word phoneme tone information to obtain the interjection information in the text data.
[0051] In the present invention, before generating word phoneme tone information, the text data can be regularized by the second front-end branch sub-network, and this regularization process uses regular expressions to identify interjection information in the text. Through regularization, interjection information can be extracted from the text data, and the interjection information includes the position, form and frequency of the interjection, etc., which is used for subsequent tone generation or other speech processing tasks. The present invention uses regularization to identify interjection information, and can more accurately consider the existence and characteristics of interjections when generating word phoneme tone information.
[0052] In one embodiment, a fourth front-end branch sub-network may be provided in the front-end network for regularizing the text data, so that the second front-end branch sub-network is only used for recognizing phoneme tones of the text data.
[0053] Based on the above embodiment, the style information includes voice style information and scene style information; The first encoding of the style information to obtain a target style feature includes: The speech style information and the scene style information are input into a style encoding network to obtain the target style features output by the style encoding network, wherein the style encoding network is trained based on sample style feature vectors marked with speech style labels and scene style labels.
[0054] In the present invention, the style information includes voice style information and scene style information, wherein the voice style information involves aspects such as formality, relaxation and humor, and the scene style information involves aspects such as business meetings and explanations.
[0055] In the present invention, the style encoding network is a neural network structure, which is used to convert the input speech style information and scene style information into target style features. The style encoding network is obtained through training, and the training data includes sample style feature vectors marked with speech style labels and scene style labels.
[0056] Furthermore, the style encoding network processes the input style information, thereby encoding the input information into a feature vector and outputting the corresponding target style features for use in adjusting the speech style in tasks such as speech synthesis and speech conversion and adapting to different scenario requirements.
[0057] Based on the above embodiment, the second encoding of the target text information and the target style combined features to obtain high-dimensional speech features includes: The target text information and the target style merged features are input into a back-end encoder network to obtain the high-dimensional speech features output by the back-end encoder network, wherein the back-end encoder network is trained based on sample high-dimensional speech features corresponding to sample text information and the sample style merged features.
[0058] In the present invention, the back-end encoder network is also a neural network structure, which is used to convert the target text information and the target style merged features into high-dimensional speech features. The network outputs the corresponding high-dimensional speech features by learning the feature representation of the sample text and the sample style. The high-dimensional speech features represent the high-dimensional representation of the target text information and the target style merged features in the speech field, and contain the speech features of the corresponding input information, such as the character's intonation and rhythm information, which are used for subsequent speech synthesis or other related tasks. Through the back-end encoder network, the present invention can efficiently and accurately convert the target text information and the target style merged features into high-dimensional speech features, thereby improving the efficiency of speech generation tasks.
[0059] On the basis of the above embodiment, the step of adjusting the high-dimensional speech feature according to the style information to obtain the target speech feature corresponding to the style information includes: performing style correction processing on the high-dimensional speech feature according to the target style feature corresponding to the style information and the stress prosody information to obtain the high-dimensional speech feature after the style correction processing; According to the word phoneme tone information, predict the duration information of each phoneme in the text data; According to the duration information of each phoneme, the high-dimensional speech feature after the style correction processing is up-sampled and adjusted to obtain the target speech feature.
[0060] In the present invention, according to the given style information and stress prosody information, the high-dimensional speech features are subjected to style correction processing to make them meet the requirements of the target style. At the same time, the word phoneme tone information is used to predict the duration of each phoneme in the text data. This process is to make appropriate duration adjustment for the generated speech in the subsequent steps to make it more natural and fluent.
[0061] In speech processing, the duration of each phoneme may be different. These duration information reflects the duration of the phoneme in the speech signal and is an important factor to be considered in the speech synthesis process. According to the duration information of each phoneme, the present invention performs upsampling adjustment on the high-dimensional speech features after style correction processing, so that the duration of each phoneme in the generated speech matches its duration in the original text, and obtains speech features that meet the duration requirements of the target speech, which are then used in the final speech synthesis process to generate a natural and fluent target speech.
[0062] On the basis of the above embodiment, the method of upsampling and adjusting the high-dimensional speech features after the style correction processing according to the word phoneme tone information to obtain the target speech features includes: Determine the key speech action of the target character animation corresponding to the style information, and obtain the rhythm duration corresponding to the key speech action; Acquire the voice playing time of the target character animation, and modify the time information of each phoneme in the text data according to the ratio between the rhythm time and the voice playing time to obtain modified time information; According to the modified duration information, up-sampling adjustment is performed on the high-dimensional speech features after the style correction processing to obtain the target speech features.
[0063] In the present invention, the key speech actions of the corresponding target character animation are determined according to the style information of the audio, that is, which actions should be synchronized with the speech content in the audio to ensure that the animation and the audio are expressed in the same form. After the key speech actions are determined, the rhythm durations corresponding to these actions are obtained, thereby obtaining the duration of each action in the animation, so as to correctly express the speech content in the audio in the animation.
[0064] Further, the speech playing duration of the audio-related part in the target character animation is determined. In the present invention, the speech playing duration can be the length of the entire animation clip, or the duration of a specific part associated with the audio. Then, according to the proportional relationship between each rhythm duration and the speech playing duration of the target character animation, the duration information of each phoneme in the text data is modified, that is, the duration of each phoneme is adjusted using the proportional relationship so that it is consistent with the playing duration of the target character animation. After modification, the duration information of each phoneme in the text data will match the speech playing duration of the target character animation, and finally, according to the modified duration information of each phoneme, the high-dimensional speech feature after the style correction process is up-sampled and adjusted, and a speech feature that is more in line with the target speech duration requirement is obtained.
[0065] On the basis of the above embodiment, the method of performing style correction processing on the high-dimensional speech feature according to the target style feature corresponding to the style information and the stress prosody information to obtain the high-dimensional speech feature after the style correction processing includes: The target style features, the stress prosody information and the high-dimensional speech features are input into a prosody-style adaptation network to obtain high-dimensional speech features after style correction output by the prosody-style adaptation network, wherein the prosody-style adaptation network is trained based on sample style features, sample stress prosody information and corresponding sample high-dimensional speech features.
[0066] In the present invention, the prosody style adaptation network is a neural network model, which is trained based on sample style features, sample stress prosody information and corresponding sample high-dimensional speech features. The neural network model can make corresponding style corrections to the input high-dimensional speech features by learning the style and prosody information in the sample data, thereby improving the quality and fidelity of the speech part in speech synthesis or animation.
[0067] On the basis of the above embodiment, after generating the target speech according to the target speech feature, the method further includes: receiving a second input, wherein the second input includes the text data and style change information, wherein the style change information includes voice style change information and scene style change information; In response to the second input, the text data and the style change information are input into the style encoding network to obtain a new target style feature, so as to change the style of the target speech through the new target style feature.
[0068] In the present invention, when text data and style change information are input into the style encoding network, new target style features are obtained, which describe the new style that the target speech should have, including changes in pitch, speech speed, intonation, etc. Further, these new target style features are used to change the style of the target speech, thereby changing the style of the target speech.
[0069] On the basis of the above embodiment, before acquiring the target text information corresponding to the text data, the method further includes: The text data is preprocessed to obtain preprocessed text data, wherein the preprocessing includes text cleaning processing, text standardization processing and word segmentation processing.
[0070] In the present invention, noise, useless information or unnecessary characters in the text are removed by text cleaning, thereby improving the efficiency of subsequent processing steps. By text standardization, text data is converted into a unified format or a standardized form, for example, to handle spelling errors, and standardized text helps to improve the consistency and accuracy of text processing. By word segmentation, text data is divided into words or phrases, providing a basis for subsequent language understanding and processing. The present invention preprocesses text data to improve the quality and consistency of text data, and provides a better basis for subsequent processing and analysis.
[0071] Figure 2 The overall flow chart of the speech generation method provided by the present invention can be referred to Figure 2 As shown, the process of intelligently generating live voice based on text provided by the present invention is described, and the specific process is as follows: First, input the text into the front-end network to obtain the language ID, phoneme tone, common rhythm and stress of the whole sentence in the text, and interjection paralanguage tag of each word in the text (according to the actual voice scene, the corresponding front-end network structure is selected to achieve it). In this process, the results obtained are independent of the voice style ID. The main purpose is to process the text and enhance generalization. For texts of various languages in different scenarios, the rhythmic rhythm, pronunciation of digital symbols, pronunciation of polyphonic words and English words in the text can be judged. For example, the "hahaha" in the text is converted into a laughter tag instead of two separate "ha1ha1" phonemes.
[0072] Furthermore, the style IDs (including the voice style ID and scene style ID specified by the user) are uniformly encoded into style features, and then the obtained text information (such as language, phoneme tone and stress rhythm) and style features are merged and input into the back-end encoder network to obtain high-dimensional speech features. In the present invention, the style ID is used to control the text information to synthesize high-dimensional speech features, including the speaker's intonation and rhythm information.
[0073] Furthermore, high-dimensional speech features, universal prosody-stress features (i.e., stress and prosody) and style features are input into the prosody-style adaptation network to make certain stylized corrections to the speech features. At the same time, a duration prediction network is used to output the duration of each phoneme. Specifically, the duration prediction network is a model based on a neural network. The training data includes phonemes and their corresponding durations. During the training phase, the network learns the relationship between phoneme features and durations. The present invention modifies the high-dimensional speech features of universal prosody-stress coding based on the style features, and uses this to predict the duration of each phoneme with personal style characteristics and scene characteristics.
[0074] Finally, the rhythm duration of the key actions of the animated character is obtained, and the predicted phoneme duration (that is, the phoneme duration information obtained by the duration prediction network) is modified according to the ratio between the rhythm duration and the overall voice playback duration of the animated character. The high-dimensional speech features are then upsampled using the modified phoneme duration, and the upsampled features are synthesized into the output speech through the back-end vocoder network, thereby adapting to the action rhythm and synthesizing the final speech.
[0075] The front-end network in the present invention is trained with a large amount of text, and can directly predict the stress and rhythm of the voice through the text. In view of the different live broadcast habits of each person, the hierarchical rhythm control idea is adopted, the front end is used to generate the initial rhythm solution, and the back end is corrected according to the live broadcast style ID, which can adapt to the voice rhythm of each speaker and restore the live broadcast voice more vividly. In addition, a special decoupling modeling is made for the tone, so that the AI anchor can synthesize various interjections and laughter, such as the labels of various interjections and paralanguages in the live broadcast, including special laughter "roar", "hehe", "hehe" and sighs, smacking lips, hissing, etc. At the same time, the present invention decouples the language through the language ID and the mixed Chinese and English phonemes, which can make the Chinese voice fluently speak various English trademark names and English short sentences, so that the final generated voice can adapt to the rhythm of the key human body movements given in the live broadcast, on the one hand, avoiding the cost of modifying the animation, and on the other hand, making the action and sound rhythm consistent in the terminal effect.
[0076] Figure 3 A flow chart of the virtual human voice generation method provided by the present invention is shown in FIG. Figure 3 As shown, the present invention provides a method for generating a virtual human voice based on the voice generation method described in the above embodiments, comprising: Step 301: Receive a first input, where the first input includes text data and style information.
[0077] In the present invention, the text data is mainly the text content to be generated for speech, and the style information specifies the style of the target speech generated based on the text data, including language style, emotional tendency and sound characteristics, so as to ensure that the generated speech results meet specific requirements.
[0078] Step 302: Receive a third input, wherein the third input includes action information of a virtual person, and association information between a duration corresponding to the action information and the text data.
[0079] In the present invention, virtual people refer to characters in computer graphics or virtual reality, which can act according to specific programs or instructions. The action information of virtual people usually refers to the actions performed by these characters, such as walking, waving, talking, etc. In the present invention, each action is preset with a duration in advance as action information. For example, a virtual person's waving action may last for a few seconds, while a talking action may last for tens of seconds or longer.
[0080] In the present invention, the association information between text data and action information mainly refers to the correspondence between a word or sentence in the text data and an action performed by a virtual person. For example, when a virtual person speaks a text, the corresponding action information includes mouth movements, body postures, etc. This association information can ensure that the actions of the virtual person are consistent with his or her verbal expression or behavior.
[0081] Step 303: Generate a target voice based on the associated information, the text data, the style information and the action information, wherein the duration of the target voice is determined based on the duration of the action information.
[0082] In the present invention, the associated information is used to ensure that the content and style of the generated voice match the text data and style information, and the duration of the voice corresponds to the duration of the action information. Then, the target voice is generated based on the input text data and style information, combined with the action information. In the generation process, the duration of the action information is taken into account to ensure that the duration of the voice matches the duration of the virtual person performing the action. For example, the speech speed of an audio is very fast, but the actual motion capture actor speaks much slower. In order to make the final action and voice rhythm match, the generated voice rhythm will be adapted according to the action duration. Preferably, the duration of the target voice is equal to the duration of the action information. For example, if the action information specifies that an action lasts for 5 seconds, then the duration of the generated target voice will also be 5 seconds. The voice of the corresponding length is generated according to the duration of the action information, thereby achieving a more accurate and natural speech synthesis effect.
[0083] The virtual human speech generation method provided by the present invention encodes the target style features obtained by encoding the style information and the target text information corresponding to the text data to obtain high-dimensional speech features, and then adjusts the high-dimensional speech features according to the style information to obtain the target speech generated by the target speech features, so that the speech generated by the text content is more accurate and more consistent with the real speech.
[0084] On the basis of the above embodiment, the generating of the target speech based on the association information, according to the text data, the style information and the action information includes: The duration of each phoneme in the target speech is adjusted based on the duration of the action information.
[0085] In the present invention, the duration of each phoneme in the target speech is adjusted according to the duration specified in the action information to ensure that the overall speech duration meets the requirements, that is, for different phonemes, their duration needs to be dynamically adjusted according to the action information, so that the generated speech is more natural and in line with the actual situation.
[0086] On the basis of the above embodiment, the generating of the target speech based on the association information and according to the text data, the style information and the action information further includes: Extracting target text information of the text data, wherein the target text information includes at least one of text word language information, word phoneme tone information, and stress rhythm information; Performing a first encoding on the style information to obtain a target style feature; Performing a second encoding on the target text information and the target style feature to obtain a high-dimensional speech feature; The high-dimensional speech feature is adjusted according to the target style feature and the duration of the action information to obtain a target speech feature corresponding to the style information, and a target speech is generated according to the target speech feature.
[0087] In the present invention, different languages have different pronunciation rules and speech features, and the language type used by the text data is determined, for example, English, Chinese, and French. At the same time, by analyzing the text, the phoneme and tone information therein can be identified for subsequent accurate speech synthesis or pronunciation, wherein the phoneme is the basic segment that constitutes the smallest speech unit in the language, and the tone is the characteristic of the pitch change in the language. It should be noted that in the present invention, the tone is mainly for the Chinese live broadcast scene, and for English or other languages, only the phoneme information in the text data can be extracted. As for the stress and rhythm in the text data, which are important rhythm and intonation features in language expression, the present invention provides recognition of stress and rhythm in text data, which helps to achieve more natural and fluent speech synthesis, and makes the synthesized speech closer to the expression of human natural language.
[0088] Furthermore, the specific style types that need to be reflected in speech synthesis, such as formal, relaxed, solemn and humorous, and specific scenes such as business meetings, family gatherings and travel commentaries, are obtained through the user end. These specified style types and scenes constitute style information, which will be used in the subsequent encoding process. In the present invention, according to the style information specified by the user end, it is converted into target style features through a certain encoding method, for example, the style information is converted into a numerical vector or other forms of representation. These features will be used in subsequent speech synthesis or other related tasks to ensure that the generated speech has the specific style and emotional tendency expected by the user, so as to accurately reflect the style features expected by the user in the speech synthesis process.
[0089] In the present invention, the target text information is the target text information extracted from the text data in the above embodiment, including one or more features of language, phoneme tone, stress rhythm, etc. The target text information is used as the input of speech synthesis, and describes the specific content to be converted into speech. The target style feature is the feature obtained by encoding in the above embodiment to represent the user-specified speech style category and scene category, and the target style feature describes the specific style and emotional tendency that the user expects to be reflected in speech synthesis.
[0090] Furthermore, the target text information and target style features are subjected to a second encoding, and this encoding process converts the text information and style features into high-dimensional vectors or other forms of representation, thereby obtaining high-dimensional speech features representing the target text information and target style features. These features contain rich representations of text content and style information, and can be used in subsequent speech synthesis or other related tasks to generate high-quality speech with the style and emotion expected by the user.
[0091] Finally, the high-dimensional speech features are adjusted according to the style information and action information. This adjustment process involves changing the voice, pitch, and rhythm of the speech features to match the target style information. After adjustment, the target speech features corresponding to the style information are obtained. The target speech features contain the information of the speech style and emotional tendency specified by the user. At the same time, the speech features are adapted to the corresponding animation actions and are the basis for generating the target speech. Finally, the target speech features can be converted into sound signals through the vocoder network to generate the target speech that meets the user's expectations, with high speech quality and accuracy.
[0092] The speech generation system provided by the present invention is described below. The speech generation system described below and the speech generation method described above can be referenced to each other.
[0093] Figure 4 A schematic diagram of the structure of the speech generation system provided by the present invention, such as Figure 4As shown, the present invention provides a speech generation system, including a receiving module 401, a processing module 402, a first encoding module 403, a second encoding module 404 and a speech generation module 405, wherein the receiving module 401 is used to receive a first input, the first input including text data and style information; the processing module 402 is used to obtain target text information of the text data; the first encoding module 403 is used to perform a first encoding on the style information to obtain a target style feature; the second encoding module 404 is used to perform a second encoding on the target text information and the target style feature to obtain a high-dimensional speech feature; the speech generation module 405 is used to adjust the high-dimensional speech feature according to the style information, obtain a target speech feature corresponding to the style information, and generate a target speech according to the target speech feature.
[0094] The speech generation system provided by the present invention encodes the target style features obtained by encoding the style information and the target text information corresponding to the text data to obtain high-dimensional speech features, and then adjusts the high-dimensional speech features according to the style information to obtain the target speech generated by the target speech features, so that the speech generated by the text content is more accurate and more consistent with the real speech.
[0095] The system provided by the present invention is used to execute the above-mentioned method embodiments. Please refer to the above-mentioned embodiments for the specific processes and detailed contents, which will not be repeated here.
[0096] Figure 5 A schematic diagram of the structure of an electronic device provided by the present invention, such as Figure 5 As shown, the electronic device may include: a processor (Processor) 501, a communication interface (Communications Interface) 502, a memory (Memory) 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504. The processor 501 may call the logic instructions in the memory 503 to execute the method provided by the above embodiments, the method comprising: receiving a first input, the first input comprising text data and style information; obtaining target text information corresponding to the text data; performing a first encoding on the style information to obtain a target style feature; performing a second encoding on the target text information and the target style feature to obtain a high-dimensional speech feature; adjusting the high-dimensional speech feature according to the style information to obtain a target speech feature corresponding to the style information, and generating a target speech according to the target speech feature; Or, receive a first input, the first input including text data and style information; receive a third input, the third input including action information of a virtual person, and association information between a duration corresponding to the action information and the text data; based on the association information, generate a target speech according to the text data, the style information and the action information, the duration of the target speech being determined based on the duration of the action information.
[0097] In addition, the logic instructions in the above-mentioned memory 503 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0098] On the other hand, the present invention also provides a computer program product, the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, when the program instructions are executed by a computer, the computer can execute the method provided by the above embodiments, the method comprising: receiving a first input, the first input including text data and style information; obtaining target text information corresponding to the text data; performing a first encoding on the style information to obtain a target style feature; performing a second encoding on the target text information and the target style feature to obtain a high-dimensional speech feature; adjusting the high-dimensional speech feature according to the style information to obtain a target speech feature corresponding to the style information, and generating a target speech according to the target speech feature; Or, receive a first input, the first input including text data and style information; receive a third input, the third input including action information of a virtual person, and association information between a duration corresponding to the action information and the text data; based on the association information, generate a target speech according to the text data, the style information and the action information, the duration of the target speech being determined based on the duration of the action information.
[0099] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented by a processor to execute the method provided in each of the above embodiments, the method comprising: receiving a first input, the first input comprising text data and style information; obtaining target text information corresponding to the text data; performing a first encoding on the style information to obtain a target style feature; performing a second encoding on the target text information and the target style feature to obtain a high-dimensional speech feature; adjusting the high-dimensional speech feature according to the style information to obtain a target speech feature corresponding to the style information, and generating a target speech according to the target speech feature; Or, receive a first input, the first input including text data and style information; receive a third input, the third input including action information of a virtual person, and association information between a duration corresponding to the action information and the text data; based on the association information, generate a target speech according to the text data, the style information and the action information, the duration of the target speech being determined based on the duration of the action information.
[0100] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0101] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A speech generation method, characterized in that: include: receiving a first input, the first input comprising text data and style information; Acquire target text information corresponding to the text data; Performing a first encoding on the style information to obtain a target style feature; Performing a second encoding on the target text information and the target style feature to obtain a high-dimensional speech feature; The high-dimensional speech features are adjusted according to the style information to obtain target speech features corresponding to the style information, and a target speech is generated according to the target speech features.
2. The speech generation method according to claim 1, characterized in that: The obtaining of target text information corresponding to the text data includes: The text data is input into a front-end network to obtain the target text information output by the front-end network, wherein the front-end network includes any one or more of a first front-end branch sub-network, a second front-end branch sub-network, and a third front-end branch sub-network, wherein: The first front-end branch sub-network is trained based on sample text data marked with word language labels, and the first front-end branch sub-network is used to identify the language of the text data to obtain text word language information corresponding to the text data; The second front-end branch sub-network is trained based on sample text data marked with phoneme tone labels, and the second front-end branch sub-network is used to convert the text data into corresponding pronunciation annotation information, and generate corresponding word phoneme tone information according to the pronunciation annotation information; The third front-end branch sub-network is obtained by training a deep learning network based on sample text data marked with stress and prosody labels, and is used to predict stress and prosody information in the text data.
3. The speech generation method according to claim 2, characterized in that: The second front-end branch sub-network is also used to judge the polyphones in the text data, and output the corresponding word phoneme tone information according to the polyphone information in the judgment result; The second front-end branch sub-network is also used to perform regularization processing on the text data before generating the word phoneme tone information to obtain the interjection information in the text data.
4. The speech generation method according to claim 1, characterized in that: The style information includes voice style information and scene style information; The first encoding of the style information to obtain a target style feature includes: The speech style information and the scene style information are input into a style encoding network to obtain the target style features output by the style encoding network, wherein the style encoding network is trained based on sample style feature vectors marked with speech style labels and scene style labels.
5. The speech generation method according to claim 4, characterized in that: The second encoding of the target text information and the target style feature to obtain a high-dimensional speech feature includes: The target text information and the target style features are input into a back-end encoder network to obtain the high-dimensional speech features output by the back-end encoder network, wherein the back-end encoder network is trained based on sample high-dimensional speech features corresponding to sample text information and the sample style combined features.
6. The speech generation method according to claim 2, characterized in that: The step of adjusting the high-dimensional speech feature according to the style information to obtain a target speech feature corresponding to the style information includes: performing style correction processing on the high-dimensional speech feature according to the target style feature corresponding to the style information and the stress prosody information to obtain the high-dimensional speech feature after the style correction processing; According to the word phoneme tone information, predict the duration information of each phoneme in the text data; According to the duration information of each phoneme, the high-dimensional speech feature after the style correction processing is up-sampled and adjusted to obtain the target speech feature.
7. The speech generation method according to claim 6, characterized in that: The step of upsampling and adjusting the high-dimensional speech features after the style correction process according to the duration information of each phoneme to obtain the target speech features includes: Determine the key speech action of the target character animation corresponding to the style information, and obtain the rhythm duration corresponding to the key speech action; Acquire the voice playing time of the target character animation, and modify the time information of each phoneme in the text data according to the ratio between the rhythm time and the voice playing time to obtain modified time information; According to the modified duration information, up-sampling adjustment is performed on the high-dimensional speech features after the style correction processing to obtain the target speech features.
8. The speech generation method according to claim 6, characterized in that: The step of performing style correction processing on the high-dimensional speech feature according to the target style feature corresponding to the style information and the stress prosody information to obtain the high-dimensional speech feature after the style correction processing includes: The target style features, the stress prosody information and the high-dimensional speech features are input into a prosody-style adaptation network to obtain high-dimensional speech features after style correction output by the prosody-style adaptation network, wherein the prosody-style adaptation network is trained based on sample style features, sample stress prosody information and corresponding sample high-dimensional speech features.
9. The speech generation method according to claim 2, characterized in that: The second front-end branch sub-network is trained by the following steps: Obtaining sample text data marked with phoneme tone tags; Annotating interjections in the sample text data marked with phoneme and tone labels with corresponding interjection labels, thereby constructing a phoneme and tone sample text data set; The neural network model is trained using the training samples in the phoneme tone sample text data set to obtain the second front-end branch sub-network.
10. The speech generation method according to claim 4, characterized in that: After generating the target speech according to the target speech feature, the method further includes: receiving a second input, wherein the second input includes the text data and style change information, wherein the style change information includes voice style change information and scene style change information; In response to the second input, the text data and the style change information are input into the style encoding network to obtain a new target style feature, so as to change the style of the target speech through the new target style feature.
11. A method for generating virtual human voice, characterized in that: include: receiving a first input, the first input comprising text data and style information; receiving a third input, the third input comprising action information of the virtual person, and association information between a duration corresponding to the action information and the text data; Based on the association information, a target voice is generated according to the text data, the style information and the action information, and the duration of the target voice is determined based on the duration of the action information.
12. A speech generation system, characterized in that: include: A receiving module, configured to receive a first input, wherein the first input includes text data and style information; A processing module, used for obtaining target text information of the text data; A first encoding module, used for performing a first encoding on the style information to obtain a target style feature; A second encoding module, used for performing a second encoding on the target text information and the target style feature to obtain a high-dimensional speech feature; The speech generation module is used to adjust the high-dimensional speech features according to the style information, obtain target speech features corresponding to the style information, and generate target speech according to the target speech features.
13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 11 is implemented.
14. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.
15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.
Citation Information
Cited By
Live broadcast voice generation method and system
CN119052515A
Personalized speech synthesis method, device and equipment based on scene perception and natural language description
CN120148475A
Method, device and equipment for personalized speech synthesis based on scene perception and natural language description
CN120148475B
