Voice generation method and device, storage medium and program product

Abstract

The embodiment of the invention provides a voice generation method and device, a storage medium and a program product. The method comprises the following steps: acquiring a text, inputting a model cue word and the text into a label generation model, and generating voice parameter labels of a plurality of text segments under the guidance of the model cue word; and inputting the plurality of text segments and the respective voice parameter tags into a voice generation model, generating a plurality of target voice segments based on the plurality of text segments and the respective voice parameter tags, and splicing to obtain a target voice corresponding to the text. Through the mode, the voice parameter label of the text segment can be accurately generated based on the label generation model and the model prompt word, and voice generation can be accurately performed based on the voice generation model and the voice parameter label. Each speech segment of the generated target speech has the speech speed, the volume and / or the pause duration required by the speech parameter label, so that the speech segments are more natural, and the adaptation degree with the text is higher.

Description

Technical Field

[0001] The present application relates to the field of speech generation technology, and in particular to a speech generation method, device, storage medium and program product. Background Art

[0002] With the rapid development of mobile Internet, more and more scenarios are using speech generation services. At present, the existing speech generation methods, which generate speech based on text information, are mostly stereotyped templated speech, which is generally unnatural and has low adaptability to text information. Summary of the invention

[0003] Multiple aspects of the present application provide a speech generation method, device, storage medium and program product for more accurately generating speech.

[0004] The embodiment of the present application provides a speech generation method, comprising: obtaining a target text; inputting a model prompt word and the target text into a label generation model based on artificial intelligence (AI); under the guidance of the model prompt word, identifying multiple text segments in the target text, and generating speech parameter labels for each of the multiple text segments; the model prompt word requires that the speech parameter labels generated for the text segments include: speaking rate, volume and / or pause duration; inputting the multiple text segments and their respective speech parameter labels into a speech generation model, generating multiple target speech segments based on the multiple text segments and their respective speech parameter labels, each target speech segment having a speaking rate, volume and / or pause duration required by the corresponding speech parameter label; and splicing the multiple target speech segments to obtain a target speech corresponding to the target text.

[0005] Optionally, the model prompt words include: role prompt words and voice parameter label prompt words; the model prompt words and the target text are input into a label generation model based on artificial intelligence AI, and under the guidance of the model prompt words, multiple text fragments in the target text are identified, and voice parameter labels for each of the multiple text fragments are generated, including: the model prompt words and the target text are input into the label generation model, and under the guidance of the role prompt words, multiple text fragments in the target text are identified, and according to the voice parameter label prompt words, voice parameter labels for each of the multiple text fragments are generated.

[0006] Optionally, under the guidance of the role prompt word, multiple text fragments in the target text are identified, including: under the guidance of the role prompt word, independent semantic information contained in the target text is identified, and according to the independent semantic information contained in the target text, the target text is divided into multiple text fragments, each text fragment has independent semantic information or the target text includes multiple pre-added segmentation markers, under the guidance of the role prompt word, multiple segmentation markers in the target text are identified, and according to the multiple segmentation markers, the target text is divided into multiple text fragments, each text has independent semantic information.

[0007] Optionally, according to the voice parameter label prompt words, voice parameter labels for each of the multiple text fragments are generated, including: according to the voice parameter label prompt words, semantic features of each of the multiple text fragments are extracted, and according to the semantic features of each of the multiple text fragments, voice parameter labels for each of the multiple text fragments are generated.

[0008] Optionally, based on the respective semantic features of the multiple text fragments, speech parameter labels for the multiple text fragments are generated, including: predicting target emotion information corresponding to the multiple text fragments based on the respective semantic features of the multiple text fragments; and determining the speech parameter labels for the multiple text fragments based on the target emotion information corresponding to the multiple text fragments.

[0009] Optionally, based on the respective semantic features of the multiple text fragments, target emotion information corresponding to the multiple text fragments is predicted, including: obtaining a facial image of the target user when inputting the target text; predicting first emotion information corresponding to the multiple text fragments based on the expression features of the facial image; predicting second emotion information corresponding to the multiple text fragments based on the respective semantic features of the multiple text fragments; and determining target emotion information corresponding to the multiple text fragments based on the first emotion information and the second emotion information corresponding to the multiple text fragments.

[0010] Optionally, the multiple text segments and their respective voice parameter labels are input into a speech generation model, and multiple target voice segments are generated based on the multiple text segments and their respective voice parameter labels, including: inputting the multiple text segments and their respective voice parameter labels into the speech generation model, performing text-to-speech processing on the multiple text segments respectively to obtain initial voice segments corresponding to the multiple text segments; and adjusting the speaking speed, volume and / or pause duration of the initial voice segments corresponding to the multiple text segments respectively according to the voice parameter labels of the multiple text segments to obtain multiple target voice segments.

[0011] Optionally, according to the respective speech parameter labels of the multiple text segments, the speaking speed, volume and / or pause duration of the initial speech segments corresponding to the multiple text segments are adjusted to obtain multiple speech segments, including: for any text segment, when the speech parameter label of any text segment includes a pause duration, after the initial speech segment corresponding to the any text segment, a silent speech segment with a duration equal to the pause duration is inserted to obtain a target speech segment.

[0012] Optionally, the multiple text segments and their respective voice parameter labels are input into a speech generation model, and multiple target speech segments are generated based on the multiple text segments and their respective voice parameter labels, including: inputting the multiple text segments and their respective voice parameter labels into a speech generation model, performing feature extraction on each text segment and its voice parameter label to obtain text features and label features corresponding to each text segment; performing speech generation according to the text features and label features corresponding to each text segment to obtain a target speech segment corresponding to each text segment.

[0013] Optionally, before inputting the multiple text fragments and their respective speech parameter labels into the speech generation model, the method further includes: splicing each text fragment and its speech parameter label to obtain a spliced ​​text, and converting the spliced ​​text from an unstructured text format to a structured text format, so that the speech generation model can perform speech generation processing.

[0014] An embodiment of the present application also provides an electronic device, comprising: a memory and a processor; wherein the memory is used to: store one or more computer instructions; and the processor is used to execute the one or more computer instructions to: execute the steps in the speech generation method.

[0015] The embodiment of the present application also provides a computer-readable storage medium, which, when the computer program is executed by a processor, enables the processor to implement the steps in the speech generation method.

[0016] An embodiment of the present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, enables the processor to implement the steps in the speech generation method.

[0017] In this embodiment, a text can be obtained, and the model prompt words and the text can be input into the label generation model. Under the guidance of the model prompt words, the speech parameter labels of multiple text segments are generated; multiple text segments and their respective speech parameter labels are input into the speech generation model, and multiple target speech segments are generated based on the multiple text segments and their respective speech parameter labels, so as to obtain the target speech corresponding to the text by splicing. In this way, the speech parameter labels of the text segments can be generated more accurately based on the label generation model and the model prompt words, and the speech generation can be performed more accurately based on the speech generation model and the speech parameter labels. Each speech segment of the generated target speech has the speech speed, volume and / or pause duration required by the speech parameter label, so it is more natural and more compatible with the text. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 A flowchart of a speech generation method provided by an exemplary embodiment of the present application;

[0020] Figure 2 A flowchart of a speech generation method in an actual scenario provided by an exemplary embodiment of the present application;

[0021] Figure 3 A schematic diagram of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0023] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation portals for users to choose to authorize or refuse. In addition, the various models involved in this application (including but not limited to language models or large models) are in compliance with relevant laws and standards.

[0024] With the rapid development of mobile Internet, more and more scenarios are using speech generation services. At present, the existing speech generation methods, which generate speech based on text information, are mostly stereotyped templated speech, which is generally unnatural and has a low degree of adaptability to text information. For example, the speech generated based on text information has the same speaking speed and volume, and the same pause time is used between speech segments, which is very unnatural for the speech listener.

[0025] In response to the above technical problems, in an embodiment of the present application, the speech parameter labels of the text fragments can be generated more accurately based on the label generation model and the model prompt words, and the speech generation can be performed more accurately based on the speech generation model and the speech parameter labels. Each speech segment of the generated target speech has at least one of the speaking speed, volume and pause duration required by the speech parameter label, and is therefore more natural and more adapted to the text.

[0026] Figure 1 The speech generation method provided by the exemplary embodiment of the present application is as follows: Figure 1 As shown, the method may include the following steps:

[0027] Step 11: Get the target text.

[0028] Step 12: Input the model prompt words and the target text into a label generation model based on artificial intelligence (AI). Under the guidance of the model prompt words, identify multiple text fragments in the target text and generate speech parameter labels for the multiple text fragments. The model prompt words require that the speech parameter labels generated for the text fragments include: speaking speed, volume and / or pause duration.

[0029] Step 13: Input multiple text segments and their respective speech parameter labels into a speech generation model, and generate multiple target speech segments based on the multiple text segments and their respective speech parameter labels, each target speech segment having a speech rate, volume and / or pause duration required by the corresponding speech parameter label.

[0030] Step 14: splice multiple target speech segments to obtain a target speech corresponding to the target text.

[0031] In this embodiment, the target text may be any type of text, such as advertising promotion text, self-introduction text, speech text, etc.; the target text may be input by a user, or may be generated by a text generation model based on artificial intelligence (AI), which is not limited in this embodiment.

[0032] In this embodiment, the model prompt words can be used to guide the label generation model to generate a speech parameter label for the text segment from the set speech parameter dimension according to the semantic features of the text segment, and the set speech parameter dimension includes at least one of speech rate, volume and pause duration. Accordingly, the speech parameter label includes: at least one of a speech rate label, a volume label and a pause duration label. Among them, the label generation model is based on artificial intelligence AI, and through pre-training, it can have the ability to generate speech parameter labels for text segments from the set speech parameter dimension according to the semantic features of the text segment.

[0033] Based on this, the model prompt words and the target text are input into a label generation model based on artificial intelligence AI. Under the guidance of the model prompt words, the label generation model can identify multiple text fragments in the target text and generate speech parameter labels for each of the multiple text fragments.

[0034] In this embodiment, in addition to a label generation model based on artificial intelligence AI, a speech generation model is also provided. The speech generation model has the ability to generate a speech segment for each text segment based on the speech parameters of each text segment through pre-training.

[0035] Based on this, after generating the speech parameter labels of the multiple text segments, the multiple text segments and the respective speech parameter labels can be input into the speech generation model, and multiple target speech segments can be generated based on the multiple text segments and the respective speech parameter labels. Each target speech segment has at least one of the speech speed, volume and pause duration required by the corresponding speech parameter label. The multiple target speech segments are spliced ​​to obtain the target speech corresponding to the target text.

[0036] In this embodiment, the speech parameter labels of the text segments can be generated more accurately based on the label generation model and the model prompt words, and the speech generation can be performed more accurately based on the speech generation model and the speech parameter labels. Each speech segment of the generated target speech has at least one of the speaking speed, volume and pause duration required by the speech parameter label, and is therefore more natural and more adapted to the text.

[0037] In some optional embodiments, the model prompt words in the aforementioned embodiments may include: role prompt words and voice parameter label prompt words. Among them, the role prompt words can be used to prompt the label generation model to identify the text segment and play a role in generating voice parameter labels for the text segment, and the voice parameter label prompt words can be used to limit which voice parameter labels the label generation model generates for each text segment.

[0038] Based on this, step 12 in the aforementioned embodiment, “inputting the model prompt words and the target text into the label generation model based on artificial intelligence AI, identifying multiple text segments in the target text under the guidance of the model prompt words, and generating voice parameter labels for the multiple text segments respectively”, can be implemented based on the following steps:

[0039] Step 121: input the model prompt word and the target text into the label generation model, and under the guidance of the role prompt word, identify multiple text segments in the target text.

[0040] The step 121 can be implemented based on the following two implementations:

[0041] Implementation method 1: The label generation model may include: a semantic recognition layer and a text segmentation layer. The model prompt words and the target text are input into the semantic recognition layer, and under the guidance of the role prompt words, the independent semantic information contained in the target text is identified, wherein the independent semantic information contained in the target text can be used to characterize the text content of the target text. The independent semantic information contained in the target text may be multiple semantic information, and each semantic information in the independent semantic information contained in the target text can be segmented into a segment. Optionally, the segment can be a sentence, a paragraph, or multiple sentences, which is not limited in this embodiment.

[0042] The independent semantic information contained in the target text is input into the text segmentation layer, and the target text is segmented into multiple text segments according to the independent semantic information contained in the target text, and each text segment has independent semantic information. For example: the independent semantic information contained in the target text may be "I am a virtual digital person, I can restore the movements and expressions of real people in a real, natural and smooth manner, and I have a variety of styles for you to choose from", then according to the independent semantic information contained in the target text, the target text can be segmented into the following three text segments (text segment P1, text segment P2 and text segment P3), text segment P1 has the following independent semantic information: "I am a virtual digital person"; text segment P2 has the following independent semantic information: "I can restore the movements and expressions of real people in a real, natural and smooth manner"; text segment P3 has the following independent semantic information: "I have a variety of styles for you to choose from".

[0043] That is to say, in the first embodiment, the role prompt word has the function of prompting the tag generation model to perform semantic recognition and text segmentation. Therefore, the tag generation model can segment the target text into multiple text segments with independent semantic information under the guidance of the role prompt word.

[0044] Implementation method 2: The label generation model may include: a tag recognition layer and a text segmentation layer. The target text may include multiple pre-added segmentation tags. Based on this, the role prompt word and the target text are input into the tag recognition layer, and under the guidance of the role prompt word, multiple segmentation tags in the target text are identified; the multiple segmentation tags and the target text are input into the text segmentation layer, and the target text is segmented into multiple text segments according to the multiple segmentation tags, and each text has independent semantic information.

[0045] That is to say, in the second implementation mode, the role prompt word has the function of prompting the label generation model to perform segmentation mark recognition and text segmentation. Human participation is used to tell the label generation model in advance how to segment the text, and to provide prompts to the label generation model through segmentation marks.

[0046] Through the above two implementations, the label generation model can more accurately identify multiple text segments in the target text under the guidance of the role prompt words. Afterwards, the following step 122 can be performed.

[0047] Step 122: Generate speech parameter labels for the multiple text segments according to the speech parameter label prompts.

[0048] The label generation model may further include: a feature extraction layer and a label generation layer. Based on this, the feature extraction layer may be used to extract the semantic features of each of the multiple text segments according to the voice parameter label prompt words; and the label generation layer may be used to generate the voice parameter labels of each of the multiple text segments according to the semantic features of each of the multiple text segments.

[0049] Optionally, when generating speech parameter labels for multiple text segments according to their respective semantic features, the target emotion information corresponding to the multiple text segments can be first predicted according to their respective semantic features. The target emotion information corresponding to each text segment is used to describe the emotion expressed by the text segment. For example, anger, happiness, sadness, etc., which are not limited in this embodiment. Afterwards, the speech parameter labels for the multiple text segments can be determined according to the target emotion information corresponding to the multiple text segments. It should be noted that this method aims to further more accurately determine the speech parameter labels by identifying the emotions expressed by the text segments. For example, under the emotion of anger, the speech speed is usually faster and the volume is usually louder.

[0050] The embodiments of the present application do not limit the specific implementation method of the above "predicting target emotion information corresponding to multiple text fragments based on the respective semantic features of multiple text fragments". In some exemplary embodiments, the facial image of the target user when entering the target text can be first obtained, for example, the facial images of the target users when entering multiple text fragments. Predict the first emotion information corresponding to the multiple text fragments based on the expression features of the facial images. For example, when the target user has a ferocious facial expression when entering a certain text fragment, it means that the target user is in an angry mood, then the first emotion information corresponding to the text fragment is the anger emotion.

[0051] Afterwards, the second emotion information corresponding to the multiple text segments can be predicted based on the respective semantic features of the multiple text segments; and the target emotion information corresponding to the multiple text segments can be determined based on the first emotion information and the second emotion information corresponding to the multiple text segments. Specifically, the first emotion information and the second emotion information corresponding to any text segment can be weightedly calculated based on the respective weight information of the preset expression features and semantic features to obtain the target emotion information corresponding to the text segment.

[0052] In this way, with the help of the user's facial image, the emotion expressed by the text segment can be more accurately identified, and then the speech parameter label can be more accurately determined by identifying the emotion expressed by the text segment.

[0053] In some optional embodiments, step 13 in the aforementioned embodiment, “inputting multiple text segments and their respective voice parameter labels into a speech generation model, and generating multiple target speech segments based on the multiple text segments and their respective voice parameter labels”, can be implemented based on the following implementations:

[0054] Implementation method 1: The speech generation model may include: a speech conversion layer and a speech adjustment layer.

[0055] The multiple text segments and their respective speech parameter labels are input into the speech conversion layer in the speech generation model, and the speech conversion layer can perform text-to-speech processing on the multiple text segments respectively to obtain initial speech segments corresponding to the multiple text segments. The initial speech segments corresponding to the multiple text segments and the speech parameter labels of the multiple text segments are input into the speech adjustment layer, and at least one of the speech speed, volume and pause length of the initial speech segments corresponding to the multiple text segments is adjusted according to the speech parameter labels of the multiple text segments to obtain multiple target speech segments.

[0056] Specifically, for any text segment, when the speech parameter tag of any text segment includes a pause duration, a silent speech segment with a duration equal to the pause duration can be inserted after the initial speech segment corresponding to any text segment to obtain a target speech segment.

[0057] Implementation method 2: The speech generation model may include: a feature extraction layer and a speech generation layer.

[0058] Multiple text segments and their respective speech parameter labels are input into the feature extraction layer, and the feature extraction layer is used to extract features of each text segment and its speech parameter label to obtain text features and label features corresponding to each text segment; the text features and label features corresponding to each text segment are input into the speech generation layer, and speech generation is performed according to the text features and label features corresponding to each text segment to obtain a target speech segment corresponding to each text segment.

[0059] In the above implementation mode 1, an initial speech segment can be generated first, and then the initial speech segment can be adjusted according to the speech parameter label to obtain multiple target speech segments. Different from the implementation mode 1, in the implementation mode 2, speech generation can be performed directly based on the speech generation model according to the text features and label features corresponding to each text segment to obtain multiple target speech segments.

[0060] Optionally, before inputting multiple text segments and their respective voice parameter tags into the speech generation model, each text segment and its voice parameter tag may be spliced ​​to obtain a spliced ​​text, and the spliced ​​text is converted from an unstructured text format to a structured text format, so that the speech generation model can perform speech generation processing. The spliced ​​text with a structured text format means that the spliced ​​text has a clear format and organizational form, and the structured text format may be a Json (JavaScript Object Notation) format or other formats, which is not limited in this embodiment.

[0061] In some optional embodiments, sample model prompt words, sample texts and sample speech parameter labels may also be obtained, and the sample model prompt words and sample texts may be input into a preset label generation network. Under the supervision of the sample speech parameter labels, the label generation network may be iteratively trained with the sample texts with the goal of converging the loss function of the label generation network to a specified range, thereby obtaining a label generation model. The loss function is used to characterize the error between the speech parameter labels of the multiple text segments generated by the label generation network under the guidance of the sample model prompt words and the sample speech parameter labels.

[0062] In each iteration round, under the guidance of the sample model prompt words, multiple text fragments in the sample text are identified, and sample speech parameter labels for each of the multiple text fragments are generated. According to the sample speech parameter labels for each of the multiple text fragments and the sample speech parameter labels, the loss function is calculated. If the loss function does not converge to the specified range, the parameters of the label generation network are adjusted and the next iteration round is entered; if the loss function converges to the specified range, the iterative training is stopped and the label generation code model is output.

[0063] In some optional embodiments, multiple sample text segments, multiple speech parameter labels of each sample text segment, and multiple sample speech segments can also be obtained and input into a preset speech generation network. Under the supervision of the sample speech, the speech generation network is trained using the multiple sample text segments and the speech parameter labels of each sample text segment to obtain a speech generation model with the goal of converging the loss function of the speech generation network to a specified range. The loss function is used to describe the error between the multiple speech segments generated by the speech generation network according to the multiple sample text segments and their respective speech parameter labels and the multiple sample speech segments.

[0064] The following will further explain the speech generation method in combination with actual scenarios.

[0065] like Figure 2 As shown, in an actual scenario, the large model (i.e., the label generation model in the aforementioned embodiment) can generate a target text with a label. For example, the target text with a label can be:

[0066] I am a virtual digital person. <style speed＝1.1mute＝500volume＝90>我能够真实自然流畅的还原真人的动作和表情< / style> <style speed＝0.9mute＝1000volume＝120>有多种风格供你选择

[0067] <style speed＝1mute＝1000volume＝100>。

[0068] 之后，可将上述中的目标文本，根据标签进行拆分，从而解析成以下的json列表，即将从非结构化的文本格式转换为结构化的文本格式。[

[0070] {

[0071] "text":"我是一个虚拟的数字人，",

[0072] "speed":1.1,

[0073] "mute":500,

[0074] valume:90

[0075] },

[0076] {

[0077] "text":"我能够真实自然流畅的还原真人的动作和表情",

[0078] "speed":0.9,

[0079] "mute":1000,

[0080] valume:120

[0081] },

[0082] {

[0083] "text":"有多种风格供你选择",

[0084] "speed":1,

[0085] "mute":1000,

[0086] valume:100

[0087] },]

[0089] 之后，可根据上述的json列表，逐个生成每个语音片段。比如第二个语音片段，对应的文本片段是：我能够真实自然流畅的还原真人的动作和表情。语速是：0.9倍速，语句之后的停顿是120毫秒，音量是常规的120％。首先，在文本转语音的时候，使用0.9倍速生成。在生成之后，调整音频文件的音量为原来的120％。最后，再将一个120毫秒的静默视频拼接到调整过音量之后的语音片段后面，从而达到停顿的效果。需要说明的是，如果文本片段并未设置对应的语音参数标签，则该文本片段对应的语音片段默认使用1倍速、正常音量以及无停顿。

[0090] 基于上述步骤，将上述各个文本片段都按照格式生成语音片段之后，可将各个语音片段拼接起来，上传到云存储，并将拼接得到的目标语音的语音链接返回至调用方，即目标文本的发出方。通过这种方式，最终得到的目标语音的声音听起来更自然，更像真人在说话，增加真实性。

[0091] 需要说明的是，上述实施例所提供方法的各步骤的执行主体均可以是同一设备，或者，该方法也由不同设备作为执行主体。比如，步骤11至步骤14的执行主体可以为设备A；又比如，步骤11到12的执行主体可以为设备A，步骤13到14的执行主体可以为设备B；等等。

[0092] 另外，在上述实施例及附图中的描述的一些流程中，包含了按照特定顺序出现的多个操作，但是应该清楚了解，这些操作可以不按照其在本文中出现的顺序来执行或并行执行，操作的序号如12、13等，仅仅是用于区分开各个不同的操作，序号本身不代表任何的执行顺序。另外，这些流程可以包括更多或更少的操作，并且这些操作可以按顺序执行或并行执行。

[0093] 需要说明的是，本文中的"第一”、"第二”等描述，是用于区分不同的消息、设备、模块等，不代表先后顺序，也不限定"第一”和"第二”是不同的类型。

[0094] 图3是本申请一示例性实施例提供的电子设备的结构示意图，该电子设备适用于前述实施例提供的语音生成方法，如图3所示，该电子设备可包括：存储器301、处理器302以及通信组件303。

[0095] 存储器301，用于存储计算机程序，并可被配置为存储其它各种数据以支持在电子设备上的操作。这些数据的示例包括用于在电子设备上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。

[0096] 在一些示例性的实施例中，处理器302，与存储器301耦合，用于执行存储器301中的计算机程序，以用于：获取目标文本；将模型提示词和所述目标文本输入到基于人工智能AI的标签生成模型，在所述模型提示词的引导下，识别所述目标文本中的多个文本片段，生成所述多个文本片段各自的语音参数标签；所述模型提示词要求为文本片段生成的语音参数标签包括：语速、音量和 / 或停顿时长；将所述多个文本片段以及各自的语音参数标签输入语音生成模型，基于所述多个文本片段以及各自的语音参数标签生成多个目标语音片段，每个目标语音片段具有对应语音参数标签要求的语速、音量和 / 或停顿时长；将所述多个目标语音片段进行拼接，以得到所述目标文本对应的目标语音。

[0097] 可选地，所述模型提示词包括：角色提示词和语音参数标签提示词；处理器302将模型提示词和所述目标文本输入到基于人工智能AI的标签生成模型，在所述模型提示词的引导下，识别所述目标文本中的多个文本片段，生成所述多个文本片段各自的语音参数标签时，具体用于：将所述模型提示词和所述目标文本输入到标签生成模型，在所述角色提示词的引导下，识别所述目标文本中的多个文本片段，并按照所述语音参数标签提示词，生成所述多个文本片段各自的语音参数标签。

[0098] 可选地，处理器302在所述角色提示词的引导下，识别所述目标文本中的多个文本片段时，具体用于：在所述角色提示词的引导下，识别所述目标文本包含的独立语义信息，根据所述目标文本包含的独立语义信息，将所述目标文本切分为多个文本片段，每个文本片段具有独立的语义信息；或者所述目标文本包括预先添加的多个分段标记，在所述角色提示词的引导下，识别所述目标文本中的多个分段标记，根据所述多个分段标记将所述目标文本切分为多个文本片段，每个文本具有独立的语义信息。

[0099] 可选地，处理器302按照所述语音参数标签提示词，生成所述多个文本片段各自的语音参数标签时，具体用于：按照所述语音参数标签提示词，提取所述多个文本片段各自的语义特征，根据所述多个文本片段各自的语义特征，生成所述多个文本片段各自的语音参数标签。

[0100] 可选地，处理器302根据所述多个文本片段各自的语义特征，生成所述多个文本片段各自的语音参数标签时，具体用于：根据所述多个文本片段各自的语义特征，预测所述多个文本片段对应的目标情绪信息；根据所述多个文本片段对应的目标情绪信息，确定所述多个文本片段各自的语音参数标签。

[0101] 可选地，处理器302根据所述多个文本片段各自的语义特征，预测所述多个文本片段对应的目标情绪信息时，具体用于：获取目标用户在输入所述目标文本时的面部图像；根据所述面部图像的表情特征，预测所述多个文本片段对应的第一情绪信息；根据所述多个文本片段各自的语义特征，预测所述多个文本片段对应的第二情绪信息；根据所述多个文本片段对应的第一情绪信息和第二情绪信息，确定所述多个文本片段对应的目标情绪信息。

[0102] 可选地，处理器302将所述多个文本片段以及各自的语音参数标签输入语音生成模型，基于所述多个文本片段以及各自的语音参数标签生成多个目标语音片段，包括：将所述多个文本片段以及各自的语音参数标签输入所述语音生成模型，对所述多个文本片段分别进行文本转语音处理，以得到所述多个文本片段各自对应的初始语音片段；根据所述多个文本片段各自的语音参数标签，对所述多个文本片段各自对应的初始语音片段的语速、音量和 / 或停顿时长的进行调整，以得到多个目标语音片段。

[0103] 可选地，处理器302根据所述多个文本片段各自的语音参数标签，对所述多个文本片段各自对应的初始语音片段的语速、音量和 / 或停顿时长的进行调整，以得到多个语音片段时，具体用于：针对任一文本片段，在所述任一文本片段的语音参数标签包括停顿时长的情况下，在所述任一文本片段对应的初始语音片段后，插入时长为所述停顿时长的静默语音片段，以得到目标语音片段。

[0104] 可选地，处理器302将所述多个文本片段以及各自的语音参数标签输入语音生成模型，基于所述多个文本片段以及各自的语音参数标签生成多个目标语音片段时，具体用于：将所述多个文本片段以及各自的语音参数标签输入语音生成模型，对每个文本片段和其语音参数标签进行特征提取，以得到每个文本片段对应的文本特征和标签特征；根据每个文本片段对应的文本特征和标签特征进行语音生成，以得到每个文本片段对应的目标语音片段。

[0105] 可选地，处理器302将所述多个文本片段以及各自的语音参数标签输入语音生成模型之前，还用于：将每个文本片段和其语音参数标签进行拼接，以得到拼接文本，并将所述拼接文本从非结构化的文本格式转换为结构化的文本格式，以便于所述语音生成模型进行语音生成处理。

[0106] 进一步，如图3所示，该电子设备还包括：显示器304、电源组件305、音频组件306等其它组件。图3中仅示意性给出部分组件，并不意味着电子设备只包括图3所示组件。

[0107] 本申请实施例还提供一种计算机可读存储介质，当所述计算机程序被处理器执行时，致使所述处理器能够实现所述语音生成方法中的步骤。

[0108] 本申请实施例还提供一种计算机程序产品，包括计算机程序 / 指令，当所述计算机程序 / 指令被处理器执行时，执行所述语音生成方法中的步骤。

[0109] 在本实施例中，可获取文本，将模型提示词和文本输入到标签生成模型，在模型提示词的引导下，生成多个文本片段的语音参数标签；将多个文本片段以及各自的语音参数标签输入语音生成模型，基于多个文本片段以及各自的语音参数标签生成多个目标语音片段，从而拼接得到文本对应的目标语音。通过这种方式，可基于标签生成模型和模型提示词较为准确地生成文本片段的语音参数标签，并基于语音生成模型和语音参数标签较为准确地进行语音生成，生成的目标语音的每个语音片段具有语音参数标签要求的语速、音量和 / 或停顿时长，因而更加自然，与文本的适配度更高。

[0110] 上述存储器可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(Static Random-Access Memory，SRAM)，电可擦除可编程只读存储器(Electrically Erasable Programmable Read Only Memory，EEPROM)，可擦除可编程只读存储器(Erasable Programmable Read Only Memory，EPROM)，可编程只读存储器(Programmable Read-Only Memory，PROM)，只读存储器(Read-Only Memory，ROM)，磁存储器，快闪存储器，磁盘或光盘。

[0111] 上述通信组件被配置为便于通信组件所在设备和其他设备之间有线或无线方式的通信。通信组件所在设备可以接入基于通信标准的无线网络，如WiFi，2G、3G、4G / LTE、5G等移动通信网络，或它们的组合。在一个示例性实施例中，通信组件经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，通信组件还包括近场通信(Near Field Communication，NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(Radio Frequency Identification，RFID)技术，红外数据协会(InfraredData Association，IrDA)技术，超宽带(Ultra Wide Band，UWB)技术，蓝牙(BlueTooth，BT)技术和其他技术来实现。

[0112] 上述显示器包括屏幕，其屏幕可以包括液晶显示器(Liquid Crystal Display，LCD)和触摸面板(TouchPanel，TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与触摸或滑动操作相关的持续时间和压力。

[0113] 上述电源组件，为电源组件所在设备的各种组件提供电力。电源组件可以包括电源管理系统，一个或多个电源，及其他与为电源组件所在设备生成、管理和分配电力相关联的组件。

[0114] 上述音频组件，可被配置为输出和 / 或输入音频信号。例如，音频组件包括一个麦克风(Microphone，MIC)，当音频组件所在设备处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器或经由通信组件发送。在一些实施例中，音频组件还包括一个扬声器，用于输出音频信号。

[0115] 本领域内的技术人员应明白，本申请的实施例可提供为方法、系统、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可读存储介质(包括但不限于磁盘存储器、只读光盘(Compact Disc Read-Only Memory，CD-ROM)、光学存储器等)上实施的计算机程序产品的形式。

[0116] 本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和 / 或方框图来描述的。应理解可由计算机程序指令实现流程图和 / 或方框图中的每一流程和 / 或方框、以及流程图和 / 或方框图中的流程和 / 或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和 / 或方框图一个方框或多个方框中指定的功能的装置。

[0117] 这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和 / 或方框图一个方框或多个方框中指定的功能。

[0118] 这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和 / 或方框图一个方框或多个方框中指定的功能的步骤。

[0119] 在一个典型的配置中，计算设备包括一个或多个处理器(Central ProcessingUnit，CPU)、输入 / 输出接口、网络接口和内存。

[0120] 内存可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RandomAccess Memory，RAM)和 / 或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。

[0121] 计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(Phase-change Random AccessMemory，PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(Dynamic RandomAccess Memory，DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(Digital Video Disc，DVD)或其他光学存储、磁盒式磁带，磁带磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。按照本文中的界定，计算机可读介质不包括暂存电脑可读媒体(transitory media)，如调制的数据信号和载波。

[0122] 还需要说明的是，术语"包括”、"包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、商品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、商品或者设备所固有的要素。在没有更多限制的情况下，由语句"包括一个……”限定的要素，并不排除在包括要素的过程、方法、商品或者设备中还存在另外的相同要素。

[0123] 以上仅为本申请的实施例而已，并不用于限制本申请。对于本领域技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原理之内所作的任何修改、等同替换、改进等，均应包含在本申请的权利要求范围之内。< / style>

Claims

1. A speech generation method, characterized in that: include: Get the target text; Inputting the model prompt words and the target text into a label generation model based on artificial intelligence AI, and under the guidance of the model prompt words, identifying multiple text segments in the target text, and generating speech parameter labels for each of the multiple text segments; The model prompt word requires that the speech parameter labels generated for the text segment include: speech rate, volume and / or pause duration; Inputting the multiple text segments and their respective speech parameter tags into a speech generation model, generating multiple target speech segments based on the multiple text segments and their respective speech parameter tags, each target speech segment having a speech rate, volume and / or pause duration required by the corresponding speech parameter tag; The multiple target speech segments are concatenated to obtain a target speech corresponding to the target text.

2. The method according to claim 1, characterized in that: The model prompt words include: role prompt words and voice parameter label prompt words; the model prompt words and the target text are input into a label generation model based on artificial intelligence AI, and under the guidance of the model prompt words, multiple text segments in the target text are identified, and voice parameter labels for each of the multiple text segments are generated, including: The model prompt words and the target text are input into a label generation model, and under the guidance of the role prompt words, multiple text segments in the target text are identified, and speech parameter labels for each of the multiple text segments are generated according to the speech parameter label prompt words.

3. The method according to claim 2, characterized in that Under the guidance of the role prompt word, multiple text segments in the target text are identified, including: Under the guidance of the role prompt words, the independent semantic information contained in the target text is identified, and according to the independent semantic information contained in the target text, the target text is divided into a plurality of text segments, each of which has independent semantic information; or The target text includes a plurality of pre-added segmentation marks. Under the guidance of the role prompt words, the plurality of segmentation marks in the target text are identified, and the target text is segmented into a plurality of text segments according to the plurality of segmentation marks, each of which has independent semantic information.

4. The method according to claim 3, characterized in that Generating speech parameter labels for the plurality of text segments according to the speech parameter label prompts includes: According to the voice parameter label prompt words, the semantic features of each of the multiple text segments are extracted, and according to the semantic features of each of the multiple text segments, the voice parameter labels of each of the multiple text segments are generated.

5. The method according to claim 4, characterized in that Generating speech parameter labels for the multiple text segments according to the semantic features of the multiple text segments respectively includes: Predicting target emotion information corresponding to the multiple text segments according to the respective semantic features of the multiple text segments; According to the target emotion information corresponding to the multiple text segments, the speech parameter labels of the multiple text segments are determined.

6. The method according to claim 5, characterized in that Predicting target emotion information corresponding to the multiple text segments according to the respective semantic features of the multiple text segments includes: Acquire a facial image of a target user when the target user inputs the target text; Predicting first emotion information corresponding to the plurality of text segments according to the expression features of the facial image; Predicting second emotion information corresponding to the multiple text segments according to the respective semantic features of the multiple text segments; According to the first emotion information and the second emotion information corresponding to the multiple text segments, target emotion information corresponding to the multiple text segments is determined.

7. The method according to any one of claims 1 to 6, characterized in that: Inputting the plurality of text segments and their respective speech parameter labels into a speech generation model, and generating a plurality of target speech segments based on the plurality of text segments and their respective speech parameter labels, comprising: Inputting the multiple text segments and their respective speech parameter labels into the speech generation model, and performing text-to-speech processing on the multiple text segments respectively to obtain initial speech segments corresponding to the multiple text segments respectively; According to the speech parameter labels of the multiple text segments, the speech rate, volume and / or pause duration of the initial speech segments corresponding to the multiple text segments are adjusted to obtain multiple target speech segments.

8. The method according to claim 7, characterized in that According to the speech parameter tags of the multiple text segments, the speech rate, volume and / or pause duration of the initial speech segments corresponding to the multiple text segments are adjusted to obtain multiple speech segments, including: For any text segment, when the speech parameter tag of the text segment includes a pause duration, a silent speech segment with a duration equal to the pause duration is inserted after the initial speech segment corresponding to the text segment to obtain a target speech segment.

9. The method according to any one of claims 1 to 6, characterized in that: Inputting the plurality of text segments and their respective speech parameter labels into a speech generation model, and generating a plurality of target speech segments based on the plurality of text segments and their respective speech parameter labels, comprising: Inputting the multiple text segments and their respective voice parameter labels into a speech generation model, performing feature extraction on each text segment and its voice parameter label to obtain text features and label features corresponding to each text segment; Speech generation is performed according to the text features and label features corresponding to each text segment to obtain a target speech segment corresponding to each text segment.

10. The method according to any one of claims 1 to 6, characterized in that: Before inputting the plurality of text segments and their respective speech parameter labels into the speech generation model, the method further includes: Each text segment and its speech parameter label are concatenated to obtain a concatenated text, and the concatenated text is converted from an unstructured text format to a structured text format, so that the speech generation model can perform speech generation processing.

11. A terminal device, characterized in that: include: A memory and a processor; wherein the memory is used to: store one or more computer instructions; The processor is configured to execute the one or more computer instructions to perform the steps of the method according to any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that: When the computer program is executed by a processor, the processor is enabled to implement the steps of the method according to any one of claims 1 to 10.

13. A computer program product, characterized in that The method comprises a computer program / instruction, which, when executed by a processor, enables the processor to implement the steps of the method according to any one of claims 1 to 10.

Images