Voice generation method and device, storage medium and electronic equipment

Abstract

The invention discloses a voice generation method and device, a storage medium and electronic equipment, and relates to the field of artificial intelligence and voice synthesis. The method comprises the steps of obtaining a to-be-processed text; converting the to-be-processed text into a first text according to the target mark library; analyzing the first text to obtain a control parameter list corresponding to S sub-texts in the first text; the to-be-processed text and the control parameter list corresponding to the S sub-texts are input into a target model, target voice information for the to-be-processed text is generated according to the target model, and the target model is a pre-trained model used for generating voice information meeting the preset requirement. The preset requirement is used for constraining output characteristics of the target voice information. The technical problem that speech emotion, intonation and professional pronunciation cannot be accurately controlled and synthesized in the existing speech synthesis field is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and speech synthesis, and in particular, to a speech generation method and device, a storage medium and an electronic device. BACKGROUND

[0002] With the continuous evolution of artificial intelligence technology, speech synthesis (TTS) systems have become an important bridge for human-computer interaction, widely used in intelligent assistants, educational software and accessibility services, and many other fields. Although deep learning-driven TTS models have significantly improved the naturalness and fluency of speech, in practical applications, they still face the challenge of precise expression control.

[0003] Specifically, most systems can only provide limited pre-set emotional styles when processing emotional speech synthesis, lacking the ability to dynamically adjust the emotional details of the text, resulting in insufficient emotional expression and naturalness of the synthesized speech. In addition, the control of intonation and speech rate is often too rough, failing to meet the precise control requirements for reading rhythm, pause time, stress position, and other details, especially in scenarios such as drama and news broadcasting, where high-precision requirements for intonation changes cannot be fully met. Furthermore, the accurate pronunciation of specific vocabulary in professional fields, especially the synthesis of industry-specific terms such as medical, legal, and technical terms, faces the problem of insufficient training data or poor model adaptability, often leading to pronunciation errors and affecting the professionalism and credibility of speech synthesis. In response to these challenges, existing technologies have not yet provided a general, flexible, and high-precision control method that can simultaneously consider emotional expression, intonation adjustment, and accurate control of professional pronunciation.

[0004] To address the above problems, no effective solutions have been proposed so far. SUMMARY

[0005] The present application provides a speech generation method, device, storage medium and electronic device to at least solve the technical problem of being unable to accurately control the emotion, intonation and professional pronunciation of synthesized speech in the existing field of speech synthesis.

[0006] According to an aspect of the present application, a voice generation method is provided, comprising: obtaining a text to be processed; converting the text to be processed into a first text according to a target tag library, wherein the target tag library is a set of a plurality of predefined tags and attributes corresponding to each tag, wherein each tag is used to identify a speech segment in the text to be processed that needs to be processed, and the attribute is used to adjust the output characteristics of the voice information corresponding to the text to be processed; parsing the first text to obtain a control parameter list corresponding to S subtexts in the first text, wherein S is an integer greater than or equal to 1, and the control parameter list includes an attribute corresponding to each subtext and an attribute value corresponding to the attribute; inputting the text to be processed and the control parameter list corresponding to the S subtexts into a target model to generate target voice information for the text to be processed according to the target model, wherein the target model is a pre-trained model used to generate voice information that meets predetermined requirements, and the predetermined requirements are used to constrain the output characteristics of the target voice information.

[0007] Optionally, the target tag library is obtained by the following steps: setting a target tag set, wherein the target tag set includes a plurality of predefined tags; setting a target attribute set, wherein the attributes in the target attribute set are control parameters associated with the tags in the target tag set; and determining the target tag library according to the target tag set and the target attribute set.

[0008] Optionally, the target attribute set includes at least the following attributes: a first attribute, wherein the first attribute is used to adjust the playback speed of the generated voice information; a second attribute, wherein the second attribute is used to adjust the tone of the generated voice information; a third attribute, wherein the third attribute is used to adjust the emotion of the generated voice information; a fourth attribute, wherein the fourth attribute is used to adjust the pronunciation of the text; and a fifth attribute, wherein the fifth attribute is used to represent the domain information to which the text belongs.

[0009] Optionally, parsing the first text to obtain a control parameter list corresponding to S subtexts in the first text includes: identifying the tags in the first text to obtain a first tag set; extracting the attributes corresponding to each tag in the first tag set and the attribute values corresponding to the attributes to obtain a first attribute set; determining the nesting relationship between the tags in the first tag set; dividing the first text into S subtexts according to the first tag set and the nesting relationship, and generating a control parameter list for each subtext based on the first tag set, the nesting relationship, and the first attribute set.

[0010] Optionally, the first text is divided into S subtexts according to the first set of marks and the nesting relationship, and a corresponding control parameter list is generated for each subtext based on the first set of marks, the nesting relationship, and the first set of attributes, including: dividing the first text into S subtexts according to the position information of each mark in the first set of marks and the nesting relationship; determining the attribute corresponding to each subtext and the attribute value corresponding to the attribute from the first set of attributes based on the nesting relationship and at least one mark corresponding to each subtext; and generating a corresponding control parameter list for each subtext according to the attribute corresponding to each subtext and the attribute value corresponding to the attribute.

[0011] Optionally, the control parameter list corresponding to the to-be-processed text and the S subtexts is input into the target model, and target voice information for the to-be-processed text is generated according to the target model, including: updating the control parameter list corresponding to each subtext based on a mapping rule; wherein the mapping rule includes a mapping relationship between each attribute in the control parameter list and a parameter in the target model, and the mapping rule is used to map each attribute in the control parameter list corresponding to each subtext to a corresponding control vector, wherein the control vector is used to guide the target model to generate voice information; and inputting the to-be-processed text and the updated control parameter list corresponding to each subtext into the target model, and outputting target voice information for the to-be-processed text according to the target model.

[0012] Optionally, the control parameter list corresponding to the to-be-processed text and the S subtexts is input into the target model, and target voice information for the to-be-processed text is generated according to the target model, including: preprocessing the to-be-processed text to obtain a target text, wherein the preprocessing is used to convert the to-be-processed text into a unified format; converting the target text into a phoneme sequence based on a fourth attribute and a fifth attribute in the control parameter list corresponding to each subtext, wherein the fourth attribute is used to adjust the pronunciation of the text, and the fifth attribute is used to represent the domain information to which the text belongs; encoding the phoneme sequence and the control vector in the control parameter list corresponding to each subtext through an encoding module of the target model to obtain a target vector and a target control vector set corresponding to each subtext; predicting the acoustic characteristics of the output voice information for the to-be-processed text according to the target vector and the target control vector set corresponding to each subtext through an acoustic feature prediction module of the target model; and generating target voice information for the to-be-processed text according to the acoustic characteristics and the target control vector set corresponding to each subtext through a vocoder of the target model.

[0013] According to another aspect of the present application, a voice generation apparatus is also provided, comprising: an obtaining unit configured to obtain a to-be-processed text; a converting unit configured to convert the to-be-processed text into a first text according to a target tag library, wherein the target tag library is a set of a plurality of predefined tags and corresponding attributes of each tag, wherein each tag is used to identify a speech segment in the to-be-processed text that needs to be processed, and the attributes are used to adjust output characteristics of corresponding voice information of the to-be-processed text; an analyzing unit configured to analyze the first text to obtain a control parameter list corresponding to S subtexts in the first text, wherein S is an integer greater than or equal to 1, and the control parameter list includes attributes corresponding to each subtext and attribute values corresponding to the attributes; and a generating unit configured to input the to-be-processed text and the control parameter list corresponding to the S subtexts into a target model, and generate target voice information for the to-be-processed text according to the target model, wherein the target model is a pre-trained model used to generate voice information meeting preset requirements, and the preset requirements are used to constrain output characteristics of the target voice information.

[0014] According to another aspect of the present application, a computer-readable storage medium is also provided, comprising a stored executable program, wherein when the executable program is executed, the computer-readable storage medium controls a device where the computer-readable storage medium is located to perform the voice generation method described above.

[0015] According to another aspect of the present application, an electronic device is also provided, comprising one or more processors and a memory, the memory being configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the voice generation method described above.

[0016] In the present application, first, the text to be processed is acquired, then the text to be processed is converted into a first text according to a target mark library, wherein the target mark library is a set of a plurality of predefined marks and attributes corresponding to each mark, wherein each mark is used to identify a segment of the text to be processed, and the attribute is used to adjust the output characteristics of the voice information corresponding to the text to be processed, then the first text is parsed to obtain a control parameter list corresponding to S subtexts in the first text, wherein S is an integer greater than or equal to 1, the control parameter list includes an attribute corresponding to each subtext and an attribute value corresponding to the attribute, and finally the text to be processed and the control parameter list corresponding to the S subtexts are input into a target model to generate target voice information for the text to be processed according to the target model, wherein the target model is a pre-trained model used to generate voice information meeting preset requirements, wherein the preset requirements are used to constrain the output characteristics of the target voice information. That is, by embedding customizable marks in the text and converting them into model control parameters, the purpose of accurately controlling the output characteristics of the synthesized voice is achieved, thereby achieving the technical effects of improving the naturalness, emotional richness and professional field adaptability of the synthesized voice, and further solving the technical problems of being unable to accurately control the synthesized voice emotion, tone and professional pronunciation in the existing voice synthesis field. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of this application and help to explain the present application, but do not limit the present application in any way. In the drawings:

[0018] Figure 1 is a flowchart of an optional voice generation method according to an embodiment of the present application;

[0019] Figure 2 is a schematic diagram of an optional voice generation device according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor should fall within the scope of protection of the present application.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0022] It should be noted that the collected information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for display, analyzed data, etc.) involved in the present application are information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards, necessary security measures are taken, do not violate public order and good customs, and provide corresponding operation portal for user to choose authorization or refusal. For example, the system and related users or institutions are provided with an interface to provide the user with a corresponding operation portal for the user to choose to agree or refuse the automatic decision result; if the user chooses to refuse, the expert decision process is entered.

[0023] According to an embodiment of the present application, a method embodiment of a speech generation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0024] It should be noted that the speech synthesis system can be the execution subject of the speech generation method according to the embodiments of the present application. It can be understood that the speech generation method provided by the embodiments of the present application can also be executed by other systems or devices as the execution subject, and the embodiments of the present application do not make specific limitations.

[0025] Figure 1 is a flowchart of an optional speech generation method according to an embodiment of the present application, as shown in Figure 1 The method comprises the following steps:

[0026] Step S101, obtaining a text to be processed.

[0027] Optionally, the text to be processed refers to the original input text for the speech synthesis system to convert into speech, which can be any written content that needs to be converted into speech, such as articles, news, dialogue text, etc.

[0028] Optionally, the speech synthesis system receives or reads the text data submitted by the user as the starting point of the subsequent speech synthesis process. This step ensures that the content to be processed by the system is clearly and completely defined.

[0029] Step S102: converting the text to be processed into a first text according to a target tag library.

[0030] In step S102, the target tag library is a set of pre-defined tags and attributes corresponding to each tag.

[0031] In step S102, each tag is used to identify a speech segment in the text to be processed, and the attribute is used to adjust the output characteristics of the corresponding speech information of the text to be processed.

[0032] Optionally, the target tag library contains a series of pre-defined tags (such as <voice>, <phoneme> 、 <prosody>and their attributes, for inserting control information in the text.

[0033] Optionally, converting into the first text means that, based on the original text, by inserting the marks in the target mark library, a marked text with voice control information is generated.

[0034] Optionally, the voice synthesis system uses the target mark library to check the text to be processed, and the user or automated tool can insert marks in the text, which contain specific attributes such as speed, pitch, emotion, etc., to guide the voice synthesis system on how to process specific segments. This process provides additional control instructions for voice synthesis, enabling the system to adjust the characteristics of the output voice according to the marks, thus more accurately reflecting the author's intentions.

[0035] Step S103, parsing the first text to obtain a control parameter list corresponding to S subtexts in the first text.

[0036] In step S103, S is an integer greater than or equal to 1, and the control parameter list includes attributes corresponding to each subtext and attribute values corresponding to the attributes.

[0037] Optionally, the control parameter list is a parameter set extracted by the parsing module in the system from the first text to guide voice synthesis. It includes specific attributes and their attribute values corresponding to each segment, such as speed 1.2x, pitch reduction 20%, emotion marked as "happy", etc.

[0038] Optionally, the parsing module reads the first text, identifies the marks and their attributes, and then converts these marks into a structured data format such as a JSON list, which contains the control parameters of S subtexts (S≥1) in the first text. This step is to "translate" the marks inserted by the user or system into control instructions that the system can understand, preparing data for the next parameter mapping and model input.

[0039] Step S104, inputting the text to be processed and the control parameter list corresponding to S subtexts into the target model, and generating target voice information for the text to be processed according to the target model.

[0040] In step S104, the target model is a pre-trained model used to generate voice information that meets predetermined requirements.

[0041] In step S104, the predetermined requirements are used to constrain the output characteristics of the target voice information.

[0042] Optionally, the target model is a pre-trained deep learning model for generating speech information, and the model is optimized to identify and respond to the control parameters, ensuring that the output speech meets the preset expression requirements.

[0043] Optionally, the speech synthesis system inputs the original text to be processed and the control parameter list corresponding to the S subtexts into the target model. The target model adjusts its behavior according to these parameters, for example, adjusting the speech speed, tone, or emotional color, and finally outputting target speech information with user-specified characteristics. This step is the core of the entire technical implementation, and through the flexibility and learning ability of the model, it ensures the controllability and high quality of the output speech, solving the problems of traditional speech synthesis systems in emotion, tone, and professional pronunciation control.

[0044] As can be seen from steps S101 to S104, in the present application, first, the text to be processed is obtained, then the text to be processed is converted into a first text according to a target mark library, wherein the target mark library is a set of a plurality of predefined marks and attributes corresponding to each mark, wherein each mark is used to identify a speech segment in the text to be processed that needs to be processed, and the attribute is used to adjust the output characteristics of the speech information corresponding to the text to be processed, then the first text is parsed to obtain a control parameter list corresponding to S subtexts in the first text, wherein S is an integer greater than or equal to 1, the control parameter list includes an attribute corresponding to each subtext and an attribute value corresponding to the attribute, and finally the text to be processed and the control parameter list corresponding to the S subtexts are input into a target model to generate target speech information for the text to be processed according to the target model, wherein the target model is a pre-trained model for generating speech information that meets preset requirements, wherein the preset requirements are used to constrain the output characteristics of the target speech information. That is, by embedding customizable marks in the text and converting them into model control parameters, the purpose of accurately controlling the output characteristics of speech synthesis is achieved, thereby achieving the technical effect of improving the naturalness, emotional richness, and professional field adaptability of speech synthesis, and further solving the technical problem of being unable to accurately control the emotion, tone, and professional pronunciation of synthesized speech in the existing field of speech synthesis.

[0045] In an optional embodiment, the target mark library is obtained by the following steps: first, the speech synthesis system sets a target mark set, wherein the target mark set includes a plurality of predefined marks, then sets a target attribute set, wherein the attributes in the target attribute set are control parameters associated with the marks in the target mark set, and then determines the target mark library according to the target mark set and the target attribute set.

[0046] Optionally, the speech synthesis system first defines a set of marks, which will be used as containers to control the characteristics of synthesized speech. The mark set includes but is not limited to <voice>、 <phoneme>、 <prosody>, respectively, corresponding to voice global control, phoneme-level pronunciation control, intonation and emotion control. For example, <voice>The tags can be used to adjust the pace and tone of the entire sentence, while <phoneme>Tags are used to specify the way a particular word is pronounced.

[0047] Optionally, the system then sets a series of attributes for each of the aforementioned tags, which act as control parameters to adjust specific characteristics of the synthesized speech. For example, <voice>The tags can be associated with speed, pitch, and volume attributes, <prosody>The tags can be associated with emotion, emphasis, and pause attributes. The setting of the attribute set enables each tag to carry more rich control information to fine-tune the output characteristics of the voice.

[0048] Optionally, through the combination of the first two steps, the system forms a target tag library, which is a comprehensive control tag system containing all predefined tags and their corresponding attributes. The establishment of the target tag library provides users with an intuitive and flexible control means, enabling them to directly embed tags and attributes when inputting text to guide the voice synthesis system to generate voice according to specific requirements.

[0049] From the above, through the specific implementation of constructing the target tag library and applying it to voice synthesis described above, the embodiment realizes fine control of the synthesis voice process. Users can directly add tags in the text to guide the voice synthesis system to make specific adjustments to the speech rate, tone, emotion, and professional pronunciation, so that the final synthesized voice can better fit the application scenario and improve the expressiveness and professionalism of voice synthesis. Compared with the traditional fixed model parameter voice synthesis method, the embodiment significantly enhances the flexibility and adaptability of the system, reduces the difficulty of user operation, and at the same time ensures the high quality of the synthesized voice, providing strong technical support for the wide application of voice synthesis technology.

[0050] In an optional embodiment, the target attribute set includes at least the following attributes: a first attribute, wherein the first attribute is used to adjust the playback speed of the generated voice information; a second attribute, wherein the second attribute is used to adjust the tone of the generated voice information; a third attribute, wherein the third attribute is used to adjust the emotion of the generated voice information; a fourth attribute, wherein the fourth attribute is used to adjust the pronunciation of the text; and a fifth attribute, wherein the fifth attribute is used to represent the domain information to which the text belongs.

[0051] Optionally, first, the voice synthesis system needs to define a target attribute set, which contains multiple attributes for fine-tuning various aspects of voice synthesis. Specifically, the set includes at least five core attributes:

[0052] The first attribute, for example, speed, is used to adjust the playback speed of the generated voice information, so that the voice can become slower or faster according to the scene requirements to adapt to the needs of different listener groups such as hearing-impaired people, children, or fast-paced news broadcasts.

[0053] The second attribute, such as pitch, is used to adjust the tone of the generated voice information, so that the voice can more vividly reflect the rising or falling trend of the text, enhancing the richness of expression.

[0054] A third attribute, such as emotion, is used to adjust the emotional color of the generated speech information, enabling the synthesized speech to convey the emotional state behind the text, such as excitement, sadness, warning, etc. This is an aspect that traditional speech synthesis systems often overlook or handle inadequately, and is crucial for improving the naturalness and immersion of speech synthesis.

[0055] A fourth attribute, such as phoneme or pronunciation, is used to adjust the pronunciation of specific words, which is particularly important for handling multi-syllable words, professional terms, or foreign words. Through this attribute, the system can avoid common pronunciation errors and improve the professionalism and accuracy of speech synthesis.

[0056] A fifth attribute, such as domain, is used to represent the domain information of the text, such as legal terminology, medical vocabulary, etc. This helps the system to call appropriate pronunciation rules and intonation styles according to different domain requirements, enhancing the domain adaptation ability of speech synthesis.

[0057] Optionally, once the target attribute set is defined, the user can apply the target tag set to the text to be processed, adding the above attributes and their specific values to specific speech segments. For example, <voice speed="1.5" pitch="0.9">Emergency notice: please evacuate immediately< / prosody> < / voice> or <prosody emotion="calm" domain="finance">Today's closing stock market situation...< / phoneme> < / voice> < / prosody> .

[0058] From the above, through the target attribute set, the embodiment significantly improves the controllability, semantic accuracy, emotional expression ability and domain adaptation level of speech synthesis. Users can intuitively add control marks in the text to guide the speech synthesis system to make fine voice adjustment, including adjusting the playback speed to adapt to different audiences, changing the pitch to reflect the ups and downs of the sentence, injecting emotional color to enhance the authenticity of expression, specifying accurate pronunciation to reduce the misunderstanding of homophones or professional terms, and automatically adapting the tone style according to the domain information to ensure that the speech synthesis result is both professional and personalized. This method greatly improves the customization ability and user experience of speech synthesis, making it have a wider application prospect in education, entertainment, customer service and professional fields, and can meet the speech synthesis needs in diversified scenarios. At the same time, it also provides developers with a more flexible integration method, promoting the deep integration of speech synthesis technology and various industry applications.

[0059] In an optional embodiment, the speech synthesis system first identifies the marks in the first text to obtain a first mark set, then extracts the attributes corresponding to each mark in the first mark set and the attribute values corresponding to the attributes to obtain a first attribute set, then determines the nesting relationship between the marks in the first mark set, and finally divides the first text into S subtexts according to the first mark set and the nesting relationship, and generates a corresponding control parameter list for each subtext based on the first mark set, the nesting relationship and the first attribute set.

[0060] Optionally, the speech synthesis system first reads the first text with extended marks, and identifies all control marks in the text through built-in mark recognition algorithms (such as regular expression matching). These marks constitute the first mark set, which covers multiple dimensions of speech control, such as speed, pitch, emotion, etc. The system further extracts the control attributes related to each mark and their specific attribute values. For example, the speed attribute and its value 1.2 are extracted from <voice speed="1.2" / >, and the emotion and pause attributes and their corresponding values sad and long are extracted from <prosody emotion="sad" pause="long" / >. These attributes and their values will constitute the first attribute set.

[0061] Optionally, the system analyzes the nesting level of each mark in the first text to identify which marks are under the control of higher-level marks. This step is very important because it determines the influence range of the control parameters. For example, <voice> ...< / voice> internal <prosody> ... <prosody>The marking will be subjected to the outer layer <voice>The global control of the tags while retaining the local control ability of each tag itself.

[0062] Optionally, the system segments the first text into S independent subtexts according to the first set of tags and the nesting structure among the tags, where S is any integer greater than or equal to 1. Each subtext is an independent unit of speech synthesis with its own set of control parameters. The system generates a control parameter list for each subtext, which lists all applicable tag attributes and their values. For example, for the subtext "This is a test.", if it is surrounded by <voice pitch="1.1" / > and <prosody emphasis="high">, then its control parameter list might contain "pitch=1.1" and "emphasis=high".

[0063] From the above, it can be seen that through the above specific implementation steps, the speech synthesis system can achieve fine parsing of the first text and effective extraction of control parameters, and then customize accurate control strategies for each subtext. This method not only enhances the system's understanding and processing ability of complex text structures, but also ensures that control parameters can accurately act on target subtexts, and will not cause control failure or poor effect due to the confusion of tag nesting. Compared with traditional single model control, this embodiment significantly improves the flexibility and accuracy of speech synthesis, so that the speech output can be closer to the user's expectation, whether it is adjusting the speech speed to adapt to a specific audience, or enhancing the realism and infectivity of speech expression through pitch and emotion tags, can be effectively realized. In addition, the parsing ability of nested relationships enables the system to cope with multi-level pragmatic control requirements, laying a solid foundation for realizing advanced emotional speech synthesis, accurate pronunciation of professional terms, and domain-specific intonation style. Overall, this series of steps greatly enhances the controllability and expressiveness of speech synthesis technology, broadens its boundaries in personalized services, education and training, media communication and professional field applications, and promotes the development of speech synthesis technology towards more intelligent and humanized direction.

[0064] In an optional embodiment, the speech synthesis system first divides the first text into S subtexts according to the position information and nesting relationship of each tag in the first set of tags, then determines the attribute corresponding to each subtext and the attribute value corresponding to the attribute from the first attribute set based on the nesting relationship and at least one tag corresponding to each subtext, and then generates a corresponding control parameter list for each subtext according to the attribute corresponding to each subtext and the attribute value corresponding to the attribute.

[0065] Optionally, the voice synthesis system first parses the first text and obtains the position information of each tag in the first tag set, including the start and end positions, and identifies the nesting relationship between the tags. For example, the system identifies that the <voicespeed="1.2"> tag covers the entire text segment, while the <prosody emphasis="high"> tag is located within a specific sentence in the text segment, which determines that the voice speed control acts on the whole, while the emphasis control only acts on the sentence.

[0066] Optionally, according to the nesting level and position information of the tags, the system divides the first text into S subtexts. Each subtext is an independent voice synthesis unit, and its control parameters are limited to the combination of the outermost tag and the internal nested tag. For example, if the text segment contains <voice>and <prosody>Two markers, then the system can divide the text into two parts, one of which contains <prosody>The text of the sentence is considered as one subtext, influenced by the specific attributes of the tag, while the remaining text is considered as another subtext, influenced only by the attributes of the <voice>The impact of the tags.

[0067] Optionally, for each subtext, the system extracts attributes and their attribute values from the first attribute set based on its corresponding tag set and nesting relationship. For example, for a subtext tagged as <prosody emotion="happy" pitch="1.1">, the system generates a control parameter list containing emotion="happy" and pitch="1.1". This process ensures that the synthesized speech for each subtext accurately reflects the user's control intentions set through the tags.

[0068] Optionally, finally, the system generates a control parameter list for each subtext, combining all applicable attribute values together. This list will be sent to the speech synthesis model along with the subtext, guiding the model to generate synthesized speech that meets the user's set parameters. Through the impact analysis between attributes, the system can ensure that even under complex tag structures, the synthesis process accurately reflects the independent control requirements of each subtext.

[0069] From the above, it can be seen that the implementation of the embodiment significantly improves the precision and efficiency of text control in the speech synthesis process. By accurately analyzing the position information and nesting relationship of the tags, the system can intelligently segment the text into multiple subtext units and generate customized control parameter lists for each subtext. This method avoids the excessive impact of global tags on local segments, allowing users to independently and meticulously control different parts of the text, further enhancing the flexibility and accuracy of speech synthesis in expressing emotions, adjusting speech speed, and controlling intonation.

[0070] Unlike traditional models that apply control parameters uniformly to the entire text passage, the embodiment can achieve personalized regulation of different sentences or phrases in the same text, ensuring that the synthesized speech is both coherent and accurately reflects the subtle changes and author's intentions of the text. This is particularly important for creating more natural and expressive synthesized speech, especially in multi-character dialogue, emotional reading, or professional field broadcasting applications, significantly improving the quality of speech synthesis and audience acceptance. In addition, this method simplifies the user interface, allowing even non-technical users to easily control speech synthesis through intuitive tag languages, greatly reducing the threshold for using speech synthesis technology and promoting its popularization and innovative applications in a wider range of fields.

[0071] In an optional embodiment, the voice synthesis system updates the control parameter list corresponding to each subtext based on a mapping rule, wherein the mapping rule includes a mapping relationship between each attribute in the control parameter list and a parameter in the target model, the mapping rule is used to map each attribute in the control parameter list corresponding to each subtext to a corresponding control vector, wherein the control vector is used to guide the target model to generate voice information, and then the to-be-processed text and the updated control parameter list corresponding to each subtext are input into the target model, and the target voice information for the to-be-processed text is output according to the target model.

[0072] Optionally, the mapping rule is a bridge connecting the user control parameters and the internal parameters of the target model. First, the system needs to define clear mapping rules, including but not limited to how to map the attributes in the control parameter list (such as speech rate, pitch, and emotional state) to the corresponding control vectors in the target model. For example, the speed attribute is mapped to the duration prediction parameter in the model, the pitch attribute is mapped to the fundamental frequency adjustment factor, and the emotion attribute is mapped to a specific emotion encoding or style vector.

[0073] Optionally, after the system parses the first text and generates the control parameter list, the mapping rule will be applied to these lists to update the control parameters corresponding to each subtext in real time. The updating process involves the quantization processing of attribute values to ensure that they can be effectively recognized and applied by the model. For example, the speed attribute value is converted from "1.2" in the text label to a duration prediction coefficient that the model can recognize, and "happy" in the emotion attribute is translated into an encoded happy emotion vector.

[0074] Optionally, through the mapping rule, each attribute is converted into its corresponding expression in the target model, i.e., a control vector. These control vectors can guide the model to adjust specific parameters when synthesizing voice, such as controlling the behavior of the model's rhythm prediction, pitch adjustment, or emotion encoder, so that it better meets the user's expression requirements. Finally, the system inputs the labeled to-be-processed text and the updated control parameter list into the target model. The model generates voice information consistent with the user's intention according to the content of the input text and the guidance of the control vector. This process combines the semantic content of the text and the user-specified voice characteristics, realizes the synchronous transmission of semantics and emotions, and enhances the personalization and customization capabilities of voice synthesis.

[0075] From the above, by updating the control parameter list of each subtext in real time and converting it into a control vector understandable by the target model, the system can respond to the needs of users or content creators in real time, ensuring that the synthesized speech not only follows the basic semantics of the text but also incorporates rich expression colors and rhythm changes, such as simulating different emotions like anger, joy, calmness, or adjusting speech speed and tone according to different scenarios. The key to this method is that it breaks the hard binding of control parameters and model behavior in traditional systems, achieving flexible conversion of control parameters to internal parameters of the model through dynamic mapping rules. This not only improves the response speed of the model to user feedback, but also enables the model to adapt to various complex voice control requirements, opening up more possibilities for the application of voice synthesis technology, especially in the fields of education, entertainment, customer service, and professional broadcasting, which can significantly improve user experience and promote the widespread application and deep integration of voice synthesis technology.

[0076] In addition, the dynamic and customizable nature of the mapping rules also provides greater innovation space for system developers, who can flexibly adjust the mapping logic according to the characteristics of different models and application scenarios, further optimizing the results of voice synthesis, ensuring that the synthesized speech achieves the best in terms of sound quality, expressiveness, and professionalism, thus promoting the development of voice synthesis technology towards a more intelligent, natural, and professional direction.

[0077] In an alternative embodiment, the voice synthesis system preprocesses the text to be processed to obtain a target text, wherein the preprocessing is used to convert the text to be processed into a unified format, then based on the fourth attribute and the fifth attribute in the control parameter list corresponding to each subtext, the target text is converted into a phoneme sequence, wherein the fourth attribute is used to adjust the pronunciation of the text, and the fifth attribute is used to represent the domain information to which the text belongs, then the encoding module of the target model is used to encode the phoneme sequence and the control vector in the control parameter list corresponding to each subtext respectively to obtain the target vector and the target control vector set corresponding to each subtext, the acoustic feature prediction module of the target model is used to predict the acoustic features of the output speech information for the text to be processed according to the target vector and the target control vector set corresponding to each subtext, and finally the vocoder of the target model is used to generate the target speech information for the text to be processed according to the acoustic features and the target control vector set corresponding to each subtext.

[0078] Optionally, the voice synthesis system first preprocesses the text to be processed, which aims to standardize the text format and ensure that all marks and attributes meet the specifications of internal processing. The preprocessing step may include removing any non-text characters other than HTML / XML tags, unifying the text encoding to a specified format, and preprocessing and encoding special characters or professional terms so that they can be correctly recognized and parsed by subsequent processing modules.

[0079] Optionally, then, the system performs phoneme conversion on the target text based on a fourth attribute (for adjusting pronunciation of the text) and a fifth attribute (for representing domain information to which the text belongs) in the control parameter list corresponding to each subtext. Specifically, the fourth attribute is, for example, zhong< / voice> < / prosody> < / prosody> < / voice> < / voice> < / prosody> < / prosody> < / phoneme> The fifth attribute, such as <domain type="finance"> financial terms, represents the field of the text, which helps the system to call the correct field dictionary or pronunciation rules. Through this conversion, the original text is decomposed into a phoneme sequence, preparing the input at the phoneme level for the subsequent speech synthesis process.

[0080] Optionally, the system encodes the phoneme sequence and the control vectors in the control parameter list corresponding to each subtext through the encoding module of the target model. The phoneme sequence is encoded into a vector representation that the model can understand, and at the same time, the control vectors are also encoded or converted into control signals inside the model, ensuring that the model can accurately integrate the user's voice control information into the speech synthesis process.

[0081] Optionally, the encoded phoneme sequence and control vector set are sent together to the acoustic feature prediction module of the target model. This module uses the encoded information to predict the acoustic features of the output speech information for the text to be processed, including pitch, duration, intensity, etc. The control vectors guide the model to adjust the prediction results at this stage, so that the acoustic features can reflect the pronunciation manner and field-specific intonation style specified in the fourth and fifth attributes, thereby achieving fine control of the synthesized speech.

[0082] Optionally, finally, through the vocoder module of the target model, based on the predicted acoustic features and the target control vector set corresponding to each subtext, the final target speech information for the text to be processed is generated. The vocoder converts the acoustic features into speech waveforms, and at the same time, through the guidance of the target control vector set, ensures that the synthesized speech of each subtext accurately reflects the user's requirements for pronunciation, speech rate, tone, and emotion, as well as the field-specific expression style.

[0083] From the above, it can be seen that the embodiment shows how to deeply integrate the user's specified control information into the speech synthesis process through preprocessing, phoneme conversion, and model-driven control vector application, thereby generating highly customized and specialized speech output. The main technical effects of this method are reflected in the following aspects: through the accurate control of the fourth attribute, the system can handle the pronunciation problems of multi-sound characters, professional terms, and foreign words, ensuring that the synthesized speech is more accurate and natural in pronunciation; the introduction of the fifth attribute enables the system to identify the field of the text and call the corresponding field dictionary or pronunciation rules, achieving the adaptation of the field-specific intonation, especially for professional fields such as medicine, law, and finance, which can significantly improve the professionalism and credibility of speech synthesis; through model-driven encoding and prediction, control parameters not only play a role in the phoneme sequence generation stage, but also continuously guide in the acoustic feature prediction and speech information generation stage, ensuring the user's full-range control over the speech synthesis process.

[0084] In an alternative embodiment, to solve the technical problems existing in the prior art voice synthesis system in terms of expression control, pronunciation accuracy, emotional expression and field adaptation, the embodiment proposes: by introducing a self-defined syntax markup language in the text to be synthesized, the voice synthesis process can receive and parse these marks, thereby realizing flexible control of multi-dimensional synthesis parameters such as speech rate, pitch, emotion, tone, multi-sound word pronunciation, professional term processing, etc. This method does not rely on major modifications to the existing voice synthesis model structure, but through four core steps of marking-parsing-mapping-control, it forms a loosely coupled integration with mainstream language synthesis systems, and has flexibility, scalability and universality, and can be widely applied to various voice synthesis scenarios. Specifically as follows:

[0085] (1) Self-defined extended markup syntax:

[0086] The markup language based on XML / HTML structure has the following characteristics: support for rich parameters such as speech rate, pitch, emotion, pronunciation, etc.; allow tag combination nesting and self-defined extension, and the structure is clear and easy to parse.

[0087] (2) Marking parsing and parameter extraction:

[0088] The system introduces a parsing module to extract control parameters from the text, generate structured configurations, and support complex markup combinations through regular expressions or DOM tree parsing techniques.

[0089] (3) Control parameter mapping and injection:

[0090] Map the parsed parameters to control vectors recognizable by the voice synthesis model: speed control duration prediction; pitch adjustment frequency; emotion injection emotion encoding; phoneme forced pronunciation; domain call domain dictionary.

[0091] (4) Integrated into the voice synthesis process:

[0092] It can be seamlessly embedded into mainstream voice synthesis frameworks: inserted into the text preprocessing or acoustic feature prediction stage; work with the post-processing module; pass control parameters through the interface.

[0093] From the above, the method proposed in the embodiment has made key innovations and systematic improvements on the basis of the original voice synthesis technology, especially in terms of voice expression controllability, pronunciation accuracy, emotional and tone expression ability, field adaptability, user operation convenience, etc. It has obvious beneficial effects and the following five major advantages over the prior art:

[0094] (1) Realize fine-grained control of the voice synthesis process:

[0095] The existing voice synthesis system mainly relies on a trained model to automatically infer expression parameters such as speech speed, tone, and emotion, and lacks a clear external control interface. The present embodiment achieves direct control of voice synthesis output behavior from the text input end by embedding structured extended markup language in the text. Users can specify speech speed adjustment, tone variation, pause control, stress position, and the like at the word, sentence, or even segment level, thereby achieving more flexible, natural, and realistic voice synthesis results. This method significantly improves the expressiveness of voice synthesis, making it applicable to scenarios such as dramatic readings, audio novels, personalized voice assistants, and other scenarios that require high expressiveness.

[0096] (2) Improved pronunciation accuracy of multi-syllable words and professional terms:

[0097] The problem of multi-syllable words in Chinese has always been one of the challenges that voice synthesis systems cannot fully solve, especially in texts with insufficient context such as news, classical Chinese, and poetry. The present embodiment supports explicit specification of multi-syllable word pronunciation through the use of special tags, allowing users to specify the pronunciation of multi-syllable words in a more flexible and accurate manner. <phoneme>or <pinyin>The mark manually specifies the pronunciation, and in combination with the mark analysis system, the model default behavior can be directly covered to ensure the accuracy of the pronunciation of the keywords. Especially in professional fields such as law, medicine, finance, etc., professional terms can be accurately marked and mapped to customized phonemes to realize standardized pronunciation of terms, thereby significantly improving the professionalism and credibility of the speech synthesis system.

[0098] (3) The performance ability of voice emotion and tone is enhanced:

[0099] Traditional speech synthesis systems are mainly based on neutral speech, and a few systems that introduce emotion models face the problems of large training data and uncontrollable emotional style. The embodiment directly guides the system to express specific semantic emotions such as joy, sadness, anger, encouragement, calmness, etc. through emotion and tone control marks, thereby enriching the performance level of speech synthesis. This mechanism can be used in combination with emotion embedding and style transfer modules to provide highly controllable and customizable voice emotional expression ability for content creators and voice broadcasters, and is particularly suitable for scenarios such as character dubbing, educational voice, intelligent customer service, etc. that have interactive and emotional needs.

[0100] (4) Good compatibility and extensibility, suitable for multiple speech synthesis frameworks:

[0101] The method proposed in the embodiment does not depend on a specific speech synthesis model structure, adopts a modular design, and can be flexibly embedded into mainstream speech synthesis architectures. The control parameters can be mapped as a supplement or replacement of the existing input of the model, and the model behavior is guided through a control vector, so it has high compatibility. At the same time, the mark syntax supports user-defined extension, which facilitates the addition of new control dimensions such as background sound control, character switching, dialect switching, etc. in different application scenarios, meeting the development trend of speech synthesis from "general" to "diversification" and "contextualization".

[0102] (5) Reduce the threshold for use and improve user interaction experience:

[0103] Unlike traditional systems that require modification of model code or configuration files, the embodiment allows users to directly embed marks in text in a HTML-like manner to control the speech synthesis process, greatly simplifying the operation process. Ordinary users do not need programming experience and can perform personalized operations such as speed adjustment and emotion control by mastering basic mark syntax. For content creators, educators or industry users, templates, tool plugins or visual editors can be provided for batch operations to improve speech synthesis efficiency, reduce technical barriers, and expand speech application scenarios.

[0104] In an alternative embodiment, in order to better realize flexible control and expression enhancement of the speech synthesis process, the embodiment provides a language generation method based on text extension marks, including the following core implementation details:

[0105] (1) Definition of Extended Markup Language Specification: First, a set of syntax specifications for the text of the language synthesis is made. The markup language has the following features:

[0106] Tag form: XML / HTML style, such as <voice> ...< / voice> , <phoneme> 、 <prosody>etc.

[0107] Attribute support: Each tag can be attached with multiple attributes (such as speed, pitch, emotion, etc.);

[0108] Nested support: Supports multi-layer nesting, allowing different dimensional parameters to take effect on the same text segment;

[0109] Strong scalability: Supports user-defined tags and attributes, facilitating future expansion.

[0110] Examples:

[0111] <voice speed="1.2" pitch="0.8">Welcome to the intelligent synthesis system< / prosody> < / phoneme> < / pinyin> < / phoneme> < / voice>

[0112] <phoneme pinyin="xíng">xing< / prosody> < / phoneme>

[0113] <prosody emotion="happy" emphasis="medium">Today is a good day!

[0114] (2) Text markup injection and input construction: Users can embed the above markup into the original text through a visual editor, a text template, or an automated tool to form an input text with control information. The system takes this input as the starting content of the language synthesis process.

[0115] Example:

[0116] <voice speed="1.0" pitch="1.1">

[0117] <prosody emotion="angry">Why are you late again?

[0118] < / voice>

[0119] (3) The tag analysis module parses the control information: the extended tag analysis module in the system is responsible for converting the tagged text into structured intermediate data. The parsing process includes: tag identification and attribute extraction; separation of text and control instructions; parsing and merging of multi-layer nested relationships; generating a control parameter table corresponding to each speech segment.

[0120] The parsing result is as follows (pseudo-json format data):

[0121]

[0122] (4) Control parameter mapping to TTS model input vector: the system maps the control parameters obtained by parsing into a form that can be recognized by the TTS model, for example:

[0123] speed is mapped to duration predictor;

[0124] pitch is mapped to control channel;

[0125] emotion is mapped to emotion embedding;

[0126] phoneme directly replaces the default phoneme prediction result;

[0127] domain controls model selection or domain dictionary invocation.

[0128] This mapping process can be completed by lookup table, neural network encoder or vector splicing.

[0129] (5) Interface integration with the speech synthesis system: This method does not rely on modifying the TTS main model structure, but only through intermediate parameter injection or auxiliary module control method, adapting to mainstream speech synthesis systems, which can be integrated in the following stages: pre-processing stage: for phoneme replacement, text regularization; coding stage: for inputting emotion, speech rate, etc. as additional vectors; acoustic feature generation stage: for adjusting rhythm, pitch features; vocoder stage: further strengthening timbre, style control.

[0130] From the above, the embodiment expands the markup language, constructs the text markup injection and input, parses the control information through the markup analysis module, maps the control parameters to the speech synthesis model input vector, and integrates with the speech synthesis system without invasion. Through the above implementation, the embodiment realizes fine-grained control of the speech synthesis process, improves the pronunciation accuracy of multi-sound words and professional terms, enhances the performance of speech emotion and tone, has good compatibility and expandability, adapts to various speech synthesis frameworks, reduces the use threshold, and improves the user interaction experience.

[0131] The embodiment of the application also provides a speech generation device. It should be noted that the speech generation device of the embodiment of the application can be used to execute the speech generation method provided by the embodiment of the application. The speech generation device provided by the embodiment of the application is introduced as follows.

[0132] According to the embodiment of the application, a device for implementing the above speech generation method is also provided, Figure 2 is a schematic diagram of an optional speech generation device according to the embodiment of the application, as Figure 2 shown, the device includes: an acquisition unit 201, configured to acquire a to-be-processed text; a conversion unit 202, configured to convert the to-be-processed text into a first text according to a target markup library, wherein the target markup library is a set of a plurality of predefined markups and attributes corresponding to each markup, wherein each markup is used to identify a speech segment in the to-be-processed text that needs to be processed, and the attribute is used to adjust the output characteristics of the speech information corresponding to the to-be-processed text; an analysis unit 203, configured to analyze the first text to obtain a control parameter list corresponding to S subtexts in the first text, wherein S is an integer greater than or equal to 1, and the control parameter list includes an attribute corresponding to each subtext and an attribute value corresponding to the attribute; a generation unit 204, configured to input the to-be-processed text and the control parameter list corresponding to the S subtexts into a target model, and generate target speech information for the to-be-processed text according to the target model, wherein the target model is a pre-trained model used to generate speech information meeting preset requirements, and the preset requirements are used to constrain the output characteristics of the target speech information.

[0133] Optionally, the voice generation apparatus further comprises a first setting unit, a second setting unit and a first determining unit. The first setting unit is configured to set a target mark set, wherein the target mark set comprises a plurality of predefined marks. The second setting unit is configured to set a target attribute set, wherein the attributes in the target attribute set are control parameters associated with the marks in the target mark set. The first determining unit is configured to determine a target mark library according to the target mark set and the target attribute set.

[0134] Optionally, the second setting unit comprises a first attribute, a second attribute, a third attribute, a fourth attribute and a fifth attribute. The first attribute is configured to adjust the playing speed of the generated voice information. The second attribute is configured to adjust the tone of the generated voice information. The third attribute is configured to adjust the emotion of the generated voice information. The fourth attribute is configured to adjust the pronunciation of the text. The fifth attribute is configured to represent the domain information to which the text belongs.

[0135] Optionally, the parsing unit 203 comprises a first identifying subunit, a first extracting subunit, a first determining subunit and a first generating subunit. The first identifying subunit is configured to identify the marks in the first text to obtain a first mark set. The first extracting subunit is configured to extract the attribute corresponding to each mark in the first mark set and the attribute value corresponding to the attribute to obtain a first attribute set. The first determining subunit is configured to determine the nesting relationship between the marks in the first mark set. The first generating subunit is configured to divide the first text into S subtexts according to the first mark set and the nesting relationship, and generate a corresponding control parameter list for each subtext based on the first mark set, the nesting relationship and the first attribute set.

[0136] Optionally, the first generating subunit comprises a first dividing module, a first determining module and a first generating module. The first dividing module is configured to divide the first text into S subtexts according to the position information of each mark in the first mark set and the nesting relationship. The first determining module is configured to determine the attribute corresponding to each subtext and the attribute value corresponding to the attribute from the first attribute set based on the nesting relationship and the at least one mark corresponding to each subtext. The first generating module is configured to generate a corresponding control parameter list for each subtext according to the attribute corresponding to each subtext and the attribute value corresponding to the attribute.

[0137] Optionally, the generating unit 204 comprises a first updating subunit and a second generating subunit. The first updating subunit is configured to update the control parameter list corresponding to each subtext based on a mapping rule. The mapping rule comprises a mapping relationship between each attribute in the control parameter list and a parameter in the target model. The mapping rule is used to map each attribute in the control parameter list corresponding to each subtext to a corresponding control vector. The control vector is used to guide the target model to generate the voice information. The second generating subunit is configured to input the to-be-processed text and the updated control parameter list corresponding to each subtext into the target model, and output the target voice information for the to-be-processed text according to the target model.

[0138] Optionally, the second generating subunit comprises a first processing module, a first conversion module, a first encoding module, a first output module, and a second generating module. The first processing module is configured to pre-process the to-be-processed text to obtain a target text. The pre-processing is used to convert the to-be-processed text into a unified format. The first conversion module is configured to convert the target text into a phoneme sequence based on a fourth attribute and a fifth attribute in the control parameter list corresponding to each subtext. The fourth attribute is used to adjust the pronunciation of the text, and the fifth attribute is used to represent the domain information to which the text belongs. The first encoding module is configured to encode the phoneme sequence and the control vector in the control parameter list corresponding to each subtext through an encoding module of the target model to obtain a target vector and a target control vector set corresponding to each subtext. The first output module is configured to predict the acoustic characteristics of the output voice information for the to-be-processed text according to the target vector and the target control vector set corresponding to each subtext through an acoustic characteristic prediction module of the target model. The second generating module is configured to generate the target voice information for the to-be-processed text according to the acoustic characteristics and the target control vector set corresponding to each subtext through a vocoder of the target model.

[0139] According to another aspect of the present application, a computer readable storage medium is also provided, which comprises a stored executable program. When the executable program is executed, the computer readable storage medium controls the device where the computer readable storage medium is located to perform the voice generation method described above.

[0140] According to another aspect of the present application, an electronic device is also provided, which comprises one or more processors and a memory. The memory is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the voice generation method described above.

[0141] The above sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0142] In the above-described embodiments of the present application, the description of each embodiment focuses on different aspects, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0143] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other ways. Among them, the above-described device embodiments are only schematic, for example, the division of the units can be a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or modules shown or discussed can be indirect coupling or communication connection through some interfaces, units or modules, which can be electrical or other forms.

[0144] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place or distributed on multiple units. Part or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.

[0145] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0146] The integrated unit, if realized in the form of software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part of the prior art that makes a contribution or the whole or part of the technical solutions can be embodied in the form of software product, which is stored in a storage medium and includes a plurality of instructions for making a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and various program code storage media.

[0147] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A speech generation method, characterized in that: include: Get the text to be processed; Converting the to-be-processed text into a first text according to a target tag library, wherein the target tag library is a collection of a plurality of predefined tags and attributes corresponding to each tag, wherein each tag is used to identify a segment to be processed in the to-be-processed text, and the attributes are used to adjust output characteristics of speech information corresponding to the to-be-processed text; Parsing the first text to obtain a control parameter list corresponding to S subtexts in the first text, where S is an integer greater than or equal to 1, and the control parameter list includes an attribute corresponding to each subtext and an attribute value corresponding to the attribute; The control parameter list corresponding to the text to be processed and the S sub-texts is input into the target model, and target speech information for the text to be processed is generated based on the target model, wherein the target model is a pre-trained model for generating speech information that meets preset requirements, wherein the preset requirements are used to constrain the output characteristics of the target speech information.

2. The speech generation method according to claim 1, wherein: The target marker library is obtained by the following steps: Setting a target tag set, wherein the target tag set includes a plurality of predefined tags; Setting a target attribute set, wherein the attributes in the target attribute set are control parameters associated with the tags in the target tag set; The target tag library is determined according to the target tag set and the target attribute set.

3. The speech generation method according to claim 2, wherein: The target attribute set includes at least the following attributes: a first attribute, wherein the first attribute is used to adjust the playback speed of the generated voice information; a second attribute, wherein the second attribute is used to adjust the pitch of the generated voice information; a third attribute, wherein the third attribute is used to adjust the emotion of the generated voice information; a fourth attribute, wherein the fourth attribute is used to adjust the pronunciation of the text; The fifth attribute is used to represent the domain information to which the text belongs.

4. The speech generation method according to claim 1, wherein: The first text is parsed to obtain a list of control parameters corresponding to the S subtexts in the first text, including: Identify tokens in the first text to obtain a first token set; Extracting an attribute corresponding to each tag in the first tag set and an attribute value corresponding to the attribute to obtain a first attribute set; Determining a nesting relationship between each tag in the first tag set; The first text is divided into the S subtexts according to the first tag set and the nesting relationship, and a corresponding control parameter list is generated for each subtext based on the first tag set, the nesting relationship and the first attribute set.

5. The speech generation method according to claim 4, characterized in that Dividing the first text into the S subtexts according to the first tag set and the nesting relationship, and generating a corresponding control parameter list for each subtext based on the first tag set, the nesting relationship, and the first attribute set, including: Dividing the first text into the S subtexts according to the position information of each tag in the first tag set and the nesting relationship; Based on the nested relationship and the at least one tag corresponding to each subtext, determining an attribute corresponding to the subtext and an attribute value corresponding to the attribute from the first attribute set; A corresponding control parameter list is generated for each subtext according to the attribute corresponding to each subtext and the attribute value corresponding to the attribute.

6. The speech generation method according to claim 5, characterized in that Inputting the control parameter list corresponding to the text to be processed and the S subtexts into a target model, and generating target speech information for the text to be processed according to the target model, including: updating the control parameter list corresponding to each subtext based on the mapping rule; The mapping rule includes a mapping relationship between each attribute in the control parameter list and a parameter in the target model, and the mapping rule is used to map each attribute in the control parameter list corresponding to each subtext to a corresponding control vector, wherein the control vector is used to guide the target model to generate speech information; The text to be processed and the updated control parameter list corresponding to each sub-text are input into the target model, and target speech information for the text to be processed is output according to the target model.

7. The speech generation method according to claim 6, characterized in that Inputting the text to be processed and the updated control parameter list corresponding to each subtext into the target model, and generating target speech information for the text to be processed according to the target model, including: Preprocessing the text to be processed to obtain a target text, wherein the preprocessing is used to convert the text to be processed into a unified format; Converting the target text into a phoneme sequence based on a fourth attribute and a fifth attribute in the control parameter list corresponding to each subtext, wherein the fourth attribute is used to adjust the pronunciation of the text and the fifth attribute is used to represent the domain information to which the text belongs; Encode the phoneme sequence and the control vectors in the control parameter list corresponding to each subtext respectively by the encoding module of the target model to obtain a target vector and a target control vector set corresponding to each subtext; Predicting, by the acoustic feature prediction module of the target model, acoustic features of the output speech information of the text to be processed based on the target vector and the target control vector set corresponding to each subtext; The target speech information for the text to be processed is generated according to the acoustic features and the target control vector set corresponding to each sub-text through the vocoder of the target model.

8. A speech generating device, characterized in that: include: Get the unit and get the text to be processed; a conversion unit, configured to convert the to-be-processed text into a first text according to a target tag library, wherein the target tag library is a collection of a plurality of predefined tags and attributes corresponding to each tag, wherein each tag is used to identify a segment to be processed in the to-be-processed text, and the attributes are used to adjust output characteristics of speech information corresponding to the to-be-processed text; a parsing unit, configured to parse the first text to obtain a control parameter list corresponding to S subtexts in the first text, where S is an integer greater than or equal to 1, and the control parameter list includes an attribute corresponding to each subtext and an attribute value corresponding to the attribute; A generation unit is used to input the control parameter list corresponding to the text to be processed and the S sub-texts into a target model, and generate target speech information for the text to be processed based on the target model, wherein the target model is a pre-trained model for generating speech information that meets preset requirements, wherein the preset requirements are used to constrain the output characteristics of the target speech information.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located executes the speech generation method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the speech generation method described in any one of claims 1 to 7.

Images