Live broadcast verbal skill generation method and system supporting voice broadcast of multiple virtual characters

Abstract

The invention provides a live broadcast verbal skill generation method and system supporting voice broadcast of multiple virtual characters, and aims to improve interactivity and interestingness of live broadcast contents. The method is implemented through the following steps that firstly, a grammatical structure containing a start identifier, role identifiers, a content identifier and an end identifier is designed, and a user can define multi-role dialogue content through the identifiers; secondly, the system analyzes the verbal skill text, generates a syntax tree structure, and configures personalized voice features, such as timbre and intonation, for each role; secondly, converting the verbal skill content into an independent audio file by using a text-to-speech (TTS) technology, and supporting parallel processing to improve the efficiency; and finally, the system merges the generated audio files according to a preset sequence to generate a natural and smooth complete audio. The system ensures that the user can quickly obtain the generated multi-role voice file, meets the requirements of real-time performance and high efficiency, and is suitable for a live broadcast platform and other scenes needing multi-role voice interaction. Compared with the prior art, the method and the device have remarkable advantages in diversity and efficiency of voice generation and user experience.

Description

Technical Field

[0001] The present invention relates to the technical field of live broadcast systems, and in particular to a method and system for generating audio media by setting multiple virtual characters to broadcast live speech on a client or a server. Background Art

[0002] At present, with the help of AI voice generation technology, the technology of converting text live broadcasting into voice media and using it in the live broadcast room has become relatively mature. These AI-generated voices are mainly used to replace real anchors to realize automated content broadcasting. However, existing technical solutions usually only support the voice broadcasting of a single virtual character, resulting in insufficient interactivity and fun in the live broadcast process, and a relatively single format.

[0003] In live broadcasts, the use of multi-person voice broadcasts has gradually become a trend. The interaction of multiple characters can enhance the audience's sense of participation and the live broadcast. However, existing AI technology has not yet been able to effectively implement the voice broadcast of multiple virtual characters. How to use AI technology to generate the voices of multiple virtual characters in live broadcast scenarios and achieve natural and coherent multi-person broadcasts in the same speech has become an urgent problem to be solved. Summary of the invention

[0004] The present invention provides a live broadcast script generation method and system that supports multiple virtual character voice broadcasts, aiming to solve the problem of monotonous broadcast form and lack of interactivity of a single virtual character in the prior art. The present invention realizes the voice interaction and broadcast of multiple virtual characters in the live broadcast scene through innovative speech grammar structure and multi-character voice configuration, significantly improving the vividness of the live broadcast content.

[0005] The present invention mainly includes the following innovative contents: Multi-role dialogue setting : The present invention designs a grammatical structure for generating multi-role live dialogue content. The grammatical structure includes a start identifier (StartMarker), a role identifier (RoleID), a content identifier (Content) and an end identifier (EndMarker). Users can define the dialogue content of multiple virtual characters in the text through these identifiers. The lines of each character are embedded between the corresponding identifiers to generate a structured text of a multi-role dialogue. Through this grammatical structure, users can easily set the dialogue content of different characters, making the interaction between multiple characters more natural and flexible.

[0006] Syntax parsing and voice configuration: The system parses the text of the speech input by the user, generates the corresponding syntax tree structure, identifies and splits the default paragraph and the paragraph content of each virtual character. Then, based on the identifier of each character, the system prompts the user to select the corresponding voice features for different characters, including timbre, speaking speed, intonation, etc. The voice features of each virtual character can be personalized according to the needs of the user, ensuring that the voice performance of different characters in the audio is significantly different, enhancing the distinction and diversity between the characters.

[0007] Text-to-speech (TTS) generation : In the process of generating speech, the system uses text-to-speech (TTS) technology to convert the parsed multi-character dialogue content into independent audio files. The system can process the audio generation tasks of multiple characters in parallel, which improves the efficiency of audio file generation. This step ensures that each character's lines can be presented in the form of high-quality speech, and that the voice characteristics of each character can accurately reflect the user's settings.

[0008] Audio Merge and Output : After all the audio files of the characters are generated, the system will merge them in the preset grammatical structure order to generate a complete audio output file. During the merging process, the system will automatically adjust the connection between the audio files to ensure the coherence and natural fluency of the character dialogue. The generated merged audio file can be directly used in the live broadcast scene. Users can choose to play it in the live broadcast room or save it for later use.

[0009] Instant feedback and efficient application : The system architecture design of the present invention ensures that users can quickly obtain the generated multi-role voice file after entering the speech text. After the text submitted by the user is processed by the system, the corresponding audio feedback will be obtained in a short time, which meets the requirements of the live broadcast scene for real-time and efficiency. In addition, the generated audio file can be directly applied to various live broadcast platforms, or used in other scenarios that require multi-role voice interaction.

[0010] Through the above innovative steps, the present invention realizes the automated processing flow from raw text to high-quality multi-role voice broadcast, significantly improving the interactivity and fun of live broadcast content. The present invention is not only applicable to live broadcast scenarios, but can also be widely used in fields such as intelligent customer service and virtual assistants that require multi-role voice broadcast. Compared with the prior art, the present invention has significant advantages in the diversity, efficiency and user experience of voice generation, and has high technical advancement and application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to fully explain the technical details of the embodiments of the present invention and the technical effects achieved, the technical solutions in the embodiments of the present invention will be described clearly and in detail in conjunction with the accompanying drawings. These drawings are intended to intuitively show the structure, construction and operation process of the present invention so that readers can deeply understand the innovations and technical features of the present invention. Please note that the drawings are only schematic representations and are not intended to limit the present invention.

[0012] Figure 1 is an example diagram schematically showing a system for generating speech sounds according to an embodiment of the present disclosure;

[0013] Figure 2 is an example diagram schematically showing a syntax structure tree according to an embodiment of the present disclosure;

[0014] Figure 3 is a flowchart schematically illustrating an information processing method according to an embodiment of the present disclosure;

[0015] Figure 4 FIG. 4 is a flowchart schematically showing an information processing method according to another embodiment of the present disclosure. DETAILED DESCRIPTION

[0016] The present invention provides a method and system for generating live broadcast scripts that supports multiple virtual character voice broadcasts. The following describes in detail the implementation of the system, covering the entire process from script text input to generating a complete audio file.

[0017] Text input and analysis S101: The user submits a live broadcast script containing multiple virtual character lines through a client device (such as a PC or mobile terminal). The text adopts a predetermined grammatical structure, including a start identifier (StartMarker), a role identifier (RoleID), a content identifier (Content), and an end identifier (EndMarker). For example, the text input by the user is: This promotion is 20 yuan per order. Do you think it is cheap? {#1 Very cheap.} Yes. {#2 Free shipping, really worth it.} Hurry up and place your order.

[0018] S102: The system receives and parses the text submitted by the user. Figure 2 The syntax tree structure shown in the figure splits the text into different paragraphs and generates a syntax tree. Each paragraph consists of four child nodes: StartMarker: Marks the start of a paragraph.

[0019] RoleID: Identifies the role of the current paragraph, such as "#1" or "#2".

[0020] Content: The actual dialogue content of the character.

[0021] EndMarker: Marks the end of a paragraph.

[0022] By parsing these identifiers, the system forms a structured syntax tree for subsequent processing.

[0023] Character voice feature configuration S103: The system prompts the user to select corresponding voice features for each virtual character based on the character information in the parsed syntax tree. The user can set parameters such as timbre, speaking speed, and intonation for each character to ensure that the voice performance of each character is significantly different. This step is done by Figure 3 The "audio generation module" shown is implemented, and the sound features selected by the user will be used for subsequent text-to-speech processing.

[0024] Text-to-speech (TTS) processing S104: The system performs text-to-speech (TTS) processing on the lines of each character based on the voice features of the character configured by the user, and generates an independent audio file. Figure 4 The "audio generation module" shown in the figure is used to improve efficiency through parallel processing technology. Each character's lines are converted into audio files and saved as independent files.

[0025] Audio Merge and Output S105: After generating the audio files of all the characters, the system merges these audio files in the order defined in the syntax tree. The system automatically adjusts the connection between the audio files to ensure the coherence and natural fluency of the character dialogues. Figure 3 As shown in the figure, the complete audio file generated after completion can be used directly for live broadcast or saved for later use. The system merges the audio files into a whole output file according to the order of the characters in the text.

[0026] Real-time feedback and system optimization S106: The system design ensures that users can quickly obtain the generated multi-role voice files after submitting the script. The system ensures the real-time and efficient audio generation through optimized processing flow. Figure 4 In the process shown, after the data submitted by the user is parsed and processed, the generated audio file will be fed back to the user in a short time. This design not only meets the real-time requirements of the live broadcast scene, but also ensures the high quality and personalized requirements of the generated audio files in other application scenarios such as intelligent customer service and virtual assistants.

[0027] This implementation method covers the entire process from user input of multi-role speech text to the generation of high-quality audio files through detailed steps. The system realizes the automated processing of multi-role voice broadcasts by parsing text, configuring voice features, performing text-to-speech processing, merging audio files, and providing real-time feedback. The design of the system significantly improves the interactivity and fun of live broadcast content, while ensuring the efficient generation and high-quality output of audio files.

[0028] Figure 2 The grammar tree structure of a live broadcast text parsing process that supports multiple virtual character voice broadcasts is shown. This structure is used to split complex text content into multiple grammar paragraphs so that the corresponding audio files can be generated later. The following is a detailed description of the grammar structure:

[0029] Talk text node Definition: The dialogue text node is the root node of the syntax tree structure, which contains all the lines in the entire text.

[0030] Function: This node is used as the starting point for text parsing. The system obtains and processes the entire speech text through this node. The content in the root node includes the lines and dialogues of all characters. Whether it is the lines of the default character or the specified character, they are all summarized and processed in this node.

[0031] Example: This promotion is 20 yuan per order. Do you think it is cheap? {#1 Very cheap.} Yes. {#2 Free shipping, really worth it.} Hurry up and place your order.

[0032] Default Role Paragraph Definition: Default role paragraphs are lines that do not explicitly specify a role. These lines appear directly in the syntax tree without additional role identifiers.

[0033] Function: The lines in the default role paragraphs will be read by the system's default role. In the syntax tree structure, these paragraphs directly display the line text and are processed as the default role's voice during the audio generation process.

[0034] Example: This promotion is 20 yuan per order. Do you think it is cheap? Yes. Hurry up and place your order.

[0035] Processing: The system incorporates these paragraphs into the audio generation process, without the need for additional parsing of character information, and directly converts the lines into audio.

[0036] Other role paragraphs Definition: Other character paragraphs are those lines that clearly specify the virtual characters. The system decomposes these paragraphs into multiple sub-nodes according to specific grammatical rules.

[0037] Subnode Description: StartMarker: Used to mark the beginning of a paragraph, usually with curly brackets "{". The presence of a start marker indicates the starting point of the paragraph.

[0038] RoleID: This is used to identify which virtual character the current paragraph belongs to, such as "#1" or "#2". This identifier specifies the speaker of the line, ensuring that the lines of different characters can be correctly identified and processed.

[0039] Content: Indicates the actual lines of the characters. For example, Character 1's lines are "It's very cheap.", and Character 2's lines are "It's free shipping, what a bargain.".

[0040] EndMarker: It is used to mark the end of a paragraph, usually with curly brackets "}". The existence of the end marker indicates the end point of the paragraph, and the content after the end marker does not belong to the current character's lines.

[0041] Example: {#1It’s very cheap.} {#2It’s free shipping, what a bargain.} Processing: The system parses the child nodes of each character paragraph and converts them into independent dialogue content. The speaker of the dialogue is determined by the character identifier and the content is converted into an audio file for the corresponding character.

[0042] Application of syntax tree structure pass Figure 2 The syntax tree structure shown in the figure enables the system to flexibly parse and process the lines of each virtual character. The syntax tree structure ensures the organization and clarity of the text content, so that the lines of each character can be accurately recognized and converted into audio files.

[0043] Finally, the system merges all parsed audio files in the order defined by the syntax tree to generate a complete speech output file. This process ensures the naturalness and interactivity of the character dialogue, allowing natural dialogue and interaction between multiple virtual characters during the live broadcast.

[0044] Through the definition and processing steps of the above-mentioned grammatical structure, the present invention realizes efficient and accurate multi-role voice broadcasting, and provides technical support for the richness and interactivity of live broadcast content.

[0045] Figure 3A process for converting speech text into speech files is shown. The following are the detailed steps of the process: S301: Obtain the complete speech text entered by the user. The system receives the text data containing the complete speech content entered by the user. The user can manually enter the text through the client interface, or import the content from an existing text file. The purpose is to ensure that the system obtains the complete speech text that needs to be processed and provide basic data for subsequent processing steps.

[0046] S302: Analyze the text and build a syntax tree. The system parses the acquired text and generates a corresponding syntax tree structure according to the preset syntax rules. During the parsing process, the system identifies the role identifiers, lines, and tag information in the text to build a structured syntax tree so that the system can effectively process and separate different paragraphs and roles in the text.

[0047] S303: For each virtual character in the text, prompt the user to select the corresponding voice feature. Based on the parsed syntax tree information, the system prompts the user to select the appropriate voice feature for each virtual character. The user can select parameters such as timbre, speaking speed, and intonation to match the voice characteristics of each virtual character. Ensure that the voice features of each character are personalized so that the voices of different characters can be clearly distinguished in the generated audio file.

[0048] S304: Get the syntax tree and voice features of the speech text. The system combines the previously generated syntax tree and the voice features selected by the user to prepare for the subsequent speech synthesis work. This step merges the parsed syntax tree with the set voice features as the basic data for speech generation. Ensure that the speech synthesis process can generate high-quality audio files based on the structured text and the specified voice features.

[0049] S305: Analyze the number of paragraphs and start processing from the first paragraph. The system identifies the number of paragraphs in the script text and starts processing from the first paragraph. This step splits the text into paragraphs so that it can be processed and generated into audio files one by one. Each paragraph is processed sequentially to ensure that the content of each character's lines can be independently generated and accurately synthesized.

[0050] S306: Obtain the role, content, and voice features selected by the user for the paragraph. Based on the information in the syntax tree, extract the role identifier, line content, and voice features selected by the user for the role of the currently processed paragraph. Provide accurate role information and personalized sound features for speech synthesis of each paragraph to ensure that the generated audio file meets expectations.

[0051] S307: Generate the speech of the paragraph and save it as an independent audio file. Use text-to-speech (TTS) technology to convert the extracted paragraph content into an audio file. The audio file generated for each paragraph is saved independently to ensure that the lines of each character are clear and discernible. Convert the text content into a high-quality audio file to prepare for the subsequent audio merging step.

[0052] S308: Determine whether all paragraphs have been processed. The system checks whether all paragraphs have been processed. If not, return to step S306 to continue processing the next paragraph; if all paragraphs have been processed, proceed to the next step. Ensure that the audio file of each paragraph has been generated and saved to ensure the integrity and accuracy of the final audio output.

[0053] S309: Process the next paragraph. Continue processing the paragraphs that have not yet generated audio files. Repeat steps S306 to S308 until the audio files of all paragraphs are generated. Make sure to process all paragraphs and generate all the required audio files to prepare for the merging step.

[0054] S310: Arrange the audio files of each paragraph in the order set by the syntax tree. The system arranges all generated independent audio files according to the order defined in the syntax tree. According to the order of the character dialogues in the text, ensure that the order of the audio files is consistent with the script text. Provide the correct audio file order for the final audio merging step to ensure the coherence and logic of the dialogue content.

[0055] S311: Combine the audio files of each paragraph into a complete voice file in the arranged order. The system combines the arranged audio files in order to generate a complete voice output file. The file will contain the lines of all characters and will be played in the preset order. Provide a complete, multi-character interactive audio file for live broadcast or other application scenarios to ensure the continuity and integrity of the audio.

[0056] Through the above processing flow, the system can efficiently convert the speech text into multi-role voice files that meet user needs and meet the audio generation requirements of various application scenarios.

[0057] process Figure 4 The following is a description of the workflow of a speech audio generation system: S401: User submits raw data. The user submits raw data including the speech text, grammar tree structure and audio features corresponding to the role to the system entrance through a client device (such as a mobile client or PC). This step ensures the integrity and diversity of the input data and provides a sufficient basis for subsequent processing. It is used to collect all necessary data input by the user, including text content, grammar structure and audio features of the role, to ensure that subsequent processing can be based on comprehensive data.

[0058] S402: Import and parse data. After receiving the raw data submitted by the user, the system automatically imports it into the speech information processing module. This module is responsible for parsing the input data, extracting key information such as speech text, grammatical structure and character audio features, so as to prepare for subsequent audio synthesis. The purpose is to convert the raw data into structured information that the system can process, extract the necessary data for generating audio files, and prepare for the audio generation module.

[0059] S403: Generate high-quality speech audio files. After the data is parsed, the speech information is passed to the audio generation module. In this module, the system uses advanced audio synthesis technology to generate corresponding high-quality speech audio files based on the parsed text, syntax tree structure and audio features. The purpose is to use audio synthesis technology to convert the parsed text and character audio features into audio files to ensure that the generated audio files meet high quality standards.

[0060] S404: Return the audio file to the user. After being processed, the generated audio file is returned to the user through the output port of the system. This step ensures the rapid feedback and efficient use of the audio file, and can meet the user's demand for customized and high-quality audio files. The purpose is to provide the generated audio file to the user, ensuring that the audio file can be delivered to the user quickly and accurately to meet his needs.

[0061] Through the above steps, the present invention establishes a complete process from user submission of raw data to generation of high-quality speech audio files. This process not only improves the efficiency of audio file generation, but also ensures that the final generated audio file meets the personalized needs of users, and has significant technical progress and practical application value.

Claims

1. A live broadcast script generation system supporting multiple virtual character voice broadcasts, characterized in that: include: A speech input module, used to receive speech texts containing multiple virtual characters input by a user, wherein the speech texts follow a specific grammatical structure and contain role identifiers and content; A syntax parsing module, connected to the speech input module, is used to parse the input speech text, generate a corresponding syntax tree structure, and identify the default paragraph and the role paragraph; A character voice configuration module, connected to the grammar parsing module, configured to configure corresponding voice features for each virtual character according to the parsed character paragraphs; A speech generation module, connected to the character speech configuration module, for generating independent audio files for each virtual character according to the grammar tree structure and the configured speech features; An audio merging module, connected to the speech generating module, for merging the independent audio files of multiple virtual characters into a complete audio file according to the order set by the grammar; The audio output module is connected to the audio merging module and is used to output the merged audio file to the user or the live broadcast platform.

2. The system according to claim 1, wherein: The speech input module includes a client device, which allows a user to input speech text containing a role identifier through a mobile phone application or a computer terminal.

3. The system according to claim 1, wherein: The syntax parsing module can identify the start identifier and the end identifier in the speech text, and parse the corresponding content into nodes in the syntax tree structure.

4. The system according to claim 1, wherein: The character voice configuration module further includes a timbre selection unit for assigning different voice features to different virtual characters to enhance the distinction and personalization between the characters.

5. The system according to claim 1, wherein: The speech generation module adopts text-to-speech (TTS) technology to generate high-quality virtual character audio files based on the speech features in the character speech configuration module.

6. The system according to claim 1, wherein: The audio merging module can ensure that the dialogues between virtual characters in the merged audio file are natural and coherent through timeline control and voice feature matching.

7. The system according to claim 1, wherein: The audio output module further includes a streaming media interface for directly transmitting the merged audio file to a platform required by the user for real-time playback.

Images