Text-to-speech real-time streaming method, system, device, medium and program product

Abstract

This invention discloses a method, system, device, medium, and program product for real-time text-to-speech streaming conversion. The method includes: first, receiving a user input request, and then generating a text content block based on the user request; identifying the type of the text content block by monitoring the thinking state, and writing it into the corresponding inference queue and response queue. After the inference content in the inference queue meets a first preset condition, the inference content is synthesized using a first preset speech synthesis parameter, and the inference content and inference audio block are streamed using a unified data structure. When the response queue is first written with response content, the system switches to the response stage, and the response content is synthesized using a second preset speech synthesis parameter; after all the remaining inference content in the inference queue is output and cleared, the response content and response audio block are streamed using a unified data structure. This invention significantly reduces the initial packet latency between the response text and audio, and effectively improves the smoothness of switching between inference and response content.

Description

Technical Field

[0001] This invention relates to the field of large model technology, and in particular to a method, system, device, medium and program product for real-time text-to-speech streaming conversion. Background Technology

[0002] Existing AI voice dialogue systems mostly adopt the mode of generating text first and then calling the entire text-to-speech (TTS) function, which has the following drawbacks:

[0003] 1. High latency for the first packet, resulting in long waiting times for users;

[0004] 2. The speech performance of the reasoning process (REASONING) and the final answer (TEXT) cannot be distinguished, making it difficult for users to understand the AI's thinking process;

[0005] 3. The interruption function (Stop / Cancel) only applies to the text side. On the audio side, there is often a delay in the ending sound or the audio is out of sync with the interface.

[0006] 4. When accessing from multiple distributed terminals (web pages, apps, mini programs, etc.), audio needs to be repeatedly fetched and spliced, lacking a unified streaming distribution mechanism;

[0007] 5. Buffering strategies are mostly fixed block sizes or simple time slices, which cannot simultaneously take into account the continuity of thinking content and the rapid initial response of answer content.

[0008] Therefore, there is an urgent need to invent a real-time text-to-speech streaming method to solve the problems of high first-packet latency and easy confusion between reasoning and response processes in existing technologies. Summary of the Invention

[0009] In view of this, embodiments of the present invention provide a method, system, device, medium, and program product for real-time text-to-speech streaming conversion, which at least partially solves the problems existing in the prior art.

[0010] Other features and advantages of the invention will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0011] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0012] According to a first aspect of the present invention, a method for real-time text-to-speech streaming conversion is provided, the method comprising:

[0013] Requests to receive user input;

[0014] Generate text content blocks based on user requests;

[0015] The text content block type is identified by listening to the thinking state and written into the corresponding content queue, which includes a reasoning queue and a response queue.

[0016] After the reasoning content in the reasoning queue meets the first preset condition, the reasoning content is synthesized using the first preset speech synthesis parameters to obtain a reasoning audio block, and the reasoning content and the reasoning audio block are streamed out using a unified data structure.

[0017] After the answer queue is written with answer content for the first time, the system switches to the answer stage and uses the second preset speech synthesis parameters to synthesize the answer content to obtain an answer audio block. After all the remaining inference content in the inference queue is output and cleared, the answer content and the answer audio block are streamed using the unified data structure.

[0018] Furthermore, the unified data structure is used to simultaneously carry text segmented streams and audio segmented streams;

[0019] The data types of the unified data structure include text segments, audio segments, audio segment start signals, and audio segment end signals.

[0020] The audio segment start signal is used to trigger a local player warm-start and pre-buffering before responding to an audio block;

[0021] The audio segment end signal is used to determine the end point of audio playback and trigger resource reclamation.

[0022] Further, after the reasoning content in the reasoning queue satisfies the first preset condition, the reasoning content is synthesized using the first preset speech synthesis parameters, including:

[0023] Determine whether the length of the reasoning content in the reasoning queue is less than a preset length threshold;

[0024] If the length of the reasoning content in the reasoning queue is less than a preset length threshold, then continue to wait for the reasoning content in the reasoning queue to be aggregated;

[0025] If the length of the inference content in the inference queue is greater than or equal to a preset length threshold, then the inference content in the inference queue is synthesized using the first preset speech synthesis parameters to obtain an inference audio block.

[0026] Furthermore, after the response queue is first written with response content, the system switches to the response phase and uses the second preset speech synthesis parameters to synthesize the response content to obtain a response audio block, including:

[0027] Once the answer queue is written with answer content for the first time, the system switches to the answering phase.

[0028] Determine whether the length of the answer content in the answer queue meets the second preset condition;

[0029] If the answer content in the answer queue meets the second preset condition, then continue to wait for the answer content in the answer queue to be aggregated;

[0030] If the answer content in the answer queue does not meet the second preset condition, the answer content in the answer queue is synthesized using the second preset speech synthesis parameters to obtain an answer audio block.

[0031] Furthermore, after all residual reasoning content in the reasoning queue has been output and cleared, the answer content and the answer audio block are streamed using the unified data structure, including:

[0032] After switching to the response phase, the inference queue is flushed.

[0033] The residual inference content in the inference queue is synthesized using the first preset speech synthesis parameters to obtain a residual inference audio block. The residual inference content and the residual inference audio block are then streamed using a unified data structure, and the inference queue is cleared.

[0034] After the inference queue is cleared, the answer content and the answer audio block will be streamed using the unified data structure.

[0035] Furthermore, the method also includes:

[0036] Generate a distributed audio stream associated with the session using audio blocks;

[0037] Each terminal subscribes to the distributed audio stream to obtain and play audio segments. The terminals include web pages, apps, and mini-programs.

[0038] The distributed audio stream supports synchronous playback and catch-up playback on various terminals.

[0039] According to a second aspect of the present invention, a real-time text-to-speech streaming system is provided, the system comprising: an LLM output layer, a state-aware layer, a TTS scheduling layer, a buffer manager, and a distributed publish-subscribe module;

[0040] The LLM output layer is used to receive user input requests and generate text content blocks based on the user requests;

[0041] The state awareness layer is used to identify the type of text content block by listening to the thinking state and write it into the corresponding content queue, which includes an inference queue and an answer queue.

[0042] The TTS scheduling layer and the buffer manager are used to synthesize the inference content in the inference queue using first preset speech synthesis parameters after the inference content in the inference queue meets the first preset condition, to obtain an inference audio block, and to stream the inference content and the inference audio block using a unified data structure; when the response queue is first written with response content, the system switches to the response stage, and synthesizes the response content using second preset speech synthesis parameters to obtain a response audio block; after all the remaining inference content in the inference queue is output and cleared, the response content and the response audio block are streamed using the unified data structure;

[0043] The distributed publish-subscribe module is used to generate a distributed audio stream associated with a session using audio blocks. Each terminal subscribes to the distributed audio stream to obtain and play audio segments.

[0044] According to a third aspect of the present invention, a real-time text-to-speech streaming device is provided, the device comprising: a processor and a memory;

[0045] The memory is used to store one or more program instructions;

[0046] The processor is configured to run one or more program instructions to perform the steps of a real-time text-to-speech streaming method as described in any of the preceding claims.

[0047] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, wherein when executed by a processor, the computer program implements the steps of a real-time text-to-speech streaming method as described in any of the preceding claims.

[0048] According to a fifth aspect of the present invention, a computer program product is provided, the computer program product comprising a computing program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to implement the steps of a real-time text-to-speech streaming method as described in any of the preceding claims.

[0049] This invention discloses a method, system, device, medium, and program product for real-time text-to-speech streaming conversion. The method includes: first, receiving a user input request, and then generating a text content block based on the user request; identifying the type of the text content block by monitoring the thinking state, and writing it into the corresponding inference queue and response queue. After the inference content in the inference queue meets a first preset condition, the inference content is synthesized using a first preset speech synthesis parameter, and the inference content and inference audio block are streamed using a unified data structure. When the response queue is first written with response content, the system switches to the response stage, and the response content is synthesized using a second preset speech synthesis parameter; after all the remaining inference content in the inference queue is output and cleared, the response content and response audio block are streamed using a unified data structure. This invention significantly reduces the initial packet latency between the response text and audio, and effectively improves the smoothness of switching between inference and response content. Attached Figure Description

[0050] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0051] Figure 1 A flowchart illustrating a real-time text-to-speech streaming conversion method provided in an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of the architecture of a real-time text-to-speech streaming conversion system provided in an embodiment of the present invention;

[0053] Figure 3 A schematic diagram illustrating the principle and flow of a real-time text-to-speech streaming conversion method provided in an embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram illustrating the multi-terminal interaction principle of a distributed audio stream in a real-time text-to-speech streaming conversion method provided in an embodiment of the present invention. Detailed Implementation

[0055] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0056] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0057] Figure 1 A flowchart of a real-time text-to-speech streaming method according to an embodiment of the present invention is shown.

[0058] like Figure 1 As shown, the real-time text-to-speech streaming conversion method according to an embodiment of the present invention may include steps S100, S200, S300, S401 and S402.

[0059] Specifically, taking the scenario of a user's request "Please check the progress of one of my orders" as an example, we can illustrate the complete chain from receiving the request to responding.

[0060] In step S100, a request input by the user is received.

[0061] Specifically, the above steps include:

[0062] Upon receiving user input requests, the client sends a message in a unified format, which serves as the entry point for subsequent state awareness and message fragmentation.

[0063] Next, in step S200, a text content block is generated based on the user request.

[0064] Specifically, the above steps include:

[0065] The LLM server retrieves recent session summaries and relevant data (order status, latest logistics milestones, etc.) based on the user's request "Please check the progress of one of my orders," thus forming the inference context.

[0066] Then, in step S300, the text content block type is identified by listening to the thinking state and written into the corresponding content queue, which includes a reasoning queue and a response queue.

[0067] Figure 3 The diagram illustrates the principle and flow of a real-time text-to-speech streaming conversion method provided by an embodiment of the present invention.

[0068] Specifically, the above steps include:

[0069] By listening to thinking_started and thinking_completed in the generation pipeline, the text content block type is identified and written to the corresponding content queue, which includes the reasoning queue and the answer queue.

[0070] This invention uses BufferManager to maintain two logical content queues (ReasoningQueue and AnswerQueue) to achieve adaptive buffering. The parameters of the content queues dynamically include the maximum splicing time, the minimum chunk byte threshold, and the real-time callback trigger threshold.

[0071] In step S401, after the reasoning content in the reasoning queue meets the first preset condition, the reasoning content is synthesized using the first preset speech synthesis parameters to obtain the reasoning audio block, and the reasoning content and the reasoning audio block are streamed out using a unified data structure.

[0072] Specifically, the above steps include:

[0073] After entering the thinking phase, determine whether the length of the reasoning content in the reasoning queue is less than the preset length threshold;

[0074] If the length of the reasoning content in the reasoning queue is less than the preset length threshold, then continue to wait for the reasoning content in the reasoning queue to be aggregated;

[0075] If the length of the inference content in the inference queue is greater than or equal to a preset length threshold, then the inference content in the inference queue is synthesized using the first preset speech synthesis parameters to obtain an inference audio block.

[0076] Preferably, the first preset speech synthesis parameters are a "thinking tone" with a low pitch, slight pauses, and a slow speech rate, which in turn indicate that the answer is still being constructed.

[0077] Then, the reasoning content and reasoning audio blocks are streamed using a unified data structure.

[0078] The unified data structure mentioned above is MessageStreamChunk, used to simultaneously carry text segment streams and audio segment streams. This invention adds AUDIO audio segment, TTS_START audio segment start signal, and TTS_COMPLETE audio segment end signal to the original MessageStreamChunk type enumeration, allowing text and audio segments to be interleaved in a single sequence and marking the boundaries of the synthesis stage. The audio segment start signal is used to trigger a local player warm-start and pre-buffering before responding to an audio block, and the audio segment end signal is used to determine the end point of audio playback and trigger resource reclamation.

[0079] In step S402, after the answer queue is written with the answer content for the first time, the system switches to the answer stage and uses the second preset speech synthesis parameters to synthesize the answer content to obtain the answer audio block. After all the remaining inference content in the inference queue is output and cleared, the answer content and the answer audio block are streamed with a unified data structure.

[0080] Specifically, the above steps include:

[0081] Once the answer queue is written with answer content for the first time, the system switches to the answering phase.

[0082] Determine whether the length of the answer content in the answer queue meets the second preset condition, wherein the second preset condition is a preset length threshold or a preset punctuation sentence segmentation;

[0083] If the content of the answers in the answer queue meets the second preset condition, then continue to wait for the content of the answers in the answer queue to be aggregated;

[0084] If the response content in the response queue does not meet the second preset condition, the response content in the response queue is synthesized using the second preset speech synthesis parameters to obtain the response audio block.

[0085] Meanwhile, after switching to the answering stage, the inference queue is flushed by calling flush_remaining_reasoning(), which forcibly empties the thinking speech. The remaining inference content in the inference queue is synthesized using the first preset speech synthesis parameters to obtain the remaining inference audio block. The remaining inference content and the remaining inference audio block are streamed with a unified data structure, and the inference queue is cleared to prevent the loss of the end of the inference sentence.

[0086] This embodiment of the invention utilizes the flush_remaining_reasoning() mechanism to ensure that the audio during the inference phase is not truncated due to state switching.

[0087] After the inference queue is cleared, the answer content and answer audio block will be streamed using a unified data structure.

[0088] Preferably, the second preset speech synthesis parameter is a "response intonation" using normal or clearer pronunciation parameters.

[0089] In this embodiment of the invention, the core answer is segmented according to length and punctuation, and quickly converted into audio using a minimum buffering strategy, providing almost instantaneous feedback. While maintaining the coherence of the reasoning speech, it effectively improves the timeliness of the first packet of the answer speech, thus accelerating the first packet of the answer.

[0090] In addition, the real-time text-to-speech streaming conversion method provided in this embodiment of the invention further includes:

[0091] Audio blocks are used to generate a distributed audio stream RedisStreamPubSub associated with the session. RedisStreamPubSub maintains the audio stream key corresponding to the session ID. Each terminal subscribes to the audio block queue of the same session to obtain and play audio segments. The terminals mentioned above include web terminals, APP terminals and mini-program terminals.

[0092] Upon access, playback can start from the most recent offset or the real-time head, supporting catch-up mode and synchronous mode. When a new device joins the same session, it quickly plays the generated inference audio by offset and catches up with the current answer, thus achieving a consistent experience across multiple devices.

[0093] This invention utilizes RedisStreamPubSub to achieve low-latency synchronous playback of multiple devices subscribing to the same session audio stream.

[0094] Figure 4 This diagram illustrates the multi-terminal interaction principle of a distributed audio stream in a real-time text-to-speech streaming conversion method provided by an embodiment of the present invention.

[0095] In addition, the real-time text-to-speech streaming conversion method provided in this embodiment of the invention further includes:

[0096] Based on MessageStopChecker, when a user interruption or a new question is detected, handle_stream_termination() is called. At the same time, a termination command is sent to the TTS synthesis module and the audio distribution module, and a termination flag is set to prevent subsequent text or audio chunks from entering the downstream, thereby achieving streaming interruption synchronization. For example, if the user sends "cancel" or a new question, the system sets the termination flag; stops subsequent text and audio chunks, and the client immediately ends playback to avoid the tail sound.

[0097] The embodiments of the present invention employ streaming interruption checks to ensure that text and audio are terminated simultaneously and accurately.

[0098] In addition, the real-time text-to-speech streaming conversion method provided in this embodiment of the invention further includes:

[0099] Audit and playback: All text and audio segments are recorded with timestamps and verification information, which can reconstruct the entire process of reasoning, clarification, tool calls, and responses for quality inspection and compliance.

[0100] Abnormal degradation: Continue to output text even when speech synthesis is unavailable; automatically resume speech in the next round after service recovery without interrupting the conversation.

[0101] Parameter expansion: The tone and buffer threshold can be adapted to the scenario, user preference or language characteristics; adding a new terminal or language only requires supplementing the tone configuration and subscription processing, without changing the core structure.

[0102] Corresponding to the above-mentioned real-time text-to-speech streaming conversion method, the present invention also provides a real-time text-to-speech streaming conversion system, the system including an LLM output layer, a state-aware layer, a TTS scheduling layer, a buffer manager, and a distributed publish-subscribe module;

[0103] The LLM output layer described above is used to receive user input requests and generate text content blocks based on the user requests;

[0104] The aforementioned state awareness layer is used to identify the type of text content blocks by listening to the thinking state and write them into the corresponding content queue, which includes an inference queue and an answer queue.

[0105] The aforementioned TTS scheduling layer and buffer manager are used to synthesize the inference content in the inference queue using the first preset speech synthesis parameters after the inference content in the inference queue meets the first preset condition, thereby obtaining an inference audio block. The inference content and the inference audio block are then streamed using a unified data structure. When the response queue is first written with response content, the system switches to the response stage and synthesizes the response content using the second preset speech synthesis parameters to obtain a response audio block. After all the remaining inference content in the inference queue has been output and cleared, the response content and the response audio block are then streamed using a unified data structure.

[0106] The aforementioned distributed publish-subscribe module is used to generate a distributed audio stream associated with a session using audio blocks. Each terminal subscribes to the distributed audio stream to obtain and play audio segments.

[0107] Figure 2 A schematic diagram of the architecture of the real-time text-to-speech streaming conversion system provided by the present invention is shown.

[0108] In addition, embodiments of the present invention also provide a real-time text-to-speech streaming conversion device, the device comprising: a processor and a memory; the memory being used to store one or more program instructions; the processor being used to run one or more program instructions to perform the steps of a real-time text-to-speech streaming conversion method as described above.

[0109] In addition, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a real-time text-to-speech streaming conversion method as described above.

[0110] In addition, embodiments of the present invention also provide a computer program product, which includes computer program instructions that, when executed by a processor, implement the steps of a real-time text-to-speech streaming conversion method as described above.

[0111] The real-time text-to-speech streaming conversion method, system, device, medium, and program product provided in this invention have the following advantages:

[0112] This invention significantly reduces the initial packet latency of the core response by triggering the audio during the response phase directly after the text segment arrives with minimal buffering. Simultaneously, by distinguishing between the thinking tone and the response tone, it allows users to recognize the current state of the AI, effectively enhancing interpretability. Furthermore, through aggregation buffering during the Reasoning phase, it maintains smooth thinking speech, ensuring no sentence endings are lost during transitions, effectively optimizing content continuity during transitions.

[0113] This invention also utilizes a synchronous interruption mechanism to avoid the "tail note" problem where audio continues to play even after text has stopped, thus improving interaction consistency. Furthermore, through Redis streaming distribution, multiple devices can play almost synchronously in real time, supporting catch-up playback.

[0114] Furthermore, this embodiment of the invention currently only adds three types and expands the buffer and state components in the unified message structure, which helps to simplify subsequent expansion. At the same time, retaining offsets and timestamps facilitates quality inspection and traceability, as well as playback and auditing.

[0115] In this embodiment of the invention, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in this embodiment of the invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in this embodiment of the invention can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor reads information from the storage medium and, in conjunction with its hardware, completes the steps of the above methods. The storage medium can be memory, for example, volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM).The storage media described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable types of memory. Those skilled in the art will recognize that the functions described in the above examples can be implemented using a combination of hardware and software. When applied software, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of computer programs from one place to another. Storage media can be any available medium accessible to general-purpose or special-purpose computers. Although the invention has been described in detail above with general description and specific embodiments, modifications or improvements can be made to it, which will be apparent to those skilled in the art. Therefore, such modifications or improvements made without departing from the spirit of the invention are all within the scope of protection claimed by the invention.

[0116] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes, or alterations made by those skilled in the art using the disclosed technical content shall fall within the protection scope of the present invention.

Claims

1. A method for real-time text-to-speech streaming conversion, characterized in that, The method includes: Requests to receive user input; Generate text content blocks based on user requests; The text content block type is identified by listening to the thinking state and written into the corresponding content queue, which includes a reasoning queue and a response queue. After the reasoning content in the reasoning queue meets the first preset condition, the reasoning content is synthesized using the first preset speech synthesis parameters to obtain a reasoning audio block, and the reasoning content and the reasoning audio block are streamed out using a unified data structure. After the answer queue is written with answer content for the first time, the system switches to the answer stage and uses the second preset speech synthesis parameters to synthesize the answer content to obtain an answer audio block. After all the remaining inference content in the inference queue is output and cleared, the answer content and the answer audio block are streamed using the unified data structure.

2. The method for real-time text-to-speech streaming conversion according to claim 1, characterized in that, The unified data structure is used to simultaneously carry text segmented streams and audio segmented streams; The data types of the unified data structure include text segments, audio segments, audio segment start signals, and audio segment end signals. The audio segment start signal is used to trigger a local player warm-start and pre-buffering before responding to an audio block; The audio segment end signal is used to determine the end point of audio playback and trigger resource reclamation.

3. The method for real-time text-to-speech streaming conversion according to claim 1, characterized in that, After the reasoning content in the reasoning queue meets the first preset condition, the reasoning content is synthesized using the first preset speech synthesis parameters, including: Determine whether the length of the reasoning content in the reasoning queue is less than a preset length threshold; If the length of the reasoning content in the reasoning queue is less than a preset length threshold, then continue to wait for the reasoning content in the reasoning queue to be aggregated; If the length of the inference content in the inference queue is greater than or equal to a preset length threshold, then the inference content in the inference queue is synthesized using the first preset speech synthesis parameters to obtain an inference audio block.

4. The method for real-time text-to-speech streaming conversion according to claim 1, characterized in that, After the response queue is first written with response content, the system switches to the response phase and uses the second preset speech synthesis parameters to synthesize the response content, resulting in a response audio block, including: Once the answer queue is written with answer content for the first time, the system switches to the answering phase. Determine whether the length of the answer content in the answer queue meets the second preset condition; If the answer content in the answer queue meets the second preset condition, then continue to wait for the answer content in the answer queue to be aggregated; If the answer content in the answer queue does not meet the second preset condition, the answer content in the answer queue is synthesized using the second preset speech synthesis parameters to obtain an answer audio block.

5. The method for real-time text-to-speech streaming conversion according to claim 1, characterized in that, After all residual reasoning content in the reasoning queue has been output and cleared, the answer content and the answer audio block are streamed using the unified data structure, including: After switching to the response phase, the inference queue is flushed. The residual inference content in the inference queue is synthesized using the first preset speech synthesis parameters to obtain a residual inference audio block. The residual inference content and the residual inference audio block are then streamed using a unified data structure, and the inference queue is cleared. After the inference queue is cleared, the answer content and the answer audio block will be streamed using the unified data structure.

6. The method for real-time text-to-speech streaming conversion according to claim 1, characterized in that, The method further includes: Generate a distributed audio stream associated with the session using audio blocks; Each terminal subscribes to the distributed audio stream to obtain and play audio segments. The terminals include web pages, apps, and mini-programs. The distributed audio stream supports synchronous playback and catch-up playback on various terminals.

7. A real-time text-to-speech streaming system, characterized in that, The system includes: an LLM output layer, a state awareness layer, a TTS scheduling layer, a buffer manager, and a distributed publish-subscribe module; The LLM output layer is used to receive user input requests and generate text content blocks based on the user requests; The state awareness layer is used to identify the type of text content block by listening to the thinking state and write it into the corresponding content queue, which includes an inference queue and an answer queue. The TTS scheduling layer and the buffer manager are used to synthesize the inference content in the inference queue using first preset speech synthesis parameters after the inference content in the inference queue meets the first preset condition, to obtain an inference audio block, and to stream the inference content and the inference audio block using a unified data structure; when the response queue is first written with response content, the system switches to the response stage, and synthesizes the response content using second preset speech synthesis parameters to obtain a response audio block; after all the remaining inference content in the inference queue is output and cleared, the response content and the response audio block are streamed using the unified data structure; The distributed publish-subscribe module is used to generate a distributed audio stream associated with a session using audio blocks. Each terminal subscribes to the distributed audio stream to obtain and play audio segments.

8. A real-time text-to-speech streaming device, characterized in that, The device includes: a processor and a memory; The memory is used to store one or more program instructions; The processor is configured to run one or more program instructions to perform the steps of a real-time text-to-speech streaming method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a real-time text-to-speech streaming method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes computer program instructions that, when executed by a processor, implement the steps of a real-time text-to-speech streaming method as described in any one of claims 1 to 6.

Images