Chat-TTS-model-based natural speech reading method and system

Abstract

A Chat-TTS-model-based natural speech reading method and system, which relate to the technical field of speech reading. The method comprises: deploying a Chat-TTS model on a server side; a user uploading novel text by means of a front-end browser, and the front-end browser performing automatic chapter segmentation processing; the front-end browser using WebSocket to transmit, to the server side, text data obtained after the chapter segmentation processing, and the server side using the Chat-TTS model to convert the novel text into a PCM-format audio file of a specified duration; and the front-end browser using Jessibuca.js to play the PCM-format audio file and maintaining synchronization of novel content with audio. The present invention is used for converting text into audio, which can bring an immersive reading experience to a user, and supports adjusting an audio caching duration on the basis of a requirement, so as to meet the requirements of different users.

Description

A natural speech reading method and system based on Chat-TTS model

[0001] This application claims priority to Chinese Patent Application No. 202411579643.6, filed on November 7, 2024, entitled "A Natural Speech Reading Method and System Based on Chat-TTS Model", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This invention relates to the field of speech reading technology, specifically a natural speech reading method and system based on the Chat-TTS model. Background Technology

[0003] Traditional e-readers primarily rely on text display, lacking voice interaction and emotional expression, making it difficult to meet users' personalized reading needs. In recent years, with the rapid development of artificial intelligence technology, Chat-TTS (text-to-speech) technology has matured, capable of generating natural and fluent speech based on text content, and possessing emotional expression capabilities. The Chat-TTS model is a text-to-speech model specifically designed for dialogue scenarios. It supports multiple languages, including English and Chinese. The largest model was trained using 100,000 hours of Chinese and English data, while the open-source version on Huggingface is a 40,000-hour trained version without SFT (Sentence Transfer Function). To ensure high-quality and natural voice synthesis, the Chat-TTS model employs various advanced technologies, such as neural networks, attention mechanisms, emotional expression, and style transfer.

[0004] Jessibuca.js is a high-performance web player based on WebAssembly that supports multiple audio formats, including PCM. PCM is an uncompressed audio format that preserves the original audio information and ensures high-fidelity sound quality. Jessibuca.js uses WebAssembly technology to compile C++ code into WebAssembly modules, which then run in a browser, thus achieving high-performance audio playback.

[0005] However, existing voice readers still need improvement in terms of functionality, ease of use, and user experience. For example, some voice readers can only convert text into monotonous speech, lacking emotional expression; some cannot achieve synchronized playback of text and speech, affecting the user's reading experience; and some voice readers are complex to operate and difficult to learn. Summary of the Invention

[0006] This invention addresses the needs and shortcomings of current technological development by providing a natural speech reading method and system based on the Chat-TTS model, bringing users an immersive reading experience and supporting audio caching for specified durations to meet the needs of different users.

[0007] Firstly, the present invention provides a natural speech reading method based on a Chat-TTS model, and the technical solution adopted to solve the above-mentioned technical problems is as follows:

[0008] A natural speech reading method based on a Chat-TTS model includes the following steps:

[0009] S1. Deploy the Chat-TTS model on the server side;

[0010] S2. Users upload novel text through the front-end browser and the text is automatically divided into chapters.

[0011] S3. The front-end browser uses WebSocket to transmit the text data output in step S2 to the server. The server uses the Chat-TTS model to convert the novel text into a PCM format audio file of a specified duration.

[0012] S4. The front-end browser uses Jessibuca.js to play PCM format audio files while keeping the novel content synchronized with the audio.

[0013] Optionally, perform step S1 to deploy the Chat-TTS model on the server side. The specific deployment process is as follows:

[0014] S1.1 Preparing the server environment: Ensure that the server has sufficient computing resources and storage space to meet the running requirements of the Chat-TTS model. At the same time, install the necessary operating system and dependency libraries.

[0015] S1.2 Download Chat-TTS model: Obtain the Chat-TTS model from official channels and extract it to the specified directory on the server;

[0016] S1.3 Configuring Model Parameters: Adjust the parameters of the Chat-TTS model according to the server hardware configuration and actual needs to optimize the speech synthesis effect;

[0017] S1.4 Deployment Model: Deploy the Chat-TTS model to the server and ensure that the Chat-TTS model can run normally.

[0018] Optionally, in step S2, the user uploads the novel text through the front-end browser and the text is automatically divided into chapters. The specific implementation is as follows:

[0019] S2.1. Upload the novel via the web page of the front-end browser;

[0020] S2.2 The front-end browser uses natural language processing technology to automatically divide the uploaded novel into chapters: First, it reads the text content of the novel and analyzes the paragraph structure; then, based on the chapter title and paragraph length, it automatically identifies the chapter boundaries, generates a chapter directory, and realizes automatic chapter division.

[0021] Optionally, in step S3, the front-end browser uses WebSocket to transmit the text data output in step S2 to the server. The server then uses the Chat-TTS model to convert the novel text into a PCM format audio file of a specified duration. The specific implementation is as follows:

[0022] S3.1 The front-end browser uses the WebSocket protocol to establish a connection with the server to achieve real-time data transmission;

[0023] S3.2 After the user selects a chapter of the novel, the front-end browser sends the selected text data to the server via WebSocket;

[0024] S3.3 Server-side audio caching based on user-specified duration: First, the server receives text data transmitted from the front-end browser, then uses the Chat-TTS model to convert the text into a PCM format audio file of a specified duration, and finally caches the generated audio file on the server.

[0025] Optionally, in step S4, the front-end browser uses Jessibuca.js to play PCM format audio while maintaining synchronization between the novel content and the audio. The specific implementation is as follows:

[0026] Use the Jessibuca.js player to play PCM format audio files cached on the server side;

[0027] During playback, users can control the audio playback progress, adjust the playback speed and volume to meet their personalized needs.

[0028] During playback, the audio playback progress is monitored, and the scrolling position of the novel content is updated in real time; users can click on the novel content to jump to the next audio playback position.

[0029] Secondly, the present invention provides a natural speech reading system based on a Chat-TTS model, and the technical solution adopted to solve the above-mentioned technical problems is as follows:

[0030] A natural speech reading system based on a Chat-TTS model, comprising:

[0031] The model deployment module is used to deploy Chat-TTS models on the server side;

[0032] The upload section module is used to assist users in uploading novel text through the front-end browser and automatically dividing it into chapters;

[0033] The transmission call module is used to transmit the text data output by the front-end browser to the server and call the Chat-TTS model to convert the novel text into a PCM format audio file of a specified duration.

[0034] The cache transmission module is used to cache the PCM format audio files converted from Chat-TTS models on the server side and transmit them to the front-end browser;

[0035] The audio playback module is used to play PCM format audio files using Jessibuca.js, while keeping the novel content synchronized with the audio.

[0036] Optionally, the process for deploying the Chat-TTS model on the server side using the model deployment module is as follows:

[0037] Prepare the server environment: Ensure that the server has sufficient computing resources and storage space to meet the running requirements of the Chat-TTS model. At the same time, install the necessary operating system and dependency libraries.

[0038] Download the Chat-TTS model: Obtain the Chat-TTS model from official channels and extract it to the specified directory on the server;

[0039] Configure model parameters: Adjust the parameters of the Chat-TTS model according to the server hardware configuration and actual needs to optimize the speech synthesis effect;

[0040] Deployment Model: Deploy the Chat-TTS model to the server and ensure that the Chat-TTS model can function properly.

[0041] Optionally, the upload segmentation modules involved specifically include:

[0042] The upload unit assists users in uploading novel texts through a web page in a front-end browser;

[0043] The segmentation unit is used to automatically divide uploaded novels into chapters using natural language processing technology. The specific process includes: first, reading the text content of the novel and analyzing the paragraph structure; then, automatically identifying chapter boundaries based on chapter titles and paragraph lengths, generating a chapter index, and realizing automatic chapter division.

[0044] Optionally, the transport call modules involved specifically include:

[0045] The connection unit is used to establish a connection between the front-end browser and the server using the WebSocket protocol to achieve real-time data transmission.

[0046] The transmission unit is used to send the text content selected by the user to the server via WebSocket;

[0047] The calling unit is used to invoke the Chat-TTS model deployed on the server side. The Chat-TTS model converts the sent text content into a PCM format audio file of a specified duration.

[0048] Optionally, the audio playback module involved includes:

[0049] An audio playback unit is used to play PCM format audio files using Jessibuca.js;

[0050] The audio control unit is used to view and adjust the playback progress, adjust the playback speed and playback volume to meet the user's personalized needs;

[0051] The audio monitoring unit is used to monitor the audio playback progress and update the scrolling position of the novel content in real time.

[0052] The audio jump unit is used to jump to different audio playback modes based on the user's click operation.

[0053] The natural speech reading method and system based on the Chat-TTS model of the present invention have the following advantages compared with the prior art:

[0054] 1. This invention can bring users an immersive reading experience and supports adjusting the audio buffer duration according to needs to meet the needs of different users;

[0055] 2. This invention leverages the ease of learning and use, and rich APIs of Chat-TTS and JavaScript to simplify a large amount of business-irrelevant code, making the code logic simpler and clearer, which is beneficial to the development work of coders; Chat-TTS technology can generate natural and fluent speech with emotional expression, providing users with an immersive reading experience; Jessibuca.js can achieve high-performance audio playback through WebAssembly technology, providing users with a smooth voice reading experience. Attached Figure Description

[0056] Figure 1 is a flowchart of the method according to Embodiment 1 of the present invention;

[0057] Figure 2 is a module connection block diagram of Embodiment 2 of the present invention. Detailed Implementation

[0058] To make the technical solution, the technical problem solved, and the technical effect of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments.

[0059] Example 1:

[0060] Referring to Figure 1, this embodiment proposes a natural speech reading method based on a Chat-TTS model, which includes the following steps:

[0061] S1. Deploy the Chat-TTS model on the server side. The specific deployment process is as follows:

[0062] S1.1 Preparing the server environment: Ensure that the server has sufficient computing resources and storage space to meet the running requirements of the Chat-TTS model. At the same time, install the necessary operating system and dependency libraries.

[0063] S1.2 Download Chat-TTS model: Obtain the Chat-TTS model from official channels and extract it to the specified directory on the server;

[0064] S1.3 Configuring Model Parameters: Adjust the parameters of the Chat-TTS model according to the server hardware configuration and actual needs to optimize the speech synthesis effect;

[0065] S1.4 Deployment Model: Deploy the Chat-TTS model to the server and ensure that the Chat-TTS model can run normally.

[0066] S2. Users upload novel text through the front-end browser and the text is automatically divided into chapters. The specific implementation is as follows:

[0067] S2.1. Upload the novel via the web page of the front-end browser;

[0068] S2.2 The front-end browser uses natural language processing technology to automatically divide the uploaded novel into chapters: First, it reads the text content of the novel and analyzes the paragraph structure; then, based on the chapter title and paragraph length, it automatically identifies the chapter boundaries, generates a chapter directory, and realizes automatic chapter division.

[0069] S3. The front-end browser uses WebSocket to transmit the text data output in step S2 to the server. The server uses the Chat-TTS model to convert the novel text into a PCM format audio file of a specified duration. The specific implementation is as follows:

[0070] S3.1 The front-end browser uses the WebSocket protocol to establish a connection with the server to achieve real-time data transmission;

[0071] S3.2 After the user selects a chapter of the novel, the front-end browser sends the selected text data to the server via WebSocket;

[0072] S3.3 Server-side audio caching based on user-specified duration: First, the server receives text data transmitted from the front-end browser, then uses the Chat-TTS model to convert the text into a PCM format audio file of a specified duration, and finally caches the generated audio file on the server.

[0073] S4. The front-end browser uses Jessibuca.js to play PCM format audio files while maintaining synchronization between the novel content and the audio. The specific implementation is as follows:

[0074] Use the Jessibuca.js player to play PCM format audio files cached on the server side;

[0075] During playback, users can control the audio playback progress, adjust the playback speed and volume to meet their personalized needs.

[0076] During playback, the audio playback progress is monitored, and the scrolling position of the novel content is updated in real time; users can click on the novel content to jump to the next audio playback position.

[0077] Example 2:

[0078] Referring to Figure 2, this embodiment proposes a natural speech reading system based on a Chat-TTS model, which includes:

[0079] The model deployment module is used to deploy Chat-TTS models on the server side;

[0080] The upload section module is used to assist users in uploading novel text through the front-end browser and automatically dividing it into chapters;

[0081] The transmission call module is used to transmit the text data output by the front-end browser to the server and call the Chat-TTS model to convert the novel text into a PCM format audio file of a specified duration.

[0082] The cache transmission module is used to cache the PCM format audio files converted from Chat-TTS models on the server side and transmit them to the front-end browser;

[0083] The audio playback module is used to play PCM format audio files using Jessibuca.js, while keeping the novel content synchronized with the audio.

[0084] In this embodiment, the process by which the model deployment module deploys the Chat-TTS model on the server side is as follows:

[0085] Prepare the server environment: Ensure that the server has sufficient computing resources and storage space to meet the running requirements of the Chat-TTS model. At the same time, install the necessary operating system and dependency libraries.

[0086] Download the Chat-TTS model: Obtain the Chat-TTS model from official channels and extract it to the specified directory on the server;

[0087] Configure model parameters: Adjust the parameters of the Chat-TTS model according to the server hardware configuration and actual needs to optimize the speech synthesis effect;

[0088] Deployment Model: Deploy the Chat-TTS model to the server and ensure that the Chat-TTS model can function properly.

[0089] In this embodiment, the upload segmentation module specifically includes:

[0090] The upload unit assists users in uploading novel texts through a web page in a front-end browser;

[0091] The segmentation unit is used to automatically divide uploaded novels into chapters using natural language processing technology. The specific process includes: first, reading the text content of the novel and analyzing the paragraph structure; then, automatically identifying chapter boundaries based on chapter titles and paragraph lengths, generating a chapter index, and realizing automatic chapter division.

[0092] In this embodiment, the transmission and invocation module specifically includes:

[0093] The connection unit is used to establish a connection between the front-end browser and the server using the WebSocket protocol to achieve real-time data transmission.

[0094] The transmission unit is used to send the text content selected by the user to the server via WebSocket;

[0095] The calling unit is used to invoke the Chat-TTS model deployed on the server side. The Chat-TTS model converts the sent text content into a PCM format audio file of a specified duration.

[0096] In this embodiment, the audio playback module includes:

[0097] An audio playback unit is used to play PCM format audio files using Jessibuca.js;

[0098] The audio control unit is used to view and adjust the playback progress, adjust the playback speed and playback volume to meet the user's personalized needs;

[0099] The audio monitoring unit is used to monitor the audio playback progress and update the scrolling position of the novel content in real time.

[0100] The audio jump unit is used to jump to different audio playback modes based on the user's click operation.

[0101] In summary, the natural speech reading method and system based on the Chat-TTS model of this invention can bring users an immersive reading experience and support adjusting the audio buffer duration according to needs to meet the needs of different users.

[0102] The above specific examples illustrate the principles and implementation methods of the present invention in detail. These embodiments are merely for the purpose of helping to understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, any improvements and modifications made to the present invention by those skilled in the art without departing from the principles of the present invention should fall within the patent protection scope of the present invention.

Claims

1. A natural speech reading method based on a Chat-TTS model, characterized in that, Includes the following steps: S1. Deploy the Chat-TTS model on the server side; S2. Users upload novel text through the front-end browser and the text is automatically divided into chapters. S3. The front-end browser uses WebSocket to transmit the text data output in step S2 to the server. The server uses the Chat-TTS model to convert the novel text into a PCM format audio file of a specified duration. S4. The front-end browser uses Jessibuca.js to play PCM format audio files while keeping the novel content synchronized with the audio.

2. The natural speech reading method based on the Chat-TTS model according to claim 1, characterized in that, Execute step S1 to deploy the Chat-TTS model on the server side. The specific deployment process is as follows: S1.1 Preparing the server environment: Ensure that the server has sufficient computing resources and storage space to meet the running requirements of the Chat-TTS model. At the same time, install the necessary operating system and dependency libraries. S1.2 Download Chat-TTS model: Obtain the Chat-TTS model from official channels and extract it to the specified directory on the server; S1.3 Configuring Model Parameters: Adjust the parameters of the Chat-TTS model according to the server hardware configuration and actual needs to optimize the speech synthesis effect; S1.4 Deployment Model: Deploy the Chat-TTS model to the server and ensure that the Chat-TTS model can run normally.

3. The natural speech reading method based on the Chat-TTS model according to claim 1, characterized in that, In step S2, the user uploads the novel text through the front-end browser and the text is automatically divided into chapters. The specific implementation is as follows: S2.

1. Upload the novel via the web page of the front-end browser; S2.2 The front-end browser uses natural language processing technology to automatically divide the uploaded novel into chapters: First, it reads the text content of the novel and analyzes the paragraph structure; then, based on the chapter title and paragraph length, it automatically identifies the chapter boundaries, generates a chapter directory, and realizes automatic chapter division.

4. The natural speech reading method based on the Chat-TTS model according to claim 1, characterized in that, In step S3, the front-end browser uses WebSocket to transmit the text data output in step S2 to the server. The server then uses the Chat-TTS model to convert the novel text into a PCM audio file of a specified duration. The specific implementation is as follows: S3.1 The front-end browser uses the WebSocket protocol to establish a connection with the server to achieve real-time data transmission; S3.2 After the user selects a chapter of the novel, the front-end browser sends the selected text data to the server via WebSocket; S3.3 Server-side audio caching based on user-specified duration: First, the server receives text data transmitted from the front-end browser, then uses the Chat-TTS model to convert the text into a PCM format audio file of a specified duration, and finally caches the generated audio file on the server.

5. A natural speech reading method based on a Chat-TTS model according to claim 1, characterized in that, In step S4, the front-end browser uses Jessibuca.js to play PCM format audio while maintaining synchronization between the novel content and the audio. The specific implementation is as follows: Use the Jessibuca.js player to play PCM format audio files cached on the server side; During playback, users can control the audio playback progress, adjust the playback speed and volume to meet their personalized needs. During playback, the audio playback progress is monitored, and the scrolling position of the novel content is updated in real time; users can click on the novel content to jump to the next audio playback position.

6. A natural speech reading system based on a Chat-TTS model, characterized in that, It includes: The model deployment module is used to deploy Chat-TTS models on the server side; The upload section module is used to assist users in uploading novel text through the front-end browser and automatically dividing it into chapters; The transmission call module is used to transmit the text data output by the front-end browser to the server and call the Chat-TTS model to convert the novel text into a PCM format audio file of a specified duration. The cache transmission module is used to cache the PCM format audio files converted from Chat-TTS models on the server side and transmit them to the front-end browser; The audio playback module is used to play PCM format audio files using Jessibuca.js, while keeping the novel content synchronized with the audio.

7. A natural speech reading system based on a Chat-TTS model according to claim 6, characterized in that, The process by which the model deployment module deploys the Chat-TTS model on the server side is as follows: Prepare the server environment: Ensure that the server has sufficient computing resources and storage space to meet the running requirements of the Chat-TTS model. At the same time, install the necessary operating system and dependency libraries. Download the Chat-TTS model: Obtain the Chat-TTS model from official channels and extract it to the specified directory on the server; Configure model parameters: Adjust the parameters of the Chat-TTS model according to the server hardware configuration and actual needs to optimize the speech synthesis effect; Deployment Model: Deploy the Chat-TTS model to the server and ensure that the Chat-TTS model can function properly.

8. A natural speech reading system based on a Chat-TTS model according to claim 6, characterized in that, The upload segmentation module specifically includes: The upload unit assists users in uploading novel texts through a web page in a front-end browser; The segmentation unit is used to automatically divide uploaded novels into chapters using natural language processing technology. The specific process includes: first, reading the text content of the novel and analyzing the paragraph structure; then, automatically identifying chapter boundaries based on chapter titles and paragraph lengths, generating a chapter index, and realizing automatic chapter division.

9. A natural speech reading system based on a Chat-TTS model according to claim 6, characterized in that, The transmission and invocation module specifically includes: The connection unit is used to establish a connection between the front-end browser and the server using the WebSocket protocol to achieve real-time data transmission. The transmission unit is used to send the text content selected by the user to the server via WebSocket; The calling unit is used to invoke the Chat-TTS model deployed on the server side. The Chat-TTS model converts the sent text content into a PCM format audio file of a specified duration.

10. A natural speech reading system based on a Chat-TTS model according to claim 6, characterized in that, The audio playback module includes: An audio playback unit is used to play PCM format audio files using Jessibuca.js; The audio control unit is used to view and adjust the playback progress, adjust the playback speed and playback volume to meet the user's personalized needs; The audio monitoring unit is used to monitor the audio playback progress and update the scrolling position of the novel content in real time. The audio jump unit is used to jump to different audio playback modes based on the user's click operation.

Images