Natural voice reading method and system based on Chat-TTS model

Abstract

The invention discloses a Chat-TTS model-based natural voice reading method and system, and relates to the technical field of voice reading, and the method comprises the steps: deploying a Chat-TTS model at a server side; a user uploads a novel text through a front-end browser and automatically divides the novel text into chapters and sections; the front-end browser uses WebSocket to transmit text data subjected to chapter division processing to a server side, and the server side uses a Chat-TTS model to convert a novel text into a PCM format audio file with specified duration; and the front-end browser plays the audio file in the PCM format by using Jessibuca.js, and keeps the synchronization of the novel content and the audio at the same time. The method and the device are used for converting the text into the audio, immersive reading experience can be brought to the user, and the audio caching duration can be adjusted according to requirements so as to meet the requirements of different users.

Description

Technical Field

[0001] The present invention relates to the field of speech reading technology, in particular to a natural speech reading method and system based on a Chat-TTS model. Background Art

[0002] Traditional e-readers mainly rely on text display, lack voice interaction and emotional expression, and are difficult to meet users' personalized reading needs. In recent years, with the rapid development of artificial intelligence technology, Chat-TTS model (chat text to speech synthesis) technology has gradually matured, capable of generating natural and fluent speech based on text content, and has the ability to express emotions. The Chat-TTS model is a text-to-speech model designed specifically for conversational scenarios. It supports multiple languages, including English and Chinese. The largest model uses 100,000 hours of Chinese and English data for training. The open source version in Huggingface is a 40,000-hour training and unencrypted version. To ensure the high quality and naturalness of voice synthesis, the Chat-TTS model uses a variety of advanced technologies, such as neural networks, attention mechanisms, emotional expression, and style transfer.

[0003] Jessibuca.js is a high-performance Web player based on WebAssembly that supports multiple audio formats, including PCM. PCM is an uncompressed audio format that preserves the original audio information and ensures high fidelity of the sound quality. Jessibuca.js compiles C++ code into WebAssembly modules through WebAssembly technology and runs in the browser, thus achieving high-performance audio playback.

[0004] However, existing voice readers still need to be improved in terms of functionality, ease of operation and user experience. For example, some voice readers can only convert text into monotonous voice and lack the ability to express emotions. Some voice readers cannot achieve synchronous playback of text and voice, which affects the user's reading experience. At the same time, some voice readers are complex to operate and difficult to use. Summary of the invention

[0005] In view of the needs and shortcomings of current technological development, the present invention provides a natural voice reading method and system based on the Chat-TTS model, which brings an immersive reading experience to users and supports audio caching of specified duration to meet the needs of different users.

[0006] In the first aspect, the present invention provides a natural speech reading method based on the Chat-TTS model, and the technical solution adopted to solve the above technical problems is as follows:

[0007] A natural speech reading method based on Chat-TTS model comprises the following steps:

[0008] S1. Deploy the Chat-TTS model on the server side;

[0009] S2, the user uploads the novel text through the front-end browser and automatically divides it into chapters;

[0010] S3, the front-end browser uses WebSocket to transmit the text data output in step S2 to the server, and the server uses the Chat-TTS model to convert the novel text into a PCM format audio file of a specified length;

[0011] S4. The front-end browser uses Jessibuca.js to play audio files in PCM format while keeping the novel content synchronized with the audio.

[0012] Optionally, execute step S1 to deploy the Chat-TTS model on the server. The specific deployment process is as follows:

[0013] S1.1. Prepare the server environment: Ensure that the computing resources and storage space of the server meet the operating requirements of the Chat-TTS model. At the same time, install the necessary operating system and dependent libraries;

[0014] S1.2. Download the Chat-TTS model: Obtain the Chat-TTS model from the official channel and decompress it to the specified directory on the server;

[0015] S1.3. Configure model parameters: According to the server hardware configuration and actual needs, adjust the parameters of the Chat-TTS model to optimize the speech synthesis effect;

[0016] S1.4. Deploy model: Deploy the Chat-TTS model to the server and ensure that the Chat-TTS model can run normally.

[0017] Optionally, step S2 is performed, the user uploads the novel text through the front-end browser and automatically divides it into chapters, and the specific implementation operation is as follows:

[0018] S2.1, upload the novel through the web page of the front-end browser;

[0019] S2.2. The front-end browser uses natural language processing technology to automatically divide the uploaded novel into chapters: first, the text content of the novel is read and the paragraph structure is analyzed; then, based on the chapter title and paragraph length, the chapter boundary is automatically identified, and a chapter directory is generated to achieve automatic chapter division.

[0020] Optionally, step S3 is performed, the front-end browser transmits the text data output in step S2 to the server using WebSocket, and the server uses the Chat-TTS model to convert the novel text into a PCM format audio file of a specified length. The specific implementation operation is as follows:

[0021] S3.1, the front-end browser uses the WebSocket protocol to establish a connection with the server to achieve real-time data transmission;

[0022] S3.2, after the user selects a novel chapter, the front-end browser sends the selected text data to the server through WebSocket;

[0023] S3.3. The server caches the audio according to the duration specified by the user: First, the text data transmitted by the front-end browser is received, and then the Chat-TTS model is used to convert the text into a PCM format audio file of the specified duration, and finally the generated audio file is cached to the server.

[0024] Optionally, step S4 is performed, and the front-end browser uses Jessibuca.js to play the audio in PCM format while keeping the novel content and the audio synchronized. The specific implementation operation is as follows:

[0025] Use Jessibuca.js player to play PCM audio files cached on the server;

[0026] During playback, you can check the playback progress, adjust the playback speed and volume through audio playback control to meet the user's personalized needs;

[0027] During the playback process, the audio playback progress is monitored and the scrolling position of the novel content is updated in real time; users are supported to click on the novel content to jump to the audio playback.

[0028] In a second aspect, the present invention provides a natural speech reading system based on the Chat-TTS model, and the technical solution adopted to solve the above technical problems is as follows:

[0029] A natural speech reading system based on Chat-TTS model, comprising:

[0030] Model deployment module, used to deploy the Chat-TTS model on the server side;

[0031] The upload and segmentation module is used to assist users in uploading novel texts through the front-end browser and automatically divide them into chapters;

[0032] The transmission calling module is used to transmit the text data output by the front-end browser to the server, and call the Chat-TTS model to convert the novel text into a PCM format audio file of a specified length;

[0033] The cache transmission module is used to cache the PCM format audio files converted and output by the Chat-TTS model on the server side and transmit them to the front-end browser;

[0034] An audio playback module that uses Jessibuca.js to play audio files in PCM format while keeping the novel content synchronized with the audio.

[0035] Optionally, the model deployment module deploys the Chat-TTS model on the server side as follows:

[0036] Prepare the server environment: Ensure that the computing resources and storage space available on the server meet the operating requirements of the Chat-TTS model. At the same time, install the necessary operating system and dependent libraries.

[0037] Download the Chat-TTS model: Get the Chat-TTS model from the official channel and decompress it to the specified directory on the server;

[0038] Configure model parameters: adjust the parameters of the Chat-TTS model according to the server hardware configuration and actual needs to optimize the speech synthesis effect;

[0039] Deploy the model: Deploy the Chat-TTS model to the server and ensure that the Chat-TTS model can run normally.

[0040] Optionally, the upload segment modules involved include:

[0041] An upload unit, used to assist users in uploading novel texts through a web page of a front-end browser;

[0042] The segmentation unit is used to automatically divide the uploaded novel into chapters using natural language processing technology. The specific process includes: first reading the text content of the novel and analyzing the paragraph structure; then automatically identifying the chapter boundaries based on the chapter title and paragraph length, generating a chapter directory, and realizing automatic chapter division.

[0043] Optionally, the transmission calling module involved specifically includes:

[0044] The connection unit is used to establish a connection between the front-end browser and the server using the WebSocket protocol to achieve real-time data transmission;

[0045] The transmission unit is used to send the text content selected by the user to the server through WebSocket;

[0046] The calling unit is used to call the Chat-TTS model deployed on the server. The Chat-TTS model converts the sent text content into a PCM format audio file of a specified length.

[0047] Optionally, the audio playback modules involved include:

[0048] Audio playback unit, used to play PCM audio files using Jessibuca.js;

[0049] Audio control unit, used to check and adjust the playback progress, speed and volume to meet the user's personalized needs;

[0050] The audio monitoring unit is used to monitor the audio playback progress and update the scrolling position of the novel content in real time;

[0051] The audio jump unit is used to realize the jump playback of audio according to the user's click operation.

[0052] The natural speech reading method and system based on the Chat-TTS model of the present invention have the following beneficial effects compared with the prior art:

[0053] 1. The present invention can bring an immersive reading experience to users and support adjusting the audio cache duration according to needs to meet the needs of different users;

[0054] 2. The present invention can simplify a large amount of business-independent codes with the help of Chat-TTS and JavaScript's easy-to-learn, easy-to-use and rich APIs, making the code logic simpler and clearer, which is beneficial to the development work of coders; Chat-TTS technology can generate natural and fluent speech with emotional expression ability, providing users with an immersive reading experience; Jessibuca.js can achieve high-performance audio playback through WebAssembly technology, providing users with a smooth voice reading experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Attached Figure 1 is a flow chart of a method according to Embodiment 1 of the present invention;

[0056] Attached Figure 2 This is a module connection block diagram of the second embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the technical solution, the technical problem solved and the technical effect of the present invention more clearly understood, the technical solution of the present invention is clearly and completely described below in conjunction with specific embodiments.

[0058] Embodiment 1:

[0059] Combined with Figure 1 This embodiment proposes a natural speech reading method based on the Chat-TTS model, which includes the following steps:

[0060] S1. Deploy the Chat-TTS model on the server. The specific deployment process is as follows:

[0061] S1.1. Prepare the server environment: Ensure that the computing resources and storage space of the server meet the operating requirements of the Chat-TTS model. At the same time, install the necessary operating system and dependent libraries;

[0062] S1.2. Download the Chat-TTS model: Obtain the Chat-TTS model from the official channel and decompress it to the specified directory on the server;

[0063] S1.3. Configure model parameters: According to the server hardware configuration and actual needs, adjust the parameters of the Chat-TTS model to optimize the speech synthesis effect;

[0064] S1.4. Deploy model: Deploy the Chat-TTS model to the server and ensure that the Chat-TTS model can run normally.

[0065] S2. Users upload novel texts through the front-end browser and automatically divide them into chapters. The specific implementation operations are as follows:

[0066] S2.1, upload the novel through the web page of the front-end browser;

[0067] S2.2. The front-end browser uses natural language processing technology to automatically divide the uploaded novel into chapters: first, the text content of the novel is read and the paragraph structure is analyzed; then, based on the chapter title and paragraph length, the chapter boundary is automatically identified, and a chapter directory is generated to achieve automatic chapter division.

[0068] S3, the front-end browser uses WebSocket to transmit the text data output in step S2 to the server, and the server uses the Chat-TTS model to convert the novel text into a PCM format audio file of a specified length. The specific implementation operations are as follows:

[0069] S3.1, the front-end browser uses the WebSocket protocol to establish a connection with the server to achieve real-time data transmission;

[0070] S3.2, after the user selects a novel chapter, the front-end browser sends the selected text data to the server through WebSocket;

[0071] S3.3. The server caches the audio according to the duration specified by the user: First, the text data transmitted by the front-end browser is received, and then the Chat-TTS model is used to convert the text into a PCM format audio file of the specified duration, and finally the generated audio file is cached to the server.

[0072] S4. The front-end browser uses Jessibuca.js to play the audio file in PCM format, while keeping the novel content and audio synchronized. The specific implementation operations are as follows:

[0073] Use Jessibuca.js player to play PCM audio files cached on the server;

[0074] During playback, you can check the playback progress, adjust the playback speed and volume through audio playback control to meet the user's personalized needs;

[0075] During the playback process, the audio playback progress is monitored and the scrolling position of the novel content is updated in real time; users are supported to click on the novel content to jump to the audio playback.

[0076] Embodiment 2:

[0077] Combined with Figure 2 This embodiment proposes a natural speech reading system based on the Chat-TTS model, which includes:

[0078] Model deployment module, used to deploy the Chat-TTS model on the server side;

[0079] The upload and segmentation module is used to assist users in uploading novel texts through the front-end browser and automatically divide them into chapters;

[0080] The transmission calling module is used to transmit the text data output by the front-end browser to the server, and call the Chat-TTS model to convert the novel text into a PCM format audio file of a specified length;

[0081] The cache transmission module is used to cache the PCM format audio files converted and output by the Chat-TTS model on the server side and transmit them to the front-end browser;

[0082] An audio playback module that uses Jessibuca.js to play audio files in PCM format while keeping the novel content synchronized with the audio.

[0083] In this embodiment, the process of the model deployment module deploying the Chat-TTS model on the server side is as follows:

[0084] Prepare the server environment: Ensure that the computing resources and storage space available on the server meet the operating requirements of the Chat-TTS model. At the same time, install the necessary operating system and dependent libraries.

[0085] Download the Chat-TTS model: Get the Chat-TTS model from the official channel and decompress it to the specified directory on the server;

[0086] Configure model parameters: adjust the parameters of the Chat-TTS model according to the server hardware configuration and actual needs to optimize the speech synthesis effect;

[0087] Deploy the model: Deploy the Chat-TTS model to the server and ensure that the Chat-TTS model can run normally.

[0088] In this embodiment, the upload segmentation module involved specifically includes:

[0089] An upload unit, used to assist users in uploading novel texts through a web page of a front-end browser;

[0090] The segmentation unit is used to automatically divide the uploaded novel into chapters using natural language processing technology. The specific process includes: first reading the text content of the novel and analyzing the paragraph structure; then automatically identifying the chapter boundaries based on the chapter title and paragraph length, generating a chapter directory, and realizing automatic chapter division.

[0091] In this embodiment, the transmission calling module involved specifically includes:

[0092] The connection unit is used to establish a connection between the front-end browser and the server using the WebSocket protocol to achieve real-time data transmission;

[0093] The transmission unit is used to send the text content selected by the user to the server through WebSocket;

[0094] The calling unit is used to call the Chat-TTS model deployed on the server. The Chat-TTS model converts the sent text content into a PCM format audio file of a specified length.

[0095] In this embodiment, the audio playback module involved includes:

[0096] Audio playback unit, used to play PCM audio files using Jessibuca.js;

[0097] Audio control unit, used to check and adjust the playback progress, speed and volume to meet the user's personalized needs;

[0098] The audio monitoring unit is used to monitor the audio playback progress and update the scrolling position of the novel content in real time;

[0099] The audio jump unit is used to realize the jump playback of audio according to the user's click operation.

[0100] In summary, the natural speech reading method and system based on the Chat-TTS model of the present invention can bring an immersive reading experience to users, and support adjusting the audio cache duration according to needs to meet the needs of different users.

[0101] The above specific examples are used to explain the principles and implementation methods of the present invention in detail. These examples are only used to help understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, any improvements and modifications made by technicians in this technical field without departing from the principles of the present invention should fall within the scope of patent protection of the present invention.

Claims

1. A natural speech reading method based on Chat-TTS model, characterized in that: The steps include: S1. Deploy the Chat-TTS model on the server side; S2, the user uploads the novel text through the front-end browser and automatically divides it into chapters; S3, the front-end browser uses WebSocket to transmit the text data output in step S2 to the server, and the server uses the Chat-TTS model to convert the novel text into a PCM format audio file of a specified length; S4. The front-end browser uses Jessibuca.js to play audio files in PCM format while keeping the novel content synchronized with the audio.

2. A natural speech reading method based on Chat-TTS model according to claim 1, characterized in that: Execute step S1 to deploy the Chat-TTS model on the server. The specific deployment process is as follows: S1.

1. Prepare the server environment: Ensure that the computing resources and storage space of the server meet the operating requirements of the Chat-TTS model. At the same time, install the necessary operating system and dependent libraries; S1.

2. Download the Chat-TTS model: Obtain the Chat-TTS model from the official channel and decompress it to the specified directory on the server; S1.

3. Configure model parameters: According to the server hardware configuration and actual needs, adjust the parameters of the Chat-TTS model to optimize the speech synthesis effect; S1.

4. Deploy model: Deploy the Chat-TTS model to the server and ensure that the Chat-TTS model can run normally.

3. The natural speech reading method based on the Chat-TTS model according to claim 1, characterized in that: Execute step S2, the user uploads the novel text through the front-end browser and automatically divides it into chapters. The specific implementation operation is as follows: S2.1, upload the novel through the web page of the front-end browser; S2.

2. The front-end browser uses natural language processing technology to automatically divide the uploaded novel into chapters: first, the text content of the novel is read and the paragraph structure is analyzed; then, based on the chapter title and paragraph length, the chapter boundary is automatically identified, and a chapter directory is generated to achieve automatic chapter division.

4. The natural speech reading method based on the Chat-TTS model according to claim 1, characterized in that: Execute step S3. The front-end browser transmits the text data output in step S2 to the server using WebSocket. The server uses the Chat-TTS model to convert the novel text into a PCM format audio file of a specified length. The specific implementation operations are as follows: S3.1, the front-end browser uses the WebSocket protocol to establish a connection with the server to achieve real-time data transmission; S3.2, after the user selects a novel chapter, the front-end browser sends the selected text data to the server through WebSocket; S3.

3. The server caches the audio according to the duration specified by the user: First, the text data transmitted by the front-end browser is received, and then the Chat-TTS model is used to convert the text into a PCM format audio file of the specified duration, and finally the generated audio file is cached to the server.

5. The natural speech reading method based on the Chat-TTS model according to claim 1, characterized in that: Execute step S4, the front-end browser uses Jessibuca.js to play the audio in PCM format, while keeping the novel content and the audio synchronized. The specific implementation operations are as follows: Use Jessibuca.js player to play PCM audio files cached on the server; During playback, you can check the playback progress, adjust the playback speed and volume through audio playback control to meet the user's personalized needs; During the playback process, the audio playback progress is monitored and the scrolling position of the novel content is updated in real time; users are supported to click on the novel content to jump to the audio playback.

6. A natural speech reading system based on Chat-TTS model, characterized in that: It includes: Model deployment module, used to deploy the Chat-TTS model on the server side; The upload and segmentation module is used to assist users in uploading novel texts through the front-end browser and automatically divide them into chapters; The transmission calling module is used to transmit the text data output by the front-end browser to the server, and call the Chat-TTS model to convert the novel text into a PCM format audio file of a specified length; The cache transmission module is used to cache the PCM format audio files converted and output by the Chat-TTS model on the server side and transmit them to the front-end browser; An audio playback module that uses Jessibuca.js to play audio files in PCM format while keeping the novel content synchronized with the audio.

7. A natural speech reading system based on Chat-TTS model according to claim 6, characterized in that: The process of the model deployment module deploying the Chat-TTS model on the server side is as follows: Prepare the server environment: Ensure that the computing resources and storage space available on the server meet the operating requirements of the Chat-TTS model. At the same time, install the necessary operating system and dependent libraries. Download the Chat-TTS model: Get the Chat-TTS model from the official channel and decompress it to the specified directory on the server; Configure model parameters: adjust the parameters of the Chat-TTS model according to the server hardware configuration and actual needs to optimize the speech synthesis effect; Deploy the model: Deploy the Chat-TTS model to the server and ensure that the Chat-TTS model can run normally.

8. A natural speech reading system based on Chat-TTS model according to claim 6, characterized in that: The upload segmentation module specifically includes: An upload unit, used to assist users in uploading novel texts through a web page of a front-end browser; The segmentation unit is used to automatically divide the uploaded novel into chapters using natural language processing technology. The specific process includes: first reading the text content of the novel and analyzing the paragraph structure; then automatically identifying the chapter boundaries based on the chapter title and paragraph length, generating a chapter directory, and realizing automatic chapter division.

9. A natural speech reading system based on Chat-TTS model according to claim 6, characterized in that: The transmission calling module specifically includes: The connection unit is used to establish a connection between the front-end browser and the server using the WebSocket protocol to achieve real-time data transmission; The transmission unit is used to send the text content selected by the user to the server through WebSocket; The calling unit is used to call the Chat-TTS model deployed on the server. The Chat-TTS model converts the sent text content into a PCM format audio file of a specified length.

10. A natural speech reading system based on Chat-TTS model according to claim 6, characterized in that: The audio playback module includes: Audio playback unit, used to play PCM audio files using Jessibuca.js; Audio control unit, used to check and adjust the playback progress, speed and volume to meet the user's personalized needs; The audio monitoring unit is used to monitor the audio playback progress and update the scrolling position of the novel content in real time; The audio jump unit is used to realize the jump playback of audio according to the user's click operation.

Images