Text-to-voice conversion method and device

Abstract

The invention discloses a text-to-speech method and device, and relates to the technical field of artificial intelligence speech. The method comprises the following steps: training an audio sample, and generating a timbre feature model weight and a model vocoder; receiving a new timbre registration request containing user metadata, a path of timbre feature model weight and audio data, distributing a unique identifier for a new timbre, and setting an initial state as an inactive state; selecting a target timbre from the plurality of timbres, and setting the state of the target timbre as an activated state; receiving a synthesis request, and segmenting the text into ordered text segments; and loading the target timbre in the activated state, processing the text segments in sequence by using the timbre feature model weight corresponding to the target timbre, generating audio features corresponding to the text segments, converting the audio features into an audio data stream by using a model vocoder, and transmitting the audio data stream to a user. The problems that an existing speech synthesis method is long in generation time and does not support low-delay streaming playing are solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence speech technology, and in particular to a text-to-speech method and apparatus. Background Technology

[0002] Text-to-Speech (TTS), a key technology in the field of artificial intelligence, aims to naturally convert input text information into speech output in natural language.

[0003] Currently, existing speech synthesis systems like KAN-TTS typically generate a complete audio file all at once when converting text to speech. Specifically, after receiving the input text, the system uses a pre-trained speech synthesis model to comprehensively analyze and process the entire text content. The model then generates a corresponding complete audio data stream based on the text and saves it as an audio file. Users who want to receive the speech output must wait for the system to complete the entire audio file generation process and then download the file to their local device before playback, impacting the real-time interactive experience.

[0004] Therefore, existing speech synthesis methods suffer from long generation times and do not support low-latency streaming playback. Summary of the Invention

[0005] In this embodiment of the application, a text-to-speech method is provided, which solves the problems of long generation time and lack of support for low-latency streaming playback in existing speech synthesis methods.

[0006] In a first aspect, embodiments of this application provide a text-to-speech method, which includes: training audio samples to generate timbre feature model weights and a model vocoder; receiving a new timbre registration request containing user metadata, timbre feature model weights, a path, and audio data, assigning a unique identifier to the new timbre and setting its initial state to an inactive state; in application mode, selecting a target timbre from multiple timbres and setting the target timbre's state to an active state; receiving a synthesis request to segment the text into multiple ordered text segments; loading the target timbre in the active state, processing the text segments sequentially using the timbre feature model weights corresponding to the target timbre, generating audio features corresponding to the text segments, converting the audio features into an audio data stream using the model vocoder, and transmitting the audio data stream to the user.

[0007] One possible implementation also includes: stopping the synthesis request, releasing the currently occupied resources, and waiting for a new synthesis request after all text fragments have been processed or an interrupt signal for the synthesis request has been received.

[0008] In one possible implementation, the training of audio samples to generate timbre feature model weights and a model vocoder includes: labeling audio samples using a data annotation tool and associating audio samples with corresponding text information; organizing interval annotation information into a data catalog and recording abnormal samples separately; preprocessing audio samples to extract and normalize Mel spectrogram features to obtain key features of the audio samples; wherein, key features include frame-by-frame pitch and energy information; for audio samples with interval annotations, parsing out syllable-level durations; if the number of frames corresponding to the parsed syllable-level durations is inconsistent with the number of Mel frames obtained based on Mel spectrogram features, fine-tuning the end frame position of the audio sample; filtering out audio samples lacking key features, and obtaining... After obtaining the filtered audio samples, a training set and a validation set are generated from them. A distributed initialization method is used to train the audio samples based on the training set, allowing multiple computing nodes to participate in the training process simultaneously. The parameters of the timbre feature model are updated step by step according to the preset number of steps, and the changes in the loss value and learning rate in each training round are recorded. When the preset stopping condition is reached, the weights of the timbre feature model are generated. The timbre feature model is evaluated based on the changes in the loss value and learning rate, and gradient pruning and learning rate scheduling are performed during the training process. An adversarial training method is used to generate the model vocoder, distinguishing the starting steps and respective loss terms of the generator and discriminator of the model vocoder, and all losses are calculated only within the scope of valid frames.

[0009] In one possible implementation, receiving a new timbre registration request containing user metadata, a path to the timbre feature model weights, and audio data, assigning a unique identifier to the new timbre, and setting its initial state to inactive includes: the user metadata including a name and description set for the new timbre for identification and display on the user interface; the path to the timbre feature model weights pointing to the actual storage path of the timbre feature model weights and the model vocoder generated during the timbre feature model training process; and the audio data being a network link to a preview audio file stored in a file service system accessible externally.

[0010] In one possible implementation, the step of selecting a target timbre from multiple timbres in the application mode and setting the target timbre's state to an active state includes: the timbre management unit receiving a display instruction from the client, displaying all available timbres in the database according to the display instruction, and returning a complete set of information including a unique identifier, name, audio data, and timbre to the user interface; when the user selects a target timbre for real-time human-computer interaction, the client sends an update instruction containing the target timbre's unique identifier; after receiving the update instruction, the timbre management unit uniformly sets the states of all timbres in the database corresponding to the application mode to an inactive state, and sets the state of the target timbre selected by the user to an active state, ensuring that only one target timbre is active in any application mode.

[0011] In one possible implementation, upon receiving an interruption signal for the synthesis request, the synthesis request is stopped, the currently occupied resources are released, and a new synthesis request is awaited. This includes: providing an API endpoint; when the client or upper-layer business logic calls the API endpoint and sends an interruption signal for the synthesis request, changing the global status flag or directly terminating the synthesis of the current text; checking the global status flag before processing each new text fragment; if an interruption signal for the synthesis request is detected, stopping the synthesis request, releasing the currently occupied resources, and waiting for a new synthesis request.

[0012] Secondly, embodiments of this application provide a text-to-speech device, comprising: a training module for training audio samples and generating timbre feature model weights and a model vocoder; a setting module for receiving a new timbre registration request containing user metadata, timbre feature model weights, a path, and audio data, assigning a unique identifier to the new timbre, and setting its initial state to an inactive state; a selection module for selecting a target timbre from multiple timbres in an application mode, and setting the target timbre's state to an active state; a segmentation module for receiving a synthesis request and segmenting the text into multiple ordered text segments; and a conversion module for loading the target timbre in the active state, processing the text segments sequentially using the timbre feature model weights corresponding to the target timbre, generating audio features corresponding to the text segments, converting the audio features into an audio data stream using the model vocoder, and transmitting the audio data stream to the user.

[0013] Thirdly, embodiments of this application provide a text-to-speech server, including a memory and a processor; the memory is used to store computer-executable instructions; the processor is used to execute the computer-executable instructions to implement the method described in the first aspect or any possible implementation of the first aspect.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium storing executable instructions, which, when executed by a computer, enable the method described in the first aspect or any possible implementation thereof.

[0015] One or more technical solutions provided in the embodiments of this application have at least the following technical effects: This application provides a text-to-speech method that generates timbre feature model weights and a model vocoder by training audio samples. It receives a new timbre registration request containing user metadata, the path of the timbre feature model weights, and audio data, assigns a unique identifier to the new timbre, and sets its initial state to inactive. In application mode, a target timbre is selected from multiple timbres, and its state is set to active. A synthesis request is received, and the text is segmented into multiple ordered text segments. The active target timbre is loaded, and the text segments are processed sequentially using the timbre feature model weights corresponding to the target timbre to generate audio features corresponding to the text segments. The model vocoder is used to convert the audio features into an audio data stream, which is then transmitted to the user. This processing method breaks the traditional mode of generating a complete audio file at once, allowing users to obtain speech output in real time during processing without long waiting times, shortening the response time of speech synthesis, and meeting the requirements of real-time interactive scenarios. It solves the problems of long generation time and lack of support for low-latency streaming playback in existing speech synthesis methods. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of a text-to-speech method provided in an embodiment of this application; Figure 2 A schematic diagram of a text-to-speech device provided in an embodiment of this application; Figure 3 This is a schematic diagram of a text-to-speech server provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0019] The following description of some technologies involved in the embodiments of this application is provided to aid understanding and should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, some descriptions of well-known functions and structures are omitted in the following description.

[0020] This application provides a text-to-speech method, such as... Figure 1 As shown, the method includes steps S101 to S105. Wherein, Figure 1 This is merely one execution order shown in the embodiments of this application and does not represent the only execution order for a text-to-speech method. The execution order may vary depending on whether the final result can be achieved. Figure 1 The steps shown can be performed in parallel or in reverse order.

[0021] S101: Train audio samples to generate timbre feature model weights and model vocoder.

[0022] Train audio samples to generate timbre feature model weights and model vocoders, including the following:

[0023] Audio samples are labeled using data annotation tools, which associate the audio samples with the corresponding text information and annotate the start and end times of semantic units in the audio samples. The annotations of the start and end times are used as interval annotations.

[0024] Specifically, this application uses the KAN-TTS model for data annotation, training, and inference. Through the aforementioned annotation method, the corresponding text and time range of each semantic unit in the audio can be clearly identified, providing an accurate data foundation for subsequent model training. The data annotation tool used in this application can be the TTS-Autolabel automated data annotation tool.

[0025] The interval labeling information is organized into a data catalog, and abnormal samples are recorded separately.

[0026] Specifically, abnormal samples may be samples that do not meet the requirements due to collection problems, labeling errors, or other reasons. Recording them separately helps to handle or exclude these samples in a special way later.

[0027] The audio samples are preprocessed to extract and normalize Mel spectrum features, thereby obtaining key features of the audio samples. These key features include frame-by-frame pitch and energy information.

[0028] Specifically, audio sample preprocessing involves amplitude normalization and silence removal. Mel-spectral features effectively reflect the spectral characteristics of audio and are one of the key features commonly used in speech processing. Pitch and energy information reflect the pitch and intensity variations of audio samples.

[0029] For audio samples with interval labels, the syllable-level duration is parsed out. If the number of frames corresponding to the parsed syllable-level duration is inconsistent with the number of Mel frames obtained based on Mel spectrum features, the position of the end frame of the audio sample is fine-tuned.

[0030] Specifically, syllable-level duration can describe the duration of each syllable in audio in more detail. However, during parsing, there may be discrepancies between the number of frames corresponding to the parsed syllable-level duration and the number of Mel frames obtained based on Mel spectral features. In this case, it is necessary to fine-tune the position of the last frame of the audio sample to ensure consistency between the syllable-level duration and Mel spectral features in the time dimension, thereby improving the accuracy of model training. The position of the last frame can be the last three frames.

[0031] Filter out audio samples lacking key features to obtain filtered audio samples, and generate training and validation sets from the filtered audio samples.

[0032] Specifically, the training set is used for model training, and the validation set is used to evaluate and validate the model's performance during training. The training set can account for 80% of the model's performance, and the validation set can account for 20%.

[0033] A distributed initialization method is adopted to train audio samples based on the training set, allowing multiple computing nodes to participate in the training process simultaneously. The parameters of the timbre feature model are updated step by step according to the preset number of steps, and the changes in loss value and learning rate in each round of training are recorded.

[0034] Specifically, distributed initialization allows multiple computing nodes to participate in the training process simultaneously, fully utilizing computing resources and improving training efficiency. A computing node is the basic unit in distributed initialization training, referring to a physical or virtual device with independent computing capabilities (CPU / GPU / TPU). When dealing with small-scale samples (less than 10,000 samples), the preset number of steps can be 10,000 to 20,000. The parameters of the timbre feature model are updated progressively according to the preset number of steps. During each training step, the changes in the loss value and learning rate in each round of training are recorded. The loss value reflects the difference between the model's prediction and the true value, while the learning rate controls the step size of the model parameter updates.

[0035] When the preset stopping condition is reached, the weights of the timbre feature model are generated.

[0036] Specifically, the stopping condition can be set according to the actual situation, such as when the number of training rounds reaches a preset number.

[0037] The timbre feature model is evaluated based on the changes in loss value and learning rate, and gradient pruning and learning rate scheduling are performed during training.

[0038] Specifically, the purpose of the evaluation is to understand the model's performance and generalization ability in order to determine whether the model has achieved the expected results. During training, gradient clipping and learning rate scheduling are also performed to prevent problems such as overfitting. Gradient clipping can limit the magnitude of gradients, avoiding gradient explosion or gradient vanishing; learning rate scheduling can dynamically adjust the learning rate according to the training situation, improving the stability and effectiveness of training.

[0039] An adversarial training method is used to generate a model vocoder, which distinguishes the starting steps of the generator and the discriminator of the model vocoder and their respective loss terms, and all losses are calculated only within the effective frame range.

[0040] Specifically, adversarial training is an effective training method that can improve model performance. When generating a vocoder, it's necessary to distinguish the starting steps of the generator and discriminator, as well as their respective loss terms. The generator is responsible for generating audio data, while the discriminator is responsible for determining whether the generated audio data is authentic. Furthermore, all losses are calculated only within the valid frame range, allowing for a more accurate evaluation of the model's performance in the valid audio portion and improving the generation quality of the vocoder.

[0041] Furthermore, during the inference phase, an open network interface is used to enable interaction with external systems, and it is also responsible for loading pre-prepared resources and checkpoints obtained from model training. The input text is split into individual sentences based on specific delimiters, and then audio synthesis is performed on each sentence sequentially. Before audio synthesis, old files in the storage area are cleaned up to ensure a clean storage environment. After audio synthesis is complete, the generated files are saved according to predetermined naming rules to ensure standardized and consistent file naming. Finally, a notification is sent to external services to inform them that a new audio file has been generated.

[0042] S102: Receive a new timbre registration request containing user metadata, timbre feature model weights, path and audio data, assign a unique identifier to the new timbre and set its initial state to inactive.

[0043] Receive a new timbre registration request containing user metadata, timbre feature model weights, path and audio data, assign a unique identifier to the new timbre and set its initial state to inactive, including the following:

[0044] User metadata includes a name and description assigned to a new tone, used for identification and display on the user interface.

[0045] The path of the timbre feature model weights points to the actual storage path of the timbre feature model weights and the model vocoder generated during the timbre feature model training process.

[0046] The audio data is a network link to a preview audio file stored in a file service system that is accessible externally.

[0047] Specifically, this application implements structured recording and management of timbres, with recording fields covering multiple aspects of information: a unique identifier and display name used for interface display and command positioning; the network address of audio samples for quick playback and management by the front end; an activation status flag used to uniquely select timbre switching in different application modes; a path field for the weight of timbre feature models that associate different timbre synthesis with external or subsequent synthesis components; and creation and update timestamps for recording changes. When the front-end timbre configuration page is opened, it requests a complete timbre list from the back end, displaying the name, description, and sample audio in card format, and also supports creation, editing, deletion, and status switching operations.

[0048] S103: In application mode, select a target timbre from multiple timbres and set the target timbre to the active state.

[0049] In application mode, select a target tone from multiple tones and set the target tone to the active state, including the following.

[0050] The timbre management unit receives display instructions from the client, displays all available timbres in the database according to the display instructions, and returns a complete set of information including unique identifiers, names, audio data and timbres to the user interface.

[0051] When a user selects a target tone for real-time human-computer interaction, the client sends an update command containing a unique identifier for the target tone.

[0052] After receiving the update command, the timbre management unit sets the status of all timbres in the database corresponding to the application mode to the inactive state, and sets the status of the target timbre selected by the user to the active state, ensuring that only one target timbre is active in any application mode.

[0053] Specifically, in actual user operation scenarios, when a user wants to switch a certain timbre on the interface and make it the target timbre that is active in the current application mode, the front end will send a PUT / voices / :id / activate request to the back end, where :id represents the unique identifier of the selected target timbre.

[0054] Upon receiving the request, the controller forwards it to the management service for processing. The service layer employs a clear-then-set update strategy, completing the entire operation within a single transaction. The status of all sounds in the current application mode is uniformly set to False (inactive); only the selected target sound is set to True (active). After the transaction is successfully committed, the backend returns a success response. This update method ensures, at the data level, that only one record is always active in any application mode, effectively avoiding multiple active states due to concurrent requests. Upon receiving the success response from the backend, the frontend promptly refreshes the list data, ensuring that the active indicator displayed on the interface remains consistent with the final state on the backend.

[0055] S104: Receive a synthesis request and divide the text into multiple ordered text segments.

[0056] Specifically, a pre-built interface can be used to receive synthesis requests from clients, where the text to be converted is the text to be processed. Upon receiving the request, the received text is precisely segmented into multiple ordered text fragments according to a preset segmentation strategy, such as rules based on punctuation marks (e.g., periods, question marks, exclamation marks), semantic pause markers, or specific character lengths. These text fragments are arranged in the order they appear in the original text.

[0057] S105: Load the target timbre in the active state, process the text segment sequentially using the timbre feature model weights corresponding to the target timbre, generate the audio features corresponding to the text segment, use the model vocoder to convert the audio features into an audio data stream, and transmit the audio data stream to the user.

[0058] Specifically, this application possesses real-time text-to-speech capabilities, implemented using the WebSocket (streaming protocol) streaming synthesis interface. During real-time speech synthesis, the client only needs to send the text to be converted to the system's streaming synthesis interface, without specifying timbre information in the request. Upon receiving the request, the server proactively queries the timbre management unit to obtain the unique timbre record marked as active in the current application mode. After obtaining this record, the streaming synthesis unit extracts the path of the speech synthesis timbre feature model weights, loads the corresponding timbre feature model weights, then segments the received text and generates audio data segment by segment. Each generated audio segment is immediately sent to the client via the streaming protocol for immediate playback.

[0059] This application also includes: stopping the synthesis request, releasing the currently occupied resources, and waiting for a new synthesis request after all text fragments have been processed or an interrupt signal for the synthesis request has been received.

[0060] Upon receiving an interrupt signal for a synthesis request, stop the synthesis request, release the currently occupied resources, and wait for a new synthesis request, including the following:

[0061] Provide an API endpoint. When the client or upper-layer business logic calls the API endpoint and sends an interrupt signal for the synthesis request, change the global state flag or directly terminate the synthesis of the current text.

[0062] Before processing each new text segment, check the global state flags.

[0063] If an interruption signal is detected in the synthesis request, stop the synthesis request, release the currently occupied resources, and wait for a new synthesis request.

[0064] Specifically, if a client initiates an interrupt signal for a synthesis request via an API endpoint or sends an interrupt signal for a synthesis request via a WebSocket interface during a session, the server's interrupt control unit will act immediately. It will set the interrupt flag polled by the streaming synthesis loop to True, or directly send a cancellation signal to the thread / coroutine where the synthesis task resides. If necessary, it will also close the output queue to prevent further data transmission. Before proceeding to the next segment, it will read the interrupt flag at predetermined checkpoints. Once an interrupt signal is detected, it will exit the loop in an orderly manner according to the preset code path, stopping model inference and audio frame generation. Simultaneously, it will perform resource cleanup, including releasing currently occupied resources, reclaiming buffers, and closing transmission channels to ensure no residual data continues to be sent. After the termination operation is complete, it will return a confirmation message indicating that the process has stopped via an existing session channel or control endpoint, allowing the client interface to update its status and stop playback. The interrupt control unit receives stop commands (receives interrupt signals for synthesis requests) from clients or upper-layer business logic via the provided API endpoint, changes global status flags, or directly terminates the process, and implements interrupt handling based on these flags to ensure stable system operation and reasonable resource release.

[0065] This application also provides a text-to-speech device 200, such as... Figure 2 As shown, the device includes: a training module 201, a setting module 202, a selection module 203, a segmentation module 204, and a conversion module 205.

[0066] The training module 201 is used to train audio samples and generate timbre feature model weights and model vocoders.

[0067] The setting module 202 is used to receive new timbre registration requests containing user metadata, timbre feature model weights, paths, and audio data, assign a unique identifier to the new timbre, and set the initial state to an inactive state.

[0068] The selection module 203 is used to select a target timbre from multiple timbres in application mode and set the status of the target timbre to active state.

[0069] The segmentation module 204 is used to receive a synthesis request and segment the text into multiple ordered text fragments.

[0070] The conversion module 205 is used to load the target timbre in the active state, process the text segment in sequence using the timbre feature model weights corresponding to the target timbre, generate the audio features corresponding to the text segment, convert the audio features into an audio data stream using a model vocoder, and transmit the audio data stream to the user.

[0071] Some modules in the apparatus described in this application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0072] The apparatus or module described in the above embodiments can be implemented by a computer chip or physical entity, or by a product with a certain function. For ease of description, the above apparatus is described by dividing it into various modules according to their functions. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0073] The methods, apparatus, or modules described in this application can be implemented in a computer-readable program code manner. The controller can be implemented in any suitable manner, for example, as a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of a memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code manner, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included within it for implementing various functions can also be considered as structures within the hardware component. Alternatively, the device used to implement various functions can be viewed as either a software module that implements the method or a structure within a hardware component.

[0074] like Figure 3As shown, this application embodiment also provides a text-to-speech server, including a memory 301 and a processor 302; the memory 301 is used to store computer-executable instructions; the processor 302 is used to execute the computer-executable instructions to implement the text-to-speech method described above in this application embodiment.

[0075] This application also provides a computer-readable storage medium storing executable instructions, which, when executed by a computer, can implement the text-to-speech method described above in this application.

[0076] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, or it can be embodied in the process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in the embodiments of this application.

[0077] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this application can be used in numerous general-purpose or special-purpose computer system environments or configurations.

[0078] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of this application.

Claims

1. A method of text-to-speech, characterized by, The method comprises the following steps: training audio samples, generating timbre feature model weights and model vocoder; receiving a new timbre registration request containing user metadata, the path of timbre feature model weights, and audio data, assigning a unique identifier to the new timbre and setting the initial state to an inactive state; in the application mode, selecting a target timbre from multiple timbres and setting the state of the target timbre to an active state; receiving a synthesis request, splitting the text into multiple ordered text segments; loading the target timbre in the active state, processing the text segments in order using the timbre feature model weights corresponding to the target timbre, generating audio features corresponding to the text segments, converting the audio features into an audio data stream using the model vocoder, and transmitting the audio data stream to the user.

2. The text-to-speech method of claim 1, wherein, It also includes: when all text segments are processed or an interrupt signal is received, stop the synthesis request, release the currently occupied resources, and wait for a new synthesis request.

3. The text-to-speech method of claim 1, wherein, The method of training audio samples, generating timbre feature model weights and model vocoder comprises the following steps: annotating the audio samples using a data annotation tool, and associating the audio samples with the corresponding text information; organize the interval annotation information into a data directory, and record the abnormal samples separately; preprocessing the audio samples, extracting and normalizing the mel spectrum features, and obtaining the key features of the audio samples; wherein the key features include frame-by-frame pitch and energy information; for audio samples with interval annotations, parse the syllable-level duration, and if the number of frames corresponding to the syllable-level duration is inconsistent with the number of mel frames obtained based on the mel spectrum features, fine-tune the end frame position of the audio sample; filtering audio samples that lack key features to obtain filtered audio samples, and generating a training set and a validation set from the filtered audio samples; using a distributed initialization method to train the audio samples based on the training set, allowing multiple computing nodes to participate in the training process simultaneously, gradually updating the parameters of the timbre feature model according to the preset number of steps, and recording the loss value and learning rate changes of each training round; when the preset stopping condition is reached, generate the timbre feature model weights; based on the changes in loss value and learning rate, evaluate the timbre feature model and perform gradient clipping and learning rate scheduling during the training process; generate the model vocoder using the adversarial training method, distinguish the starting steps of the generator and the discriminator of the model vocoder, and calculate all losses only within the effective frame range.

4. The text-to-speech method of claim 1, wherein, The method of receiving a new timbre registration request containing user metadata, the path of timbre feature model weights, and audio data, assigning a unique identifier to the new timbre and setting the initial state to an inactive state, comprises the following steps: The user metadata includes the name and description set for the new timbre, which is used for identification and display on the user interface; The path of timbre feature model weights points to the actual storage path of the timbre feature model weights and the model vocoder generated during the training process of the timbre feature model; The audio data is a network link pointing to a preview audio file, which is stored in a file service system accessible to the outside.

5. The text-to-speech method of claim 4, wherein, The selecting a target timbre from the plurality of timbres and setting a state of the target timbre to an active state in the application mode comprises: The timbre management unit receives a display instruction of the client, displays all available timbres in the database according to the display instruction, and returns a complete information set containing a unique identifier, a name, audio data and a timbre to the user interface; When the user selects a target timbre for real-time human-computer interaction, the client sends an update instruction containing a unique identifier of the target timbre; After the timbre management unit receives the update instruction, it sets the state of all timbres in the database in the application mode to an inactive state, and sets the state of the target timbre selected by the user to an active state, ensuring that only one target timbre is in the active state in any application mode.

6. The text-to-speech method of claim 2, wherein, After receiving the interruption signal of the synthesis request, stop the synthesis request, release the currently occupied resources, and wait for a new synthesis request, comprising: Providing an API endpoint, when the client or upper business logic calls the API endpoint and sends an interruption signal of the synthesis request, changing the global state flag or directly terminating the synthesis of the current text; Before processing each new text segment, check the global state flag; If an interruption signal of the synthesis request is found, stop the synthesis request, release the currently occupied resources, and wait for a new synthesis request.

7. A text-to-speech apparatus, characterized by Comprising: A training module for training audio samples, generating timbre feature model weights and model vocoders; A setting module for receiving a new timbre registration request containing user metadata, paths of timbre feature model weights and audio data, assigning a unique identifier to the new timbre and setting the initial state to an inactive state; A selection module for selecting a target timbre from a plurality of timbres in the application mode, and setting the state of the target timbre to an active state; A segmentation module for receiving a synthesis request and segmenting text into a plurality of ordered text segments; A conversion module for loading a target timbre in an active state, processing text segments in order using timbre feature model weights corresponding to the target timbre, generating audio features corresponding to the text segments, converting the audio features into an audio data stream using a model vocoder, and transmitting the audio data stream to the user.

8. A text-to-speech server, characterized by Comprising a memory and a processor; The memory is used to store computer executable instructions; The processor is used to execute the computer executable instructions to implement the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores executable instructions, and the computer executes the executable instructions to implement the method of any one of claims 1-6. The computer readable storage medium stores executable instructions, and the computer executes the executable instructions to implement the method of any one of claims 1-6.

Images