Speech generation model training method, speech generation method, and system

Abstract

The application discloses a speech generation model training method, relates to the field of speech synthesis, and comprises the following steps: inputting target text into a context semantic analysis model to generate labeled text, dividing the labeled text into multiple text segments, obtaining multiple candidate speeches corresponding to each text segment through a teacher speech model and a reference speech, determining a preferred sample according to the multiple candidate speeches, determining a non-preferred sample through unlabeled target text, converting the preferred sample and the non-preferred sample into corresponding discrete word sequences, and performing supervised training and preference training on a target speech generation model based on the discrete speech word sequences and the labeled text; in the reinforcement optimization stage, the same labeled text and the reference speech are input into the trained target speech generation model to generate multiple candidate speech sequences, and a group relative strategy optimization method is used to further update the target speech generation model. The application can realize accurate word-level prosody control and maintain naturalness without manual annotation.

Description

Technical Field

[0001] This invention relates to the field of speech synthesis technology, specifically to a method and system for training speech generation models. Background Technology

[0002] Text-to-speech systems have evolved from traditional modular architectures to unified generative frameworks based on large language models. These frameworks can utilize discrete acoustic lexical units and textual lexical units for joint modeling, achieving zero-sample timbre replication and long speech generation. However, applications such as advertising voice-over, audio content, and chatbots require not only global style control but also fine-grained prosodic control, such as stress, weak forms, fast pronunciation, and slow pronunciation, on specific words or phrases. While existing models excel at sentence-level or document-level style transfer, they struggle to stably achieve word-level prosodic adjustment. This is primarily due to the scarcity of prosodic data precisely aligned to the word level and the high cost of manual annotation. There is an inherent conflict between local prosodic constraints and autoregressive continuous generation mechanisms. Furthermore, existing training methods focus on label imitation or single acoustic metrics, making it difficult to balance local controllability with overall naturalness.

[0003] Existing technologies, represented by the CosyVoice series, can achieve zero-sample timbre replication and overall speech rate and emotion control. However, when embedding word-level prosodic tags in the text, the model often ignores the tags or causes the control effect to spread, resulting in abrupt pauses, prosodic discontinuities, and timbre drift. Another type of solution, such as FastSpeech 2 and FastPitch, controls speech by predicting acoustic parameters such as pitch and duration. However, these rely on manual annotation, and fixed thresholds are difficult to adapt to different contexts, easily degenerating into mechanical stretching or compression. Neither type of solution can simultaneously meet the requirements of word-level prosodic control accuracy and overall speech naturalness.

[0004] The root cause of the above bottlenecks is that: supervised fine-tuning can only mimic the correspondence between labels and speech, but cannot effectively punish problems such as label ignoring and control overstepping; overemphasizing local control will destroy the global acoustic prior of the pre-trained model, resulting in timbre drift and rhythm instability; and the model's self-generated training data will amplify its own defects. Summary of the Invention

[0005] This invention aims to address, to a certain extent, one of the technical problems in related technologies. To this end, this invention provides a speech generation model training method that, while achieving word-level prosodic control, maintains speech intelligibility, timbre consistency, and overall naturalness, thus solving the problem of timbre drift and rhythmic instability caused by overemphasizing local control.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for training a speech generation model, the method comprising:

[0008] Obtain the target text, input the target text into the context semantic analysis model, and generate tagged text, wherein the tags are used to annotate prosodic control requirements;

[0009] The labeled text is divided into multiple text segments according to the label boundaries and punctuation boundaries. Multiple candidate voices corresponding to each text segment are obtained through the teacher's voice model and reference voice. Preference samples are determined based on the multiple candidate voices, and non-preference samples are determined through the unlabeled target text.

[0010] The preferred samples and the non-preferred samples are converted into corresponding discrete word sequences, and the target speech generation model is trained based on the discrete word sequences and the labeled text.

[0011] The target speech generation model, trained using reference speech as speaker prompt input, generates multiple candidate speech sequences for the same labeled text, calculates the comprehensive reward for each candidate speech sequence, and optimizes and updates the target speech generation model based on the comprehensive reward using a group-relative strategy.

[0012] Optionally, obtaining multiple candidate voices corresponding to each text segment through the teacher's voice model and reference voice includes:

[0013] The teacher's speech model is used to generate each text segment through multiple rounds of autoregression. For the first text segment, it is generated directly. For the second and subsequent text segments, an acoustic word segment from the end of the previous text segment is extracted as the autoregression generation prefix of the current text segment, so that an acoustic bridge is formed between adjacent text segments.

[0014] By using an acoustic bridging mechanism to use the acoustic lexical unit at the end of the previous segment as a prefix for the next segment, the pitch, energy, and rhythm information between adjacent segments can be naturally continued. This avoids splicing breaks and abrupt pauses caused by independent generation, and significantly improves the overall coherence and naturalness of the segmented generated speech.

[0015] Optionally, determining preferred samples based on the plurality of candidate speech and determining non-preferred samples through unlabeled target text includes:

[0016] The candidate speech samples are filtered according to the label type to obtain the label-consistent speech samples, and the label-consistent speech samples are determined as the preferred samples.

[0017] The baseline speech of the target text is generated using the teacher's speech model, and the baseline speech is identified as a non-preference sample.

[0018] By automatically constructing preference sample pairs that reflect the comparison between "meeting and not meeting prosodic control requirements" using preference and non-preference samples, key training signals are provided for subsequent direct preference optimization training. These signals can be directly used to distinguish between good and bad and guide the model to learn the correct prosodic mapping.

[0019] Optionally, the tag type includes any one or more of strong reading tags, weak reading tags, fast reading tags, and slow reading tags; the step of filtering the multiple candidate speech based on the tag type to obtain tag-consistent speech includes:

[0020] If the text segment contains the strong read label or the weak read label, extract the speaker embedding of the reference speech, calculate the cosine similarity between each candidate speech of the text segment and the speaker embedding of the reference speech, and select the candidate speech with the highest cosine similarity as the label-consistent speech.

[0021] When the text segment contains the fast reading tag, the candidate speech with the shortest speech word sequence or the shortest non-silence duration is selected as the tag-consistent speech.

[0022] When the text segment contains the slow reading tag, the candidate speech with the longest speech word sequence or the longest non-silence duration is selected as the tag-consistent speech.

[0023] When a text segment contains multiple tags, first set a timbre similarity threshold, and then sort the candidate speech according to duration to select the speech with the same tag among the candidate speech that meets the timbre similarity threshold, or calculate a comprehensive score based on timbre similarity, speech duration, intelligibility and naturalness to select the speech with the same tag.

[0024] Differentiated screening strategies are adopted for different tag types to accurately match various rhythm control targets while taking into account timbre stability and overall quality.

[0025] Optionally, the discrete word sequence includes preferred sample words and unpreferred sample words, and the training of the target speech generation model based on the discrete word sequence and the labeled text includes:

[0026] Calculate the supervised loss based on the labeled text and the preferred sample terms;

[0027] The preferred sample lexical and the non-preferred sample lexical are respectively input into the current policy model and the frozen reference model, and the conditional log-likelihood ratio of the current policy model relative to the frozen reference model is calculated. The current policy model is the target speech generation model being optimized, and the frozen reference model is a copy of the target speech generation model with fixed parameters.

[0028] The preference loss is obtained by expanding the difference in the conditional log-likelihood ratio between the preferred sample word and the non-preferred sample word using the preference loss coefficient.

[0029] The weighted sum of the supervision loss and the preference loss is used as the total loss for model training.

[0030] By establishing a basic mapping between labels and acoustic performance through supervised loss, and by using preference loss to amplify the relative probability difference between preferred and non-preferred samples, the model can learn to distinguish between good and bad samples based on imitating correct samples, thereby effectively suppressing local prosodic control failures such as label ignoring and control overstepping.

[0031] Optionally, calculating the comprehensive reward for each candidate speech sequence separately includes:

[0032] Speech recognition is performed on the candidate speech sequence to obtain the recognized text. The punctuation marks in the recognized text and the target text are uniformly mapped to boundary words. The character error rate containing pause boundary information is calculated based on the boundary words. The pause perception recognition reward is determined according to the character error rate.

[0033] The candidate speech sequence is input into the emotion recognition model to obtain the posterior probability of emotion. Based on the posterior probability of emotion, the decoupled emotion reward is obtained by taking neutral emotion as the anchor point and combining the dominant emotion and retention coefficient of the reference speech.

[0034] The baseline duration is calculated based on the number of text characters or phonemes and the natural silence budget. The actual duration is obtained based on the non-silence duration of the candidate speech sequence after speech activity detection. The direction coefficient is determined based on the speed control label. A directional duration reward is obtained based on the baseline duration, the actual duration, and the direction coefficient.

[0035] The comprehensive reward is obtained by weighting the pause perception recognition reward, the decoupled emotion reward, and the directional duration reward.

[0036] By using three dimensions—pause perception, emotion decoupling, and duration direction—the intelligibility and pause rationality of the generated speech, global emotional stability and local strength and weakness comparison, and speech rate control direction are quantitatively evaluated and weighted and integrated. This enables the comprehensive reward to accurately evaluate the quality of candidate speech from multiple perspectives, guiding reinforcement learning to improve the accuracy of local prosodic control while taking into account the overall naturalness.

[0037] Optionally, the step of optimizing and updating the target speech generation model using a group-relative strategy based on the comprehensive reward includes:

[0038] The comprehensive reward is standardized within the candidate group corresponding to the same labeled text condition to obtain the relative advantage of each candidate speech sequence relative to the mean within the group.

[0039] Based on the relative advantage, calculate the policy gradient of the relative policy optimization group, and update the parameters of the current policy model;

[0040] During the update process, strategy pruning is incorporated to limit the step size of a single update. The word-level KL divergence between the current strategy model and the frozen reference model is calculated, and the word-level KL divergence is added as a penalty term to the optimization objective to constrain the offset of the current strategy model relative to the frozen reference model during the optimization of the group of relative strategies to not exceed a preset offset threshold.

[0041] By standardizing intra-group rewards and controlling the step size of each update through policy pruning, and by introducing word-level KL divergence penalty with the frozen reference model as the anchor point, the optimization process of reinforcement learning balances the improvement of local prosodic control accuracy with the preservation of global acoustic priors.

[0042] Furthermore, the present invention also provides a speech generation method, the method comprising:

[0043] Obtain the text to be synthesized and a reference speech, wherein the text to be synthesized contains word-level or fragment-level prosodic control tags;

[0044] The text to be synthesized and the reference speech are input into the target speech generation model to determine the synthesized speech of the text to be synthesized. The target speech generation model is trained based on the speech generation model training method.

[0045] This method directly calls the target speech generation model that has been trained and optimized through multiple stages. Without the need for large-scale manual annotation, it can synthesize speech that accurately responds to local prosodic control requirements while maintaining timbre consistency and overall naturalness.

[0046] Furthermore, the present invention also provides a speech generation model training system, the system comprising:

[0047] The tag generation module is used to acquire target text, input the target text into the context semantic analysis model, and generate tagged text, wherein the tags are used to annotate prosodic control requirements;

[0048] The training sample construction module is used to divide the labeled text into multiple text segments according to the label boundaries and punctuation boundaries, obtain multiple candidate voices corresponding to each text segment through the teacher's voice model and reference voice, determine the preferred samples based on the multiple candidate voices, and determine the non-preferred samples through the unlabeled target text.

[0049] The training module is used to convert the preferred samples and the non-preferred samples into corresponding discrete word sequences, and to train the target speech generation model based on the discrete word sequences and the labeled text.

[0050] The model optimization module is used to train the target speech generation model using the reference speech as the speaker prompt input, generate multiple candidate speech sequences for the same labeled text, calculate the comprehensive reward for each candidate speech sequence, and optimize and update the target speech generation model based on the comprehensive reward using a group relative strategy.

[0051] The reasoning process for the beneficial effects of the speech generation model training system provided by this invention is similar to that of the aforementioned speech generation model training method, and will not be repeated here.

[0052] Furthermore, the present invention also provides a speech generation system, the system comprising:

[0053] The acquisition module is used to acquire the text to be synthesized and the reference speech, wherein the text to be synthesized contains word-level or fragment-level prosodic control tags;

[0054] The synthesis module is used to input the text to be synthesized and the reference speech into the target speech generation model to determine the synthesized speech of the text to be synthesized. The target speech generation model is trained based on the speech generation model training method.

[0055] The reasoning process for the beneficial effects of the speech generation system provided by this invention is similar to that of the aforementioned speech generation methods, and will not be repeated here.

[0056] Furthermore, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the speech generation model training method or speech generation method described above.

[0057] In addition, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the speech generation model training method or speech generation method described above.

[0058] These features and advantages of the present invention will be disclosed in detail in the following specific embodiments and accompanying drawings. The preferred embodiments or means of the present invention will be shown in detail in conjunction with the accompanying drawings, but are not intended to limit the technical solutions of the present invention. In addition, each of these features, elements and components appearing in the following text and drawings is a plurality of, and different symbols or numbers are used for convenience of representation, but all represent parts with the same or similar construction or function. Attached Figure Description

[0059] The present invention will be further described below with reference to the accompanying drawings:

[0060] Figure 1 This is a flowchart of a speech generation model training method according to an embodiment of this application;

[0061] Figure 2 This is a flowchart illustrating the construction of a preference sample pair according to an embodiment of this application;

[0062] Figure 3 This is a flowchart of a joint supervision and preference training process according to an embodiment of this application;

[0063] Figure 4 This is a flowchart of a multi-dimensional reward enhancement optimization process according to an embodiment of this application;

[0064] Figure 5 This is a flowchart illustrating the overall process of a fine-grained prosodic controllable speech synthesis method based on multi-stage preference alignment, according to an embodiment of this application. Detailed Implementation

[0065] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described are intended to explain the present invention and should not be construed as limiting the invention.

[0066] The terms "an embodiment," "example," or "trademark" used in this specification refer to a particular feature, structure, or characteristic described in connection with the embodiment itself that may be included in at least one embodiment disclosed in this patent. The phrase "in an embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment.

[0067] Definitions of abbreviations and key terms

[0068] TTS (Text-to-Speech): A technology that converts text into speech waveforms.

[0069] Zero-shot speech synthesis: The model synthesizes speech with corresponding timbre based on a speaker reference speech without additional training for the target speaker.

[0070] Text lexical units: Sequence units obtained by discretizing synthetic text through the text segmenter of a large language model. They can be used to represent characters, punctuation marks, and prosody control tags.

[0071] Acoustic lexicons: speech representations obtained by discretization by a speech encoder or audio segmenter and that can be predicted autoregressively by a generative model.

[0072] Pairs of preferred samples: Pairs of preferred and non-preferred samples formed for the same input conditions, used to train the model to improve the probability of selecting the target behavior.

[0073] SFT (Supervised Fine-Tuning): Minimizes the negative log-likelihood using conditional target speech samples, enabling the model to establish a fundamental mapping between control labels and local acoustic performance.

[0074] DPO (Direct Preference Optimization): Directly expands the conditional likelihood difference between preferred and unpreferred samples by using preferred samples, without needing to train a separate explicit reward model.

[0075] GRPO (Group Relative Policy Optimization): A reinforcement learning method that generates multiple candidates for the same input, calculates the advantage based on the relative reward within the group, and updates the generation policy.

[0076] ASR (Automatic Speech Recognition): Used to recognize generated speech and evaluate text intelligibility and pause boundary consistency.

[0077] CER (Character Error Rate): The character-level editing error rate of the identified text relative to the target text.

[0078] Speaker embedding: A vector extracted from speech by a speaker representation model, used to evaluate the timbre similarity between candidate speech and reference speech.

[0079] Global acoustic priors: The overall generation rules of coherent pronunciation, timbre, rhythm and naturalness formed by the pre-trained speech generation model.

[0080] Example:

[0081] This embodiment provides a method for training a speech generation model. Figure 1 This is a flowchart of a speech generation model training method according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps:

[0082] Step S101: Obtain the target text, input the target text into the context semantic analysis model, and generate tagged text, where the tags are used to annotate the prosodic control requirements.

[0083] Contextual semantic analysis models are large language models (such as DeepSeek-V3) that can understand the semantics, syntax, and logical weight of an entire sentence and automatically determine which words should be stressed, which should be unstressed, or read quickly or slowly. A large-scale unlabeled text corpus is acquired, and the text is input into this large language model for contextual semantics and prosodic analysis. Strong, weak, fast, and slow pronunciation labels are then inserted into the ordinary text.

[0084] Taking DeepSeek-V3 as an example, prompts allow it to determine, based on language expression and semantic focus, which words or phrases need to be emphasized, weakened, accelerated, or decelerated:

[0085] <strong> words or phrases< / strong> Enhance local accents and intensity;

[0086] <weak> words or phrases< / weak> Reduce local accents and intensity;

[0087] <fast> words or phrases< / fast> Increase the speed of speech in certain areas;

[0088] <slow> words or phrases< / slow> Slow down the pace of speech in certain areas.

[0089] For example, the output structured text is: <strong> Alvarez< / strong> Act in accordance with instructions.

[0090] It should be noted that prosody control tags are not limited to strong, weak, fast, and slow pronunciations, but may also include pitch rise and fall, range, energy, pauses, rhythm, emotion, dialect, pronunciation style, or combinations thereof; tags may be in XML, JSON, special tokens, numerical vectors, or natural language instructions.

[0091] Step S102: Divide the labeled text into multiple text segments according to the label boundaries and punctuation boundaries. Obtain multiple candidate voices corresponding to each text segment through the teacher's voice model and reference voice. Determine the preferred samples based on the multiple candidate voices and determine the non-preferred samples through the unlabeled target text.

[0092] This embodiment utilizes a teacher speech model to generate speech with corresponding prosodic characteristics. The teacher speech model is an external speech synthesizer independent of the target model. It is responsible for accurately generating demonstration speech with prosodic labels, serving as the "standard answer" for the target model's learning, thus preventing the target model from amplifying its own shortcomings when generating data itself. During the generation process, the acoustic lexical units at the end of the previous text segment are used as prefixes for the next text segment, ensuring acoustic continuity between different prosodic segments. Subsequently, the target speech is selected from multiple candidates based on speaker similarity or speech lexical unit length, and paired with baseline speech without corresponding prosodic labels to form a preferred sample pair.

[0093] It should be noted that the teacher's speech model can be an autoregressive, non-autoregressive, diffusion, stream matching, or neural codec architecture; acoustic bridging can use discrete acoustic tokens, as well as Mel frames, continuous latent variables, hidden states, or waveform segments.

[0094] Step S103: Convert the preferred and unpreferred samples into corresponding discrete word sequences, and train the target speech generation model based on the discrete word sequences and labeled text.

[0095] The target speech generation model is trained using joint supervised fine-tuning and direct preference optimization. Supervised fine-tuning establishes a fundamental mapping between prosodic labels and local acoustic representations; direct preference optimization increases the generation probability of label-consistent speech relative to the baseline speech, thereby reducing label ignoring, label overreach, and local control failures. In this embodiment, the target speech generation model is a TTS (Text-to-Speech) large language model.

[0096] Step S104: Using the reference speech as the speaker prompt input to train the target speech generation model, generate multiple candidate speech sequences for the same labeled text, calculate the comprehensive reward for each candidate speech sequence, and optimize and update the target speech generation model based on the comprehensive reward using a group relative strategy.

[0097] For the same input, multiple candidate speech are generated, and rewards are calculated from three dimensions: text intelligibility and pause boundaries, global sentiment and local intensity, and baseline duration and local speech rate. The model is further updated through group relative policy optimization, and policy pruning and word-level KL constraints are used to limit the model's deviation relative to the pre-trained model.

[0098] Through steps S101-S104, the text is automatically labeled with prosodic tags using a contextual semantic analysis model. Combined with a teacher model, labeled preferred samples and unlabeled non-preferred samples are generated, fundamentally solving the problems of high cost and poor consistency in fine-grained prosodic data annotation. The target model is trained based on discrete word sequences, establishing a fundamental mapping between prosodic tags and acoustic performance. During the reinforcement learning phase, multiple candidate generation under the same conditions and comprehensive reward evaluation, along with group-based relative strategy optimization, update the model, improving local prosodic control accuracy while ensuring overall naturalness and timbre consistency through multi-dimensional reward design. The entire method achieves stable and precise prosodic control while reducing data dependence.

[0099] The method in this embodiment operates on a basic speech synthesis backbone network. Optionally, the backbone network uses CosyVoice2, which includes a text encoder, a speech encoder, a TTS large language model, an acoustic decoder, and a vocoder. In the speech synthesis stage, tagged text is input into the text encoder to obtain text units; reference speech is input into the speech encoder to obtain reference speaker embeddings; the TTS large language model generates target speech units based on the text units and the reference speaker embeddings; the acoustic decoder converts the target speech units into Mel spectra; and the vocoder generates speech waveforms based on the Mel spectra. The target speech synthesis model determined by this training method can generate speech with target timbre and specified local prosody based on tagged text and reference speech, maintaining overall naturalness and timbre consistency while improving word-level prosodic control accuracy.

[0100] In this embodiment, training and inference can be performed on local devices, cloud servers, or edge devices; acoustic decoders and vocoders can be combined into an end-to-end decoding network; alternatively, only acoustic tokens can be output for downstream systems to decode.

[0101] Optionally, multiple candidate voices corresponding to each text segment can be obtained through the teacher's speech model and reference speech, including:

[0102] The teacher's speech model is used to generate each text segment through multiple rounds of autoregression. The first text segment is generated directly, while for the second and subsequent text segments, an acoustic word segment from the end of the previous text segment is extracted as the autoregression prefix for the current text segment, so that an acoustic bridge is formed between adjacent text segments.

[0103] The tagged text is divided into multiple sequentially arranged text segments based on prosodic control of label and punctuation boundaries. A teacher speech model (e.g., the IndexTTS2 model) is then used to perform multiple rounds of autoregressive generation on each text segment. The first text segment is generated directly; for the second and subsequent text segments, a segment of speech words from the end of the previous segment is extracted and used as the autoregressive generation prefix for the current segment. This word-level continuous generation mechanism forms an acoustic bridge between adjacent segments, allowing the pitch, energy, timbre, and rhythm information of the previous segment to be passed to the next segment, reducing abrupt pauses and boundary breaks caused by independent generation and direct splicing.

[0104] Optionally, determining preferred samples based on multiple candidate speech and determining non-preferred samples using unlabeled target text includes:

[0105] Multiple candidate speech samples are filtered based on label type to obtain speech samples with consistent labels, which are then identified as preferred samples.

[0106] The baseline speech of the target text is generated using a teacher speech model, and the baseline speech is identified as a non-biased sample.

[0107] Figure 2 This is a flowchart of a preference sample pair construction process according to an embodiment of this application, such as... Figure 2 As shown, by filtering multiple candidate speech samples according to label type, speech samples with consistent labels are obtained as preferred samples, and the baseline speech of unlabeled text is used as non-preferred samples. In this way, preferred sample pairs reflecting the comparison between "meeting and not meeting prosodic control requirements" are automatically constructed, providing key training signals that can be directly used to distinguish between good and bad and guide the model to learn the correct prosodic mapping for subsequent direct preference optimization training.

[0108] Optionally, the label type includes any one or more of strong reading labels, weak reading labels, fast reading labels, and slow reading labels; multiple candidate speech samples are filtered according to the label type to obtain speech samples with consistent labels, including:

[0109] When a text segment contains strong or weak reading labels, the speaker embedding of the reference speech is extracted, the cosine similarity between each candidate speech segment and the speaker embedding of the reference speech is calculated, and the candidate speech with the highest cosine similarity is selected as the label-consistent speech.

[0110] When a text segment contains a fast-reading tag, the candidate speech with the shortest speech word sequence or the shortest non-silence duration is selected as the tag-consistent speech.

[0111] When a text segment contains a slow reading label, the candidate speech with the longest speech word sequence or the longest non-silence duration is selected as the label-consistent speech.

[0112] When a text segment contains multiple tags, first set a timbre similarity threshold, then sort the candidate speech that meets the timbre similarity threshold by duration and select the speech with the same tag, or calculate a comprehensive score based on timbre similarity, speech duration, intelligibility and naturalness and select the speech with the same tag.

[0113] For each text segment, generate N candidate speech samples, and use different candidate selection methods based on the label type:

[0114] For strong or weak reading labels, speaker embeddings of each candidate speech and reference speech are extracted, and the cosine similarity between the two is calculated. The candidate with the highest similarity is selected to reduce timbre drift caused by local changes in strength.

[0115] For fast reading tags, the speech word length or non-silence duration generated by autoregression is used as an indicator of speech rate, and the candidate with the shortest word sequence or the shortest non-silence duration is selected; for slow reading tags, the candidate with the longest word sequence or the longest non-silence duration is selected.

[0116] When the same segment contains multiple control targets, you can first set a speaker similarity threshold, and then sort the candidates that meet the threshold by duration.

[0117] A comprehensive score can also be calculated based on speaker similarity, duration, intelligibility, and naturalness. For example, speaker embeddings of candidate and reference speech are extracted, and cosine similarity is calculated as timbre similarity score to measure timbre consistency; the duration scoring direction is determined according to the label type—if a fast reading label is included, the shorter the actual non-silence duration, the higher the score; if a slow reading label is included, the longer the actual non-silence duration, the higher the score, thus quantifying the compliance of speech rate control; the candidate speech is input into an Automatic Speech Recognition (ASR) model to obtain the recognized text, and the character error rate with the target text is calculated, with 1 minus the character error rate as the intelligibility score; a pre-trained speech quality assessment model or MOS predictor is used to score the candidate speech as the naturalness score. The scores of the above four dimensions are multiplied by the corresponding weight coefficients (the sum of the weights is 1, which can be dynamically adjusted according to the application scenario) and summed to obtain the comprehensive score of each candidate speech. Finally, the candidate speech with the highest comprehensive score is selected as the label-consistent speech, so as to take into account the stability of timbre, the precision of speech rate control, the clarity of content and the naturalness of listening when multiple labels coexist.

[0118] This embodiment employs a differentiated screening strategy for different tag types—strong / weak reading tags prioritize timbre similarity to prevent timbre drift, while fast / slow reading tags select the fastest or slowest candidates based on the length of the speech word sequence or the extreme value of non-silence duration. When multiple tags coexist, thresholds or comprehensive scores are used to balance multiple requirements such as timbre and duration, thereby ensuring both accurate matching of various prosodic control targets and timbre stability and overall quality.

[0119] Optionally, the discrete word sequence includes preferred and unpreferred sample words. Training the target speech generation model based on the discrete word sequence and labeled text includes:

[0120] The supervised loss is calculated based on the labeled text and preferred sample terms.

[0121] The preferred sample words and unpreferred sample words are input into the current policy model and the frozen reference model, respectively. The conditional log-likelihood ratio of the current policy model to the frozen reference model is calculated. The current policy model is the target speech generation model that is being optimized, and the frozen reference model is a copy of the target speech generation model with fixed parameters.

[0122] The preference loss is obtained by expanding the difference in the conditional log-likelihood ratio between preferred sample words and unpreferred sample words using the preference loss coefficient.

[0123] The weighted sum of the supervision loss and the preference loss is used as the total loss for model training.

[0124] Will prefer voice y w The unpreferred speech y1 is input into the speech encoder and converted into the corresponding discrete speech word sequence.

[0125] First, supervised fine-tuning is performed to establish a basic mapping between strong, weak, fast, and slow pronunciation labels and their corresponding local speech lexical distributions in the TTS large language model. Using labeled text conditions and preferred speech lexical units as training data, the supervised loss L is calculated. SFT :

[0126] ;

[0127] in, This represents the probability that the current model generates complete preferred speech lexical units under condition x.

[0128] Then, direct preference optimization is performed. Preferred and unpreferred speech terms are input into the current policy model and the frozen reference model, respectively. Their relative conditional generation probabilities are calculated, and the probabilistic advantage of preferred samples over unpreferred samples is amplified through preference loss.

[0129] ;

[0130] Among them, L DPO For preference loss, The current strategy is relative to the frozen reference strategy. coefficient Scaling log-likelihood ratio.

[0131] The joint training objective is:

[0132] ;

[0133] Among them, L total λ represents the total loss during model training, and λ is the balance coefficient between the supervision loss and the preference loss.

[0134] This embodiment establishes a basic mapping between labels and acoustic performance through supervised loss, while using preference loss to amplify the relative probability difference between preferred and non-preferred samples. This enables the model to learn to distinguish between good and bad samples based on imitating correct samples, thereby effectively suppressing local prosodic control failures such as label ignoring and control overstepping. Figure 3 This is a flowchart of a joint supervision and preference training process according to an embodiment of this application.

[0135] Optionally, the comprehensive reward for each candidate speech sequence can be calculated separately, including:

[0136] Speech recognition is performed on candidate speech sequences to obtain recognized text. Punctuation marks in the recognized text and target text are uniformly mapped to boundary words. The character error rate containing pause boundary information is calculated based on the boundary words, and the pause perception recognition reward is determined according to the character error rate.

[0137] The candidate speech sequence is input into the emotion recognition model to obtain the posterior probability of emotion. Based on the posterior probability of emotion, the neutral emotion is used as the anchor point, and the dominant emotion and retention coefficient of the reference speech are combined to obtain the decoupled emotion reward.

[0138] The baseline duration is calculated based on the number of text characters or phonemes and the natural silence budget. The actual duration is obtained based on the non-silence duration of the candidate speech sequence after speech activity detection. The directional coefficient is determined based on the speed control label. The directional duration reward is obtained based on the baseline duration, the actual duration, and the directional coefficient.

[0139] The reward is calculated by weighting the pause perception recognition reward, the decoupling emotion reward, and the directional duration reward.

[0140] The model trained under joint supervision and preferences is used as the current policy model. For the same labeled text and reference speech, G candidate speech word sequences are generated and converted into candidate speech using an acoustic decoder. The following reward is then calculated for each candidate speech:

[0141] Pause-aware recognition reward: Candidate speech is input into the ASR model to obtain the recognized text. Different punctuation marks in both the target and recognized texts are uniformly mapped to boundary words "|", and then the character error rate containing pause boundary information is calculated. This reward simultaneously constrains the text content and pause positions, preventing misspellings, omissions, or abnormal pauses during reinforcement learning. The pause-aware recognition reward R... ASR Represented as:

[0142] ;

[0143] Where γ is the recognition reward scaling factor and CER is the character error rate.

[0144] Decoupling Emotional Reward: Candidate speech is input into the emotion recognition model to obtain the posterior probability of the generated speech's emotion. Neutral emotion is used as the global anchor point, and a portion of the reference emotion is retained when the dominant emotion of the reference speech is non-neutral. This reward is used to suppress excessively strong global emotion, avoiding the global emotion masking the differences between strong and weak local readings, while maintaining overall emotional features consistent with the reference speech. The decoupled emotional reward R... emo Represented as:

[0145] ;

[0146] Where, p gen(e) represents the probability that the generated speech belongs to the emotion category e, e * The dominant emotion in speech. I(e) * (≠neutral) is an indicator function; its value is 1 when the dominant emotion of the reference speech is non-neutral, and 0 otherwise. gen (neutral) represents the posterior probability that a candidate speech belongs to a neutral emotion. η is the emotion preservation coefficient.

[0147] Directional duration bonus: The base duration T is calculated based on the number of text characters or phonemes and the natural silence budget. base The non-silence duration T is obtained from the candidate speech through speech activity detection. y For fast-reading tags, candidates with shorter durations relative to the baseline are rewarded; for slow-reading tags, candidates with longer durations relative to the baseline are rewarded; for text without speed tags, no directional duration reward is applied. Directional duration reward R Len It can be represented as:

[0148] ;

[0149] Where, d tag This indicates the direction coefficient determined by the speed control label.

[0150] The above rewards are weighted to obtain the comprehensive reward:

[0151] ;

[0152] Among them, w ASR For R ASR The weight parameter, w emo For R emo The weight parameter, w Len For R Len The weight parameters.

[0153] Figure 4 This is a flowchart of a multi-dimensional reward reinforcement optimization process according to an embodiment of this application, such as... Figure 4 As shown, the intelligibility and pause rationality of the generated speech, the global emotional stability and local strength and weakness comparison, and the speech rate control direction are quantitatively evaluated and weighted by three dimensions: pause perception, emotion decoupling, and duration direction. This enables the comprehensive reward to accurately evaluate the quality of candidate speech from multiple perspectives, guiding reinforcement learning to improve the accuracy of local prosodic control while taking into account the overall naturalness.

[0154] Optionally, based on comprehensive rewards, the target speech generation model can be optimized and updated using a group-relative strategy, including:

[0155] The comprehensive reward is standardized within the candidate group corresponding to the same labeled text condition to obtain the relative advantage of each candidate speech sequence relative to the mean within the group.

[0156] Based on the relative advantage, calculate the policy gradient of the relative policy optimization group, and update the parameters of the current policy model.

[0157] During the update process, policy pruning is incorporated to limit the step size of a single update. The word-level KL divergence between the current policy model and the frozen reference model is calculated and incorporated as a penalty term into the optimization objective to constrain the offset of the current policy model relative to the frozen reference model during the group relative policy optimization process to not exceed a preset offset threshold.

[0158] The comprehensive reward is standardized within the candidate group corresponding to the same input to obtain the relative advantage of each candidate. Based on the relative advantage, a group-based relative strategy is used to optimize and update the TTS large language model. At the same time, policy pruning and word-level KL penalty are added to limit the deviation of the current model relative to the frozen reference model.

[0159] Taking zero-sample Chinese speech synthesis as an example, the plain text is taken from subsets of AISHELL-3 and THCHS-30; the semantic weight and rhythm are analyzed by a contextual language model, and strong, weak, fast, and slow pronunciation labels are inserted outside words or phrases. The style cues for strong and weak pronunciations are taken from a speech pool with corresponding expressiveness, and the speaker reference speech is taken from multi-speaker speech data.

[0160] The teacher speech model performs multiple rounds of autoregressive generation on each segment. In each round, the final acoustic token of the previous segment is retained as a prefix for the next segment. Each segment is sampled multiple times: strong or weak reading segments are selected based on speaker embedding cosine similarity; fast reading segments are selected based on the shortest acoustic token sequence; and slow reading segments are selected based on the longest acoustic token sequence. The selected continuous speech is paired with the unlabeled baseline speech to form preference sample pairs. This generates 9,330 speech samples for joint SFT-DPO training, and an additional 3,224 text prompts are used for reinforcement optimization.

[0161] The joint SFT-DPO training uses the Adam optimizer for 13 epochs with a learning rate of 1×10⁻⁵, a DPO scaling factor β of 0.1, and a maximum gradient norm of 5.0. After joint training, G=8 candidates are sampled for each input with a sampling temperature of 1.0. The GRPO stage uses the Adam optimizer with a learning rate of 1×10⁻⁶, a pruning parameter ε of 0.2, and a token-level KL penalty coefficient of 0.01. ASR w emo w LenThe coefficients are 0.6, 0.15, and 0.25, respectively; the recognition reward scaling factor γ is 3.0, and the emotion preservation factor η is 0.6. The above values ​​are reproducible real-world examples and do not constitute a limitation on the scope of protection.

[0162] After training, the target model can not only replicate the timbre of the reference speech with zero samples, but also perform local strong readings, weak readings, fast readings, and slow readings on specified words or phrases. The recognition reward, sentiment reward, and duration reward in the policy optimization constrain intelligibility and pauses, global sentiment and local intensity, and speech rate direction, respectively, thereby reducing acoustic distortion caused by single-objective optimization.

[0163] Figure 5 This is a flowchart illustrating the overall process of a fine-grained prosodic controllable speech synthesis method based on multi-stage preference alignment, according to an embodiment of this application.

[0164] Using the above method, fine-grained prosodic control tags such as strong reading, weak reading, fast reading, and slow reading are automatically inserted into ordinary text using a contextual semantic model, and corresponding speech is generated through a teacher speech model independent of the target model. At the same time, combined with segmented continuation, acoustic bridging and multi-candidate screening mechanisms, preferred speech samples that are consistent with prosodic tags, have stable timbre and continuous context are obtained.

[0165] Preference sample pairs are formed by combining preferred speech with prosodic labels and baseline speech without corresponding prosodic labels. During the discrete speech lexical generation stage, supervised loss and preference loss are jointly used for training, so that the model can learn the correspondence between labels and local acoustic performance, and suppress problems such as ignoring labels, control overruns and local control failures.

[0166] The model combines pause perception recognition rewards, decoupled emotion rewards anchored by neutral emotions, and directional duration rewards, and uses them for group-based strategy optimization. This allows the model to enhance its ability to control local strength and speed while maintaining speech intelligibility, timbre consistency, pause rationality, and overall naturalness.

[0167] In addition, this embodiment also provides a speech generation method, the method including:

[0168] Obtain the text to be synthesized and the reference speech. The text to be synthesized contains word-level or fragment-level prosodic control tags.

[0169] The text to be synthesized and the reference speech are input into the target speech generation model to determine the synthesized speech of the text to be synthesized. The target speech generation model is trained based on the speech generation model training method described above.

[0170] This speech generation method employs a target speech generation model optimized through the aforementioned multi-stage training method. In real-world inference scenarios, it can directly receive the text to be synthesized and output synthesized speech with fine-grained prosodic control. Since the training process has enabled the model to fully grasp the stable mapping relationship between prosodic labels and local acoustic representations through preference alignment and reinforcement learning, it can automatically generate speech with both local control precision and overall naturalness without requiring additional prosodic annotations or post-processing adjustments in practical use. This achieves seamless transfer of optimization results from the training stage to the inference stage, significantly reducing the barrier to entry and deployment complexity of fine-grained prosodic controllable speech synthesis.

[0171] Furthermore, this embodiment also provides a speech generation model training system, which is used to implement the above-described speech generation model training method embodiments and preferred embodiments, and will not be repeated hereafter. The speech generation model training system includes:

[0172] The tag generation module is used to acquire target text, input the target text into the context semantic analysis model, and generate tagged text, where tags are used to annotate prosodic control requirements;

[0173] The training sample construction module is used to divide the labeled text into multiple text segments according to the label boundaries and punctuation boundaries. Multiple candidate voices corresponding to each text segment are obtained through the teacher's speech model and reference speech. Preferred samples are determined based on multiple candidate voices, and non-preferred samples are determined through the unlabeled target text.

[0174] The training module is used to convert preferred and unpreferred samples into corresponding discrete word sequences, and to train the target speech generation model based on the discrete word sequences and labeled text.

[0175] The model optimization module is used to train the target speech generation model with reference speech as speaker prompt input. It generates multiple candidate speech sequences for the same labeled text, calculates the comprehensive reward for each candidate speech sequence, and optimizes and updates the target speech generation model based on the comprehensive reward using a group relative strategy.

[0176] Furthermore, this embodiment also provides a speech generation system for implementing the above-described speech generation method embodiments and preferred implementations, details of which will not be repeated hereafter. The speech generation system includes:

[0177] The acquisition module is used to acquire the text to be synthesized and the reference speech. The text to be synthesized contains word-level or fragment-level prosodic control tags.

[0178] The synthesis module is used to input the text to be synthesized and the reference speech into the target speech generation model to determine the synthesized speech of the text to be synthesized. The target speech generation model is trained based on the above speech generation model training method.

[0179] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0180] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0181] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0182] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0183] Meanwhile, this embodiment also provides a computer device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the above-described speech generation model training method or speech generation method. The steps of the speech generation model training method or speech generation method here can be steps from the memory analysis methods of the various embodiments described above.

[0184] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. Accordingly, the computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can implement the methods of any of the above embodiments. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0185] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Those skilled in the art should understand that the present invention includes, but is not limited to, the contents described in the accompanying drawings and the specific embodiments above. Any modifications that do not depart from the functional and structural principles of the present invention will be included within the scope of the claims.

Claims

1. A method for training a speech generation model, characterized in that, The method includes: Obtain the target text, input the target text into the context semantic analysis model, and generate tagged text, wherein the tags are used to annotate prosodic control requirements; The labeled text is divided into multiple text segments according to the label boundaries and punctuation boundaries. Multiple candidate voices corresponding to each text segment are obtained through the teacher's voice model and reference voice. Preference samples are determined based on the multiple candidate voices, and non-preference samples are determined through the unlabeled target text. The preferred samples and the non-preferred samples are converted into corresponding discrete word sequences, and the target speech generation model is trained based on the discrete word sequences and the labeled text. The target speech generation model is trained using the reference speech as the speaker prompt input. Multiple candidate speech sequences are generated for the same labeled text. The comprehensive reward of each candidate speech sequence is calculated. Based on the comprehensive reward, the target speech generation model is optimized and updated using a group relative strategy.

2. The speech generation model training method according to claim 1, characterized in that, The process of obtaining multiple candidate voices corresponding to each text segment through the teacher's voice model and reference voice includes: The teacher's speech model is used to generate each text segment through multiple rounds of autoregression. For the first text segment, it is generated directly. For the second and subsequent text segments, an acoustic word segment from the end of the previous text segment is extracted as the autoregression generation prefix of the current text segment, so that an acoustic bridge is formed between adjacent text segments.

3. The speech generation model training method according to claim 1, characterized in that, The step of determining preferred samples based on the multiple candidate speech samples and determining non-preferred samples through unlabeled target text includes: The candidate speech samples are filtered according to the label type to obtain the label-consistent speech samples, and the label-consistent speech samples are determined as the preferred samples. The baseline speech of the target text is generated using the teacher's speech model, and the baseline speech is identified as a non-preference sample.

4. The speech generation model training method according to claim 3, characterized in that, The tag types include any one or more of strong reading tags, weak reading tags, fast reading tags, and slow reading tags; the step of filtering the multiple candidate speech based on the tag types to obtain tag-consistent speech includes: If the text segment contains the strong read label or the weak read label, extract the speaker embedding of the reference speech, calculate the cosine similarity between each candidate speech of the text segment and the speaker embedding of the reference speech, and select the candidate speech with the highest cosine similarity as the label-consistent speech. When the text segment contains the fast reading tag, the candidate speech with the shortest speech word sequence or the shortest non-silence duration is selected as the tag-consistent speech. When the text segment contains the slow reading tag, the candidate speech with the longest speech word sequence or the longest non-silence duration is selected as the tag-consistent speech. When a text segment contains multiple tags, first set a timbre similarity threshold, and then sort the candidate speech according to duration to select the speech with the same tag among the candidate speech that meets the timbre similarity threshold, or calculate a comprehensive score based on timbre similarity, speech duration, intelligibility and naturalness to select the speech with the same tag.

5. The speech generation model training method according to claim 1, characterized in that, The discrete word sequence includes preferred sample words and unpreferred sample words. Training the target speech generation model based on the discrete word sequence and the labeled text includes: Calculate the supervised loss based on the labeled text and the preferred sample terms; The preferred sample lexical and the non-preferred sample lexical are respectively input into the current policy model and the frozen reference model, and the conditional log-likelihood ratio of the current policy model relative to the frozen reference model is calculated. The current policy model is the target speech generation model being optimized, and the frozen reference model is a copy of the target speech generation model with fixed parameters. The preference loss is obtained by expanding the difference in the conditional log-likelihood ratio between the preferred sample word and the non-preferred sample word using the preference loss coefficient. The weighted sum of the supervision loss and the preference loss is used as the total loss for model training.

6. The speech generation model training method according to claim 1, characterized in that, The calculation of the comprehensive reward for each candidate speech sequence includes: Speech recognition is performed on the candidate speech sequence to obtain the recognized text. The punctuation marks in the recognized text and the target text are uniformly mapped to boundary words. The character error rate containing pause boundary information is calculated based on the boundary words. The pause perception recognition reward is determined according to the character error rate. The candidate speech sequence is input into the emotion recognition model to obtain the posterior probability of emotion. Based on the posterior probability of emotion, the decoupled emotion reward is obtained by taking neutral emotion as the anchor point and combining the dominant emotion and retention coefficient of the reference speech. The baseline duration is calculated based on the number of text characters or phonemes and the natural silence budget. The actual duration is obtained based on the non-silence duration of the candidate speech sequence after speech activity detection. The direction coefficient is determined based on the speed control label. A directional duration reward is obtained based on the baseline duration, the actual duration, and the direction coefficient. The comprehensive reward is obtained by weighting the pause perception recognition reward, the decoupled emotion reward, and the directional duration reward.

7. The speech generation model training method according to claim 5, characterized in that, The step of optimizing and updating the target speech generation model based on the comprehensive reward using a group-relative strategy includes: The comprehensive reward is standardized within the candidate group corresponding to the same labeled text condition to obtain the relative advantage of each candidate speech sequence relative to the mean within the group. Based on the relative advantage, calculate the policy gradient of the relative policy optimization group, and update the parameters of the current policy model; During the update process, strategy pruning is incorporated to limit the step size of a single update. The word-level KL divergence between the current strategy model and the frozen reference model is calculated, and the word-level KL divergence is added as a penalty term to the optimization objective to constrain the offset of the current strategy model relative to the frozen reference model during the optimization of the group of relative strategies to not exceed a preset offset threshold.

8. A speech generation method, characterized in that, The method includes: Obtain the text to be synthesized and a reference speech, wherein the text to be synthesized contains word-level or fragment-level prosodic control tags; The text to be synthesized and the reference speech are input into the target speech generation model to determine the synthesized speech of the text to be synthesized. The target speech generation model is trained based on the speech generation model training method of any one of claims 1 to 7.

9. A speech generation model training system, characterized in that, The system includes: The tag generation module is used to acquire target text, input the target text into the context semantic analysis model, and generate tagged text, wherein the tags are used to annotate prosodic control requirements; The training sample construction module is used to divide the labeled text into multiple text segments according to the label boundaries and punctuation boundaries, obtain multiple candidate voices corresponding to each text segment through the teacher's speech model and reference speech, determine the preferred samples based on the multiple candidate voices, and determine the non-preferred samples through the unlabeled target text. The training module is used to convert the preferred samples and the non-preferred samples into corresponding discrete word sequences, and to train the target speech generation model based on the discrete word sequences and the labeled text. The model optimization module is used to train the target speech generation model using the reference speech as the speaker prompt input, generate multiple candidate speech sequences for the same labeled text, calculate the comprehensive reward for each candidate speech sequence, and optimize and update the target speech generation model based on the comprehensive reward using a group relative strategy.

10. A speech generation system, characterized in that, The system includes: The acquisition module is used to acquire the text to be synthesized and the reference speech, wherein the text to be synthesized contains word-level or fragment-level prosodic control tags; A synthesis module is used to input the text to be synthesized and the reference speech into a target speech generation model to determine the synthesized speech of the text to be synthesized, wherein the target speech generation model is trained based on the speech generation model training method of any one of claims 1 to 7.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the speech generation model training method according to any one of claims 1 to 7, or the speech generation method according to claim 8.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the speech generation model training method of any one of claims 1 to 7, or the speech generation method of claim 8.

Images