Voice synthesis model training method, voice synthesis method, device, electronic equipment, computer readable storage medium and computer program product

Abstract

The application provides a speech synthesis model training method, a speech synthesis method, a device, an electronic device, a computer readable storage medium and a computer program product. The method comprises: synthesizing a first target speech based on a first text feature of a text sample and a first reference speech feature of a reference speech through a speech synthesis model; determining a speech length reward value based on a deviation between a first speech speed of the first target speech and a second speech speed of the reference speech; determining a prosody alignment reward value based on a matching degree between a first pause structure of the first target speech and a second pause structure pre-constructed for the text sample; determining an entropy regular reward value based on a difference between a distribution entropy when the first target speech is synthesized and a target entropy value labeled for the text sample; and training the speech synthesis model based on the speech length reward value, the entropy regular reward value and the prosody alignment reward value. The application can improve the quality of the speech synthesized by the speech synthesis model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a training method for a speech synthesis model, a speech synthesis method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the rapid development of deep learning technology, text-to-speech (TTS) technology has been widely applied in various scenarios such as intelligent customer service, audiobooks, in-vehicle navigation, and virtual digital humans. However, the training objectives for speech synthesis models in related technologies can easily lead to an average distribution of speech features (i.e., oversmoothing), resulting in the loss of high-frequency details in the synthesized speech, making the sound quality unclear and rich, and failing to meet the needs of high-quality voice interaction. Summary of the Invention

[0003] This application provides a training method for a speech synthesis model, a speech synthesis method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the quality of speech synthesized by the speech synthesis model.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a method for training a speech synthesis model, the method comprising:

[0006] The speech synthesis model extracts first text features from a text sample and first reference speech features from a reference speech, and synthesizes a first target speech based on the first reference speech features and the first text features. A speech length reward value is determined based on the deviation between a first speech rate of the first target speech and a second speech rate of the reference speech. The pause structure of the first target speech is analyzed to obtain a first pause structure, and a prosodic alignment reward value is determined based on the matching degree between the first pause structure and a second pause structure pre-constructed for the text sample. The distribution entropy of the speech synthesis model when synthesizing the first target speech is determined, and an entropy regularization reward value is determined based on the difference between the distribution entropy and the target entropy value labeled for the text sample. The speech synthesis model is trained based on the speech length reward value, the entropy regularization reward value, and the prosodic alignment reward value.

[0007] This application also provides a speech synthesis method, the method comprising:

[0008] The process involves acquiring a text to be processed and a voiceprint reference identifier; inputting the text to be processed and the voiceprint reference identifier into a pre-trained speech synthesis model; extracting second text features from the text to be processed through the speech synthesis model; and synthesizing a second target speech based on the second reference speech features corresponding to the voiceprint reference identifier and the second text features; the voiceprint of the second target speech is consistent with the voiceprint indicated by the voiceprint reference identifier; wherein, the speech synthesis model is trained based on a speech length reward value, an entropy regularization reward value, and a prosody alignment reward value; the speech length reward value is determined based on the deviation between the first speech rate of the first target speech and the second speech rate of the reference speech, the first target speech being synthesized by the speech synthesis model based on the first text features of the text sample and the first reference speech features of the reference speech; the entropy regularization reward value is determined based on the difference between the distribution entropy of the first target speech and the target entropy value of the text sample; the prosody alignment reward value is determined based on the matching degree between the first pause structure of the first target speech and the second pause structure pre-constructed for the text sample.

[0009] This application provides a training device for a speech synthesis model, including:

[0010] The first speech synthesis module is used to extract the first text features of the text sample and the first reference speech features of the reference speech through a speech synthesis model, and synthesize the first target speech based on the first reference speech features and the first text features.

[0011] The first reward value determination module is used to determine the speech length reward value based on the degree of deviation between the first speech rate of the first target speech and the second speech rate of the reference speech;

[0012] The second reward value determination module is used to parse the pause structure of the first target speech to obtain the first pause structure of the first target speech, and determine the prosodic alignment reward value based on the matching degree between the first pause structure and the second pause structure pre-constructed for the text sample.

[0013] The third reward value determination module is used to determine the distribution entropy of the speech synthesis model when synthesizing the first target speech, and to determine the entropy regularization reward value based on the difference between the distribution entropy and the target entropy value labeled for the text sample.

[0014] The training module is used to train the speech synthesis model based on the speech length reward value, the entropy regularization reward value, and the prosody alignment reward value.

[0015] In the above scheme, the first reward value determination module is further configured to determine the first speech rate of the first target speech as the first ratio of the text length of the text sample to the speech duration of the first target speech; determine the second ratio between the first speech rate and the second speech rate of the reference speech; and determine the speech length reward value based on the second ratio and the tolerable ratio range; wherein the tolerable ratio range represents the degree to which the first speech rate is allowed to deviate from the second speech rate.

[0016] In the above scheme, the training device for the speech synthesis model further includes an offline construction module, which is used to construct a second pause structure in the following ways: constructing a reference example based on the reference text and the pause structure annotated for the reference text; constructing an inference prompt based on the text sample and the prompt word template, and inputting the inference prompt and the reference example into the pre-trained inference model, so that the inference model, under the guidance of the inference prompt and the requirements of the reference example, infers the pause structure of the text sample and generates the second pause structure of the text sample.

[0017] In the above scheme, the second reward value determination module is further used to perform timestamp parsing on the first target speech to obtain the silence period in the first target speech and the silence duration corresponding to the silence period; and to map the silence duration to a pause structure using a preset mapping rule to obtain the first pause structure of the first target speech; wherein, the preset mapping rule is associated with the language corresponding to the text sample.

[0018] In the above scheme, there are multiple second pause structures; the second reward value determination module is further used to perform structural matching between the first pause structure and each of the second pause structures to obtain a matching degree; if the matching degree between the first pause structure and at least one of the second pause structures exceeds a first matching degree threshold, a preset first value is determined as the prosody alignment reward value; if the matching degree between the first pause structure and each of the second pause structures does not exceed the first matching degree threshold, a preset second value is determined as the prosody alignment reward value; the first value is greater than the second value.

[0019] In the above scheme, there are multiple first target speech samples; the third reward value determination module is further used to determine the distribution entropy of the speech synthesis model when synthesizing each first target speech sample, and to determine the first mean of all the distribution entropies; to determine the first difference between the first mean and the target entropy value labeled for the text sample; if the first difference is greater than a preset third value, the first product value between the first difference and a preset penalty coefficient is determined as the entropy regularization reward value; if the first difference is less than or equal to the third value, the third value is determined as the entropy regularization reward value.

[0020] In the above scheme, the third reward value determination module is further used to determine the target entropy value by: acquiring a first speech sample set constructed for the text sample, wherein multiple first speech samples in the first speech sample set are synthesized based on the text sample; determining the sample distribution entropy when synthesizing each first speech sample, and determining the second mean and standard deviation of all the sample distribution entropies; determining the second product value between the standard deviation and a preset adjustment coefficient, and determining the sum of the second product value and the second mean as the target entropy value; wherein the preset adjustment coefficient is used to regulate the coverage of the target entropy value on the first speech sample set, so as to define the allowable fluctuation range of the sample distribution entropy from the second mean.

[0021] In the above scheme, the third reward value determination module is further used to acquire multiple candidate speech samples, which are synthesized based on the text samples; based on a pre-trained speech quality assessment model, the quality of each candidate speech sample is assessed to obtain a quality score of each candidate speech sample as the speech of the text sample, and the candidate speech samples with quality scores higher than a quality threshold are used as first speech samples to obtain a first speech sample set composed of multiple first speech samples; or, for each candidate speech sample, the signal-to-noise ratio of the candidate speech sample is determined, and the candidate speech samples with signal-to-noise ratios higher than a first signal-to-noise ratio threshold are used as first speech samples to obtain a first speech sample set composed of multiple first speech samples.

[0022] In the above scheme, the training device for the speech synthesis model further includes a fourth reward value determination module, which is used to transcribe the first target speech into text to obtain the text to be verified, and determine an accuracy reward value based on the difference between the text to be verified and the text sample; determine a voiceprint similarity reward value based on the similarity between the first voiceprint feature of the first target speech and the second voiceprint feature of the reference speech; determine a voiceprint comparison reward value based on the similarity between the first voiceprint feature and the third voiceprint feature of the comparison speech, as well as the similarity between the first voiceprint feature and the second voiceprint feature; the training module is also used to train the speech synthesis model based on the speech length reward value, the entropy regularization reward value, the prosody alignment reward value, the accuracy reward value, the voiceprint similarity reward value, and the voiceprint comparison reward value.

[0023] In the above scheme, the fourth reward value determination module is further used to determine the character error rate of the text to be verified based on the text sample; normalize the character error rate based on the text length of the text sample to obtain the normalized character error rate; and determine the second difference between the preset benchmark value and the normalized character error rate as the accuracy reward value.

[0024] In the above scheme, the fourth reward value determination module is further used to determine the comparison speech in the following ways: constructing a second speech sample set, wherein the second speech samples in the second speech sample set satisfy at least one of the following preset mutual exclusion conditions: the identity label corresponding to the second speech sample is different from the identity label corresponding to the reference speech; the similarity between the voiceprint features of the second speech sample and the second voiceprint features of the reference speech is lower than a similarity threshold; filtering out second speech samples that meet preset filtering conditions in the second speech sample set, and determining the filtered second speech samples as the comparison speech; wherein the preset filtering conditions include at least one of the following: the similarity between the voiceprint features of the second speech sample and the second voiceprint features of the reference speech is within a preset threshold range, the upper limit of the preset threshold range is less than the similarity threshold, and the lower limit of the preset threshold range is greater than the mean of the similarity between the voiceprint features of the second speech sample and the second voiceprint features; the signal-to-noise ratio of the second speech sample is higher than a second signal-to-noise ratio threshold; the speech duration of the second speech sample is greater than a duration threshold; the matching degree between the attribute labels of the second speech sample and the attribute labels of the reference speech is higher than a second matching degree threshold.

[0025] In the above scheme, the fourth reward value determination module is further used to determine a voiceprint comparison loss value based on the similarity between the first voiceprint feature and the third voiceprint feature of the contrasting speech, as well as the similarity between the first voiceprint feature and the second voiceprint feature; obtain multiple pre-constructed loss value intervals where there is no overlap, with different loss value intervals corresponding to different voiceprint reward values, and the smaller the loss value interval, the larger the voiceprint reward value corresponding to it; determine the target loss value interval to which the voiceprint comparison loss value belongs in the multiple loss value intervals, and determine the voiceprint reward value corresponding to the target loss value interval as the voiceprint comparison reward value.

[0026] In the above scheme, the first speech synthesis module is further configured to perform multiple feature samplings on the first text features under the constraint of the first reference speech features to generate multiple candidate speech features, and decode each of the candidate speech features to obtain multiple first target speech features; the training module is further configured to determine the total reward value of each first target speech feature based on the speech length reward value, the entropy regularization reward value, the prosody alignment reward value, the accuracy reward value, the voiceprint similarity reward value, and the voiceprint comparison reward value corresponding to the first target speech feature; determine the relative advantage value of the first target speech feature based on the statistical distribution of the total reward value of the first target speech feature among the total reward values ​​of the multiple first target speech features; and update the policy parameters of the speech synthesis model based on the relative advantage value of each first target speech feature to maximize the total reward value of all first target speech features; wherein, the policy parameters are used to guide the speech synthesis model to perform multiple feature samplings on the first text features.

[0027] This application provides a speech synthesis device, including:

[0028] The information acquisition module is used to acquire the text to be processed and the voiceprint reference identifier;

[0029] The second speech synthesis module is used to input the text to be processed and the voiceprint reference identifier into a pre-trained speech synthesis model, extract the second text features of the text to be processed through the speech synthesis model, and synthesize the second target speech based on the second reference speech features corresponding to the voiceprint reference identifier and the second text features; the voiceprint of the second target speech is consistent with the voiceprint indicated by the voiceprint reference identifier.

[0030] The speech synthesis model is trained based on speech length reward, entropy regularization reward, and prosody alignment reward. The speech length reward is determined based on the deviation between a first speech rate of a first target speech and a second speech rate of a reference speech. The first target speech is synthesized by the speech synthesis model based on a first text feature of a text sample and a first reference speech feature of the reference speech. The entropy regularization reward is determined based on the difference between the distribution entropy of the first target speech and the target entropy value of the text sample. The prosody alignment reward is determined based on the matching degree between a first pause structure of the first target speech and a second pause structure pre-constructed for the text sample.

[0031] This application provides an electronic device, the electronic device comprising:

[0032] Memory is used to store executable instructions or computer programs.

[0033] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the training method or speech synthesis method of the speech synthesis model provided in the embodiments of this application.

[0034] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the training method or speech synthesis method of the speech synthesis model provided in this application.

[0035] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the training method or speech synthesis method of the speech synthesis model provided in this application.

[0036] The embodiments of this application have the following beneficial effects:

[0037] In the embodiments of this application, when training a speech synthesis model, the first text features of the text sample and the first reference speech features of the reference speech are extracted by the speech synthesis model. A first target speech is synthesized based on the first reference speech features and the first text features. Then, a speech length reward value is determined based on the deviation between the first speech rate of the first target speech and the second speech rate of the reference speech. The pause structure of the first target speech is analyzed to obtain the first pause structure of the first target speech. Based on the matching degree between the first pause structure and the second pause structure pre-constructed for the text sample, a prosodic alignment reward value is determined. The distribution entropy of the speech synthesis model when synthesizing the first target speech is determined, and an entropy regularization reward value is determined based on the difference between the distribution entropy and the target entropy value labeled for the text sample. Finally, the speech synthesis model is trained based on the speech length reward value, the entropy regularization reward value, and the prosodic alignment reward value. Thus, the speech length... The degree reward value can directly constrain the duration features of the first target speech, making the rhythm of the speech synthesized by the speech synthesis model more consistent with the speaking habits of the reference speech; the prosodic alignment reward value can force the speech synthesis model to learn the correct punctuation and pause logic, enhancing the semantic intelligibility and rhythmic sense of the first target speech; the entropy regularization reward value can regularize the prediction distribution of the speech synthesis model, preventing both overly smooth phenomena caused by excessively low entropy (i.e., dull and mechanical speech) and unstable generation or noise caused by excessively high entropy, thus preserving rich acoustic details and expressiveness while ensuring speech clarity. It can be seen that, through the above methods, non-differential or higher-order perceptual indicators such as speech rate, prosody, and acoustic distribution can be directly transformed into optimization objectives, guiding the speech synthesis model to optimize collaboratively in multiple dimensions, improving the quality of the synthesized speech, and ultimately generating a highly realistic, emotionally rich, and rhythmically accurate first target speech. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the architecture of the speech synthesis system provided in the embodiments of this application;

[0039] Figure 2A This is a first structural schematic diagram of the electronic device provided in the embodiments of this application;

[0040] Figure 2B This is a schematic diagram of the second structure of the electronic device provided in the embodiments of this application;

[0041] Figure 3 This is a schematic diagram of the first process of the training method for the speech synthesis model provided in the embodiments of this application;

[0042] Figure 4 This is a flowchart illustrating the method for determining the speech length reward value provided in an embodiment of this application;

[0043] Figure 5 This is a flowchart illustrating the method for determining the prosody alignment reward value provided in an embodiment of this application;

[0044] Figure 6 This is a flowchart illustrating the method for determining the entropy regularization reward value provided in an embodiment of this application;

[0045] Figure 7 This is a schematic diagram of the second process of the training method for the speech synthesis model provided in the embodiments of this application;

[0046] Figure 8 This is a flowchart illustrating the method for determining the accuracy reward value provided in the embodiments of this application;

[0047] Figure 9 This is a flowchart illustrating the method for determining the voiceprint comparison reward value provided in the embodiments of this application;

[0048] Figure 10 This is a schematic flowchart of the speech synthesis method provided in the embodiments of this application;

[0049] Figure 11 This is a schematic diagram of the architecture of the speech synthesis model provided in the embodiments of this application;

[0050] Figure 12 This is a flowchart of the reward evaluation process for the multi-reward evaluation module provided in this application embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0052] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0053] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0054] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0055] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0056] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0057] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0058] 1) Client, also known as user terminal, refers to the program that provides local services to users in contrast to the server. Except for some applications that can only run locally, it is generally installed on ordinary client machines and needs to cooperate with the server to run. That is, there needs to be a corresponding server and service program in the network to provide the corresponding services. Thus, a specific communication connection needs to be established between the client and the server to ensure the normal operation of the application.

[0059] 2) Large Language Models (LLMs): These are machine learning models designed to understand and generate human language. Trained on massive amounts of text data, they can perform a wide range of tasks, including text summarization, translation, sentiment analysis, and more. LLMs are characterized by their enormous scale, containing billions of parameters that help them learn complex patterns in language data. They are typically based on deep learning architectures. In short, LLMs are deep learning models trained on vast amounts of text data, containing billions or even more parameters. They can be used to generate natural language text and understand its meaning. Through training, the models learn the statistical regularities and semantic relationships of language to build a vast language knowledge base, thereby simulating human language understanding and generation capabilities. Large language models possess the following characteristics: learning ability: trained on massive amounts of text data, large language models can learn rich linguistic knowledge and expressions, including grammar, semantics, and common expression habits; pattern recognition: large language models can identify common text patterns and semantic relationships, such as co-occurrence relationships between words, logical structure of sentences, and semantic roles; context understanding: large language models can capture contextual information in text, understand the influence of previous text on subsequent text, and generate corresponding responses based on the context; generation ability: large language models can generate relevant natural language text based on input information, including answering questions, generating articles, and engaging in dialogue; and ambiguity resolution: despite the polysemy and ambiguity of language, large language models resolve ambiguity through contextual information and linguistic rules, providing more accurate and appropriate text generation or understanding.

[0060] This application provides a training method for a speech synthesis model, a speech synthesis method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the speech synthesis quality of the speech synthesis model.

[0061] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the speech synthesis system 100 provided in the embodiments of this application. In order to support a speech synthesis application and the training application of the speech synthesis model, the terminal 401 connects to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0062] In practical applications, a trained speech synthesis model can be deployed in terminal 401. This speech synthesis model can be applied in various scenarios such as dialogue assistants, intelligent customer service, audiobooks, news broadcasting, short videos and film dubbing, digital human broadcasting and multi-role reading, barrier-free reading and assisted communication. Specifically, in any application scenario, terminal 401 responds to the user's speech synthesis command, obtains the text to be processed and the voiceprint reference identifier; inputs the text to be processed and the voiceprint reference identifier into the pre-trained speech synthesis model, extracts the second text features of the text to be processed through the speech synthesis model, and synthesizes the second target speech based on the second reference speech features corresponding to the voiceprint reference identifier and the second text features; the voiceprint of the second target speech is consistent with the voiceprint indicated by the voiceprint reference identifier.

[0063] As an example, if a user uses a client with audiobook functionality on terminal 401, and generates a reading instruction for a certain text based on the voice of the target person, this reading instruction can be used as a speech synthesis instruction. The text selected by the user is the text to be processed, and the voice of the target person is the voiceprint reference identifier.

[0064] In practical applications, the training method for the speech synthesis model can be executed independently by server 200 or terminal 401. The following explanation uses server 200 as an example. Responding to the training instruction for the speech synthesis model, server 200 extracts the first text features of the text sample and the first reference speech features of the reference speech through the speech synthesis model, and synthesizes the first target speech based on the first reference speech features and the first text features. Based on the deviation between the first speech rate of the first target speech and the second speech rate of the reference speech, a speech length reward value is determined. The pause structure of the first target speech is analyzed to obtain the first pause structure of the first target speech, and based on the matching degree between the first pause structure and the second pause structure pre-constructed for the text sample, a prosody alignment reward value is determined. The distribution entropy of the speech synthesis model when synthesizing the first target speech is determined, and based on the difference between the distribution entropy and the target entropy value labeled for the text sample, an entropy regularization reward value is determined. The speech synthesis model is trained based on the speech length reward value, the entropy regularization reward value, and the prosody alignment reward value. After training is complete, server 200 can send the trained speech synthesis model to terminal 401 to deploy the trained speech synthesis model on terminal 401, so as to execute the speech synthesis method based on the speech synthesis model.

[0065] In practical applications, the training method for the speech synthesis model can also be executed collaboratively by the terminal 401 and the server 200. Specifically, the terminal 401 deploys the speech synthesis model to be trained. In response to the training instruction for the speech synthesis model, the terminal 401 sends a sample acquisition request to the server 200. In response to the sample acquisition request, the server 200 sends text samples and reference speech to the terminal 401. Terminal 401 extracts first text features from a text sample and first reference speech features from a reference speech using a speech synthesis model, and synthesizes first target speech based on the first reference speech features and the first text features. It determines a speech length reward value based on the deviation between the first speech rate of the first target speech and the second speech rate of the reference speech. It analyzes the pause structure of the first target speech to obtain the first pause structure, and determines a prosodic alignment reward value based on the matching degree between the first pause structure and a pre-constructed second pause structure for the text sample. It determines the distribution entropy of the speech synthesis model when synthesizing the first target speech, and determines an entropy regularization reward value based on the difference between the distribution entropy and the target entropy value labeled for the text sample. The speech synthesis model is then trained based on the speech length reward value, the entropy regularization reward value, and the prosodic alignment reward value. Afterwards, terminal 401 can directly execute the speech synthesis method based on the trained speech synthesis model.

[0066] The following describes the training method or electronic device for the speech synthesis model provided in the embodiments of this application. The electronic device implementing the training method or speech synthesis method for the speech synthesis model in the embodiments of this application can be a terminal, a server, or a combination of both. Therefore, the executing entity of each step will not be repeated below. In some embodiments, the terminal can be implemented as a laptop computer, tablet computer, desktop computer, set-top box, smartphone, smart speaker, smartwatch, smart TV, vehicle terminal, and other types of terminals.

[0067] In some embodiments, the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal and server can be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment.

[0068] See Figure 2A and Figure 2B , Figure 2A This is a first structural schematic diagram of the electronic device provided in an embodiment of this application. Figure 2BThis is a schematic diagram of the second structure of the electronic device provided in the embodiments of this application. Figure 2A and Figure 2B The illustrated electronic device 400 includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the electronic device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2A and Figure 2B The general labeled all buses as Bus System 440.

[0069] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0070] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0071] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0072] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0073] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0074] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0075] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0076] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with user interface 430 (e.g., a display screen, a speaker, etc.).

[0077] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0078] In some embodiments, the training apparatus for the speech synthesis model provided in this application can be implemented in software. Figure 2A A training device 455A for a speech synthesis model stored in memory 450 is shown. This device can be software in the form of programs and plugins, and includes the following software modules: a first speech synthesis module 4551A, a first reward value determination module 4552A, a second reward value determination module 4553A, a third reward value determination module 4554A, and a training module 4555A. These modules are logically connected and can therefore be arbitrarily combined or further split according to their implemented functions. The functions of each module will be described below.

[0079] In some embodiments, the speech synthesis apparatus provided in this application can be implemented in software. Figure 2B A speech synthesis device 455B stored in memory 450 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: an information acquisition module 4551B and a second speech synthesis module 4552B. These modules are logically connected and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0080] In other embodiments, the training device or speech synthesis device for the speech synthesis model provided in this application can be implemented in hardware. As an example, the training device or speech synthesis device for the speech synthesis model provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the training method or speech synthesis method for the speech synthesis model provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0081] Below, the training method for the speech synthesis model provided in the embodiments of this application will be described in conjunction with the accompanying drawings. As mentioned earlier, the electronic device 400 that implements the training method for the speech synthesis model in the embodiments of this application can be a terminal, a server, or a combination of both. Therefore, the executing entity of each step will not be described again below.

[0082] The training method of the speech synthesis model in this application embodiment is described using a server as the execution entity as an example. See also Figure 3 , Figure 3 This is a schematic diagram of the first process of the training method for the speech synthesis model provided in this application embodiment, which will be combined with Figure 3 The steps shown are explained.

[0083] In step 101, the first text features of the text sample and the first reference speech features of the reference speech are extracted by the speech synthesis model, and the first target speech is synthesized based on the first reference speech features and the first text features.

[0084] In practical applications, the speech synthesis model first receives input data (including text samples and reference speech) for synthesis. The text sample refers to the text content to be converted into speech, which can be a sentence, a paragraph, or a specific phoneme sequence, serving as the content basis for speech synthesis; the reference speech refers to the audio clip used to provide target voiceprint, timbre, prosodic style, or background environment information, serving as style cues for speech synthesis.

[0085] Specifically, the process of extracting the first text features of a text sample and the first reference speech features of a reference speech through a speech synthesis model can include: parsing the input text sample (such as converting characters to phonemes, semantic encoding, etc.) through the text front-end or encoder inside the speech synthesis model, and converting it into a high-dimensional vector or discrete vector sequence that can be understood by the machine, i.e., the first text features, which represent the semantic and pronunciation content of the speech to be synthesized; at the same time, the audio encoder or feature extractor inside the speech synthesis model performs acoustic analysis on the reference speech, and extracts vectors or discrete vector sequences that represent the speaker's identity, intonation style and acoustic details, i.e., the first reference speech features.

[0086] Subsequently, in the step of synthesizing the first target speech based on the first reference speech features and the first text features, the speech synthesis model can fuse or concatenate the aforementioned first text features and the first reference speech features. Using the first reference speech features as conditions, the speech synthesis model predicts the acoustic feature sequence or waveform corresponding to the first text features, thereby generating the first target speech. This first target speech not only corresponds to the text sample in content but also simulates the timbre and prosodic style of the reference speech in terms of auditory perception.

[0087] See also Figure 3 The following explanation follows step 101 above.

[0088] In step 102, a speech length reward value is determined based on the degree of deviation between the first speech rate of the first target speech and the second speech rate of the reference speech.

[0089] Step 102 is primarily used to evaluate the temporal plausibility of the generated first target speech. Specifically, the first speech rate characterizes the temporal speed or duration characteristics (e.g., syllables per second or the ratio of total duration to text length) of the first target speech generated by the speech synthesis model, while the second speech rate serves as a benchmark, reflecting the speaker's inherent speaking habits and rhythmic style in the reference speech. The degree of deviation reflects the magnitude of the difference between the rhythm of the generated first target speech and the rhythmic style of the reference speech.

[0090] The speech length reward value serves as a key constraint or feedback signal (e.g., a penalty in a reinforcement learning framework), imposing duration constraints on the speech synthesis model's generation process. By quantifying this deviation, the speech length reward value guides the speech synthesis model to avoid generating abnormal speech rates. The specific technical effect of this constraint is that when the first target speech exhibits phenomena such as excessively fast speech rate (e.g., elision, word skipping) or excessively slow speech rate (e.g., prolonged sounds, meaningless silences, or endless repetitions) due to the unstable probability distribution of the speech synthesis model, the deviation increases, resulting in a lower (or negative) speech length reward value. By maximizing this reward value during training, the speech synthesis model can learn the correct pause and duration prediction mechanisms, ensuring that the generated first target speech maintains consistency with the reference speech in terms of speech rate and rhythm, thereby improving the naturalness and stability of the synthesized speech.

[0091] In some embodiments, Figure 4 This is a flowchart illustrating the method for determining the speech length reward value provided in an embodiment of this application. See also... Figure 4 Step 102 can be achieved through steps 1021 to 1022.

[0092] In step 1021, the first ratio of the text length of the text sample to the speech duration of the first target speech is determined as the first speech rate of the first target speech.

[0093] In practice, the number of language units contained in the text sample is first counted as the text length, and the total duration of the audio waveform corresponding to the first target speech is detected as the speech duration. Then, the number of language units per unit time is calculated by division, which yields the first ratio. This first ratio is then used directly as the first speech rate to measure the speed of speech generation.

[0094] In some cases, the statistical unit for text length can be flexibly selected according to the linguistic level, which can be the number of characters, the number of words, or even the number of phonemes at a finer granular level, to adapt to different precision requirements for speech rate control; the method for measuring speech duration can be set to only count the cumulative duration of effective vocal segments, that is, to remove the duration of silent segments from the total duration, thereby calculating a purer effective speech rate and avoiding interference from long pauses on speech rate calculation.

[0095] As an example, if the text sample consists of 10 Chinese characters and the total duration of the waveform of the first target speech is 5 seconds, then the first ratio is 2 characters / second, which is the first speech rate.

[0096] In step 1022, a second ratio between the first speech rate and the second speech rate of the reference speech is determined, and a speech length reward value is determined based on the second ratio and the tolerable ratio range.

[0097] Among them, the tolerable ratio range represents the degree to which the first speech rate is allowed to deviate from the second speech rate.

[0098] In practical applications, a second speech rate of a predetermined reference speech is obtained, and the first speech rate is divided by the second speech rate to obtain a normalized second ratio. Then, it is determined whether the second ratio falls within a preset tolerable ratio range. If it falls within the range, a positive reward is given (e.g., the speech length reward value is set to 1). If it falls outside the range, a penalty of zero or negative value is given (e.g., the speech length reward value is set to 0). Finally, the value is output as the speech length reward value.

[0099] In some cases, the calculation function for speech length reward can be designed as a piecewise linear or nonlinear function. When the second ratio is within the tolerable ratio range, the speech length reward remains at 1, indicating that the generated speech rate is within a reasonable range. When the second ratio is below the lower limit of the range (meaning the speech rate is too slow) or above the upper limit of the range (meaning the speech rate is too fast), the speech length reward decreases linearly or exponentially with the increase of the deviation distance, thereby imposing a stronger penalty on severe speech rate abnormalities. In addition, the boundary threshold of the tolerable ratio range can be dynamically adjusted according to the style type of the reference speech. For styles with large inherent speech rate variations, the range can be appropriately widened.

[0100] As an example, if the second speech rate is 4 words / second and the first speech rate is 2 words / second, then the second ratio is 0.5. If the tolerable ratio range is set to [0.7, 1.3], since 0.5 is less than 0.7, the speech synthesis model will calculate a negative number as the speech length reward value based on the difference.

[0101] By introducing the calculation mechanism of the first and second ratios, the absolute duration constraint is transformed into a relative speech rate consistency constraint, eliminating the influence of different text sample lengths on the evaluation criteria and achieving standardized measurement of the rhythm of the generated speech. At the same time, a flexible acceptable range is defined using the tolerable ratio range, which allows the first target speech to retain a certain natural prosodic fluctuation within a reasonable range, while also accurately identifying and suppressing malicious errors caused by the instability of the model generation path through a powerful penalty mechanism, such as extremely slow speech rate caused by infinite loops or extremely fast speech rate caused by omissions, thereby significantly improving the robustness of the synthesized speech.

[0102] See also Figure 3 The following explanation follows step 102 above.

[0103] In step 103, the pause structure of the first target speech is parsed to obtain the first pause structure of the first target speech, and the prosodic alignment reward value is determined based on the matching degree between the first pause structure and the second pause structure pre-constructed for the text sample.

[0104] Here, the first pause structure refers to the discrete symbol sequence representing the rhythmic pause information actually presented by the first target speech at the acoustic level (representing the position and level of prosodic boundaries), used to reflect how the first target speech is actually read. The pause levels it includes usually include prosodic word boundaries, prosodic phrase boundaries, and intonation phrase boundaries. The second pause structure refers to the discrete symbol sequence representing the ideal rhythmic pause information that the text sample should have at the linguistic logical level, used to specify how the text sample should theoretically be read.

[0105] The prosodic alignment reward value is a numerical feedback signal used to quantify the degree of consistency between the first and second pause structures. The higher the value, the closer the actual pause position of the generated speech is to the theoretical optimal pause position.

[0106] By introducing a dual-pause structure contrast mechanism, the elusive sense of rhythm in speech synthesis is transformed into a computable supervisory signal. This allows the reinforcement learning model to not only focus on the clarity of speech quality but also be forced to learn rhythmic patterns that match the semantic logic of the text. This alignment-based reward mechanism can effectively correct the "flat and monotonous" or "randomly segmented" phenomena that are prone to occur in long sentence synthesis models, ensuring that the synthesized speech is both fluent and conforms to the grammatical logic of human language, significantly improving the prosodic naturalness and intelligibility of speech synthesis.

[0107] In some embodiments, the second pause structure can be pre-built before training the speech synthesis model. Specifically, it can be built by: constructing a reference example based on the reference text and the pause structure annotated for the reference text; constructing an inference prompt based on the text sample and the prompt word template, and inputting the inference prompt and the reference example into the pre-trained inference model, so that the inference model, guided by the inference prompt and required by the reference example, infers the pause structure of the text sample and generates the second pause structure of the text sample.

[0108] Here, "reference example" refers to exemplary data containing standard text input and its corresponding correct prosodic annotation output. In this application, it is used to demonstrate the expected input-output mapping relationship to an inference model (e.g., a large language model) through a context learning mechanism. Specifically, one or more data points are selected from a high-quality annotated speech synthesis dataset, the original text content corresponding to the data is extracted as reference text, and a manually verified sequence of prosodic boundary symbols is extracted as a pause structure. Then, the reference text and the pause structure are concatenated according to a preset format template (e.g., "Input: [Text] -> Output: [Annotation Result]") to form a reference example in string form.

[0109] Inference prompts refer to instruction texts used to stimulate the specific task capabilities of an inference model. In this application, they are used to explicitly instruct the inference model to perform the prosodic pause prediction task. The inference model refers to a pre-trained generative model with large-scale general language knowledge. In this application, it is used to act as a high-precision prosodic annotator.

[0110] In practical applications, a prompt word template containing a task description (such as "Please imitate the above example and add prosodic boundaries to the following text") is designed. This prompt word template is placed after the reference example constructed in the preceding order, and the text sample to be processed is attached to form a complete input prompt word (i.e., inference prompt). The inference prompt is input into a pre-trained inference model with frozen parameters. The inference model analyzes the pattern of the reference example based on the attention mechanism and predicts the position and type of pause mark to be inserted in the text sample by combining its own linguistic knowledge. It outputs a text sequence containing the mark, and finally obtains the second pause structure after parsing.

[0111] In some cases, the construction of inference prompts can adopt a thought chain strategy. The prompt word template guides the inference model to first analyze the grammatical components (such as subject-verb-object structure) or semantic groups of the text sample, and then generate pause structures based on the analysis results, thereby improving the accuracy of inference for long and difficult sentences. For the selection and invocation of inference models, an integrated inference mechanism can be introduced. That is, multiple inference prompts with different wordings are constructed and input into the model respectively, or multiple inference models with different architectures are invoked at the same time, and the multiple candidate pause structures generated by them are voted on, and the result with the highest frequency is selected as the final second pause structure.

[0112] By leveraging the powerful semantic understanding and context learning capabilities of pre-trained inference models, this method can achieve high-accuracy prosodic pause prediction directly through a small sample size without requiring manual annotation of large amounts of domain-specific data or specialized model fine-tuning. Compared to traditional rule-based or shallow statistical model-based approaches, this method can better handle text samples containing complex grammatical structures or ambiguous semantics, and the generated second pause structure is more in line with human natural language habits. This provides a high-quality benchmark that is close to human level for measuring the pause accuracy of speech synthesis models.

[0113] In some embodiments, the step 103 of "parse the pause structure of the first target speech to obtain the first pause structure of the first target speech" can be implemented in the following way: perform timestamp parsing on the first target speech to obtain the silence period in the first target speech and the silence duration corresponding to the silence period; use a preset mapping rule to map the silence duration to a pause structure to obtain the first pause structure of the first target speech; wherein, the preset mapping rule is associated with the language corresponding to the text sample.

[0114] Here, the silent period refers to a continuous time interval in the speech waveform where the amplitude is lower than a specific threshold or is marked as non-articulated content in the alignment information, and is used in this application to locate the pause positions in speech; the silent duration refers to the length value of this continuous time interval, and is used in this application to quantify the length of the pause. The preset mapping rule refers to a predefined logical judgment criterion for converting continuous time values into discrete prosodic level symbols, and is used in this application to convert acoustic features into linguistic features; the first pause structure refers to a prosodic boundary sequence deduced backward based on the generated first target speech, and is used in this application as the basis for evaluating the actual performance of the generation result of the evaluation model.

[0115] In practical applications, obtain the phoneme-level or character-level alignment information generated during the generation of the first target speech, or directly perform energy analysis on the audio waveform of the generated first target speech, identify the time difference between the end moment of each language unit (such as a character or a word) and the start moment of the next language unit, use this time difference as the silent period, and obtain the silent duration in milliseconds or seconds by calculating the difference between the end moment and the start moment. Traverse each silent period and its silent duration extracted for the first target speech, load the corresponding duration threshold table according to the language of the text sample being processed, compare the silent duration with the threshold interval, determine the corresponding prosodic level symbol, and insert this prosodic level symbol into the corresponding text position to form the first pause structure with explicit prosodic markings.

[0116] As an example, for the first target speech "How are you", the parsing result shows that the character "好" ends at 1.5 seconds and the character "吗" starts at 1.8 seconds. Then, identify the time interval from 1.5 seconds to 1.8 seconds as a silent period, and the corresponding silent duration is 0.3 seconds. The preset mapping rule sets that in the Chinese context, [100ms, 300ms) is mapped to #1, and [300ms, 600ms) is mapped to #2; if the silent duration at a certain place is 0.3 seconds (300ms), it is mapped to #2. Finally, the obtained first pause structure is "你好#2吗".

[0117] In some cases, the timestamp parsing process can combine speech activity detection technology with forced alignment technology. First, forced alignment is used to obtain a rough word boundary, and then speech activity detection technology is used in the boundary neighborhood to finely detect the actual acoustic silence boundary, so as to eliminate the slight silence caused by non-prosodic pauses such as plosive occlusion segments. In addition, an adaptive threshold algorithm based on energy and zero-crossing rate can be introduced to dynamically determine the silence judgment criteria in the background noise environment, so as to prevent the omission of weak silence segments due to excessive background noise of the generated speech. The preset mapping rules can be designed as a dynamic mapping mechanism related to the global speech rate, rather than a fixed absolute time threshold. That is, the average speech rate of the first target speech is calculated first, and the standard threshold is scaled according to the average speech rate (for example, when the speech rate is fast, a short silence is also considered a high-level pause), so as to adapt to the prosodic judgment under different speaking rhythms. At the same time, differentiated rule bases are constructed for different languages. For Chinese, the focus is on the hierarchical division of prosodic words (#1), prosodic phrases (#2), and intonation phrases (#3), while for English, the focus is on the distinction between pauses between words and pauses between sentences, to ensure that the mapping logic stored in the rule base conforms to the linguistic rules of the specific language.

[0118] By transforming the continuous temporal features (silence duration) at the acoustic level into discrete symbolic features (first pause structure) at the linguistic level, this approach successfully bridges the gap between audio signals and textual logic. This allows the evaluation of the naturalness of generated speech prosody to move beyond vague, subjective auditory perception and instead transform into quantifiable and comparable structured data. In particular, by introducing language-related pre-defined mapping rules, the pause parsing process adheres to the prosodic norms of a specific language, accurately distinguishing between physiological breathing and semantic pauses. This provides a high-precision, linguistically meaningful input for subsequent reward value calculations, effectively enhancing the model's control over fine-grained prosodic features.

[0119] In some embodiments, Figure 5 This is a flowchart illustrating the method for determining the prosodic alignment reward value provided in an embodiment of this application. See also... Figure 5 The number of second pause structures is multiple; the "determine the prosodic alignment reward value based on the matching degree between the first pause structure and the second pause structure pre-constructed for the text sample" in step 103 can be implemented through steps 1031 to 1033.

[0120] In step 1031, the first pause structure and each of the second pause structures are structurally matched to obtain the matching degree.

[0121] Here, the matching degree refers to a numerical index that quantifies the similarity between the first pause structure and the second pause structure in terms of prosodic boundary position and level. In this application, it is used to evaluate whether the generated speech hits any reasonable prosodic pattern.

[0122] In practical applications, multiple lists of second pause structures pre-generated for text samples are traversed, and the first pause structure is compared with each second pause structure in the list. During the comparison, characters or words in the text are used as anchors to check whether the pause symbols inserted at the corresponding positions are consistent. The F1 value or the reciprocal of the sequence edit distance is calculated as the similarity score between the two, thus obtaining a set of matching scores.

[0123] For example, if the first pause structure is “A#1B#2C”, and the three second pause structures are “A#1B#2C”, “A#0B#2C”, and “A#1B#0C”, the calculated matching degrees are 1.0, 0.8, and 0.6, respectively.

[0124] In some cases, the matching degree can be calculated using a weighted algorithm based on prosodic hierarchy weights. This algorithm not only determines the presence or absence of pause marks but also calculates the distance between pause levels. For example, the difference between "prosodic word boundary (#1)" and "prosodic phrase boundary (#2)" is determined to be smaller than the difference between "no pause (#0)" and "long pause (#3)". A weighted matching score is calculated using a pre-set confusion matrix, which more subtly reflects the closeness between the generated result and the reference standard. In addition, semantic-prosodic joint matching based on vector space can be used to convert the first and second pause structures into embedded vectors and calculate cosine similarity to tolerate minor local differences and focus on the consistency of the overall rhythmic pattern.

[0125] In step 1032, if the matching degree between the first pause structure and at least one second pause structure exceeds the first matching degree threshold, the preset first value is determined as the prosody alignment reward value.

[0126] Here, the first value refers to a higher scalar feedback value (positive reward) used to incentivize the reinforcement learning model to update positively, which in this application is used to reinforce the speech synthesis model to generate behaviors that conform to the diverse prosodic habits of humans; the first matching degree threshold is the passing grade for determining whether the generated prosodic structure is qualified.

[0127] In practical applications, a first matching degree threshold (e.g., 0.9) is set, and it is checked whether any value in the set of matching degree values ​​calculated in the previous step is greater than or equal to the threshold. As long as there is a second pause structure in the list that matches the first pause structure, the generated result is determined to be valid, and a preset positive number (e.g., +1.0) is directly selected as the prosody alignment reward value of the current time step or the current sample and fed back to the policy network.

[0128] For example, if the first pause structure matches a standard answer in the candidate set with a score of 0.95 (above 0.9), it can be considered a "hit" and awarded a +1 bonus.

[0129] In some cases, the first value can be designed as a dynamic function positively correlated with the highest matching degree, rather than a fixed value. That is, the more the value exceeds the threshold, the higher the reward (e.g., a matching degree of 0.9 earns 1 point, and a matching degree of 1.0 earns 1.2 points), to encourage the speech synthesis model to pursue a perfect match. At the same time, the first matching degree threshold can be adaptively adjusted according to the length or syntactic complexity of the text sample. The threshold can be appropriately lowered for long and difficult sentences and raised for short sentences to balance the learning efficiency of samples of different difficulty.

[0130] In step 1033, if the matching degree between the first pause structure and each of the second pause structures does not exceed the first matching degree threshold, the preset second value is determined as the prosody alignment reward value; the first value is greater than the second value.

[0131] Here, the second value refers to a low scalar feedback value (usually negative or zero) used to penalize the reinforcement learning model or suppress its generative behavior, which in this application is used to correct the erroneous behavior of the speech synthesis model in generating speech that does not conform to any reasonable prosodic pattern.

[0132] In practical applications, after traversing and comparing all second pause structures, if no matching score reaches the first matching score threshold, it indicates that the prosodic structure of the generated speech deviates from all possible correct pronunciations (i.e., there are broken sentences or serious rhythmic errors). In this case, a preset negative number (such as -1.0) or zero value can be selected as the prosodic alignment reward value. For example, if the highest matching score between the generated result and all reference answers is only 0.4, it can be judged as "missing" and a penalty of -0.5 can be given.

[0133] In some cases, the setting of the second value can introduce a graded penalty mechanism based on the severity of the error. The minimum distance between the first pause structure and all second pause structures is calculated. The greater the distance (i.e., the more outrageous the error), the larger the absolute value of the negative reward (e.g., -0.1 for a small deviation and -1.0 for a large deviation), thereby providing denser gradient signals to guide the model to gradually approach the correct region. In addition, differentiated penalties can be applied in combination with the type of alignment error. Severe segmentation errors that destroy semantic integrity are severely penalized, while minor hierarchical errors that only affect tone are lightly penalized.

[0134] By introducing a flexible "one-to-many" matching mechanism, the diversity of prosodic expression in natural language is fully respected. That is, the same sentence can have multiple reasonable pauses in different contexts or emotions. This mechanism avoids the "pattern collapse" problem caused by the traditional single reference standard and prevents the speech synthesis model from being forced to fit a specific, non-unique prosodic pattern. As a result, the trained speech synthesis model can generate richer, more flexible, and more natural speech. At the same time, it significantly reduces the probability of misjudgment caused by a single reference standard (that is, the generated speech is correct, but is judged wrong because it is different from the single reference answer), and improves the stability and convergence efficiency of reinforcement learning training.

[0135] See also Figure 3 The following explanation follows step 103 above.

[0136] In step 104, the distribution entropy of the speech synthesis model when synthesizing the first target speech is determined, and the entropy regularization reward value is determined based on the difference between the distribution entropy and the target entropy value labeled for the text sample.

[0137] Here, distribution entropy refers to the uncertainty measure of the probability distribution output by the speech synthesis model at each time step of the generated speech sequence. In this application, it is used to reflect the degree of randomness or exploratory ability of the speech synthesis model's generation strategy. Target entropy refers to the ideal uncertainty level that the speech synthesis model should maintain during the generation process, which is pre-set for a specific text sample. In this application, it is used as a benchmark reference to constrain the model's behavior. Entropy regularization reward value refers to the numerical feedback calculated based on the deviation between the actual distribution entropy and the ideal target entropy value. In this application, it is used to prevent the model from getting trapped in local optima (generating uniformity) or excessive randomness (generating ambiguity).

[0138] By introducing an entropy-based regularized reward mechanism, this approach effectively solves the "exploration-exploitation" dilemma within the reinforcement learning framework. It prevents the speech synthesis model from experiencing "pattern collapse" in the pursuit of high rewards, thus avoiding the model from generating monotonous and unchanging speech in order to secure a high score. At the same time, by setting a target entropy value as an anchor point, it prevents the uncontrollable generation results and ambiguous pronunciation caused by maximum entropy exploration. This allows the generated speech to maintain clear and accurate pronunciation while possessing richer prosodic variations and stylistic diversity, significantly improving the anthropomorphism of the synthesized first target speech.

[0139] In some embodiments, Figure 6 This is a flowchart illustrating the method for determining the entropy regularization reward value provided in an embodiment of this application. See also... Figure 6 If there are multiple target voices, step 104 can be implemented through steps 1041 to 1044.

[0140] In step 1041, the distribution entropy of the speech synthesis model when synthesizing each first target speech is determined, and the first mean of all distribution entropies is determined.

[0141] Here, the first target speech refers to multiple copies of speech waveforms or acoustic feature sequences generated by the speech synthesis model for the same text sample using different random sampling strategies (such as sampling different latent variables from a Gaussian distribution) during a single training iteration or inference process; distribution entropy refers to a metric used to measure the uncertainty of random variables, and in this application, it specifically refers to the information entropy of the probability distribution output by the policy network of the speech synthesis model; the first mean is a scalar value obtained by arithmetically averaging the distribution entropies of the above multiple acoustic feature sequence copies, used to characterize the current overall exploration level of the speech synthesis model.

[0142] In practical applications, during the training phase, after the speech synthesis model receives a text sample, it generates N first target speech samples (e.g., N=4) through parallel computation. For each first target speech sample, the policy distribution (such as the log probability of phoneme duration prediction) at each time step during its generation process is extracted, and the average entropy value of the sample is calculated to obtain N independent distribution entropy values. Subsequently, these N values ​​are added together and divided by N to calculate the first mean value representing the diversity level of the current batch generation.

[0143] Here, the calculation of the first mean can also adopt a confidence-based weighted average algorithm. A discriminator network that can evaluate speech quality scores each first target speech. The speech quality score is normalized and used as weight to sum the weights of the entropy of each distribution. This makes the first mean more reflective of the entropy level of high-quality generated samples and filters out the interference of abnormal entropy values ​​caused by generation failure. In addition, a sliding window averaging method can be used, which not only calculates the mean of the current batch, but also combines the entropy mean of historical training steps to perform an exponential moving average to obtain a more stable entropy estimate.

[0144] In step 1042, a first difference is determined between the first mean and the target entropy value labeled for the text sample.

[0145] Here, the target entropy value is a preset hyperparameter or an ideal lower bound of uncertainty calculated for a specific text; the first difference is a value that reflects the gap between the actual entropy level of the current speech synthesis model and the expected target, used to indicate whether a penalty needs to be imposed.

[0146] In practical applications, the target entropy value (e.g., 2.0) is read from the memory or dynamically retrieved based on the current text attributes. The first mean value (e.g., 1.5) calculated in the previous step is subtracted from the target entropy value to obtain the first difference value (0.5). This positive value indicates that the current model's exploration ability is insufficient and has not yet met the target requirements. If the first mean value is 2.2, then the first difference value is -0.2, indicating that the requirements have been met.

[0147] Here, the calculation of the first difference can also be combined with the concept of relative error, calculating (target entropy value - first mean) / target entropy value, thereby eliminating the influence of the size of the target entropy value on the sensitivity of the difference; or a dynamic difference calculation based on course learning can be introduced, artificially reducing the first difference in the early stage of training (for example, by multiplying it by a coefficient less than 1), tolerating the low entropy state of the speech synthesis model in the early stage, and gradually restoring the true difference as training progresses, guiding the speech synthesis model to smoothly increase the diversity of generation.

[0148] In step 1043, if the first difference is greater than a preset third value, the first product value between the first difference and the preset penalty coefficient is determined as the entropy regularization reward value.

[0149] Here, the third value is a decision boundary threshold (usually 0) used to distinguish whether the speech synthesis model meets the standard; the preset penalty coefficient is a negative weight used to convert the difference into a negative reward signal; the entropy regularized reward value is the scalar signal that is finally fed back to the speech synthesis model.

[0150] In practical applications, it is determined whether the calculated first difference (e.g., 0.2) is strictly greater than the third value (0). If the judgment condition is met, it means that the generation diversity of the speech synthesis model is lower than the preset bottom line. At this time, the preset penalty coefficient (e.g., -0.1) is called, and the first difference is multiplied by the preset penalty coefficient (0.2*-0.1=-0.02). The resulting negative value is determined as the entropy regularization reward value, thereby punishing the conservative behavior of the speech synthesis model.

[0151] Here, the preset penalty coefficient can also be adaptively adjusted using the Lagrange multiplier method. That is, a learnable parameter is set as the coefficient. When the first difference is consistently greater than the third value, the absolute value of the coefficient is automatically increased, thereby applying a stronger penalty to force the model to increase the entropy value. Conversely, the penalty is reduced. In addition, the calculation of the first product value can use a non-linear function, such as squaring the first difference and then multiplying it by the coefficient, to apply an exponentially increasing penalty for cases that deviate significantly from the target.

[0152] In step 1044, if the first difference is less than or equal to the third value, the third value is determined as the entropy regularization reward value.

[0153] In practical applications, it is determined whether the calculated first difference (e.g., -0.2) is less than or equal to the third value (0). If the judgment condition is met, it means that the distribution entropy generated by the speech synthesis model has exceeded or equaled the target entropy value, that is, the diversity has met the standard. At this time, no additional guidance or intervention is needed. Directly selecting the third value (i.e., 0) as the entropy regularization reward value means that no reward or penalty is given on this indicator, allowing the speech synthesis model to freely optimize other objectives under the premise of meeting the minimum entropy requirement.

[0154] In some cases, a "soft pruning" mechanism can be introduced, which provides a small positive incentive in a small neighborhood where the difference is close to the third value (e.g., [-0.05, 0]) to smooth the gradient of the reward function and avoid training oscillations caused by gradient abrupt changes at the threshold; or a reward upper limit can be set, and when the first difference is too small (i.e., the actual entropy is much greater than the target entropy, e.g., -1.0), a certain negative penalty is given to prevent the speech synthesis model from generating completely random noise in pursuit of high entropy.

[0155] The above approach constructs a robust minimum entropy constraint through a "batch averaging-threshold truncation" mechanism. First, by calculating the mean of multiple generated samples, the random fluctuations caused by a single sampling are smoothed out, making the entropy estimation more accurate and avoiding training instability caused by individual abnormal samples. Second, using the target entropy value as a passing grade, penalties are only imposed when the speech synthesis model's exploration is insufficient (entropy is too low), and intervention stops once the speech synthesis model reaches the target (reward is 0). This design effectively avoids the "infinite entropy increase" problem that may occur in traditional maximum entropy reinforcement learning, preventing the model from sacrificing speech clarity and coherence in pursuit of maximizing entropy. It ensures that the generated speech has sufficient prosodic variation (not monotonous) while maintaining high-quality acoustic performance (unambiguous).

[0156] In some embodiments, the target entropy value can be determined by: obtaining a first speech sample set constructed from text samples, wherein multiple first speech samples in the first speech sample set are synthesized based on text samples; determining the sample distribution entropy when synthesizing each first speech sample, and determining the second mean and standard deviation of all sample distribution entropies; determining the second product value between the standard deviation and a preset adjustment coefficient, and determining the sum of the second product value and the second mean as the target entropy value; wherein the preset adjustment coefficient is used to regulate the coverage of the target entropy value on the first speech sample set, so as to define the allowable fluctuation range of the sample distribution entropy from the second mean.

[0157] Here, the first speech sample set refers to a set of speech synthesis results pre-generated or retrieved specifically for the current input text sample, and is used in this application as a statistical benchmark to evaluate the inherent prosodic diversity potential of the text sample. The sample distribution entropy is a numerical value calculated for a single first speech sample in the set, reflecting its generation uncertainty; the second mean characterizes the average randomness of the text sample under normal circumstances; and the standard deviation characterizes the dispersion of the text prosodic variation amplitude. The preset adjustment coefficient is a scalar parameter (usually negative, used to set a lower limit) used to control the tightness of the target constraints, and in this application, it is used to define a reasonable range within which the entropy value of the speech synthesis model is allowed to deviate from the average level.

[0158] In practical applications, each first speech sample in the first speech sample set is traversed, and its corresponding generation probability distribution sequence is extracted. The entropy value of each sample is calculated using the information entropy formula, resulting in a list containing the distribution entropy of M samples. Subsequently, the distribution entropy of the samples in this list is summed and divided by M to obtain the second mean. At the same time, the variance of these distribution entropy values ​​relative to the second mean is calculated and the square root is taken to obtain the standard deviation. The calculated standard deviation (e.g., 0.2) is multiplied by a pre-set adjustment coefficient (e.g., -1.0) to obtain the second product value (-0.2). Then, this second product value is added to the second mean (1.5) (1.5 + (-0.2) = 1.3), and the result 1.3 is determined as the target entropy value of the text sample.

[0159] This approach proposes a "text-adaptive" target entropy setting mechanism. Unlike related techniques that use a fixed and uniform entropy threshold for all texts (which ignores the significant differences in prosodic ambiguity between different texts—for example, declarative sentences are usually highly deterministic and have low entropy, while interrogative sentences or poetry are more varied and have higher entropy), this application dynamically sets a "passing line" that conforms to the characteristics of each text by statistically analyzing the sample set distribution characteristics (mean and standard deviation). Using the three-standard-deviation criterion or similar logic in statistics to construct the target value ensures that the set target is both challenging and consistent with the linguistic rules of the text. This effectively avoids the pronunciation ambiguity caused by forcibly demanding high entropy for simple texts, and also avoids the prosodic monotony caused by restricting low entropy to complex texts, thus improving the scientific nature of speech synthesis model training and the naturalness of generated speech.

[0160] In some embodiments, the above-mentioned "obtaining a first speech sample set constructed for text samples" can be implemented in the following ways: obtaining multiple candidate speech samples, which are synthesized based on text samples; evaluating the quality of each candidate speech sample based on a pre-trained speech quality assessment model to obtain a quality score of each candidate speech sample as a text sample, and using candidate speech samples with quality scores higher than a quality threshold as first speech samples to obtain a first speech sample set composed of multiple first speech samples; or, for each candidate speech sample, determining the signal-to-noise ratio (SNR) of the candidate speech sample, and using candidate speech samples with SNR higher than a first SNR threshold as first speech samples to obtain a first speech sample set composed of multiple first speech samples.

[0161] Here, candidate speech samples refer to a set of initial speech waveforms or acoustic feature sequences obtained by inputting text samples into a speech synthesis system and running the generation process multiple times or through parallel sampling. This set of candidate speech samples has not yet undergone quality verification and may include various cases ranging from extremely high quality to generation failure (such as muteness or feedback). The speech quality assessment model refers to a deep neural network trained on a large amount of subjective listening test data. Its function is to simulate human auditory perception and automatically quantify the naturalness and clarity of speech. The quality score is the numerical score output by the speech quality assessment model; the quality threshold is a preset minimum acceptable score.

[0162] In practical applications, the current text sample to be processed is used as input and fed into a pre-trained speech synthesis model (which can be the current policy network or a teacher network used as a reference). During the inference phase, a random sampling mode is enabled (such as introducing Gaussian noise into the duration predictor and acoustic decoder), and the number of samplings K is set (e.g., K=200). The pre-trained speech synthesis model performs K inference operations in parallel and outputs the corresponding K original speech sequences, which are temporarily stored as a candidate speech sample set.

[0163] Load the pre-trained speech quality assessment model and convert the waveform data of each candidate speech sample obtained in the previous step into Mel spectrograms or other input features required by the model; input the features into the speech quality assessment model, and the speech quality assessment model outputs a floating-point number between 1 and 5 as a quality score; iterate through all scores, mark the candidate speech samples with scores greater than a preset threshold (such as 3.8 points) as qualified, retain them and reorganize them into the first speech sample set, and discard the samples below the quality threshold directly.

[0164] Signal-to-noise ratio (SNR) is the ratio of speech signal power to background noise power. In the application, it is used as a physical indicator to measure the purity of generated speech and to quickly identify bad samples containing artifacts or background noise. The first SNR threshold is a preset benchmark value used to determine whether the SNR of a candidate speech sample meets the acoustic quality screening conditions, and is used to determine whether the candidate speech sample is clear enough.

[0165] In practical applications, for each candidate speech sample, the signal-to-noise ratio (SNR) is estimated using a signal processing algorithm; an empirical threshold (e.g., 20dB) is set as the first SNR threshold; the calculated SNR is compared with the first SNR threshold. If it is higher than 20dB, the candidate speech sample is determined to be clear and is included in the first speech sample set; otherwise, it is considered a noise sample and is removed.

[0166] In some cases, an integrated generation strategy based on different model architectures can also be adopted. This means that in addition to using the current speech synthesis model, auxiliary models based on different architectures such as stream models, diffusion models, or generative adversarial networks can be called to generate candidate speech samples for the same text sample, thereby obtaining a wider range of acoustic representations and capturing the potential form of the text sample under different generation mechanisms. Alternatively, data augmentation techniques can be used to perform pitch and speed shifting on a small number of basic generated samples to generate derived candidate speech samples, thereby enriching the sample space.

[0167] The above method significantly improves the accuracy and robustness of target entropy calculation by introducing a "survival of the fittest" sample selection mechanism. Since random sampling (especially high-variance sampling) during speech synthesis inevitably produces a certain proportion of "broken" samples (such as broken pronunciations, meaningless noise, or excessively distorted and strange pronunciations), directly using a set containing these bad samples to calculate the mean and standard deviation would cause the statistical results to deviate significantly from the distribution characteristics of normal speech (for example, bad samples often have abnormally high entropy or extreme acoustic features), thus misleading the setting of the target entropy value. This scheme uses objective quality scores or physical signal-to-noise ratio indicators to clean the data before statistics, ensuring that the final determined first speech sample set contains only high-quality, usable speech variants. This allows the calculated distribution entropy statistics to truly reflect the prosodic diversity of the text within a "reasonable generation range," providing the model with a safe, reliable, and valuable regularization benchmark.

[0168] See also Figure 3 The following explanation follows step 104 above.

[0169] In step 105, the speech synthesis model is trained based on the speech length reward value, entropy regularization reward value, and prosody alignment reward value.

[0170] In the embodiments of this application, when training a speech synthesis model, the first text features of the text sample and the first reference speech features of the reference speech are extracted by the speech synthesis model. A first target speech is synthesized based on the first reference speech features and the first text features. Then, a speech length reward value is determined based on the deviation between the first speech rate of the first target speech and the second speech rate of the reference speech. The pause structure of the first target speech is analyzed to obtain the first pause structure of the first target speech. Based on the matching degree between the first pause structure and the second pause structure pre-constructed for the text sample, a prosodic alignment reward value is determined. The distribution entropy of the speech synthesis model when synthesizing the first target speech is determined, and an entropy regularization reward value is determined based on the difference between the distribution entropy and the target entropy value labeled for the text sample. Finally, the speech synthesis model is trained based on the speech length reward value, the entropy regularization reward value, and the prosodic alignment reward value. Thus, the speech length... The degree reward value can directly constrain the duration features of the first target speech, making the rhythm of the speech synthesized by the speech synthesis model more consistent with the speaking habits of the reference speech; the prosodic alignment reward value can force the speech synthesis model to learn the correct punctuation and pause logic, enhancing the semantic intelligibility and rhythmic sense of the first target speech; the entropy regularization reward value can regularize the prediction distribution of the speech synthesis model, preventing both overly smooth phenomena caused by excessively low entropy (i.e., dull and mechanical speech) and unstable generation or noise caused by excessively high entropy, thus preserving rich acoustic details and expressiveness while ensuring speech clarity. It can be seen that, through the above methods, non-differential or higher-order perceptual indicators such as speech rate, prosody, and acoustic distribution can be directly transformed into optimization objectives, guiding the speech synthesis model to optimize collaboratively in multiple dimensions, improving the quality of the synthesized speech, and ultimately generating a highly realistic, emotionally rich, and rhythmically accurate first target speech.

[0171] In some embodiments, Figure 7 This is a schematic diagram of the second process of the training method for the speech synthesis model provided in the embodiments of this application. See also... Figure 7 Before step 105, the accuracy reward value, voiceprint similarity reward value, and voiceprint comparison reward value can also be determined through steps 106 to 108.

[0172] In step 106, the first target speech is transcribed into text to obtain the text to be verified, and an accuracy reward value is determined based on the difference between the text to be verified and the text sample.

[0173] Here, the text to be verified refers to the text sequence derived from the speech content generated by the speech synthesis model; the accuracy reward value is a quantitative indicator used to measure the clarity, intelligibility, and whether any omissions or misreadings occur in the generated first target speech content.

[0174] In practical applications, the first target voice is input into a pre-trained automatic speech recognition model, and the speech recognition model outputs a corresponding predicted text sequence as the text to be verified. Subsequently, the edit distance algorithm can be used to calculate the character error rate or word error rate between the text to be verified and the original input text sample. Finally, the error rate is inverted or mapped through a monotonically decreasing function to obtain a positive accuracy reward value.

[0175] In some embodiments, Figure 8 is a schematic flowchart of a method for determining an accuracy reward value provided by an embodiment of the present application. Refer to Figure 8 , step 106 can be implemented through steps 1061 to 1063:

[0176] In step 1061, based on the text sample, the character error rate of the text to be verified is determined.

[0177] The character error rate refers to the number of characters for the minimum insertion, deletion, and replacement operations required to convert the text to be verified into the text sample; its use in this application is as a quantitative indicator to reflect the accuracy of the first target voice in content expression.

[0178] In practical applications, the text to be verified obtained by transcribing the first target voice is acquired, and the edit distance algorithm is used to align and compare the text to be verified with the original text sample character by character; the total number of character insertions, character deletions, and character replacements that occur during the alignment process is counted, and this total number is recorded as the original error count value.

[0179] In some cases, a semantic fuzzy alignment strategy based on pinyin or phoneme conversion can also be introduced. That is, before calculating the error rate, the text sample and the text to be verified are both converted into phoneme sequences using a text-to-phoneme tool. The penalty weight is reduced for character mismatches caused by homophone recognition errors, while the penalty weight is increased for characters with completely incorrect pronunciations, so that the calculated error rate can more accurately reflect the performance of the speech synthesis model in pronunciation accuracy rather than the recognition preference of the speech recognition model; or a key character weight assignment mechanism is adopted, and higher error weights are assigned to core information characters such as names, place names, and numbers in the text. Once these characters are incorrect, the calculated character error rate value is greatly increased. Example illustration: The text sample is "The weather is good", and the text to be verified obtained by transcribing is "The weather". After comparison, it is found that the word "good" is missing, so the character error rate is 1.

[0180] In step 1062, the character error rate is normalized based on the text length of the text sample to obtain the normalized character error rate.

[0181] In practical applications, the total number of characters L in the statistical text sample (e.g., L=10) is counted. The character error rate value determined in the previous step is divided by the total number of characters L to obtain a ratio representing the average error level per unit character, which is the normalized character error rate.

[0182] In step 1063, the second difference between the preset baseline value and the normalized character error rate is determined as the accuracy bonus value.

[0183] The preset benchmark value is a pre-defined constant reference point; its purpose in this application is to transform the error rate indicator, which has a negative connotation, into a reward score with a positive incentive effect.

[0184] In practical applications, a preset benchmark score (e.g., 1.0) is read, and the normalized error rate (e.g., 0.2) is subtracted from the preset benchmark score. The calculated result (0.8) is defined as the accuracy bonus value. When the error rate is 0, the accuracy bonus value reaches the maximum value of the preset benchmark score.

[0185] In some cases, simple subtraction calculations can be replaced with S-curve-based mapping transformations. The normalized error rate is input into a monotonically decreasing non-linear function, so that the reward value decreases slowly when the error rate is low (within an acceptable range), but drops rapidly when the error rate exceeds a certain threshold. This strengthens the model's awareness of avoiding serious pronunciation errors. Alternatively, a dynamic baseline strategy can be adopted, gradually increasing the preset baseline value according to the number of training rounds. In the later stages of training, the model must achieve a lower error rate to obtain a positive reward, thereby achieving progressive intelligibility optimization from easy to difficult.

[0186] This approach, by establishing an objective "synthesis-transcription-comparison" closed-loop evaluation mechanism, provides direct content fidelity feedback for the training of speech synthesis models. Normalization eliminates the impact of text length fluctuations on the reward signal, ensuring consistent incentive levels for the speech synthesis model when processing sentences of different lengths. Converting character-level errors into reward scores accurately identifies and constrains intelligibility issues such as missing words, omitting words, or unclear pronunciation that may occur during the synthesis process. This reward mechanism, based on objective recognition results, does not rely on manual scoring and can automatically force the model to improve the clarity and accuracy of pronunciation during large-scale reinforcement learning training, fundamentally solving the technical problem of generated target speech that is "pleasant to hear but difficult to understand."

[0187] See also Figure 7 The following explanation follows step 106 above.

[0188] In step 107, a voiceprint similarity reward value is determined based on the similarity between the first voiceprint feature of the first target speech and the second voiceprint feature of the reference speech.

[0189] Here, the first voiceprint feature is a high-dimensional vector representing the speaker's identity characteristics extracted from the generated first target speech; the second voiceprint feature is the corresponding identity vector extracted from the reference speech used as a style reference; the voiceprint similarity reward value aims to guide the first target speech generated by the speech synthesis model to approximate the target speaker (i.e., the timbre of the reference speech) in terms of timbre.

[0190] In practical applications, a pre-trained speaker encoder is invoked to map the first target speech and the reference speech to a fixed voiceprint embedding space, respectively, to obtain two high-dimensional vectors; the cosine similarity between these two vectors is calculated, and the similarity value or its scaled value is directly determined as the voiceprint similarity reward value.

[0191] See also Figure 7 The following explanation follows step 107 above.

[0192] In step 108, a voiceprint comparison reward value is determined based on the similarity between the first voiceprint feature and the third voiceprint feature of the contrasting speech, as well as the similarity between the first voiceprint feature and the second voiceprint feature.

[0193] Here, the contrastive speech refers to a speech segment belonging to a different speaker (i.e., a negative sample) than the current reference speech, used to provide contradictory acoustic information; the third voiceprint feature is the identity vector of the contrastive speech; the voiceprint contrast reward value is a signal based on a contrastive learning mechanism, designed to enhance the speech synthesis model's ability to identify the target speaker's identity.

[0194] In some embodiments, the comparison speech can be determined by: constructing a second speech sample set, wherein the second speech samples in the second speech sample set satisfy at least one of the following preset mutual exclusion conditions: the identity label corresponding to the second speech sample is different from the identity label corresponding to the reference speech; the similarity between the voiceprint features of the second speech sample and the second voiceprint features of the reference speech is lower than a similarity threshold. Second speech samples that meet preset screening conditions are selected from the second speech sample set, and the selected second speech samples are determined as comparison speech; wherein the preset screening conditions include at least one of the following: the similarity between the voiceprint features of the second speech sample and the second voiceprint features of the reference speech is within a preset threshold range, the upper limit of the preset threshold range is less than the similarity threshold, and the lower limit of the preset threshold range is greater than the mean of the similarity between the voiceprint features of the second speech sample and the second voiceprint features of the reference speech; the signal-to-noise ratio of the second speech sample is higher than a second signal-to-noise ratio threshold; the speech duration of the second speech sample is greater than a duration threshold; the matching degree between the attribute labels of the second speech sample and the attribute labels of the reference speech is higher than a second matching degree threshold.

[0195] The second speech sample set refers to a negative sample candidate pool specifically constructed for contrastive learning during training. It contains a large amount of speech data from non-target speakers, i.e., multiple second speech samples. Its purpose in this application is to provide rich candidate material, from which specific negative samples are selected for calculating the voiceprint contrast reward value as contrast speech. Contrast speech refers to the specific speech segment ultimately selected to form an opposing relationship with the current reference speech (positive sample). Its purpose in this application is to serve as the negative input in the triplet loss function or contrastive loss function, forcing the first target speech generated by the speech synthesis model to be far removed from the contrast speech in the feature space. The preset threshold range is a manually set range of similarity values ​​used to define "difficult negative samples." Its purpose in this application is to filter out speech that has a certain similarity to the target speaker but is not the same person, increasing the difficulty for the speech synthesis model to distinguish timbre, thereby improving training effectiveness.

[0196] In practical applications, all speech files in the speech database are traversed, and the metadata information associated with each speech file (such as speaker identifiers, i.e., identity tags) and pre-extracted voiceprint features are read. The identity tag of the current reference speech is set as the exclusion target. If the identity tag of a speech sample is inconsistent with the identity tag of the reference speech, or if the cosine similarity value between the voiceprint features of the speech sample and the second voiceprint features of the reference speech is lower than the pre-set similarity threshold (e.g., 0.8), then the speech sample is determined to be the speech of a non-target speaker and is added to the second speech sample set.

[0197] Next, a multi-dimensional scan is performed on the samples in the second speech sample set. First, the similarity between the voiceprint features of each second speech sample and the second voiceprint features of the reference speech is calculated. Second speech samples with similarity within a preset threshold range (e.g., between 0.3 and 0.6) are retained. These second speech samples are neither completely unrelated (too low similarity, too easy to distinguish) nor potentially mislabeled samples (too high similarity). Then, the signal-to-noise ratio (SNR) is detected, and second speech samples with an SNR lower than the second SNR threshold (e.g., 15 dB) are removed. At the same time, the speech duration is checked, and second speech samples shorter than the duration threshold (e.g., 2 seconds) are discarded. Finally, attribute labels (including at least one of language, speaker gender, vocal range, and style) are compared, and second speech samples with an attribute label matching degree higher than the second matching degree threshold are preferentially selected. Second speech samples that meet at least one of the above conditions are identified as comparison speech.

[0198] This approach, employing a two-step strategy of refined mutual exclusion construction and conditional selection, enhances the effectiveness and robustness of contrastive learning. First, by filtering through signal-to-noise ratio and duration, it ensures that the contrastive speech used as negative samples possesses high-quality acoustic features, preventing the speech synthesis model from misinterpreting noise patterns as identity differences and thus guaranteeing the purity of voiceprint feature extraction. Second, by setting similarity threshold ranges (mining difficult negative samples) and attribute label matching (e.g., selecting the same gender as a negative sample), a highly challenging discrimination task is implemented. This forces the speech synthesis model to move beyond relying solely on obvious features (such as gender differences and silence length) to distinguish positive and negative samples, instead requiring it to deeply learn and capture the unique formants, fundamental frequency trajectories, and other essential voiceprint features of the target speaker. Finally, this rigorous selection mechanism effectively prevents "false negative samples" (samples actually from the same person but with incorrect labels) from being mixed into the training, avoiding damage to the model's feature space and thus training a speech synthesis model with extremely high identity recognition and intra-class compactness.

[0199] In some embodiments, Figure 9 This is a flowchart illustrating the method for determining the voiceprint comparison reward value provided in an embodiment of this application. See also... Figure 9 Step 108 can be achieved through steps 1081 to 1083:

[0200] In step 1081, the voiceprint contrast loss value is determined based on the similarity between the first voiceprint feature and the third voiceprint feature of the contrasting speech, as well as the similarity between the first voiceprint feature and the second voiceprint feature.

[0201] The voiceprint contrast loss value is a scalar value calculated based on triplet loss or contrastive learning principle. It is used to measure the effect of the generated first target speech in the feature space in bringing it closer to the reference speech and pushing it away from the contrast speech. The smaller the value, the better the discrimination effect. Its purpose in this application is to transform the vector distance relationship of voiceprint features into an optimizable scalar index.

[0202] In practical applications, the cosine similarity between the first voiceprint feature and the second voiceprint feature (positive sample) is first calculated, and the cosine similarity between the first voiceprint feature and the third voiceprint feature (negative sample) is also calculated. The voiceprint contrast loss value is then calculated based on the loss function of contrastive learning.

[0203] In step 1082, multiple pre-constructed loss value intervals with no overlap are obtained. Different loss value intervals correspond to different voiceprint reward values, and the smaller the loss value interval, the larger the corresponding voiceprint reward value.

[0204] The loss value interval is a continuous range of loss values ​​divided into several discrete, non-overlapping segments; its purpose in this application is to perform hierarchical quantization of the loss value in order to assign a stepped reward signal.

[0205] In practical applications, a set of boundary thresholds (such as 0.1, 0.3, 0.5) are predefined, and multiple loss value intervals are constructed accordingly (such as interval A: [0, 0.1), interval B: [0.1, 0.3), interval C: [0.3, 0.5), interval D: [0.5, positive infinity)). A fixed reward score is assigned to each loss value interval as a voiceprint reward value, following the principle of "better performance, better reward". For example, interval A corresponds to +10 points, interval B corresponds to +5 points, interval C corresponds to +1 point, and interval D corresponds to -5 points (penalty).

[0206] In step 1083, the target loss value interval to which the voiceprint contrast loss value belongs among multiple loss value intervals is determined, and the voiceprint reward value corresponding to the target loss value interval is determined as the voiceprint contrast reward value.

[0207] The target loss value range is the specific range that is hit by the currently calculated specific loss value; in this application, it is used as an index key value to find and determine the final voiceprint reward value.

[0208] In practical applications, the voiceprint contrast loss value calculated in the first step is compared with the upper and lower bounds of each loss value interval in the second step. Once the loss value is found to fall within a certain interval, that interval is locked as the target loss value interval. The preset mapping score (voiceprint reward value) of the target loss value interval is directly read and output as the final voiceprint contrast reward value to the reinforcement learning algorithm.

[0209] The above approach cleverly solves the problems of numerical instability and large gradient variance that exist when continuous loss values ​​are directly used for reinforcement learning, through a three-level processing mechanism of "loss calculation - interval quantization - reward mapping". First, it filters out the small fluctuations in voiceprint features and provides a more stable and clear step-by-step optimization objective for the speech synthesis model through discrete interval mapping, avoiding model oscillations on small numerical errors. Second, this non-linear reward design allows for the artificial adjustment of optimization preferences. For example, by setting a steep score difference, it provides the model with strong positive feedback at key similarity thresholds, thereby accelerating the convergence of the model's ability to distinguish between "similar" and "unsimilar" features. Finally, it encapsulates the complex voiceprint comparison logic into a standardized reward value, which can be easily weighted and fused with rewards from other dimensions (such as prosody and accuracy), achieving dimensional unification for multi-objective optimization.

[0210] See also Figure 7 Step 105 can be achieved through step 1051:

[0211] In step 1051, the speech synthesis model is trained based on speech length reward value, entropy regularization reward value, prosody alignment reward value, accuracy reward value, voiceprint similarity reward value, and voiceprint comparison reward value.

[0212] In practical applications, preset weight coefficients can be assigned to the above six reward values, and they can be weighted and summed to obtain the total reward value. The total reward value can then be backpropagated using a reinforcement learning algorithm to update the parameters of the policy network in the speech synthesis model, so that it tends to obtain a higher total score in subsequent generation.

[0213] Through the above approach, a comprehensive and multi-dimensional constraint system is constructed, effectively resolving the contradiction between "diversity" and "controllability" in end-to-end speech synthesis. First, an accuracy reward value (intelligibility constraint) is introduced as a "safety net" to prevent speech synthesis models from producing unclear pronunciation or semantic loss in pursuit of high entropy (diversity), ensuring the accuracy of the generated target speech content. Second, by combining voiceprint similarity and voiceprint contrast reward values ​​(contrastive learning mechanism), not only is the generated timbre required to be close to the reference speech, but it is also explicitly required to be far from non-target timbre (the timbre of the contrast speech), improving the timbre cloning purity and identity decoupling ability of the speech synthesis model in multi-speaker scenarios, avoiding timbre leakage or averaging problems. Finally, content accuracy, timbre fidelity, and the prosodic alignment reward value, duration accuracy (speech length reward value), and entropy regularization reward from previous steps are integrated into the same reinforcement learning framework, enabling the speech synthesis model to autonomously learn expressive and natural prosodic rhythms while ensuring speech clarity and timbre realism.

[0214] In some embodiments, the "synthesizing of the first target speech based on the first reference speech features and the first text features" in step 101 can be implemented as follows: under the constraint of the first reference speech features, the first text features are sampled multiple times to generate multiple candidate speech features, and each candidate speech feature is decoded to obtain multiple first target speech features. Correspondingly, Figure 7 Step 1051 can be implemented as follows: For each first target speech, determine the total reward value of the first target speech based on the speech length reward value, entropy regularization reward value, prosody alignment reward value, accuracy reward value, voiceprint similarity reward value, and voiceprint comparison reward value corresponding to the first target speech; determine the relative advantage value of the first target speech based on the statistical distribution of the total reward value of the first target speech among the total reward values ​​of multiple first target speeches; update the policy parameters of the speech synthesis model based on the relative advantage value of each first target speech to maximize the total reward value of all first target speeches; wherein, the policy parameters are used to guide the speech synthesis model to perform multiple feature samplings on the first text features.

[0215] Feature sampling refers to randomly extracting specific feature values ​​based on probability distribution during the inference process of a speech synthesis model, instead of directly selecting the output with the highest probability. In this application, its purpose is to introduce randomness and diversity into the generation process, allowing the same input to produce multiple different first target speech, supporting subsequent exploratory training. The relative advantage value refers to the superiority of the total reward value of a particular first target speech relative to the current batch average or benchmark value. In this application, it serves as a weight coefficient for gradient updates, indicating whether the speech synthesis model should strengthen or suppress the probability of generating that first target speech. Policy parameters refer to the neural network weights in the speech synthesis model responsible for determining the feature generation probability distribution. In this application, they serve as optimization objects, adjusting the generation strategy by updating parameters to make the generated first target speech more consistent with multi-dimensional reward objectives.

[0216] In practical applications, the first reference speech features are used as style constraints, and the first text features are used as content inputs to the encoder of the speech synthesis model. When the decoder generates acoustic features (such as Mel spectrum) at each step, it outputs a probability distribution (such as the mean and variance of a Gaussian distribution). Based on this probability distribution, it independently performs N (such as N=4) random sampling operations to obtain N sets of slightly different feature sequences (i.e., candidate speech features). Then, the vocoder is used to restore these N sets of feature sequences to waveforms, thereby obtaining N first target speech.

[0217] For each generated first target speech, six reward values ​​are independently calculated (speech length reward, entropy regularization reward, prosodic alignment reward, accuracy reward, voiceprint similarity reward, and voiceprint comparison reward). A set of weighted coefficient vectors is set, and the six reward values ​​are weighted and summed. The resulting scalar result is the total reward value of that first target speech. Then, the total reward values ​​of N first target speech generated in the same sampling are collected, and the expected value (mean) and standard deviation of this set of values ​​are calculated. For the i-th first target speech, its total reward value is subtracted from the mean, and then divided by the standard deviation. The resulting value is taken as the relative advantage value of that first target speech.

[0218] Finally, a loss function is constructed using the policy gradient algorithm, which is the negative log probability multiplied by the relative advantage value. For samples with a positive advantage value, the speech synthesis model increases the probability of generating the features of that sample through backpropagation, while for samples with a negative advantage value, it decreases the probability of generating them. The policy parameters are then updated iteratively through an optimizer.

[0219] Through the above approach, a training mechanism with self-game properties is constructed using a reinforcement learning closed loop of sampling-evaluation-advantage calculation. First, the multiple sampling mechanism breaks the averaging effect caused by traditional maximum likelihood estimation training, allowing the speech synthesis model to explore multiple possibilities of sound and learn the best from them, thus solving the problem of bland prosody in synthesized speech. Second, using relative advantage values ​​based on statistical distribution instead of absolute reward values ​​effectively eliminates reward fluctuations caused by differences in the basic difficulty of different samples, significantly reducing gradient variance and making the training process more stable and convergent. Finally, this strategy of directly optimizing the generated results end-to-end can simultaneously improve multiple non-differential indicators such as accuracy, voiceprint, and prosody, enabling the final speech synthesis model to have robust capabilities in generating high-quality, highly realistic speech under complex constraints.

[0220] The speech synthesis method provided in the embodiments of this application will be described below with reference to the accompanying drawings. As mentioned above, the electronic device 400 that implements the speech synthesis method of the embodiments of this application can be a terminal, a server, or a combination of both. Therefore, the executing entity of each step will not be described again below.

[0221] The speech synthesis method of this application will be described using a terminal as an example. See also... Figure 10 , Figure 10 This is a flowchart illustrating the speech synthesis method provided in the embodiments of this application, which will be combined with... Figure 10 The steps shown are explained.

[0222] In step 201, the text to be processed and the voiceprint reference identifier are obtained.

[0223] A voiceprint reference identifier is a data index used to uniquely identify a specific speaker or vocal style. It can take the form of a speaker number, a specific reference audio file itself, or a pre-extracted and stored voiceprint embedding vector. In this application, it is used as conditional constraint information input to the speech synthesis model to guide the speech synthesis model to generate speech with specific vocal attributes.

[0224] In practical applications, the terminal can receive speech synthesis requests sent by the client through the application programming interface. The data packet of the request contains the string content to be synthesized as the text to be processed, as well as the target speaker code selected by the user or an audio file uploaded by the user as the voiceprint reference identifier. The backend of the terminal cleans and regularizes the received text (such as converting numbers into Chinese characters), and parses the voiceprint reference identifier for subsequent use.

[0225] In step 202, the text to be processed and the voiceprint reference identifier are input into the pre-trained speech synthesis model. The second text features of the text to be processed are extracted by the speech synthesis model, and the second target speech is synthesized based on the second reference speech features corresponding to the voiceprint reference identifier and the second text features. The voiceprint of the second target speech is consistent with the voiceprint indicated by the voiceprint reference identifier.

[0226] The second text feature refers to the sequence of hidden context vectors generated by the text encoder during the inference phase of the speech synthesis model after performing deep semantic and prosodic analysis on the text to be processed. In this application, its purpose is to determine the linguistic content, pronunciation, and basic prosodic structure of the synthesized speech. The second reference speech feature refers to the high-dimensional voiceprint representation vector mapped or extracted from the voiceprint reference identifier. In this application, its purpose is to provide global timbre style control for the synthesis process, ensuring that the generated speech sounds like the target speaker.

[0227] In practical applications, the text to be processed is converted into a phoneme sequence and fed into the text encoder of a pre-trained speech synthesis model to generate a hidden layer vector sequence, i.e., the second text feature. Simultaneously, if the voiceprint reference identifier is an identifier, the corresponding vector is looked up from a pre-set embedding table; if it is audio, the vector is extracted using a voiceprint extractor to obtain the second reference speech feature. Subsequently, the second text feature and the second reference speech feature are fused through concatenation or an attention mechanism, input into the decoder to predict the Mel spectrum, and finally, a vocoder is used to convert the spectrum into a time-domain waveform, thus obtaining the second target speech. The second target speech is used to play to the user.

[0228] Among them, the speech synthesis model can be based on Figure 3The training method shown is used for training. Specifically, the speech synthesis model is trained based on speech length reward, entropy regularization reward, and prosody alignment reward. The speech length reward is determined based on the deviation between the first speech rate of the first target speech and the second speech rate of the reference speech. The first target speech is synthesized by the speech synthesis model based on the first text features of the text sample and the first reference speech features of the reference speech. The entropy regularization reward is determined based on the difference between the distribution entropy of the first target speech and the target entropy value of the text sample. The prosody alignment reward is determined based on the matching degree between the first pause structure of the first target speech and the second pause structure pre-constructed for the text sample.

[0229] The training process for speech synthesis models can be found in the documentation for... Figure 3 The description of the embodiments is omitted here.

[0230] This approach achieves complete decoupling and flexible recombination of text content and timbre features. On the application side, there's no need to retrain the model; simply changing the input speaker reference identifier allows the generation of thousands of different second target speech timbres using the same pre-trained model, reducing the deployment cost and storage space of multi-speaker speech synthesis systems. Furthermore, thanks to the contrastive learning and multi-dimensional reward mechanism introduced during training, the second target speech generated by this inference method accurately replicates the subtle speaker features indicated by the speaker reference identifier in terms of timbre similarity, while maintaining extremely high speech naturalness and prosodic expressiveness.

[0231] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario. First, the terms used in the embodiments of this application will be explained, including:

[0232] 1) Text-to-speech generation is the process of automatically converting input text into audible speech, with the goal of generating clear, natural, rhythmic, and controllable speech.

[0233] 2) Speech discrete representation, also known as speech token, compresses and quantizes continuous speech signals into discrete symbol sequences through a speech encoder, which is used to facilitate autoregressive modeling and streaming generation.

[0234] 3) Single-Codebook: Speech discrete representation uses only one codebook to carry speech information. Compared with multi-codebook schemes, it is more compact and has lower latency, but it is more prone to decoding instability and prosodic distortion.

[0235] 4) The speech large language model is based on an autoregressive architecture. It predicts speech feature sequences (discrete speech representation sequences) under text conditions, and then restores the waveform through a decoder or vocoder.

[0236] 5) Group relative policy optimization, a reinforcement learning-based policy optimization method, samples multiple outputs (group) for the same input at once, and updates the policy based on the relative advantage within the group (advantage normalization), reducing reward noise and training instability.

[0237] 6) Multiple rewards: For the training task of speech large language model, multiple computable reward items (such as length consistency, stability, prosodic alignment, intelligibility, speaker similarity, etc.) are designed and combined according to weights to form the total reward signal.

[0238] 7) Length penalty: penalizes the synthesized speech length or duration if it deviates from the target range, suppressing distortions such as excessively long trailing or excessively short truncation.

[0239] 8) Entropy regularization encourages the policy distribution to maintain an appropriate entropy value, reducing decoding instability caused by collapse and abnormal peak selection.

[0240] 9) Prosody alignment reward: Matches and scores the pauses or rhythmic structures that can be inferred from the text with the actual pauses or rhythmic structures of the synthesized speech, and is used to explicitly guide prosody.

[0241] In related technologies, speech synthesis is a key technology for converting text information into audible speech, and it has been widely used in scenarios such as intelligent assistants, content production, intelligent customer service, digital human broadcasting, and accessibility assistance. In recent years, with the development of large language model-related technologies, the TTS field has gradually formed an autoregressive generation paradigm based on discrete speech representation: the waveform is encoded into a discrete representation sequence by a speech encoder, then the autoregressive model predicts the discrete speech identifier sequence under text conditions, and finally the speech is restored by a decoder. This paradigm has advantages in system uniformity, scalability, and streaming generation, especially the single codebook scheme, because the discrete representation sequence is more compact and the link is simpler, making it suitable for low latency and online deployment.

[0242] From the perspective of zero-shot TTS technology systems, related solutions typically fall into three categories: First, LLM-based discrete acoustic representation generation schemes, where the speech synthesis model directly predicts speech token sequences and decodes them to obtain speech, resulting in a relatively simple and stream-compatible structure; Second, generative acoustic generation schemes such as diffusion models or stream matching, which typically model in continuous space, achieving high naturalness and fine-grained control, but with significant inference computation overhead; Third, coarse-to-fine pipeline schemes combining autoregressive models with diffusion or stream matching modules, where the upstream autoregressive model predicts semantics or coarse-grained speech tokens, and the downstream module compensates for and refines acoustic details, achieving a balance between semantic accuracy and sound quality to some extent, but with complex system composition and long link lengths.

[0243] In the implementation path of single-codebook speech large language models, there are two tendencies: semantic-dominated and joint semantic and acoustic modeling. Semantic-dominated schemes typically rely on subsequent modules to supplement acoustic details; joint semantic and acoustic modeling schemes simultaneously carry linguistic content and paralinguistic information within a single speech token sequence, possessing stronger expressive power and lower end-to-end latency. However, they are more prone to introducing suboptimal decoding strategies during unified modeling and autoregressive sampling, leading to problems such as prosodic instability, decreased speaker consistency, and uncontrolled duration distribution. Since these problems are essentially closely related to autoregressive sampling strategies, reinforcement learning is considered an effective way to directly optimize the policies of this type of speech synthesis model. Related technologies have attempted to use computable indicators such as automatic speech recognition error and speaker similarity as reward signals, and fine-tuned them using frameworks such as Proximal Policy Optimization (PPO) and Monte Carlo Policy Gradient (REINFORCE). Some studies have also proposed differentiable reward frameworks, transforming reward optimization into a supervised objective through differentiable approximation. Meanwhile, uncertainty-aware optimization and back-reasoning sample construction methods have also been used to alleviate the problems of feedback sparsity and training instability. However, the above-mentioned solutions still have shortcomings in terms of training complexity, dependence on preference data, scalability, and the ability to explicitly constrain prosodic structure, and it is still difficult to form a systematic solution that simultaneously satisfies stability, prosodic naturalness, and speaker consistency.

[0244] In challenging applications such as zero-sample and long text processing, existing autoregressive sampling-based speech-language models still generally suffer from unstable prosodic organization. This manifests as inconsistencies between pause positions, sentence structure, rhythm arrangement, and natural language reading habits, thus affecting the naturalness and listenability of the speech. Furthermore, under conditions involving multiple speakers or reference timbres, speaker feature drift easily occurs during generation, reducing the consistency of timbre within the same speech segment. Simultaneously, autoregressive generation is prone to unstable duration distribution, including premature termination, abnormal prolongation, local repetition, and stuttering. These problems are more pronounced in long sentences, complex punctuation, and cross-linguistic text scenarios, leading to insufficient reliability and controllability of the output.

[0245] To address the aforementioned issues, relevant engineering practices often improve sound quality and detail coherence by introducing downstream acoustic refinement modules such as diffusion or flow matching, or by employing coarse-to-fine pipelines to enhance acoustic reconstruction. However, these post-processing modules primarily operate at the acoustic detail level, lacking targeted strategy-level correction mechanisms for issues such as pause structure deviations, unreasonable rhythmic organization, and loss of duration control caused by upstream autoregressive strategies. Therefore, they struggle to fundamentally improve prosodic consistency and generation stability. Furthermore, the methods in these technologies still face limitations in large-scale applications: PPO-based methods typically require components such as value models, resulting in more complex training paths and higher stability requirements; direct preference optimization-based methods rely on high-quality preference pairs, significantly increasing data acquisition and consistency control costs when dealing with multilingual, multi-speaker, and multi-style requirements, and are highly sensitive to preference noise; differentiable reward frameworks usually require additional training of evaluation models or the introduction of approximate sampling mechanisms, increasing training resource overhead. More importantly, most related technologies still rely on indirect indicators or weak rules to constrain prosody, lacking a general mechanism to directly convert interpretable structures such as pause structures and prosodic templates into training signals. This makes it difficult for speech synthesis models to stably obtain rhythms and phrasing patterns that conform to human preferences.

[0246] To address the aforementioned shortcomings, the inventors propose a scalable reinforcement learning optimization framework for large language models of single-codebook languages. This framework uses group relative policy optimization to stably update policy parameters under a mechanism of intra-group relative advantage normalization, reducing training oscillations and decreasing reliance on dense preferences for data. In reward design, objective metrics such as intelligibility and speaker similarity (based on cosine similarity of speaker verification embeddings) are jointly modeled with rule-based rewards such as length consistency constraints and entropy constraints to suppress premature termination and abnormal prolongation, and reduce generation random fluctuations. Furthermore, pause structure templates generated offline by the inference-based large model are introduced. During online training, pause structures are extracted from synthesized speech and matched for scoring, thus forming an explicit supervised closed loop for prosodic rhythm. Through this combination of technologies, this application can improve the stability, prosodic naturalness, and speaker consistency of speech generation while maintaining the compactness of the single-codebook link and streaming capabilities. It is also suitable for engineering training and deployment under large-scale data and different model sizes.

[0247] This application applies to text-driven zero-shot speech synthesis services, including but not limited to dialogue assistants and intelligent customer service, audiobooks and news broadcasts, short videos and film dubbing, digital human broadcasting and multi-role reading, accessible reading and assisted communication, and other application scenarios. When applying a speech synthesis model, its input typically includes at least the text to be synthesized, and optionally includes a reference voice of the target speaker or a speaker identifier to specify timbre consistency requirements. The output of the speech synthesis model is a speech waveform that is consistent with the text content, has a timbre consistent with the target speaker, and features more natural and stable pauses, phrasing, and rhythm. To meet the requirements of online deployment and streaming output, this invention is based on a single-codebook large language model, maintaining a compact link, and directly optimizing the autoregressive decoding strategy through reinforcement learning to reduce the risk of long text generation failure and prosodic instability.

[0248] Figure 11 This is a schematic diagram of the architecture of the speech synthesis model provided in the embodiments of this application. See also: Figure 11 The speech synthesis model includes a text processing module, a speech representation module, a single-codebook large language model, a speech decoding module, a multi-reward evaluation module, and an optimization module. The training process of the speech synthesis model mainly involves feature extraction of data, model inference sampling, speech decoding and generation, multi-dimensional reward evaluation, and parameter optimization and update closed loop, specifically including the following steps and module interaction processes:

[0249] First, the speech synthesis model receives two input data sources: the input text S (i.e., the text sample) and the reference audio. (i.e., reference audio). For the input text S, it is fed into the text processing module for preprocessing and feature transformation to obtain the corresponding text representation sequence (text token sequence, i.e., the first text feature); simultaneously, for the reference audio... The input is then fed into the speech representation module for acoustic feature extraction and discretization to obtain the corresponding reference speech representation sequence (reference speech token sequence, also known as the first reference speech feature).

[0250] Subsequently, the generated text token sequence and the reference speech token sequence are used as conditional inputs to the single-codebook speech large language model. This single-codebook speech large language model performs autoregressive prediction or probability sampling based on the input sequence information, and outputs N sets of sampled speech representation sequences (sampled speech token sequences, i.e., multiple candidate speech features).

[0251] Next, these N sets of sampled speech token sequences are input to the speech decoding module. The speech decoding module is responsible for restoring the discrete sampled speech token sequences into continuous waveform data, thereby generating N sets of combined speech. ,in, Represents the generated first Synthesized speech, This represents the total number of samples generated, i.e., the total number of sampled speech representation sequences. These N combinations of generated speech are then fed into a multi-reward evaluation module. The multi-reward evaluation module comprehensively scores or evaluates the quality of the generated synthesized speech based on various preset reward mechanisms (such as speech rate, prosody, and voiceprint consistency). The evaluation results are transmitted to an optimization module, which calculates the gradient or policy advantage based on the evaluation feedback and updates the parameters of the single-codebook speech large language model.

[0252] Through the above steps, a complete reinforcement learning loop is formed, continuously iterating and optimizing the parameters of the large language model to improve the quality of speech synthesis.

[0253] Figure 12 This is a flowchart of the reward evaluation process for the multi-reward evaluation module provided in this application embodiment. See also: Figure 12 , Figure 12 This document illustrates a detailed architecture diagram of the multi-reward evaluation module provided in an embodiment of this application. The multi-reward evaluation module comprises six sub-modules: an intelligibility reward module, a speaker similarity reward module, a length penalty reward module, an entropy regularization reward module, an anchor point comparison reward module, and a prosodic alignment reward module. The multi-reward evaluation module primarily uses synthesized speech and reference audio as input. It calculates reward values ​​for different dimensions through the six parallel sub-modules, and finally sums them to obtain the total reward. The specific processing flow of each sub-module is as follows:

[0254] For the intelligibility reward module, the synthesized speech is first input into the pre-trained speech recognition model for recognition to obtain the corresponding transcribed text (i.e., the text to be verified); then, the edit distance is calculated based on the transcribed text and the real text (i.e., the text sample), and the intelligibility reward (i.e., the accuracy reward value) is output according to the calculation result.

[0255] For the speaker similarity reward module, the synthesized speech is input into the pre-trained voice verification model to extract speaker embedding features (i.e., the voiceprint features of the synthesized speech and the voiceprint features of the reference audio). Then, the cosine similarity calculation method is used to calculate the similarity between the voiceprint features of the synthesized speech (first voiceprint feature) and the voiceprint features of the reference audio (second voiceprint feature), and the speaker similarity reward (i.e., voiceprint similarity reward value) is output.

[0256] For the length penalty reward module, the audio duration of the input synthesized speech is first calculated; then, the degree of deviation of the speech rate is determined based on the reference speech rate threshold (i.e., the tolerance ratio range), and the length penalty reward (i.e., the speech length reward value) is output according to the degree of deviation.

[0257] For the entropy regularization reward module, the synthetic speech entropy (i.e., the first mean) generated by the speech synthesis model is first calculated; then, it is compared with the preset high-quality audio entropy (i.e., the target entropy value), and the entropy regularization reward (i.e., the entropy regularization reward value) is output based on the difference value.

[0258] For the anchor point comparison reward module, in addition to receiving synthesized speech and reference audio, it also needs to acquire a negative sample audio set (i.e., a set of comparison speech). All audio is processed through a feature extraction model. For the target audio, speaker embeddings are extracted; for the negative sample audio set, a negative sample anchor point set is extracted. Subsequently, the comparison distance is calculated based on these two types of features, and the anchor point comparison reward (i.e., the voiceprint comparison reward value) is output.

[0259] The prosodic alignment reward module comprises two processing stages: offline and online. In the offline stage, a pre-trained inference model is used to predict and construct a set of pause templates for specific texts based on the training text dataset. In the online stage, the synthesized speech is processed using a speech recognition model (or a forced alignment tool) to extract audio pause timestamps. These timestamps are then matched against the pause template set constructed in the offline stage according to pause rules, and the prosodic alignment reward (i.e., the prosodic alignment reward value) is output based on the matching degree.

[0260] Finally, the reward values ​​output by the above six modules are weighted and summed to obtain the total reward used to optimize the speech synthesis model.

[0261] Next, combined Figure 11 and Figure 12 The training method of the speech synthesis model in the embodiments of this application will be described in detail.

[0262] This application presents a reinforcement learning optimization method for single-codebook text-to-speech large language models, aimed at improving the performance of autoregressive speech synthesis in terms of generation stability, prosodic naturalness, and speaker consistency. This application is based on single-codebook discrete speech representation: the speech waveform is encoded into a single-codebook speech token sequence by a single-layer vector quantization speech large language model, and the decoder can restore the speech token sequence to the speech waveform to obtain synthesized speech. On this basis, the single-codebook speech large language model takes the text token sequence and optional speaker conditions as input, autoregressively generates a speech token sequence, and decodes it to obtain synthesized speech. This application models this autoregressive generation process as a policy. The sequential decision-making process optimizes the policy parameters through group relative policy optimization. Iterative optimization is performed to align it with natural reading preferences under the constraints of multi-objective reward signals.

[0263] like Figure 11As shown, the speech synthesis model includes at least a text processing module, a speech representation module, a single-codebook speech large language model, a speech decoding module, a multi-reward evaluation module, and an optimization module. The text processing module normalizes the input text and converts it into a text token sequence; the single-codebook speech large language model generates a speech token sequence under the constraints of the text token sequence and optional speaker conditions; the speech decoding module restores the speech token sequence to the generated speech waveform, obtaining the synthesized speech; the multi-reward evaluation module calculates multi-dimensional rewards for the synthesized speech and the generation process; the optimization module normalizes the relative advantage within each group based on the reward results of the candidate groups and updates the policy parameters, thus forming a closed-loop training mechanism. For the prosodic alignment reward, it includes two stages: offline prosodic template generation and online prosodic structure extraction and matching. In the offline stage, the inference large language model generates a set of pause templates from the training text dataset; in the online stage, pause structures are extracted from the generated speech and matched with the template set to obtain the reward value.

[0264] Specifically, this application performs group sampling on the same input condition during the training phase to obtain multiple candidate outputs and calculate the reward. Then, it updates the policy parameters through group relative policy optimization. The specific process can be summarized as follows: given the input text S and optional reference audio... Under the given conditions, use the current strategy. N candidate speech token sequences (i.e., sampled speech token sequences) are generated through sampling. Each candidate speech token sequence is decoded into synthesized speech A, and the multi-reward evaluation module calculates the individual rewards and the total reward. The optimization module performs relative comparison of rewards within the candidate groups and performs advantage normalization to obtain a stable update signal for iteratively updating the parameters. In the above process, the generation process of a single-codebook speech large language model can be represented as a trajectory. ,in, It can represent a sequence of candidate voice tokens; That is, all the known information that a single-codebook speech large language model refers to when predicting the next articulation point; That is, at time T, the speech token predicted by the single-codebook speech big language model for the current step.

[0265] The optimization objective of the single-codebook speech large language model is to maximize the expected cumulative reward, expressed by formula (1):

[0266] (1)

[0267] in, This represents the cumulative reward of the speech synthesis model. That is, all the known information that a single-codebook speech large language model refers to when predicting the next articulation point; That is, at time t, the speech token predicted by the single-codebook speech big language model for the current step; express Instant reward, indicating the state Take action below Contribution to the final quality of synthesized speech.

[0268] in, A weighted combination of multiple explainable rewards is used, as detailed in formula (2):

[0269] (2)

[0270] in, Indicates a comprehensibility bonus; Indicates speaker similarity reward; Indicates length penalty / reward; This indicates an entropy-regularized reward; Indicates a reward for rhythmic alignment; Indicates the anchor point comparison reward; , , , , , These represent the weighting coefficients set for each reward. These coefficients are adjustable and are used to balance objectives such as content consistency, speaker consistency, duration stability, generation stability, and prosodic alignment. The optimization module uses a relative advantage normalization mechanism within candidate groups to keep the advantage function within a stable range, thereby reducing the impact of reward noise on training and improving training stability, while avoiding dependence on dense preferences for data.

[0271] The multi-reward evaluation module calculates six types of rewards for each candidate generated result and combines them into a total reward signal. Among them, the intelligibility reward is used to constrain the generated speech to be consistent with the input text; the speaker similarity reward is used to constrain the timbre to be consistent with the target speaker; the length penalty reward is used to constrain the distribution of generated duration and suppress truncation and trailing; the entropy regularization reward is used to suppress random fluctuations caused by unstable sampling; and the prosodic alignment reward is used to explicitly supervise the pause structure and promote the rhythm of sentence breaks to conform to natural reading preferences. The calculation method of each reward item is explained below.

[0272] Explanation of the comprehensibility bonus.

[0273] This application uses a pre-trained automatic speech recognition model to transcribe the synthesized speech A. And calculate the edit distance normalization value corresponding to the character error rate or word error rate with the input text S to construct the comprehensibility reward, as shown in formula (3):

[0274] (3)

[0275] in, Indicates a comprehensibility reward. This represents the transcribed text of the synthesized speech A (i.e., the text to be verified). This represents the input text (i.e., a text sample). This represents the edit distance between the transcribed text and the input text. This intelligibility reward encourages single-codebook speech-language models to generate speech that is semantically consistent with the input text, suppressing misreading, omissions, and semantic deviations.

[0276] Explanation of speaker similarity rewards.

[0277] This application employs a sound and color verification model to verify the synthesized speech A and the reference audio respectively. Extracting speaker embeddings , And calculate the cosine similarity as the speaker similarity reward, see formula (4) for details:

[0278] (4)

[0279] in, Indicates speaker similarity reward; This represents the embedding features of the synthesized speech, i.e., the first voiceprint feature; This represents the embedded features of the reference audio, i.e., the second voiceprint features. Speaker similarity rewards are used to constrain timbre consistency and reduce the risk of speaker feature drift during long text generation.

[0280] Explanation of the length penalty reward.

[0281] This application addresses the premature termination and abnormal tailing problems common in autoregressive models by introducing a length penalty reward to stabilize the generated duration distribution. Let the synthesized speech duration be T, and the target duration be estimated from the ratio of the reference text length to the reference speech rate. Let the reference speech rate be calculated from the paired reference input, with a tolerance interval of [a, b]. Then the length reward is defined as formula (5):

[0282] (5)

[0283] in, Indicates length penalty / reward; This indicates the speech rate of the synthesized speech, i.e., the first speech rate. Indicates the length of the input text. Indicates the duration of the synthesized speech; Indicates a reference speaking speed; express Not belonging to The length bonus is used to constrain the speech rate ratio within a reasonable range, thereby reducing the probability of truncation and excessive lengthening, and improving the controllability and stability of output duration.

[0284] The entropy regularization reward is explained.

[0285] This application suppresses unstable sampling by regularizing the policy entropy. Let the average value of the generated sequence token distribution entropy be... And estimate the target entropy using high-quality samples. Then the entropy reward is defined by formula (6):

[0286] (6)

[0287] in, This indicates an entropy-regularized reward; This represents the target entropy, i.e., the target entropy value. This represents the average value of the distribution entropy of the generated token sequence, i.e., the first mean. This indicates a preset hyperparameter (positive number), and the preceding negative sign indicates that this is a "penalty" mechanism that controls the severity of the penalty.

[0288] Entropy regularization penalties are imposed on situations where the entropy exceeds the target entropy, which is used to suppress rhythmic jitter and unstable generation paths caused by excessive randomness, thereby obtaining smoother and more predictable generation results.

[0289] Explanation of the anchor point comparison reward.

[0290] To improve speaker consistency in single-codebook text-to-speech large language models in zero-shot and long-text synthesis scenarios, and to suppress speaker feature drift during generation, this scheme sets up anchor-point contrast rewards in the multi-reward evaluation module. Anchor-point contrast rewards construct contrast constraints by introducing an anchor-point negative sample speech set, ensuring that the generated speech not only approaches the target reference speaker in the speaker embedding space but also moves away from non-target speakers, thereby obtaining a more discriminative reward signal and improving the stability and robustness of reinforcement learning optimization. The calculation process of anchor-point contrast rewards includes four parts: anchor-point negative sample set construction, speaker embedding extraction, contrast distance calculation, and reward mapping. First, the anchor-point negative sample speech set... The source can be a training corpus, a constructed speaker voice database, or historical speech resources available from the business side. The anchor point negative sample set is used to provide non-target speaker constraint signals, with the basic requirement being that they are similar to the target reference speech. They do not belong to the same speaker. To ensure the validity, comparability, and stability of the reward signal of negative samples, the anchor negative sample speech set can be constructed and screened according to the following criteria, which can be used individually or in combination:

[0291] First, the identity mutual exclusion standard. When the corpus contains speaker identifiers, those that do not match the speaker's identity are directly removed. For speech samples with the same speaker identifier, only the speech samples from different speakers are retained as negative sample candidates. When the corpus does not have a clear speaker identifier, the speaker embedding extraction model can first extract the embeddings from the corpus, and then obtain a set of samples suspected to be from the same speaker through clustering or thresholding. Samples with embedding similarity higher than the threshold for identifying the same speaker are identified as belonging to the same speaker and are removed, thus forming a similarity with the same speaker. A set of mutually exclusive negative sample candidates. To reduce training noise caused by misjudgments, a safety margin can be introduced when removing samples from the same speaker, that is, samples with similarity close to a threshold are removed by a stricter threshold or their sampling probability is reduced.

[0292] Secondly, quality and comparability standards. To avoid the reward signal being dominated by noise, differences in recording conditions, or abnormal speech segments, anchor negative samples should ideally meet basic audio quality requirements and be comparable to the target reference speech. The following screening rules can be adopted: remove samples with excessively low signal-to-noise ratio, severe reverberation, strong background music, obvious clipping distortion, multiple speakers, or long non-speech segments; remove excessively short speech segments (e.g., below a preset minimum duration threshold) to avoid unstable embedding; uniformly resample the sampling rate to a preset value and uniformly use single-channel input; normalize the amplitude or prune the dynamic range to improve embedding consistency. The above quality screening can be performed offline when constructing the anchor library or as a filtering condition when extracting anchor samples online.

[0293] Third, the selection criteria for difficult-to-bear samples. To enhance the sensitivity of anchor-point contrast reward to speaker drift, this scheme can select difficult-to-bear samples, that is, under the premise of satisfying identity mutual exclusion constraints, selecting samples that are similar to the target reference speech in the embedding space. Negative samples that are highly similar but still below the threshold for the same speaker. Specifically, candidate negative samples can be calculated first. and The similarity between speaker embeddings is used to limit them to a preset range. Within this framework, insufficient learning signals are eliminated by removing overly similar negative samples (too low similarity), while avoiding the misclassification of samples from suspected speakers (too high similarity) into the negative sample set. Difficult negative samples are preferably mixed with random negative samples, making the training both discriminative and stable.

[0294] Fourth, distribution matching criteria (optional). To reduce the interference of factors irrelevant to the speaker on the reward, the anchor negative samples can be matched with... The distribution attributes are constrained to ensure that the speakers are as similar as possible in terms of language, gender, vocal range, speaking style, recording conditions, or content length. This makes the contrast constraint more focused on differences in speaker identity rather than differences in content or environment. For cross-language training, anchor negative samples that are in the same language or language set as the target sample can be preferred. For multi-style training, anchor negative samples can be preferred from sample sets of the same or similar styles to improve the directionality of the reward.

[0295] Fifth, the construction method and size control of the anchor negative sample set. The anchor negative sample set can be constructed independently for each training sample, that is, each One corresponding group Alternatively, a shared anchor library can be pre-built and dynamically extracted during training according to the above rules. The number of anchors M can be set to a fixed value or randomly selected within a preset range based on training efficiency and discriminative requirements; and the anchor library can be stratified for sampling, such as by similarity difficulty or by speaker attributes, to stabilize the training signal and avoid anchor sampling bias.

[0296] Secondly, this application employs a unified speaker embedding extraction model (i.e., a feature extraction model) to generate embedding vectors, which is fine-tuned for speaker verification tasks. A unified preprocessing procedure is performed on the input speech, including resampling to a preset sampling rate, single-channelization, and necessary amplitude normalization. Frame-level features are then extracted and converged through a speaker representation network to obtain a fixed-length speaker embedding vector. Let the synthesized speech be A, and the target reference speech be... The anchor point negative sample speech set is The embedding vectors are as follows: , , To ensure comparability of the embedding space, the above embeddings can be further normalized. Furthermore, the anchor point comparison reward uses information noise comparison estimation to construct the comparison distance. The comparison distance is defined as formula (7):

[0297] (7)

[0298] in, This represents the voiceprint contrast loss value. The first voiceprint feature representing synthesized speech; This represents the second voiceprint feature of the reference audio. The third voiceprint feature represents the i-th contrast speech (i.e., the anchor negative sample speech); The temperature parameter is used. A smaller contrast distance indicates that the generated speech is closer to the target reference speaker and has a higher distinguishability from the anchor negative sample. Finally, to stabilize the continuous distance for reinforcement learning training, this application maps the contrast distance to the anchor contrast reward value. Threshold segmentation mapping can be used to reduce reward noise; a threshold is set... And set discrete reward values. , , ,when High rewards at the right time ,when Take a medium reward ,when Take low reward The above mapping can also be expressed as an indicator function, see formula (8):

[0299] (8)

[0300] Among them, I( () is an indicator function that takes the value 1 if the condition is true and 0 otherwise; This refers to the anchor point comparison reward. The anchor point comparison reward can be used as an enhancement of the speaker similarity reward to directly replace the original similarity reward, or it can be used as an independent reward item to be incorporated into a multi-reward weighted combination. It is used to update the strategy parameters during the optimization of the group relative strategy, thereby more effectively suppressing speaker drift and improving timbre consistency and stability when synthesizing long texts and complex conditions.

[0301] The rhyme alignment reward is explained.

[0302] The prosodic alignment reward consists of two stages: offline annotation and online comparison. In the offline annotation stage, an inference-based large model is used to generate one or more reasonable pause templates for the input text S under limited sample cues, forming a set of pause structure templates. (That is, including multiple second pause structures). For Chinese, a hierarchical pause notation system can be used to represent different pause duration levels, while for English, a prosodic hierarchical annotation system can be used to represent prosodic words and phrases. During the online comparison phase, timestamps are analyzed on the generated speech during training, and the predicted pause structures are obtained through rule mapping based on the duration of silence segments. (i.e., the first pause structure), and match it with the template set, when A binary reward is given when any structure in the template set is satisfied, as shown in formula (9):

[0303] (9)

[0304] in, Indicates the predicted pause structure The case where it does not match any pause structure template.

[0305] Through the above mechanism, this application transforms the interpretable prosodic element of pause structure into a scalable training signal, enabling the strategy to converge toward natural reading preferences in terms of sentence segmentation and rhythm organization.

[0306] After training, during the inference phase, the input text and optional speaker conditions are used. The single-codebook speech large language model's policy model autoregressively generates speech tokens and decodes and outputs speech waveforms, supporting low-latency streaming synthesis. For scenarios requiring higher sound quality, acoustic refinement modules such as stream matching can be added after the autoregressive output for post-processing enhancement. However, this post-processing does not change the improvement effect achieved by this application on the stability and prosodic structure of the autoregressive policy through group relative policy optimization and multi-reward mechanism.

[0307] This application addresses the common problems of unstable generation and unnatural prosody in the practical deployment of single-codebook text-to-speech large language models. By introducing a reinforcement learning training framework based on group relative policy optimization, and combining multi-dimensional reward signals such as intelligibility, speaker similarity, length consistency, entropy stability, and prosodic alignment, the autoregressive decoding strategy is directly optimized. This significantly improves the overall usability and engineering stability of speech synthesis without altering the compact link and streaming output advantages of the single-codebook system. Compared to schemes relying solely on supervised learning or single objective metric optimization, this application more effectively reduces the probability of premature termination, abnormal trailing, stuttering, and repetition in long text scenarios, making the duration distribution and speech rate of generated speech more stable and controllable. Simultaneously, entropy constraints suppress unnecessary random fluctuations, improving the determinism and consistency of the decoding process and reducing instability phenomena such as fluctuating prosodic speed and misaligned pauses. Furthermore, this application introduces a prosodic alignment mechanism that generates pause structure templates offline using a large inference model and extracts pause structures from the generated speech for matching and scoring during online training. This enables the model to learn sentence breaks and pause organization that conforms to natural reading habits at the policy level, thereby significantly improving rhythm and prosodic naturalness and making the output speech closer to the pauses and sentence structures of real human reading. At the same time, this application constrains the consistency between the generated speech and the target reference timbre through speaker similarity rewards, which helps reduce speaker timbre drift and enhances timbre stability and identity preservation ability when synthesizing long sentences and multiple sentences in succession. By constraining the consistency between the generated content and the input text through intelligibility rewards, the risk of misreading, omissions, and semantic deviations can be reduced, improving usability and credibility.

[0308] In terms of application effectiveness, this application can provide more stable and reliable speech generation capabilities for scenarios such as voice assistants, intelligent customer service, audiobooks and news broadcasts, digital human broadcasts, and film and short video dubbing. For businesses that require continuous synthesis of long texts, this application can reduce the manual intervention and rework costs caused by generation interruptions and abnormal trailing, and improve end-to-end delivery efficiency. For content production scenarios that emphasize natural sentence breaks and rhythmic expression, this application can improve the consistency of reading rhythm and auditory perception, thereby increasing user acceptance and satisfaction with synthesized speech. For digital human or character dubbing scenarios that emphasize timbre consistency and identity preservation, this application can improve the timbre stability of long speech segments, enhancing character consistency and immersive experience. In addition, since this application uses a group relative strategy optimization mechanism for training optimization, it reduces the dependence on dense preferences for data, has good scalable training and cross-data scale transfer capabilities, is suitable for continuous iteration under different model and data scale conditions, and has strong engineering promotion value and implementation feasibility.

[0309] The following description continues to illustrate the exemplary structure of the speech synthesis model training device 455A provided in the embodiments of this application as a software module. In some embodiments, such as... Figure 2A As shown, the software modules in the training device 455A for the speech synthesis model stored in the memory 450 may include:

[0310] The first speech synthesis module 4551A is used to extract the first text features of the text sample and the first reference speech features of the reference speech through a speech synthesis model, and synthesize the first target speech based on the first reference speech features and the first text features.

[0311] The first reward value determination module 4552A is used to determine the speech length reward value based on the degree of deviation between the first speech rate of the first target speech and the second speech rate of the reference speech.

[0312] The second reward value determination module 4553A is used to parse the pause structure of the first target speech, obtain the first pause structure of the first target speech, and determine the prosody alignment reward value based on the matching degree between the first pause structure and the second pause structure pre-constructed for the text sample.

[0313] The third reward value determination module 4554A is used to determine the distribution entropy of the speech synthesis model when synthesizing the first target speech, and to determine the entropy regularization reward value based on the difference between the distribution entropy and the target entropy value labeled for the text sample.

[0314] The training module 4555A is used to train the speech synthesis model based on speech length reward value, entropy regularization reward value, and prosody alignment reward value.

[0315] In some embodiments, the first reward value determination module 4552A is further configured to determine a first speech rate of the first target speech by a first ratio of the text length of the text sample to the speech duration of the first target speech; determine a second ratio between the first speech rate and a second speech rate of the reference speech; and determine a speech length reward value based on the second ratio and a tolerable ratio range; wherein the tolerable ratio range characterizes the degree to which the first speech rate is allowed to deviate from the second speech rate.

[0316] In some embodiments, the training device 455A for the speech synthesis model further includes an offline construction module for constructing a second pause structure by: constructing a reference example based on a reference text and pause structures annotated for the reference text; constructing an inference prompt based on a text sample and a prompt word template, and inputting the inference prompt and the reference example into a pre-trained inference model, so that the inference model, guided by the inference prompt and required by the reference example, infers the pause structure of the text sample and generates a second pause structure of the text sample.

[0317] In some embodiments, the second reward value determination module 4553A is further configured to perform timestamp parsing on the first target speech to obtain the silence period in the first target speech and the silence duration corresponding to the silence period; and to map the silence duration to a pause structure using a preset mapping rule to obtain the first pause structure of the first target speech; wherein the preset mapping rule is associated with the language corresponding to the text sample.

[0318] In some embodiments, the number of second pause structures is multiple; the second reward value determination module 4553A is further configured to perform structural matching between the first pause structure and each of the second pause structures to obtain a matching degree; if the matching degree between the first pause structure and at least one of the second pause structures exceeds a first matching degree threshold, a preset first value is determined as a rhythm alignment reward value; if the matching degree between the first pause structure and each of the second pause structures does not exceed the first matching degree threshold, a preset second value is determined as a rhythm alignment reward value; the first value is greater than the second value.

[0319] In some embodiments, the number of first target speech is multiple; the third reward value determination module 4554A is further configured to determine the distribution entropy of the speech synthesis model when synthesizing each first target speech, and determine the first mean of all distribution entropies; determine the first difference between the first mean and the target entropy value labeled for the text sample; if the first difference is greater than a preset third value, determine the first product value between the first difference and the preset penalty coefficient as the entropy regularization reward value; if the first difference is less than or equal to the third value, determine the third value as the entropy regularization reward value.

[0320] In some embodiments, the third reward value determination module 4554A is further configured to determine the target entropy value by: acquiring a first speech sample set constructed for text samples, wherein multiple first speech samples in the first speech sample set are synthesized based on text samples; determining the sample distribution entropy when synthesizing each first speech sample, and determining the second mean and standard deviation of all sample distribution entropies; determining the second product value between the standard deviation and a preset adjustment coefficient, and determining the sum of the second product value and the second mean as the target entropy value; wherein the preset adjustment coefficient is used to regulate the coverage of the target entropy value on the first speech sample set, so as to define the allowable fluctuation range of the sample distribution entropy from the second mean.

[0321] In some embodiments, the third reward value determination module 4554A is further configured to acquire multiple candidate speech samples, which are synthesized based on text samples; perform quality evaluation on each candidate speech sample based on a pre-trained speech quality assessment model to obtain the quality score of each candidate speech sample as a text sample, and take the candidate speech sample with a quality score higher than a quality threshold as the first speech sample to obtain a first speech sample set composed of multiple first speech samples; or, for each candidate speech sample, determine the signal-to-noise ratio of the candidate speech sample, and take the candidate speech sample with a signal-to-noise ratio higher than a first signal-to-noise ratio threshold as the first speech sample to obtain a first speech sample set composed of multiple first speech samples.

[0322] In some embodiments, the training device 455A for the speech synthesis model further includes a fourth reward value determination module, which is used to transcribe the first target speech into text to obtain a text to be verified, and determine an accuracy reward value based on the difference between the text to be verified and the text sample; determine a voiceprint similarity reward value based on the similarity between the first voiceprint feature of the first target speech and the second voiceprint feature of the reference speech; determine a voiceprint comparison reward value based on the similarity between the first voiceprint feature and the third voiceprint feature of the comparison speech, as well as the similarity between the first voiceprint feature and the second voiceprint feature; the training module is also used to train the speech synthesis model based on the speech length reward value, entropy regularization reward value, prosody alignment reward value, accuracy reward value, voiceprint similarity reward value, and voiceprint comparison reward value.

[0323] In some embodiments, the fourth reward value determination module is further configured to determine the character error rate of the text to be verified based on the text sample; normalize the character error rate based on the text length of the text sample to obtain the normalized character error rate; and determine the second difference between the preset benchmark value and the normalized character error rate as the accuracy reward value.

[0324] In some embodiments, the fourth reward value determination module is further configured to determine the comparison speech by: constructing a second speech sample set, wherein the second speech samples in the second speech sample set satisfy at least one of the following preset mutual exclusion conditions: the identity label corresponding to the second speech sample is different from the identity label corresponding to the reference speech; the similarity between the voiceprint features of the second speech sample and the second voiceprint features of the reference speech is lower than a similarity threshold; filtering out second speech samples that meet preset filtering conditions from the second speech sample set, and determining the filtered second speech samples as comparison speech; wherein the preset filtering conditions include at least one of the following: the similarity between the voiceprint features of the second speech sample and the second voiceprint features of the reference speech is within a preset threshold range, the upper limit of the preset threshold range is less than the similarity threshold, and the lower limit of the preset threshold range is greater than the mean of the similarity between the voiceprint features of the second speech sample and the second voiceprint features; the signal-to-noise ratio of the second speech sample is higher than a second signal-to-noise ratio threshold; the speech duration of the second speech sample is greater than a duration threshold; the matching degree between the attribute labels of the second speech sample and the attribute labels of the reference speech is higher than a second matching degree threshold.

[0325] In some embodiments, the fourth reward value determination module is further configured to determine a voiceprint comparison loss value based on the similarity between the first voiceprint feature and the third voiceprint feature of the contrasting speech, and the similarity between the first voiceprint feature and the second voiceprint feature; obtain multiple pre-constructed loss value intervals where there is no overlap, with different loss value intervals corresponding to different voiceprint reward values, and the smaller the loss value interval, the larger the corresponding voiceprint reward value; determine the target loss value interval to which the voiceprint comparison loss value belongs among the multiple loss value intervals, and determine the voiceprint reward value corresponding to the target loss value interval as the voiceprint comparison reward value.

[0326] In some embodiments, the first speech synthesis module 4551A is further configured to perform multiple feature samplings on the first text features under the constraint of the first reference speech features to generate multiple candidate speech features, and decode each candidate speech feature to obtain multiple first target speech; the training module 4555A is further configured to determine the total reward value of each first target speech based on the speech length reward value, entropy regularization reward value, prosody alignment reward value, accuracy reward value, voiceprint similarity reward value, and voiceprint comparison reward value corresponding to the first target speech; determine the relative advantage value of the first target speech based on the statistical distribution of the total reward value of the first target speech among the total reward values ​​of multiple first target speech; and update the policy parameters of the speech synthesis model based on the relative advantage value of each first target speech to maximize the total reward value of all first target speech; wherein, the policy parameters are used to guide the speech synthesis model to perform multiple feature samplings on the first text features.

[0327] The following continues to describe the exemplary structure of the speech synthesis device 455B provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2B As shown, the software module stored in the speech synthesis device 455B in the memory 450 may include:

[0328] The information acquisition module 4551B is used to acquire the text to be processed and the voiceprint reference identifier.

[0329] The second speech synthesis module 4552B is used to input the text to be processed and the voiceprint reference identifier into the pre-trained speech synthesis model, extract the second text features of the text to be processed through the speech synthesis model, and synthesize the second target speech based on the second reference speech features corresponding to the voiceprint reference identifier and the second text features; the voiceprint of the second target speech is consistent with the voiceprint indicated by the voiceprint reference identifier.

[0330] The speech synthesis model is trained based on speech length reward, entropy regularization reward, and prosody alignment reward. The speech length reward is determined based on the deviation between the first speech rate of the first target speech and the second speech rate of the reference speech. The first target speech is synthesized by the speech synthesis model based on the first text features of the text sample and the first reference speech features of the reference speech. The entropy regularization reward is determined based on the difference between the distribution entropy of the first target speech and the target entropy value of the text sample. The prosody alignment reward is determined based on the matching degree between the first pause structure of the first target speech and the second pause structure pre-constructed for the text sample.

[0331] This application provides a computer program product, which includes a computer program or computer-executable instructions. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the training method or speech synthesis method of the speech synthesis model provided in this application. For example, ... Figure 3 The training method of the speech synthesis model shown or Figure 10 The illustrated speech synthesis method involves an electronic device's processor reading a computer program or computer-executable instructions from a computer-readable storage medium, executing the computer program or computer-executable instructions, causing the electronic device to perform the speech synthesis model training method or speech synthesis method described in the embodiments of this application.

[0332] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the training method or speech synthesis method of the speech synthesis model provided in this application. For example, ... Figure 3 The training method of the speech synthesis model shown or Figure 10The speech synthesis method shown.

[0333] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0334] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0335] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0336] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0337] In summary, through the embodiments of this application, when training the speech synthesis model, the first text features of the text sample and the first reference speech features of the reference speech are extracted by the speech synthesis model. A first target speech is synthesized based on the first reference speech features and the first text features. Then, based on the deviation between the first speech rate of the first target speech and the second speech rate of the reference speech, a speech length reward value is determined. The pause structure of the first target speech is analyzed to obtain the first pause structure of the first target speech. Based on the matching degree between the first pause structure and the second pause structure pre-constructed for the text sample, a prosodic alignment reward value is determined. The distribution entropy of the speech synthesis model when synthesizing the first target speech is determined, and based on the difference between the distribution entropy and the target entropy value labeled for the text sample, an entropy regularization reward value is determined. Finally, the speech synthesis model is trained based on the speech length reward value, the entropy regularization reward value, and the prosodic alignment reward value. Thus, the speech... The length reward directly constrains the duration features of the first target speech, making the rhythm of the speech synthesized by the speech synthesis model more consistent with the speaking habits of the reference speech. The prosodic alignment reward forces the speech synthesis model to learn the correct punctuation and pause logic, enhancing the semantic intelligibility and rhythmic sense of the first target speech. The entropy regularization reward regularizes the prediction distribution of the speech synthesis model, preventing both overly smooth phenomena caused by excessively low entropy (i.e., dull and mechanical speech) and unstable generation or noise caused by excessively high entropy. Thus, while ensuring speech clarity, it retains rich acoustic details and expressiveness. It can be seen that through the above methods, non-differential or higher-order perceptual indicators such as speech rate, prosody, and acoustic distribution can be directly transformed into optimization objectives, guiding the speech synthesis model to optimize collaboratively in multiple dimensions, improving the quality of the synthesized speech, and ultimately generating a highly realistic, emotionally rich, and rhythmically accurate first target speech.

[0338] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A training method for a speech synthesis model, characterized in that, The method includes: The speech synthesis model extracts the first text features of the text sample and the first reference speech features of the reference speech, and synthesizes the first target speech based on the first reference speech features and the first text features. The speech length reward value is determined based on the degree of deviation between the first speech rate of the first target speech and the second speech rate of the reference speech. The pause structure of the first target speech is parsed to obtain the first pause structure of the first target speech, and the prosodic alignment reward value is determined based on the matching degree between the first pause structure and the second pause structure pre-constructed for the text sample. Determine the distribution entropy of the speech synthesis model when synthesizing the first target speech, and determine the entropy regularization reward value based on the difference between the distribution entropy and the target entropy value labeled for the text sample; The first target speech is transcribed into text to obtain the text to be verified, and an accuracy reward value is determined based on the difference between the text to be verified and the text sample. Based on the similarity between the first voiceprint feature of the first target speech and the second voiceprint feature of the reference speech, a voiceprint similarity reward value is determined. A voiceprint comparison reward value is determined based on the similarity between the first voiceprint feature and the third voiceprint feature of the comparison speech, as well as the similarity between the first voiceprint feature and the second voiceprint feature; wherein, the comparison speech is a speech segment belonging to a different speaker from the reference speech; The speech synthesis model is trained based on the speech length reward value, the entropy regularization reward value, the prosody alignment reward value, the accuracy reward value, the voiceprint similarity reward value, and the voiceprint comparison reward value.

2. The method according to claim 1, characterized in that, The determination of the speech length reward value based on the deviation between the first speech rate of the first target speech and the second speech rate of the reference speech includes: The first ratio of the text length of the text sample to the speech duration of the first target speech is determined as the first speech rate of the first target speech. A second ratio is determined between the first speech rate and the second speech rate of the reference speech, and a speech length reward value is determined based on the second ratio and the tolerable ratio range; The tolerable ratio range represents the degree to which the first speech rate is allowed to deviate from the second speech rate.

3. The method according to claim 1, characterized in that, The second pause structure is constructed in the following way: A reference example is constructed based on the reference text and the pause structure annotated for the reference text; Based on the text sample and the prompt word template, a reasoning prompt is constructed, and the reasoning prompt and the reference example are input into a pre-trained reasoning model, so that the reasoning model, guided by the reasoning prompt and required by the reference example, performs reasoning on the pause structure of the text sample and generates a second pause structure of the text sample.

4. The method according to claim 1, characterized in that, The step of parsing the pause structure of the first target speech to obtain the first pause structure of the first target speech includes: The first target speech is timestamped to obtain the silence periods in the first target speech and the silence duration corresponding to the silence periods; The silence duration is mapped to a pause structure using a preset mapping rule to obtain the first pause structure of the first target speech; wherein the preset mapping rule is associated with the language corresponding to the text sample.

5. The method according to claim 1, characterized in that, The number of second pause structures is multiple; determining the prosodic alignment reward value based on the matching degree between the first pause structure and the second pause structure pre-constructed for the text sample includes: The first pause structure and each of the second pause structures are structurally matched to obtain the matching degree; If the matching degree between the first pause structure and at least one of the second pause structures exceeds the first matching degree threshold, the preset first value is determined as the prosody alignment reward value. If the matching degree between the first pause structure and each of the second pause structures does not exceed the first matching degree threshold, the preset second value is determined as the rhythm alignment reward value; the first value is greater than the second value.

6. The method according to claim 1, characterized in that, The number of the first target speech is multiple; determining the distribution entropy of the speech synthesis model when synthesizing the first target speech, and determining the entropy regularization reward value based on the difference between the distribution entropy and the target entropy value labeled for the text sample, includes: Determine the distribution entropy of the speech synthesis model when synthesizing each of the first target speech, and determine the first mean of all the distribution entropies; Determine a first difference between the first mean and the target entropy value labeled for the text sample; If the first difference is greater than a preset third value, the first product between the first difference and the preset penalty coefficient is determined as the entropy regularization reward value. If the first difference is less than or equal to the third value, the third value is determined as the entropy regularization reward value.

7. The method according to claim 1, characterized in that, The target entropy value is determined in the following way: Obtain a first speech sample set constructed based on the text sample, wherein multiple first speech samples in the first speech sample set are synthesized based on the text sample; Determine the sample distribution entropy when synthesizing each of the first speech samples, and determine the second mean and standard deviation of all said sample distribution entropies; Determine the second product value between the standard deviation and the preset adjustment coefficient, and determine the sum of the second product value and the second mean value as the target entropy value; The preset adjustment coefficient is used to regulate the coverage of the target entropy value on the first speech sample set, so as to define the allowable fluctuation range of the sample distribution entropy from the second mean.

8. The method according to claim 7, characterized in that, The step of obtaining the first speech sample set constructed for the text sample includes: Multiple candidate speech samples are obtained, and the candidate speech samples are synthesized based on the text samples; Based on a pre-trained speech quality assessment model, the quality of each candidate speech sample is assessed to obtain a quality score for each candidate speech sample as part of the text sample. Candidate speech samples with quality scores higher than a quality threshold are designated as first speech samples, resulting in a first speech sample set composed of multiple first speech samples; or... For each candidate speech sample, the signal-to-noise ratio (SNR) of the candidate speech sample is determined, and the candidate speech sample with an SNR higher than a first SNR threshold is taken as the first speech sample, thus obtaining a first speech sample set composed of multiple first speech samples.

9. The method according to claim 1, characterized in that, The process of determining an accuracy reward value based on the difference between the text to be verified and the text sample includes: Based on the text sample, determine the character error rate of the text to be verified; The character error rate is normalized based on the text length of the text sample to obtain the normalized character error rate; The second difference between the preset baseline value and the normalized character error rate is determined as the accuracy reward value.

10. The method according to claim 1, characterized in that, The compared speech was determined in the following way: Construct a second speech sample set, wherein the second speech samples in the second speech sample set satisfy at least one of the following preset mutual exclusion conditions: the identity label corresponding to the second speech sample is different from the identity label corresponding to the reference speech, and the similarity between the voiceprint features of the second speech sample and the second voiceprint features of the reference speech is lower than a similarity threshold. Second speech samples that meet preset screening conditions are selected from the second speech sample set, and the selected second speech samples are determined as the comparison speech; wherein, the preset screening conditions include at least one of the following: The similarity between the voiceprint features of the second speech sample and the second voiceprint features of the reference speech is within a preset threshold range. The upper limit of the preset threshold range is less than the similarity threshold, and the lower limit of the preset threshold range is greater than the mean of the similarity between the voiceprint features of the second speech sample and the second voiceprint features. The signal-to-noise ratio of the second speech sample is higher than the second signal-to-noise ratio threshold; The duration of the second speech sample is greater than the duration threshold; The matching degree between the attribute labels of the second speech sample and the attribute labels of the reference speech is higher than the second matching degree threshold.

11. The method according to claim 1, characterized in that, The determination of the voiceprint comparison reward value based on the similarity between the first voiceprint feature and the third voiceprint feature of the contrasting speech, as well as the similarity between the first voiceprint feature and the second voiceprint feature, includes: Based on the similarity between the first voiceprint feature and the third voiceprint feature of the contrasting speech, as well as the similarity between the first voiceprint feature and the second voiceprint feature, the voiceprint contrast loss value is determined. Obtain multiple pre-constructed loss value intervals that do not overlap. Different loss value intervals correspond to different voiceprint reward values, and the smaller the loss value interval, the larger the corresponding voiceprint reward value. Determine the target loss value interval to which the voiceprint contrast loss value belongs among the multiple loss value intervals, and determine the voiceprint reward value corresponding to the target loss value interval as the voiceprint contrast reward value.

12. The method according to claim 1, characterized in that, The synthesis of the first target speech based on the first reference speech features and the first text features includes: Under the constraint of the first reference speech features, the first text features are sampled multiple times to generate multiple candidate speech features, and each candidate speech feature is decoded to obtain multiple first target speech features. The training of the speech synthesis model based on the speech length reward value, the entropy regularization reward value, the prosodic alignment reward value, the accuracy reward value, the voiceprint similarity reward value, and the voiceprint comparison reward value includes: For each of the first target speech segments, the total reward value of the first target speech segment is determined based on the speech length reward value, the entropy regularization reward value, the prosody alignment reward value, the accuracy reward value, the voiceprint similarity reward value, and the voiceprint comparison reward value corresponding to the first target speech segment. Based on the statistical distribution of the total reward value of the first target speech among the total reward values ​​of multiple first target speech, the relative advantage value of the first target speech is determined; The policy parameters of the speech synthesis model are updated based on the relative advantage value of each of the first target speech to maximize the total reward value of all the first target speech; wherein the policy parameters are used to guide the speech synthesis model to perform multiple feature samplings on the first text features.

13. A speech synthesis method, characterized in that, The method includes: Obtain the text to be processed and the voiceprint reference identifier; The text to be processed and the voiceprint reference identifier are input into a pre-trained speech synthesis model. The speech synthesis model extracts the second text features of the text to be processed, and synthesizes the second target speech based on the second reference speech features corresponding to the voiceprint reference identifier and the second text features. The voiceprint of the second target speech is consistent with the voiceprint indicated by the voiceprint reference identifier. The speech synthesis model is trained based on speech length reward, entropy regularization reward, prosodic alignment reward, accuracy reward, voiceprint similarity reward, and voiceprint contrast reward. The speech length reward is determined based on the deviation between a first speech rate of the first target speech and a second speech rate of the reference speech. The first target speech is synthesized by the speech synthesis model based on a first text feature of a text sample and a first reference speech feature of the reference speech. The entropy regularization reward is determined based on the difference between the distribution entropy of the first target speech and the target entropy value of the text sample. The prosodic alignment reward is based on the first... The matching degree between the pause structure and the second pause structure pre-constructed for the text sample is determined; the accuracy reward value is determined based on the difference between the text to be verified obtained by transcribing the first target speech and the text sample; the voiceprint similarity reward value is determined based on the similarity between the first voiceprint feature of the first target speech and the second voiceprint feature of the reference speech; the voiceprint comparison reward value is determined based on the similarity between the first voiceprint feature and the third voiceprint feature of the comparison speech, as well as the similarity between the first voiceprint feature and the second voiceprint feature, wherein the comparison speech is a speech segment belonging to a different speaker than the reference speech.

14. A training device for a speech synthesis model, characterized in that, The device includes: The first speech synthesis module is used to extract the first text features of the text sample and the first reference speech features of the reference speech through a speech synthesis model, and synthesize the first target speech based on the first reference speech features and the first text features. The first reward value determination module is used to determine the speech length reward value based on the degree of deviation between the first speech rate of the first target speech and the second speech rate of the reference speech; The second reward value determination module is used to parse the pause structure of the first target speech to obtain the first pause structure of the first target speech, and determine the prosodic alignment reward value based on the matching degree between the first pause structure and the second pause structure pre-constructed for the text sample. The third reward value determination module is used to determine the distribution entropy of the speech synthesis model when synthesizing the first target speech, and to determine the entropy regularization reward value based on the difference between the distribution entropy and the target entropy value labeled for the text sample. The fourth reward value determination module is used to transcribe the first target speech into text to obtain the text to be verified, and determine an accuracy reward value based on the difference between the text to be verified and the text sample; determine a voiceprint similarity reward value based on the similarity between the first voiceprint feature of the first target speech and the second voiceprint feature of the reference speech; and determine a voiceprint comparison reward value based on the similarity between the first voiceprint feature and the third voiceprint feature of the comparison speech, as well as the similarity between the first voiceprint feature and the second voiceprint feature; wherein, the comparison speech is a speech segment belonging to a different speaker from the reference speech; The training module is used to train the speech synthesis model based on the speech length reward value, the entropy regularization reward value, the prosody alignment reward value, the accuracy reward value, the voiceprint similarity reward value, and the voiceprint comparison reward value.

15. The apparatus according to claim 14, characterized in that, The first reward value determination module is also used for: The first ratio of the text length of the text sample to the speech duration of the first target speech is determined as the first speech rate of the first target speech. A second ratio is determined between the first speech rate and the second speech rate of the reference speech, and a speech length reward value is determined based on the second ratio and the tolerable ratio range; The tolerable ratio range represents the degree to which the first speech rate is allowed to deviate from the second speech rate.

16. The apparatus according to claim 14, characterized in that, The number of the second pause structures is multiple; the second reward value determination module is used for: The first pause structure and each of the second pause structures are structurally matched to obtain the matching degree; If the matching degree between the first pause structure and at least one of the second pause structures exceeds the first matching degree threshold, the preset first value is determined as the prosody alignment reward value. If the matching degree between the first pause structure and each of the second pause structures does not exceed the first matching degree threshold, the preset second value is determined as the prosody alignment reward value. The first value is greater than the second value.

17. The apparatus according to claim 14, characterized in that, The number of the first target speech is multiple; the third reward value determination module is further used for: Determine the distribution entropy of the speech synthesis model when synthesizing each of the first target speech, and determine the first mean of all the distribution entropies; Determine a first difference between the first mean and the target entropy value labeled for the text sample; If the first difference is greater than a preset third value, the first product between the first difference and the preset penalty coefficient is determined as the entropy regularization reward value. If the first difference is less than or equal to the third value, the third value is determined as the entropy regularization reward value.

18. The apparatus according to claim 14, characterized in that, The fourth reward value determination module is used to determine the comparison speech in the following way: Construct a second speech sample set, wherein the second speech samples in the second speech sample set satisfy at least one of the following preset mutual exclusion conditions: the identity label corresponding to the second speech sample is different from the identity label corresponding to the reference speech, and the similarity between the voiceprint features of the second speech sample and the second voiceprint features of the reference speech is lower than a similarity threshold. Second speech samples that meet preset screening conditions are selected from the second speech sample set, and the selected second speech samples are determined as the comparison speech; wherein, the preset screening conditions include at least one of the following: The similarity between the voiceprint features of the second speech sample and the second voiceprint features of the reference speech is within a preset threshold range. The upper limit of the preset threshold range is less than the similarity threshold, and the lower limit of the preset threshold range is greater than the mean of the similarity between the voiceprint features of the second speech sample and the second voiceprint features. The signal-to-noise ratio of the second speech sample is higher than the second signal-to-noise ratio threshold; The duration of the second speech sample is greater than the duration threshold; The matching degree between the attribute labels of the second speech sample and the attribute labels of the reference speech is higher than the second matching degree threshold.

19. The apparatus according to claim 14, characterized in that, The fourth reward value determination module is also used for: Based on the similarity between the first voiceprint feature and the third voiceprint feature of the contrasting speech, as well as the similarity between the first voiceprint feature and the second voiceprint feature, the voiceprint contrast loss value is determined. Obtain multiple pre-constructed loss value intervals that do not overlap. Different loss value intervals correspond to different voiceprint reward values, and the smaller the loss value interval, the larger the corresponding voiceprint reward value. Determine the target loss value interval to which the voiceprint contrast loss value belongs among the multiple loss value intervals, and determine the voiceprint reward value corresponding to the target loss value interval as the voiceprint contrast reward value.

20. A speech synthesis device, characterized in that, The device includes: The information acquisition module is used to acquire the text to be processed and the voiceprint reference identifier; The second speech synthesis module is used to input the text to be processed and the voiceprint reference identifier into a pre-trained speech synthesis model, extract the second text features of the text to be processed through the speech synthesis model, and synthesize the second target speech based on the second reference speech features corresponding to the voiceprint reference identifier and the second text features; the voiceprint of the second target speech is consistent with the voiceprint indicated by the voiceprint reference identifier. The speech synthesis model is trained based on speech length reward, entropy regularization reward, prosodic alignment reward, accuracy reward, voiceprint similarity reward, and voiceprint contrast reward. The speech length reward is determined based on the deviation between a first speech rate of the first target speech and a second speech rate of the reference speech. The first target speech is synthesized by the speech synthesis model based on a first text feature of a text sample and a first reference speech feature of the reference speech. The entropy regularization reward is determined based on the difference between the distribution entropy of the first target speech and the target entropy value of the text sample. The prosodic alignment reward is based on the first... The matching degree between the pause structure and the second pause structure pre-constructed for the text sample is determined; the accuracy reward value is determined based on the difference between the text to be verified obtained by transcribing the first target speech and the text sample; the voiceprint similarity reward value is determined based on the similarity between the first voiceprint feature of the first target speech and the second voiceprint feature of the reference speech; the voiceprint comparison reward value is determined based on the similarity between the first voiceprint feature and the third voiceprint feature of the comparison speech, as well as the similarity between the first voiceprint feature and the second voiceprint feature, wherein the comparison speech is a speech segment belonging to a different speaker than the reference speech.

21. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the method according to any one of claims 1 to 13.

22. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the method described in any one of claims 1 to 13.

23. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the method according to any one of claims 1 to 13.

Images