Speech synthesis method and device, electronic equipment and storage medium

Abstract

The invention provides a speech synthesis method and device, electronic equipment and a storage medium, and the method comprises the steps: selecting a target speech synthesis model from a plurality of speech synthesis models based on the speech synthesis preference of a user, and carrying out the speech synthesis through employing the target speech synthesis model based on a to-be-synthesized text, and obtaining a synthesized speech meeting the speech synthesis preference; each speech synthesis model is obtained based on preference data sets corresponding to a plurality of preferences and preference weight configuration training, and the preference weight configurations used during training of different speech synthesis models are different, so that the defect that in a traditional scheme, a human preference alignment method for the speech synthesis models can only align single human preferences is overcome; according to the invention, the defects that the speech synthesis model cannot meet diversified human preference requirements in the prior art are overcome, the speech synthesis model is trained in a multi-target direct preference optimization mode, the training process is simpler and more efficient, human preference alignment can be carried out from multiple dimensions, and the diversified human preference requirements are met by giving preference weights.

Description

Technical Field

[0001] The present invention relates to the field of speech synthesis technology, and in particular to a speech synthesis method, device, electronic equipment and storage medium. Background Art

[0002] In recent years, speech synthesis models have become the mainstream technology in the field of speech synthesis due to their powerful speech generation capabilities and zero-sample adaptability to unseen speakers. However, despite the many advantages of speech synthesis models, they still face an important challenge in practical applications, namely how to make the quality of synthesized speech consistent with human subjective evaluation. In the training process of traditional speech synthesis models, indicators such as MOS (Mean Opinion Score) scores of human subjective evaluation are not directly introduced, resulting in a certain deviation between the training objectives and human evaluation standards. This deviation makes it difficult for the speech synthesized by the trained model to fully meet human expectations.

[0003] In response to this, researchers proposed a method based on human feedback reinforcement learning and a method based on direct preference optimization. The former pre-trains the reward model and applies the reinforcement learning algorithm to adjust the speech synthesis model so that it generates speech with a higher reward score. However, this method relies on online sampling and takes a long time, resulting in low training efficiency of the model. The latter optimizes the classification function to make the model output closer to the distribution of real data, thereby aligning the speech synthesis model with human preferences. Although the above two methods can adjust the speech synthesis model to align it with human preferences, the human preferences they align are specific. However, in fact, human preferences for synthesized speech are diverse, and the preference emphasis in different scenarios is also different. The current solution cannot adapt to the diverse human preference needs in different scenarios. Summary of the invention

[0004] The present invention provides a speech synthesis method, device, electronic device and storage medium, which are used to solve the defect that the human preference alignment method for speech synthesis models in the prior art can only align a single human preference and cannot meet the diverse human preference needs. The human preference alignment of the speech synthesis model can be performed from multiple dimensions, and different preference weight configurations can be set to meet the diverse human preference needs.

[0005] The present invention provides a speech synthesis method, comprising: Determine the text to be synthesized and the user's speech synthesis preference; Based on the speech synthesis preference, a target speech synthesis model is selected from a plurality of speech synthesis models, and based on the text to be synthesized, the target speech synthesis model is applied to perform speech synthesis to obtain a synthesized speech that meets the speech synthesis preference; Among them, each speech synthesis model is trained based on preference data sets and preference weight configurations corresponding to multiple preferences. Different speech synthesis models use different preference weight configurations when training. Each preference data set contains multiple sets of preference data corresponding to preferences, and each set of preference data includes sample text corresponding to the preference, sample preference speech and sample non-preferential speech.

[0006] According to a speech synthesis method provided by the present invention, a preference data set corresponding to each preference is determined based on the following steps: Determine a sample text corresponding to the preference, and a prompt voice corresponding to the sample text; Based on an initial speech synthesis model, determining a plurality of initial synthesized speech corresponding to the sample text and the prompt speech; the initial speech synthesis model is trained based on a pre-training data set, and the pre-training data set includes a pre-training text and a corresponding pre-training prompt speech; Determine a speech score of each initial synthesized speech under a corresponding preference, and determine a sample preferred speech and a sample non-preferred speech corresponding to the corresponding preference from each initial synthesized speech based on the speech score corresponding to each initial synthesized speech; Based on the sample text and prompt voice corresponding to the preference, and the corresponding sample preferred voice and the sample non-preferred voice, a preference data set corresponding to the preference is constructed.

[0007] According to a speech synthesis method provided by the present invention, the step of determining a speech score of each initial synthesized speech under a corresponding preference, and determining a sample preferred speech and a sample non-preferred speech corresponding to the corresponding preference from each initial synthesized speech based on the speech score corresponding to each initial synthesized speech, comprises: Determining a coarse-grained speech score of each of the initial synthesized speech under the plurality of preferences; Based on the coarse-grained speech scores of the initial synthesized speech corresponding to the plurality of preferences, a plurality of candidate synthesized speech are screened from the initial synthesized speech; Determine a fine-grained speech score for each candidate synthesized speech under a corresponding preference; Based on the fine-grained speech scores corresponding to the candidate synthesized speech, sample preferred speech and sample non-preferred speech corresponding to the candidate synthesized speech are determined from the candidate synthesized speech.

[0008] According to a speech synthesis method provided by the present invention, the multiple preferences include naturalness preference, comprehension preference and similarity preference; the method of determining sample preferred speech and sample non-preferred speech corresponding to the corresponding preference from the candidate synthesized speech based on the fine-grained speech score corresponding to each candidate synthesized speech includes: From the candidate synthesized speech corresponding to the naturalness preference, select the candidate synthesized speech with the highest and lowest fine-grained speech scores under the naturalness preference as the sample preferred speech and the sample non-preferred speech corresponding to the naturalness preference, respectively; From the candidate synthesized speech corresponding to the intelligibility preference, select the candidate synthesized speech with the highest and lowest fine-grained speech scores under the intelligibility preference as the sample preferred speech and the sample non-preferred speech corresponding to the intelligibility preference, respectively; From the candidate synthesized speech corresponding to the similarity preference, the candidate synthesized speech with the highest and lowest fine-grained speech scores under the similarity preference are selected as the sample preferred speech and the sample non-preferred speech corresponding to the similarity preference, respectively.

[0009] According to a speech synthesis method provided by the present invention, the determining, based on an initial speech synthesis model, a plurality of initial synthesized speech corresponding to the sample text and the prompt speech comprises: Performing phoneme conversion on the sample text to obtain a sample phoneme sequence; Performing feature encoding on the prompt voice to obtain sample voice features; Embedding the sample phoneme sequence and the sample speech feature, and concatenating the sample phoneme vector representation and the sample speech vector representation obtained by the embedded coding to obtain a sample fusion vector representation; Based on the sample fusion vector representation, the initial speech synthesis model is applied to perform speech synthesis to obtain multiple initial synthesized speech.

[0010] According to a speech synthesis method provided by the present invention, each speech synthesis model is trained based on the following steps: Selecting a preference data set corresponding to any one preference from the preference data sets corresponding to the multiple preferences as a target preference data set; Based on the initial speech synthesis model, determining the sample synthesized speech corresponding to the sample text and the prompt speech in each group of preference data in the target preference data set; Based on a reward model for other preferences, determining a reward score for the sample synthesized speech corresponding to each set of preference data under the other preferences; the reward model is trained based on the preference data set corresponding to the other preferences; Based on the sample preferred speech and sample non-preferred speech in each group of preference data, the sample synthesized speech corresponding to each group of preference data, the reward score of the sample synthesized speech corresponding to other preferences, and the preference weight configuration corresponding to the initial speech synthesis model, the initial speech synthesis model is iterated on parameters to obtain a speech synthesis model.

[0011] According to a speech synthesis method provided by the present invention, based on the sample preferred speech and sample non-preferred speech in each group of preference data, the sample synthesized speech corresponding to each group of preference data, the reward score of the sample synthesized speech corresponding to other preferences, and the preference weight configuration corresponding to the initial speech synthesis model, the initial speech synthesis model is iterated to obtain the speech synthesis model, including: Determine a first loss based on the sample preferred speech and the sample non-preferred speech in each group of preference data, the sample synthesized speech corresponding to each group of preference data, and the weight of any preference in the preference weight configuration corresponding to the initial speech synthesis model; Determine a second loss based on the sample preferred speech and the sample non-preferred speech in each group of preference data, the sample synthesized speech and its reward score corresponding to other preferences, and the weight of the other preferences in the preference weight configuration corresponding to the initial speech synthesis model; Based on the first loss and the second loss, parameters of the initial speech synthesis model are iterated to obtain a speech synthesis model.

[0012] The present invention also provides a speech synthesis device, comprising: A data determination unit, used to determine the text to be synthesized and the user's speech synthesis preference; A speech synthesis unit, configured to select a target speech synthesis model from a plurality of speech synthesis models based on the speech synthesis preference, and perform speech synthesis based on the text to be synthesized by applying the target speech synthesis model to obtain a synthesized speech that meets the speech synthesis preference; Among them, each speech synthesis model is trained based on preference data sets and preference weight configurations corresponding to multiple preferences. Different speech synthesis models use different preference weight configurations when training. Each preference data set contains multiple sets of preference data corresponding to preferences, and each set of preference data includes sample text corresponding to the preference, sample preference speech and sample non-preferential speech.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements any of the above-mentioned speech synthesis methods when executing the computer program.

[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the speech synthesis method described in any one of the above is implemented.

[0015] The speech synthesis method, device, electronic device and storage medium provided by the present invention select a target speech synthesis model from multiple speech synthesis models according to the user's speech synthesis preference, and apply the target speech synthesis model to perform speech synthesis according to the text to be synthesized, so as to obtain synthesized speech that meets the speech synthesis preference; wherein each speech synthesis model is trained based on a preference data set and a preference weight configuration corresponding to multiple preferences, and different speech synthesis models are trained using different preference weight configurations, thereby overcoming the defect that the human preference alignment method for speech synthesis models in traditional solutions can only align a single human preference and cannot meet the diverse human preference needs. The speech synthesis model is trained by a multi-objective direct preference optimization method, which not only makes the training process simpler and more efficient, but also can perform human preference alignment from multiple dimensions, and meet the diverse human preference needs by assigning weights to each preference. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 It is a flow chart of the speech synthesis method provided by the present invention; Figure 2 It is a framework example diagram of the model training process provided by the present invention; Figure 3 It is a structural schematic diagram of the speech synthesis device provided by the present invention; Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Currently, there are two main methods for aligning human preferences for speech synthesis models. The first is based on RLHF (Reinforcement Learning from Human Feedback). This method trains a reward model on a dataset that reflects human preferences and applies a reinforcement learning algorithm to adjust the speech synthesis model to optimize it towards generating speech with a high reward score while ensuring that the deviation from the original model is minimized. However, the process of sampling and generating speech with this method is time-consuming and the model training efficiency is low.

[0020] The other is based on the DPO (Direct Preference Optimization) method. This method is different from traditional preference-driven reinforcement learning. It directly optimizes the closed-form loss of preference data without relying on an explicit reward function, avoiding the problem of time-consuming online sampling. Compared with the RLHF method, training is more efficient and stable.

[0021] However, although both of the above methods can adjust the speech synthesis model to align it with human preferences, the aligned human preferences are specific and difficult to meet the diverse human preference needs. In actual applications, people have different preferences for generated speech in different scenarios. For example, in the sound reproduction scenario, people are more concerned about whether the model can generate speech that is highly similar to the timbre of the prompt speech; in the audio book scenario, people are more concerned about whether the model can generate speech with high naturalness and expressiveness. Obviously, neither RLHF nor DPO can do this at present, and cannot adapt to the diverse preference needs in different scenarios.

[0022] Furthermore, in order to address the limitations of single-target preference alignment, there are studies on the use of multi-target RLHF to optimize speech synthesis models. This method trains respective reward models for different goals, and performs weighted optimization on the scoring results of multiple reward models to achieve alignment of multiple human preferences. However, this method also has disadvantages such as cumbersome procedures and unstable training. In the field of speech synthesis, when the model encounters diverse needs, the preferences of multiple targets often show certain conflicts. For example, the generated highly intelligible speech may not be highly natural. Therefore, how to achieve the alignment of diverse human preferences while maintaining efficient and stable training is still a problem to be solved in the field of speech synthesis.

[0023] In this regard, the present invention provides a speech synthesis method, which aims to solve the problem that RLHF and DPO optimization are difficult to satisfy the diversity of human preferences. The speech synthesis model is trained by multi-objective direct preference optimization, which abandons the dependence on RLHF, makes training simpler and more efficient, and satisfies diverse human preference needs by assigning weights to each target (preference).

[0024] Figure 1 It is a flow chart of the speech synthesis method provided by the present invention, such as Figure 1 As shown, the method includes: Step 110, determining the text to be synthesized and the user's speech synthesis preference; Step 120, based on the speech synthesis preference, select a target speech synthesis model from multiple speech synthesis models, and perform speech synthesis using the target speech synthesis model based on the text to be synthesized to obtain synthesized speech that meets the speech synthesis preference; Among them, each speech synthesis model is trained based on preference data sets and preference weight configurations corresponding to multiple preferences. Different speech synthesis models use different preference weight configurations when training. Each preference data set contains multiple sets of preference data corresponding to preferences, and each set of preference data includes sample text corresponding to the preference, sample preference speech and sample non-preferential speech.

[0025] Specifically, considering that the current RLHF method and DPO method can only align a single human preference, people's preferences for generated speech are diverse in actual applications, and the preference emphasis on generated speech in different scenarios is also different. For example, in the sound reproduction scenario, people pay more attention to the similarity of the generated speech, while in the audiobook scenario, they pay more attention to whether the generated speech is highly natural and expressive. Obviously, these two methods that can only align a single human preference are not suitable for different scenarios and cannot meet the diverse human preference needs in different scenarios.

[0026] In view of this, in an embodiment of the present invention, it is proposed that multi-target human preference alignment can be performed. In order to avoid the limitations of the current multi-target RLHF, in an embodiment of the present invention, a preference data set corresponding to multiple preferences is selected for model training. During the training process, corresponding weights are assigned to different preferences to obtain diversified speech synthesis models. Therefore, in actual application, according to the user's preferences, a model adapted to it can be selected for speech synthesis, and a synthesized speech that meets the user's expectations can be obtained, thereby fully meeting the diverse human preference needs.

[0027] It is understandable that in actual application, before performing speech synthesis, it is first necessary to determine the text to be synthesized, which contains the text information corresponding to the desired synthesized speech. The text can be text directly input by the user, or text automatically generated by the computer during human-computer interaction, or text obtained by acquiring an image through an image acquisition device and performing OCR (Optical Character Recognition) on the image. The embodiment of the present invention does not specifically limit this. The image acquisition device here can be a scanner, a mobile phone, a camera, etc.

[0028] While determining the text to be synthesized, in order to make the synthesized speech more in line with the user's expectations, in an embodiment of the present invention, the user's speech synthesis preference can also be determined. The speech synthesis preference is used to characterize the user's requirements / expectations for the synthesized speech obtained by this speech synthesis. For example, it can be a synthesized speech with a timbre as similar as possible, a synthesized speech that is more realistic and easier to understand, etc.

[0029] Furthermore, considering that the effect of synthesized speech obtained by performing speech synthesis directly based on the text to be synthesized is mostly poor, especially in terms of timbre, pronunciation habits, etc., there are obvious deficiencies, and the difference from the real speech is obvious, and the effect of speech synthesis is not good. In view of this situation, in the embodiment of the present invention, after obtaining the text to be synthesized and before performing speech synthesis, it is necessary to obtain a segment of real speech as a reference for the subsequent speech synthesis process. The real speech can be the user's speech or the speech of other people, which can be determined according to the usage scenario and the user's speech synthesis preference, and the embodiment of the present invention does not make specific limitations on this.

[0030] After that, speech synthesis can be performed to obtain synthesized speech that meets the user's expectations, that is, synthesized speech that matches the user's speech synthesis preference. Considering that the current speech synthesis model only aligns with a single human preference, it cannot meet the diverse human preference needs and cannot synthesize synthesized speech that meets the user's expectations, in an embodiment of the present invention, a preference data set corresponding to multiple preferences is pre-established, and this preference data set is used for model training. Preference weight configuration is added during training. By assigning corresponding weights to different preferences, a diverse speech synthesis model can be trained. Based on this, speech synthesis in a variety of scenarios can be achieved, which can meet the diverse human preference needs.

[0031] Specifically, here, according to the user's speech synthesis preference, a model that matches or conforms to the user's expectations / requirements can be selected from multiple pre-trained speech synthesis models as the target speech synthesis model. For example, when the user's speech synthesis preference is to synthesize highly natural and realistic speech, a speech synthesis model with a corresponding preference weight configuration that reflects a greater focus on naturalness and similarity can be selected from multiple speech synthesis models. Specifically, a model whose weights corresponding to the naturalness preference and similarity preference are higher than the weights of other preferences can be selected as the target speech synthesis model. For another example, when the user's speech synthesis preference is to synthesize speech that is easy to understand and read, a speech synthesis model that focuses on intelligibility can be selected from multiple speech synthesis models. Specifically, a model whose weight corresponding to the intelligibility preference is significantly higher than the weights of other preferences can be selected as the target speech synthesis model.

[0032] After that, the target speech synthesis model can be applied to perform speech synthesis to obtain synthesized speech. Here, specifically, the target speech synthesis model can be applied to perform speech synthesis on the basis of the text to be synthesized and the real speech, so as to obtain synthesized speech that meets the user's speech synthesis preference. That is, the text to be synthesized and the real speech are input into the target speech synthesis model, so that the target speech synthesis model refers to the real speech and performs speech synthesis according to the content of the text to be synthesized, and obtains a synthesized speech that has the characteristics of the real speech and is consistent with the content of the text to be synthesized, and outputs it, so as to obtain the user's synthesized speech that meets the speech synthesis preference.

[0033] It is worth noting that before the text to be synthesized and the real speech are input into the target speech synthesis model, in the embodiment of the present invention, it is necessary to pre-train the speech synthesis model. Considering that the single target preference alignment cannot meet the diversified preference requirements, in the embodiment of the present invention, when training the speech synthesis model, a multi-target alignment method is adopted, specifically, a preference data set corresponding to multiple preferences is first constructed, where the multiple preferences may include naturalness preference, intelligibility preference, similarity preference, completeness preference, fluency preference, etc. The preference data set corresponding to each preference contains multiple sets of preference data, and each set of preference data contains sample text, sample preference speech and sample non-preferred speech corresponding to the preference. The sample preference speech and sample non-preferred speech can be understood as positive samples and negative samples under the corresponding preference, specifically, the speech that meets the corresponding preference and the speech that does not meet the corresponding preference corresponding to the sample text. The sample text can be understood as the text to be synthesized in the training process, which can be obtained from a public speech synthesis data set.

[0034] After determining the preference data sets corresponding to multiple preferences, corresponding weights can also be set for each preference. The weight can indicate the importance of this preference in the speech synthesis model training process. The larger the weight, the more emphasis the training process places on this ability of the model. For example, the larger the weight of the naturalness preference, the more attention is paid to the naturalness of the model in speech synthesis during model training, and the goal is to make the synthesized speech output by the model more natural. Conversely, the lower the weight, the less attention is paid to this ability of the model during training. By setting different weights for different preferences, the combination of different weights can make the model training process focus on different points, so that a variety of speech synthesis models can be obtained. Among them, each speech synthesis model corresponds to a unique set of weights, and the weight groups (preference weight configuration composed of weights corresponding to multiple preferences) used in the training of different speech synthesis models are different.

[0035] In the embodiment of the present invention, based on the preference data sets and preference weight configurations corresponding to multiple preferences, multiple speech synthesis models are trained to align multiple human preferences and meet the diverse human preference needs in different scenarios. This not only avoids the problem of limited application and inability to meet the diverse needs of users in a single preference alignment solution, but also overcomes the defects of unstable training and conflicting speech synthesis in the multi-objective RLHF solution. Under the guidance of the preference weight configuration, the preference data sets corresponding to multiple preferences are used to train the model, so that the model can be optimized and updated in the direction guided by the weights. While highlighting one or more capabilities, the optimization and update of other items will not be ignored. Therefore, the quality of the synthesized speech can be guaranteed in the application stage, and the problem of poor quality of the synthesized speech can be avoided.

[0036] The speech synthesis method provided by the present invention selects a target speech synthesis model from multiple speech synthesis models according to the user's speech synthesis preference, and applies the target speech synthesis model to perform speech synthesis according to the text to be synthesized, so as to obtain synthesized speech that meets the speech synthesis preference; wherein each speech synthesis model is trained based on a preference data set and a preference weight configuration corresponding to multiple preferences, and different speech synthesis models are trained using different preference weight configurations, thereby overcoming the defect that the human preference alignment method for speech synthesis models in traditional solutions can only align a single human preference and cannot meet the diverse human preference needs. The speech synthesis model is trained by a multi-objective direct preference optimization method, which not only makes the training process simpler and more efficient, but also can perform human preference alignment from multiple dimensions, and meet the diverse human preference needs by assigning weights to each preference.

[0037] Based on the above embodiment, the preference data set corresponding to each preference is determined based on the following steps: Determine a sample text corresponding to the preference and a prompt voice corresponding to the sample text; Based on an initial speech synthesis model, a plurality of initial synthesized speech corresponding to the sample text and the prompt speech is determined; the initial speech synthesis model is trained based on a pre-training data set, and the pre-training data set includes a pre-training text and a corresponding pre-training prompt speech; Determine a speech score of each initial synthesized speech under a corresponding preference, and determine a sample preferred speech and a sample non-preferred speech corresponding to the corresponding preference from each initial synthesized speech based on the speech score corresponding to each initial synthesized speech; Based on the sample texts and prompt voices of the corresponding preferences, as well as the sample preferred voices and sample non-preferred voices, a preference data set of the corresponding preferences is constructed.

[0038] Specifically, the preferred data set used for model training can be determined by the following steps: The following will take any preference as an example to illustrate the construction of the preference data set: First, the sample text under the preference can be determined. The sample text here can be obtained from a public speech synthesis dataset, and there can be multiple sample texts. After determining the sample text, the real speech needs to be determined as well, which is called the prompt speech corresponding to the sample text. It can be used to prompt the model to perform speech synthesis based on the sample text during the training process and output the synthesized speech.

[0039] Next, speech synthesis can be performed based on the sample text and the prompt voice. Specifically, based on the sample text and the prompt voice, an initial speech synthesis model is applied to perform speech synthesis, thereby obtaining multiple initial synthesized voices output by the initial speech synthesis model. Here, the initial speech synthesis model can be an initial model directly constructed for the training process, or it can be a model obtained after pre-training the initial model. In order to ensure the quality of the initial synthesized voice, in the embodiment of the present invention, the initial model is preferably pre-trained, specifically, the initial model is trained using the pre-trained text in the pre-training data set and the corresponding pre-trained prompt voice, and the pre-trained initial model is used as the initial speech synthesis model. Among them, the initial model is constructed on the Transformer architecture, and its training method is autoregressive, and the loss function adopts cross entropy loss.

[0040] Here, pre-training specifically includes two stages, namely the model pre-training stage and the supervised fine-tuning stage. The two stages use different training data sets. In short, the pre-training data set contains the training data sets of the two stages, and the training data set of each stage contains the pre-training text and the corresponding pre-training prompt voice. Among them, the pre-training stage uses a large amount of data (tens of thousands of hours of voice data) to train the initial model, and the supervised fine-tuning stage uses the data set of downstream tasks (speech synthesis tasks in various scenarios) for supervised fine-tuning. After two stages of training, a model with good speech synthesis capabilities can be obtained.

[0041] After that, the speech score of each initial synthesized speech under the corresponding preference can be determined, that is, the score of each initial synthesized speech output by the initial speech synthesis model on the preference can be determined. For example, when the preference is a naturalness preference, the score of each generated initial synthesized speech on the naturalness preference needs to be determined. Here, the process of scoring each initial synthesized speech can be performed manually or implemented by a currently mature scoring algorithm. Furthermore, according to the speech score of each initial synthesized speech corresponding to the preference, a sample preferred speech and a sample non-preferred speech can be selected from each initial synthesized speech. Here, specifically, based on the speech score, an initial synthesized speech that meets the preference (such as an initial synthesized speech with a high score) can be selected as a sample preferred speech corresponding to the preference, and an initial synthesized speech that does not meet the preference (such as an initial synthesized speech with a low score) can be selected as a sample non-preferred speech corresponding to the preference.

[0042] Then, the preference data set corresponding to the preference can be constructed based on the selected sample preferred speech, sample non-preferred speech, and the sample text and corresponding prompt speech under the preference obtained in advance. It should be noted that the preference data set corresponding to the preference often contains more than one set of sample texts and their corresponding data. Therefore, in the process of constructing the preference data set, multiple sample texts can be obtained, and multiple initial synthesized speech can be generated for each sample text, so as to determine the sample preferred speech and sample non-preferred speech corresponding to each sample text, and then multiple sets of preference data can be constructed, based on which a complete preference data set can be constructed.

[0043] Based on the above embodiment, based on the initial speech synthesis model, multiple initial synthesized speech corresponding to the sample text and the prompt speech are determined, including: Perform phoneme conversion on the sample text to obtain a sample phoneme sequence; Perform feature encoding on the prompt speech to obtain sample speech features; Embedding the sample phoneme sequence and the sample speech feature, and concatenating the sample phoneme vector representation and the sample speech vector representation obtained by the embedded coding to obtain a sample fusion vector representation; Based on the sample fusion vector representation, an initial speech synthesis model is applied to perform speech synthesis to obtain multiple initial synthesized speech.

[0044] Specifically, the process of generating the initial synthesized speech may include: Before the data is input into the initial speech synthesis model for speech synthesis, it needs to be processed. Specifically, the sample text can be first converted into phonemes to process the text sequence in the sample text into a phoneme sequence, thereby obtaining a sample phoneme sequence. Next, the prompt speech can be feature encoded, specifically using a pre-trained audio codec to perform feature encoding to extract discrete speech representations, namely, sample speech features.

[0045] After that, the sample phoneme sequence and the sample speech features can be embedded and coded, and the sample phoneme vector representation and the sample speech vector representation obtained by the embedded coding can be spliced ​​to obtain the sample fusion vector representation. Specifically, the converted phoneme sequence and the extracted discrete speech representation are embedded and coded, and the two are spliced ​​in the time series dimension, and the spliced ​​features are used as the input of the initial speech synthesis model to wait for speech synthesis. That is, the sample fusion vector representation obtained by splicing can be input into the initial speech synthesis model, so that the initial speech synthesis model performs speech synthesis based on it and outputs multiple initial synthesized speech.

[0046] Based on the above embodiment, determining the speech score of each initial synthesized speech under the corresponding preference, and determining the sample preferred speech and the sample non-preferred speech corresponding to the corresponding preference from each initial synthesized speech based on the speech score corresponding to each initial synthesized speech, including: determining a coarse-grained speech score for each initial synthesized speech under a plurality of preferences; Based on the coarse-grained speech scores of the initial synthesized speech corresponding to the plurality of preferences, a plurality of candidate synthesized speech are screened from the initial synthesized speech; Determine a fine-grained speech score for each candidate synthesized speech under a corresponding preference; Based on the fine-grained speech scores corresponding to the candidate synthesized speech, the sample preferred speech and the sample non-preferred speech corresponding to the candidate synthesized speech are determined from the candidate synthesized speech.

[0047] Specifically, the process of determining the speech scores of each initial synthesized speech under the corresponding preference, and determining the sample preferred speech and the sample non-preferred speech corresponding to the corresponding preference from each initial synthesized speech based on the speech scores may specifically include: After multiple initial synthesized speech are generated by the initial speech synthesis model, each initial synthesized speech can be scored. The scoring process can be divided into two steps. The first is the rough scoring. In this stage, each initial synthesized speech needs to be scored in multiple dimensions to obtain its score under multiple preferences, that is, to determine the coarse-grained speech score of each initial synthesized speech under multiple preferences. For example, multiple initial synthesized speech generated under the naturalness preference can be scored under multiple preferences, so as to obtain the score of each initial synthesized speech under the naturalness preference, intelligibility preference, similarity preference, etc., which is the coarse-grained speech score.

[0048] After that, data screening can be performed based on the coarse-grained speech score corresponding to each initial synthesized speech obtained by the rough score, so as to filter out valid data from the generated initial synthesized speech; specifically, when the coarse-grained speech score corresponding to an initial synthesized speech indicates that it is unqualified in any one or more of the multiple preferences, that is, its coarse-grained speech score in this or these preferences is too low, lower than the set scoring threshold (e.g., 3 points can be set as the scoring threshold for a full score of 5 points), it is considered to be unqualified in this or these preferences. At this time, it can be regarded as invalid data and eliminated, thereby screening out candidate synthesized speech that are qualified in all preferences from the initial synthesized speech.

[0049] Furthermore, after obtaining the candidate synthesized speech, the second stage of scoring, i.e., fine scoring, can be started. Specifically, the multiple candidate synthesized speech obtained by the previous step is scored on the corresponding preference. That is, when the preference data set to be constructed is the preference data set corresponding to the naturalness preference, then the multiple candidate synthesized speech can be scored on the naturalness preference at this time. Different from the coarse scoring in the first stage, for all preferences, this stage only scores the preference dimensions to be constructed, and a more accurate and detailed speech score can be obtained, i.e., the fine-grained speech score of each candidate synthesized speech under the corresponding preference. Then, according to this fine-grained speech score, the sample preferred speech and sample non-preferred speech under the corresponding preference can be selected from each candidate synthesized speech. Specifically, according to the fine-grained speech score, the candidate synthesized speech that meets the corresponding preference (such as the candidate synthesized speech with a high fine-grained speech score) is selected as the sample preferred speech under the corresponding preference, and the candidate synthesized speech that does not meet the corresponding preference (such as the candidate synthesized speech with a low fine-grained speech score) is selected as the sample non-preferred speech.

[0050] Based on the above embodiment, the multiple preferences include naturalness preference, intelligibility preference and similarity preference; based on the fine-grained speech score corresponding to each candidate synthesized speech, the sample preferred speech and the sample non-preferred speech corresponding to the corresponding preference are determined from each candidate synthesized speech, including: From the candidate synthesized speech corresponding to the naturalness preference, select the candidate synthesized speech with the highest and lowest fine-grained speech scores under the naturalness preference as the sample preferred speech and sample non-preferred speech corresponding to the naturalness preference, respectively; From the candidate synthesized speech corresponding to the intelligibility preference, select the candidate synthesized speech with the highest and lowest fine-grained speech scores under the intelligibility preference as the sample preferred speech and the sample non-preferred speech corresponding to the intelligibility preference, respectively; From the candidate synthesized speech corresponding to the similarity preference, the candidate synthesized speech with the highest and lowest fine-grained speech scores under the similarity preference are selected as the sample preferred speech and sample non-preferred speech corresponding to the similarity preference, respectively.

[0051] Specifically, according to the fine-grained speech score, the process of determining the corresponding preferred sample preferred speech and the sample non-preferred speech from each candidate synthesized speech specifically includes: In an embodiment of the present invention, the multiple preferences are naturalness preference, understanding preference and similarity preference. When determining the sample preferred speech and sample non-preferred speech under each preference, the fine-grained speech score under the preference can be directly used to select the highest-scoring candidate synthesized speech as the sample preferred speech and the lowest-scoring candidate as the sample non-preferred speech from multiple candidate synthesized speech. In this way, the sample preferred speech and sample non-preferred speech under each preference can be determined.

[0052] In detail, when determining the sample preferred speech and the sample non-preferred speech under the naturalness preference, the candidate synthetic speech with the highest fine-grained speech score under the naturalness preference is directly selected from the candidate synthetic speech corresponding to the naturalness preference as the sample preferred speech corresponding to the naturalness preference. At the same time, the candidate synthetic speech with the lowest fine-grained speech score under the naturalness preference is selected as the sample non-preferred speech corresponding to the naturalness preference.

[0053] When determining the sample preferred speech and sample non-preferred speech under the intelligibility preference, the candidate synthesized speech with the highest fine-grained speech score under the intelligibility preference is directly selected from the candidate synthesized speech corresponding to the intelligibility preference as the sample preferred speech corresponding to the intelligibility preference. At the same time, the candidate synthesized speech with the lowest fine-grained speech score under the intelligibility preference is selected as the sample non-preferred speech corresponding to the intelligibility preference.

[0054] When determining the sample preferred speech and sample non-preferred speech under the similarity preference, the candidate synthesized speech with the highest fine-grained speech score under the similarity preference is directly selected from the candidate synthesized speech corresponding to the similarity preference as the sample preferred speech corresponding to the similarity preference. At the same time, the candidate synthesized speech with the lowest fine-grained speech score under the similarity preference is selected as the sample non-preferred speech corresponding to the similarity preference.

[0055] It should be noted that, whether it is when scoring the similarity preference in the coarse scoring stage or when scoring the candidate synthesized speech under the similarity preference in the fine scoring stage, the data itself is not scored only, but the speech is compared with the prompt speech under the corresponding preference, and the similarity is judged by comparison.

[0056] In the embodiment of the present invention, a preference data set is constructed from three dimensions: naturalness, intelligibility, and similarity. Based on the constructed preference data set, model training is performed in combination with preference weight configuration, which can achieve multi-objective human preference alignment, thereby meeting the human preference needs of multiple samples in different scenarios.

[0057] Based on the above embodiment, each speech synthesis model is trained based on the following steps: Selecting a preference data set corresponding to any one preference from preference data sets corresponding to multiple preferences as a target preference data set; Based on the initial speech synthesis model, determine the sample synthesized speech corresponding to the sample text and prompt speech in each group of preference data in the target preference data set; Based on the reward model of other preferences, the reward score of the sample synthesized speech corresponding to each group of preference data under other preferences is determined; the reward model is trained based on the preference data set corresponding to other preferences; Based on the sample preferred speech and sample non-preferred speech in each group of preference data, the sample synthesized speech corresponding to each group of preference data, the reward score of the sample synthesized speech corresponding to other preferences, and the preference weight configuration corresponding to the initial speech synthesis model, the parameters of the initial speech synthesis model are iterated to obtain the speech synthesis model.

[0058] Specifically, considering the single preference alignment method, the application of reward models also faces challenges. That is, in the current single preference alignment scheme for speech synthesis models, speech recognition models and voiceprint recognition models are usually used as reward models. Although the speech recognition model can evaluate the intelligibility of speech synthesis, when there are misreadings in the generated speech, the speech recognition model often ignores this point, resulting in inaccurate scoring and lack of reliability. Although the voiceprint recognition model has the ability to distinguish different speakers, it still lacks in distinguishing the speech generated by the model from the real speech.

[0059] Based on this, in an embodiment of the present invention, it is proposed to use a constructed preference data set to train a reward model to overcome the defect that human preferences cannot be accurately reflected when speech recognition and voiceprint recognition models are directly used as reward models. The trained reward model is applied to the training process of the speech synthesis model to assist the training of the speech synthesis model, thereby better ensuring the model training effect and achieving multi-objective human preference alignment.

[0060] Specifically, here, a preference data set corresponding to any preference may be selected from the constructed preference data sets corresponding to multiple preferences as the target preference data set; that is, from the multiple preference data sets In the example, select any preferred dataset , as the target preference dataset, which is used to train the speech synthesis model, and other preference datasets are used to train the reward model. , and They respectively represent the preference data set corresponding to naturalness preference, the preference data set corresponding to intelligibility preference, and the preference data set corresponding to similarity preference.

[0061] Next, the initial speech synthesis model can be used to generate synthesized speech for each group of preference data in the target preference data set. That is, the initial speech synthesis model can be used to perform speech synthesis based on the sample text and corresponding prompt speech in each group of preference data in the target preference data set, thereby obtaining the sample synthesized speech output by the initial speech synthesis model. Afterwards, the reward model corresponding to other preferences can be applied to score the sample synthesized speech corresponding to each group of preference data output by the initial speech synthesis model, thereby obtaining the reward score of the sample synthesized speech corresponding to each group of preference data under other preferences. For example, when the target preference data set is When , it is necessary to first perform speech synthesis through the initial speech synthesis model to obtain The sample synthesized speech corresponding to each group of preference data in and The corresponding reward model is The sample synthesized speech corresponding to each group of preference data is scored to obtain The sample synthesized speech corresponding to each group of preference data is and Reward score corresponding to preference.

[0062] It is worth noting that before applying the reward model of other preferences for scoring, it is necessary to pre-train the reward model. Specifically, the preference data set corresponding to other preferences can be applied to train the initial reward model, so as to obtain the trained reward model corresponding to other preferences. Here, the initial reward model is constructed on the basis of the initial speech synthesis model and the linear projection layer. Specifically, a linear projection layer for classification can be added on the basis of the initial speech synthesis model, and the initial reward model is constructed accordingly. When training the initial reward model, the loss function used is the binary classification loss.

[0063] After that, the loss of the initial speech synthesis model on the speech synthesis task can be measured according to the sample preferred speech and sample non-preferred speech in each group of preference data in the target preference data set, the sample synthesized speech corresponding to each group of preference data, the reward score of the sample synthesized speech corresponding to other preferences, and the preference weight configuration corresponding to the initial speech synthesis model, and the parameters of the initial speech synthesis model can be iterated based on this loss to obtain the speech synthesis model.

[0064] It should be noted that in the multi-objective direct preference optimization scheme provided by the present invention, during model training, the initial speech synthesis model can be set Set weight , each set of weights can be expressed as , That is, a set of weights, which is also the preference weight configuration of the model. , and The weights of the three preferences are respectively, and the weight of each preference in each set of weights is between 0 and 1, and the sum of the weights of the three preferences is 1. Different preference weight configurations can be trained to obtain Different speech synthesis models can better meet people's diverse preference needs.

[0065] It is worth noting that after selecting the preference dataset corresponding to one of the preferences as the target preference dataset, the preference datasets corresponding to the other preferences are automatically used for reward model training. After the target preference data set, the remaining preference data sets and Separately train the intelligibility reward model and the similarity reward model. No need to reselect them in reverse, that is, no need to reselect or The target preference dataset is used to repeat the above training process.

[0066] Based on the above embodiment, based on the sample preferred speech and sample non-preferred speech in each group of preference data, the sample synthesized speech corresponding to each group of preference data, the reward score of the sample synthesized speech corresponding to other preferences, and the preference weight configuration corresponding to the initial speech synthesis model, the initial speech synthesis model is iterated on parameters to obtain a speech synthesis model, including: Determine a first loss based on the sample preferred speech and the sample non-preferred speech in each group of preference data, the sample synthesized speech corresponding to each group of preference data, and the weight of the preference in the preference weight configuration corresponding to the initial speech synthesis model; Determine a second loss based on the sample preferred speech and the sample non-preferred speech in each group of preference data, the sample synthesized speech and its reward score corresponding to other preferences, and the weights of other preferences in the preference weight configuration corresponding to the initial speech synthesis model; Based on the first loss and the second loss, parameters of the initial speech synthesis model are iterated to obtain a speech synthesis model.

[0067] Specifically, the process of iterating the parameters of the initial speech synthesis model to obtain the speech synthesis model may include: In an embodiment of the present invention, the overall loss of the initial speech synthesis model can be obtained by weighted optimization through multiple preferences. Specifically, the overall loss can be divided into two parts, one of which is measured by the binary cross entropy loss function BCEloss (Binary Cross Entropy Loss). Specifically, the sample preferred speech and sample non-preferred speech in each group of preference data in the target preference data set, as well as the sample synthesized speech corresponding to each group of preference data, can be used to determine the closeness of the sample synthesized speech generated by the model to the distribution of the sample preferred speech and the sample non-preferred speech, respectively. On this basis, combined with the weight of the preference in the preference weight configuration, the loss of the model is determined, that is, the first loss. The other part can be expressed by with margin, that is, the loss on the reward score, which is called the second loss; specifically, it can be determined by the sample preferred speech and sample non-preferred speech in each group of preference data in the target preference data set, as well as the sample synthesized speech and its corresponding reward score for other preferences, the difference between the score of the sample synthesized speech output by the initial speech synthesis model and the score of the sample preferred speech and the sample non-preferred speech by the reward model of other preferences, and on this basis, combined with the weight of other preferences in the preference weight configuration, the second loss is calculated.

[0068] After that, the first loss and the second loss can be weighted to determine the overall loss, and the initial speech synthesis model can be updated according to the overall loss to obtain the speech synthesis model. Here, the parameters of the initial speech synthesis model can be adjusted according to the calculated overall loss, so that the sample synthesized speech output by the model after the parameter adjustment matches the corresponding preference weight configuration as much as possible, and is close to the corresponding sample preference speech, and away from the sample non-preferential speech, so that the model output can be aligned with the human preference represented by the corresponding preference weight configuration, and finally a trained speech synthesis model can be obtained. It should be noted that after the reward model is pre-trained using the preference data set corresponding to other preferences, the reward model is frozen during the training process of the initial speech synthesis model and does not participate in the update.

[0069] Figure 2 This is a schematic diagram of the model training process provided by the present invention. Figure 2As shown, when When you prefer a dataset for your target, you can apply and Train the reward model for intelligibility separately and similarity reward model Furthermore, after obtaining the sample synthesized speech through the initial speech synthesis model, the output of the model, the sample preferred speech and the sample non-preferred speech, the reward score corresponding to other preferences of the model output, and the preference weight configuration can be combined. , the first loss and the second loss are determined by BCE loss and with margin, and the overall loss of the model is measured accordingly to update the model parameters, and finally the speech synthesis model corresponding to the trained preference weight configuration is obtained.

[0070] In the embodiment of the present invention, during the training process, each set of weights in the multiple preference weight configurations is optimized and updated using the above method, so that multiple speech synthesis models aligned with human preferences with different weights in three dimensions can be obtained. Then, in the actual application process, according to different speech synthesis preferences, a corresponding speech synthesis model can be selected from the multiple trained speech synthesis models. For example, for users with high requirements for the naturalness of synthesized speech, the preferred weight configuration can be selected. A large speech synthesis model can be used to perform speech synthesis, which can obtain synthesized speech that meets user expectations. It can meet the diverse speech synthesis needs of users in different scenarios and has strong scene adaptability.

[0071] The speech synthesis device provided by the present invention is described below. The speech synthesis device described below and the speech synthesis method described above can be referred to each other.

[0072] Figure 3 Schematic diagram of the structure of the speech synthesis device provided by the present invention. Figure 3 As shown, the device comprises: A data determination unit 310, used to determine the text to be synthesized and the user's speech synthesis preference; A speech synthesis unit 320, configured to select a target speech synthesis model from a plurality of speech synthesis models based on the speech synthesis preference, and perform speech synthesis based on the text to be synthesized by applying the target speech synthesis model to obtain a synthesized speech that meets the speech synthesis preference; Among them, each speech synthesis model is trained based on preference data sets and preference weight configurations corresponding to multiple preferences. Different speech synthesis models use different preference weight configurations when training. Each preference data set contains multiple sets of preference data corresponding to preferences, and each set of preference data includes sample text corresponding to the preference, sample preference speech and sample non-preferential speech.

[0073] The speech synthesis device provided by the present invention selects a target speech synthesis model from multiple speech synthesis models according to the user's speech synthesis preference, and applies the target speech synthesis model to perform speech synthesis according to the text to be synthesized, so as to obtain synthesized speech that meets the speech synthesis preference; wherein each speech synthesis model is trained based on a preference data set and a preference weight configuration corresponding to multiple preferences, and different speech synthesis models are trained using different preference weight configurations, thereby overcoming the defect that the human preference alignment method for the speech synthesis model in the traditional solution can only align a single human preference and cannot meet the diverse human preference needs. The speech synthesis model is trained by a multi-objective direct preference optimization method, which not only makes the training process simpler and more efficient, but also can perform human preference alignment from multiple dimensions, and meet the diverse human preference needs by assigning weights to each preference.

[0074] Based on the above embodiment, the device further includes a data set construction unit, which is used to: Determine a sample text corresponding to the preference, and a prompt voice corresponding to the sample text; Based on an initial speech synthesis model, determining a plurality of initial synthesized speech corresponding to the sample text and the prompt speech; the initial speech synthesis model is trained based on a pre-training data set, and the pre-training data set includes a pre-training text and a corresponding pre-training prompt speech; Determine a speech score of each initial synthesized speech under a corresponding preference, and determine a sample preferred speech and a sample non-preferred speech corresponding to the corresponding preference from each initial synthesized speech based on the speech score corresponding to each initial synthesized speech; Based on the sample text and prompt voice corresponding to the preference, and the corresponding sample preferred voice and the sample non-preferred voice, a preference data set corresponding to the preference is constructed.

[0075] Based on the above embodiment, the data set construction unit is used for: Determining a coarse-grained speech score of each of the initial synthesized speech under the plurality of preferences; Based on the coarse-grained speech scores of the initial synthesized speech corresponding to the plurality of preferences, a plurality of candidate synthesized speech are screened from the initial synthesized speech; Determine a fine-grained speech score for each candidate synthesized speech under a corresponding preference; Based on the fine-grained speech scores corresponding to the candidate synthesized speech, sample preferred speech and sample non-preferred speech corresponding to the candidate synthesized speech are determined from the candidate synthesized speech.

[0076] Based on the above embodiment, the multiple preferences include naturalness preference, understanding preference and similarity preference; the data set construction unit is used to: From the candidate synthesized speech corresponding to the naturalness preference, select the candidate synthesized speech with the highest and lowest fine-grained speech scores under the naturalness preference as the sample preferred speech and the sample non-preferred speech corresponding to the naturalness preference, respectively; From the candidate synthesized speech corresponding to the intelligibility preference, select the candidate synthesized speech with the highest and lowest fine-grained speech scores under the intelligibility preference as the sample preferred speech and the sample non-preferred speech corresponding to the intelligibility preference, respectively; From the candidate synthesized speech corresponding to the similarity preference, the candidate synthesized speech with the highest and lowest fine-grained speech scores under the similarity preference are selected as the sample preferred speech and the sample non-preferred speech corresponding to the similarity preference, respectively.

[0077] Based on the above embodiment, the data set construction unit is used for: Performing phoneme conversion on the sample text to obtain a sample phoneme sequence; Performing feature encoding on the prompt voice to obtain sample voice features; Embedding the sample phoneme sequence and the sample speech feature, and concatenating the sample phoneme vector representation and the sample speech vector representation obtained by the embedded coding to obtain a sample fusion vector representation; Based on the sample fusion vector representation, the initial speech synthesis model is applied to perform speech synthesis to obtain multiple initial synthesized speech.

[0078] Based on the above embodiment, the device further includes a model training unit, which is used to: Selecting a preference data set corresponding to any one preference from the preference data sets corresponding to the multiple preferences as a target preference data set; Based on the initial speech synthesis model, determining the sample synthesized speech corresponding to the sample text and the prompt speech in each group of preference data in the target preference data set; Based on a reward model for other preferences, determining a reward score for the sample synthesized speech corresponding to each set of preference data under the other preferences; the reward model is trained based on the preference data set corresponding to the other preferences; Based on the sample preferred speech and sample non-preferred speech in each group of preference data, the sample synthesized speech corresponding to each group of preference data, the reward score of the sample synthesized speech corresponding to other preferences, and the preference weight configuration corresponding to the initial speech synthesis model, the initial speech synthesis model is iterated on parameters to obtain a speech synthesis model.

[0079] Based on the above embodiment, the model training unit is used to: Determine a first loss based on the sample preferred speech and the sample non-preferred speech in each group of preference data, the sample synthesized speech corresponding to each group of preference data, and the weight of any preference in the preference weight configuration corresponding to the initial speech synthesis model; Determine a second loss based on the sample preferred speech and the sample non-preferred speech in each group of preference data, the sample synthesized speech and its reward score corresponding to other preferences, and the weight of the other preferences in the preference weight configuration corresponding to the initial speech synthesis model; Based on the first loss and the second loss, parameters of the initial speech synthesis model are iterated to obtain a speech synthesis model.

[0080] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the speech synthesis method, which includes: determining the text to be synthesized and the speech synthesis preference of the user; based on the speech synthesis preference, selecting a target speech synthesis model from multiple speech synthesis models, and applying the target speech synthesis model to perform speech synthesis based on the text to be synthesized to obtain a synthesized speech that meets the speech synthesis preference; wherein each speech synthesis model is trained based on a preference data set and a preference weight configuration corresponding to multiple preferences, and different speech synthesis models are trained using different preference weight configurations, and each preference data set contains multiple sets of preference data corresponding to the preference, and each set of preference data includes a sample text corresponding to the preference, a sample preference speech and a sample non-preferred speech.

[0081] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0082] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the speech synthesis method provided by the above-mentioned methods, and the method includes: determining the text to be synthesized and the user's speech synthesis preference; based on the speech synthesis preference, selecting a target speech synthesis model from multiple speech synthesis models, and applying the target speech synthesis model to perform speech synthesis based on the text to be synthesized to obtain synthesized speech that meets the speech synthesis preference; wherein each speech synthesis model is trained based on preference data sets and preference weight configurations corresponding to multiple preferences, and different speech synthesis models are trained using different preference weight configurations, and each preference data set contains multiple groups of preference data corresponding to preferences, and each group of preference data includes sample text, sample preference speech, and sample non-preferential speech corresponding to the preference.

[0083] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the speech synthesis method provided by the above-mentioned methods, the method comprising: determining a text to be synthesized and a user's speech synthesis preference; based on the speech synthesis preference, selecting a target speech synthesis model from a plurality of speech synthesis models, and applying the target speech synthesis model to perform speech synthesis based on the text to be synthesized to obtain a synthesized speech that meets the speech synthesis preference; wherein each speech synthesis model is trained based on a preference data set and a preference weight configuration corresponding to a plurality of preferences, different speech synthesis models are trained using different preference weight configurations, each preference data set contains a plurality of groups of preference data corresponding to a preference, and each group of preference data includes a sample text corresponding to the preference, a sample preference speech, and a sample non-preferential speech.

[0084] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0085] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A speech synthesis method, characterized in that: include: Determine the text to be synthesized and the user's speech synthesis preference; Based on the speech synthesis preference, a target speech synthesis model is selected from a plurality of speech synthesis models, and based on the text to be synthesized, the target speech synthesis model is applied to perform speech synthesis to obtain a synthesized speech that meets the speech synthesis preference; Among them, each speech synthesis model is trained based on preference data sets and preference weight configurations corresponding to multiple preferences. Different speech synthesis models use different preference weight configurations when training. Each preference data set contains multiple sets of preference data corresponding to preferences, and each set of preference data includes sample text corresponding to the preference, sample preference speech and sample non-preferential speech.

2. The speech synthesis method according to claim 1, characterized in that: The preference data set corresponding to each preference is determined based on the following steps: Determine a sample text corresponding to the preference, and a prompt voice corresponding to the sample text; Based on an initial speech synthesis model, determining a plurality of initial synthesized speech corresponding to the sample text and the prompt speech; the initial speech synthesis model is trained based on a pre-training data set, and the pre-training data set includes a pre-training text and a corresponding pre-training prompt speech; Determine a speech score of each initial synthesized speech under a corresponding preference, and determine a sample preferred speech and a sample non-preferred speech corresponding to the corresponding preference from each initial synthesized speech based on the speech score corresponding to each initial synthesized speech; Based on the sample text and prompt voice corresponding to the preference, and the corresponding sample preferred voice and the sample non-preferred voice, a preference data set corresponding to the preference is constructed.

3. The speech synthesis method according to claim 2, characterized in that: The step of determining a speech score of each initial synthesized speech under a corresponding preference, and determining a sample preferred speech and a sample non-preferred speech corresponding to the corresponding preference from each initial synthesized speech based on the speech score corresponding to each initial synthesized speech, comprises: Determining a coarse-grained speech score of each of the initial synthesized speech under the plurality of preferences; Based on the coarse-grained speech scores of the initial synthesized speech corresponding to the plurality of preferences, a plurality of candidate synthesized speech are screened from the initial synthesized speech; Determine a fine-grained speech score for each candidate synthesized speech under a corresponding preference; Based on the fine-grained speech scores corresponding to the candidate synthesized speech, sample preferred speech and sample non-preferred speech corresponding to the candidate synthesized speech are determined from the candidate synthesized speech.

4. The speech synthesis method according to claim 3, characterized in that: The multiple preferences include naturalness preference, intelligibility preference and similarity preference; the step of determining sample preferred speech and sample non-preferred speech corresponding to the corresponding preference from the candidate synthesized speech based on the fine-grained speech scores corresponding to the candidate synthesized speech includes: From the candidate synthesized speech corresponding to the naturalness preference, select the candidate synthesized speech with the highest and lowest fine-grained speech scores under the naturalness preference as the sample preferred speech and the sample non-preferred speech corresponding to the naturalness preference, respectively; From the candidate synthesized speech corresponding to the intelligibility preference, select the candidate synthesized speech with the highest and lowest fine-grained speech scores under the intelligibility preference as the sample preferred speech and the sample non-preferred speech corresponding to the intelligibility preference, respectively; From the candidate synthesized speech corresponding to the similarity preference, the candidate synthesized speech with the highest and lowest fine-grained speech scores under the similarity preference are selected as the sample preferred speech and the sample non-preferred speech corresponding to the similarity preference, respectively.

5. The speech synthesis method according to any one of claims 2 to 4, characterized in that: The determining, based on the initial speech synthesis model, a plurality of initial synthesized speech corresponding to the sample text and the prompt speech comprises: Performing phoneme conversion on the sample text to obtain a sample phoneme sequence; Performing feature encoding on the prompt voice to obtain sample voice features; Embedding the sample phoneme sequence and the sample speech feature, and concatenating the sample phoneme vector representation and the sample speech vector representation obtained by the embedded coding to obtain a sample fusion vector representation; Based on the sample fusion vector representation, the initial speech synthesis model is applied to perform speech synthesis to obtain multiple initial synthesized speech.

6. The speech synthesis method according to any one of claims 2 to 4, characterized in that: Each speech synthesis model is trained based on the following steps: Selecting a preference data set corresponding to any one preference from the preference data sets corresponding to the multiple preferences as a target preference data set; Based on the initial speech synthesis model, determining the sample synthesized speech corresponding to the sample text and the prompt speech in each group of preference data in the target preference data set; Based on the reward model of other preferences, determining the reward score of the sample synthesized speech corresponding to each group of preference data under the other preferences; The reward model is trained based on the preference data set corresponding to the other preferences; Based on the sample preferred speech and sample non-preferred speech in each group of preference data, the sample synthesized speech corresponding to each group of preference data, the reward score of the sample synthesized speech corresponding to other preferences, and the preference weight configuration corresponding to the initial speech synthesis model, the initial speech synthesis model is iterated on parameters to obtain a speech synthesis model.

7. The speech synthesis method according to claim 6, characterized in that: The method of performing parameter iteration on the initial speech synthesis model based on the sample preferred speech and sample non-preferred speech in each group of preference data, the sample synthesized speech corresponding to each group of preference data, the reward score of the sample synthesized speech corresponding to other preferences, and the preference weight configuration corresponding to the initial speech synthesis model to obtain the speech synthesis model includes: Determine a first loss based on the sample preferred speech and the sample non-preferred speech in each group of preference data, the sample synthesized speech corresponding to each group of preference data, and the weight of any preference in the preference weight configuration corresponding to the initial speech synthesis model; Determine a second loss based on the sample preferred speech and the sample non-preferred speech in each group of preference data, the sample synthesized speech and its reward score corresponding to other preferences, and the weight of the other preferences in the preference weight configuration corresponding to the initial speech synthesis model; Based on the first loss and the second loss, parameters of the initial speech synthesis model are iterated to obtain a speech synthesis model.

8. A speech synthesis device, characterized in that: include: A data determination unit, used to determine the text to be synthesized and the user's speech synthesis preference; A speech synthesis unit, configured to select a target speech synthesis model from a plurality of speech synthesis models based on the speech synthesis preference, and perform speech synthesis based on the text to be synthesized by applying the target speech synthesis model to obtain a synthesized speech that meets the speech synthesis preference; Among them, each speech synthesis model is trained based on preference data sets and preference weight configurations corresponding to multiple preferences. Different speech synthesis models use different preference weight configurations when training. Each preference data set contains multiple sets of preference data corresponding to preferences, and each set of preference data includes sample text corresponding to the preference, sample preference speech and sample non-preferential speech.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the speech synthesis method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the speech synthesis method according to any one of claims 1 to 7 is implemented.

Images