Speech synthesis method, speech synthesis device, electronic device, and storage medium

Abstract

The voice synthesis method, voice synthesis device, electronic equipment and storage medium provided by the present application relate to the fields of artificial intelligence and financial technology. The method comprises the following steps: extracting speaker features from sample voice data sets through a feature embedder in an initial speaker verification model to obtain sample set speaker features; classifying the sample set speaker features through a feature classifier in the initial speaker verification model to obtain a sample verification speaker category, then adjusting parameters of the initial speaker verification model to obtain a target speaker verification model; extracting speaker features from target voice data through a feature embedder in the target speaker verification model to obtain target speaker features; and generating voice according to the target speaker features and target text features to obtain target synthesized voice data. The present application can alleviate the adverse effects of background noise in voice data and improve the accuracy of voice synthesis.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence technology and financial technology, and in particular to a speech synthesis method, a speech synthesis device, an electronic device and a storage medium. Background Art

[0002] Current zero-shot speech synthesis (Zero-Shot TTS) technology can generate realistic speech using limited target speaker data. However, most existing zero-shot TTS systems require high-quality, noise-free target speaker data to effectively extract speaker features. This is difficult for resource-constrained scenarios (such as financial services). For example, in insurance or banking systems in financial services scenarios, target speakers (such as customer service representatives) are usually recorded in noisy environments, and the recorded speech data contains background noise. Background noise affects the extraction of speaker features from the speech data, resulting in poor accuracy of speaker features, and thus poor accuracy of speech synthesis. Summary of the Invention

[0003] The main purpose of the embodiments of the present application is to propose a speech synthesis method, a speech synthesis device, an electronic device and a storage medium, which can alleviate the adverse effects of background noise in speech data and improve the accuracy of speech synthesis.

[0004] To achieve the above objectives, a first aspect of an embodiment of the present application provides a speech synthesis method, the method comprising:

[0005] Acquire a sample voice data group; wherein the sample voice data group includes two sample voice data;

[0006] Performing speaker feature extraction on the two sample speech data of the sample speech data group by using a feature embedder in a preset initial speaker verification model to obtain speaker features of the sample group;

[0007] Performing speaker classification on the speaker features of the sample group by using the feature classifier in the initial speaker verification model to obtain a sample verification speaker category of the sample speech data group; wherein the sample verification speaker category indicates whether the speaker to which one sample speech data in the sample speech data group belongs is the same as or different from the speaker to which another sample speech data belongs;

[0008] Computing a contrastive learning loss based on the sample verification speaker category to obtain speaker contrastive loss data;

[0009] Adjusting parameters of the initial speaker verification model according to the speaker contrast loss data to obtain a target speaker verification model;

[0010] obtaining target speech data, extracting speaker features of the target speech data by a feature embedder in the target speaker verification model to obtain target speaker features;

[0011] generating speech according to the target speaker features and preset target text features to obtain target synthesized speech data.

[0012] Optionally, after the initial speaker verification model is adjusted in parameters according to the speaker contrast loss data to obtain the target speaker verification model, the method further comprises:

[0013] updating the target speaker verification model according to the sample speech data set, specifically comprising:

[0014] determining one of the two sample speech data in the sample speech data set as sample individual speech data;

[0015] extracting speaker features of the sample individual speech data by a feature embedder in the target speaker verification model to obtain sample individual speaker features;

[0016] performing speech synthesis according to the sample individual speaker features and preset sample text features to obtain sample synthesized speech data;

[0017] performing speaker loss calculation according to the sample individual speech data and the sample synthesized speech data to obtain speaker loss data;

[0018] updating the target speaker verification model according to the speaker loss data.

[0019] Optionally, the speaker loss calculation according to the sample individual speech data and the sample synthesized speech data to obtain speaker loss data comprises:

[0020] performing speaker recognition on the sample individual speech data to obtain a first speaker, and performing speaker recognition on the sample synthesized speech data to obtain a second speaker;

[0021] performing speaker difference calculation according to the first speaker and the second speaker to obtain speaker verification loss data;

[0022] performing spectrum recognition on the sample individual speech data to obtain a first speech spectrum, and performing spectrum recognition on the sample synthesized speech data to obtain a second speech spectrum;

[0023] performing spectrum error calculation according to the first speech spectrum and the second speech spectrum to obtain speaker reconstruction loss data;

[0024] Data fusion is performed according to the speaker verification loss data and the speaker reconstruction loss data to obtain the speaker loss data.

[0025] Optionally, determining one of the two sample voice data in the sample voice data group as sample independent voice data includes:

[0026] Determine two of the sample voice data in the sample voice data group as first sample voice data and second sample voice data respectively;

[0027] If the first sample voice data and the second sample voice data are voice data generated under the same background noise state, determining the first sample voice data or the second sample voice data as the sample independent voice data;

[0028] If the first sample voice data and the second sample voice data are voice data generated under different background noise conditions, obtaining the background noise contained in the first sample voice data to obtain a first sample background noise, and obtaining the background noise contained in the second sample voice data to obtain a second sample background noise;

[0029] Performing a noise comparison between the first sample background noise and the second sample background noise to obtain a noise comparison result;

[0030] If the intensity of the first sample background noise is greater than or equal to the intensity of the second sample background noise, determining the first sample voice data as the sample independent voice data;

[0031] If the intensity of the first sample background noise is less than the intensity of the second sample background noise, the second sample voice data is determined as the sample independent voice data.

[0032] Optionally, before extracting speaker features from the two sample speech data of the sample speech data group using a feature embedder in a preset initial speaker verification model to obtain speaker features of the sample group, the method further comprises:

[0033] Determine the two sample voice data as first sample voice data and second sample voice data respectively;

[0034] Performing speaker feature extraction on the first sample speech data by the feature embedder to obtain a first speaker feature, and performing speaker feature extraction on the second sample speech data by the feature embedder to obtain a second speaker feature;

[0035] performing noise recognition on the first speaker feature to obtain a first noise feature, and performing noise recognition on the second speaker feature to obtain a second noise feature;

[0036] performing a denoising degree evaluation based on the first noise feature and the second noise feature to obtain a denoising score of the feature embedder;

[0037] The feature embedder is adjusted according to the denoising score.

[0038] Optionally, the performing denoising degree evaluation according to the first noise feature and the second noise feature to obtain a denoising score of the feature embedder includes:

[0039] Determining a first feature denoising sub-score based on the strength of the first noise feature, and determining a second feature denoising sub-score based on the strength of the second noise feature;

[0040] If the first sample speech data and the second sample speech data are speech data generated under the same background noise state, performing feature difference analysis on the first noise feature and the second noise feature to obtain a stable denoising sub-score;

[0041] The denoising score is obtained by performing score aggregation based on the first feature denoising sub-score, the second feature denoising sub-score, and the stable denoising sub-score.

[0042] Optionally, adjusting the feature embedder according to the denoising score comprises:

[0043] If the denoising score is greater than or equal to a preset denoising score threshold, adjusting parameters of the feature embedder according to the denoising score;

[0044] If the denoising score is less than the denoising score threshold, a denoising layer is added to the feature embedder to perform structural adjustment on the feature embedder.

[0045] To achieve the above-mentioned object, a second aspect of an embodiment of the present application provides a speech synthesis device, comprising:

[0046] A sample data acquisition module is used to acquire a sample voice data group; wherein the sample voice data group includes two sample voice data;

[0047] A sample feature extraction module is used to extract speaker features from the two sample speech data of the sample speech data group using a feature embedder in a preset initial speaker verification model to obtain speaker features of the sample group;

[0048] a speaker classification module, configured to perform speaker classification on the speaker features of the sample group using a feature classifier in the initial speaker verification model to obtain a sample verification speaker category of the sample speech data group; wherein the sample verification speaker category indicates whether the speaker to which one sample speech data in the sample speech data group belongs is the same as or different from the speaker to which another sample speech data belongs;

[0049] a contrastive loss calculation module, configured to perform contrastive learning loss calculation based on the sample to verify the speaker category and obtain speaker contrastive loss data;

[0050] a model parameter adjustment module, configured to adjust parameters of the initial speaker verification model according to the speaker contrast loss data to obtain a target speaker verification model;

[0051] a target data acquisition module, configured to acquire target speech data, and perform speaker feature extraction on the target speech data using a feature embedder in the target speaker verification model to obtain target speaker features;

[0052] The speech generation module is used to generate speech according to the target speaker characteristics and preset target text characteristics to obtain target synthesized speech data.

[0053] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the speech synthesis method described in the first aspect.

[0054] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, it implements the speech synthesis method described in the first aspect above.

[0055] The speech synthesis method, speech synthesis device, electronic device, and storage medium proposed in this application first obtain a sample speech data set, which includes two sample speech data. The two sample speech data may belong to the same speaker or different speakers. Furthermore, a feature embedder in an initial speaker verification model extracts speaker features from the two sample speech data to obtain sample group speaker features. In this way, the sample speaker features can indicate the speakers corresponding to the two sample speech data. Furthermore, a feature classifier in the initial speaker verification model performs speaker classification on the sample group speaker features to obtain a sample verification speaker category for the sample speech data set. In this way, the sample verification speaker category can be used to evaluate the accuracy of speaker feature extraction performed by the feature embedder. Furthermore, a contrastive learning loss is calculated based on the sample verification speaker category to obtain speaker contrast loss data. Then, the parameters of the initial speaker verification model are adjusted based on the speaker contrast loss data to obtain a target speaker verification model. This enables the feature embedder of the target speaker verification model to distinguish between speaker features and noise features, enabling accurate characterization of the target speaker under noisy conditions. In actual use, the target speech data is obtained, and the speaker features of the target speech data are extracted by the feature embedder in the target speaker verification model to obtain the target speaker features. Then, speech generation is performed based on the target speaker features and the preset target text features to obtain the target synthesized speech data. In this way, the background noise in the target speech data can be effectively removed by the feature embedder in the target speaker verification model, and the output target speaker features can accurately characterize the target speaker corresponding to the target speech data, so that the generated target synthesized speech data is more real and realistic. In summary, the present application can alleviate the adverse effects of background noise in speech data and improve the accuracy of speech synthesis. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flowchart of the speech synthesis method provided in an embodiment of the present application;

[0057] Figure 2 is a flowchart of a speech synthesis method provided by another embodiment of the present application;

[0058] Figure 3 yes Figure 2 Flowchart of step 204 in FIG.

[0059] Figure 4 is a flowchart of a speech synthesis method provided by another embodiment of the present application;

[0060] Figure 5 yes Figure 4 Flowchart of step 401 in FIG.

[0061] Figure 6 yes Figure 4 Flowchart of step 404 in FIG.

[0062] Figure 7 This is a block diagram of the module structure of the speech synthesis device provided in an embodiment of the present application;

[0063] Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0065] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0067] First, let’s analyze some of the terms used in this application:

[0068] Artificial intelligence (AI) is a new technical discipline that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. A branch of computer science, AI seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thinking. It also encompasses the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0069] Natural language processing (NLP): NLP uses computers to process, understand, and apply human languages ​​(such as Chinese and English). A branch of artificial intelligence, NLP is an interdisciplinary field between computer science and linguistics, often referred to as computational linguistics. Natural language processing encompasses grammatical analysis, semantic analysis, and discourse comprehension. Natural language processing is commonly used in technical fields such as machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information and image processing, information extraction and filtering, text classification and clustering, public opinion analysis, and opinion mining. It encompasses data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research related to language processing, and linguistics research related to language computing.

[0070] Zero-Shot TTS is a speech synthesis technology that can generate the speech of any speaker without specific speech data. This technology utilizes deep learning and neural networks. By training on datasets of multiple speakers, it can generate output that reflects the speech style of a specific speaker based on text input. Zero-Shot TTS eliminates the need to collect independent speech data for each speaker, significantly improving synthesis efficiency and flexibility.

[0071] Currently, insurance and banking systems in financial services are becoming increasingly automated and intelligent. Speech synthesis technology is particularly widely used in financial services. Speech synthesis aims to extract speaker features from speech data and then generate target synthesized speech based on these features and textual features, without requiring the speaker to speak the entire text. Current zero-shot speech synthesis technologies can generate realistic speech using limited speaker data. However, these technologies still have the following shortcomings in terms of data efficiency and noise robustness: 1. High demand for speaker data: Most existing zero-shot TTS systems require high-quality, noise-free speaker data to effectively extract speaker features. 2. Unrobust speaker feature extraction: Many models lack robustness in speaker embedding representations, especially in noisy conditions. The extracted features may be contaminated, resulting in reduced speaker consistency in the generated speech. 3. Lack of efficient self-supervision mechanisms: Traditional TTS systems require large amounts of labeled data for training. However, current zero-shot TTS models have limited optimization for data efficiency and are unable to fully utilize the potential information in unlabeled data. 4. Inadequate noise processing capabilities: In real-world applications, speaker data may contain ambient noise. Existing systems lack effective processing capabilities for this noise, resulting in unwanted noise artifacts in the generated speech. 5. Poor naturalness and clarity of synthesized speech: Existing methods often struggle to maintain the naturalness of speech and the timbre of the target speaker while maintaining clarity and coherence, particularly in the presence of background noise.

[0072] Based on this, the embodiments of the present application propose a speech synthesis method, a speech synthesis device, an electronic device and a computer-readable storage medium, which can alleviate the adverse effects of background noise in speech data and improve the accuracy of speech synthesis.

[0073] The speech synthesis method provided in the embodiments of the present application can be applied to terminals and servers, and can also be software running on the server. The server can be configured as an independent physical server, or as a server cluster or distributed system consisting of multiple physical servers. It can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the speech synthesis method, etc., but is not limited to the above forms.

[0074] The present application can be used in many general or special computer system environments or configurations. For example: server computers, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0075] The embodiments of the present application provide a speech synthesis method, a speech synthesis device, an electronic device, and a computer-readable storage medium, which are specifically illustrated by the following embodiments. First, the speech synthesis method in the embodiments of the present application is described.

[0076] It should be noted that in each specific implementation method of the present application, when it comes to processing data related to the user's identity or characteristics, such as the user's voice data, the user's permission or consent must be obtained first. Moreover, the collection, use and processing of this data will comply with relevant laws, regulations and standards.

[0077] Reference Figure 1 , Figure 1 This is an optional flowchart of the speech synthesis method provided in an embodiment of the present application, which may include but is not limited to steps 101 to 107.

[0078] Step 101: Acquire a sample speech data set; wherein the sample speech data set includes two sample speech data;

[0079] Step 102: extract speaker features from two sample speech data of the sample speech data group using a feature embedder in a preset initial speaker verification model to obtain speaker features of the sample group;

[0080] Step 103, performing speaker classification on the speaker features of the sample group using the feature classifier in the initial speaker verification model to obtain the sample verification speaker category of the sample speech data group;

[0081] Step 104 , performing contrastive learning loss calculation based on the sample verification speaker category to obtain speaker contrastive loss data;

[0082] Step 105 , adjusting parameters of the initial speaker verification model according to the speaker contrast loss data to obtain a target speaker verification model;

[0083] Step 106: acquiring target speech data, and performing speaker feature extraction on the target speech data using a feature embedder in a target speaker verification model to obtain target speaker features;

[0084] Step 107 : generating speech based on the target speaker characteristics and the preset target text characteristics to obtain target synthesized speech data.

[0085] In steps 101 to 107 illustrated in the embodiment of the present application, speaker features are first extracted from two sample speech data using a feature embedder in an initial speaker verification model to obtain speaker features for a sample group. In this way, the sample speaker features can indicate the speakers corresponding to the two sample speech data. Furthermore, a feature classifier in the initial speaker verification model outputs a sample verification speaker category. In this way, the accuracy of the speaker feature extraction performed by the feature embedder can be evaluated using the sample verification speaker category. Furthermore, speaker contrast loss data is calculated based on the sample verification speaker category. The initial speaker verification model is then parameterized based on the speaker contrast loss data to obtain a target speaker verification model. This enables the feature embedder of the target speaker verification model to distinguish between speaker features and noise features, enabling accurate characterization of the target speaker under noisy conditions. In actual use, the feature embedder in the target speaker verification model extracts speaker features from the target speech data to obtain target speaker features. Speech generation is then performed based on the target speaker features and preset target text features to obtain target synthesized speech data. In this way, the feature embedder can effectively remove background noise from the target speech data, and the output target speaker features can accurately characterize the target speaker corresponding to the target speech data, thereby generating more realistic and lifelike target synthesized speech data. In summary, this application can alleviate the adverse effects of background noise in speech data and improve the accuracy of speech synthesis.

[0086] When the embodiment of the present application is applied to customer service, online consultation and other scenarios in insurance systems or banking systems in financial service scenarios, it is only necessary for the customer service or the person being asked to output a segment of speaker data to generate target synthesized speech data based on the speaker data. The speaker data is the same as the speaker of the target synthesized speech data, but the speech content is different. In this way, not only can the accuracy of speaker feature extraction for customer service or the person being asked be improved, but the adverse effects of the length or difficulty of the speech content on the customer service or the person being asked be reduced, service efficiency is improved, and user experience is enhanced. For example, for a customer service scenario, when launching a new service (such as a pet insurance product), there is no need for the customer service to enter the complete content corresponding to the new service, but to arbitrarily enter a segment of customer service speech data, and then the feature embedder in the target speaker verification model is used to extract the speaker features of the customer service speech data to obtain the customer service speaker features, and then perform speech generation together with the text features of the content corresponding to the new service to obtain the target synthesized speech data. In this way, the target synthesized speech data in the financial service scenario does not need to be recorded by the customer service based on the complete content, and is not affected by the background noise in the customer service speech data. It can accurately retain the speaker features of the customer service and improve the accuracy of speech synthesis.

[0087] When the embodiments of the present application are applied to digital medical, medical health and other scenarios in medical scenarios, only the customer service or the inquiry object (such as a patient) needs to output a piece of speaker data, and the target synthesized speech data can be generated based on the speaker data. The speaker data is the same as the speaker of the target synthesized speech data, but the speech content is different. For example, in a digital medical scenario, if the doctor's office changes, the customer service does not need to enter the complete office name and office location, but can arbitrarily enter a piece of customer service speech data, and then use the feature embedder in the target speaker verification model to extract the speaker features of the customer service speech data to obtain the customer service speaker features, and then perform speech generation together with the text features representing the office name and office location to obtain the target synthesized speech data. In this way, the target synthesized speech data in the medical scenario does not need to be recorded by the customer service based on the complete content, and is not affected by the background noise in the customer service speech data. It can accurately retain the speaker features of the customer service and improve the accuracy of speech synthesis.

[0088] In step 101 of some embodiments, a sample speech data set is obtained. The sample speech data set includes two sample speech data sets. One of the two sample speech data sets is used as the first sample speech data set, and the other sample speech data set is used as the second sample speech data set. The speaker of the first sample speech data set and the speaker of the second sample speech data set may be the same or different. The generalization capability of the model can also be improved by adding artificial noise and diversifying the sample speech data.

[0089] In one embodiment, if the speaker of the first sample voice data is the same as the speaker of the second sample voice data, the speech content of the first sample voice data is different from the speech content of the second sample voice data.

[0090] In one example, when the speech synthesis method is applied to a terminal, the sample speech data set can be obtained through recording, Bluetooth transmission, wired transmission, or downloading. When the sample speech data set is obtained through recording, the terminal is equipped with a microphone, and the microphone is used to collect audio to achieve the recording of the speech sample data set. When the speech synthesis method is applied to a server, the sample speech data set can be uploaded to the server by the terminal or downloaded from another server or database.

[0091] In one embodiment, referring to Figure 2 Before step 102, the speech synthesis method may further include:

[0092] Step 201: Determine two sample voice data as first sample voice data and second sample voice data respectively;

[0093] Step 202: extracting speaker features from the first sample speech data using a feature embedder to obtain a first speaker feature, and extracting speaker features from the second sample speech data using a feature embedder to obtain a second speaker feature.

[0094] Step 203: performing noise recognition on the first speaker feature to obtain a first noise feature, and performing noise recognition on the second speaker feature to obtain a second noise feature;

[0095] Step 204: Evaluate the denoising degree based on the first noise feature and the second noise feature to obtain a denoising score of the feature embedder;

[0096] Step 205: Adjust the feature embedder according to the denoising score.

[0097] In step 203, noise recognition can be performed using a noise recognition model. A noise recognition model refers to a model used to recognize noise. Common noise recognition models include deep learning models, traditional machine learning models, and adaptive filters.

[0098] In step 204, a stronger intensity of the first noise feature indicates more noise in the first speaker feature, thereby indicating a lower level of denoising. A weaker intensity of the first noise feature indicates less noise in the first speaker feature, thereby indicating a higher level of denoising. A stronger intensity of the second noise feature indicates more noise in the second speaker feature, thereby indicating a lower level of denoising. A weaker intensity of the second noise feature indicates less noise in the second speaker feature, thereby indicating a higher level of denoising. The level of denoising can be assessed using a scoring function or scoring model.

[0099] In one embodiment, referring to Figure 3 , step 204 may include:

[0100] Step 301, determining a first feature denoising sub-score based on the strength of the first noise feature, and determining a second feature denoising sub-score based on the strength of the second noise feature;

[0101] Step 302: If the first sample speech data and the second sample speech data are speech data generated under the same background noise conditions, then feature difference analysis is performed on the first noise feature and the second noise feature to obtain a stable denoising sub-score;

[0102] Step 303 : performing score aggregation based on the first feature denoising sub-score, the second feature denoising sub-score, and the stable denoising sub-score to obtain a denoising score.

[0103] In step 301, the higher the intensity of the first noise feature, the higher the first feature denoising sub-score. The weaker the intensity of the first noise feature, the lower the first feature denoising sub-score. The higher the intensity of the second noise feature, the higher the second feature denoising sub-score. The weaker the intensity of the second noise feature, the lower the second feature denoising sub-score. An intensity score mapping function / mapping table can be set to obtain the first feature denoising sub-score and the second feature denoising sub-score based on the table lookup. The intensity of the first noise feature is generally reflected in the amplitude of the noise spectrum or the amplitude of the noise power spectrum. In daily life, the stronger the noise, the noisier it is.

[0104] In step 302, if the first sample speech data and the second sample speech data are speech data generated under the same background noise conditions, it means that the background noise contained in the first sample speech data and the second sample speech data is basically the same. In this case, the first noise feature and the second noise feature should be basically the same, otherwise it means that the performance of the feature embedder is unstable and the speaker feature extraction cannot be performed stably. The first noise feature and the second noise feature can be analyzed for feature differences using a deep learning model (such as a CNN model) to obtain a stable denoising sub-score. The higher the stable denoising sub-score, the higher the stability of the feature embedder, and the lower the stable denoising sub-score, the lower the stability of the feature embedder.

[0105] In step 303, the first feature denoiser score and the second feature denoiser score are proportional to the denoising score, and the stable denoiser score is inversely proportional to the denoising score. The score aggregation can be achieved by calculating a weighted average or other ways to obtain the denoising score.

[0106] The embodiment of steps 301 to 303 has the benefit of introducing a stable denoiser score to improve the accuracy of determining the denoising score, thereby improving the accuracy of evaluating the performance of the feature embedder.

[0107] In an embodiment, before step 303, step 204 can further include: if the first sample speech data and the second sample speech data are speech data generated in different background noise states, generating a stable denoiser score with a score of zero. In this way, the case of two speech data generated in different background noise states can be compatible, improving the applicability.

[0108] In step 205, the higher the denoising score, the stronger the denoising ability of the feature embedder, and the smaller the adjustment of the feature embedder. The lower the denoising score, the weaker the denoising ability of the feature embedder, and the larger the adjustment of the feature embedder.

[0109] The embodiment of steps 201 to 205 is that before outputting the sample group speaker features by using the feature embedder, the feature embedder can be adjusted according to the denoising score to improve the accuracy of the sample group speaker features, thereby reducing the number of adjustments of the parameters of the feature embedder, saving processing resources and improving the efficiency of speech synthesis.

[0110] In an embodiment, step 205 can include: if the denoising score is greater than or equal to a preset denoising score threshold, adjusting the parameters of the feature embedder according to the denoising score; and if the denoising score is less than the denoising score threshold, adding a denoising layer to the feature embedder to adjust the structure of the feature embedder.

[0111] The embodiment has the benefit of selecting parameter adjustment or structural adjustment of the feature embedder by the denoising score, which can improve the adjustment flexibility and improve the denoising performance of the feature embedder.

[0112] In step 102 of some embodiments, the feature embedder in the preset initial speaker verification model is used to extract speaker features from two sample speech data of the sample speech data group to obtain the sample group speaker features.

[0113] The initial speaker verification model is a neural network model used to verify whether two speech data samples are spoken by the same speaker, such as the CAM model or the CAM++ model. The initial speaker verification model includes a feature embedder and a feature classifier. The feature embedder is used to perform deep feature embedding on the speech data. The feature classifier is used to classify the speaker based on the output of the feature embedder. In step 102, the feature embedder's feature embedding capabilities are primarily used to extract speaker features from the two sample speech data samples, resulting in a sample group speaker feature that effectively characterizes the speakers corresponding to the two sample speech data samples.

[0114] In one embodiment, the feature embedder includes a mel spectrum enhancement layer, a convolution layer, and a pooling layer. Step 102 may include:

[0115] Performing mel-spectrogram enhancement on one sample speech data set of the sample speech data set through the mel-spectrogram enhancement layer to obtain a first enhanced mel-spectrogram speaker feature, and performing mel-spectrogram enhancement on another sample speech data set of the sample speech data set to obtain a second enhanced speaker mel-spectrogram feature;

[0116] Perform feature convolution on the first enhanced speaker mel-spectrogram feature and the second enhanced speaker mel-spectrogram feature through a convolution layer to obtain an initial set of speaker features;

[0117] The initial group of speaker features are aggregated through the pooling layer to obtain the sample group speaker features.

[0118] The benefit of this embodiment is that speaker features can be extracted with high accuracy even when the speech data contains background noise.

[0119] In one embodiment, the mel spectrum enhancement layer includes a first convolutional layer, a complex encoding layer, a complex decoding layer, and a second convolutional layer. The complex encoding layer includes a first complex two-dimensional convolutional layer (specifically, a first real convolutional layer, a first imaginary convolutional layer), a first complex batch normalization layer, and a first real activation function layer. The complex decoding layer includes a second complex two-dimensional convolutional layer (specifically, a second real convolutional layer, a second imaginary convolutional layer), a second complex batch normalization layer, and a second real activation function layer.

[0120] In step 103 of some embodiments, the speaker features of the sample group are classified by a feature classifier in the initial speaker verification model to obtain a sample verification speaker category for the sample speech data group. The sample verification speaker category indicates whether the speaker to which one sample speech data in the sample speech data group belongs is the same as or different from the speaker to which another sample speech data belongs.

[0121] In one embodiment, the feature classifier includes a fully connected layer, a normalization layer, and an activation function layer. The speaker features of the sample group are input into the fully connected layer, the output of the fully connected layer is input into the normalization layer, and the output of the normalization layer is input into the activation function layer to obtain the speaker category of the sample verification.

[0122] In step 104 of some embodiments, a contrastive learning loss is calculated based on the sample verification speaker category to obtain speaker contrastive loss data. Contrastive learning is a learning method that focuses on extracting meaningful representations by comparing positive and negative examples. It uses the assumption that in the learned embedding space, similar instances should be closer, while dissimilar instances should be farther apart. Specifically in this embodiment, if the sample speech data group is a positive sample, the speaker contrastive loss data should tend to indicate that the speaker to which the sample verification speaker category indicates one sample speech data in the sample speech data group belongs is the same as the speaker to which the other sample speech data belongs; if the sample speech data group is a negative sample, the speaker contrastive loss data should tend to indicate that the speaker to which the sample verification speaker category indicates one sample speech data in the sample speech data group belongs is different from the speaker to which the other sample speech data belongs. A cross-entropy loss function, etc., can be selected for contrastive learning loss calculation.

[0123] In step 105 of some embodiments, parameters of the initial speaker verification model are adjusted based on the speaker contrast loss data to obtain a target speaker verification model. During the parameter adjustment process, the speaker contrast loss data is compared with a preset speaker contrast loss threshold. If the speaker contrast loss data is greater than the speaker contrast loss threshold, the parameters of the initial speaker verification model are continuously adjusted until the speaker contrast loss data is less than or equal to the speaker contrast loss threshold. At this point, the parameter adjustment is stopped to obtain the target speaker verification model.

[0124] In one embodiment, referring to Figure 4 After step 105, the speech synthesis method further includes: updating the target speaker verification model according to the sample speech data set, specifically including:

[0125] Step 401, determining one of the two sample voice data in the sample voice data group as sample independent voice data;

[0126] Step 402: extract speaker features from the sample individual speech data using a feature embedder in the target speaker verification model to obtain sample individual speaker features.

[0127] Step 403: Perform speech synthesis based on the sample individual speaker characteristics and preset sample text characteristics to obtain sample synthesized speech data;

[0128] Step 404: Calculate speaker loss based on the sample individual speech data and the sample synthesized speech data to obtain speaker loss data;

[0129] Step 405: Update the target speaker verification model according to the speaker loss data.

[0130] In step 401, one of the two sample speech data in the sample speech data group may be randomly determined as the sample independent speech data, or the first or the second of the two sample speech data may be determined as the sample independent speech data each time.

[0131] In one embodiment, referring to Figure 5 , step 401 may include:

[0132] Step 501: Determine two sample voice data in the sample voice data group as first sample voice data and second sample voice data respectively;

[0133] Step 502: If the first sample voice data and the second sample voice data are voice data generated under the same background noise state, the first sample voice data or the second sample voice data is determined as sample independent voice data;

[0134] Step 503: If the first sample voice data and the second sample voice data are voice data generated under different background noise conditions, obtaining the background noise contained in the first sample voice data to obtain a first sample background noise, and obtaining the background noise contained in the second sample voice data to obtain a second sample background noise;

[0135] Step 504: performing a noise comparison between the first sample background noise and the second sample background noise to obtain a noise comparison result;

[0136] Step 505: If the intensity of the first sample background noise is greater than or equal to the intensity of the second sample background noise, the first sample voice data is determined as sample independent voice data;

[0137] Step 506: If the intensity of the first sample background noise is less than the intensity of the second sample background noise, the second sample speech data is determined as sample independent speech data.

[0138] The benefit of the above embodiment is that by distinguishing between the same background noise state and different background noise states, different methods of determining the sample individual voice data from the two sample voice data are set, thereby ensuring that the sample individual voice data is the sample voice data with stronger background noise (at least not weaker than the other sample voice data), and reducing the processing steps such as calculating the noise intensity under the same background noise state, thereby saving processing resources.

[0139] In one embodiment, referring to Figure 6 , step 404 may include:

[0140] Step 601: Perform speaker recognition on the sample individual speech data to obtain a first speaker, and perform speaker recognition on the sample synthesized speech data to obtain a second speaker;

[0141] Step 602 , performing speaker difference calculation based on the first speaker and the second speaker to obtain speaker verification loss data;

[0142] Step 603: performing spectrum recognition on the sample individual speech data to obtain a first speech spectrum, and performing spectrum recognition on the sample synthesized speech data to obtain a second speech spectrum;

[0143] Step 604 , calculating a spectrum error based on the first speech spectrum and the second speech spectrum to obtain speaker reconstruction loss data;

[0144] Step 605 : Perform data fusion based on the speaker verification loss data and the speaker reconstruction loss data to obtain speaker loss data.

[0145] Specifically, speaker recognition can be performed using a speaker recognition model, such as ERes2Net. Speaker differences can be calculated using a cross-entropy loss function. Spectral recognition can be performed using a spectrum recognition model or spectrum recognition algorithm. Spectral error can be calculated using the L2 norm. Data fusion can be performed using a weighted average function or other data fusion formulas.

[0146] The benefit of the embodiment of steps 601 to 605 is that speaker loss data is obtained by jointly constructing speaker verification loss data and speaker reconstruction loss data, thereby improving the accuracy of model updating.

[0147] In step 106 of some embodiments, target speech data is acquired, and speaker features are extracted from the target speech data using a feature embedder in a target speaker verification model to obtain target speaker features.

[0148] In one embodiment, the feature embedder includes a mel spectrum enhancement layer, a convolution layer, and a pooling layer. Step 106 may include:

[0149] Through the Mel spectrum enhancement layer, the target speech data is enhanced by Mel spectrum to obtain the target enhanced speaker Mel spectrum features;

[0150] Perform feature convolution on the target enhanced speaker Mel spectrum features through the convolution layer to obtain the initial speaker features;

[0151] The initial speaker features are aggregated through the pooling layer to obtain the target speaker features.

[0152] The benefit of this embodiment is that speaker features can be extracted with high accuracy even when the target speech data contains background noise.

[0153] In step 107 of some embodiments, speech generation is performed based on the target speaker characteristics and preset target text features to obtain target synthesized speech data. The target text features are obtained by obtaining the target text and encoding the target text using a text encoder to obtain the target text features. The text encoder can use a Transformer-based encoder to generate text embeddings in combination with contextual information. The text encoder uses position embedding and a semantic attention mechanism to capture the intonation, stress, and pause information required for speech generation.

[0154] In one embodiment, step 107 may include: constructing a speech generation model based on a generative framework that integrates variational inference and a generative adversarial network; performing speech generation on the target speaker features and preset target text features using the speech generation model to obtain target synthesized speech data. Specifically, the generative framework that integrates variational inference and a generative adversarial network is a VITS (Variational Inference Text-to-Speech) architecture, which generates speech waveforms through variational inference. The speech generation model includes: a speaker-conditioned generator for introducing speaker embedding during the generation process to ensure that the timbre of the speech is consistent with that of the target speaker; and a dynamic denoising module: removing artifacts and noise residues during the generation phase through speech feature enhancement and filtering mechanisms.

[0155] In summary, the present application can achieve at least the following beneficial effects: (1) Significantly improve noise robustness: Through the self-supervised speaker verification loss and denoising mechanism, it is possible to extract robust speaker features in low-quality or noisy environments and generate clear and natural speech. (2) High data efficiency: The self-supervised learning method is used to reduce the dependence on labeled data, and high-quality speech generation can also be achieved in scenarios with limited resources. (3) Strong timbre matching: The speaker verification loss is introduced to ensure that the generated speech is consistent with the timbre of the target speaker, making it suitable for personalized speech synthesis needs. (4) Improved speech naturalness: Combined with the VITS framework and dynamic denoising mechanism, the sound quality of the generated speech is more natural and smooth, and there are no artifacts. (5) Wide range of application scenarios: It can be applied to scenarios such as virtual assistants, personalized speech synthesis, and barrier-free communication in financial technology scenarios, and is particularly suitable for speech generation needs in low-resource or noisy environments.

[0156] See also Figure 7 The embodiment of the present application also provides a speech synthesis device, which can implement the above-mentioned speech synthesis method. Figure 7This is a module structure block diagram of the speech synthesis device provided in an embodiment of the present application, which includes: a sample data acquisition module 701, a sample feature extraction module 702, a speaker classification module 703, a contrast loss calculation module 704, a model parameter adjustment module 705, a target data acquisition module 706 and a speech generation module 707. Among them, the sample data acquisition module 701 is used to obtain a sample speech data group; wherein the sample speech data group includes two sample speech data; the sample feature extraction module 702 is used to extract speaker features of the two sample speech data of the sample speech data group using a feature embedder in a preset initial speaker verification model to obtain speaker features of the sample group; the speaker classification module 703 is used to perform speaker classification on the speaker features of the sample group using a feature classifier in the initial speaker verification model to obtain a sample verification speaker category of the sample speech data group; the contrastive loss calculation module 704 is used to calculate contrastive learning loss based on the sample verification speaker category to obtain speaker contrastive loss data; the model parameter adjustment module 705 is used to adjust the parameters of the initial speaker verification model based on the speaker contrastive loss data to obtain a target speaker verification model; the target data acquisition module 706 is used to obtain target speech data, and extract speaker features of the target speech data using a feature embedder in the target speaker verification model to obtain target speaker features; and the speech generation module 707 is used to generate speech based on the target speaker features and preset target text features to obtain target synthesized speech data.

[0157] In one embodiment, after adjusting the parameters of the initial speaker verification model according to the speaker contrast loss data to obtain the target speaker verification model, the speech synthesis device further includes: a parameter secondary updating module for updating the target speaker verification model according to the sample speech data group.

[0158] In one embodiment, before performing speaker feature extraction on two sample speech data of a sample speech data group by a feature embedder in a preset initial speaker verification model to obtain speaker features of the sample group, the speech synthesis device further includes: an embedder adjustment module, which is used to: determine the two sample speech data as first sample speech data and second sample speech data, respectively; perform speaker feature extraction on the first sample speech data by the feature embedder to obtain a first speaker feature, and perform speaker feature extraction on the second sample speech data by the feature embedder to obtain a second speaker feature; perform noise recognition on the first speaker feature to obtain a first noise feature, and perform noise recognition on the second speaker feature to obtain a second noise feature; perform denoising degree evaluation based on the first noise feature and the second noise feature to obtain a denoising score of the feature embedder; and adjust the feature embedder based on the denoising score.

[0159] It should be noted that the specific implementation of the voice synthesis device is basically the same as the specific embodiment of the voice synthesis method described above, and will not be repeated here.

[0160] The embodiments of the present application further provide an electronic device, which comprises a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing the connection communication between the processor and the memory, and the program is executed by the processor to realize the voice synthesis method. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0161] Please refer to Figure 8 , Figure 8 The hardware structure of the electronic device of another embodiment is illustrated, which comprises:

[0162] The processor 801 can be implemented in the form of a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to realize the technical solutions provided by the embodiments of the present application.

[0163] The memory 802 can be implemented in the form of a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 802 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are saved in the memory 802 and called and executed by the processor 801 to realize the voice synthesis method of the embodiments of the present application.

[0164] The input / output interface 803 is used to realize information input and output.

[0165] The communication interface 804 is used to realize the communication interaction between the device and other devices, which can realize communication through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.).

[0166] The bus 805 transmits information between various components (such as the processor 801, the memory 802, the input / output interface 803, and the communication interface 804) of the device.

[0167] The processor 801, the memory 802, the input / output interface 803, and the communication interface 804 realize the communication connection between each other inside the device through the bus 805.

[0168] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned speech synthesis method.

[0169] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0170] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0171] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0172] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0173] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0174] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0175] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0176] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0177] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0178] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0179] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes multiple instructions for enabling an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0180] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A speech synthesis method, characterized in that: The method comprises: Acquire a sample voice data group; wherein the sample voice data group includes two sample voice data; Determine the two sample voice data as first sample voice data and second sample voice data respectively; Performing speaker feature extraction on the first sample speech data using a feature embedder in a preset initial speaker verification model to obtain a first speaker feature, and performing speaker feature extraction on the second sample speech data using the feature embedder to obtain a second speaker feature; performing noise recognition on the first speaker feature to obtain a first noise feature, and performing noise recognition on the second speaker feature to obtain a second noise feature; performing a denoising degree evaluation based on the first noise feature and the second noise feature to obtain a denoising score of the feature embedder; If the denoising score is greater than or equal to a preset denoising score threshold, adjusting parameters of the feature embedder according to the denoising score; If the denoising score is less than the denoising score threshold, adding a denoising layer to the feature embedder to adjust the structure of the feature embedder; Performing speaker feature extraction on the two sample speech data of the sample speech data group by the feature embedder to obtain speaker features of the sample group; Performing speaker classification on the speaker features of the sample group by using the feature classifier in the initial speaker verification model to obtain a sample verification speaker category of the sample speech data group; wherein the sample verification speaker category indicates whether the speaker to which one sample speech data in the sample speech data group belongs is the same as or different from the speaker to which another sample speech data belongs; Computing a contrastive learning loss based on the sample verification speaker category to obtain speaker contrastive loss data; Adjusting parameters of the initial speaker verification model according to the speaker contrast loss data to obtain a target speaker verification model; Acquire target speech data, and perform speaker feature extraction on the target speech data using a feature embedder in the target speaker verification model to obtain target speaker features; Speech generation is performed based on the target speaker characteristics and preset target text characteristics to obtain target synthesized speech data.

2. The method according to claim 1, characterized in that After adjusting parameters of the initial speaker verification model according to the speaker contrast loss data to obtain a target speaker verification model, the method further includes: Updating the target speaker verification model according to the sample speech data group specifically includes: determining one of the two sample voice data of the sample voice data group as sample individual voice data; Performing speaker feature extraction on the sample individual speech data using a feature embedder in the target speaker verification model to obtain sample individual speaker features; Performing speech synthesis based on the sample individual speaker characteristics and preset sample text characteristics to obtain sample synthesized speech data; performing speaker loss calculation based on the sample individual speech data and the sample synthesized speech data to obtain speaker loss data; The target speaker verification model is updated according to the speaker loss data.

3. The method according to claim 2, characterized in that The performing speaker loss calculation based on the sample individual speech data and the sample synthesized speech data to obtain speaker loss data includes: Performing speaker recognition on the sample separate speech data to obtain a first speaker, and performing speaker recognition on the sample synthesized speech data to obtain a second speaker; performing speaker difference calculation based on the first speaker and the second speaker to obtain speaker verification loss data; Performing spectrum recognition on the sample independent speech data to obtain a first speech spectrum, and performing spectrum recognition on the sample synthesized speech data to obtain a second speech spectrum; Calculating a spectrum error based on the first speech spectrum and the second speech spectrum to obtain speaker reconstruction loss data; Data fusion is performed according to the speaker verification loss data and the speaker reconstruction loss data to obtain the speaker loss data.

4. The method according to claim 2, characterized in that The step of determining one of the two sample voice data in the sample voice data group as sample independent voice data includes: Determine two of the sample voice data in the sample voice data group as first sample voice data and second sample voice data respectively; If the first sample voice data and the second sample voice data are voice data generated under the same background noise state, determining the first sample voice data or the second sample voice data as the sample independent voice data; If the first sample voice data and the second sample voice data are voice data generated under different background noise conditions, obtaining the background noise contained in the first sample voice data to obtain a first sample background noise, and obtaining the background noise contained in the second sample voice data to obtain a second sample background noise; Performing a noise comparison between the first sample background noise and the second sample background noise to obtain a noise comparison result; If the intensity of the first sample background noise is greater than or equal to the intensity of the second sample background noise, determining the first sample voice data as the sample independent voice data; If the intensity of the first sample background noise is less than the intensity of the second sample background noise, the second sample voice data is determined as the sample independent voice data.

5. The method according to any one of claims 1 to 4, characterized in that The performing denoising degree evaluation according to the first noise feature and the second noise feature to obtain a denoising score of the feature embedder includes: Determining a first feature denoising sub-score based on the strength of the first noise feature, and determining a second feature denoising sub-score based on the strength of the second noise feature; If the first sample speech data and the second sample speech data are speech data generated under the same background noise state, performing feature difference analysis on the first noise feature and the second noise feature to obtain a stable denoising sub-score; The denoising score is obtained by performing score aggregation based on the first feature denoising sub-score, the second feature denoising sub-score, and the stable denoising sub-score.

6. A speech synthesis device, characterized in that: The device comprises: A sample data acquisition module is used to acquire a sample voice data group; wherein the sample voice data group includes two sample voice data; an embedder adjustment module, configured to: determine the two sample speech data as first sample speech data and second sample speech data, respectively; perform speaker feature extraction on the first sample speech data using a feature embedder in a preset initial speaker verification model to obtain a first speaker feature, and perform speaker feature extraction on the second sample speech data using the feature embedder to obtain a second speaker feature; perform noise identification on the first speaker feature to obtain a first noise feature, and perform noise identification on the second speaker feature to obtain a second noise feature; perform denoising evaluation based on the first noise feature and the second noise feature to obtain a denoising score of the feature embedder; if the denoising score is greater than or equal to a preset denoising score threshold, adjust parameters of the feature embedder based on the denoising score; if the denoising score is less than the denoising score threshold, add a denoising layer to the feature embedder to adjust the structure of the feature embedder; a sample feature extraction module, configured to extract speaker features from the two sample speech data of the sample speech data group through the feature embedder to obtain speaker features of the sample group; a speaker classification module, configured to perform speaker classification on the speaker features of the sample group using a feature classifier in the initial speaker verification model to obtain a sample verification speaker category of the sample speech data group; wherein the sample verification speaker category indicates whether the speaker to which one sample speech data in the sample speech data group belongs is the same as or different from the speaker to which another sample speech data belongs; a contrastive loss calculation module, configured to perform contrastive learning loss calculation based on the sample to verify the speaker category and obtain speaker contrastive loss data; a model parameter adjustment module, configured to adjust parameters of the initial speaker verification model according to the speaker contrast loss data to obtain a target speaker verification model; a target data acquisition module, configured to acquire target speech data, and perform speaker feature extraction on the target speech data using a feature embedder in the target speaker verification model to obtain target speaker features; The speech generation module is used to generate speech according to the target speaker characteristics and preset target text characteristics to obtain target synthesized speech data.

7. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the speech synthesis method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the speech synthesis method according to any one of claims 1 to 5 is implemented.

Images