Speech synthesis method, speech synthesis device, electronic equipment and storage medium

Abstract

The invention provides a speech synthesis method, a speech synthesis device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence and the field of financial science and technology. The method comprises the following steps: performing speaker feature extraction on a sample voice data group through a feature embedder in an initial speaker verification model to obtain a sample group speaker feature; performing speaker classification on the speaker features of the sample group through a feature classifier in the initial speaker verification model to obtain a sample verification speaker category, and then performing parameter adjustment on the initial speaker verification model to obtain a target speaker verification model; speaker feature extraction is carried out on the target voice data through a feature embedder in a target speaker verification model, and target speaker features are obtained; and performing voice generation according to the target speaker features and the target text features to obtain target synthetic voice data. According to the invention, adverse effects caused by background noise in the voice data can be relieved, and the accuracy of voice synthesis is improved.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence technology and financial technology, and in particular to a speech synthesis method, a speech synthesis device, an electronic device and a storage medium. Background Art

[0002] Current zero-shot speech synthesis (Zero-Shot TTS) technology can generate realistic speech using limited target speaker data. However, most existing zero-shot TTS systems require high-quality, noise-free target speaker data to effectively extract speaker features. This is difficult for resource-constrained scenarios (such as financial service scenarios). For example, in insurance systems or banking systems in financial service scenarios, target speakers (such as customer service) are usually recorded in a noisy environment, and the recorded speech data contains background noise. Background noise affects the extraction of speaker features from speech data, resulting in poor accuracy of speaker features, which in turn leads to poor accuracy of speech synthesis. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a speech synthesis method, a speech synthesis device, an electronic device and a storage medium, which can alleviate the adverse effects of background noise in speech data and improve the accuracy of speech synthesis.

[0004] To achieve the above object, a first aspect of an embodiment of the present application provides a speech synthesis method, the method comprising:

[0005] Acquire a sample voice data group; wherein the sample voice data group includes two sample voice data;

[0006] Performing speaker feature extraction on two sample speech data of the sample speech data group through a feature embedder in a preset initial speaker verification model to obtain speaker features of the sample group;

[0007] Performing speaker classification on the speaker features of the sample group by the feature classifier in the initial speaker verification model to obtain a sample verification speaker category of the sample speech data group; wherein the sample verification speaker category indicates whether the speaker to which one sample speech data of the sample speech data group belongs is the same as or different from the speaker to which another sample speech data belongs;

[0008] Comparative learning loss calculation is performed based on the sample verification speaker category to obtain speaker contrast loss data;

[0009] Adjusting parameters of the initial speaker verification model according to the speaker contrast loss data to obtain a target speaker verification model;

[0010] Acquire target speech data, and perform speaker feature extraction on the target speech data through a feature embedder in the target speaker verification model to obtain target speaker features;

[0011] Speech generation is performed according to the target speaker characteristics and preset target text characteristics to obtain target synthesized speech data.

[0012] Optionally, after adjusting parameters of the initial speaker verification model according to the speaker contrast loss data to obtain a target speaker verification model, the method further includes:

[0013] Updating the target speaker verification model according to the sample speech data group specifically includes:

[0014] Determine one of the two sample voice data of the sample voice data group as sample individual voice data;

[0015] Extracting speaker features from the sample individual speech data by using a feature embedder in the target speaker verification model to obtain sample individual speaker features;

[0016] Performing speech synthesis according to the sample individual speaker characteristics and preset sample text characteristics to obtain sample synthesized speech data;

[0017] Calculating speaker loss based on the sample individual speech data and the sample synthesized speech data to obtain speaker loss data;

[0018] The target speaker verification model is updated according to the speaker loss data.

[0019] Optionally, the performing speaker loss calculation according to the sample individual speech data and the sample synthesized speech data to obtain speaker loss data includes:

[0020] Performing speaker recognition on the sample separate speech data to obtain a first speaker, and performing speaker recognition on the sample synthesized speech data to obtain a second speaker;

[0021] Calculating a speaker difference between the first speaker and the second speaker to obtain speaker verification loss data;

[0022] Performing spectrum recognition on the sample separate speech data to obtain a first speech spectrum, and performing spectrum recognition on the sample synthesized speech data to obtain a second speech spectrum;

[0023] Calculate spectrum error based on the first speech spectrum and the second speech spectrum to obtain speaker reconstruction loss data;

[0024] Data fusion is performed according to the speaker verification loss data and the speaker reconstruction loss data to obtain the speaker loss data.

[0025] Optionally, determining one of the two sample voice data in the sample voice data group as sample independent voice data includes:

[0026] Determine two of the sample voice data in the sample voice data group as first sample voice data and second sample voice data respectively;

[0027] If the first sample voice data and the second sample voice data are voice data generated under the same background noise state, determining the first sample voice data or the second sample voice data as the sample independent voice data;

[0028] If the first sample voice data and the second sample voice data are voice data generated under different background noise conditions, obtaining the background noise contained in the first sample voice data to obtain the first sample background noise, and obtaining the background noise contained in the second sample voice data to obtain the second sample background noise;

[0029] Performing a noise comparison on the first sample background noise and the second sample background noise to obtain a noise comparison result;

[0030] If the intensity of the first sample background noise is greater than or equal to the intensity of the second sample background noise, determining the first sample voice data as the sample separate voice data;

[0031] If the intensity of the first sample background noise is less than the intensity of the second sample background noise, the second sample voice data is determined as the sample independent voice data.

[0032] Optionally, before extracting speaker features from the two sample speech data of the sample speech data group by a feature embedder in a preset initial speaker verification model to obtain speaker features of the sample group, the method further comprises:

[0033] Determine the two sample voice data as first sample voice data and second sample voice data respectively;

[0034] Performing speaker feature extraction on the first sample speech data by the feature embedder to obtain a first speaker feature, and performing speaker feature extraction on the second sample speech data by the feature embedder to obtain a second speaker feature;

[0035] Performing noise recognition on the first speaker feature to obtain a first noise feature, and performing noise recognition on the second speaker feature to obtain a second noise feature;

[0036] Performing a denoising degree evaluation based on the first noise feature and the second noise feature to obtain a denoising score of the feature embedder;

[0037] The feature embedder is adjusted according to the denoising score.

[0038] Optionally, the performing denoising degree evaluation according to the first noise feature and the second noise feature to obtain a denoising score of the feature embedder includes:

[0039] Determining a first feature denoising sub-score based on the strength of the first noise feature, and determining a second feature denoising sub-score based on the strength of the second noise feature;

[0040] If the first sample speech data and the second sample speech data are speech data generated under the same background noise state, performing feature difference analysis on the first noise feature and the second noise feature to obtain a stable denoising sub-score;

[0041] The denoising score is obtained by performing score aggregation according to the first feature denoising sub-score, the second feature denoising sub-score and the stable denoising sub-score.

[0042] Optionally, adjusting the feature embedder according to the denoising score comprises:

[0043] If the denoising score is greater than or equal to a preset denoising score threshold, adjusting parameters of the feature embedder according to the denoising score;

[0044] If the denoising score is less than the denoising score threshold, a denoising layer is added to the feature embedder to perform structural adjustment on the feature embedder.

[0045] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a speech synthesis device, the device comprising:

[0046] A sample data acquisition module, used to acquire a sample voice data group; wherein the sample voice data group includes two sample voice data;

[0047] A sample feature extraction module, used for extracting speaker features from two sample speech data of the sample speech data group through a feature embedder in a preset initial speaker verification model to obtain speaker features of the sample group;

[0048] A speaker classification module, configured to perform speaker classification on the speaker features of the sample group by using the feature classifier in the initial speaker verification model to obtain a sample verification speaker category of the sample speech data group; wherein the sample verification speaker category indicates whether the speaker to which one sample speech data of the sample speech data group belongs is the same as or different from the speaker to which another sample speech data belongs;

[0049] A contrast loss calculation module, used to perform contrast learning loss calculation based on the sample to verify the speaker category and obtain speaker contrast loss data;

[0050] A model parameter adjustment module, used to adjust the parameters of the initial speaker verification model according to the speaker contrast loss data to obtain a target speaker verification model;

[0051] A target data acquisition module is used to acquire target speech data, and extract speaker features from the target speech data through a feature embedder in the target speaker verification model to obtain target speaker features;

[0052] The speech generation module is used to generate speech according to the target speaker characteristics and preset target text characteristics to obtain target synthesized speech data.

[0053] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the speech synthesis method described in the first aspect when executing the computer program.

[0054] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program. When the computer program is executed by a processor, the speech synthesis method described in the first aspect is implemented.

[0055] The speech synthesis method, speech synthesis device, electronic device and storage medium proposed in the present application first obtain a sample speech data group, which includes two sample speech data, and the two sample speech data may belong to the same speaker or different speakers. Further, the speaker features of the two sample speech data are extracted by the feature embedder in the initial speaker verification model to obtain the speaker features of the sample group, so that the sample speaker features can indicate the speakers corresponding to the two sample speech data. Further, the speaker features of the sample group are classified by the feature classifier in the initial speaker verification model to obtain the sample verification speaker category of the sample speech data group, so that the accuracy of the speaker feature extraction by the feature embedder can be evaluated by the sample verification speaker category. Further, the contrast learning loss is calculated according to the sample verification speaker category to obtain the speaker contrast loss data, and then the parameters of the initial speaker verification model are adjusted according to the speaker contrast loss data to obtain the target speaker verification model, so that the feature embedder of the target speaker verification model can distinguish between speaker features and noise features, and can accurately characterize the target speaker under noise conditions. In actual use, the target speech data is obtained, and the speaker features of the target speech data are extracted through the feature embedder in the target speaker verification model to obtain the target speaker features, and then speech generation is performed according to the target speaker features and the preset target text features to obtain the target synthesized speech data. In this way, the background noise in the target speech data can be effectively removed by the feature embedder in the target speaker verification model, and the output target speaker features can accurately characterize the target speaker corresponding to the target speech data, so that the generated target synthesized speech data is more real and realistic. In summary, the present application can alleviate the adverse effects of background noise in speech data and improve the accuracy of speech synthesis. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flow chart of a speech synthesis method provided in an embodiment of the present application;

[0057] Figure 2 is a flow chart of a speech synthesis method provided by another embodiment of the present application;

[0058] Figure 3 yes Figure 2 Flowchart of step 204 in;

[0059] Figure 4 is a flow chart of a speech synthesis method provided by another embodiment of the present application;

[0060] Figure 5 yes Figure 4 Flow chart of step 401 in;

[0061] Figure 6 yes Figure 4 Flowchart of step 404 in;

[0062] Figure 7 is a block diagram of the module structure of the speech synthesis device provided in an embodiment of the present application;

[0063] Figure 8 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0065] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0067] First, some nouns involved in this application are analyzed:

[0068] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0069] Natural language processing (NLP): NLP uses computers to process, understand and apply human languages ​​(such as Chinese, English, etc.). NLP is a branch of artificial intelligence and an interdisciplinary subject between computer science and linguistics. It is often referred to as computational linguistics. Natural language processing includes grammatical analysis, semantic analysis, and text comprehension. Natural language processing is commonly used in technical fields such as machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information image processing, information extraction and filtering, text classification and clustering, public opinion analysis and opinion mining. It involves data mining related to language processing, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computing.

[0070] Zero-Shot TTS is a technology for synthesizing speech that can generate the speech of any speaker without specific speech data. This technology uses deep learning and neural networks, and by training on data sets of multiple speakers, it can generate output in the speech style of a specific speaker after inputting text. Zero-Shot TTS does not need to collect independent speech data for each speaker, which greatly improves synthesis efficiency and flexibility.

[0071] At present, insurance systems, banking systems, etc. in financial service scenarios are becoming more and more automated and intelligent. Among them, speech synthesis technology is particularly widely used in financial service scenarios. Speech synthesis aims to extract speaker features from speaker voice data, and then generate target synthesized speech based on the speaker features and text features of the text without the speaker speaking the complete text. Current zero-shot speech synthesis technology can generate realistic speech using limited speaker data. However, these technologies still have the following deficiencies in data efficiency and noise robustness: 1. High demand for speaker data: Most existing zero-shot TTS systems require high-quality, noise-free speaker data to effectively extract speaker features. 2. Speaker feature extraction is not robust: Many models lack robustness when embedding speakers, especially under noisy conditions, the extracted features may be contaminated, resulting in decreased speaker consistency in the generated speech. 3. Lack of efficient self-supervision mechanism: Traditional TTS systems require large-scale labeled data for training, while current zero-shot TTS models have limited optimization of data efficiency and cannot fully utilize the potential information in unlabeled data. 4. Inadequate noise processing capabilities: In practical applications, speaker data may contain environmental noise. Existing systems lack effective processing of these noises, resulting in unnecessary noise artifacts in the generated speech. 5. Poor naturalness and clarity of synthesized speech: Existing methods often have difficulty in balancing the clarity and coherence of speech while maintaining the naturalness of speech and the timbre of the target speaker, especially in the presence of background noise.

[0072] Based on this, the embodiments of the present application propose a speech synthesis method, a speech synthesis device, an electronic device and a computer-readable storage medium, which can alleviate the adverse effects of background noise in speech data and improve the accuracy of speech synthesis.

[0073] The speech synthesis method provided in the embodiment of the present application can be applied to the terminal and the server side, and can also be software running on the server side. The server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the speech synthesis method, etc., but is not limited to the above forms.

[0074] The present application can be used in many general or special computer system environments or configurations. For example: server computers, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0075] The embodiments of the present application provide a speech synthesis method, a speech synthesis device, an electronic device and a computer-readable storage medium, which are specifically described through the following embodiments. First, the speech synthesis method in the embodiments of the present application is described.

[0076] It should be noted that in each specific implementation method of the present application, when it comes to performing relevant processing based on the user's voice data or other data related to the user's identity or characteristics, the user's permission or consent must be obtained first, and the collection, use and processing of such data will comply with relevant laws, regulations and standards.

[0077] Reference Figure 1 , Figure 1 It is an optional flowchart of the speech synthesis method provided in an embodiment of the present application, which may include but is not limited to steps 101 to 107.

[0078] Step 101, obtaining a sample voice data group; wherein the sample voice data group includes two sample voice data;

[0079] Step 102, extracting speaker features from two sample speech data of the sample speech data group by using a feature embedder in a preset initial speaker verification model to obtain speaker features of the sample group;

[0080] Step 103, performing speaker classification on the speaker features of the sample group by using the feature classifier in the initial speaker verification model to obtain a sample verification speaker category of the sample speech data group;

[0081] Step 104, performing contrastive learning loss calculation based on sample verification speaker categories to obtain speaker contrastive loss data;

[0082] Step 105, adjusting parameters of the initial speaker verification model according to the speaker contrast loss data to obtain a target speaker verification model;

[0083] Step 106, obtaining target speech data, and extracting speaker features from the target speech data using a feature embedder in a target speaker verification model to obtain target speaker features;

[0084] Step 107, generating speech according to the target speaker characteristics and the preset target text characteristics to obtain target synthesized speech data.

[0085] In the steps 101 to 107 shown in the embodiment of the present application, speaker features are first extracted from two sample speech data by the feature embedder in the initial speaker verification model to obtain speaker features of the sample group, so that the sample speaker features can indicate the speakers corresponding to the two sample speech data. Further, the sample verification speaker category is output by the feature classifier in the initial speaker verification model, so that the accuracy of speaker feature extraction by the feature embedder can be evaluated by the sample verification speaker category. Further, speaker contrast loss data is calculated according to the sample verification speaker category, and then the parameters of the initial speaker verification model are adjusted according to the speaker contrast loss data to obtain the target speaker verification model, so that the feature embedder of the target speaker verification model can distinguish between speaker features and noise features, and can accurately characterize the target speaker under noise conditions. In actual use, speaker features are extracted from the target speech data by the feature embedder in the target speaker verification model to obtain target speaker features, and then speech generation is performed according to the target speaker features and preset target text features to obtain target synthetic speech data. In this way, the background noise in the target speech data can be effectively removed by the feature embedder, and the output target speaker features can accurately characterize the target speaker corresponding to the target speech data, so that the generated target synthesized speech data is more real and realistic. In summary, the present application can alleviate the adverse effects of background noise in speech data and improve the accuracy of speech synthesis.

[0086] When the embodiment of the present application is applied to the customer service, online consultation and other scenarios in the insurance system or banking system in the financial service scenario, only the customer service or the inquiry object needs to output a section of speaker data, and the target synthesized voice data can be generated based on the speaker data. The speaker data is the same as the speaker of the target synthesized voice data, but the speech content is different. In this way, it can not only improve the accuracy of speaker feature extraction for customer service or inquiry objects, but also reduce the adverse effects of the length or difficulty of the speech content on customer service or inquiry objects, improve service efficiency, and enhance user experience. For example, for the customer service scenario, when launching a new service (such as a pet insurance product), there is no need for the customer service to enter the complete content corresponding to the new service, but to arbitrarily enter a section of customer service voice data, and then extract the speaker feature of the customer service voice data through the feature embedder in the target speaker verification model to obtain the customer service speaker feature, and then generate voice together with the text feature of the content corresponding to the new service to obtain the target synthesized voice data. In this way, the target synthesized voice data in the financial service scenario does not need to be recorded by the customer service according to the complete content, and is not affected by the background noise in the customer service voice data, and can accurately retain the speaker features of the customer service, thereby improving the accuracy of speech synthesis.

[0087] When the embodiments of the present application are applied to digital medical, medical health and other scenarios in medical scenarios, only the customer service or the inquiry object (such as a patient) needs to output a piece of speaker data, and the target synthesized speech data can be generated based on the speaker data. The speaker data is the same as the speaker of the target synthesized speech data, but the speech content is different. For example, for digital medical scenarios, if the doctor's office changes, the customer service does not need to enter the complete office name and office location, but arbitrarily enters a piece of customer service speech data, and then uses the feature embedder in the target speaker verification model to extract speaker features from the customer service speech data to obtain customer service speaker features, and then generates speech together with the text features representing the office name and office location to obtain the target synthesized speech data. In this way, the target synthesized speech data in the medical scenario does not need to be recorded by the customer service based on the complete content, and is not affected by the background noise in the customer service speech data. It can accurately retain the speaker features of the customer service and improve the accuracy of speech synthesis.

[0088] In step 101 of some embodiments, a sample speech data set is obtained. The sample speech data set includes two sample speech data. One of the two sample speech data is used as the first sample speech data, and the other sample speech data is used as the second sample speech data. The speaker of the first sample speech data and the speaker of the second sample speech data may be the same or different. The generalization ability of the model can also be improved by adding artificial noise and diversifying the sample speech data.

[0089] In one embodiment, if the speaker of the first sample voice data is the same as the speaker of the second sample voice data, the speech content of the first sample voice data is different from the speech content of the second sample voice data.

[0090] In one example, when the speech synthesis method is applied to a terminal, the sample speech data group can be obtained by recording, Bluetooth transmission, wired transmission, or downloading. When the sample speech data group is obtained by recording, the terminal is correspondingly configured with a microphone, and the microphone is used to collect audio to achieve the recording of the speech sample data group. When the speech synthesis method is applied to a server, the sample speech data group can be uploaded to the server by the terminal, or downloaded by the server from other servers or databases.

[0091] In one embodiment, referring to Figure 2 Before step 102, the speech synthesis method may further include:

[0092] Step 201, determining two sample voice data as first sample voice data and second sample voice data respectively;

[0093] Step 202, extracting speaker features from the first sample speech data through a feature embedder to obtain a first speaker feature, and extracting speaker features from the second sample speech data through a feature embedder to obtain a second speaker feature;

[0094] Step 203, performing noise recognition on the first speaker feature to obtain a first noise feature, and performing noise recognition on the second speaker feature to obtain a second noise feature;

[0095] Step 204, evaluating the denoising degree according to the first noise feature and the second noise feature to obtain a denoising score of the feature embedder;

[0096] Step 205, adjusting the feature embedder according to the denoising score.

[0097] In step 203, noise recognition can be performed using a noise recognition model. A noise recognition model refers to a model used to recognize noise. Common noise recognition models include deep learning models, traditional machine learning models, and adaptive filters.

[0098] In step 204, the stronger the strength of the first noise feature, the more noise there is in the first speaker feature, and thus the lower the denoising degree is. The weaker the strength of the first noise feature, the less noise there is in the first speaker feature, and thus the higher the denoising degree is. The stronger the strength of the second noise feature, the more noise there is in the second speaker feature, and thus the lower the denoising degree is. The weaker the strength of the second noise feature, the less noise there is in the second speaker feature, and thus the higher the denoising degree is. The denoising degree can be evaluated by a scoring function or a scoring model.

[0099] In one embodiment, referring to Figure 3 , step 204 may include:

[0100] Step 301, determining a first feature denoising sub-score according to the strength of the first noise feature, and determining a second feature denoising sub-score according to the strength of the second noise feature;

[0101] Step 302, if the first sample speech data and the second sample speech data are speech data generated under the same background noise state, then feature difference analysis is performed on the first noise feature and the second noise feature to obtain a stable denoising sub-score;

[0102] Step 303 , performing score aggregation according to the first feature denoising sub-score, the second feature denoising sub-score and the stable denoising sub-score to obtain a denoising score.

[0103] In step 301, the higher the intensity of the first noise feature, the higher the first feature denoising sub-score. The weaker the intensity of the first noise feature, the lower the first feature denoising sub-score. The higher the intensity of the second noise feature, the higher the second feature denoising sub-score. The weaker the intensity of the second noise feature, the lower the second feature denoising sub-score. An intensity score mapping function / mapping table can be set to obtain the first feature denoising sub-score and the second feature denoising sub-score according to the table lookup. The intensity of the first noise feature is generally reflected in the amplitude of the noise spectrum or in the amplitude of the noise power spectrum. In daily life, the stronger the noise, the noisier it is.

[0104] In step 302, if the first sample speech data and the second sample speech data are speech data generated under the same background noise state, it means that the background noise contained in the first sample speech data and the second sample speech data is basically the same. In this case, the first noise feature and the second noise feature should be basically the same, otherwise it means that the performance of the feature embedder is unstable and the speaker feature extraction cannot be performed stably. The first noise feature and the second noise feature can be analyzed for feature differences through a deep learning model (such as a CNN model) to obtain a stable denoising sub-score. The higher the stable denoising sub-score, the higher the stability of the feature embedder, and the lower the stable denoising sub-score, the lower the stability of the feature embedder.

[0105] In step 303, the first feature denoising sub-score and the second feature denoising sub-score are proportional to the denoising score, and the stable denoising sub-score is inversely proportional to the denoising score. The denoising score can be obtained by calculating a weighted average or other methods to achieve score aggregation.

[0106] The benefit of the embodiment of the above steps 301 to 303 is that a stable denoising sub-score is introduced to improve the accuracy of determining the denoising score, thereby improving the accuracy of evaluating the performance of the feature embedder.

[0107] In one embodiment, before step 303, step 204 may further include: if the first sample speech data and the second sample speech data are speech data generated under different background noise conditions, generating a stable denoising sub-score with a score of 0. In this way, it is compatible with the situation where two speech data are generated under different background noise conditions, thereby improving applicability.

[0108] In step 205, a higher denoising score indicates a stronger denoising capability of the feature embedder, and thus a smaller adjustment is made to the feature embedder. A lower denoising score indicates a weaker denoising capability of the feature embedder, and thus a larger adjustment is made to the feature embedder.

[0109] The embodiment of the above steps 201 to 205 is that before using the feature embedder to output the speaker features of the sample group, the feature embedder can be adjusted according to the denoising score to improve the accuracy of the speaker features of the sample group, thereby reducing the number of times the parameters of the feature embedder are adjusted, saving processing resources and improving speech synthesis efficiency.

[0110] In one embodiment, step 205 may include: if the denoising score is greater than or equal to a preset denoising score threshold, adjusting parameters of the feature embedder according to the denoising score; if the denoising score is less than the denoising score threshold, adding a denoising layer to the feature embedder to adjust the structure of the feature embedder.

[0111] The benefit of the above embodiment is that by selecting whether to adjust the parameters or the structure of the feature embedder through the denoising score, the adjustment flexibility can be improved and the denoising performance of the feature embedder can be improved.

[0112] In step 102 of some embodiments, speaker features are extracted from two sample speech data of the sample speech data group by a feature embedder in a preset initial speaker verification model to obtain speaker features of the sample group.

[0113] The initial speaker verification model is a neural network model used to verify whether the speakers of two speech data are the same, such as a CAM model, a CAM++ model, etc. The initial speaker verification model includes a feature embedder and a feature classifier. The feature embedder is used to perform deep feature embedding on the speech data. The feature classifier is used to perform speaker classification on the output of the feature embedder. In step 102, the speaker features of the two sample speech data are extracted mainly by using the feature embedding capability of the feature embedder to obtain the speaker features of the sample group, which can well characterize the speakers corresponding to the two sample speech data.

[0114] In one embodiment, the feature embedder includes a Mel spectrum enhancement layer, a convolution layer, and a pooling layer, and step 102 may include:

[0115] Performing Mel-spectrogram enhancement on one sample speech data of the sample speech data group through the Mel-spectrogram enhancement layer to obtain a first enhanced Mel-spectrogram speaker feature, and performing Mel-spectrogram enhancement on another sample speech data of the sample speech data group to obtain a second enhanced speaker Mel-spectrogram feature;

[0116] Performing feature convolution on the first enhanced speaker mel-spectrogram feature and the second enhanced speaker mel-spectrogram feature through a convolution layer to obtain an initial group of speaker features;

[0117] The initial group of speaker features are aggregated through the pooling layer to obtain the sample group of speaker features.

[0118] The benefit of this embodiment is that speaker features can be extracted with high accuracy when the speech data contains background noise.

[0119] In one embodiment, the Mel spectrum enhancement layer includes a first convolution layer, a complex encoding layer, a complex decoding layer, and a second convolution layer. The complex encoding layer includes a first complex two-dimensional convolution layer (specifically including a first real convolution layer, a first imaginary convolution layer), a first complex batch normalization layer, and a first real activation function layer. The complex decoding layer includes a second complex two-dimensional convolution layer (specifically including a second real convolution layer, a second imaginary convolution layer), a second complex batch normalization layer, and a second real activation function layer.

[0120] In step 103 of some embodiments, the speaker features of the sample group are classified by the feature classifier in the initial speaker verification model to obtain the sample verification speaker category of the sample speech data group. The sample verification speaker category indicates whether the speaker to which one sample speech data of the sample speech data group belongs is the same as or different from the speaker to which another sample speech data belongs.

[0121] In one embodiment, the feature classifier includes a fully connected layer, a normalized layer, and an activation function layer. The speaker features of the sample group are input into the fully connected layer, the output of the fully connected layer is input into the normalized layer, and the output of the normalized layer is input into the activation function layer to obtain the speaker category of the sample verification.

[0122] In step 104 of some embodiments, a contrastive learning loss is calculated based on the sample verification speaker category to obtain speaker contrastive loss data. Contrastive learning is a learning method that focuses on extracting meaningful representations by comparing positive and negative examples. It uses the assumption that in the learned embedding space, similar instances should be closer, while dissimilar instances should be farther away. Specifically in this embodiment, if the sample speech data group is a positive sample, the speaker contrastive loss data should tend to make the speaker to which one sample speech data of the sample speech data group indicated by the sample verification speaker category belongs to be the same as the speaker to which another sample speech data belongs; if the sample speech data group is a negative sample, the speaker contrastive loss data should tend to make the speaker to which one sample speech data of the sample speech data group indicated by the sample verification speaker category belongs to be different from the speaker to which another sample speech data belongs. A cross entropy loss function or the like can be selected for contrastive learning loss calculation.

[0123] In step 105 of some embodiments, the parameters of the initial speaker verification model are adjusted according to the speaker contrast loss data to obtain a target speaker verification model. During the parameter adjustment process, the speaker contrast loss data is compared with a preset speaker contrast loss threshold. If the speaker contrast loss data is greater than the speaker contrast loss threshold, the parameters of the initial speaker verification model are continuously adjusted until the speaker contrast loss data is less than or equal to the speaker contrast loss threshold, and the parameter adjustment is stopped to obtain the target speaker verification model.

[0124] In one embodiment, referring to Figure 4 After step 105, the speech synthesis method further includes: updating the target speaker verification model according to the sample speech data group, specifically including:

[0125] Step 401, determining one of the two sample voice data in the sample voice data group as sample independent voice data;

[0126] Step 402, extracting speaker features from the sample individual speech data through a feature embedder in the target speaker verification model to obtain sample individual speaker features;

[0127] Step 403, performing speech synthesis according to the sample individual speaker characteristics and the preset sample text characteristics to obtain sample synthesized speech data;

[0128] Step 404, calculating speaker loss based on the sample individual speech data and the sample synthesized speech data to obtain speaker loss data;

[0129] Step 405: Update the target speaker verification model according to the speaker loss data.

[0130] In step 401, one of the two sample speech data in the sample speech data group may be randomly determined as the sample independent speech data, or the first or the second of the two sample speech data may be determined as the sample independent speech data each time.

[0131] In one embodiment, referring to Figure 5 , step 401 may include:

[0132] Step 501, determining two sample voice data in the sample voice data group as first sample voice data and second sample voice data respectively;

[0133] Step 502: if the first sample voice data and the second sample voice data are voice data generated under the same background noise state, the first sample voice data or the second sample voice data is determined as sample independent voice data;

[0134] Step 503, if the first sample voice data and the second sample voice data are voice data generated under different background noise states, then obtaining the background noise contained in the first sample voice data to obtain the first sample background noise, and obtaining the background noise contained in the second sample voice data to obtain the second sample background noise;

[0135] Step 504, performing a noise comparison between the first sample background noise and the second sample background noise to obtain a noise comparison result;

[0136] Step 505: if the intensity of the first sample background noise is greater than or equal to the intensity of the second sample background noise, the first sample voice data is determined as sample independent voice data;

[0137] Step 506: If the intensity of the first sample background noise is less than the intensity of the second sample background noise, the second sample speech data is determined as sample independent speech data.

[0138] The benefit of the above embodiment is that by distinguishing between the same background noise state and different background noise states, different methods of determining sample single voice data from two sample voice data are set, thereby ensuring that the sample single voice data is the sample voice data with stronger background noise (at least not weaker than the other sample voice data), and reducing the processing steps such as calculating the intensity of the noise under the same background noise state, thereby saving processing resources.

[0139] In one embodiment, referring to Figure 6 , step 404 may include:

[0140] Step 601, performing speaker recognition on the sample individual speech data to obtain a first speaker, and performing speaker recognition on the sample synthesized speech data to obtain a second speaker;

[0141] Step 602, performing speaker difference calculation based on the first speaker and the second speaker to obtain speaker verification loss data;

[0142] Step 603, performing spectrum recognition on the sample individual speech data to obtain a first speech spectrum, and performing spectrum recognition on the sample synthesized speech data to obtain a second speech spectrum;

[0143] Step 604, calculating a spectrum error based on the first speech spectrum and the second speech spectrum to obtain speaker reconstruction loss data;

[0144] Step 605 , performing data fusion according to the speaker verification loss data and the speaker reconstruction loss data to obtain speaker loss data.

[0145] Specifically, speaker recognition can be performed using a speaker recognition model, such as ERes2Net. Speaker difference calculation can be performed using a cross entropy loss function. Spectrum recognition can be performed using a spectrum recognition model or a spectrum recognition algorithm. Spectrum error calculation can be performed using the L2 norm. Data fusion can be performed using a weighted average function or other data fusion formulas.

[0146] The benefit of the above-mentioned embodiment of steps 601 to 605 is that speaker loss data is obtained by jointly constructing speaker verification loss data and speaker reconstruction loss data, thereby improving the accuracy of model updating.

[0147] In step 106 of some embodiments, target speech data is acquired, and speaker features are extracted from the target speech data by a feature embedder in a target speaker verification model to obtain target speaker features.

[0148] In one embodiment, the feature embedder includes a mel spectrum enhancement layer, a convolution layer, and a pooling layer, and step 106 may include:

[0149] Through the Mel spectrum enhancement layer, the target speech data is enhanced by Mel spectrum to obtain the Mel spectrum features of the target enhanced speaker;

[0150] Perform feature convolution on the target enhanced speaker Mel spectrum features through the convolution layer to obtain the initial speaker features;

[0151] The initial speaker features are aggregated through the pooling layer to obtain the target speaker features.

[0152] The benefit of this embodiment is that speaker features can be extracted with high accuracy when the target speech data contains background noise.

[0153] In step 107 of some embodiments, speech generation is performed according to the target speaker characteristics and the preset target text characteristics to obtain target synthesized speech data. The target text characteristics are obtained by obtaining the target text, and encoding the target text through a text encoder to obtain the target text characteristics. The text encoder can use a Transformer-based encoder to generate text embeddings in combination with context information. In the text encoder, the intonation, stress, and pause information required for speech generation are captured through position embedding and semantic attention mechanisms.

[0154] In one embodiment, step 107 may include: constructing a speech generation model based on a generation framework that integrates variational inference and generative adversarial networks; performing speech generation on target speaker features and preset target text features through the speech generation model to obtain target synthesized speech data. Specifically, the generation framework that integrates variational inference and generative adversarial networks is a VITS (Variational Inference Text-to-Speech) architecture, which generates speech waveforms through variational inference. The speech generation model includes: a speaker-conditioned generator, which is used to introduce speaker embedding during the generation process to ensure that the timbre of the speech is consistent with that of the target speaker; a dynamic denoising module: through speech feature enhancement and filtering mechanisms, artifacts and noise residues are removed during the generation stage.

[0155] In summary, the present application can at least achieve the following beneficial effects: (1) Significantly improve noise robustness: Through self-supervised speaker verification loss and denoising mechanism, it is possible to extract robust speaker features in low-quality or noisy environments and generate clear and natural speech. (2) High data efficiency: The self-supervised learning method is used to reduce the dependence on labeled data, and high-quality speech generation can also be achieved in scenarios with limited resources. (3) Strong timbre matching: The speaker verification loss is introduced to ensure the consistency of the timbre of the generated speech with that of the target speaker, making it suitable for personalized speech synthesis needs. (4) Improved speech naturalness: Combined with the VITS framework and dynamic denoising mechanism, the sound quality of the generated speech is more natural and smooth, and there are no artifacts. (5) Wide application scenarios: It can be applied to virtual assistants, personalized speech synthesis, barrier-free communication and other scenarios in financial technology scenarios, and is particularly suitable for speech generation needs in low-resource or noisy environments.

[0156] See also Figure 7 The embodiment of the present application also provides a speech synthesis device, which can implement the above-mentioned speech synthesis method. Figure 7This is a block diagram of the module structure of the speech synthesis device provided in an embodiment of the present application, and the device includes: a sample data acquisition module 701, a sample feature extraction module 702, a speaker classification module 703, a contrast loss calculation module 704, a model parameter adjustment module 705, a target data acquisition module 706 and a speech generation module 707. The sample data acquisition module 701 is used to acquire a sample speech data group, wherein the sample speech data group includes two sample speech data; the sample feature extraction module 702 is used to extract speaker features of two sample speech data of the sample speech data group through a feature embedder in a preset initial speaker verification model to obtain speaker features of the sample group; the speaker classification module 703 is used to perform speaker classification on the speaker features of the sample group through a feature classifier in the initial speaker verification model to obtain a sample verification speaker category of the sample speech data group; the contrast loss calculation module 704 is used to perform contrast learning loss calculation according to the sample verification speaker category to obtain speaker contrast loss data; the model parameter adjustment module 705 is used to adjust the parameters of the initial speaker verification model according to the speaker contrast loss data to obtain a target speaker verification model; the target data acquisition module 706 is used to acquire target speech data, extract speaker features of the target speech data through a feature embedder in the target speaker verification model to obtain target speaker features; the speech generation module 707 is used to generate speech according to the target speaker features and preset target text features to obtain target synthesized speech data.

[0157] In one embodiment, after adjusting the parameters of the initial speaker verification model according to the speaker contrast loss data to obtain the target speaker verification model, the speech synthesis device further includes: a parameter secondary updating module for updating the target speaker verification model according to the sample speech data set.

[0158] In one embodiment, before extracting speaker features from two sample speech data of a sample speech data group by a feature embedder in a preset initial speaker verification model to obtain speaker features of the sample group, the speech synthesis device further includes: an embedder adjustment module, which is used to: determine the two sample speech data as first sample speech data and second sample speech data, respectively; extract speaker features from the first sample speech data by the feature embedder to obtain a first speaker feature, and extract speaker features from the second sample speech data by the feature embedder to obtain a second speaker feature; perform noise recognition on the first speaker feature to obtain a first noise feature, and perform noise recognition on the second speaker feature to obtain a second noise feature; perform denoising degree evaluation based on the first noise feature and the second noise feature to obtain a denoising score of the feature embedder; and adjust the feature embedder based on the denoising score.

[0159] It should be noted that the specific implementation of the speech synthesis device is basically the same as the specific embodiment of the above-mentioned speech synthesis method, and will not be repeated here.

[0160] The embodiment of the present application also provides an electronic device, which includes: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program is executed by the processor to realize the above-mentioned speech synthesis method. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0161] See also Figure 8 , Figure 8 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0162] The processor 801 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0163] The memory 802 may be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 may store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 802, and the processor 801 calls and executes the speech synthesis method of the embodiment of this application;

[0164] Input / output interface 803, used to implement information input and output;

[0165] The communication interface 804 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0166] A bus 805 that transmits information between the various components of the device (e.g., the processor 801, the memory 802, the input / output interface 803, and the communication interface 804);

[0167] The processor 801 , the memory 802 , the input / output interface 803 and the communication interface 804 are connected to each other in communication within the device via a bus 805 .

[0168] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned speech synthesis method.

[0169] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0170] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0171] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0172] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0173] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0174] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0175] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0176] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0177] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0178] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0179] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0180] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A speech synthesis method, characterized in that: The method comprises: Acquire a sample voice data group; wherein the sample voice data group includes two sample voice data; Performing speaker feature extraction on two sample speech data of the sample speech data group through a feature embedder in a preset initial speaker verification model to obtain speaker features of the sample group; Performing speaker classification on the speaker features of the sample group by the feature classifier in the initial speaker verification model to obtain a sample verification speaker category of the sample speech data group; wherein the sample verification speaker category indicates whether the speaker to which one sample speech data of the sample speech data group belongs is the same as or different from the speaker to which another sample speech data belongs; Comparative learning loss calculation is performed based on the sample verification speaker category to obtain speaker contrast loss data; Adjusting parameters of the initial speaker verification model according to the speaker contrast loss data to obtain a target speaker verification model; Acquire target speech data, and perform speaker feature extraction on the target speech data through a feature embedder in the target speaker verification model to obtain target speaker features; Speech generation is performed according to the target speaker characteristics and preset target text characteristics to obtain target synthesized speech data.

2. The method according to claim 1, characterized in that After adjusting the parameters of the initial speaker verification model according to the speaker contrast loss data to obtain a target speaker verification model, the method further includes: Updating the target speaker verification model according to the sample speech data group specifically includes: Determine one of the two sample voice data of the sample voice data group as sample individual voice data; Extracting speaker features from the sample individual speech data by using a feature embedder in the target speaker verification model to obtain sample individual speaker features; Performing speech synthesis according to the sample individual speaker characteristics and preset sample text characteristics to obtain sample synthesized speech data; Calculating speaker loss based on the sample individual speech data and the sample synthesized speech data to obtain speaker loss data; The target speaker verification model is updated according to the speaker loss data.

3. The method according to claim 2, characterized in that The step of calculating speaker loss according to the sample individual speech data and the sample synthesized speech data to obtain speaker loss data includes: Performing speaker recognition on the sample separate speech data to obtain a first speaker, and performing speaker recognition on the sample synthesized speech data to obtain a second speaker; Calculating a speaker difference between the first speaker and the second speaker to obtain speaker verification loss data; Performing spectrum recognition on the sample separate speech data to obtain a first speech spectrum, and performing spectrum recognition on the sample synthesized speech data to obtain a second speech spectrum; Calculate spectrum error based on the first speech spectrum and the second speech spectrum to obtain speaker reconstruction loss data; Data fusion is performed according to the speaker verification loss data and the speaker reconstruction loss data to obtain the speaker loss data.

4. The method according to claim 2, characterized in that: The step of determining one of the two sample voice data of the sample voice data group as sample independent voice data comprises: Determine two of the sample voice data in the sample voice data group as first sample voice data and second sample voice data respectively; If the first sample voice data and the second sample voice data are voice data generated under the same background noise state, determining the first sample voice data or the second sample voice data as the sample independent voice data; If the first sample voice data and the second sample voice data are voice data generated under different background noise conditions, obtaining the background noise contained in the first sample voice data to obtain the first sample background noise, and obtaining the background noise contained in the second sample voice data to obtain the second sample background noise; Performing a noise comparison on the first sample background noise and the second sample background noise to obtain a noise comparison result; If the intensity of the first sample background noise is greater than or equal to the intensity of the second sample background noise, determining the first sample voice data as the sample separate voice data; If the intensity of the first sample background noise is less than the intensity of the second sample background noise, the second sample voice data is determined as the sample independent voice data.

5. The method according to any one of claims 1 to 4, characterized in that: Before extracting speaker features from the two sample speech data of the sample speech data group by the feature embedder in the preset initial speaker verification model to obtain speaker features of the sample group, the method further includes: Determine the two sample voice data as first sample voice data and second sample voice data respectively; Performing speaker feature extraction on the first sample speech data by the feature embedder to obtain a first speaker feature, and performing speaker feature extraction on the second sample speech data by the feature embedder to obtain a second speaker feature; Performing noise recognition on the first speaker feature to obtain a first noise feature, and performing noise recognition on the second speaker feature to obtain a second noise feature; Performing a denoising degree evaluation based on the first noise feature and the second noise feature to obtain a denoising score of the feature embedder; The feature embedder is adjusted according to the denoising score.

6. The method according to claim 5, characterized in that The step of evaluating the denoising degree according to the first noise feature and the second noise feature to obtain a denoising score of the feature embedder includes: Determining a first feature denoising sub-score based on the strength of the first noise feature, and determining a second feature denoising sub-score based on the strength of the second noise feature; If the first sample speech data and the second sample speech data are speech data generated under the same background noise state, performing feature difference analysis on the first noise feature and the second noise feature to obtain a stable denoising sub-score; The denoising score is obtained by performing score aggregation according to the first feature denoising sub-score, the second feature denoising sub-score and the stable denoising sub-score.

7. The method according to claim 5, characterized in that The adjusting the feature embedder according to the denoising score comprises: If the denoising score is greater than or equal to a preset denoising score threshold, adjusting parameters of the feature embedder according to the denoising score; If the denoising score is less than the denoising score threshold, a denoising layer is added to the feature embedder to perform structural adjustment on the feature embedder.

8. A speech synthesis device, characterized in that: The device comprises: A sample data acquisition module, used to acquire a sample voice data group; wherein the sample voice data group includes two sample voice data; A sample feature extraction module, used for extracting speaker features from two sample speech data of the sample speech data group through a feature embedder in a preset initial speaker verification model to obtain speaker features of the sample group; A speaker classification module, configured to perform speaker classification on the speaker features of the sample group by using the feature classifier in the initial speaker verification model to obtain a sample verification speaker category of the sample speech data group; wherein the sample verification speaker category indicates whether a speaker to which one sample speech data of the sample speech data group belongs is the same as or different from a speaker to which another sample speech data belongs; A contrast loss calculation module, used to perform contrast learning loss calculation based on the sample to verify the speaker category and obtain speaker contrast loss data; A model parameter adjustment module, used to adjust the parameters of the initial speaker verification model according to the speaker contrast loss data to obtain a target speaker verification model; A target data acquisition module is used to acquire target speech data, and extract speaker features from the target speech data through a feature embedder in the target speaker verification model to obtain target speaker features; The speech generation module is used to generate speech according to the target speaker characteristics and preset target text characteristics to obtain target synthesized speech data.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the speech synthesis method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the speech synthesis method according to any one of claims 1 to 7 is implemented.

Images