A speech data synthesis method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to the technical field of speech processing, and discloses a speech data synthesis method and device, an electronic device and a storage medium. The method comprises: obtaining a speech synthesis task, the speech synthesis task comprising text information corresponding to a target language type and a style identifier corresponding to an original language type; predicting a target style feature using the text information and the style identifier, the target style feature being a style feature obtained by processing an original language feature, the original language feature being a language feature corresponding to the style identifier under the original language type; and synthesizing target speech data corresponding to the target language type based on the text information and the target style feature. The method provided by the present disclosure processes the style feature using the original language feature of the original language type in the process of predicting the style feature based on the text information and the style identifier, obtains the target style feature, thereby ensuring the accuracy of the style feature obtained during cross-language migration and enhancing the effect of the synthesized audio.

Description

Technical Field

[0001] This disclosure relates to the field of speech processing, specifically to a method, apparatus, electronic device, and storage medium for synthesizing speech data. Background Technology

[0002] With the development of audio processing technology, speech synthesis is being applied more and more, for example in scenarios such as intelligent question answering, voice broadcasting, audiobooks, and virtual anchors. Current end-to-end speech synthesis models can directly output the corresponding speech based on the input text. However, end-to-end speech synthesis technology produces relatively uniform speech styles, and in cross-language transfer speech synthesis, it cannot accurately obtain cross-language stylistic features, resulting in poor synthesized audio quality. Summary of the Invention

[0003] In view of this, embodiments of the present disclosure provide a method, apparatus, electronic device, and storage medium for synthesizing speech data to solve the problem that in the process of cross-language transfer speech synthesis, it is impossible to accurately obtain cross-language style features, resulting in poor synthesized audio quality.

[0004] In a first aspect, embodiments of this disclosure provide a method for synthesizing speech data, the method comprising: Obtain a speech synthesis task, wherein the speech synthesis task includes text information corresponding to the target language type and style identifiers corresponding to the original language type; The target style features are predicted using the text information and the style identifier, wherein the target style features are style features obtained by processing the original language features, and the original language features are the language features corresponding to the style identifier under the original language type; Based on the text information and the target style features, target speech data corresponding to the target language type is synthesized.

[0005] Secondly, embodiments of this disclosure provide a speech data synthesis apparatus, the apparatus comprising: The acquisition module is used to acquire the speech synthesis task, wherein the speech synthesis task includes text information corresponding to the target language type and style identifiers corresponding to the original language type; The prediction module is used to predict target style features using the text information and the style identifier, wherein the target style features are style features obtained by processing the original language features, and the original language features are the language features corresponding to the style identifier under the original language type; The synthesis module is used to synthesize target speech data corresponding to the target language type based on the text information and the target style features.

[0006] Thirdly, embodiments of this disclosure provide an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the method for synthesizing voice data as described in the first aspect or any corresponding embodiment.

[0007] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer instructions for causing a computer to execute the speech data synthesis method of the first aspect or any corresponding embodiment described above.

[0008] The technical solutions provided in this disclosure have the following advantages: In the process of predicting style features based on text information and style identifiers, the method provided in this disclosure uses the original language features of the original language type to process the style features and obtain the target style features, thereby ensuring the accuracy of the style features obtained during cross-language transfer and enhancing the effect of synthesized audio. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0010] Figure 1 This is a schematic flowchart of a method for synthesizing speech data according to some embodiments of the present disclosure; Figure 2 This is a flowchart illustrating another method for synthesizing speech data according to some embodiments of the present disclosure; Figure 3 This is a flowchart illustrating a training method for a style prediction model according to some embodiments of the present disclosure; Figure 4 This is a structural block diagram of a speech data synthesis apparatus according to some embodiments of the present disclosure; Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this disclosure. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure. The embodiments of this disclosure provide a novel method, apparatus, electronic device, and storage medium for editing power grid topology. The method provided by the embodiments of this disclosure can be applied to any required electronic device, such as a server, terminal, or other electronic device, and is not specifically limited here. For ease of description, it will be referred to as an electronic device below.

[0012] According to one aspect of the present disclosure, an embodiment of a method for synthesizing speech data is provided. Figure 1 This is a flowchart of a speech data synthesis method provided according to embodiments of this disclosure, such as... Figure 1 As shown, the process includes the following steps: Step S11: Obtain the speech synthesis task, wherein the speech synthesis task includes text information corresponding to the target language type and style identifiers corresponding to the original language type.

[0013] The method provided in this disclosure can be applied to smart devices with voice processing capabilities, such as: headphones, voice recorders, smart home terminals, business smart terminals (including: video phones, smart conference desktop terminals, etc.), wearable devices (including smartwatches, smart glasses, etc.), financial smart terminals, as well as smartphones, tablets, in-vehicle devices, computers, etc. Users can upload voice synthesis tasks through smart devices. The voice synthesis task includes: text information corresponding to the target language type and style identifiers corresponding to the original language type. The target language type can be English, Korean, Japanese, etc., and the original language type can be Chinese, etc., without specific limitations on the language type.

[0014] Specifically, when a user uses a client with speech synthesis capabilities on a smart device, after uploading text information in the target language, the client's interactive interface provides the user with multiple candidate style identifiers to choose from. The system then retrieves the style identifier selected by the user from these candidate identifiers. Each candidate style identifier corresponds to a style, which can be understood as the speaking style of a language object in different scenarios or with different emotions. For example, the style of speech activities such as news broadcasting, reading, dialogue, and recitation under different scenarios or emotions. Finally, the retrieved text information and style identifiers are used to generate a speech synthesis task.

[0015] Step S12: Predict target style features using text information and style identifiers. The target style features are style features obtained by processing the original language features. The original language features are the language features corresponding to the style identifiers under the original language type.

[0016] In this embodiment, the smart device can extract text features from text information and obtain preset text features. The association between the style identifier and the style features is then determined. Based on this association, an initial style feature corresponding to the text features and the style identifier is predicted. The style feature can be used to characterize a person's speaking style, speaking characteristics, or language expressiveness. Style features may include, but are not limited to, emotional features, scene features, and dialect features. Then, the language features corresponding to the style identifier under the original language type are obtained, and the original language features are separated from the initial style feature. The separated initial style feature is then used as the target style feature.

[0017] It should be noted that, since this application embodiment implements cross-language type synthesized speech data, in existing cross-language type synthesized speech data processes, such as directly migrating a Chinese-style news live broadcast to an English-style news live broadcast, the style features contain some acoustic style features, as well as some linguistic features. If the linguistic features are not separated, a Chinese accent will appear, ultimately affecting the quality of the synthesized speech. Therefore, it is necessary to separate the original linguistic features of the original language type from the initial style features to ensure that the final target style features are standard style features.

[0018] Step S13: Synthesize target speech data corresponding to the target language type based on text information and target style features.

[0019] In this embodiment of the disclosure, the synthesis of target speech data corresponding to the target language type based on text information and target style features includes the following steps A1-A4: Step A1: Detect the target text features corresponding to the text information, and determine the target prosodic features based on the target text features and target style features.

[0020] In this embodiment, detecting the target text features corresponding to the text information can be achieved by: inputting the text information into a text processing model, extracting the target text features from the text information, and then converting the target text features into a phoneme sequence. The final target prosodic features are then determined based on the phoneme sequence and the target style features. Specifically, the target prosodic features can be determined by combining the target style features with the duration boundaries of each phoneme in the phoneme sequence.

[0021] In another embodiment of this application, prosodic prediction based on text features and style features includes: adding feature encoding to target text features and target style features; performing prosodic prediction based on the target text features and target style features with added feature encoding to obtain target prosodic features. Target prosodic features belong to a phonological structure of language and are closely related to other linguistic structures such as syntax, discourse structure, and information structure. Prosodic features can be divided into three main aspects: intonation, temporal distribution, and stress. It should be noted that prosodic features are related to time; prosodic features are a sequence, and prosodic features differ at different times. Adding feature encoding to target text features and target style features can enhance the richness of prosodic features.

[0022] Step A2: Obtain the timbre features corresponding to the speech object.

[0023] In this embodiment, the speech object can be the speaker of the speech data to be synthesized. Audio data of the speech object is acquired and input into a timbre detection model to obtain the timbre features of the speech object. Timbre features may include, but are not limited to, features such as pitch, length, and tone.

[0024] Step A3: Synthesize target acoustic features based on target text features, target prosodic features, and timbre features.

[0025] In this embodiment, target text features, target prosodic features, and timbre features are input into a speech synthesis model. The speech synthesis model includes a feature processor and a decoder. The feature processor processes the target text features and target prosodic features to obtain processed text features and processed prosodic features, and then passes the processed text features and processed prosodic features to the decoder. The decoder decodes and synthesizes based on the processed text features, processed prosodic features, and timbre features to obtain target acoustic features, which may be Mel-spectral coefficient features.

[0026] Step A4: Convert the target acoustic features into speech waveforms to obtain target speech data.

[0027] In this embodiment, the speech synthesis model further includes a vocoder. The vocoder receives the target acoustic features transmitted by the encoder, converts the target acoustic features into a speech waveform, then obtains an audio template, and adds the speech waveform to the audio template to obtain the target speech data.

[0028] The method provided in this disclosure separates the original language features of the original language type from the initial style features during the process of predicting style features based on text information and style identifiers, thereby obtaining the target style features. This ensures the accuracy of the style features obtained during cross-language transfer and enhances the effect of synthesized audio.

[0029] According to another aspect of the embodiments of this disclosure, a method embodiment for synthesizing voice data is provided. Figure 2 This is a flowchart of a speech data synthesis method according to another embodiment of the present disclosure, such as... Figure 2 As shown, the method includes: Step S21: Obtain the speech synthesis task, wherein the speech synthesis task includes text information corresponding to the target language type and style identifiers corresponding to the original language type. For detailed explanation, please refer to the relevant descriptions in the above embodiments, which will not be repeated here.

[0030] Step S22: Predict target style features using text information and style identifiers. The target style features are style features obtained by processing the original language features. The original language features are the language features corresponding to the style identifiers under the original language type.

[0031] In this embodiment of the disclosure, the target style features are predicted using text information and style identifiers, including the following steps B1-B3: Step B1: Obtain a pre-trained style prediction model, wherein the style prediction model includes a first prediction network and a first processing network.

[0032] Step B2: Extract the target text features corresponding to the text information through the first prediction network, and predict the initial style features based on the target text features and style identifiers.

[0033] Step B3: Obtain the original language features corresponding to the style identifier in the original language type through the first processing network, and separate the original language features from the initial style features to obtain the target style features.

[0034] In this embodiment, text information and style identifiers are input into the prediction network of the style prediction model. The prediction network extracts target text features from the text information and, based on the preset correspondence between text features, style identifiers, and style features, determines the initial style features corresponding to the target text features and style identifiers. It should be noted that the initial style features obtained at this time carry both acoustic style features and linguistic features. The prediction network passes the initial style features to the processing network, which obtains the original linguistic features corresponding to the style identifiers in the original language type and separates the linguistic features from the initial style features to obtain the target style features.

[0035] Step S23: Synthesize target speech data corresponding to the target language type based on text information and target style features. For detailed explanation, please refer to the relevant descriptions in the above embodiments; they will not be repeated here.

[0036] The method provided in this disclosure, in the process of predicting style features based on text information and style identifiers, uses the processing network in the style prediction model to separate the original language features of the original language type from the initial style features, thereby obtaining the target style features. This ensures the accuracy of the style features obtained during cross-language transfer and enhances the effect of synthesized audio.

[0037] Figure 3 This is a flowchart of a training method for a style prediction model provided according to embodiments of this disclosure, such as... Figure 3 As shown, the training methods for style prediction models include: Step S31: Obtain training sample data, wherein the training sample data includes text samples of the first language type and style identifier samples of the second language type.

[0038] In this embodiment of the disclosure, the training sample data includes: text samples of a first language type and style identifier samples of a second language type. The first language type can be English, Korean, Japanese, etc., and the original language type can be Chinese, etc. The language type is not specifically limited here. The source of the training sample data is not limited. The text samples of the first language type can be obtained by real-time acquisition and recognition through recording equipment, or they can be, for example, text corresponding to radio programs, audio text of television programs, text of real human speech, etc.

[0039] Step S32: Input the text sample and style identifier sample into the initial style prediction model to obtain the output style features.

[0040] In this embodiment of the disclosure, the initial style prediction model includes a second prediction network and a second processing network.

[0041] In this embodiment of the disclosure, text information and style identifiers are input into an initial style prediction model to obtain output style features, including the following steps C1-C2: Step C1: Extract text features corresponding to text samples through the second prediction network, and predict style features based on text features and style label samples.

[0042] In this embodiment, text samples and style identifiers are input into the prediction network of the style prediction model. The prediction network extracts target text features from the text information and determines the initial style features corresponding to the text features and style identifier samples based on the preset correspondence between text features, style identifiers, and style features. It should be noted that the initial style features obtained at this time carry acoustic style features and linguistic features. The prediction network then passes the initial style features to the processing network.

[0043] Step C2: Obtain the language features corresponding to the style identifier in the second language type through the second processing network, and separate the language features corresponding to the second language type from the style features to obtain the output style features.

[0044] In this embodiment of the disclosure, the processing network obtains the linguistic features corresponding to the style identifier in the second language type, and separates the linguistic features from the initial style features to obtain the style features. Specifically, the processing network calculates the mutual information between the linguistic features and the style features corresponding to the second language type; when the mutual information is minimal, the linguistic features are separated from the style features to obtain the output style features.

[0045] It should be noted that when training the initial style prediction model using training sample data, minimizing the mutual information between style features and oracle features is the training objective. Minimizing the mutual information between style features and oracle features ensures that the minimization of mutual information between style features and linguistic features is completely independent and does not interfere with each other. This allows for the separation of linguistic features from the original language type during cross-language type transfer, thereby improving the realism of the synthesized speech.

[0046] Step S33: Determine the target loss between the output style features and the standard style features, wherein the standard style features are obtained based on the audio data corresponding to the first language type.

[0047] In this embodiment of the disclosure, determining the target loss between the output style features and the standard style features includes: acquiring audio data of a first language type corresponding to the style identifier sample, and extracting the acoustic features corresponding to the audio data; inputting the acoustic features into a style detection model to obtain the standard style features corresponding to the acoustic features; calculating the mean square error between the output style features and the standard style features, and using the mean square error as the target loss.

[0048] In the disclosed embodiments, a style detection model is also set up to ensure the accuracy of the output style features of the initial style prediction model. First, audio data of the first language type corresponding to the style identifier sample is acquired. For example, the audio data of the first language type corresponding to the style identifier sample is Chinese audio data in the style of news broadcasting. Then, an acoustic detection model is used to extract acoustic features (e.g., timbre, prosody, etc.) from the Chinese audio data, and these acoustic features are input into the style detection model. The style detection model determines the standard style features corresponding to the acoustic features based on the mapping relationship between the acoustic features and style features. Finally, the mean squared error between the output style features and the standard style features is calculated, and this mean squared error is used as the target loss.

[0049] As an example, the audio data corresponding to the first language type of the style identifier sample is audio data recorded using emotion H (corresponding to serious style feature), language type E (style feature English), and scene C (style feature news broadcasting). This audio data is news broadcasting in English with a serious tone, denoted as HEC. Here, "HEC" is the style identifier, corresponding to the three style identifiers of emotion, language type, and scene. Then, this audio data is input into the acoustic detection model, which uses the acoustic detection model to extract the acoustic features of the Chinese audio data. At this point, the acoustic features can be understood as the key features of news broadcasting in English. Then, the acoustic features are input into the style detection model, which determines the standard style features corresponding to the acoustic features based on the mapping relationship between acoustic features and style features. That is, it determines which features in the acoustic features belong to seriousness (emotion), which features belong to pronunciation (language type), and so on.

[0050] Step S34: Correct the initial style prediction model based on the target loss until a trained style prediction model is obtained.

[0051] In this embodiment of the disclosure, if it is determined that the target loss does not fall within the preset loss range, it is determined that the current initial style prediction model has not met the training requirements. Therefore, it is necessary to update / optimize the parameters of the prediction network and the processing network in the initial style prediction model based on the target loss, and continue to train the updated prediction network and the updated processing network until the target loss of the final output style feature falls within the preset loss range, which indicates that the initial style prediction model has met the training requirements.

[0052] The training method provided in this embodiment ensures the accuracy of the output style features by setting up a style detection model and calculating the loss using the output style features of the style prediction model and the standard style features output by the style detection model. Furthermore, it minimizes the mutual information between style features and linguistic features as the training objective, ensuring that minimizing the mutual information between style features and linguistic features is completely independent and does not interfere with each other, thereby further guaranteeing the accuracy of the style features.

[0053] This embodiment also provides a speech data synthesis apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0054] This embodiment provides a speech data synthesis device, such as... Figure 4 As shown, it includes: The acquisition module 41 is used to acquire the speech synthesis task, wherein the speech synthesis task includes text information corresponding to the target language type and style identifiers corresponding to the original language type; Prediction module 42 is used to predict target style features using text information and style identifiers. The target style features are style features obtained by processing the original language features. The original language features are the language features corresponding to the style identifiers under the original language type. Synthesis module 43 is used to synthesize target speech data corresponding to the target language type based on text information and target style features.

[0055] In this embodiment of the disclosure, the prediction module 42 is used to obtain a pre-trained style prediction model, wherein the style prediction model includes a first prediction network and a first processing network; the first prediction network extracts target text features corresponding to text information, and predicts initial style features based on the target text features and style identifiers; the first processing network obtains the original language features corresponding to the style identifiers in the original language type, and separates the language features from the initial style features to obtain the target style features.

[0056] In this embodiment of the disclosure, the speech data synthesis apparatus further includes a training module, which includes: The acquisition unit is used to acquire training sample data, wherein the training sample data includes text samples of the first language type and style identifier samples of the second language type. The input unit is used to input text samples and style identifier samples into the initial style prediction model to obtain output style features; The loss calculation unit is used to determine the target loss between the output style features and the standard style features, wherein the standard style features are obtained based on the audio data corresponding to the first language type; The calibration module is used to calibrate the initial style prediction model based on the target loss until a trained style prediction model is obtained.

[0057] In this embodiment of the disclosure, the initial style prediction model includes: a second prediction network and a second processing network; The input unit is used to extract text features corresponding to text samples through the second prediction network, and predict style features based on text features and style identifiers; the second processing network is used to obtain the language features corresponding to the style identifiers in the second language type, and the language features corresponding to the second language type are separated from the style features to obtain the output style features.

[0058] In this embodiment of the disclosure, the input unit is used to calculate the mutual information between the language features and style features corresponding to the second language type through a processing network; when the mutual information is minimal, the language features are separated from the style features to obtain the output style features.

[0059] In this embodiment of the disclosure, the loss calculation unit is used to acquire audio data of the first language type corresponding to the style identifier sample, and extract the acoustic features corresponding to the audio data; input the acoustic features into the style detection model to obtain the standard style features corresponding to the acoustic features; calculate the mean square error between the output style features and the standard style features, and use the mean square error as the target loss.

[0060] In this embodiment of the disclosure, the synthesis module 43 is used to detect the target text features corresponding to the text information, and determine the target prosodic features based on the target text features and the target style features; obtain the timbre features corresponding to the speech object; synthesize the target acoustic features based on the target text features, the target prosodic features and the timbre features; and convert the target acoustic features into speech waveforms to obtain target speech data.

[0061] In this embodiment, the speech data synthesis device is presented in the form of a functional unit. Here, a unit refers to an ASIC circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0062] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0063] This disclosure also provides an electronic device having the above-described features. Figure 5 The device shown is for synthesizing speech data.

[0064] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an optional embodiment of this disclosure, such as... Figure 5 As shown, the electronic device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 5 Take a processor 10 as an example.

[0065] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GPA), or any combination thereof.

[0066] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0067] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device as displayed on a mini-program landing page. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0068] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0069] The electronic device also includes input and output devices. The processor 10, memory 20, input devices, and output devices can be connected via a bus or other means.

[0070] Input devices can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as touchscreens, keypads, mice, trackpads, touchpads, joysticks, one or more mouse buttons, trackballs, joysticks, etc. Output devices may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0071] The electronic device also includes a communication interface 30 for communicating with other devices or communication networks.

[0072] This disclosure also provides a computer-readable storage medium in which the methods described in this disclosure can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium after being downloaded over a network. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium may be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0073] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for synthesizing speech data, characterized in that, The method includes: Obtain a speech synthesis task, wherein the speech synthesis task includes text information corresponding to the target language type and style identifiers corresponding to the original language type; The target style features are predicted using the text information and the style identifier. The target style features are obtained by calculating the mutual information between the original language features and the initial style features, and separating the original language features from the initial style features when the mutual information reaches the minimum value. The initial style features are predicted based on the target text features corresponding to the text information and the style identifier. The original language features are the language features corresponding to the style identifier under the original language type. Based on the text information and the target style features, target speech data corresponding to the target language type is synthesized.

2. The method according to claim 1, characterized in that, The method of predicting target style features using the text information and the style identifier includes: Obtain a pre-trained style prediction model, wherein the style prediction model includes a first prediction network and a first processing network; The first prediction network extracts the target text features corresponding to the text information, and predicts the initial style features based on the target text features and the style identifier. The first processing network obtains the original language features corresponding to the original language type of the style identifier, and separates the original language features from the initial style features to obtain the target style features.

3. The method according to claim 2, characterized in that, The training methods for the style prediction model include: Acquire training sample data, wherein the training sample data includes text samples of a first language type and style identifier samples of a second language type; Input the text sample and the style identifier sample into the initial style prediction model to obtain the output style features; Determine the target loss between the output style features and the standard style features, wherein the standard style features are obtained based on the audio data corresponding to the first language type; The initial style prediction model is corrected based on the target loss until a trained style prediction model is obtained.

4. The method according to claim 3, characterized in that, The initial style prediction model includes a second prediction network and a second processing network; The step of inputting the text sample and the style identifier sample into the initial style prediction model to obtain the output style features includes: The second prediction network extracts the text features corresponding to the text samples, and predicts style features based on the text features and the style identifier samples; The second processing network obtains the language features corresponding to the style identifier in the second language type, and separates the language features corresponding to the second language type from the style features to obtain the output style features.

5. The method according to claim 4, characterized in that, The step of separating the language features corresponding to the second language type from the style features to obtain the output style features includes: The processing network calculates the mutual information between the linguistic features and the style features corresponding to the second language type; When the mutual information is minimized, the language features are separated from the style features to obtain the output style features.

6. The method according to claim 3, characterized in that, Determining the target loss between the output style features and the standard style features includes: Obtain the audio data of the first language type corresponding to the style identifier sample, and extract the acoustic features corresponding to the audio data; The acoustic features are input into the style detection model to obtain the standard style features corresponding to the acoustic features; Calculate the mean squared error between the output style feature and the standard style feature, and use the mean squared error as the target loss.

7. The method according to claim 1, characterized in that, Based on the text information and the target style features, target speech data corresponding to the target language type is synthesized, including: Detect the target text features corresponding to the text information, and determine the target prosodic features based on the target text features and the target style features; Obtain the timbre features corresponding to the speech object; Based on the target text features, target prosodic features, and the timbre features, target acoustic features are synthesized. The target acoustic features are converted into speech waveforms to obtain the target speech data.

8. A speech data synthesis apparatus, characterized in that, The device includes: The acquisition module is used to acquire the speech synthesis task, wherein the speech synthesis task includes: text information corresponding to the target language type and style identifiers corresponding to the original language type; The prediction module is used to predict target style features using the text information and the style identifier. The target style features are obtained by calculating the mutual information between the original language features and the initial style features, and separating the original language features from the initial style features when the mutual information reaches the minimum value. The initial style features are predicted based on the target text features corresponding to the text information and the style identifier. The original language features are the language features corresponding to the style identifier under the original language type. The synthesis module is used to synthesize target speech data corresponding to the target language type based on the text information and the target style features.

9. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the method for synthesizing speech data according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the method for synthesizing speech data according to any one of claims 1 to 7.

Images