Speech synthesis method and device based on artificial intelligence, computer equipment and medium

Abstract

The invention belongs to the technical field of artificial intelligence, and relates to an artificial intelligence-based speech synthesis method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining a to-be-synthesized target text; performing feature conversion on the target text based on a conversion network in a preset target speech synthesis model to obtain acoustic features; converting the acoustic features based on an acoustic mode encoder to obtain an acoustic embedding sequence; adjusting the length of the acoustic embedding sequence based on a phoneme duration adapter to obtain a target embedding vector; fusing the target embedding vector and a preset target style feature based on a style decorator to obtain fused embedding; performing Mel spectrogram generation on fusion embedding based on a Mel spectrogram encoder and a linear layer to obtain a target Mel spectrogram; and performing voice conversion on the target Mel-frequency spectrogram based on a vocoder to obtain target voice. In addition, the target voice may be stored in the blockchain. According to the invention, natural, accurate and high-quality target voice can be efficiently and accurately generated.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence development technology and the field of financial technology, and in particular to an artificial intelligence-based speech synthesis method, apparatus, computer equipment and storage medium. Background Art

[0002] In the field of financial technology, with the continuous advancement of technology, text-to-speech (TTS) technology has become an important bridge connecting the digital world and human auditory experience. This technology can convert electronic text content into speech close to human natural voice in real time, greatly enriching the dimension of digital interaction and improving user experience. Especially in the fields of financial information broadcasting, customer service automation, and smart investment consulting, TTS technology plays an irreplaceable role with its high efficiency and convenience.

[0003] However, despite the significant progress made in TTS technology, most of the mainstream TTS systems on the market are still built on the phoneme-driven model. The phoneme-driven model breaks down the text into basic speech units, namely phonemes, and combines these phonemes into a continuous speech stream according to preset rules or statistical models. This method has achieved certain results in the accuracy and fluency of speech synthesis, but its inherent limitations are also becoming increasingly prominent. Specifically, traditional phoneme-driven models often rely too much on fixed rules and parameters, and it is difficult to flexibly adapt to these natural changes, resulting in the synthesized speech sounding monotonous and lacking in changes, and it is impossible to generate natural, high-quality speech. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a speech synthesis method, apparatus, computer equipment and storage medium based on artificial intelligence to solve the technical problem that the existing phoneme-driven models often rely too much on fixed rules and parameters and are difficult to flexibly adapt to these natural changes, resulting in the synthesized speech appearing monotonous and lacking in changes, and unable to generate natural, high-quality speech.

[0005] In order to solve the above technical problems, the present application provides a speech synthesis method based on artificial intelligence, which adopts the following technical solutions:

[0006] Obtain the target text to be synthesized;

[0007] Based on the conversion network in the preset target speech synthesis model, the target text is subjected to feature conversion processing to obtain corresponding acoustic features; wherein the target speech synthesis model at least includes a conversion network, an acoustic pattern encoder, a phoneme duration adapter, a style decorator, a mel-spectrogram encoder and a linear layer;

[0008] Based on the acoustic mode encoder, the acoustic feature is converted to obtain a corresponding acoustic embedding sequence;

[0009] Performing a length adjustment process on the acoustic embedding sequence based on the phoneme duration adapter to obtain a corresponding target embedding vector;

[0010] Based on the style decorator, the target embedding vector is fused with a preset target style feature to obtain a corresponding fused embedding;

[0011] Performing mel-spectrogram generation processing on the fused embedding based on the mel-spectrogram encoder and the linear layer to obtain a corresponding target mel-spectrogram;

[0012] The target Mel-spectrogram is subjected to speech conversion processing based on a preset vocoder to obtain a corresponding target speech.

[0013] Furthermore, the step of performing feature conversion processing on the target text based on the conversion network in the preset target speech synthesis model to obtain corresponding acoustic features specifically includes:

[0014] Inputting the target text into the conversion network;

[0015] The target text is converted by the conversion network to obtain a corresponding phoneme sequence and style sequence;

[0016] The acoustic feature is generated based on the phoneme sequence and the style sequence.

[0017] Furthermore, the step of performing a length adjustment process on the acoustic embedding sequence based on the phoneme duration adapter to obtain a corresponding target embedding vector specifically includes:

[0018] Performing duration prediction processing on the acoustic embedding vector in the acoustic embedding sequence based on the duration predictor in the phoneme duration adapter to obtain a corresponding duration;

[0019] Get the specified length of the preset specified Mel spectrum graph;

[0020] Based on the specified length and the duration, using a length adjuster in the phoneme duration adapter to adjust the length of the acoustic embedding vector in the acoustic embedding sequence to obtain an adjusted specified acoustic embedding vector;

[0021] The specified acoustic embedding vector is used as the target embedding vector.

[0022] Furthermore, the step of fusing the target embedding vector with a preset target style feature based on the style decorator to obtain a corresponding fused embedding specifically includes:

[0023] Based on the style decorator, calling a preset deep learning layer to convert the target style features to obtain a corresponding style embedding;

[0024] Get the preset fusion strategy;

[0025] Based on the fusion strategy, the target embedding vector and the style embedding are fused to obtain a corresponding fused specified embedding;

[0026] The specified embedding is used as the fused embedding.

[0027] Furthermore, the step of performing mel-spectrogram generation processing on the fused embedding based on the mel-spectrogram encoder and the linear layer to obtain a corresponding target mel-spectrogram specifically includes:

[0028] Processing the fused embedding based on the mel-spectrogram encoder to obtain a corresponding mel-spectrogram embedding;

[0029] Embedding and mapping the mel-spectrogram into the dimension of the corresponding mel-spectrogram based on the linear layer to obtain a corresponding mapping result;

[0030] The mapping result is used as the target Mel-spectrogram.

[0031] Furthermore, the step of obtaining the target text to be synthesized specifically includes:

[0032] Get the initial text to be synthesized input by the user;

[0033] Performing space removal processing on the initial text to obtain a corresponding first text;

[0034] Perform line break removal processing on the first text to obtain a corresponding second text;

[0035] Performing punctuation removal processing on the second text to obtain a corresponding third text;

[0036] The third text is used as the target text.

[0037] Furthermore, before the step of fusing the target embedding vector with a preset target style feature based on the style decorator to obtain a corresponding fused embedding, the method further includes:

[0038] Acquire style parameters input by a specified user; wherein the style parameters at least include a speech speed parameter and an intonation parameter;

[0039] Call the preset style transfer model;

[0040] Performing style conversion processing on the style parameters based on the style conversion model to obtain corresponding style representation;

[0041] The style representation is used as the target style feature.

[0042] In order to solve the above technical problems, the embodiment of the present application further provides a speech synthesis device based on artificial intelligence, which adopts the following technical solution:

[0043] A first acquisition module is used to acquire a target text to be synthesized;

[0044] A first conversion module is used to perform feature conversion processing on the target text based on a conversion network in a preset target speech synthesis model to obtain corresponding acoustic features; wherein the target speech synthesis model at least includes a conversion network, an acoustic mode encoder, a phoneme duration adapter, a style decorator, a mel-spectrogram encoder and a linear layer;

[0045] A second conversion module, configured to convert the acoustic features based on the acoustic mode encoder to obtain a corresponding acoustic embedding sequence;

[0046] An adjustment module, configured to perform a length adjustment process on the acoustic embedding sequence based on the phoneme duration adapter to obtain a corresponding target embedding vector;

[0047] A fusion module, used for fusing the target embedding vector with a preset target style feature based on the style decorator to obtain a corresponding fused embedding;

[0048] A generation module, configured to perform a Mel-spectrogram generation process on the fused embedding based on the Mel-spectrogram encoder and the linear layer to obtain a corresponding target Mel-spectrogram;

[0049] The third conversion module is used to perform speech conversion processing on the target Mel-spectrogram based on a preset vocoder to obtain a corresponding target speech.

[0050] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0051] Obtain the target text to be synthesized;

[0052] Based on the conversion network in the preset target speech synthesis model, the target text is subjected to feature conversion processing to obtain corresponding acoustic features; wherein the target speech synthesis model at least includes a conversion network, an acoustic pattern encoder, a phoneme duration adapter, a style decorator, a mel-spectrogram encoder and a linear layer;

[0053] Based on the acoustic mode encoder, the acoustic feature is converted to obtain a corresponding acoustic embedding sequence;

[0054] Performing a length adjustment process on the acoustic embedding sequence based on the phoneme duration adapter to obtain a corresponding target embedding vector;

[0055] Based on the style decorator, the target embedding vector is fused with a preset target style feature to obtain a corresponding fused embedding;

[0056] Performing mel-spectrogram generation processing on the fused embedding based on the mel-spectrogram encoder and the linear layer to obtain a corresponding target mel-spectrogram;

[0057] The target Mel-spectrogram is subjected to speech conversion processing based on a preset vocoder to obtain a corresponding target speech.

[0058] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0059] Obtain the target text to be synthesized;

[0060] Based on the conversion network in the preset target speech synthesis model, the target text is subjected to feature conversion processing to obtain corresponding acoustic features; wherein the target speech synthesis model at least includes a conversion network, an acoustic pattern encoder, a phoneme duration adapter, a style decorator, a mel-spectrogram encoder and a linear layer;

[0061] Based on the acoustic mode encoder, the acoustic feature is converted to obtain a corresponding acoustic embedding sequence;

[0062] Performing a length adjustment process on the acoustic embedding sequence based on the phoneme duration adapter to obtain a corresponding target embedding vector;

[0063] Based on the style decorator, the target embedding vector is fused with a preset target style feature to obtain a corresponding fused embedding;

[0064] Performing mel-spectrogram generation processing on the fused embedding based on the mel-spectrogram encoder and the linear layer to obtain a corresponding target mel-spectrogram;

[0065] The target Mel-spectrogram is subjected to speech conversion processing based on a preset vocoder to obtain a corresponding target speech.

[0066] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0067] The present application first obtains a target text to be synthesized; then performs feature conversion processing on the target text based on a conversion network in a preset target speech synthesis model to obtain corresponding acoustic features; wherein the target speech synthesis model at least includes a conversion network, an acoustic mode encoder, a phoneme duration adapter, a style decorator, a Mel-spectrogram encoder and a linear layer; then, the acoustic features are converted based on the acoustic mode encoder to obtain a corresponding acoustic embedding sequence; subsequently, the acoustic embedding sequence is length-adjusted based on the phoneme duration adapter to obtain a corresponding target embedding vector; further, based on the style decorator, the target embedding vector is fused with a preset target style feature to obtain a corresponding fused embedding; and based on the Mel-spectrogram encoder and the linear layer, a Mel-spectrogram generation processing is performed on the fused embedding to obtain a corresponding target Mel-spectrogram; finally, a speech conversion processing is performed on the target Mel-spectrogram based on a preset vocoder to obtain a corresponding target speech. The present application obtains a target text to be synthesized, then performs feature conversion on the target text based on a conversion network in a target speech synthesis model to obtain acoustic features, converts the acoustic features based on an acoustic pattern encoder to obtain an acoustic embedding sequence, adjusts the length of the acoustic embedding sequence based on a phoneme duration adapter to obtain a target embedding vector, fuses the target embedding vector with a preset target style feature based on a style decorator to obtain a fused embedding, generates a Mel-spectrogram of the fused embedding based on a Mel-spectrogram encoder and a linear layer to obtain a target Mel-spectrogram, and finally performs speech conversion on the target Mel-spectrogram based on a vocoder. This can achieve efficient and accurate generation of a natural, accurate and high-quality target speech, thereby improving the accuracy and intelligence of speech synthesis. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0069] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0070] Figure 2 A flowchart of an embodiment of a speech synthesis method based on artificial intelligence according to the present application;

[0071] Figure 3 is a structural schematic diagram of an embodiment of an artificial intelligence-based speech synthesis device according to the present application;

[0072] Figure 4It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0073] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0074] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0075] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0076] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0077] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0078] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to the laptop computer 1011, tablet computer 1012 or mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0079] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0080] It should be noted that the artificial intelligence-based speech synthesis method provided in the embodiments of the present application is generally executed by a server / terminal device, and accordingly, the artificial intelligence-based speech synthesis device is generally arranged in the server / terminal device.

[0081] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0082] Continue to refer Figure 2 , showing a flow chart of an embodiment of an artificial intelligence-based speech synthesis method according to the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The artificial intelligence-based speech synthesis method provided in the embodiment of the present application can be applied to any scenario that requires speech synthesis, and the artificial intelligence-based speech synthesis method can be applied to products in these scenarios, for example, speech synthesis in the financial insurance field. The artificial intelligence-based speech synthesis method comprises the following steps:

[0083] Step S201, obtaining the target text to be synthesized.

[0084] In this embodiment, the electronic device (e.g. Figure 1The server / terminal device shown in the figure) can obtain the target text to be synthesized through a wired connection or a wireless connection. It should be pointed out that the above-mentioned wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future. The executor of the present application is specifically a speech synthesis system, or a TTS system, referred to as the system. The above-mentioned target text is a text that requires speech synthesis processing, and the target text consists of a series of words or characters. In the business scenarios in the financial field, the above-mentioned target text may include texts of various content types such as transaction texts, payment texts, and consulting texts. Among them, the specific implementation process of obtaining the target text to be synthesized above will be further described in detail in the subsequent specific embodiments of the present application, and will not be elaborated on here.

[0085] Step S202, performing feature conversion processing on the target text based on the conversion network in the preset target speech synthesis model to obtain corresponding acoustic features.

[0086] In this embodiment, the target speech synthesis model at least includes a conversion network, an acoustic mode encoder, a phoneme duration adapter, a style decorator, a mel spectrogram encoder, and a linear layer. By adding a style decorator structure to the target speech synthesis model, it is possible to distinguish the training of style features from the training of speech features, thereby simplifying the process of style adaptation. And by applying the low-rank adaptation (LoRA) technology, fine-tuning of the pre-trained model is achieved, and only the parameters need to be fine-tuned to ensure that the uniqueness of the phoneme embedding vector is maintained during the style adaptation process.

[0087] Among them, the above-mentioned specific implementation process of performing feature conversion processing on the target text based on the conversion network in the preset target speech synthesis model to obtain the corresponding acoustic features will be further described in detail in the subsequent specific embodiments of this application and will not be elaborated in detail here.

[0088] Step S203: converting the acoustic features based on the acoustic mode encoder to obtain a corresponding acoustic embedding sequence.

[0089] In this embodiment, the acoustic mode encoder may specifically adopt a variant of the Transformer architecture, that is, the standard two-layer dense network in the Transformer is replaced by two 1DCNNs followed by ReLU activation. By inputting the above-mentioned acoustic features into the acoustic mode encoder, the acoustic mode encoder converts the input sequence into an embedding using a feedforward Transformer (FFT). Specifically, each FFT block contains a self-attention block and a one-dimensional convolutional neural network (1DCNN). The self-attention mechanism captures the positional relationship in the sequence through multi-head attention, thereby realizing the conversion of acoustic features into corresponding acoustic embedding sequences. By modifying the Transformer, the model's ability to capture the close dependencies between adjacent hidden states is enhanced, which is crucial for accurately representing character or phoneme sequences and their corresponding Mel-spectrograms in speech synthesis tasks.

[0090] By converting acoustic features into acoustic embedding sequences using an acoustic pattern encoder, they can be made more suitable for further processing in a deep learning framework.

[0091] Step S204: perform length adjustment processing on the acoustic embedding sequence based on the phoneme duration adapter to obtain a corresponding target embedding vector.

[0092] In this embodiment, the specific implementation process of adjusting the length of the acoustic embedding sequence based on the phoneme duration adapter to obtain the corresponding target embedding vector will be further described in detail in subsequent specific embodiments of the present application and will not be elaborated on here.

[0093] Step S205: Based on the style decorator, the target embedding vector is fused with the preset target style feature to obtain a corresponding fused embedding.

[0094] In this embodiment, the above-mentioned specific implementation process of fusing the target embedding vector with the preset target style features based on the style decorator to obtain the corresponding fused embedding will be further described in detail in subsequent specific embodiments of the present application and will not be elaborated in detail here.

[0095] Step S206: performing mel-spectrogram generation processing on the fused embedding based on the mel-spectrogram encoder and the linear layer to obtain a corresponding target mel-spectrogram.

[0096] In this embodiment, the above-mentioned specific implementation process of performing Mel-spectrogram generation processing on the fused embedding based on the Mel-spectrogram encoder and the linear layer to obtain the corresponding target Mel-spectrogram will be further described in detail in subsequent specific embodiments of the present application and will not be elaborated on here.

[0097] Step S207: performing speech conversion processing on the target Mel-spectrogram based on a preset vocoder to obtain a corresponding target speech.

[0098] In this embodiment, the above-mentioned vocoder can specifically adopt a vocoder based on the Griffin-Lim algorithm to reconstruct speech audio from the mel-spectrogram. By inputting the target mel-spectrogram into the vocoder, the vocoder gradually approaches the real speech signal by iteratively updating the phase estimation. When the algorithm converges, the complex spectrum is converted back to the time domain signal using the inverse short-time Fourier transform (ISTFT) to obtain the final reconstructed speech signal, which is used as the above-mentioned target speech.

[0099] Among them, the Griffin-Lim algorithm focuses on reconstructing a phase estimate, which represents the position of each sine waveform in the Mel-spectrogram within its cycle, and is reflected in each time frame. This phase estimate is achieved through an iterative update process, based on the phase information and complex spectrum of the previous step. Through this iterative process, the Griffin-Lim algorithm is able to gradually improve the phase estimate, thereby improving the quality of the speech audio reconstructed from the Mel-spectrogram. When the Griffin-Lim algorithm converges, the final complex spectrum is calculated and then converted back to the time domain signal through the inverse short-time Fourier transform (ISTFT) to reconstruct the speech signal. This conversion process allows the original speech waveform to be recovered from the frequency domain representation.

[0100] The present application first obtains a target text to be synthesized; then performs feature conversion processing on the target text based on a conversion network in a preset target speech synthesis model to obtain corresponding acoustic features; wherein the target speech synthesis model at least includes a conversion network, an acoustic mode encoder, a phoneme duration adapter, a style decorator, a Mel-spectrogram encoder and a linear layer; then, the acoustic features are converted based on the acoustic mode encoder to obtain a corresponding acoustic embedding sequence; subsequently, the acoustic embedding sequence is length-adjusted based on the phoneme duration adapter to obtain a corresponding target embedding vector; further, based on the style decorator, the target embedding vector is fused with a preset target style feature to obtain a corresponding fused embedding; and based on the Mel-spectrogram encoder and the linear layer, a Mel-spectrogram generation processing is performed on the fused embedding to obtain a corresponding target Mel-spectrogram; finally, a speech conversion processing is performed on the target Mel-spectrogram based on a preset vocoder to obtain a corresponding target speech. The present application obtains a target text to be synthesized, then performs feature conversion on the target text based on a conversion network in a target speech synthesis model to obtain acoustic features, converts the acoustic features based on an acoustic pattern encoder to obtain an acoustic embedding sequence, adjusts the length of the acoustic embedding sequence based on a phoneme duration adapter to obtain a target embedding vector, fuses the target embedding vector with a preset target style feature based on a style decorator to obtain a fused embedding, generates a Mel-spectrogram of the fused embedding based on a Mel-spectrogram encoder and a linear layer to obtain a target Mel-spectrogram, and finally performs speech conversion on the target Mel-spectrogram based on a vocoder. This can achieve efficient and accurate generation of a natural, accurate and high-quality target speech, thereby improving the accuracy and intelligence of speech synthesis.

[0101] In some optional implementations, step S202 includes the following steps:

[0102] The target text is input into the conversion network.

[0103] In this embodiment, the above-mentioned conversion network can specifically use a pre-trained G2P model, and the G2P model can be a rule-based method, a statistical model or a deep learning model.

[0104] The target text is converted through the conversion network to obtain a corresponding phoneme sequence and style sequence.

[0105] In this embodiment, by inputting the above target text into the conversion network, the conversion network will use the grapheme-to-phoneme technology to convert the target text into corresponding phoneme sequences and style sequences, which are collectively referred to as acoustic features.

[0106] The acoustic feature is generated based on the phoneme sequence and the style sequence.

[0107] In this embodiment, the generated phoneme sequence and style sequence are collectively referred to as acoustic features. The acoustic features are the input of the subsequent acoustic model encoder.

[0108] The present application inputs the target text into the conversion network; then converts the target text through the conversion network to obtain a corresponding phoneme sequence and style sequence; and subsequently generates the acoustic features based on the phoneme sequence and the style sequence. The present application inputs the target text into the conversion network, and then converts the target text based on the use of the conversion network to obtain a corresponding phoneme sequence and style sequence, so that the corresponding acoustic features can be generated quickly and accurately based on the obtained phoneme sequence and style sequence, thereby improving the generation efficiency and generation accuracy of the acoustic features.

[0109] In some optional implementations of this embodiment, step S204 includes the following steps:

[0110] Based on the duration predictor in the phoneme duration adapter, duration prediction processing is performed on the acoustic embedding vector in the acoustic embedding sequence to obtain a corresponding duration.

[0111] In this embodiment, in human speech, the duration of each phoneme varies from sentence to sentence. A phoneme duration adapter is introduced to solve the length mismatch problem between phonemes and spectrograms. The phoneme duration adapter adjusts the length of the embedding vector or style embedding vector of each phoneme to match the length of the specified Mel-spectrogram. The phoneme duration adapter contains two core components: a duration predictor and a length regulator. The duration predictor is responsible for predicting the duration of each acoustic feature. These predicted durations are then used by the length regulator to adjust the length of each acoustic embedding vector to generate an adaptive embedding vector. Among them, the acoustic embedding vectors in the above-mentioned acoustic embedding sequence can be processed by duration prediction using the duration predictor to obtain the corresponding duration.

[0112] Gets the specified length of the preset specified Mel-spectrogram.

[0113] In this embodiment, the above-mentioned specified Mel-spectrogram refers to a Mel-spectrogram that is expected to be generated, and the corresponding specified length can be obtained by performing a length detection on the specified Mel-spectrogram.

[0114] Based on the specified length and the duration, the length of the acoustic embedding vector in the acoustic embedding sequence is adjusted using a length adjuster in the phoneme duration adapter to obtain an adjusted specified acoustic embedding vector.

[0115] In this embodiment, by using the obtained duration, the length adjuster in the phoneme duration adapter is used to adjust the length of the acoustic embedding vector in the above acoustic embedding sequence so that the acoustic embedding sequence matches the specified length of the above specified Mel-spectrogram, thereby obtaining the adjusted specified acoustic embedding vector and using it as the final target embedding vector.

[0116] The specified acoustic embedding vector is used as the target embedding vector.

[0117] In this embodiment, the adjusted target embedding vector is the input of the subsequent style decorator.

[0118] The present application performs duration prediction processing on the acoustic embedding vector in the acoustic embedding sequence based on the duration predictor in the phoneme duration adapter to obtain the corresponding duration; then obtains the specified length of the preset specified Mel-spectrogram; then, based on the specified length and the duration, uses the length adjuster in the phoneme duration adapter to adjust the length of the acoustic embedding vector in the acoustic embedding sequence to obtain the adjusted specified acoustic embedding vector; subsequently, uses the specified acoustic embedding vector as the target embedding vector. The present application performs duration prediction processing on the acoustic embedding vector in the acoustic embedding sequence based on the use of the duration predictor in the phoneme duration adapter to obtain the corresponding duration; then, according to the specified length and the preset specified length of the specified Mel-spectrogram, uses the length adjuster in the phoneme duration adapter to adjust the length of the acoustic embedding vector in the acoustic embedding sequence, thereby realizing the rapid and accurate generation of the corresponding target embedding vector, improving the generation efficiency of the target embedding vector, and ensuring the data accuracy of the obtained target embedding vector.

[0119] In some optional implementations, step S205 includes the following steps:

[0120] Based on the style decorator, a preset deep learning layer is called to convert the target style features to obtain a corresponding style embedding.

[0121] In this embodiment, the style decorator is a module including a deep learning layer. The deep data layer refers to an FFT block with an FFT function for processing style features. The target style features can be converted into corresponding style embeddings by using the deep learning layer.

[0122] Style decorators are used for style integration, or fusion, which refers to the process of combining style features with phoneme features to generate a fused embedding vector, i.e., fused embedding. Similar to the decorator pattern in object-oriented programming (OOP), this structure allows additional functionality to be dynamically added to a single object without changing the core behavior of the object. A style decorator is a structure specifically designed to integrate style features while maintaining the uniqueness of existing acoustic features. There are two goals for a style decorator: first, the adaptation process should not change existing acoustic features, such as phonemes; second, the style encoder model should be easy to add or remove so that style features can be integrated or excluded from the TTS system.

[0123] Traditionally, incorporating style features in feed-forward TTS systems, such as variance adapters, requires updating model parameters by using gradient descent on all layers. This approach is often rigid and costly, as adapting to new style features usually requires extensive model retraining.

[0124] Based on the low-rank adaptation (LoRA) used in large language models (LLM) and the style transfer method proposed in AutoVC, the pre-trained TTS system is regarded as a specific LLM, and each style feature is regarded as a decorator for downstream tasks. Here, the modification parameters used to update the model parameters can be approximated by trainable parameters. In this application, these parameters are trainable parameters within the FFT block, which are used to convert the style acoustic mode into a style embedding. During the forward propagation process, the fusion of the phoneme embedding and the style embedding can be expressed as the addition of the two. To implement the style decorator, the style feature is first transformed through an FFT block: then, the style embedding is added to the phoneme embedding to obtain the fused embedding. This structure allows the adaptation of new style features by adjusting the trainable parameters within the FFT block, without the need to retrain the entire model. In this way, the style decorator provides an efficient and flexible way to integrate style features into the TTS system while keeping the core functionality of the system unchanged. This approach not only simplifies the process of style adaptation, but also enables the system to quickly adapt to new style requirements without time-consuming model retraining.

[0125] The above strategy brings two significant advantages. Preservation of unique acoustic features: In this process, the phoneme embeddings do not change because the original model parameters W remain unchanged and only the modified parameters are trained to adapt to the new style features. Efficient adaptation: Introducing new style features into a system that already contains the original style features only requires updating the parameters related to the new style features, i.e., the modified parameters. Compared with the traditional feedforward structure, which requires comprehensive updates on all layers, including existing and new parameters, the style decorator structure simplifies this process to a way that the modified parameters adopted to update the model parameters can be approximated by trainable parameters, which is not only less resource intensive but also allows faster adaptation.

[0126] Get the preset fusion strategy.

[0127] In this embodiment, the above fusion strategy preferably adopts the strategy of addition. In other embodiments, other strategies, such as weighted addition and the like, may also be adopted.

[0128] The target embedding vector and the style embedding are fused based on the fusion strategy to obtain a corresponding fused specified embedding.

[0129] In this embodiment, the target embedding vector and the style embedding are added together according to the strategy content of the above fusion strategy to complete the corresponding fusion processing and obtain the fused embedding, that is, the above specified embedding.

[0130] The specified embedding is used as the fused embedding.

[0131] In this embodiment, the resulting fused embedding is the input to the Mel spectrogram encoder. The fused embedding is then processed through a Mel spectrogram encoder consisting of multiple layers of FFT to generate a Mel spectrogram embedding. This is followed by a linear layer that maps the dimensions of the Mel spectrogram to produce the output. The goal of the target speech synthesis model training is to minimize the mean square error (MSE) loss, ensuring that the output of the linear layer is closely aligned with the target output, which represents the Mel spectrogram of the actual speech in the frequency domain.

[0132] In this way, the style decorator provides an efficient and flexible way to integrate style features into the TTS system while keeping the core functionality of the system unchanged. This approach not only simplifies the process of style adaptation, but also enables the system to quickly adapt to new style requirements without time-consuming model retraining.

[0133] This application calls a preset deep learning layer based on the style decorator to convert the target style features to obtain the corresponding style embedding; then obtains a preset fusion strategy; then fuses the target embedding vector with the style embedding based on the fusion strategy to obtain the corresponding fused specified embedding; and subsequently uses the specified embedding as the fused embedding. This application calls a preset deep learning layer based on the use of a style decorator to convert the target style features to obtain the corresponding style embedding, and then fuses the target embedding vector with the style embedding based on the use of a fusion strategy, so that the corresponding fused embedding can be generated efficiently and accurately, which improves the construction efficiency of the fused embedding and ensures the accuracy of the obtained fused embedding.

[0134] In some optional implementations, step S206 includes the following steps:

[0135] The fused embedding is processed based on the mel-spectrogram encoder to obtain a corresponding mel-spectrogram embedding.

[0136] In this embodiment, the mel-spectrogram encoder is a deep learning model, which can be a convolutional neural network (CNN) or a Transformer. The mel-spectrogram encoder receives the fused embedding as input and outputs the corresponding mel-spectrogram embedding.

[0137] The mel-spectrogram is embedded and mapped to the dimension of the corresponding mel-spectrogram based on the linear layer to obtain a corresponding mapping result.

[0138] In this embodiment, the above mel-spectrogram is embedded and mapped to the dimension of the mel-spectrogram by using a linear layer, so as to obtain a corresponding mapping result.

[0139] The mapping result is used as the target Mel-spectrogram.

[0140] In this embodiment, the output target mel-spectrogram is the input of the subsequent vocoder.

[0141] The present application processes the fused embedding based on the Mel-spectrogram encoder to obtain the corresponding Mel-spectrogram embedding; then maps the Mel-spectrogram embedding to the dimension of the corresponding Mel-spectrogram based on the linear layer to obtain the corresponding mapping result; and subsequently uses the mapping result as the target Mel-spectrogram. The present application processes the fused embedding based on the use of a Mel-spectrogram encoder to obtain the corresponding Mel-spectrogram embedding, and then maps the Mel-spectrogram embedding to the dimension of the corresponding Mel-spectrogram based on the use of a linear layer, so that the required target Mel-spectrogram can be generated efficiently and accurately, the generation efficiency of the target Mel-spectrogram is improved, and the accuracy of the obtained target Mel-spectrogram is ensured.

[0142] In some optional implementations of this embodiment, step S201 includes the following steps:

[0143] Get the initial text to be synthesized input by the user.

[0144] In this embodiment, the initial text to be synthesized may be an initial version of text that is input by a user through a system interface and needs to be processed by speech synthesis.

[0145] The initial text is processed by removing spaces to obtain a corresponding first text.

[0146] In this embodiment, the above-mentioned space removal process refers to removing the extra spaces that may exist at the beginning and end of the text to make the text more compact. Specifically, the space removal process can be performed by using the strip() method of the string.

[0147] The first text is processed by removing line breaks to obtain a corresponding second text.

[0148] In this embodiment, the above line break removal process refers to eliminating the extra line breaks that may exist in the text to ensure that the text is a continuous string. Specifically, the replace() method of the string can be used to replace \n and \r (line break and carriage return) with an empty string.

[0149] Punctuation marks are removed from the second text to obtain a corresponding third text.

[0150] In the present embodiment, the above-mentioned punctuation mark removal process refers to removing punctuation marks that do not affect semantics and speech pauses, and only retaining those punctuation marks that may affect intonation or pauses. Specifically, a list of unnecessary punctuation marks is predefined. Then each character in the text is traversed, and if it is an unnecessary punctuation mark, it is replaced with an empty string. For punctuation marks that need to be retained (such as periods, question marks, exclamation marks, etc., which usually affect the intonation or pauses of speech), they can be marked as special characters for processing in the subsequent speech synthesis stage.

[0151] The third text is used as the target text.

[0152] The present application obtains the initial text to be synthesized input by the user; then removes spaces from the initial text to obtain a corresponding first text; then removes line breaks from the first text to obtain a corresponding second text; then removes punctuation marks from the second text to obtain a corresponding third text; and finally uses the third text as the target text. The present application obtains the initial text to be synthesized input by the user, and then removes spaces, line breaks, and punctuation marks from the initial text, thereby achieving efficient and accurate text preprocessing of the initial text, effectively improving the accuracy of the initial text and ensuring that the input initial text is clean and correctly formatted before being converted into speech.

[0153] In some optional implementations of this embodiment, before step S205, the electronic device may further perform the following steps:

[0154] The style parameters input by the specified user are obtained; wherein the style parameters at least include a speaking speed parameter and a intonation parameter.

[0155] In this embodiment, the user can input the style parameters to be set through a preset system interface, and the style parameters at least include a speech rate parameter and a tone parameter. The system interface is a pre-built interface that provides the user with the function of configuring the style parameters.

[0156] Call the preset style transfer model.

[0157] In this embodiment, the style conversion model is a pre-built model that has the function of converting the style parameters input by the user into a matching style representation. This model can be rule-based (such as linear interpolation), learning-based (such as neural network), or a combination of the two.

[0158] The style parameters are subjected to style conversion processing based on the style conversion model to obtain corresponding style representation.

[0159] In this embodiment, by inputting the above-mentioned style parameters into the style conversion model, the style conversion model will perform style conversion processing on the style parameters and output the corresponding style representation, which is the final target style feature.

[0160] The style representation is used as the target style feature.

[0161] The present application obtains the style parameters input by the specified user; wherein the style parameters include at least the speech rate parameter and the intonation parameter; then calls the preset style conversion model; then performs style conversion processing on the style parameters based on the style conversion model to obtain the corresponding style representation; and then uses the style representation as the target style feature. The present application obtains the style parameters input by the specified user. Then, based on the use of the style conversion model, the style parameters are style converted, so that the corresponding target style features can be constructed quickly and accurately, thereby improving the construction efficiency and accuracy of the target style features.

[0162] In some optional implementations, the user information obtained is subject to the user's consent and complies with relevant laws and policies.

[0163] In addition, the non-Company software tools or components that appear in the embodiments of the present application are merely examples and do not represent actual use.

[0164] In addition, an innovative text-to-speech (TTS) system proposed in this application is designed to enhance the naturalness and accuracy of synthesized speech. Based on the existing TTS technology, a unique style decorator structure is adopted, which enables the deep learning model to learn style and phoneme features at the same time, and improves the adaptability and efficiency of the model through low-rank adaptation (LoRA) technology. LoRA technology makes it possible to efficiently adjust style features in pre-trained models. Extensive testing on benchmark datasets shows that this application is significantly superior to existing methods in generating natural, accurate and high-quality speech. These advances not only expand the capabilities of current TTS systems, but also promote the application of TTS systems in more dynamic and professional fields, such as interactive virtual assistants, adaptive audiobooks, and customized voices in games.

[0165] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0166] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned target voice, the above-mentioned target voice can also be stored in a node of a blockchain.

[0167] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0168] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0169] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0170] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0171] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0172] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a speech synthesis device based on artificial intelligence, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0173] like Figure 3 As shown, the artificial intelligence-based speech synthesis device 300 described in this embodiment includes: a first acquisition module 301, a first conversion module 302, a second conversion module 303, an adjustment module 304, a fusion module 305, a generation module 306 and a third conversion module 307.

[0174] in:

[0175] The first acquisition module 301 is used to acquire the target text to be synthesized;

[0176] The first conversion module 302 is used to perform feature conversion processing on the target text based on the conversion network in the preset target speech synthesis model to obtain corresponding acoustic features; wherein the target speech synthesis model at least includes a conversion network, an acoustic mode encoder, a phoneme duration adapter, a style decorator, a mel-spectrogram encoder and a linear layer;

[0177] A second conversion module 303 is used to convert the acoustic features based on the acoustic mode encoder to obtain a corresponding acoustic embedding sequence;

[0178] An adjustment module 304 is used to perform a length adjustment process on the acoustic embedding sequence based on the phoneme duration adapter to obtain a corresponding target embedding vector;

[0179] A fusion module 305 is used to fuse the target embedding vector with a preset target style feature based on the style decorator to obtain a corresponding fused embedding;

[0180] A generating module 306 is used to perform a mel-spectrogram generation process on the fused embedding based on the mel-spectrogram encoder and the linear layer to obtain a corresponding target mel-spectrogram;

[0181] The third conversion module 307 is used to perform speech conversion processing on the target Mel-spectrogram based on a preset vocoder to obtain a corresponding target speech.

[0182] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based speech synthesis method in the aforementioned implementation, and will not be repeated here.

[0183] In some optional implementations of this embodiment, the first conversion module 302 includes:

[0184] An input submodule, used for inputting the target text into the conversion network;

[0185] A first conversion submodule, configured to convert the target text through the conversion network to obtain a corresponding phoneme sequence and style sequence;

[0186] A generating submodule is used to generate the acoustic feature based on the phoneme sequence and the style sequence.

[0187] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based speech synthesis method in the aforementioned implementation, and will not be repeated here.

[0188] In some optional implementations of this embodiment, the adjustment module 304 includes:

[0189] A prediction submodule, configured to perform duration prediction processing on the acoustic embedding vector in the acoustic embedding sequence based on the duration predictor in the phoneme duration adapter to obtain a corresponding duration;

[0190] The first acquisition submodule is used to obtain a preset specified length of the Mel-spectrogram;

[0191] An adjustment submodule, configured to adjust the length of the acoustic embedding vector in the acoustic embedding sequence using a length adjuster in the phoneme duration adapter based on the specified length and the duration, so as to obtain an adjusted specified acoustic embedding vector;

[0192] The first determination submodule is used to use the specified acoustic embedding vector as the target embedding vector.

[0193] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based speech synthesis method in the aforementioned implementation, and will not be repeated here.

[0194] In some optional implementations of this embodiment, the fusion module 305 includes:

[0195] A second conversion submodule is used to call a preset deep learning layer to convert the target style feature based on the style decorator to obtain a corresponding style embedding;

[0196] The second acquisition submodule is used to acquire a preset fusion strategy;

[0197] A fusion submodule, configured to fuse the target embedding vector with the style embedding based on the fusion strategy to obtain a corresponding fused specified embedding;

[0198] The second determining submodule is used to use the specified embedding as the fused embedding.

[0199] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based speech synthesis method in the aforementioned implementation, and will not be repeated here.

[0200] In some optional implementations of this embodiment, the generating module 306 includes:

[0201] A first processing submodule, configured to process the fused embedding based on the mel-spectrogram encoder to obtain a corresponding mel-spectrogram embedding;

[0202] A mapping submodule, used for embedding and mapping the mel-spectrogram into the dimension of the corresponding mel-spectrogram based on the linear layer to obtain a corresponding mapping result;

[0203] The third determination submodule is used to use the mapping result as the target Mel-spectrogram.

[0204] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based speech synthesis method in the aforementioned implementation, and will not be repeated here.

[0205] In some optional implementations of this embodiment, the first acquisition module 301 includes:

[0206] The second acquisition submodule is used to acquire the initial text to be synthesized input by the user;

[0207] A second processing submodule is used to remove spaces from the initial text to obtain a corresponding first text;

[0208] A third processing submodule, configured to remove line breaks from the first text to obtain a corresponding second text;

[0209] A fourth processing submodule, configured to remove punctuation marks from the second text to obtain a corresponding third text;

[0210] The fourth determination submodule is used to use the third text as the target text.

[0211] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based speech synthesis method in the aforementioned implementation, and will not be repeated here.

[0212] In some optional implementations of this embodiment, the speech synthesis device based on artificial intelligence further includes:

[0213] A second acquisition module is used to acquire style parameters input by a specified user; wherein the style parameters at least include a speech speed parameter and an intonation parameter;

[0214] The calling module is used to call the preset style transfer model;

[0215] A fourth conversion module, configured to perform style conversion processing on the style parameters based on the style conversion model to obtain a corresponding style representation;

[0216] A determination module is used to use the style representation as the target style feature.

[0217] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based speech synthesis method in the aforementioned implementation, and will not be repeated here.

[0218] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0219] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0220] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0221] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of a speech synthesis method based on artificial intelligence, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0222] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions or process data stored in the memory 41, such as computer-readable instructions for running the artificial intelligence-based speech synthesis method.

[0223] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0224] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned artificial intelligence-based speech synthesis method.

[0225] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0226] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A speech synthesis method based on artificial intelligence, characterized in that: The steps include: Obtain the target text to be synthesized; Based on the conversion network in the preset target speech synthesis model, the target text is subjected to feature conversion processing to obtain corresponding acoustic features; wherein the target speech synthesis model at least includes a conversion network, an acoustic pattern encoder, a phoneme duration adapter, a style decorator, a mel-spectrogram encoder and a linear layer; Based on the acoustic mode encoder, the acoustic feature is converted to obtain a corresponding acoustic embedding sequence; Performing a length adjustment process on the acoustic embedding sequence based on the phoneme duration adapter to obtain a corresponding target embedding vector; Based on the style decorator, the target embedding vector is fused with a preset target style feature to obtain a corresponding fused embedding; Performing mel-spectrogram generation processing on the fused embedding based on the mel-spectrogram encoder and the linear layer to obtain a corresponding target mel-spectrogram; The target Mel-spectrogram is subjected to speech conversion processing based on a preset vocoder to obtain a corresponding target speech.

2. The method for speech synthesis based on artificial intelligence according to claim 1, characterized in that: The step of performing feature conversion processing on the target text based on the conversion network in the preset target speech synthesis model to obtain corresponding acoustic features specifically includes: Inputting the target text into the conversion network; The target text is converted by the conversion network to obtain a corresponding phoneme sequence and style sequence; The acoustic feature is generated based on the phoneme sequence and the style sequence.

3. The method for speech synthesis based on artificial intelligence according to claim 1, characterized in that: The step of performing a length adjustment process on the acoustic embedding sequence based on the phoneme duration adapter to obtain a corresponding target embedding vector specifically includes: Performing duration prediction processing on the acoustic embedding vector in the acoustic embedding sequence based on the duration predictor in the phoneme duration adapter to obtain a corresponding duration; Get the specified length of the preset specified Mel spectrum graph; Based on the specified length and the duration, using a length adjuster in the phoneme duration adapter to adjust the length of the acoustic embedding vector in the acoustic embedding sequence to obtain an adjusted specified acoustic embedding vector; The specified acoustic embedding vector is used as the target embedding vector.

4. The method for speech synthesis based on artificial intelligence according to claim 1, characterized in that: The step of fusing the target embedding vector with a preset target style feature based on the style decorator to obtain a corresponding fused embedding specifically includes: Based on the style decorator, calling a preset deep learning layer to convert the target style features to obtain a corresponding style embedding; Get the preset fusion strategy; Based on the fusion strategy, the target embedding vector and the style embedding are fused to obtain a corresponding fused specified embedding; The specified embedding is used as the fused embedding.

5. The method for speech synthesis based on artificial intelligence according to claim 1, characterized in that: The step of performing mel-spectrogram generation processing on the fused embedding based on the mel-spectrogram encoder and the linear layer to obtain a corresponding target mel-spectrogram specifically includes: Processing the fused embedding based on the mel-spectrogram encoder to obtain a corresponding mel-spectrogram embedding; Embedding and mapping the mel-spectrogram into the dimension of the corresponding mel-spectrogram based on the linear layer to obtain a corresponding mapping result; The mapping result is used as the target Mel-spectrogram.

6. The method for speech synthesis based on artificial intelligence according to claim 1, characterized in that: The step of obtaining the target text to be synthesized specifically includes: Get the initial text to be synthesized input by the user; Performing space removal processing on the initial text to obtain a corresponding first text; Perform line break removal processing on the first text to obtain a corresponding second text; Performing punctuation removal processing on the second text to obtain a corresponding third text; The third text is used as the target text.

7. The method for speech synthesis based on artificial intelligence according to claim 1, characterized in that: Before the step of fusing the target embedding vector with the preset target style feature based on the style decorator to obtain the corresponding fused embedding, the method further includes: Acquire style parameters input by a specified user; wherein the style parameters at least include a speech speed parameter and an intonation parameter; Call the preset style transfer model; Performing style conversion processing on the style parameters based on the style conversion model to obtain corresponding style representation; The style representation is used as the target style feature.

8. A speech synthesis device based on artificial intelligence, characterized in that: include: A first acquisition module is used to acquire a target text to be synthesized; A first conversion module is used to perform feature conversion processing on the target text based on a conversion network in a preset target speech synthesis model to obtain corresponding acoustic features; wherein the target speech synthesis model at least includes a conversion network, an acoustic mode encoder, a phoneme duration adapter, a style decorator, a mel-spectrogram encoder and a linear layer; A second conversion module, configured to convert the acoustic features based on the acoustic mode encoder to obtain a corresponding acoustic embedding sequence; An adjustment module, configured to perform a length adjustment process on the acoustic embedding sequence based on the phoneme duration adapter to obtain a corresponding target embedding vector; A fusion module, used for fusing the target embedding vector with a preset target style feature based on the style decorator to obtain a corresponding fused embedding; A generation module, configured to perform a Mel-spectrogram generation process on the fused embedding based on the Mel-spectrogram encoder and the linear layer to obtain a corresponding target Mel-spectrogram; The third conversion module is used to perform speech conversion processing on the target Mel-spectrogram based on a preset vocoder to obtain a corresponding target speech.

9. A computer device, characterized in that: It comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the artificial intelligence-based speech synthesis method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the artificial intelligence-based speech synthesis method as described in any one of claims 1 to 7 are implemented.

Images