Speech synthesis method, device and equipment of insurance service end and medium

Abstract

The invention relates to the technical field of financial science and technology, and discloses a speech synthesis method, device and equipment for an insurance service end and a medium, and the method comprises the steps: carrying out the splicing and fusion of a target text embedding feature and an acoustic feature, and obtaining a target splicing feature; inputting the target splicing features into an encoder of a pre-trained semantic feature processing model for down-sampling operation to obtain target semantic features; inputting the target semantic feature into a pre-trained noise filtering voice reconstruction model, and outputting to obtain first target voice data; and inputting the first target voice data, the target semantic feature and the target splicing feature into a decoder of a pre-trained semantic feature processing module to obtain a target voice. According to the scheme, the problem that the experience degree of a customer is poor when the customer uses an intelligent voice customer service to carry out dialogue service due to the fact that the tone quality is easily reduced when the text-to-voice conversion technology of the existing insurance service end is used for processing complex voices or frequently switched voices is solved.

Description

Technical Field

[0001] The present invention relates to the field of financial technology, and in particular to a speech synthesis method, device, equipment and medium for an insurance business end. Background Art

[0002] For financial companies such as banks or insurance companies, almost all of them use intelligent voice customer service as a way to communicate and resolve disputes in the early stage. This method can effectively reduce personnel costs, improve communication efficiency, and reduce the work pressure of staff. Intelligent voice customer service not only greatly shortens the actual call time of manual customer service, but also processes customer needs very quickly through the corresponding human-machine intelligent integration.

[0003] At present, text-to-speech (TTS) conversion technology in the insurance business has been widely used in the field of intelligent voice customer service. Traditional TTS models are prone to deterioration in sound quality when processing complex phonology or frequently switched voices. Especially under low bit rate conditions, the spectral details and sound quality of the voice will be significantly reduced, and it is impossible to generate clear, high-quality speech. In addition, the existing TTS system has limited ability to model the details of phonology, intonation, and emotion, resulting in monotonous and mechanical generated speech.

[0004] Autoregressive models (such as Tacotron and Transformer TTS) have achieved some success in speech naturalness, but they are prone to audio incoherence or speech distortion when generating long segments of speech. The generation process of the autoregressive model depends on the output of the previous step, which leads to error accumulation and affects the stability of the generation. In addition, for diverse voice styles or emotional changes, the performance of traditional models is still not ideal.

[0005] To sum up, current speech synthesis technology is developing rapidly, but the naturalness of the speech synthesized by related speech synthesis technology is not very ideal, the ability to process complex phonetics is limited and the reasoning efficiency is low, which leads to a poor experience for customers when using intelligent voice customer service for conversational services. Summary of the invention

[0006] The present invention provides a speech synthesis method, device, equipment and medium for an insurance business end, so as to solve the problem that the existing text-to-speech conversion technology for the insurance business end easily leads to a decrease in sound quality when processing complex phonetics or frequently switched voices, thereby causing a poor experience for customers when using intelligent voice customer service for conversational services.

[0007] In a first aspect, a speech synthesis method for an insurance business end is provided, comprising: Obtaining a target text and a reference audio to be synthesized from an insurance business end, encoding the target text to obtain a target text embedding feature, and encoding the reference audio to obtain a target acoustic feature; The target text embedding feature and the target acoustic feature are concatenated to obtain a target concatenated feature; the target concatenated feature is input into an encoder of a pre-trained semantic feature processing model, and the target concatenated feature is down-sampled to obtain a target semantic feature; the target semantic feature is input into a pre-trained noise-filtered speech reconstruction model to obtain first target speech data; The first target speech data, target semantic features and target concatenation features are input into a decoder of a pre-trained semantic feature processing model, and second target speech data is obtained based on the first target speech data and the target semantic features; and a target speech corresponding to the target text is obtained based on the second target speech data and the target concatenation features.

[0008] In a second aspect, a speech synthesis device for an insurance business terminal is provided, comprising: An acquisition module is used to acquire the target text and reference audio to be synthesized from the insurance business end, encode the target text to obtain the target text embedding feature, and encode the reference audio to obtain the target acoustic feature; An encoding module is used to concatenate the target text embedding feature and the target acoustic feature to obtain a target concatenated feature; input the target concatenated feature into an encoder of a pre-trained semantic feature processing model, and downsample the target concatenated feature to obtain a target semantic feature; input the target semantic feature into a pre-trained noise-filtered speech reconstruction model to obtain first target speech data; A decoding module is used to input the first target speech data, target semantic features and target splicing features into a decoder of a pre-trained semantic feature processing model, obtain second target speech data according to the first target speech data and the target semantic features; and obtain a target speech corresponding to the target text according to the second target speech data and the target splicing features.

[0009] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the speech synthesis method of the above-mentioned insurance business end when executing the computer program.

[0010] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the speech synthesis method of the above-mentioned insurance business end are implemented.

[0011] In the scheme implemented by the speech synthesis method, device, computer equipment and storage medium of the above-mentioned insurance business end, the target splicing feature can be obtained by splicing and fusing the target text embedding feature with the acoustic feature; the target splicing feature is input into the encoder of a pre-trained semantic feature processing model for downsampling operation to obtain the target semantic feature; the target semantic feature is input into a pre-trained noise filtering speech reconstruction model, and the first target speech data is output; the first target speech data, the target semantic feature and the target splicing feature are input into the decoder of the pre-trained semantic feature processing model to obtain the target speech.

[0012] The present invention performs downsampling operation through the encoder of the semantic feature processing model, and while extracting semantic features, the dimension of the input data can be reduced, so that the data dimension in the subsequent input noise filtering speech reconstruction model can be reduced, the calculation complexity can be reduced, and the speech generation efficiency can be improved; at the same time, the first target speech data, the target semantic features and the target splicing features are grouped and spliced ​​and fused, and the semantic features extracted in the early stage are integrated into the speech generation process. This structural design ensures that both the global information and local details in the speech signal can be retained and restored. Thus, the model can capture the speech details and emotional clues in the text during the speech generation process. In particular, when processing complex phonology, emotional changes or intonation fluctuations, speech with higher fidelity and natural expression can be generated. Therefore, the scheme of the present invention solves the problem that the existing text-to-speech (TTS) conversion technology is prone to cause sound quality degradation when processing complex phonology or frequently switched speech, thereby causing poor user experience when using it. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0014] Figure 1 It is a schematic diagram of an application environment of a speech synthesis method for an insurance business terminal in one embodiment of the present invention; Figure 2 It is a flow chart of a speech synthesis method of an insurance business terminal in one embodiment of the present invention; Figure 3 It is a flowchart of a specific implementation method of a speech synthesis method for an insurance business terminal in one embodiment of the present invention; Figure 4 It is a structural schematic diagram of a speech synthesis device at an insurance business end in one embodiment of the present invention; Figure 5 is a schematic diagram of a structure of a computer device in one embodiment of the present invention; Figure 6 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] The speech synthesis method for the insurance business terminal provided by the embodiment of the present invention can be applied in Figure 1 In an application environment, the application environment includes a client and a server, wherein the client can communicate with the server through a network. Specifically, the server obtains the target text to be synthesized and the reference audio of the insurance business end, encodes the target text to obtain the target text embedding feature, encodes the reference audio to obtain the target acoustic feature; splices the target text embedding feature with the target acoustic feature to obtain the target splicing feature; inputs the target splicing feature into the encoder of the pre-trained semantic feature processing model, downsamples the target splicing feature to obtain the target semantic feature; inputs the target semantic feature into the pre-trained noise filtering speech reconstruction model to obtain the first target speech data; inputs the first target speech data, the target semantic feature and the target splicing feature into the decoder of the pre-trained semantic feature processing model, obtains the second target speech data according to the first target speech data and the target semantic feature; obtains the target speech corresponding to the target text according to the second target speech data and the target splicing feature, and sends the target speech to the terminal client. Among them, the client is also called the user end, which refers to a program corresponding to the server and providing local services to customers. The client can be installed on but not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server can be implemented as an independent server or a server cluster consisting of multiple servers.

[0017] See also Figure 2 As shown, Figure 2 A flow chart of a speech synthesis method for an insurance business terminal provided by an embodiment of the present invention includes the following steps: S01: Obtain the target text and reference audio to be synthesized from the insurance business end, encode the target text to obtain target text embedding features, encode the reference audio to obtain target acoustic features.

[0018] The target text to be synthesized at the insurance business end is the text that needs to be synthesized. For example, in scenarios such as intelligent voice customer service and voice newspaper reading for financial companies such as banks or insurance companies, text needs to be converted into voice.

[0019] The reference audio is the audio of the "speaker", that is, the audio to be synthesized in the scenario of the insurance company's intelligent voice customer service has the same timbre as the synthesized voice, or the timbre of the synthesized voice is infinitely close to the audio to be synthesized. In this way, the quality of speech synthesis in the scenario of the insurance company's intelligent voice customer service can be improved, thereby improving customer satisfaction.

[0020] The target text embedding features include text features such as phonemes, words, phrases, and sentence divisions. Phonemes are the smallest sound units that make up a language and can be represented by letters or phonetic symbols. For example, in Chinese characters, a syllable may consist of an initial consonant, a final vowel, and a tone.

[0021] The target acoustic features include sound intensity, sound duration, pitch, and sound quality. Pitch refers to the change in the height of the sound, sound duration refers to the duration of the sound, and sound intensity refers to the change in the loudness of the sound. These parameters together constitute the prosodic characteristics of the language and help express emotions, attitudes, and the structure of the language.

[0022] In this embodiment, PyTorch is used to encode the target text when acquiring the embedding features of the target text. PyTorch is a Python-based scientific computing library that is mainly used in the fields of deep learning and machine learning. It was developed by Facebook's AI Research Lab (FAIR) and is known for its flexibility, ease of use, and strong community support. nnn.Module is the basic class for defining neural network models in PyTorch. Users can inherit nn.Module and define their own models. In the class constructor, the layer structure and parameters of the network can be defined.

[0023] In PyTorch, it is a common practice to use torch.nn.Embedding module class and Transformer module class to build text embedding vectors, especially in natural language processing (NLP) tasks. The Embedding module class is used to convert discrete word or character indexes into continuous vector representations, while the Transformer module class is a powerful sequence processing architecture that can capture long-term dependencies in text.

[0024] In this embodiment, the torch.nn.Embedding module class and the Transformer module class in the PyTorch library are used to implement text embedding feature extraction. The specific steps are as follows: First, build a character set containing all possible Chinese characters. This character set should contain all the Chinese characters you want to process in the model.

[0025] Then, assign a unique index to each Kanji character, which can be achieved by creating a dictionary or using other data structures; convert the input data (such as text) into a sequence of these indices, which usually involves iterating over the text, finding the position of each character in the character set, and replacing it with the corresponding index.

[0026] Next, create an Embedding module class whose input dimension is equal to the number of characters in the character set (that is, the maximum value of the index plus 1, because the index usually starts at 0), and the output dimension is the dimension of the embedding vector.

[0027] Finally, by setting the number of layers of the encoder in the Transformer module class to one or more layers, the embedding vector obtained by the Embedding module class is processed to obtain the final target text embedding features.

[0028] The present embodiment does not include the Transformer model training part, so this is not the focus of the present invention. In practical applications, it is necessary to define a loss function, an optimizer, and iterate on the training data set to update the model parameters.

[0029] In this embodiment, the acoustic features do not use a model, but directly use the short-time Fourier transform (STFT) to extract the Mel spectrum of the reference audio to obtain the target acoustic features. This is a common step in audio signal processing, especially in tasks such as speech recognition, music information retrieval, and sound classification. The Mel spectrum is a spectrum representation based on the auditory characteristics of the human ear. It better simulates the auditory perception of the human ear by nonlinearly compressing the frequency (i.e., Mel scale transform). Specifically, the basic steps for extracting the Mel spectrum of the reference audio are as follows: (1) Load the reference audio file and obtain the time series (waveform) of the reference audio.

[0030] (2) Apply short-time Fourier transform (STFT) to the reference audio waveform to obtain a spectrogram (a complex matrix in which each element represents the complex amplitude in frequency and time).

[0031] (3) Convert the spectrogram from a linear frequency scale to a Mel frequency scale. This typically involves convolving the spectrogram with a set of Mel filter banks.

[0032] (4) Apply a logarithmic transform to the Mel spectrum to obtain the logarithmic Mel spectrum (Mel spectrum for short), which is the target acoustic feature of the reference audio. The logarithmic transform helps to compress the dynamic range and make the data more suitable for subsequent processing.

[0033] In addition, the MelPu can be normalized, truncated, windowed, etc. as needed. If necessary, the MelPu can also be converted to other formats, such as MelPu frequency cepstral coefficients (MFCCs), which are obtained by applying discrete cosine transform (DCT).

[0034] S02: Concatenate the target text embedding feature and the target acoustic feature to obtain a target concatenated feature; input the target concatenated feature into an encoder of a pre-trained semantic feature processing model, and downsample the target concatenated feature to obtain a target semantic feature; input the target semantic feature into a pre-trained noise filtering speech reconstruction model to obtain first target speech data.

[0035] Among them, Figure 3 As shown, the semantic feature processing model in this embodiment is a U-Net model. The U-Net model was originally designed for image segmentation tasks. Its structure includes an encoder and a decoder, allowing the model to learn the mapping from input data to output data. However, with the development of technology, the idea of ​​the U-Net architecture has also been extended to other fields, including speech processing and natural language processing.

[0036] The U-Net model is named because its network architecture has a symmetrical encoder-decoder structure that resembles the letter "U". The encoder part extracts the features of the input data through a series of convolutional layers and downsampling (usually maximum pooling) operations. The decoder part gradually restores the dimensions of the data through a series of convolutional layers and upsampling (usually transposed convolution or bilinear interpolation) operations, and finally generates target data with the same dimension size as the input data. In U-Net, a unique feature is the use of skip connections. These connections pass the feature data from the encoder directly to the corresponding layer in the decoder, which helps to retain the detailed information of the input data and improve the accuracy of the output results of the current layer.

[0037] The noise-filtered speech reconstruction model in this embodiment is a diffusion model based on the transformer model in natural language processing, called the Diffusion Transformer (DiTs) model. It is a neural network structure based on deep learning. It is named because its core idea is to convert the input data into a form that is easier to process through the diffusion process, and then use the transformer model for efficient feature extraction and modeling. This structure has significant advantages in processing sequence data (such as text, speech, etc.), and can capture long-term dependencies in the data and generate high-quality output Specifically, the present invention inputs the target text embedding feature into the encoder of the U-Net model, and the encoder performs downsampling processing on the target splicing feature to obtain the target semantic feature; the target semantic feature is input into the pre-trained DiTs model to obtain the first target speech data. The DiTs model processes the target semantic feature as follows: The DiTs model first preprocesses the target semantic features (input data) through a diffusion process. This process aims to map the data from the original space to a higher-dimensional latent space to reveal the potential structure in the data. By adding an appropriate amount of noise, the diffusion process can enhance the robustness of the model and enable it to better cope with the uncertainty in actual situations.

[0038] After the diffusion process, the diffused data is input into the transformer model. The transformer model is composed of multiple self-attention mechanism layers, which can effectively capture the long-term dependencies in the data. Through the self-attention mechanism, the DiTs model can dynamically assign attention weights according to different parts of the input sequence (target semantic features), thereby accurately capturing key information.

[0039] In the final stage of the transformer model, the model generates a representation vector containing rich feature information. This vector can be further used for speech generation tasks, and a decoding process is required to convert the representation vector back to the original data (target semantic feature) space to obtain the first target speech data.

[0040] S03: Input the first target speech data, target semantic features and target concatenation features into a decoder of a pre-trained semantic feature processing model, and obtain second target speech data based on the first target speech data and the target semantic features; and obtain a target speech corresponding to the target text based on the second target speech data and the target concatenation features.

[0041] Specifically, the first target speech data, target semantic features and target splicing features are input into the decoder of a pre-trained U-Net model. The number of layers of the decoder is the same as the number of layers of the encoder. In this embodiment, it is set as the first decoding layer and the output layer. After the first target speech data and the target semantic features are input into the first decoding layer, the second target speech data is obtained. At this time, the data dimension of the second target speech data is the same as the data dimension of the target splicing features. After the second target speech data and the target splicing features of the same dimension are input into the output layer, the target speech corresponding to the original target text is obtained.

[0042] This application takes the scenario of intelligent voice customer service of an insurance company as an example to illustrate, and applies intelligent voice customer service to the above-mentioned steps S01, S02 and S03. Specifically, the text that needs to be converted into speech by the intelligent voice customer service is obtained as the target text to be synthesized, and the speech audio file that needs to be converted into the required timbre by the intelligent voice customer service is obtained as the reference audio, that is, the timbre of the synthesized speech needs to be infinitely close to the reference audio, the target text is encoded to obtain the target text embedding feature, and the reference audio is encoded to obtain the target acoustic feature; the target text embedding feature and the target acoustic feature are spliced ​​to obtain the target splicing feature; the target splicing feature is input into the encoder of a pre-trained U-Net model, and the target splicing feature is down-sampled to obtain the target semantic feature; the target semantic feature is input into the pre-trained diffusion transformer model to obtain the first target speech data; the first target speech data, the target semantic feature and the target splicing feature are input into the decoder of the pre-trained U-Net model, and the second target speech data is obtained according to the first target speech data and the target semantic feature; the target speech corresponding to the target text is obtained according to the second target speech data and the target splicing feature.

[0043] The target speech corresponding to the target text generated above will also be optimized through the post-processing module, which mainly includes noise removal, spectrum balance adjustment, pitch correction and other processing to ensure that the final output speech is clear and natural and conforms to the speech characteristics of the target text.

[0044] The present invention performs downsampling operation through the encoder of the semantic feature processing model, and while extracting semantic features, the dimension of the input data can be reduced, so that the data dimension in the subsequent input noise filtering speech reconstruction model can be reduced, the calculation complexity can be reduced, and the speech generation efficiency can be improved; at the same time, the first target speech data, the target semantic features and the target splicing features are grouped and spliced ​​and fused, and the semantic features extracted in the early stage are integrated into the speech generation process. This structural design ensures that both the global information and local details in the speech signal can be retained and restored. Thereby, the model can capture the speech details and emotional clues in the text during the speech generation process. In particular, when processing complex phonology, emotional changes or intonation fluctuations, speech with higher fidelity and natural expression can be generated.

[0045] In one embodiment, the step of concatenating the target text embedding feature with the target acoustic feature to obtain a target concatenated feature includes: The data dimension of the first target speech data is set to be the same as that of the target semantic feature; the data dimension includes the number of rows and columns of data.

[0046] The target acoustic feature is concatenated with the last column of the target text embedding feature to obtain the target concatenated feature.

[0047] Specifically, in this embodiment, the first target speech data and the target semantic features are both two-dimensional vectors, and the same data dimension means that the number of rows of the first target speech data is equal to the number of rows of the target semantic features, and the number of columns of the first target speech data is equal to the number of columns of the target semantic features. For example, if the data dimension of the first target speech data is two-dimensional data of 128 rows * 256 columns, then the data dimension of the target semantic features must also be two-dimensional data of 128 rows * 256 columns. In the last column of the target text embedding feature with a dimension of 128 rows * 256 columns, the target acoustic feature with a splicing dimension of 128 rows * 256 columns will obtain a target splicing feature with a dimension of 128 rows * 256 columns.

[0048] In one embodiment, obtaining the second target speech data according to the first target speech data and the target semantic feature includes: The data dimension of the first target speech data is set to be the same as that of the target semantic feature.

[0049] The first target speech data and the target semantic features are concatenated and fused in the channel dimension to obtain a first fused feature, and the first fused feature is sequentially convolved and up-sampled to obtain the second target speech data.

[0050] Among them, concatenation fusion in channel dimension refers to concatenating feature maps from different sources or different convolutional layers in a certain layer of a deep neural network in the channel dimension to form a new feature map with more channels.

[0051] Specifically, in this embodiment, the first target speech data and the target semantic features are spliced ​​and fused in the channel dimension. Before splicing, the first target speech data and the target semantic features need to be preprocessed into the same data dimension, and then they can be spliced ​​and fused to obtain the first fused feature. The number of channels of the first fused feature after splicing increases, which is equivalent to increasing the depth of the network, which helps the network learn more complex feature representations. Next, the first fused feature is convolved to extract the correlation between features through convolution operations, thereby enhancing the feature extraction ability of the network. This method can introduce more nonlinear changes in the feature map after splicing and improve the expression ability of the network. Finally, the dimension of the data after the above convolution processing is increased through upsampling until it is the same as the data dimension of the target splicing feature, and the second target speech data is obtained.

[0052] In one embodiment, obtaining a target speech corresponding to the target text according to the second target speech data and the target splicing feature includes: The data dimension of the second target speech data is set to be the same as that of the target concatenated feature.

[0053] The second target speech data is concatenated and fused with the target concatenated feature in the channel dimension to obtain a second fused feature, and the second fused feature is convolved to obtain a target speech corresponding to the target text.

[0054] Specifically, in this embodiment, the second target speech data and the target splicing feature are spliced ​​and fused in the channel dimension. Before splicing, the second target speech data and the target splicing feature need to be preprocessed into the same data dimension, and then the second fused feature can be obtained after splicing and fusion. The number of channels of the second fused feature after splicing increases, which is equivalent to increasing the depth of the network, which helps the network learn more complex feature representations. Next, the second fused feature is convoluted to obtain the target speech corresponding to the target text, and the correlation between the features is extracted through the convolution operation, thereby enhancing the feature extraction ability of the network. This method can introduce more nonlinear changes in the spliced ​​feature map and improve the expression ability of the network.

[0055] In one embodiment, the encoder model includes a first conversion encoder and a pooling layer. The target splicing feature is input into the first conversion encoder, and the semantic feature in the target splicing feature is obtained through feature extraction of the attention mechanism in the first conversion encoder; after the semantic feature is input into the pooling layer for downsampling operation, the target semantic feature is output.

[0056] Specifically, the encoder model in this embodiment is a vision transformer (ViT for short). The ViT model is a deep learning model that applies the Transformer architecture to image recognition tasks. The ViT model is used to capture complex associations in text information. ViT can effectively model long-distance dependencies through the self-attention mechanism, thereby helping the model to better understand the relationship between words in the text. The first conversion encoder is the Transformer encoder in the ViT model.

[0057] The target concatenation features are input into the ViT model. The ViT model structure treats the target concatenation features (embedded text) as data similar to "image blocks". After being processed by the Transformer encoder, more global and local semantic features are extracted to obtain the semantic features in the target concatenation features. This processing method can capture the rhythm, tone and emotional changes in the text, ensuring that the generated speech has more natural intonation and rhythm.

[0058] The pooling layer is a layer often used in convolutional neural networks. Its principle is to downsample the input feature map according to certain rules to obtain a feature map with a smaller spatial size. The pooling operation methods include the following: 1) Max Pooling: Select the largest value in each local area (pooling window) of the feature map as the output of the area. It can extract the most significant features and has strong robustness to noise and slight deformation. The commonly used max pooling operation is to use a 2x2 pooling window and a stride of 2.

[0059] 2) Average Pooling: Calculate the average value in each local area (pooling window) of the feature map as the output of the area. It can smooth the feature map and retain the background information. You can also use a 2x2 pooling window and a stride of 2.

[0060] Specifically, the semantic features in the target splicing features are input into the pooling layer for downsampling operation. In this embodiment, the semantic features in the target splicing features are downsampled by the maximum pooling method to obtain the target semantic features. At this time, the target semantic features have a feature map with a smaller spatial size and more significant features. This operation plays an important role in convolutional neural networks and helps to improve the performance and efficiency of the model.

[0061] In one embodiment, inputting the target semantic feature into a pre-trained noise-filtered speech reconstruction model includes: The noise-filtered speech reconstruction model includes a second conversion coder and a third conversion coder.

[0062] After adding preset noise data to the target semantic feature, a first noise target semantic feature is obtained.

[0063] The first noise target semantic feature is input into a second transformation encoder, and feature extraction is performed through an attention mechanism in the second transformation encoder to obtain a second noise target semantic feature.

[0064] After adding preset noise data to the second noise target semantic feature, a third noise target semantic feature is obtained.

[0065] The third noise target semantic feature is input into a third transformation encoder, and feature extraction is performed through an attention mechanism in the third transformation encoder to obtain a fourth noise target semantic feature.

[0066] After removing all preset noise data from the fourth noise target semantic feature, the first target speech data is obtained.

[0067] Among them, the noise filtering speech reconstruction model is a diffusion transformer, wherein the diffusion transformer includes a second conversion encoder and a third conversion encoder, and both encoders are transformer models. Each transformer model is stacked by multiple self-attention mechanism layers, which can effectively capture long-term dependencies in the data. Through the self-attention mechanism, the DiTs model can dynamically allocate attention weights according to different parts of the input sequence (target semantic features), thereby achieving accurate capture of key information. In this embodiment, the input data is captured multiple times through a two-layer transformer model, so that the key information in the input data can be further accurately captured.

[0068] Specifically, first, the target semantic feature is passed through the diffusion process in the DiTs model, and an appropriate amount of noise data is added to obtain the first noisy target semantic feature; the first noisy target semantic feature is input into the first transformer model (the second transformation encoder), and the feature is extracted through the attention mechanism in the transformer model to obtain the second noisy target semantic feature.

[0069] Next, the second noise target semantic feature is passed through the diffusion process in the DiTs model, and an appropriate amount of noise data is added to obtain the third noise target semantic feature; the third noise target semantic feature is input into the second transformer model (the third transformation encoder), and the feature is extracted through the attention mechanism in the transformer model to obtain the fourth noise target semantic feature.

[0070] Finally, the fourth noise target semantic feature is decoded by a corresponding decoder in the DiTs model to obtain the first target speech data.

[0071] In one embodiment, the step of concatenating and fusing the first target speech data and the target semantic features in the channel dimension includes: The target semantic features include rhythmic features, intonation features and emotional change features in the text; Allocating a rhythm weight parameter to the rhythm feature, allocating a tone weight parameter to the tone feature, and allocating an emotion weight parameter to the emotion change feature; The prosodic feature, the tonal feature and the emotional change feature are weighted according to the prosodic weight parameter, the tonal weight parameter and the emotional weight parameter to obtain a weighted target semantic feature; The first target speech data and the weighted target semantic features are concatenated and fused in the channel dimension.

[0072] Specifically, the target semantic features extracted in this embodiment include the prosodic features, tone features and emotional change features in the text, and a preset weight parameter is assigned to each of the prosodic features, tone features and emotional change features, for example, the prosodic weight parameters assigned to each of the prosodic features, tone features and emotional change features by expert experience data are: , and sentiment weight parameter ,use The rhythm feature data is weighted and The pitch features are weighted and After weighted processing of the emotion change features, the weighted target semantic features are obtained. The first target speech data and the weighted target semantic features are spliced ​​and fused in the channel dimension. Before splicing, the first target speech data and the weighted target semantic features need to be preprocessed into the same data dimension before splicing and fusion can be performed. The increase in the number of channels of the spliced ​​data features is equivalent to increasing the depth of the network, which helps the network learn more complex feature representations.

[0073] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0074] In one embodiment, a speech synthesis device for an insurance business terminal is provided, and the speech synthesis device for the insurance business terminal corresponds one-to-one to the speech synthesis method for the insurance business terminal in the above embodiment. Figure 4 As shown, the intelligent question-answering processing device includes an acquisition module 101, an encoding module 102 and a decoding module 103. Each functional module is described in detail as follows: The acquisition module 101 is used to acquire the target text and reference audio to be synthesized from the insurance business end, encode the target text to obtain the target text embedding feature, and encode the reference audio to obtain the target acoustic feature.

[0075] The encoding module 102 is used to concatenate the target text embedding feature and the target acoustic feature to obtain a target concatenated feature; input the target concatenated feature into an encoder of a pre-trained semantic feature processing model, and downsample the target concatenated feature to obtain a target semantic feature; input the target semantic feature into a pre-trained noise filtering speech reconstruction model to obtain the first target speech data.

[0076] The decoding module 103 is used to input the first target speech data, target semantic features and target concatenation features into a decoder of a pre-trained semantic feature processing model, obtain second target speech data according to the first target speech data and the target semantic features; and obtain a target speech corresponding to the target text according to the second target speech data and the target concatenation features.

[0077] In one embodiment, the determination module 102 is specifically configured to: The data dimension of the first target speech data is set to be the same as that of the target semantic feature; the data dimension includes the number of rows and columns of data.

[0078] The target acoustic feature is concatenated with the last column of the target text embedding feature to obtain the target concatenated feature.

[0079] In one embodiment, the determination module 103 is further configured to: The data dimension of the first target speech data is set to be the same as that of the target semantic feature.

[0080] The first target speech data and the target semantic features are concatenated and fused in the channel dimension to obtain a first fused feature, and the first fused feature is sequentially convolved and up-sampled to obtain the second target speech data.

[0081] In one embodiment, the determination module 102 is further configured to: The encoder model includes a first conversion encoder and a pooling layer. The target splicing feature is input into the first conversion encoder, and the semantic feature in the target splicing feature is obtained through feature extraction of the attention mechanism in the first conversion encoder; after the semantic feature is input into the pooling layer for downsampling operation, the target semantic feature is output.

[0082] In one embodiment, the recall module 103 is specifically configured to: The data dimension of the second target speech data is set to be the same as that of the target concatenated feature.

[0083] The second target speech data and the target splicing feature are spliced ​​and fused in the channel dimension to obtain a second fused feature, and the second fused feature is convolved to obtain the corresponding target speech.

[0084] In one embodiment, the sorting module 102 is specifically configured to: The noise-filtered speech reconstruction model includes a second conversion coder and a third conversion coder.

[0085] After adding preset noise data to the target semantic feature, a first noise target semantic feature is obtained.

[0086] The first noise target semantic feature is input into a second transformation encoder, and feature extraction is performed through an attention mechanism in the second transformation encoder to obtain a second noise target semantic feature.

[0087] After adding preset noise data to the second noise target semantic feature, a third noise target semantic feature is obtained.

[0088] The third noise target semantic feature is input into a third transformation encoder, and feature extraction is performed through an attention mechanism in the third transformation encoder to obtain a fourth noise target semantic feature.

[0089] After removing all preset noise data from the fourth noise target semantic feature, the first target speech data is obtained.

[0090] In one embodiment, the sorting module 103 is specifically configured to: The target semantic features include rhythmic features, intonation features and emotional change features in the text; Allocating a rhythm weight parameter to the rhythm feature, allocating a tone weight parameter to the tone feature, and allocating an emotion weight parameter to the emotion change feature; The prosodic feature, the tonal feature and the emotional change feature are weighted according to the prosodic weight parameter, the tonal weight parameter and the emotional weight parameter to obtain a weighted target semantic feature; The first target speech data and the weighted target semantic features are concatenated and fused in the channel dimension.

[0091] The present invention provides a speech synthesis device for an insurance business end. Through the encoder of the semantic feature processing model, downsampling operation is performed. While extracting semantic features, the dimension of the input data can be reduced, so that the data dimension in the subsequent input noise filtering speech reconstruction model can be reduced, the calculation complexity can be reduced, and the speech generation efficiency can be improved; at the same time, the first target speech data, the target semantic features and the target splicing features are grouped, spliced ​​and fused, and the semantic features extracted in the early stage are integrated into the speech generation process. This structural design ensures that both the global information and local details in the speech signal can be retained and restored. Thereby, the model can capture the speech details and emotional clues in the text during the speech generation process. In particular, when processing complex phonology, emotional changes or intonation fluctuations, speech with higher fidelity and natural expression can be generated.

[0092] For the specific definition of the speech synthesis device of the insurance business end, please refer to the definition of the speech synthesis method of the insurance business end in the above text, which will not be repeated here. Each module in the above-mentioned speech synthesis device of the insurance business end can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0093] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it realizes the functions or steps of a speech synthesis method service end of an insurance business end.

[0094] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, the functions or steps on the client side of a speech synthesis method of an insurance business end are realized.

[0095] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: The target text and reference audio to be synthesized at the insurance business end are obtained, the target text is encoded to obtain the target text embedding feature, and the reference audio is encoded to obtain the target acoustic feature.

[0096] The target text embedding feature and the target acoustic feature are concatenated to obtain a target concatenated feature; the target concatenated feature is input into an encoder of a pre-trained semantic feature processing model, and the target concatenated feature is down-sampled to obtain a target semantic feature; the target semantic feature is input into a pre-trained noise filtering speech reconstruction model to obtain the first target speech data.

[0097] The first target speech data, target semantic features and target concatenation features are input into a decoder of a pre-trained semantic feature processing model, and second target speech data is obtained based on the first target speech data and the target semantic features; and a target speech corresponding to the target text is obtained based on the second target speech data and the target concatenation features.

[0098] In one embodiment, a computer readable storage medium is provided on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: The target text and reference audio to be synthesized at the insurance business end are obtained, the target text is encoded to obtain the target text embedding feature, and the reference audio is encoded to obtain the target acoustic feature.

[0099] The target text embedding feature and the target acoustic feature are concatenated to obtain a target concatenated feature; the target concatenated feature is input into an encoder of a pre-trained semantic feature processing model, and the target concatenated feature is down-sampled to obtain a target semantic feature; the target semantic feature is input into a pre-trained noise filtering speech reconstruction model to obtain the first target speech data.

[0100] The first target speech data, target semantic features and target concatenation features are input into a decoder of a pre-trained semantic feature processing model, and second target speech data is obtained based on the first target speech data and the target semantic features; and a target speech corresponding to the target text is obtained based on the second target speech data and the target concatenation features.

[0101] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0102] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0103] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0104] The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein by equivalents; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should be included in the protection scope of the present invention. The non-company software tools or components that appear in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A speech synthesis method for an insurance business terminal, characterized in that: include: Obtaining a target text and a reference audio to be synthesized from an insurance business end, encoding the target text to obtain a target text embedding feature, and encoding the reference audio to obtain a target acoustic feature; The target text embedding feature and the target acoustic feature are concatenated to obtain a target concatenated feature; the target concatenated feature is input into an encoder of a pre-trained semantic feature processing model, and the target concatenated feature is down-sampled to obtain a target semantic feature; the target semantic feature is input into a pre-trained noise-filtered speech reconstruction model to obtain first target speech data; The first target speech data, target semantic features and target concatenation features are input into a decoder of a pre-trained semantic feature processing model, and second target speech data is obtained based on the first target speech data and the target semantic features; and a target speech corresponding to the target text is obtained based on the second target speech data and the target concatenation features.

2. The speech synthesis method for insurance business terminal according to claim 1, characterized in that: The step of concatenating the target text embedding feature with the target acoustic feature to obtain a target concatenated feature includes: Setting the data dimension of the first target speech data to be the same as that of the target semantic feature; the data dimension includes the number of rows and columns of the data; The target acoustic feature is concatenated with the last column of the target text embedding feature to obtain the target concatenated feature.

3. The speech synthesis method for insurance business terminal according to claim 1, characterized in that: The obtaining of second target speech data according to the first target speech data and the target semantic feature includes: Setting the data dimension of the first target speech data and the target semantic feature to be the same; The first target speech data and the target semantic features are concatenated and fused in the channel dimension to obtain a first fused feature, and the first fused feature is sequentially convolved and up-sampled to obtain the second target speech data.

4. The speech synthesis method for insurance business terminal according to claim 1, characterized in that: The step of obtaining a target speech corresponding to the target text according to the second target speech data and the target splicing feature includes: Setting the data dimension of the second target speech data to be the same as that of the target concatenated feature; The second target speech data and the target splicing feature are spliced ​​and fused in the channel dimension to obtain a second fused feature, and the second fused feature is convolved to obtain the corresponding target speech.

5. The speech synthesis method for insurance business terminal according to claim 1, characterized in that: The encoder model includes a first conversion encoder and a pooling layer. The target splicing feature is input into the first conversion encoder, and the semantic feature in the target splicing feature is obtained through feature extraction of the attention mechanism in the first conversion encoder; after the semantic feature is input into the pooling layer for downsampling operation, the target semantic feature is output.

6. The speech synthesis method for insurance business terminal as claimed in claim 2, characterized in that: Inputting the target semantic features into a pre-trained noise-filtered speech reconstruction model includes: The noise-filtered speech reconstruction model includes a second conversion encoder and a third conversion encoder; After adding preset noise data to the target semantic feature, a first noise target semantic feature is obtained; Inputting the first noise target semantic feature into a second transformation encoder, performing feature extraction through an attention mechanism in the second transformation encoder, and obtaining a second noise target semantic feature; After adding preset noise data to the second noise target semantic feature, a third noise target semantic feature is obtained; Inputting the third noise target semantic feature into a third transformation encoder, performing feature extraction through an attention mechanism in the third transformation encoder, and obtaining a fourth noise target semantic feature; After removing all preset noise data from the fourth noise target semantic feature, the first target speech data is obtained.

7. The speech synthesis method for insurance business terminal according to claim 1, characterized in that: The step of concatenating and fusing the first target speech data and the target semantic features in the channel dimension includes: The target semantic features include rhythmic features, intonation features and emotional change features in the text; Allocating a rhythm weight parameter to the rhythm feature, allocating a tone weight parameter to the tone feature, and allocating an emotion weight parameter to the emotion change feature; The prosodic feature, the tonal feature and the emotional change feature are weighted according to the prosodic weight parameter, the tonal weight parameter and the emotional weight parameter to obtain a weighted target semantic feature; The first target speech data and the weighted target semantic features are concatenated and fused in the channel dimension.

8. A speech synthesis device for an insurance business terminal, characterized in that: include: An acquisition module, used to acquire a target text and a reference audio to be synthesized from an insurance business end, encode the target text to obtain a target text embedding feature, and encode the reference audio to obtain a target acoustic feature; An encoding module is used to concatenate the target text embedding feature and the target acoustic feature to obtain a target concatenated feature; input the target concatenated feature into an encoder of a pre-trained semantic feature processing model, and downsample the target concatenated feature to obtain a target semantic feature; input the target semantic feature into a pre-trained noise-filtered speech reconstruction model to obtain first target speech data; A decoding module is used to input the first target speech data, target semantic features and target splicing features into a decoder of a pre-trained semantic feature processing model, obtain second target speech data according to the first target speech data and the target semantic features; and obtain a target speech corresponding to the target text according to the second target speech data and the target splicing features.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the speech synthesis method of the insurance business terminal as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the speech synthesis method of the insurance business terminal as described in any one of claims 1 to 7 are implemented.

Images