A speech synthesis method, device, equipment and medium for insurance business end

Abstract

The present invention relates to the field of financial technology, and discloses a speech synthesis method, apparatus, device and medium for an insurance business end, including obtaining a target splicing feature by splicing and fusing a target text embedding feature with an acoustic feature; inputting the target splicing feature into an encoder of a pre-trained semantic feature processing model for downsampling to obtain a target semantic feature; inputting the target semantic feature into a pre-trained noise-filtered speech reconstruction model to output a first target speech data; inputting the first target speech data, the target semantic feature and the target splicing feature into a decoder of a pre-trained semantic feature processing model to obtain a target speech. This solution solves the problem that the existing text-to-speech conversion technology for the insurance business end is prone to deterioration in sound quality when processing complex phonetics or frequently switched speech, thereby resulting in a poor experience for customers when using intelligent voice customer service for conversational services.

Description

Technical Field

[0001] The present invention relates to the field of financial technology, and in particular to a speech synthesis method, apparatus, device and medium for an insurance business end. Background Art

[0002] Financial companies like banks and insurance companies almost all use intelligent voice customer service as a means of early communication and dispute resolution. This approach effectively reduces personnel costs, improves communication efficiency, and reduces staff workload. Intelligent voice customer service not only significantly shortens the actual call time with human customer service but also, through the integration of human and machine intelligence, rapidly processes customer needs.

[0003] Text-to-speech (TTS) technology is currently widely used in the insurance industry for intelligent voice customer service. Traditional TTS models are prone to degrading sound quality when processing complex phonology or frequently changing speech patterns. Especially at low bit rates, the spectral detail and sound quality of speech are significantly reduced, making it impossible to generate clear, high-quality speech. Furthermore, existing TTS systems have limited ability to model the details of phonology, intonation, and emotion, resulting in a monotonous and robotic sound.

[0004] Autoregressive models (such as Tacotron and Transformer TTS) have achieved some success in improving speech naturalness, but they are prone to audio incoherence and distortion when generating long segments of speech. The generation process of autoregressive models relies on the output of the previous step, which leads to error accumulation and affects the stability of generation. Furthermore, traditional models still perform poorly for diverse speech styles or emotional variations.

[0005] In summary, current speech synthesis technology is developing rapidly, but the naturalness of the speech synthesized by related speech synthesis technology is not very ideal, the ability to process complex phonetics is limited, and the reasoning efficiency is low, resulting in a poor experience for customers when using intelligent voice customer service for conversational services. Summary of the Invention

[0006] The present invention provides a speech synthesis method, device, equipment and medium for the insurance business end, so as to solve the problem that the existing text-to-speech conversion technology of the insurance business end is prone to deterioration of sound quality when processing complex phonetics or frequently switched voices, thereby resulting in a poor experience for customers when using intelligent voice customer service for conversational services.

[0007] In a first aspect, a speech synthesis method for an insurance business terminal is provided, comprising:

[0008] Obtaining a target text and reference audio to be synthesized from an insurance business end, encoding the target text to obtain target text embedding features, and encoding the reference audio to obtain target acoustic features;

[0009] The target text embedding feature and the target acoustic feature are concatenated to obtain a target concatenated feature; the target concatenated feature is input into an encoder of a pre-trained semantic feature processing model, and the target concatenated feature is down-sampled to obtain a target semantic feature; the target semantic feature is input into a pre-trained noise-filtered speech reconstruction model to obtain first target speech data;

[0010] The first target speech data, target semantic features and target splicing features are input into a decoder of a pre-trained semantic feature processing model, and second target speech data is obtained based on the first target speech data and target semantic features; and a target speech corresponding to the target text is obtained based on the second target speech data and target splicing features.

[0011] In a second aspect, a speech synthesis device for an insurance business terminal is provided, comprising:

[0012] An acquisition module is used to acquire a target text and reference audio to be synthesized from an insurance business end, encode the target text to obtain target text embedding features, and encode the reference audio to obtain target acoustic features;

[0013] An encoding module is configured to concatenate the target text embedding feature and the target acoustic feature to obtain a target concatenated feature; input the target concatenated feature into an encoder of a pre-trained semantic feature processing model, downsample the target concatenated feature to obtain a target semantic feature; and input the target semantic feature into a pre-trained noise-filtered speech reconstruction model to obtain first target speech data;

[0014] A decoding module is used to input the first target speech data, target semantic features and target splicing features into a decoder of a pre-trained semantic feature processing model, obtain second target speech data based on the first target speech data and target semantic features; and obtain a target speech corresponding to the target text based on the second target speech data and target splicing features.

[0015] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned speech synthesis method for the insurance business end when executing the computer program.

[0016] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the speech synthesis method of the above-mentioned insurance business end are implemented.

[0017] In the solution implemented by the speech synthesis method, device, computer equipment and storage medium of the above-mentioned insurance business end, the target splicing feature can be obtained by splicing and fusing the target text embedding feature with the acoustic feature; the target splicing feature is input into the encoder of a pre-trained semantic feature processing model for downsampling operation to obtain the target semantic feature; the target semantic feature is input into a pre-trained noise filtering speech reconstruction model to output the first target speech data; the first target speech data, target semantic feature and target splicing feature are input into the decoder of the pre-trained semantic feature processing model to obtain the target speech.

[0018] This invention uses downsampling in the encoder of the semantic feature processing model to simultaneously extract semantic features and reduce the dimensionality of the input data. This reduces the data dimension of the subsequent noise-filtered speech reconstruction model, lowering computational complexity and improving speech generation efficiency. Furthermore, the first target speech data, target semantic features, and target splicing features are grouped and fused, integrating the previously extracted semantic features into the speech generation process. This structural design ensures that both global information and local details in the speech signal are preserved and restored. This enables the model to capture phonetic details and emotional cues in the text during speech generation. This allows for the generation of speech with higher fidelity and natural expression, particularly when processing complex phonology, emotional variations, or intonation. Therefore, this solution addresses the problem with existing text-to-speech (TTS) conversion technologies, which often suffer from poor sound quality when processing complex phonology or frequently changing speech, leading to a poor user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 This is a schematic diagram of an application environment of a speech synthesis method for an insurance business terminal according to an embodiment of the present invention;

[0021] Figure 2 This is a flow chart of a speech synthesis method for an insurance business terminal according to an embodiment of the present invention;

[0022] Figure 3 This is a flowchart of a specific implementation of a speech synthesis method for an insurance business terminal in one embodiment of the present invention;

[0023] Figure 4 This is a structural diagram of a speech synthesis device at an insurance business end according to an embodiment of the present invention;

[0024] Figure 5 is a structural diagram of a computer device in one embodiment of the present invention;

[0025] Figure 6 FIG. 2 is another structural diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0027] The speech synthesis method for the insurance business terminal provided by the embodiment of the present invention can be applied in the following aspects: Figure 1 In an application environment, the application environment includes a client and a server, wherein the client can communicate with the server through a network. Specifically, the server obtains the target text to be synthesized and the reference audio of the insurance business end, encodes the target text to obtain the target text embedding feature, encodes the reference audio to obtain the target acoustic feature; splices the target text embedding feature with the target acoustic feature to obtain the target splicing feature; inputs the target splicing feature into the encoder of the pre-trained semantic feature processing model, downsamples the target splicing feature to obtain the target semantic feature; inputs the target semantic feature into the pre-trained noise filtering speech reconstruction model to obtain the first target speech data; inputs the first target speech data, the target semantic feature and the target splicing feature into the decoder of the pre-trained semantic feature processing model, obtains the second target speech data according to the first target speech data and the target semantic feature; obtains the target speech corresponding to the target text according to the second target speech data and the target splicing feature, and sends the target speech to the terminal client. Among them, the client is also called the user end, which refers to the program corresponding to the server and provides local services to customers. The client can be installed on but not limited to various personal computers, laptops, smart phones, tablets and portable wearable devices. The server can be implemented as a standalone server or a server cluster consisting of multiple servers.

[0028] See also Figure 2As shown, Figure 2 A flowchart of a speech synthesis method for an insurance business terminal provided by an embodiment of the present invention includes the following steps:

[0029] S01: Obtain the target text and reference audio to be synthesized from the insurance business end, encode the target text to obtain target text embedding features, and encode the reference audio to obtain target acoustic features.

[0030] The target text to be synthesized in the insurance business is text that requires speech synthesis. For example, in scenarios such as intelligent voice customer service and voice reading for financial companies such as banks and insurance companies, text needs to be converted into speech.

[0031] The reference audio is the speaker's audio. This means that in the insurance company's intelligent voice customer service scenario, the audio to be synthesized and the synthesized voice have the same timbre, or the synthesized voice has a timbre very close to the audio to be synthesized. This improves the quality of speech synthesis in the insurance company's intelligent voice customer service scenario, thereby increasing customer satisfaction.

[0032] The target text embedding features include text features such as phonemes, phrases, and sentence divisions. Phonemes are the smallest sound units that make up a language and can be represented by letters or phonetic symbols. For example, in Chinese characters, a syllable may consist of an initial consonant, a final vowel, and a tone.

[0033] Target acoustic features include intensity, duration, pitch, and quality. Pitch refers to the variation in pitch of a sound, duration refers to the duration of a sound, and intensity refers to the variation in loudness. These parameters together constitute the prosodic characteristics of language, helping to express emotions, attitudes, and the structure of language.

[0034] In this example, PyTorch is used to encode the target text when acquiring embedding features. PyTorch is a Python-based scientific computing library primarily used in deep learning and machine learning. Developed by Facebook's AI Research Lab (FAIR), it is known for its flexibility, ease of use, and strong community support. nnn.Module is the fundamental class for defining neural network models in PyTorch. Users can inherit from nn.Module and define their own models. The class constructor defines the network's layer structure and parameters.

[0035] In PyTorch, using the torch.nn.Embedding module class and the Transformer module class to build text embedding vectors is a common practice, especially in natural language processing (NLP) tasks. The Embedding module class is used to convert discrete word or character indices into continuous vector representations, while the Transformer module class is a powerful sequence processing architecture that can capture long-term dependencies in text.

[0036] In this example, we use the torch.nn.Embedding module class and the Transformer module class in the PyTorch library to implement text embedding feature extraction. The specific steps are as follows:

[0037] First, build a character set containing all possible Chinese characters. This character set should include all the Chinese characters you want to process in your model.

[0038] Then, assign each Kanji character a unique index, which can be achieved by creating a dictionary or using other data structures; convert the input data (such as text) into a sequence of these indices, which usually involves iterating over the text, finding the position of each character in the character set, and replacing it with the corresponding index.

[0039] Next, create an Embedding module class whose input dimension is equal to the number of characters in the character set (that is, the maximum value of the index plus 1, because the index usually starts at 0), and whose output dimension is the dimension of the embedding vector.

[0040] Finally, by setting the number of layers of the encoder in the Transformer module class to one or more layers, the embedding vector obtained by the Embedding module class is processed to obtain the final target text embedding feature.

[0041] This embodiment does not include the Transformer model training part, so it is not the focus of the present invention. In practical applications, it is necessary to define a loss function, an optimizer, and iterate on the training data set to update the model parameters.

[0042] In this embodiment, the acoustic features are not modeled. Instead, the short-time Fourier transform (STFT) is used to directly extract the Mel spectrum of the reference audio to obtain the target acoustic features. This is a common step in audio signal processing, particularly in tasks such as speech recognition, music information retrieval, and sound classification. The Mel spectrum is a spectral representation based on the human auditory characteristics. It better simulates the human auditory perception by performing nonlinear frequency compression (i.e., Mel-scaling transform). Specifically, the basic steps for extracting the Mel spectrum of the reference audio are as follows:

[0043] (1) Load the reference audio file and obtain the time series (waveform) of the reference audio.

[0044] (2) Apply short-time Fourier transform (STFT) to the reference audio waveform to obtain a spectrogram (a complex matrix where each element represents the complex amplitude in frequency and time).

[0045] (3) Convert the spectrogram from a linear frequency scale to a Mel frequency scale. This typically involves convolving the spectrogram with a set of Mel filter banks.

[0046] (4) Applying a logarithmic transform to the Mel spectrum yields the logarithmic Mel spectrum (MelP), which is the target acoustic feature of the reference audio. The logarithmic transform helps compress the dynamic range and makes the data more suitable for subsequent processing.

[0047] In addition, the MelP can be normalized, truncated, windowed, etc. If necessary, the MelP can also be converted to other formats such as MelP frequency cepstral coefficients (MFCCs), which are obtained by applying discrete cosine transform (DCT).

[0048] S02: Concatenate the target text embedding feature and the target acoustic feature to obtain a target concatenated feature; input the target concatenated feature into an encoder of a pre-trained semantic feature processing model, and downsample the target concatenated feature to obtain a target semantic feature; input the target semantic feature into a pre-trained noise-filtered speech reconstruction model to obtain first target speech data.

[0049] Among them, such as Figure 3 As shown, the semantic feature processing model in this embodiment is a U-Net model. The U-Net model was originally designed for image segmentation tasks. Its structure includes an encoder and a decoder, allowing the model to learn the mapping from input data to output data. However, with the development of technology, the concepts of the U-Net architecture have also been extended and applied to other fields, including speech processing and natural language processing.

[0050] The U-Net model gets its name from its symmetrical encoder-decoder architecture, which resembles the letter "U." The encoder extracts features from the input data through a series of convolutional layers and downsampling (usually max pooling) operations. The decoder gradually restores the data's dimensions through a series of convolutional layers and upsampling (usually transposed convolution or bilinear interpolation) operations, ultimately generating target data of the same size as the input data. A unique feature of U-Net is the use of skip connections. These connections pass feature data from the encoder directly to the corresponding layers in the decoder, helping to preserve detailed information in the input data and improve the accuracy of the output results of the current layer.

[0051] The noise-filtered speech reconstruction model in this embodiment is a diffusion model based on the transformer model in natural language processing, called the Diffusion Transformer (DiTs) model. It is a neural network structure based on deep learning. It is named because its core idea is to transform the input data into a form that is easier to process through the diffusion process, and then use the transformer model for efficient feature extraction and modeling. This structure has significant advantages in processing sequential data (such as text, speech, etc.), and can capture long-term dependencies in the data and generate high-quality output.

[0052] Specifically, the present invention inputs the target text embedding features into the encoder of the U-Net model. The encoder downsamples the target splicing features to obtain the target semantic features. The target semantic features are then input into the pre-trained DiTs model to obtain the first target speech data. The DiTs model processes the target semantic features as follows:

[0053] The DiTs model first preprocesses the target semantic features (input data) through a diffusion process. This process aims to map the data from the original space to a higher-dimensional latent space to reveal the underlying structure in the data. By adding an appropriate amount of noise, the diffusion process can enhance the robustness of the model, making it better able to cope with the uncertainty in real-world situations.

[0054] After the diffusion process, the resulting data is fed into the transformer model. The transformer model, composed of multiple stacked self-attention layers, effectively captures long-term dependencies in the data. Through this self-attention mechanism, the DiTs model dynamically assigns attention weights based on different parts of the input sequence (target semantic features), accurately capturing key information.

[0055] At the final stage of the transformer model, the model generates a representation vector containing rich feature information. This vector can be further used in speech generation tasks. A decoding process is required to convert the representation vector back to the original data (target semantic feature) space to obtain the first target speech data.

[0056] S03: Input the first target speech data, target semantic features and target splicing features into the decoder of a pre-trained semantic feature processing model, and obtain second target speech data based on the first target speech data and target semantic features; and obtain a target speech corresponding to the target text based on the second target speech data and target splicing features.

[0057] Specifically, the first target speech data, target semantic features and target splicing features are input into the decoder of a pre-trained U-Net model. The number of layers of the decoder is the same as the number of layers of the encoder. In this embodiment, it is set as the first decoding layer and the output layer. After the first target speech data and the target semantic features are input into the first decoding layer, the second target speech data is obtained. At this time, the data dimension of the second target speech data is the same as the data dimension of the target splicing features. After the second target speech data and the target splicing features of the same dimension are input into the output layer, the target speech corresponding to the original target text is obtained.

[0058] This application takes the scenario of intelligent voice customer service of an insurance company as an example to illustrate, and applies intelligent voice customer service to the above-mentioned steps S01, S02 and S03. Specifically, the text that needs to be converted into speech by the intelligent voice customer service is obtained as the target text to be synthesized, and the speech audio file that needs to be converted into the required timbre by the intelligent voice customer service is obtained as the reference audio, that is, the timbre of the synthesized speech needs to be infinitely close to the reference audio, the target text is encoded to obtain the target text embedding feature, and the reference audio is encoded to obtain the target acoustic feature; the target text embedding feature and the target acoustic feature are spliced ​​to obtain the target splicing feature; the target splicing feature is input into the encoder of a pre-trained U-Net model, and the target splicing feature is down-sampled to obtain the target semantic feature; the target semantic feature is input into the pre-trained diffusion transformer model to obtain the first target speech data; the first target speech data, the target semantic feature and the target splicing feature are input into the decoder of the pre-trained U-Net model, and the second target speech data is obtained according to the first target speech data and the target semantic feature; the target speech corresponding to the target text is obtained according to the second target speech data and the target splicing feature.

[0059] The target speech corresponding to the generated target text will also be optimized through the post-processing module. This mainly includes noise removal, spectral balance adjustment, pitch correction, etc., to ensure that the final output speech is clear, natural, and conforms to the speech characteristics of the target text.

[0060] The present invention performs downsampling operations through the encoder of the semantic feature processing model. While extracting semantic features, it can reduce the dimensionality of the input data, thereby reducing the data dimension in the subsequent noise-filtered speech reconstruction model, reducing computational complexity, and improving speech generation efficiency. At the same time, the first target speech data, target semantic features, and target splicing features are grouped and fused to integrate the semantic features extracted in the early stage into the speech generation process. This structural design ensures that both global information and local details in the speech signal can be preserved and restored. This enables the model to capture speech details and emotional clues in the text during the speech generation process. In particular, when processing complex phonology, emotional changes, or intonation fluctuations, speech with higher fidelity and natural expression can be generated.

[0061] In one embodiment, concatenating the target text embedding feature with the target acoustic feature to obtain a target concatenated feature includes:

[0062] The data dimension of the first target speech data is set to be the same as that of the target semantic feature; the data dimension includes the number of rows and columns of data.

[0063] The target acoustic feature is concatenated with the last column of the target text embedding feature to obtain the target concatenated feature.

[0064] Specifically, in this embodiment, the first target speech data and the target semantic features are both two-dimensional vectors. The same data dimension means that the number of rows of the first target speech data is equal to the number of rows of the target semantic features, and the number of columns of the first target speech data is equal to the number of columns of the target semantic features. For example, if the data dimension of the first target speech data is two-dimensional data of 128 rows * 256 columns, then the data dimension of the target semantic features must also be two-dimensional data of 128 rows * 256 columns. In the last column of the target text embedding feature with a dimension of 128 rows * 256 columns, the target acoustic feature with a splicing dimension of 128 rows * 256 columns will be obtained, and a target splicing feature with a dimension of 128 rows * 256 columns will be obtained.

[0065] In one embodiment, obtaining the second target speech data according to the first target speech data and the target semantic feature includes:

[0066] The data dimension of the first target speech data is set to be the same as that of the target semantic feature.

[0067] The first target speech data and the target semantic features are concatenated and fused in the channel dimension to obtain a first fused feature, and the first fused feature is sequentially convolved and up-sampled to obtain the second target speech data.

[0068] Among them, concatenation fusion in channel dimension refers to concatenating feature maps from different sources or different convolutional layers in a certain layer of a deep neural network in the channel dimension to form a new feature map with more channels.

[0069] Specifically, in this embodiment, the first target speech data and the target semantic features are spliced ​​and fused in the channel dimension. Before splicing, the first target speech data and the target semantic features need to be preprocessed into the same data dimension, and then they can be spliced ​​and fused to obtain the first fused feature. The number of channels of the first fused feature after splicing increases, which is equivalent to increasing the depth of the network, which helps the network learn more complex feature representations. Next, the first fused feature is convolved to extract the correlation between features through the convolution operation, thereby enhancing the feature extraction ability of the network. This method can introduce more nonlinear changes in the spliced ​​feature map and improve the expression ability of the network. Finally, the dimension of the data after the above convolution processing is increased through upsampling until it is the same as the data dimension of the target splicing feature, thereby obtaining the second target speech data.

[0070] In one embodiment, obtaining a target speech corresponding to the target text according to the second target speech data and the target splicing feature includes:

[0071] The data dimension of the second target speech data is set to be the same as that of the target splicing feature.

[0072] The second target speech data and the target splicing feature are spliced ​​and fused in the channel dimension to obtain a second fused feature, and the second fused feature is convolved to obtain a target speech corresponding to the target text.

[0073] Specifically, in this embodiment, the second target speech data and the target splicing feature are spliced ​​and fused in the channel dimension. Before splicing, the second target speech data and the target splicing feature need to be preprocessed into the same data dimension, and then the second fused feature can be obtained after splicing and fusion. The number of channels of the second fused feature after splicing increases, which is equivalent to increasing the depth of the network, helping the network learn more complex feature representations. Next, the second fused feature is convolved to obtain the target speech corresponding to the target text. The correlation between the features is extracted through the convolution operation, thereby enhancing the feature extraction ability of the network. This method can introduce more nonlinear changes in the spliced ​​feature map and improve the expression ability of the network.

[0074] In one embodiment, the encoder model includes a first conversion encoder and a pooling layer. The target splicing feature is input into the first conversion encoder, and the semantic feature in the target splicing feature is obtained through feature extraction of the attention mechanism in the first conversion encoder; the semantic feature is input into the pooling layer for downsampling operation, and the target semantic feature is output.

[0075] Specifically, the encoder model in this embodiment is the Vision Transformer (ViT). The ViT model is a deep learning model that applies the Transformer architecture to image recognition tasks. The ViT model is used to capture complex relationships in text information. ViT uses a self-attention mechanism to effectively model long-range dependencies, helping the model better understand the relationships between individual words in a text. The first transform encoder is the Transformer encoder in the ViT model.

[0076] The target concatenated features are input into the ViT model. The ViT model structure treats the target concatenated features (text embeddings) as data similar to "image patches." After processing through the Transformer encoder, it extracts features with more global and local semantics, resulting in semantic features within the target concatenated features. This processing approach captures the rhythm, tone, and emotional changes in the text, ensuring that the generated speech has more natural intonation and rhythm.

[0077] The pooling layer is a layer commonly used in convolutional neural networks. Its principle is to downsample the input feature map according to certain rules to obtain a feature map with a smaller spatial size. The pooling operation methods include the following:

[0078] 1) Max Pooling: In each local region (pooling window) of the feature map, the maximum value is selected as the output for that region. This extracts the most significant features and is robust to noise and slight deformations. A common max pooling operation uses a 2x2 pooling window and a stride of 2.

[0079] 2) Average Pooling: Calculates the average value within each local region (pooling window) of the feature map as the output for that region. This smoothes the feature map while preserving background information. A 2x2 pooling window and a stride of 2 can also be used.

[0080] Specifically, the semantic features in the target concatenated features are input into the pooling layer for downsampling. This embodiment uses maximum pooling to downsample the semantic features in the target concatenated features, resulting in a target semantic feature with a smaller spatial size and a more prominent feature map. This operation plays an important role in convolutional neural networks, helping to improve the performance and efficiency of the model.

[0081] In one embodiment, inputting the target semantic features into a pre-trained noise-filtered speech reconstruction model includes:

[0082] The noise-filtered speech reconstruction model includes a second conversion encoder and a third conversion encoder.

[0083] After adding preset noise data to the target semantic feature, a first noise target semantic feature is obtained.

[0084] The first noise target semantic feature is input into a second transformation encoder, and feature extraction is performed through an attention mechanism in the second transformation encoder to obtain a second noise target semantic feature.

[0085] After adding preset noise data to the second noise target semantic feature, a third noise target semantic feature is obtained.

[0086] The third noise target semantic feature is input into a third transformation encoder, and feature extraction is performed through an attention mechanism in the third transformation encoder to obtain a fourth noise target semantic feature.

[0087] After removing all preset noise data from the fourth noise target semantic feature, the first target speech data is obtained.

[0088] Among them, the noise filtering speech reconstruction model is a diffusion transformer, wherein the diffusion transformer includes a second conversion encoder and a third conversion encoder, and both encoders are transformer models. Each transformer model is composed of multiple self-attention mechanism layers stacked together, which can effectively capture long-term dependencies in the data. Through the self-attention mechanism, the DiTs model can dynamically assign attention weights according to different parts of the input sequence (target semantic features), thereby achieving accurate capture of key information. In this embodiment, the input data is captured multiple times through a two-layer transformer model, so that the key information in the input data can be further accurately captured.

[0089] Specifically, first, the target semantic feature is passed through the diffusion process in the DiTs model, and an appropriate amount of noise data is added to obtain the first noisy target semantic feature; the first noisy target semantic feature is input into the first transformer model (the second transformation encoder), and the feature is extracted through the attention mechanism in the transformer model to obtain the second noisy target semantic feature.

[0090] Next, the second noise target semantic feature is passed through the diffusion process in the DiTs model, and an appropriate amount of noise data is added to obtain the third noise target semantic feature; the third noise target semantic feature is input into the second transformer model (the third transformation encoder), and the fourth noise target semantic feature is obtained by feature extraction through the attention mechanism in the transformer model.

[0091] Finally, the fourth noise target semantic feature is decoded by the corresponding decoder in the DiTs model to obtain the first target speech data.

[0092] In one embodiment, the step of concatenating and fusing the first target speech data and the target semantic features in the channel dimension includes:

[0093] The target semantic features include rhythmic features, tonal features and emotional change features in the text;

[0094] Allocating a rhythm weight parameter to the rhythm feature, allocating a tone weight parameter to the tone feature, and allocating an emotion weight parameter to the emotion change feature;

[0095] performing weighted processing on the prosody feature, the pitch feature, and the emotion change feature according to the prosody weight parameter, the pitch weight parameter, and the emotion weight parameter, respectively, to obtain a weighted target semantic feature;

[0096] The first target speech data and the weighted target semantic features are spliced ​​and fused in the channel dimension.

[0097] Specifically, the target semantic features extracted in this embodiment include the prosody features, tone features and emotional change features in the text, and a preset weight parameter is assigned to each of the prosody features, tone features and emotional change features, for example, the prosody weight parameters assigned to each of the prosody features, tone features and emotional change features by expert experience data are as follows: 、 and sentiment weight parameters ,use The rhythm feature data is weighted and the The pitch features are weighted and After weighting the emotional change features, weighted target semantic features are obtained. The first target speech data and the weighted target semantic features are concatenated and fused in the channel dimension. Before concatenation, the first target speech data and the weighted target semantic features need to be preprocessed to the same data dimension before concatenation and fusion can be performed. The increase in the number of channels in the concatenated data features is equivalent to increasing the depth of the network, which helps the network learn more complex feature representations.

[0098] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0099] In one embodiment, a speech synthesis device for an insurance business terminal is provided. The speech synthesis device for the insurance business terminal corresponds one-to-one with the speech synthesis method for the insurance business terminal in the above embodiment. Figure 4 As shown, the intelligent question-answering processing device includes an acquisition module 101, an encoding module 102, and a decoding module 103. The functional modules are described in detail as follows:

[0100] The acquisition module 101 is used to obtain the target text and reference audio to be synthesized from the insurance business end, encode the target text to obtain the target text embedding feature, and encode the reference audio to obtain the target acoustic feature.

[0101] The encoding module 102 is used to concatenate the target text embedding feature and the target acoustic feature to obtain a target concatenated feature; input the target concatenated feature into an encoder of a pre-trained semantic feature processing model, downsample the target concatenated feature to obtain a target semantic feature; and input the target semantic feature into a pre-trained noise-filtered speech reconstruction model to obtain first target speech data.

[0102] The decoding module 103 is used to input the first target speech data, target semantic features and target splicing features into a decoder of a pre-trained semantic feature processing model, obtain second target speech data based on the first target speech data and target semantic features; and obtain target speech corresponding to the target text based on the second target speech data and target splicing features.

[0103] In one embodiment, the determination module 102 is specifically configured to:

[0104] The data dimension of the first target speech data is set to be the same as that of the target semantic feature; the data dimension includes the number of rows and columns of data.

[0105] The target acoustic feature is concatenated with the last column of the target text embedding feature to obtain the target concatenated feature.

[0106] In one embodiment, the determination module 103 is further configured to:

[0107] The data dimension of the first target speech data is set to be the same as that of the target semantic feature.

[0108] The first target speech data and the target semantic features are concatenated and fused in the channel dimension to obtain a first fused feature, and the first fused feature is sequentially convolved and up-sampled to obtain the second target speech data.

[0109] In one embodiment, the determination module 102 is further configured to:

[0110] The encoder model includes a first conversion encoder and a pooling layer. The target splicing feature is input into the first conversion encoder, and the semantic feature in the target splicing feature is obtained through feature extraction of the attention mechanism in the first conversion encoder; the semantic feature is input into the pooling layer for downsampling operation, and the target semantic feature is output.

[0111] In one embodiment, the recall module 103 is specifically configured to:

[0112] The data dimension of the second target speech data is set to be the same as that of the target splicing feature.

[0113] The second target speech data and the target splicing feature are spliced ​​and fused in the channel dimension to obtain a second fusion feature, and the second fusion feature is convolved to obtain the corresponding target speech.

[0114] In one embodiment, the sorting module 102 is specifically configured to:

[0115] The noise-filtered speech reconstruction model includes a second conversion encoder and a third conversion encoder.

[0116] After adding preset noise data to the target semantic feature, a first noise target semantic feature is obtained.

[0117] The first noise target semantic feature is input into a second transformation encoder, and feature extraction is performed through an attention mechanism in the second transformation encoder to obtain a second noise target semantic feature.

[0118] After adding preset noise data to the second noise target semantic feature, a third noise target semantic feature is obtained.

[0119] The third noise target semantic feature is input into a third transformation encoder, and feature extraction is performed through an attention mechanism in the third transformation encoder to obtain a fourth noise target semantic feature.

[0120] After removing all preset noise data from the fourth noise target semantic feature, the first target speech data is obtained.

[0121] In one embodiment, the sorting module 103 is specifically configured to:

[0122] The target semantic features include rhythmic features, tonal features and emotional change features in the text;

[0123] Allocating a rhythm weight parameter to the rhythm feature, allocating a tone weight parameter to the tone feature, and allocating an emotion weight parameter to the emotion change feature;

[0124] performing weighted processing on the prosody feature, the pitch feature, and the emotion change feature according to the prosody weight parameter, the pitch weight parameter, and the emotion weight parameter, respectively, to obtain a weighted target semantic feature;

[0125] The first target speech data and the weighted target semantic features are spliced ​​and fused in the channel dimension.

[0126] The present invention provides a speech synthesis device for insurance businesses. Downsampling is performed by the encoder of a semantic feature processing model. While extracting semantic features, the dimensionality of the input data can be reduced, thereby reducing the data dimensionality in the subsequent noise-filtered speech reconstruction model, lowering computational complexity, and improving speech generation efficiency. Simultaneously, the first target speech data, target semantic features, and target splicing features are grouped and fused, integrating the previously extracted semantic features into the speech generation process. This structural design ensures that both global information and local details in the speech signal are preserved and restored. This enables the model to capture speech details and emotional cues in the text during the speech generation process. In particular, when processing complex phonology, emotional changes, or intonation fluctuations, speech with higher fidelity and natural expression can be generated.

[0127] The specific definition of the speech synthesis device for the insurance business end can be found in the definition of the speech synthesis method for the insurance business end described above and will not be repeated here. Each module in the speech synthesis device for the insurance business end can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the modules can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0128] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, memory, network interface and database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the service side of a speech synthesis method for an insurance business end.

[0129] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements the client-side functions or steps of a speech synthesis method for an insurance business end.

[0130] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0131] Obtain the target text and reference audio to be synthesized from the insurance business end, encode the target text to obtain target text embedding features, and encode the reference audio to obtain target acoustic features.

[0132] The target text embedding feature and the target acoustic feature are spliced ​​to obtain a target spliced ​​feature; the target spliced ​​feature is input into an encoder of a pre-trained semantic feature processing model, and the target spliced ​​feature is down-sampled to obtain a target semantic feature; the target semantic feature is input into a pre-trained noise filtering speech reconstruction model to obtain first target speech data.

[0133] The first target speech data, target semantic features and target splicing features are input into a decoder of a pre-trained semantic feature processing model, and second target speech data is obtained based on the first target speech data and target semantic features; and a target speech corresponding to the target text is obtained based on the second target speech data and target splicing features.

[0134] In one embodiment, a computer-readable storage medium is provided on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0135] Obtain the target text and reference audio to be synthesized from the insurance business end, encode the target text to obtain target text embedding features, and encode the reference audio to obtain target acoustic features.

[0136] The target text embedding feature and the target acoustic feature are spliced ​​to obtain a target spliced ​​feature; the target spliced ​​feature is input into an encoder of a pre-trained semantic feature processing model, and the target spliced ​​feature is down-sampled to obtain a target semantic feature; the target semantic feature is input into a pre-trained noise filtering speech reconstruction model to obtain first target speech data.

[0137] The first target speech data, target semantic features and target splicing features are input into a decoder of a pre-trained semantic feature processing model, and second target speech data is obtained based on the first target speech data and target semantic features; and a target speech corresponding to the target text is obtained based on the second target speech data and target splicing features.

[0138] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0139] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0140] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0141] The above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them. Although the present invention has been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. The non-Company software tools or components that appear in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A speech synthesis method for an insurance business terminal, characterized in that: include: Obtaining a target text and reference audio to be synthesized from an insurance business end, encoding the target text to obtain target text embedding features, and encoding the reference audio to obtain target acoustic features; The target text embedding feature and the target acoustic feature are concatenated to obtain a target concatenated feature; the target concatenated feature is input into an encoder of a pre-trained semantic feature processing model, and the target concatenated feature is down-sampled to obtain a target semantic feature; the target semantic feature is input into a pre-trained noise-filtered speech reconstruction model to obtain first target speech data; The first target speech data, target semantic features and target splicing features are input into a decoder of a pre-trained semantic feature processing model, and second target speech data is obtained based on the first target speech data and target semantic features; and a target speech corresponding to the target text is obtained based on the second target speech data and target splicing features.

2. The speech synthesis method for insurance business terminal according to claim 1, characterized in that: The step of concatenating the target text embedding feature and the target acoustic feature to obtain a target concatenated feature includes: Setting the data dimension of the first target speech data and the target semantic feature to be the same; the data dimension includes the number of rows and columns of data; The target acoustic feature is concatenated with the last column of the target text embedding feature to obtain the target concatenated feature.

3. The speech synthesis method for insurance business terminal according to claim 1, characterized in that: The obtaining of second target speech data according to the first target speech data and target semantic features includes: Setting the data dimension of the first target speech data and the target semantic feature to be the same; The first target speech data and the target semantic features are concatenated and fused in the channel dimension to obtain a first fused feature, and the first fused feature is sequentially convolved and up-sampled to obtain the second target speech data.

4. The speech synthesis method for insurance business terminal according to claim 1, characterized in that: The obtaining of a target speech corresponding to the target text according to the second target speech data and the target splicing feature includes: Setting the data dimension of the second target speech data to be the same as that of the target splicing feature; The second target speech data and the target splicing feature are spliced ​​and fused in the channel dimension to obtain a second fusion feature, and the second fusion feature is convolved to obtain the corresponding target speech.

5. The speech synthesis method for insurance business terminal according to claim 1, characterized in that: The encoder of the semantic feature processing model includes a first conversion encoder and a pooling layer. The target splicing feature is input into the first conversion encoder, and the semantic feature in the target splicing feature is obtained through feature extraction of the attention mechanism in the first conversion encoder; the semantic feature is input into the pooling layer for downsampling operation, and the target semantic feature is output.

6. The speech synthesis method for insurance business terminal according to claim 2, characterized in that: Inputting the target semantic features into a pre-trained noise-filtered speech reconstruction model includes: The noise-filtered speech reconstruction model includes a second conversion encoder and a third conversion encoder; After adding preset noise data to the target semantic feature, a first noise target semantic feature is obtained; Inputting the first noise target semantic feature into a second transformation encoder, and performing feature extraction through an attention mechanism in the second transformation encoder to obtain a second noise target semantic feature; After adding the preset noise data to the second noise target semantic feature, a third noise target semantic feature is obtained; Inputting the third noise target semantic feature into a third transformation encoder, performing feature extraction through an attention mechanism in the third transformation encoder, and obtaining a fourth noise target semantic feature; After removing all preset noise data from the fourth noise target semantic feature, the first target speech data is obtained.

7. The speech synthesis method for insurance business terminal according to claim 3, characterized in that: The step of splicing and fusing the first target speech data and the target semantic features in the channel dimension includes: The target semantic features include rhythmic features, tonal features and emotional change features in the text; Allocating a rhythm weight parameter to the rhythm feature, allocating a tone weight parameter to the tone feature, and allocating an emotion weight parameter to the emotion change feature; performing weighted processing on the prosody feature, the pitch feature, and the emotion change feature according to the prosody weight parameter, the pitch weight parameter, and the emotion weight parameter, respectively, to obtain a weighted target semantic feature; The first target speech data and the weighted target semantic features are spliced ​​and fused in the channel dimension.

8. A speech synthesis device for an insurance business terminal, characterized in that: include: An acquisition module is used to acquire a target text and reference audio to be synthesized from an insurance business end, encode the target text to obtain target text embedding features, and encode the reference audio to obtain target acoustic features; An encoding module is configured to concatenate the target text embedding feature and the target acoustic feature to obtain a target concatenated feature; input the target concatenated feature into an encoder of a pre-trained semantic feature processing model, downsample the target concatenated feature to obtain a target semantic feature; and input the target semantic feature into a pre-trained noise-filtered speech reconstruction model to obtain first target speech data; A decoding module is used to input the first target speech data, target semantic features and target splicing features into a decoder of a pre-trained semantic feature processing model, obtain second target speech data based on the first target speech data and target semantic features; and obtain a target speech corresponding to the target text based on the second target speech data and target splicing features.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the speech synthesis method for the insurance business terminal as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the speech synthesis method for the insurance business terminal as claimed in any one of claims 1 to 7 are implemented.

Images