A voice conversion method, device, computer equipment and storage medium

Abstract

The application discloses a speech conversion method and device, computer equipment and a storage medium, and belongs to the technical field of artificial intelligence. The application receives a speech conversion instruction, acquires a text to be converted, and performs semantic recognition on the text to be converted to acquire first semantic features; determines a dialogue spontaneous behavior matched with the text to be converted based on the first semantic features; extracts features of the dialogue spontaneous behavior to obtain first behavior features; combines the first semantic features and the first behavior features to construct a first to-be-predicted speech feature combination; inputs the first to-be-predicted speech feature combination into a pre-trained speech conversion model, and outputs a speech conversion result corresponding to the text to be converted. The application also relates to the technical field of blockchains, and the text to be converted is stored in a blockchain network. The application reduces mechanical feeling in speech conversion, improves naturalness and fluency of conversion effect, and significantly improves user experience.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and particularly relates to a speech conversion method and device, computer equipment and a storage medium. BACKGROUND

[0002] At present, financial customer service has become an indispensable part of financial institutions. With the continuous development of technology, financial customer service has gradually transitioned to automation and intelligentization to improve efficiency and customer experience.

[0003] In the field of financial customer service, there are some intelligent customer service systems that apply speech conversion technology. However, the current speech conversion technology still has some problems and deficiencies, for example, the converted speech dialogue lacks spontaneous behaviors such as pauses, repetitions, prolongations, and yawns, resulting in low naturalness of conversion effect, heavy mechanical feeling, and lack of natural fluency of human voice. During the conversation process, users usually feel unnatural voice when using speech conversion, which seriously affects the user experience and reduces the quality and acceptability of financial customer service. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a speech conversion method, device, computer equipment and storage medium to solve the technical problems of low naturalness of conversion effect, heavy mechanical feeling, and lack of natural fluency of human voice in existing speech conversion technology, which seriously affects the user experience.

[0005] To solve the above technical problems, the embodiments of the present application provide a speech conversion method, which adopts the following technical solutions:

[0006] A speech conversion method comprises:

[0007] receiving a speech conversion instruction, obtaining a text to be converted, and performing semantic recognition on the text to be converted to obtain first semantic features;

[0008] determining a dialogue spontaneous behavior matched with the text to be converted based on the first semantic features;

[0009] performing feature extraction on the dialogue spontaneous behavior to obtain first behavior features;

[0010] combining the first semantic features and the first behavior features to construct a first to-be-predicted speech feature combination;

[0011] inputting the first to-be-predicted speech feature combination into a pre-trained speech conversion model to output a speech conversion result corresponding to the text to be converted.

[0012] Further, the step of receiving a voice conversion instruction, obtaining the text to be converted, and performing semantic recognition on the text to be converted to obtain the first semantic feature specifically includes:

[0013] Receiving a voice conversion instruction, obtaining the text to be converted matched with the voice conversion instruction;

[0014] Performing part-of-speech tagging on the text to be converted to obtain text part-of-speech tags;

[0015] Performing named entity recognition on the text to be converted to obtain text named entities;

[0016] Performing semantic analysis on the text to be converted based on the text part-of-speech tags and the text named entities to obtain the first semantic feature.

[0017] Further, the step of determining the dialog spontaneous behavior matched with the text to be converted based on the first semantic feature specifically includes:

[0018] Inputting the first semantic feature into a pre-trained spontaneous behavior matching model, wherein the spontaneous behavior matching model is built based on a Transformer model architecture, and the latent space of the spontaneous behavior matching model pre-stores a plurality of dialog first behavior features;

[0019] Encoding the first semantic feature to obtain a semantic encoding feature;

[0020] Mapping the semantic encoding feature to the latent space and determining the dialog spontaneous behavior matched with the first semantic feature in the latent space.

[0021] Further, the step of mapping the semantic encoding feature to the latent space and determining the dialog spontaneous behavior matched with the first semantic feature in the latent space specifically includes:

[0022] Determining the position information of the semantic encoding feature in the latent space;

[0023] Based on the position information of the semantic encoding feature in the latent space, calculating the spatial distance from the semantic encoding feature to various dialog first behavior features;

[0024] According to the spatial distance from the semantic encoding feature to various dialog first behavior features, determining the dialog spontaneous behavior matched with the first semantic feature.

[0025] Further, the step of combining the first semantic feature and the first behavior feature to construct a first to-be-predicted voice feature combination specifically includes:

[0026] Obtaining the position information of the semantic encoding feature in the latent space to obtain first position information;

[0027] obtain second position information;

[0028] construct a position information sequence based on the first position information and the second position information;

[0029] combine the first semantic feature and the first behavior feature based on the position information sequence to obtain a first to-be-predicted speech feature combination.

[0030] Further, the speech conversion model includes a convolutional network layer, a long short-term memory network layer, and a fully connected layer. The step of inputting the first to-be-predicted speech feature combination into the pre-trained speech conversion model and outputting a speech conversion result corresponding to the to-be-converted text specifically includes:

[0031] extract local features of the first to-be-predicted speech feature combination through the convolutional network layer to obtain a plurality of first local features;

[0032] calculate hidden states of the plurality of first local features through the long short-term memory network layer to obtain a first hidden state sequence feature;

[0033] perform speech classification prediction on the first hidden state sequence feature through the fully connected layer to obtain the speech conversion result corresponding to the to-be-converted text.

[0034] Further, before inputting the first to-be-predicted speech feature combination into the pre-trained speech conversion model and outputting the speech conversion result corresponding to the to-be-converted text, the method further includes:

[0035] obtain training corpus samples, wherein the training corpus samples include mutually matched speech data samples and text data samples;

[0036] perform semantic recognition on the text data samples to obtain second semantic features;

[0037] recognize dialog spontaneous behaviors in the speech data samples, and perform feature extraction on the dialog spontaneous behaviors in the speech data samples to obtain second behavior features;

[0038] combine the second semantic features and the second behavior features to construct a second to-be-predicted speech feature combination;

[0039] input the second to-be-predicted speech feature combination into the speech conversion model;

[0040] extract local features of the second to-be-predicted speech feature combination through the convolutional network layer to obtain second local features;

[0041] process the second local features through the long short-term memory network layer to output a second hidden state sequence feature;

[0042] The hidden state sequence feature is classified and predicted by a full connection layer to generate a predicted speech sample;

[0043] The speech conversion model is iteratively updated based on the predicted speech sample and the speech data sample until the model is fitted.

[0044] To solve the above technical problems, the embodiment of the application further provides a speech conversion device, which adopts the technical scheme as follows:

[0045] A speech conversion device comprises:

[0046] A semantic recognition module is configured to receive a speech conversion instruction, acquire a text to be converted, and perform semantic recognition on the text to be converted to acquire first semantic features;

[0047] A behavior matching module is configured to determine a dialogue spontaneous behavior matched with the text to be converted based on the first semantic features;

[0048] A feature extraction module is configured to perform feature extraction on the dialogue spontaneous behavior to obtain first behavior features;

[0049] A feature combination module is configured to combine the first semantic features and the first behavior features to construct a first to-be-predicted speech feature combination;

[0050] A speech conversion module is configured to input the first to-be-predicted speech feature combination into a pre-trained speech conversion model and output a speech conversion result corresponding to the text to be converted.

[0051] To solve the above technical problems, the embodiment of the application further provides a computer device, which adopts the technical scheme as follows:

[0052] A computer device comprises a memory and a processor, the memory stores computer readable instructions, and the processor executes the computer readable instructions to realize the steps of the speech conversion method according to any one of the above.

[0053] To solve the above technical problems, the embodiment of the application further provides a computer readable storage medium, which adopts the technical scheme as follows:

[0054] A computer readable storage medium stores computer readable instructions, and the computer readable instructions are executed by a processor to realize the steps of the speech conversion method according to any one of the above.

[0055] Compared with the prior art, the embodiment of the application has the following beneficial effects:

[0056] The application discloses a speech conversion method and device, computer equipment and a storage medium, and belongs to the technical field of artificial intelligence. The application receives a speech conversion instruction, acquires a text to be converted, and performs semantic recognition on the text to be converted to acquire first semantic features; determines a dialogue spontaneous behavior matched with the text to be converted based on the first semantic features; performs feature extraction on the dialogue spontaneous behavior to obtain first behavior features; combines the first semantic features and the first behavior features to construct a first to-be-predicted speech feature combination; and inputs the first to-be-predicted speech feature combination into a pre-trained speech conversion model to output a speech conversion result corresponding to the text to be converted. The application combines semantic recognition and feature extraction of dialogue spontaneous behavior to generate a more natural speech conversion result, acquires semantic features of a text and dialogue spontaneous behavior features matched with the text, constructs a speech feature combination closer to a human expression mode, and thus reduces mechanical feeling in speech conversion, improves naturalness and fluency of conversion effect, and significantly improves user experience. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the solutions in the application, the drawings needed in the embodiments of the application will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0058] Figure 1 An exemplary system architecture diagram in which the application can be applied is shown;

[0059] Figure 2 A flowchart of one embodiment of a speech conversion method according to the application is shown;

[0060] Figure 3 A structural schematic diagram of one embodiment of a speech conversion device according to the application is shown;

[0061] Figure 4 A structural schematic diagram of one embodiment of computer equipment according to the application is shown. DETAILED DESCRIPTION

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the application belongs; the terminology used in the specification of the application is only for the purpose of describing specific embodiments and is not intended to limit the application; the specification of the application, the claims and the above description of drawings, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. The specification of the application and the claims or the above description of drawings, the terms "first", "second", etc. are used to distinguish different objects, not to describe a specific order.

[0063] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0064] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0065] like Figure 1 As shown, system architecture 100 may include terminal device 101, network 102, and server 103. Terminal device 101 may be a laptop 1011, tablet 1012, or mobile phone 1013. Network 102 is used as a medium to provide a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables.

[0066] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Various communication client applications can be installed on terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0067] Terminal device 101 can be various electronic devices with a display screen and support web browsing. In addition to laptops 1011, tablets 1012, or mobile phones 1013, terminal device 101 can also be an e-book reader, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, and desktop computer, etc.

[0068] Server 103 can be a server that provides various services, such as a backend server that provides support for the pages displayed on terminal device 101.

[0069] It should be noted that the voice conversion method provided in this application embodiment is generally executed by a server / terminal device, and correspondingly, the voice conversion device is generally set in the server / terminal device.

[0070] It should be understood that Figure 1 The number of terminal devices, networks and servers in the above system is only illustrative, and the system can have any number of terminal devices, networks and servers according to the needs of implementation.

[0071] With reference to Figure 2 , a flowchart of one embodiment of a voice conversion method according to the present application is shown. The voice conversion method includes the following steps:

[0072] S201, receiving a voice conversion instruction, obtaining a text to be converted, and performing semantic recognition on the text to be converted to obtain first semantic features.

[0073] Specifically, first, the system receives a voice conversion instruction and obtains a text to be converted. Then, the text is subjected to semantic recognition to extract first semantic features. Through semantic recognition, the system can understand important information such as the meaning, context and emotion of the text, and the first semantic features obtained can help the voice conversion model to generate voice expressions consistent with the content of the text.

[0074] Semantic recognition refers to understanding the meaning expressed by the text, including word meaning, sentence meaning, paragraph meaning and the intent of the entire article, etc. This step is an important link in the implementation of natural language processing (NLP). Semantic recognition operations include part-of-speech tagging, named entity recognition and semantic analysis. On the basis of semantic recognition, the semantic features of the text can be further extracted. These features can be structured data, such as the results of semantic role labeling, named entity recognition, etc.; or vectorized representations, such as vectors obtained by mapping words to low-dimensional vector spaces through word embedding.

[0075] S202, determining a spontaneous behavior in a dialogue that matches the text to be converted based on the first semantic features.

[0076] Spontaneous behavior in a dialogue refers to various verbal and non-verbal behaviors that naturally occur between communicators in the absence of explicit external stimuli during a dialogue. Spontaneous behavior in a dialogue includes, but is not limited to, pauses, repetitions, prolonged speech, yawning, smiling, nodding, etc. These behaviors together constitute the naturalness and fluency of the dialogue, often reflect the true state of the communicators' hearts, and are an important part of emotional communication and information transmission in the dialogue.

[0077] Specifically, the system captures the natural behavior performance related to semantics by matching the dialogue spontaneous behavior based on the first semantic feature, which can effectively improve the naturalness and flexibility of speech conversion. For example, if the text expression is a question, the dialogue spontaneous behavior with a rising tone is matched; if it is a sentence with rich emotions, the speech behavior with emotional color is matched.

[0078] In the embodiments of the present application, the matching of dialogue spontaneous behavior can be realized based on a pre-trained spontaneous behavior matching model. The spontaneous behavior matching model is built based on the Transformer model architecture. By utilizing the feature encoding and mapping functions of the Transformer model, the matching of dialogue spontaneous behavior is realized in the hidden space.

[0079] S203, feature extraction is performed on the dialogue spontaneous behavior to obtain first behavior features.

[0080] Specifically, the system deeply analyzes the speech behavior in natural language, including acoustic feature extraction (such as pitch, intensity, duration) and emotional feature extraction (such as joy, anger, sadness, and other emotional classification), and converts the spontaneous behavior (such as tone, emotion, pause, stress, etc.) in natural dialogue into specific features that can be understood and processed by the model. Through feature extraction of these behaviors, the model can capture more delicate elements of speech expression, providing basic support for generating more natural and emotional speech.

[0081] S204, combining the first semantic feature and the first behavior feature to construct a first to-be-predicted speech feature combination.

[0082] In the embodiments of the present application, the features are combined based on the position information of the features in the hidden space of the Transformer model. The Transformer model is good at processing sequence data and captures global dependencies in the input data through a self-attention mechanism. Under this mechanism, different features can obtain different weights in the hidden space of the model. The model adaptively adjusts the importance of each feature in the context, thereby realizing more accurate prediction.

[0083] Specifically, the semantic features (such as text content, context information) and behavior features (such as tone, emotion, pause, etc.) are combined through the hidden space position information of the model. The position information in the hidden space not only refers to the sequence position of the features, but also refers to the importance of the relationship between the features in the multi-dimensional space. In this way, the model can more effectively allocate and fuse the weights between different features, capturing more subtle differences in speech generation.

[0084] By combining features in the hidden space of the Transformer model, voice conversion not only accurately reflects the semantic content of the text, but also naturally expresses the dialog behavior characteristics that match it. This method significantly reduces the mechanical feeling in voice conversion, making the voice output more natural, smooth, and emotionally and interactively rich. Overall, this technical solution greatly improves the quality of voice conversion and user experience, and is particularly suitable for high-quality voice application scenarios.

[0085] S205, input the first to-be-predicted voice feature combination into the pre-trained voice conversion model, and output a voice conversion result corresponding to the to-be-converted text.

[0086] In the embodiments of the present application, the voice conversion model is constructed based on the network structure of CNN (Convolutional Neural Network) + BiLSTM (Bidirectional Long Short-Term Memory Network), which combines the advantages of CNN in feature extraction and the ability of BiLSTM in sequence modeling and capturing long-term dependencies. It is usually used to process data with sequence characteristics, especially in the field of natural language processing (NLP).

[0087] Specifically, when processing voice data, CNN is mainly used to extract local time-frequency features. Voice signals are usually represented as time-frequency graphs (such as mel-spectrograms), and CNN can capture local patterns in these images, such as frequency changes, energy distributions, etc. BiLSTM handles long-distance dependencies, and when generating the voice of the current segment, it not only considers the information of the current segment, but also refers to the context before and after. The local features extracted by CNN provide basic information for BiLSTM, while BiLSTM further integrates these features in the global context. The final generated voice not only has local naturalness, but also has global coherence and emotional expression.

[0088] Through detailed analysis and quantitative processing of dialog spontaneous behavior, the model can more accurately generate voice with natural fluency and human emotional expression, reduce mechanical feeling, and improve the naturalness and user experience of voice conversion, suitable for scenarios that require high-quality voice output, such as voice assistants, virtual customer service, etc.

[0089] In the above embodiments, the present application generates more natural voice conversion results by combining semantic recognition and dialog spontaneous behavior feature extraction. By obtaining the semantic features of the text and the dialog spontaneous behavior features that match them, a voice feature combination closer to human expression is constructed, thereby reducing the mechanical feeling in voice conversion, improving the naturalness and fluency of the conversion effect, and significantly improving user experience.

[0090] Further, the step of receiving a voice conversion instruction, obtaining a text to be converted, and performing semantic recognition on the text to be converted to obtain first semantic features specifically includes:

[0091] receiving a voice conversion instruction, obtaining a text to be converted matched with the voice conversion instruction;

[0092] performing part-of-speech tagging on the text to be converted to obtain text part-of-speech tags;

[0093] performing named entity recognition on the text to be converted to obtain text named entities;

[0094] performing semantic analysis on the text to be converted based on the text part-of-speech tags and the text named entities to obtain first semantic features.

[0095] In this embodiment, first, the system receives a voice conversion instruction and obtains a text to be converted matched with the voice conversion instruction. Then, through part-of-speech tagging, the model can identify the main action (verb), subject and object (noun, etc.), and modifier (adjective, adverb, etc.) in the sentence. Part-of-speech tagging refers to assigning a part-of-speech tag to each word, such as noun, verb, adjective, etc. Part-of-speech tagging can help the model understand the sentence structure and the role of words in the sentence, and thus have a deeper understanding of the text content. Next, through named entity recognition, the system can perform special processing on these entities in voice generation, such as using a specific tone or pause when mentioning a person's name or place name, enhancing the naturalness and accuracy of the voice. Finally, based on the results of part-of-speech tagging and named entity recognition, the system performs semantic analysis on the text to extract first semantic features. Semantic analysis not only focuses on the part-of-speech and entities of words, but also analyzes the semantic relationships at the sentence level, such as subject-predicate relationship, object modification, sentiment orientation, etc.

[0096] Through the above steps, through part-of-speech tagging, named entity recognition and semantic analysis, the structure and semantics of the text to be converted are comprehensively understood, and first semantic features are generated. This multi-level analysis method enables the model to obtain more rich semantic information, which helps to generate more natural, fluent and contextually appropriate voice.

[0097] Further, the step of determining a dialog spontaneous behavior matched with the text to be converted based on the first semantic features specifically includes:

[0098] inputting the first semantic features into a pre-trained spontaneous behavior matching model, wherein the spontaneous behavior matching model is built based on a Transformer model architecture, and the latent space of the spontaneous behavior matching model pre-stores a plurality of dialog first behavior features;

[0099] encoding the first semantic features to obtain semantic encoding features;

[0100] map the semantic encoding feature to the latent space, and determine the dialog spontaneous behavior matching the first semantic feature in the latent space.

[0101] In this embodiment, first, the system inputs the first semantic feature to the pre-trained spontaneous behavior matching model, and the latent space of the spontaneous behavior matching model pre-stores several dialog behavior features. These features are based on the training of a large amount of natural dialog data, and contain spontaneous behaviors in the dialog, such as emotional expression, tone change, pause, etc. These features exist in the form of vectors in the latent space, and the model can find the spontaneous behavior feature that best matches the current semantics in the latent space by matching the input feature.

[0102] Then, after inputting the first semantic feature to the spontaneous behavior matching model, the model encodes the semantic feature to generate a semantic encoding feature. This encoding process is to convert the semantic feature into a vector representation that the model can process. The purpose of encoding is to convert complex semantic information into a calculable feature vector, so as to compare and match with the dialog behavior features stored in the latent space.

[0103] Finally, the system maps the semantic encoding feature to the pre-trained latent space, and the latent space stores multiple spontaneous behavior features. The mapping process is actually to calculate the spatial distance between the current semantic feature and the dialog behavior features in the latent space, and find the most matching behavior feature.

[0104] Through the above steps, by matching the semantic feature with the spontaneous behavior feature in the latent space of the Transformer model, the naturalness and diversity of voice conversion are successfully improved. By using the powerful representation ability of the Transformer model, the model can accurately capture semantic information and find the most matching spontaneous behavior feature in the latent space, thereby generating voice that is closer to human natural expression.

[0105] Further, the step of mapping the semantic encoding feature to the latent space and determining the dialog spontaneous behavior matching the first semantic feature in the latent space includes:

[0106] determining the position information of the semantic encoding feature in the latent space;

[0107] based on the position information of the semantic encoding feature in the latent space, calculating the spatial distance of the semantic encoding feature to various dialog first behavior features;

[0108] determining the dialog spontaneous behavior matching the first semantic feature according to the spatial distance of the semantic encoding feature to various dialog first behavior features.

[0109] The latent space is a high-dimensional vector space that stores various characteristic vectors of spontaneous behaviors in a conversation. In this space, each characteristic vector represents a specific conversational behavior, such as different tones, emotional expressions, changes in speech speed, etc.

[0110] In this embodiment, by using the distance measurement method in the high-dimensional space, the semantic encoding features are matched with the pre-stored conversational behavior characteristics in the latent space to find the best voice form of expression. Specifically, after mapping the semantic encoding features to the latent space, the position of the semantic encoding features in the latent space is first determined, i.e., the semantic encoding features are represented as vectors and placed on a specific point in the latent space, so as to convert the semantic features from abstract concepts at the semantic level to vector representations that can be calculated and processed in the latent space.

[0111] Once the position information of the semantic encoding features in the latent space is determined, the system will calculate the distance between them and various conversational behavior characteristics in the latent space, which can use common distance measurement methods such as Euclidean distance, cosine similarity, etc. The smaller the distance, the higher the similarity between the two vectors, and by quantifying their similarity, the most suitable spontaneous behavior for the current context and semantics is selected.

[0112] Through the above steps, the precise matching of semantic features and spontaneous behaviors in a conversation is achieved by using the distance calculation method in the high-dimensional latent space. By determining the position of the semantic encoding features in the latent space and calculating the distance between them and the stored spontaneous behavior characteristics, the model can dynamically select the most suitable behavior characteristics for the current context, thereby generating more natural and fluent speech.

[0113] Further, the step of combining the first semantic feature and the first behavior feature to construct the first to-be-predicted voice feature combination specifically includes:

[0114] Obtaining the position information of the semantic encoding features in the latent space to obtain the first position information;

[0115] Obtaining the position information of the spontaneous behavior in the conversation matching the first semantic feature in the latent space to obtain the second position information;

[0116] Constructing a position information sequence based on the first position information and the second position information;

[0117] Combining the first semantic feature and the first behavior feature based on the position information sequence to obtain the first to-be-predicted voice feature combination.

[0118] The first position information is the position information of the semantic encoding features, indicating the position of the semantic features in the latent space. Through the encoding process, the semantic features are mapped to the latent space, generating a vector representing their position, which reflects the specific position of the semantic features in the high-dimensional space.

[0119] The second position information is position information of the dialog spontaneous behavior feature matched to the semantic feature in the latent space, and the position reflects a specific form of the behavior feature in the high-dimensional space, such as tone, emotion, speech speed, and the like.

[0120] In the embodiment, the position information of the semantic feature captures the deep meaning of the input text, and not only reflects the surface meaning of the text, but also includes complex semantic information such as context and emotional tendency. The position information of the dialog spontaneous behavior reflects features related to emotional expression and tone change in speech generation in the latent space. By combining the first position information (position information of the semantic feature) and the second position information (position information of the spontaneous behavior feature), the model constructs a position information sequence, and the position information sequence reflects the mutual relationship between the semantic and behavior features. Finally, based on the position information sequence, the system combines the first semantic feature and the first behavior feature to generate a new predicted speech feature combination. The combination not only contains the semantic information of the text, but also integrates the spontaneous behavior feature matched thereto, thereby forming a comprehensive speech feature representation.

[0121] Through the above steps, the position information of the semantic feature and the spontaneous behavior feature in the latent space is combined to generate a comprehensive speech feature combination, which not only retains the core information of the semantic feature, but also integrates the form of the spontaneous behavior feature, thereby generating a more natural, coherent, and emotional speech.

[0122] Further, the speech conversion model includes a convolutional network layer, a long short-term memory network layer, and a fully connected layer, and the step of inputting the first predicted speech feature combination into the pre-trained speech conversion model to output a speech conversion result corresponding to the text to be converted includes:

[0123] The convolutional network layer extracts local features of the first predicted speech feature combination to obtain a plurality of first local features;

[0124] The long short-term memory network layer calculates hidden states of the plurality of first local features to obtain a first hidden state sequence feature;

[0125] The fully connected layer performs speech classification prediction on the first hidden state sequence feature to obtain a speech conversion result corresponding to the text to be converted.

[0126] In this embodiment, a combination of convolutional neural network (CNN), bidirectional long short-term memory network (BiLSTM) and fully connected layer (Fully Connected Layer) is used to realize voice conversion. The convolutional network layer is used for local feature extraction of the input first to-be-predicted voice feature combination. CNN can effectively capture local patterns and features, and performs well in processing spatial data. The long short-term memory network layer is used to calculate the hidden state of the first local feature to generate a first hidden state sequence feature. LSTM can effectively handle long-term dependencies in time series data and overcome the gradient disappearance problem in traditional RNN (recurrent neural network). The fully connected layer predicts the voice classification of the first hidden state sequence feature and outputs the final voice conversion result. The fully connected layer is usually used to map high-dimensional features to output categories.

[0127] When the BiLSTM calculates the hidden state of the first local feature, it generates a hidden state sequence containing rich context information through the processing of the forward and backward LSTM layers. The forward LSTM layer processes in the natural order of the input sequence (usually from left to right or from front to back). In this process, the LSTM unit will receive the first local feature one by one and calculate the new hidden state and cell state based on the current input and the previous hidden state. The forward LSTM layer can capture the dependencies from front to back in the sequence, i.e., it can understand how the earlier elements in the sequence affect the subsequent elements. The backward LSTM layer processes in the reverse order of the input sequence (usually from right to left or from back to front). The LSTM unit will first receive the last local feature in the sequence and then process forward one by one until the beginning of the sequence. The backward LSTM layer can capture the dependencies from back to front in the sequence, i.e., it can understand how the later elements in the sequence affect the previous elements.

[0128] Through the above steps, local features are extracted by the convolutional network layer, time series features are processed by the BiLSTM layer, and voice classification prediction is performed by the fully connected layer. Deep learning techniques are used to realize voice conversion. The convolutional network layer captures local patterns, the LSTM layer models time dependencies, and the fully connected layer provides the final voice output. This can effectively improve the naturalness and accuracy of voice conversion and ensure that the generated voice signal has high quality and smooth performance.

[0129] Further, before inputting the first to-be-predicted voice feature combination into the pre-trained voice conversion model and outputting the voice conversion result corresponding to the to-be-converted text, the following steps are further included:

[0130] Obtain a training corpus sample, wherein the training corpus sample includes mutually matched voice data samples and text data samples;

[0131] perform semantic recognition on the text data sample to obtain second semantic features;

[0132] recognize a dialogue spontaneous behavior in the voice data sample, perform feature extraction on the dialogue spontaneous behavior in the voice data sample to obtain second behavior features;

[0133] combine the second semantic features and the second behavior features to construct a second to-be-predicted voice feature combination;

[0134] input the second to-be-predicted voice feature combination into the voice conversion model;

[0135] extract local features of the second to-be-predicted voice feature combination by a convolution network layer to obtain second local features;

[0136] process the second local features by a long short-term memory network layer to output a second hidden state sequence feature;

[0137] perform voice classification prediction on the hidden state sequence feature by a full connection layer to generate a predicted voice sample;

[0138] perform iterative update on the voice conversion model based on the predicted voice sample and the voice data sample until the model is fitted.

[0139] In this embodiment, first, the system obtains a training corpus sample, the training corpus sample including mutually matched voice data samples and text data samples, the voice data samples and the text data samples needing to correspond to each other to ensure that the model can learn the mapping relationship from text to voice. Then, the text data sample is subjected to semantic recognition to obtain second semantic features, which represent semantic information of the text, such as theme, emotion, etc. Next, the dialogue spontaneous behavior in the voice data sample is recognized, and features are extracted to obtain second behavior features, which include tone, emotion, pitch, etc. Next, the second semantic features and the second behavior features are combined according to the manner of combining the first to-be-predicted voice feature combination to construct a second to-be-predicted voice feature combination. And the constructed second to-be-predicted voice feature combination is input into the model for voice conversion training to generate a voice sample matched with the input features. In the model training process, local features of the second to-be-predicted voice feature combination are extracted by a convolution network layer to obtain second local features; the second local features are processed by a long short-term memory network layer to output a second hidden state sequence feature; voice classification prediction is performed on the hidden state sequence feature by a full connection layer to generate a predicted voice sample; the voice conversion model is iteratively updated according to the difference between the generated predicted voice sample and the actual voice data sample until the model is fitted, and a trained voice conversion model is obtained.

[0140] Through the above steps, through a detailed training and optimization process, combined with CNN, BiLSTM and fully connected layers, efficient training of the speech conversion model is realized, features are extracted from the training data, predicted speech samples are generated, and model iteration updating is performed, effectively improving the quality and naturalness of the generated speech, and comprehensively using feature extraction, sequence modeling and classification prediction technology, ensuring that the model can accurately generate speech output that matches the text content and dialogue behavior.

[0141] In the above embodiment, the application discloses a speech conversion method, which belongs to the field of artificial intelligence technology. The application receives a speech conversion instruction, obtains a text to be converted, and performs semantic recognition on the text to be converted to obtain first semantic features. Based on the first semantic features, a dialogue spontaneous behavior matching the text to be converted is determined. The dialogue spontaneous behavior is feature extracted to obtain first behavior features. The first semantic features and the first behavior features are combined to construct a first predicted speech feature combination. The first predicted speech feature combination is input into a pre-trained speech conversion model to output a speech conversion result corresponding to the text to be converted. The application generates more natural speech conversion results by combining semantic recognition and dialogue spontaneous behavior feature extraction. By obtaining semantic features of the text and dialogue spontaneous behavior features matching the text, a speech feature combination closer to human expression is constructed, thereby reducing the mechanical feeling in speech conversion, improving the naturalness and fluency of the conversion effect, and significantly improving user experience.

[0142] In this embodiment, the electronic device (for example Figure 1 The server shown in the figure can receive instructions or obtain data through wired or wireless connection. It should be noted that the above wireless connection can include but is not limited to 3G / 4G connection, Wi-Fi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra-wideband) connection, and other now known or future developed wireless connection methods.

[0143] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned text information to be converted, the above-mentioned text information to be converted can also be stored in a node of a block chain.

[0144] The blockchain referred to in the present application is a new application mode of distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm and other computer technologies. The blockchain, in essence, is a decentralized database, which is a series of data blocks associated using cryptographic methods, each data block containing information of a batch of network transactions, for verifying the validity of the information (anti-fake) and generating the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer, etc.

[0145] Embodiments of the present application can acquire and process related data based on artificial intelligence technology. Among them, artificial intelligence (Artificial Intelligence, AI) is the use of digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0146] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. The software technology of artificial intelligence mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0147] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by computer readable instructions instructing related hardware, and the computer readable instructions can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments of each method. Among them, the storage medium can be a magnetic disc, an optical disc, a read-only memory (Read-Only Memory, ROM) and other non-volatile storage media, or a random access memory (Random Access Memory, RAM) and the like.

[0148] It should be understood that although each step in the flowchart of the accompanying drawings is displayed in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and they can be executed in other orders. Moreover, at least part of the steps in the flowchart of the accompanying drawings can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or sub-steps or stages of other steps.

[0149] With further reference Figure 3 to the above Figure 2 described method, the present application provides an embodiment of a voice conversion device, which corresponds to the embodiment of the method described in Figure 2 , and the device can be applied in various electronic devices.

[0150] As shown in Figure 3 , the voice conversion device 300 described in the embodiment comprises:

[0151] a semantic recognition module 301 configured to receive a voice conversion instruction, acquire a text to be converted, and perform semantic recognition on the text to be converted to obtain first semantic features;

[0152] a behavior matching module 302 configured to determine a dialogue spontaneous behavior matched with the text to be converted based on the first semantic features;

[0153] a feature extraction module 303 configured to perform feature extraction on the dialogue spontaneous behavior to obtain first behavior features;

[0154] a feature combination module 304 configured to combine the first semantic features and the first behavior features to construct a first combination of voice features to be predicted;

[0155] a voice conversion module 305 configured to input the first combination of voice features to be predicted into a pre-trained voice conversion model and output a voice conversion result corresponding to the text to be converted.

[0156] Further, the semantic recognition module 301 is specifically configured to:

[0157] receive a voice conversion instruction, acquire a text to be converted matched with the voice conversion instruction, perform part-of-speech tagging on the text to be converted to obtain text part-of-speech tags, perform named entity recognition on the text to be converted to obtain text named entities, and perform semantic analysis on the text to be converted based on the text part-of-speech tags and the text named entities to obtain the first semantic features.

[0158] Further, the behavior matching module 302 is specifically configured to:

[0159] input the first semantic features into a pre-trained spontaneous behavior matching model, wherein the spontaneous behavior matching model is built based on a Transformer model architecture, and the hidden space of the spontaneous behavior matching model is pre-stored with a plurality of dialogue first behavior features; encode the first semantic features to obtain semantic encoding features; map the semantic encoding features to the hidden space, and determine a dialogue spontaneous behavior matched with the first semantic features in the hidden space.

[0160] Further, the behavior matching module 302 is further configured to:

[0161] determining position information of the semantic coding feature in the latent space; calculating spatial distances of the semantic coding feature to various dialogue first behavior features based on the position information of the semantic coding feature in the latent space; and determining the dialogue spontaneous behavior matching the first semantic feature according to the spatial distances of the semantic coding feature to various dialogue first behavior features.

[0162] Further, the feature combination module 304 is specifically configured to:

[0163] obtain position information of the semantic coding feature in the latent space to obtain first position information; obtain position information of the dialogue spontaneous behavior matching the first semantic feature in the latent space to obtain second position information; construct a position information sequence based on the first position information and the second position information; and combine the first semantic feature and the first behavior feature based on the position information sequence to obtain the first to-be-predicted speech feature combination.

[0164] Further, the speech conversion model includes a convolutional network layer, a long short-term memory network layer, and a fully connected layer, and the speech conversion module 305 is specifically configured to:

[0165] perform local feature extraction on the first to-be-predicted speech feature combination through the convolutional network layer to obtain a plurality of first local features; calculate hidden state of the plurality of first local features through the long short-term memory network layer to obtain a first hidden state sequence feature; and perform speech classification prediction on the first hidden state sequence feature through the fully connected layer to obtain a speech conversion result corresponding to the to-be-converted text.

[0166] Further, the speech conversion apparatus 300 further includes a model training module, and the training module is specifically configured to:

[0167] obtain a training corpus sample, wherein the training corpus sample includes mutually matched speech data samples and text data samples; perform semantic recognition on the text data samples to obtain second semantic features; recognize dialogue spontaneous behaviors in the speech data samples, and perform feature extraction on the dialogue spontaneous behaviors in the speech data samples to obtain second behavior features; combine the second semantic features and the second behavior features to construct a second to-be-predicted speech feature combination; input the second to-be-predicted speech feature combination to the speech conversion model; extract local features of the second to-be-predicted speech feature combination by the convolutional network layer to obtain second local features; process the second local features by the long short-term memory network layer to output second hidden state sequence features; perform speech classification prediction on the hidden state sequence features by the fully connected layer to generate a predicted speech sample; and iteratively update the speech conversion model based on the predicted speech sample and the speech data sample until the model is fitted.

[0168] In the above embodiment, the application discloses a speech conversion device, which belongs to the technical field of artificial intelligence. The application receives a speech conversion instruction, acquires a text to be converted, and performs semantic recognition on the text to be converted to acquire first semantic features. The application determines a dialogue spontaneous behavior matched with the text to be converted based on the first semantic features. The application performs feature extraction on the dialogue spontaneous behavior to obtain first behavior features. The application combines the first semantic features and the first behavior features to construct a first to-be-predicted speech feature combination. The application inputs the first to-be-predicted speech feature combination into a pre-trained speech conversion model to output a speech conversion result corresponding to the text to be converted. The application generates a more natural speech conversion result by combining semantic recognition and feature extraction of dialogue spontaneous behavior. By acquiring semantic features of a text and dialogue spontaneous behavior features matched with the text, the application constructs a speech feature combination closer to human expression, thereby reducing mechanical feeling in speech conversion, improving naturalness and fluency of conversion effect, and significantly improving user experience.

[0169] To solve the above technical problems, the application further provides a computer device. For details, please refer to Figure 4 , Figure 4 The basic structure block diagram of the computer device of the embodiment is shown in the figure.

[0170] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 which are connected to each other through a system bus. It should be noted that only the computer device 4 with the memory 41, the processor 42, and the network interface 43 is shown in the figure, but it should be understood that all the shown components are not required to be implemented, and more or fewer components can be alternatively implemented. Among them, those skilled in the art can understand that the computer device herein is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0171] The computer device can be a desktop computer, a notebook computer, a palm computer, a cloud server, and other computing devices. The computer device can interact with a user through a keyboard, a mouse, a remote controller, a touchpad, a voice control device, and other manners.

[0172] The memory 41 includes at least one type of readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 4. Of course, the memory 41 can also include both an internal storage unit and an external storage device of the computer device 4. In this embodiment, the memory 41 is generally used to store an operating system and various application software installed on the computer device 4, such as computer readable instructions of the speech conversion method, etc. In addition, the memory 41 can also be used to temporarily store various data that have been output or will be output.

[0173] The processor 42 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer readable instructions or process data stored in the memory 41, such as computer readable instructions of the speech conversion method.

[0174] The network interface 43 can include a wireless network interface or a wired network interface, and is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0175] In the above embodiment, the application discloses a computer device, which belongs to the technical field of artificial intelligence. The application receives a voice conversion instruction, obtains to-be-converted text, and performs semantic recognition on the to-be-converted text to obtain first semantic features. The application determines a dialogue spontaneous behavior matched with the to-be-converted text based on the first semantic features. The application performs feature extraction on the dialogue spontaneous behavior to obtain first behavior features. The application combines the first semantic features and the first behavior features to construct a first to-be-predicted voice feature combination. The application inputs the first to-be-predicted voice feature combination into a pre-trained voice conversion model to output a voice conversion result corresponding to the to-be-converted text. The application generates a more natural voice conversion result by combining semantic recognition and feature extraction of dialogue spontaneous behavior. By obtaining semantic features of text and dialogue spontaneous behavior features matched with the text, the application constructs a voice feature combination closer to human expression, thereby reducing mechanical feeling in voice conversion, improving naturalness and fluency of conversion effect, and significantly improving user experience.

[0176] The application also provides another implementation, that is, a computer readable storage medium storing computer readable instructions executable by at least one processor to cause the at least one processor to perform the steps of the voice conversion method as described above.

[0177] In the above embodiment, the application discloses a computer readable storage medium, which belongs to the technical field of artificial intelligence. The application receives a voice conversion instruction, obtains to-be-converted text, and performs semantic recognition on the to-be-converted text to obtain first semantic features. The application determines a dialogue spontaneous behavior matched with the to-be-converted text based on the first semantic features. The application performs feature extraction on the dialogue spontaneous behavior to obtain first behavior features. The application combines the first semantic features and the first behavior features to construct a first to-be-predicted voice feature combination. The application inputs the first to-be-predicted voice feature combination into a pre-trained voice conversion model to output a voice conversion result corresponding to the to-be-converted text. The application generates a more natural voice conversion result by combining semantic recognition and feature extraction of dialogue spontaneous behavior. By obtaining semantic features of text and dialogue spontaneous behavior features matched with the text, the application constructs a voice feature combination closer to human expression, thereby reducing mechanical feeling in voice conversion, improving naturalness and fluency of conversion effect, and significantly improving user experience.

[0178] Those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device) execute the method described in each embodiment of the present application.

[0179] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0180] Obviously, the above-described embodiments are only a part of the embodiments of the present application, and are not all the embodiments of the present application. The preferred embodiments of the present application are given in the drawings, but do not limit the patent scope of the present application. The present application can be realized in many different forms, and conversely, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or make equivalent replacements to some technical features. Any equivalent structure made by using the contents of the specification and drawings, directly or indirectly applied to other related technical fields, is also within the scope of the patent protection of the present application.

Claims

1. A voice conversion method characterized by, The method comprises the following steps: receiving a voice conversion instruction, obtaining a text to be converted, and performing semantic recognition on the text to be converted to obtain a first semantic feature; determining a dialogue spontaneous behavior matched with the text to be converted based on the first semantic feature; extracting features of the dialogue spontaneous behavior to obtain a first behavior feature; combining the first semantic feature and the first behavior feature to construct a first to-be-predicted voice feature combination; inputting the first to-be-predicted voice feature combination into a pre-trained voice conversion model to output a voice conversion result corresponding to the text to be converted; inputting the first semantic feature into a pre-trained spontaneous behavior matching model, wherein the spontaneous behavior matching model is built based on a Transformer model architecture, and the hidden space of the spontaneous behavior matching model pre-stores a plurality of dialogue first behavior features; encoding the first semantic feature to obtain a semantic encoding feature; The step of combining the first semantic feature and the first behavior feature to construct a first to-be-predicted voice feature combination specifically comprises: obtaining position information of the semantic encoding feature in the hidden space to obtain first position information; obtaining position information of the dialogue spontaneous behavior matched with the first semantic feature in the hidden space to obtain second position information; constructing a position information sequence based on the first position information and the second position information; combining the first semantic feature and the first behavior feature based on the position information sequence to obtain the first to-be-predicted voice feature combination.

2. The voice conversion method of claim 1, wherein, The step of receiving a voice conversion instruction, obtaining a text to be converted, and performing semantic recognition on the text to be converted to obtain a first semantic feature specifically comprises: receiving the voice conversion instruction and obtaining a text to be converted matched with the voice conversion instruction; performing part-of-speech tagging on the text to be converted to obtain a text part-of-speech label; performing named entity recognition on the text to be converted to obtain a text named entity; performing semantic analysis on the text to be converted based on the text part-of-speech label and the text named entity to obtain the first semantic feature.

3. The voice conversion method of claim 1, wherein, The step of determining a dialogue spontaneous behavior matched with the text to be converted based on the first semantic feature specifically comprises: mapping the semantic encoding feature to the hidden space and determining the dialogue spontaneous behavior matched with the first semantic feature in the hidden space.

4. The voice conversion method of claim 3, wherein, The step of mapping the semantic encoding feature to the hidden space and determining the dialogue spontaneous behavior matched with the first semantic feature in the hidden space specifically comprises: determining position information of the semantic encoding feature in the hidden space; calculating spatial distances of the semantic encoding feature to various dialogue first behavior features based on the position information of the semantic encoding feature in the hidden space; determining the dialogue spontaneous behavior matched with the first semantic feature according to the spatial distances of the semantic encoding feature to various dialogue first behavior features.

5. The voice conversion method of claim 1, wherein, The voice conversion model comprises a convolutional network layer, a long short-term memory network layer and a fully connected layer, the step of inputting the first to-be-predicted voice feature combination into the pre-trained voice conversion model and outputting a voice conversion result corresponding to the to-be-converted text specifically comprises: performing local feature extraction on the first to-be-predicted voice feature combination through the convolutional network layer to obtain a plurality of first local features; calculating hidden state of the plurality of first local features through the long short-term memory network layer to obtain a first hidden state sequence feature; performing voice classification prediction on the first hidden state sequence feature through the fully connected layer to obtain the voice conversion result corresponding to the to-be-converted text.

6. The voice conversion method of claim 5, wherein, Before the step of inputting the first to-be-predicted voice feature combination into the pre-trained voice conversion model and outputting a voice conversion result corresponding to the to-be-converted text, the method further comprises: obtaining a training corpus sample, wherein the training corpus sample comprises mutually matched voice data samples and text data samples; performing semantic recognition on the text data samples to obtain a second semantic feature; recognizing a spontaneous dialogue behavior in the voice data samples, performing feature extraction on the spontaneous dialogue behavior in the voice data samples to obtain a second behavior feature; combining the second semantic feature and the second behavior feature to construct a second to-be-predicted voice feature combination; inputting the second to-be-predicted voice feature combination into the voice conversion model; extracting local features of the second to-be-predicted voice feature combination by the convolutional network layer to obtain second local features; processing the second local features by the long short-term memory network layer to output a second hidden state sequence feature; performing voice classification prediction on the hidden state sequence feature by the fully connected layer to generate a predicted voice sample; iteratively updating the voice conversion model based on the predicted voice sample and the voice data sample until the model is fitted.

7. A voice conversion apparatus characterized by comprising: The voice conversion device implements the steps of the voice conversion method according to any one of claims 1 to 6, and the voice conversion device comprises: a semantic recognition module configured to receive a voice conversion instruction, obtain a to-be-converted text, and perform semantic recognition on the to-be-converted text to obtain a first semantic feature; a behavior matching module configured to determine a spontaneous dialogue behavior matched with the to-be-converted text based on the first semantic feature; a feature extraction module configured to perform feature extraction on the spontaneous dialogue behavior to obtain a first behavior feature; a feature combination module configured to combine the first semantic feature and the first behavior feature to construct a first to-be-predicted voice feature combination; a voice conversion module configured to input the first to-be-predicted voice feature combination into a pre-trained voice conversion model and output a voice conversion result corresponding to the to-be-converted text.

8. A computer device, comprising: A device comprising a memory and a processor, wherein the memory stores computer readable instructions, and the processor implements the steps of the voice conversion method according to any one of claims 1 to 6 when executing the computer readable instructions.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer readable instructions, which, when executed by a processor, implement the steps of the voice conversion method according to any one of claims 1 to 6.

Images