Voice conversion method and device, computer equipment and storage medium

Abstract

The invention discloses a voice conversion method and device, computer equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: receiving a voice conversion instruction, obtaining a to-be-converted text, and performing semantic recognition on the to-be-converted text to obtain a first semantic feature; determining a dialogue spontaneous behavior matched with the to-be-converted text based on the first semantic feature; performing feature extraction on the dialogue spontaneous behavior to obtain a first behavior feature; combining the first semantic feature and the first behavior feature to construct a first to-be-predicted speech feature combination; and inputting the first to-be-predicted speech feature combination into a pre-trained speech conversion model, and outputting a speech conversion result corresponding to the to-be-converted text. The invention further relates to the technical field of block chains, and the to-be-converted text is stored in a block chain network. According to the invention, the mechanical feeling in voice conversion is reduced, the naturalness and fluency of the conversion effect are improved, and the user experience is remarkably improved.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and specifically relates to a speech conversion method, apparatus, computer equipment and storage medium. Background Art

[0002] At present, financial customer service has become an indispensable part of financial institutions. With the continuous development of technology, financial customer service has gradually transitioned to automation and intelligence to improve efficiency and customer experience.

[0003] In the field of financial customer service, there are already some intelligent customer service systems that use voice conversion technology. However, there are still some problems and shortcomings in the current voice conversion technology. For example, the generated voice dialogue lacks spontaneous behaviors, such as pauses, repetitions, extensions, yawns, etc., resulting in low naturalness of the conversion effect, heavy mechanical feeling, and lack of natural fluency of human voice. During the conversation, when using voice conversion, users usually feel unnatural sounds, which seriously affects the user experience and reduces the quality and acceptability of financial customer service. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a voice conversion method, apparatus, computer equipment and storage medium to solve the technical problems of the existing voice conversion technology, such as low naturalness of the conversion effect, heavy mechanical feeling, lack of natural fluency of human voice, and serious impact on the user experience.

[0005] In order to solve the above technical problems, the present application provides a method for voice conversion, which adopts the following technical solution:

[0006] A voice conversion method, comprising:

[0007] receiving a speech conversion instruction, obtaining a text to be converted, and performing semantic recognition on the text to be converted to obtain a first semantic feature;

[0008] Determining, based on the first semantic feature, a spontaneous conversation behavior that matches the text to be converted;

[0009] Extract features of spontaneous behaviors in the conversation to obtain the first behavior features;

[0010] Combining the first semantic feature and the first behavioral feature to construct a first speech feature combination to be predicted;

[0011] The first speech feature combination to be predicted is input into a pre-trained speech conversion model, and a speech conversion result corresponding to the text to be converted is output.

[0012] Further, the step of receiving a speech conversion instruction, obtaining a text to be converted, and performing semantic recognition on the text to be converted to obtain a first semantic feature specifically includes:

[0013] Receive a voice conversion instruction and obtain a text to be converted that matches the voice conversion instruction;

[0014] Perform part-of-speech tagging on the text to be converted to obtain the text part-of-speech tag;

[0015] Perform named entity recognition on the text to be converted to obtain text named entities;

[0016] The text to be converted is semantically analyzed based on the text part-of-speech tags and text named entities to obtain the first semantic feature.

[0017] Furthermore, the step of determining the spontaneous dialogue behavior matching the text to be converted based on the first semantic feature specifically includes:

[0018] Inputting the first semantic feature into a pre-trained spontaneous behavior matching model, wherein the spontaneous behavior matching model is built based on the Transformer model architecture, and the latent space of the spontaneous behavior matching model pre-stores several first behavior features of the conversation;

[0019] Encoding the first semantic feature to obtain a semantic encoding feature;

[0020] The semantically encoded features are mapped to a latent space, and the spontaneous conversational behaviors matching the first semantic features are determined in the latent space.

[0021] Furthermore, the step of mapping the semantic encoding feature to a latent space and determining in the latent space a spontaneous dialogue behavior matching the first semantic feature specifically includes:

[0022] Determine the location information of the semantic encoding features in the latent space;

[0023] Based on the position information of the semantic coding features in the latent space, the spatial distance between the semantic coding features and the first behavior features of various dialogues is calculated;

[0024] According to the spatial distance from the semantic coding feature to the first behavior features of various conversations, the spontaneous behavior of the conversation matching the first semantic feature is determined.

[0025] Furthermore, the step of combining the first semantic feature and the first behavioral feature to construct a first speech feature combination to be predicted specifically includes:

[0026] Obtaining position information of the semantic coding feature in the latent space to obtain first position information;

[0027] Obtaining position information of spontaneous dialogue behaviors matching the first semantic feature in the latent space to obtain second position information;

[0028] constructing a position information sequence based on the first position information and the second position information;

[0029] The first semantic feature and the first behavioral feature are combined based on the position information sequence to obtain a first speech feature combination to be predicted.

[0030] Furthermore, the speech conversion model includes a convolutional network layer, a long short-term memory network layer and a fully connected layer, and the steps of inputting the first speech feature combination to be predicted into the pre-trained speech conversion model and outputting the speech conversion result corresponding to the text to be converted specifically include:

[0031] Performing local feature extraction on the first speech feature combination to be predicted through a convolutional network layer to obtain a plurality of first local features;

[0032] Calculate the hidden states of several first local features through the long short-term memory network layer to obtain the first hidden state sequence feature;

[0033] The first hidden state sequence features are used through the fully connected layer to perform speech classification prediction and obtain the speech conversion result corresponding to the text to be converted.

[0034] Furthermore, before inputting the first speech feature combination to be predicted into the pre-trained speech conversion model and outputting the speech conversion result corresponding to the text to be converted, the method further includes:

[0035] Acquire a training corpus sample, wherein the training corpus sample includes a voice data sample and a text data sample that match each other;

[0036] Performing semantic recognition on the text data sample to obtain a second semantic feature;

[0037] Recognize spontaneous conversation behaviors in the voice data sample, extract features of the spontaneous conversation behaviors in the voice data sample, and obtain second behavior features;

[0038] Combining the second semantic feature and the second behavioral feature to construct a second speech feature combination to be predicted;

[0039] Inputting the second speech feature combination to be predicted into the speech conversion model;

[0040] Extracting local features of the second speech feature combination to be predicted by the convolutional network layer to obtain second local features;

[0041] The second local feature is processed by the long short-term memory network layer to output a second hidden state sequence feature;

[0042] Perform speech classification prediction on hidden state sequence features through the fully connected layer to generate predicted speech samples;

[0043] The speech conversion model is iteratively updated based on the predicted speech samples and the speech data samples until the model is fitted.

[0044] In order to solve the above technical problems, the embodiment of the present application further provides a voice conversion device, which adopts the following technical solution:

[0045] A speech conversion device, comprising:

[0046] A semantic recognition module, used to receive a speech conversion instruction, obtain a text to be converted, and perform semantic recognition on the text to be converted to obtain a first semantic feature;

[0047] A behavior matching module, used for determining a spontaneous dialogue behavior matching the text to be converted based on the first semantic feature;

[0048] A feature extraction module, used to extract features of spontaneous behaviors of the dialogue to obtain first behavior features;

[0049] A feature combination module, used for combining the first semantic feature and the first behavioral feature to construct a first speech feature combination to be predicted;

[0050] The speech conversion module is used to input the first speech feature combination to be predicted into the pre-trained speech conversion model, and output the speech conversion result corresponding to the text to be converted.

[0051] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0052] A computer device comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of any of the above-mentioned speech conversion methods when executing the computer-readable instructions.

[0053] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0054] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of any one of the above-mentioned speech conversion methods.

[0055] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0056] The present application discloses a speech conversion method, device, computer equipment and storage medium, which belongs to the field of artificial intelligence technology. The present application receives a speech conversion instruction, obtains the text to be converted, and performs semantic recognition on the text to be converted to obtain a first semantic feature; determines the spontaneous dialogue behavior matching the text to be converted based on the first semantic feature; extracts features of the spontaneous dialogue behavior to obtain a first behavior feature; combines the first semantic feature and the first behavior feature to construct a first speech feature combination to be predicted; inputs the first speech feature combination to be predicted into a pre-trained speech conversion model, and outputs the speech conversion result corresponding to the text to be converted. The present application generates a more natural speech conversion result by combining semantic recognition and feature extraction of spontaneous dialogue behavior, and constructs a speech feature combination that is closer to human expression by obtaining the semantic features of the text and the matching spontaneous dialogue behavior features, thereby reducing the mechanical feeling in speech conversion, improving the naturalness and fluency of the conversion effect, and significantly improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0058] Figure 1 An exemplary system architecture diagram is shown in which the present application can be applied;

[0059] Figure 2 A flow chart of an embodiment of a voice conversion method according to the present application is shown;

[0060] Figure 3 A schematic structural diagram of an embodiment of a speech conversion device according to the present application is shown;

[0061] Figure 4 A schematic structural diagram of an embodiment of a computer device according to the present application is shown. DETAILED DESCRIPTION

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0063] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0064] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0065] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0066] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0067] The terminal device 101 can be various electronic devices with a display screen and supporting web browsing. In addition to the laptop computer 1011, the tablet computer 1012 or the mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop computer and a desktop computer, etc.

[0068] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0069] It should be noted that the voice conversion method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the voice conversion device is generally set in the server / terminal device.

[0070] It should be understood that Figure 1 The numbers of terminal devices, networks and servers in the embodiment are merely illustrative. The above system may have any number of terminal devices, networks and servers according to implementation requirements.

[0071] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for voice conversion according to the present application. The voice conversion method comprises the following steps:

[0072] S201, receiving a speech conversion instruction, obtaining a text to be converted, and performing semantic recognition on the text to be converted to obtain a first semantic feature.

[0073] Specifically, first, the system receives a speech conversion instruction and obtains the text to be converted. Then, the text is semantically recognized to extract the first semantic feature. Through semantic recognition, the system can understand important information such as the meaning, context, and emotion of the text. The first semantic feature obtained can help the speech conversion model generate speech expressions that are consistent with the text content.

[0074] Semantic recognition refers to understanding the meaning expressed by the text, including word meaning, sentence meaning, paragraph meaning, and the intention of the entire article. This step is an important part of natural language processing (NLP). Semantic recognition operations include part-of-speech tagging, named entity recognition, and semantic analysis. Based on semantic recognition, the semantic features of the text can be further extracted. These features can be structured data, such as the results of semantic role labeling, named entity recognition, etc.; they can also be vectorized representations, such as vectors obtained by mapping words to low-dimensional vector space through word embedding.

[0075] S202: Determine, based on the first semantic feature, a spontaneous dialogue behavior that matches the text to be converted.

[0076] Spontaneous behavior in a conversation refers to the various verbal and non-verbal behaviors that occur naturally during a conversation without explicit external stimulation. Spontaneous behavior in a conversation includes but is not limited to pauses, repetitions, prolonged speech, yawns, smiles, nods, etc. These behaviors together constitute the naturalness and fluency of a conversation, are often a true reflection of the inner state of the communicators, and are also an important part of emotional exchange and information transmission in a conversation.

[0077] Specifically, the system matches spontaneous conversation behaviors based on the first semantic feature to capture natural behavior performance related to semantics. This processing method can effectively improve the naturalness and flexibility of speech conversion. For example, if the text expresses a question, spontaneous conversation behaviors with an upward tone will be matched; if it is an emotional sentence, emotional speech behaviors will be matched.

[0078] In an embodiment of the present application, the matching of spontaneous behaviors of conversations can be achieved based on a pre-trained spontaneous behavior matching model, wherein the spontaneous behavior matching model is built based on the Transformer model architecture, and the matching of spontaneous behaviors of conversations is achieved by utilizing the feature encoding and mapping functions of the Transformer model in combination with its latent space.

[0079] S203: Extract features of spontaneous behaviors of the conversation to obtain first behavior features.

[0080] Specifically, the system conducts in-depth analysis of speech behaviors in natural language, including acoustic feature extraction (such as pitch, volume, and duration) and emotional feature extraction (such as emotional classification of joy, anger, sorrow, and happiness), and converts spontaneous behaviors in natural conversations (such as tone, emotion, pauses, stress, etc.) into specific features that can be understood and processed by the model. By extracting the features of these behaviors, the model can capture more delicate elements of speech expression and provide basic support for generating more natural and emotional speech.

[0081] S204: Combining the first semantic feature and the first behavioral feature to construct a first speech feature combination to be predicted.

[0082] In the embodiment of the present application, the Transformer model is good at processing sequence data by combining the position information of the features in the latent space of the Transformer model, and the global dependencies in the input data are captured through the self-attention mechanism. Under this mechanism, different features can obtain different weights in the latent space of the model, and the model can achieve more accurate predictions by adaptively adjusting the importance of each feature in the context.

[0083] Specifically, semantic features (such as text content, contextual information) and behavioral features (such as tone, emotion, pauses, etc.) are combined through the model's latent space position information. The position information in the latent space refers not only to the sequence position of the features, but also to the importance of the relationship between these features in the multidimensional space. In this way, the model can more effectively allocate and fuse weights between different features, capturing more subtle differences in speech generation.

[0084] By combining features in the latent space of the Transformer model, speech conversion can not only more accurately reflect the semantic content of the text, but also naturally express the matching dialogue behavior characteristics. This method significantly reduces the mechanical feeling in speech conversion, making the speech output more natural, smooth, emotional and interactive. Overall, this technical solution greatly improves the quality of speech conversion and user experience, and is particularly suitable for demanding speech application scenarios.

[0085] S205: Input the first speech feature combination to be predicted into a pre-trained speech conversion model, and output a speech conversion result corresponding to the text to be converted.

[0086] In an embodiment of the present application, the speech conversion model is constructed based on the network structure of CNN (convolutional neural network) + Bi LSTM (bidirectional long short-term memory network), which combines the advantages of CNN in feature extraction with the ability of Bi LSTM in sequence modeling and capturing long-term dependencies. It is usually used to process data with sequence characteristics, especially in the field of natural language processing (NLP).

[0087] Specifically, when processing speech data, CNN is mainly used to extract local time-frequency features. Speech signals are usually expressed as time-frequency graphs (such as Mel-spectrograms), and CNN can capture local patterns in these images through convolution operations, such as frequency changes, energy distribution, etc. Bi LSTM handles long-distance dependencies. When generating the speech of the current segment, it not only considers the information of the current segment, but also refers to the previous and subsequent contexts. The local features extracted by CNN provide basic information for Bi LSTM, and Bi LSTM further integrates these features in the global context. The final generated speech is not only local naturalness, but also global coherence and emotional expression.

[0088] Through detailed analysis and quantitative processing of spontaneous conversational behaviors, the model can more accurately generate speech with natural fluency and human emotional expression, reduce the sense of mechanicalness, and improve the naturalness of speech conversion and user experience. It is suitable for scenarios that require high-quality voice output, such as voice assistants, virtual customer service, etc.

[0089] In the above embodiments, the present application generates a more natural speech conversion result by combining semantic recognition and feature extraction of spontaneous conversation behavior. By acquiring the semantic features of the text and the matching spontaneous conversation behavior features, a speech feature combination that is closer to human expression is constructed, thereby reducing the mechanical feel in speech conversion, improving the naturalness and fluency of the conversion effect, and significantly improving the user experience.

[0090] Further, the step of receiving a speech conversion instruction, obtaining a text to be converted, and performing semantic recognition on the text to be converted to obtain a first semantic feature specifically includes:

[0091] Receive a voice conversion instruction and obtain a text to be converted that matches the voice conversion instruction;

[0092] Perform part-of-speech tagging on the text to be converted to obtain the text part-of-speech tag;

[0093] Perform named entity recognition on the text to be converted to obtain text named entities;

[0094] The text to be converted is semantically analyzed based on the text part-of-speech tags and text named entities to obtain the first semantic feature.

[0095] In this embodiment, first, the system receives a speech conversion instruction and obtains the text to be converted that matches the speech conversion instruction. Then, through part-of-speech tagging, the model can identify the main action (verb), subject and object (noun, etc.), and modifiers (adjective, adverb, etc.) in the sentence. Part-of-speech tagging refers to assigning a part-of-speech label to each word, such as noun, verb, adjective, etc. Part-of-speech tagging can help the model understand the sentence structure and the role of the word in the sentence, and then have a deeper understanding of the text content. Then, by identifying named entities, the system can perform special processing on these entities in speech generation, such as using specific intonation or pauses when mentioning names of people or places, to enhance the naturalness and accuracy of the speech. Finally, based on the results of part-of-speech tagging and named entity recognition, the system performs semantic analysis on the text and extracts the first semantic feature. The semantic analysis not only focuses on the part-of-speech and entity of the word, but also analyzes the semantic relationship at the sentence level, such as subject-predicate relationship, object modification, emotional tendency, etc.

[0096] Through the above steps, through part-of-speech tagging, named entity recognition and semantic analysis, we can fully understand the structure and semantics of the text to be converted and generate the first semantic features. This multi-level analysis method enables the model to obtain richer semantic information and helps to generate more natural, fluent and contextual speech.

[0097] Furthermore, the step of determining the spontaneous dialogue behavior matching the text to be converted based on the first semantic feature specifically includes:

[0098] Inputting the first semantic feature into a pre-trained spontaneous behavior matching model, wherein the spontaneous behavior matching model is built based on the Transformer model architecture, and the latent space of the spontaneous behavior matching model pre-stores several first behavior features of the conversation;

[0099] Encoding the first semantic feature to obtain a semantic encoding feature;

[0100] The semantically encoded features are mapped to a latent space, and the spontaneous conversational behaviors matching the first semantic features are determined in the latent space.

[0101] In this embodiment, first, the system inputs the first semantic feature into the pre-trained spontaneous behavior matching model, and the latent space of the spontaneous behavior matching model pre-stores several types of dialogue behavior features. These features are obtained based on the training of a large amount of natural dialogue data, and include spontaneous behaviors in the dialogue, such as emotional expression, intonation changes, pauses, etc. These features exist in the latent space in the form of vectors. By matching the input features, the model can find the spontaneous behavior features that best match the current semantics in the latent space.

[0102] Then, after the first semantic feature is input into the spontaneous behavior matching model, the model encodes the semantic feature to generate a semantic encoding feature. This encoding process converts the semantic feature into a vector representation that the model can process. The purpose of encoding is to convert complex semantic information into a computable feature vector so that it can be compared and matched with the dialogue behavior features stored in the latent space.

[0103] Finally, the system maps the semantic encoding features to the pre-trained latent space, which stores a variety of spontaneous behavior features. The mapping process actually calculates the spatial distance between the current semantic features and the dialogue behavior features in the latent space to find the most matching behavior features.

[0104] Through the above steps, the naturalness and diversity of speech conversion are successfully improved by matching semantic features with spontaneous behavioral features in the latent space of the Transformer model. By leveraging the powerful representation ability of the Transformer model, the model can accurately capture semantic information and find the most matching spontaneous behavioral features in the latent space, thereby generating speech that is closer to natural human expression.

[0105] Furthermore, the step of mapping the semantic encoding feature to a latent space and determining in the latent space a spontaneous dialogue behavior matching the first semantic feature specifically includes:

[0106] Determine the location information of the semantic encoding features in the latent space;

[0107] Based on the position information of the semantic coding features in the latent space, the spatial distance between the semantic coding features and the first behavior features of various dialogues is calculated;

[0108] According to the spatial distance from the semantic coding feature to the first behavior features of various conversations, the spontaneous behavior of the conversation matching the first semantic feature is determined.

[0109] The latent space is a high-dimensional vector space that stores the feature vectors of various spontaneous behaviors in conversations. In this space, each feature vector represents a specific conversation behavior, such as different tones, emotional expressions, and changes in speech speed.

[0110] In this embodiment, the semantic coding features are matched with the pre-stored dialogue behavior features in the latent space by using the distance measurement method in the high-dimensional space, so as to find the best speech representation. Specifically, after the semantic coding features are mapped to the latent space, their position in the latent space must first be determined, that is, the semantic coding features are represented as vectors and placed at a specific point in the latent space, so as to convert the semantic features from abstract concepts at the semantic level into vector representations that can be calculated and processed in the latent space.

[0111] Once the location information of the semantic encoding feature in the latent space is determined, the system will calculate the distance between it and various dialogue behavior features in the latent space. This can be done using common distance measurement methods such as Euclidean distance and cosine similarity. The smaller the distance, the higher the similarity between the two vectors. By quantifying the similarity between them, the spontaneous behavior that best matches the current context and semantics can be screened out.

[0112] Through the above steps, the distance calculation method in the high-dimensional latent space is used to achieve accurate matching of semantic features and spontaneous behaviors in conversation. By determining the position of the semantic encoding features in the latent space and calculating the distance between them and the stored spontaneous behavior features, the model can dynamically select the behavior features that best suit the current context, thereby generating more natural and fluent speech.

[0113] Furthermore, the step of combining the first semantic feature and the first behavioral feature to construct a first speech feature combination to be predicted specifically includes:

[0114] Obtaining position information of the semantic coding feature in the latent space to obtain first position information;

[0115] Obtaining position information of spontaneous dialogue behaviors matching the first semantic feature in the latent space to obtain second position information;

[0116] constructing a position information sequence based on the first position information and the second position information;

[0117] The first semantic feature and the first behavioral feature are combined based on the position information sequence to obtain a first speech feature combination to be predicted.

[0118] The first position information is the position information of the semantic coding feature, which indicates the position of the semantic feature in the latent space. The semantic feature is mapped to the latent space through the encoding process, and a vector representing its position is generated. This vector reflects the specific position of the semantic feature in the high-dimensional space.

[0119] The second position information is the position information of the spontaneous behavior feature of the conversation that best matches the semantic feature in the latent space. This position reflects the specific form of expression of the behavior feature in the high-dimensional space, such as tone, emotion, speaking speed, etc.

[0120] In this embodiment, the position information of the semantic feature captures the deep meaning of the input text, which not only reflects the surface meaning of the text, but also includes complex semantic information such as context and emotional tendency. The position information of the spontaneous behavior of the dialogue represents the features related to emotional expression, intonation change, etc. in speech generation in the latent space. By combining the first position information (the position information of the semantic feature) and the second position information (the position information of the spontaneous behavioral feature), the model constructs a position information sequence, which reflects the relationship between semantic and behavioral features. Finally, based on the position information sequence, the system combines the first semantic feature and the first behavioral feature to generate a new combination of speech features to be predicted. This combination not only contains the semantic information of the text, but also incorporates the spontaneous behavioral features that match it, thereby forming a comprehensive speech feature representation.

[0121] Through the above steps, a comprehensive speech feature combination is generated by combining the position information of semantic features and spontaneous behavioral features in the latent space. It not only retains the core information of semantic features, but also incorporates the expression form of spontaneous behavioral features, thereby generating more natural, coherent and emotional speech.

[0122] Furthermore, the speech conversion model includes a convolutional network layer, a long short-term memory network layer and a fully connected layer, and the steps of inputting the first speech feature combination to be predicted into the pre-trained speech conversion model and outputting the speech conversion result corresponding to the text to be converted specifically include:

[0123] Performing local feature extraction on the first speech feature combination to be predicted through a convolutional network layer to obtain a plurality of first local features;

[0124] Calculate the hidden states of several first local features through the long short-term memory network layer to obtain the first hidden state sequence feature;

[0125] The first hidden state sequence features are used through the fully connected layer to perform speech classification prediction and obtain the speech conversion result corresponding to the text to be converted.

[0126] In this embodiment, a combination of a convolutional neural network (CNN), a bidirectional long short-term memory network (B i LSTM) and a fully connected layer (Fully Connected Layer) is used to realize voice conversion. The convolutional network layer is used to extract local features of the input first voice feature combination to be predicted. CNN can effectively capture the patterns and features of the local area and perform well when processing spatial data. The long short-term memory network layer is used to calculate the hidden state of the first local feature and generate the first hidden state sequence feature. LSTM can effectively process the long-term dependencies in the time series data and overcome the gradient disappearance problem in the traditional RNN (recursive neural network). The fully connected layer performs voice classification prediction on the first hidden state sequence feature and outputs the final voice conversion result. The fully connected layer is usually used to map high-dimensional features to output categories.

[0127] When calculating the hidden state of the first local feature, Bi LSTM generates a hidden state sequence containing rich contextual information through the processing of the forward and backward LSTM layers. The forward LSTM layer processes the input sequence in the natural order (usually from left to right or from front to back). In this process, the LSTM unit receives the first local feature in turn and calculates the new hidden state and cell state based on the current input and the previous hidden state. The forward LSTM layer can capture the dependencies from front to back in the sequence, that is, it can understand how the earlier elements in the sequence affect the subsequent elements. The backward LSTM layer processes the input sequence in the reverse order (usually from right to left or from back to front). The LSTM unit first receives the last local feature in the sequence, and then processes forward in turn until the beginning of the sequence. The backward LSTM layer can capture the dependencies from back to front in the sequence, that is, it can understand how the later elements in the sequence affect the previous elements.

[0128] Through the above steps, the convolutional network layer extracts local features, the BiLSTM layer processes time series features, and the fully connected layer performs speech classification prediction, and the deep learning technology is used comprehensively to achieve speech conversion. The convolutional network layer captures local patterns, the LSTM layer models time dependencies, and the fully connected layer provides the final speech output, which can effectively improve the naturalness and accuracy of speech conversion and ensure that the generated speech signal has high quality and smooth performance.

[0129] Furthermore, before inputting the first speech feature combination to be predicted into the pre-trained speech conversion model and outputting the speech conversion result corresponding to the text to be converted, the method further includes:

[0130] Acquire a training corpus sample, wherein the training corpus sample includes a voice data sample and a text data sample that match each other;

[0131] Performing semantic recognition on the text data sample to obtain a second semantic feature;

[0132] Recognize spontaneous conversation behaviors in the voice data sample, extract features of the spontaneous conversation behaviors in the voice data sample, and obtain second behavior features;

[0133] Combining the second semantic feature and the second behavioral feature to construct a second speech feature combination to be predicted;

[0134] Inputting the second speech feature combination to be predicted into the speech conversion model;

[0135] Extracting local features of the second speech feature combination to be predicted by the convolutional network layer to obtain second local features;

[0136] The second local feature is processed by the long short-term memory network layer to output a second hidden state sequence feature;

[0137] Perform speech classification prediction on hidden state sequence features through the fully connected layer to generate predicted speech samples;

[0138] The speech conversion model is iteratively updated based on the predicted speech samples and the speech data samples until the model is fitted.

[0139] In this embodiment, first, the system obtains a training corpus sample, which includes a matching voice data sample and a text data sample. The voice data sample and the text data sample need to correspond to each other to ensure that the model can learn the mapping relationship from text to speech. Then, the text data sample is semantically recognized to obtain a second semantic feature, which represents the semantic information of the text, such as the theme, emotion, etc. Next, the spontaneous behavior of the conversation in the voice data sample is identified, and the feature is extracted to obtain the second behavior feature, which includes tone, emotion, pitch, etc. Next, the second semantic feature and the second behavior feature are combined in the above-mentioned way of combining the first voice feature combination to be predicted to construct a second voice feature combination to be predicted. The constructed second voice feature combination to be predicted is input into the model for voice conversion training to generate a voice sample that matches the input feature. During the model training process, the convolutional network layer extracts the local features of the second speech feature combination to be predicted to obtain the second local features; the long short-term memory network layer processes the second local features to output the second hidden state sequence features; the fully connected layer performs speech classification prediction on the hidden state sequence features to generate predicted speech samples; based on the difference between the generated predicted speech samples and the actual speech data samples, the speech conversion model is iteratively updated until the model is fitted to obtain a trained speech conversion model.

[0140] Through the above steps, through a detailed training and optimization process, combined with CNN, BiLSTM and fully connected layers, efficient training of the speech conversion model is achieved. By extracting features from the training data, generating predicted speech samples, and iteratively updating the model, the quality and naturalness of speech generation are effectively improved. The comprehensive use of feature extraction, sequence modeling and classification prediction techniques ensures that the model can accurately generate speech output that matches the text content and dialogue behavior.

[0141] In the above embodiment, the present application discloses a speech conversion method, which belongs to the field of artificial intelligence technology. The present application receives a speech conversion instruction, obtains the text to be converted, and performs semantic recognition on the text to be converted to obtain a first semantic feature; determines the spontaneous dialogue behavior matching the text to be converted based on the first semantic feature; extracts features of the spontaneous dialogue behavior to obtain a first behavior feature; combines the first semantic feature and the first behavior feature to construct a first speech feature combination to be predicted; inputs the first speech feature combination to be predicted into a pre-trained speech conversion model, and outputs a speech conversion result corresponding to the text to be converted. The present application generates a more natural speech conversion result by combining semantic recognition and feature extraction of spontaneous dialogue behavior, and constructs a speech feature combination that is closer to human expression by obtaining the semantic features of the text and the matching spontaneous dialogue behavior features, thereby reducing the mechanical sense in speech conversion, improving the naturalness and fluency of the conversion effect, and significantly improving the user experience.

[0142] In this embodiment, the electronic device (eg Figure 1 The server shown in the figure can receive instructions or obtain data through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G connection, Wi-Fi connection, Bluetooth connection, Wi MAX connection, Zigbee connection, UWB (Ultra Wi-Fi Band) connection, and other wireless connection methods currently known or to be developed in the future.

[0143] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned text information to be converted, the above-mentioned text information to be converted can also be stored in a node of a blockchain.

[0144] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0145] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0146] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0147] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0148] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0149] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a speech conversion device, and the embodiment of the device is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0150] like Figure 3 As shown, the speech conversion device 300 described in this embodiment includes:

[0151] The semantic recognition module 301 is used to receive a speech conversion instruction, obtain a text to be converted, and perform semantic recognition on the text to be converted to obtain a first semantic feature;

[0152] A behavior matching module 302, configured to determine a spontaneous dialogue behavior matching the text to be converted based on the first semantic feature;

[0153] A feature extraction module 303 is used to extract features of spontaneous behaviors of the dialogue to obtain first behavior features;

[0154] A feature combination module 304, configured to combine the first semantic feature and the first behavioral feature to construct a first speech feature combination to be predicted;

[0155] The speech conversion module 305 is used to input the first speech feature combination to be predicted into the pre-trained speech conversion model, and output the speech conversion result corresponding to the text to be converted.

[0156] Furthermore, the semantic recognition module 301 is specifically used for:

[0157] Receive a speech conversion instruction and obtain a text to be converted that matches the speech conversion instruction; perform part-of-speech tagging on the text to be converted to obtain a text part-of-speech tag; perform named entity recognition on the text to be converted to obtain a text named entity; perform semantic analysis on the text to be converted based on the text part-of-speech tag and the text named entity to obtain a first semantic feature.

[0158] Furthermore, the behavior matching module 302 is specifically used for:

[0159] The first semantic feature is input into a pre-trained spontaneous behavior matching model, wherein the spontaneous behavior matching model is built based on the Transformer model architecture, and the latent space of the spontaneous behavior matching model pre-stores several first behavior features of the conversation; the first semantic feature is encoded to obtain a semantic encoding feature; the semantic encoding feature is mapped to the latent space, and the spontaneous behavior of the conversation matching the first semantic feature is determined in the latent space.

[0160] Furthermore, the behavior matching module 302 is further used to:

[0161] Determine the position information of the semantic coding feature in the latent space; calculate the spatial distance from the semantic coding feature to various first behavior features of the dialogue based on the position information of the semantic coding feature in the latent space; determine the spontaneous behavior of the dialogue matching the first semantic feature according to the spatial distance from the semantic coding feature to various first behavior features of the dialogue.

[0162] Furthermore, the feature combination module 304 is specifically used for:

[0163] Acquire the position information of the semantic coding feature in the latent space to obtain the first position information; acquire the position information of the spontaneous behavior of the dialogue matching the first semantic feature in the latent space to obtain the second position information; construct a position information sequence based on the first position information and the second position information; combine the first semantic feature and the first behavior feature based on the position information sequence to obtain a first speech feature combination to be predicted.

[0164] Furthermore, the speech conversion model includes a convolutional network layer, a long short-term memory network layer and a fully connected layer, and the speech conversion module 305 is specifically used for:

[0165] The first speech feature combination to be predicted is subjected to local feature extraction through the convolutional network layer to obtain a number of first local features; the hidden states of a number of first local features are calculated through the long short-term memory network layer to obtain first hidden state sequence features; the first hidden state sequence features are subjected to speech classification prediction through the fully connected layer to obtain the speech conversion result corresponding to the text to be converted.

[0166] Furthermore, the speech conversion device 300 further includes a model training module, which is specifically used for:

[0167] Acquire training corpus samples, wherein the training corpus samples include voice data samples and text data samples that match each other; perform semantic recognition on the text data samples to obtain second semantic features; recognize spontaneous conversation behaviors in the voice data samples, perform feature extraction on the spontaneous conversation behaviors in the voice data samples, and obtain second behavior features; combine the second semantic features and the second behavior features to construct a second voice feature combination to be predicted; input the second voice feature combination to be predicted into a voice conversion model; extract local features of the second voice feature combination to be predicted by a convolutional network layer to obtain second local features; process the second local features by a long short-term memory network layer and output second hidden state sequence features; perform voice classification prediction on the hidden state sequence features by a fully connected layer to generate predicted voice samples; iteratively update the voice conversion model based on the predicted voice samples and the voice data samples until the model is fitted.

[0168] In the above embodiment, the present application discloses a speech conversion device, which belongs to the field of artificial intelligence technology. The present application receives a speech conversion instruction, obtains the text to be converted, and performs semantic recognition on the text to be converted to obtain a first semantic feature; determines the spontaneous dialogue behavior matching the text to be converted based on the first semantic feature; extracts features of the spontaneous dialogue behavior to obtain a first behavior feature; combines the first semantic feature and the first behavior feature to construct a first speech feature combination to be predicted; inputs the first speech feature combination to be predicted into a pre-trained speech conversion model, and outputs a speech conversion result corresponding to the text to be converted. The present application generates a more natural speech conversion result by combining semantic recognition and feature extraction of spontaneous dialogue behavior, and constructs a speech feature combination that is closer to human expression by obtaining the semantic features of the text and the matching spontaneous dialogue behavior features, thereby reducing the mechanical feeling in speech conversion, improving the naturalness and fluency of the conversion effect, and significantly improving the user experience.

[0169] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0170] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 4 having a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0171] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0172] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Med ia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the voice conversion method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0173] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the computer-readable instructions or process data stored in the memory 41, such as computer-readable instructions for running the speech conversion method.

[0174] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0175] In the above embodiment, the present application discloses a computer device, which belongs to the field of artificial intelligence technology. The present application receives a speech conversion instruction, obtains the text to be converted, and performs semantic recognition on the text to be converted to obtain a first semantic feature; determines the spontaneous dialogue behavior matching the text to be converted based on the first semantic feature; extracts features of the spontaneous dialogue behavior to obtain a first behavior feature; combines the first semantic feature and the first behavior feature to construct a first speech feature combination to be predicted; inputs the first speech feature combination to be predicted into a pre-trained speech conversion model, and outputs a speech conversion result corresponding to the text to be converted. The present application generates a more natural speech conversion result by combining semantic recognition and feature extraction of spontaneous dialogue behavior, and constructs a speech feature combination that is closer to human expression by obtaining the semantic features of the text and the matching spontaneous dialogue behavior features, thereby reducing the mechanical sense in speech conversion, improving the naturalness and fluency of the conversion effect, and significantly improving the user experience.

[0176] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the speech conversion method as described above.

[0177] In the above embodiment, the present application discloses a computer-readable storage medium, which belongs to the field of artificial intelligence technology. The present application receives a speech conversion instruction, obtains the text to be converted, and performs semantic recognition on the text to be converted to obtain a first semantic feature; determines the spontaneous dialogue behavior matching the text to be converted based on the first semantic feature; extracts features of the spontaneous dialogue behavior to obtain a first behavior feature; combines the first semantic feature and the first behavior feature to construct a first speech feature combination to be predicted; inputs the first speech feature combination to be predicted into a pre-trained speech conversion model, and outputs a speech conversion result corresponding to the text to be converted. The present application generates a more natural speech conversion result by combining semantic recognition and feature extraction of spontaneous dialogue behavior, and constructs a speech feature combination that is closer to human expression by obtaining the semantic features of the text and the matching spontaneous dialogue behavior features, thereby reducing the mechanical feeling in speech conversion, improving the naturalness and fluency of the conversion effect, and significantly improving the user experience.

[0178] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0179] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0180] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A speech conversion method, characterized in that: include: Receiving a speech conversion instruction, obtaining a text to be converted, and performing semantic recognition on the text to be converted to obtain a first semantic feature; Determining, based on the first semantic feature, a spontaneous conversation behavior that matches the text to be converted; Extracting features of the spontaneous behavior of the conversation to obtain a first behavior feature; Combining the first semantic feature and the first behavioral feature to construct a first speech feature combination to be predicted; The first speech feature combination to be predicted is input into a pre-trained speech conversion model, and a speech conversion result corresponding to the text to be converted is output.

2. The voice conversion method according to claim 1, characterized in that: The step of receiving a speech conversion instruction, obtaining a text to be converted, and performing semantic recognition on the text to be converted to obtain a first semantic feature specifically includes: Receiving the speech conversion instruction, and acquiring the text to be converted that matches the speech conversion instruction; Performing part-of-speech tagging on the text to be converted to obtain a text part-of-speech tag; Performing named entity recognition on the text to be converted to obtain text named entities; The text to be converted is semantically analyzed based on the text part-of-speech tag and the text named entity to obtain the first semantic feature.

3. The voice conversion method according to claim 1, wherein: The step of determining the spontaneous dialogue behavior matching the text to be converted based on the first semantic feature specifically includes: Inputting the first semantic feature into a pre-trained spontaneous behavior matching model, wherein the spontaneous behavior matching model is built based on a Transformer model architecture, and a latent space of the spontaneous behavior matching model pre-stores a plurality of first behavior features of the conversation; Encoding the first semantic feature to obtain a semantic coding feature; The semantic encoding feature is mapped to the latent space, and a spontaneous conversation behavior matching the first semantic feature is determined in the latent space.

4. The voice conversion method according to claim 3, characterized in that: The step of mapping the semantic encoding feature to the latent space and determining the spontaneous dialogue behavior matching the first semantic feature in the latent space specifically includes: Determining position information of the semantic encoding feature in the latent space; Based on the position information of the semantic coding feature in the latent space, calculating the spatial distance between the semantic coding feature and various first behavior features of the dialogue; According to the spatial distances from the semantic coding feature to various dialogue first behavior features, a dialogue spontaneous behavior matching the first semantic feature is determined.

5. The voice conversion method according to claim 4, characterized in that: The step of combining the first semantic feature and the first behavioral feature to construct a first speech feature combination to be predicted specifically includes: Acquire position information of the semantic coding feature in the latent space to obtain first position information; Acquire position information of the spontaneous behavior of the conversation matching the first semantic feature in the latent space to obtain second position information; constructing a position information sequence based on the first position information and the second position information; The first semantic feature and the first behavioral feature are combined based on the position information sequence to obtain the first speech feature combination to be predicted.

6. The voice conversion method according to claim 1, wherein: The speech conversion model includes a convolutional network layer, a long short-term memory network layer and a fully connected layer. The step of inputting the first speech feature combination to be predicted into the pre-trained speech conversion model and outputting the speech conversion result corresponding to the text to be converted specifically includes: Performing local feature extraction on the first combination of speech features to be predicted through the convolutional network layer to obtain a plurality of first local features; Calculating the hidden states of a plurality of the first local features through the long short-term memory network layer to obtain a first hidden state sequence feature; The first hidden state sequence feature is subjected to speech classification prediction through the fully connected layer to obtain a speech conversion result corresponding to the text to be converted.

7. The voice conversion method according to claim 6, characterized in that: Before inputting the first speech feature combination to be predicted into the pre-trained speech conversion model and outputting the speech conversion result corresponding to the text to be converted, the method further includes: Acquire a training corpus sample, wherein the training corpus sample includes a voice data sample and a text data sample that match each other; Performing semantic recognition on the text data sample to obtain a second semantic feature; Identifying spontaneous conversation behaviors in the voice data sample, and extracting features of the spontaneous conversation behaviors in the voice data sample to obtain second behavior features; Combining the second semantic feature and the second behavioral feature to construct a second speech feature combination to be predicted; Inputting the second speech feature combination to be predicted into the speech conversion model; Extracting local features of the second combination of speech features to be predicted by the convolutional network layer to obtain second local features; The long short-term memory network layer processes the second local feature and outputs a second hidden state sequence feature; Perform speech classification prediction on the hidden state sequence features through the fully connected layer to generate a predicted speech sample; The speech conversion model is iteratively updated based on the predicted speech sample and the speech data sample until the model is fitted.

8. A speech conversion device, characterized in that: include: A semantic recognition module, used to receive a speech conversion instruction, obtain a text to be converted, and perform semantic recognition on the text to be converted to obtain a first semantic feature; A behavior matching module, used for determining a spontaneous dialogue behavior matching the text to be converted based on the first semantic feature; A feature extraction module, used for extracting features of the spontaneous behavior of the conversation to obtain a first behavior feature; A feature combination module, used for combining the first semantic feature and the first behavioral feature to construct a first speech feature combination to be predicted; The speech conversion module is used to input the first speech feature combination to be predicted into a pre-trained speech conversion model, and output the speech conversion result corresponding to the text to be converted.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the speech conversion method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the speech conversion method according to any one of claims 1 to 7 are implemented.

Images