Voice generation method and device, computer equipment and storage medium

Abstract

The embodiment of the invention provides a voice generation method and device, computer equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: performing voice feature extraction on original voice data through an original voice conversion model to obtain a reference voice feature and a selected voice feature, wherein the reference voice feature has a target object sound style; performing similarity measurement on the reference voice features and the selected voice features to obtain feature similarity data; screening the selected voice features according to the feature similarity data to obtain target voice features; wherein the target voice feature does not have a target object sound style; performing model training on the original speech generation model according to the target speech features, the reference speech features and the training text to obtain a target speech generation model; and generating target voice data through the target voice generation model and the target text. According to the embodiment of the invention, the voice generation stability can be improved, the dependence on the reference audio is reduced, and the voice generation flexibility is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a speech generation method and apparatus, a computer device, and a storage medium. Background Art

[0002] Speech generation technology, also known as text-to-speech technology, is a technology that converts text information into spoken language. In related technologies, text-to-speech generation is achieved based on large speech models. However, when speech is generated only through large language models, it is easy to mix the voices of multiple speakers, resulting in unstable generated speech. In order to control the identity and style of the speaker in the generated speech, each speech generation requires providing the speaker's reference audio, which not only increases the dependence on the reference audio, but also limits the flexibility of speech generation. Therefore, how to improve the stability of generated speech, reduce the dependence on reference audio, and make speech generation more flexible has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a speech generation method and apparatus, a computer device and a storage medium, aiming to improve the stability of speech generation, reduce the dependence on reference audio, and improve the flexibility of speech generation.

[0004] To achieve the above object, a first aspect of an embodiment of the present application provides a speech generation method, the method comprising:

[0005] Get raw speech data and training text;

[0006] Extracting speech features from the original speech data through a preset original speech conversion model to obtain a speech feature set; wherein the speech feature set includes reference speech features and selected speech features, and the reference speech features have the sound style of the target object;

[0007] Performing similarity measurement on the reference speech feature and the selected speech feature to obtain feature similarity data;

[0008] Performing feature screening on the selected speech feature according to the feature similarity data to obtain a target speech feature; wherein the target speech feature does not have the voice style of the target object;

[0009] Performing model training on a preset original speech generation model according to the target speech feature, the reference speech feature and the training text to obtain a target speech generation model;

[0010] Speech generation is performed using the target speech generation model and the target text to obtain target speech data.

[0011] In some embodiments, the original speech conversion model includes: a speech encoder, a first feature extractor and a second feature extractor; the speech feature extraction of the original speech data by the preset original speech conversion model to obtain a speech feature set includes:

[0012] Extracting features from the original speech data by using the speech encoder to obtain candidate speech features;

[0013] Extracting speech features from the candidate speech features by the first feature extractor to obtain the reference speech features;

[0014] The candidate speech feature is subjected to speech feature extraction by the second feature extractor and the reference speech feature to obtain the selected speech feature; wherein the sum of the weights of the reference speech feature and the selected speech feature is 1.

[0015] In some embodiments, the step of performing feature screening on the selected speech feature according to the feature similarity data to obtain the target speech feature includes:

[0016] Performing feature quantization processing on the selected speech feature according to preset quantization features to obtain preliminary speech features;

[0017] The preliminary speech features are screened according to the feature similarity data to obtain the target speech features.

[0018] In some embodiments, the step of performing feature quantization processing on the selected speech feature according to the preset quantization feature to obtain the preliminary speech feature includes:

[0019] Calculating the feature distance between the preset quantitative feature and the selected speech feature to obtain a speech feature distance;

[0020] Filtering out a selected speech feature from the preset quantitative features according to the speech feature distance;

[0021] The selected speech features are concatenated to obtain the preliminary speech features.

[0022] In some embodiments, after performing feature screening on the selected speech feature according to the feature similarity data to obtain the target speech feature, the method further includes:

[0023] Concatenating the target speech feature and the reference speech feature to obtain a concatenated speech feature;

[0024] Perform speech reconstruction according to the concatenated speech features to obtain reconstructed speech data;

[0025] Performing loss calculation on the reconstructed speech data and the original speech data to obtain reconstructed loss data;

[0026] Perform loss calculation on a preset similarity threshold and the feature similarity data to obtain similarity loss data;

[0027] Performing object recognition according to the reference speech features to obtain object recognition information;

[0028] Calculate the loss based on the preset target object information and the object identification information to obtain identification loss data;

[0029] The model parameters of the original speech conversion model are adjusted according to the reconstruction loss data, the similarity loss data and the recognition loss data to obtain a target speech conversion model.

[0030] In some embodiments, the original speech generation model includes: an original text encoder and an original decoder; the model training of the preset original speech generation model according to the target speech feature, the reference speech feature and the training text to obtain the target speech generation model includes:

[0031] Performing semantic extraction on the training text by using the original text encoder to obtain text semantic information;

[0032] The text semantic information is converted into speech features by using the original decoder and the reference speech features to obtain predicted speech features; wherein the predicted speech features do not have the sound style of the target object;

[0033] The model parameters of the original speech generation model are adjusted according to the predicted speech features and the target speech features to obtain the target speech generation model.

[0034] In some embodiments, the original text encoder includes: a word segmentation layer, a semantic analysis layer and an encoding layer; the semantic extraction of the training text by the original text encoder to obtain text semantic information includes:

[0035] Perform word segmentation processing on the training text through the word segmentation layer to obtain training phrases;

[0036] Performing semantic analysis on the training phrase through the semantic analysis layer to obtain semantic analysis information;

[0037] Semantic extraction is performed on the training text through the encoding layer and the semantic parsing information to obtain the text semantic information.

[0038] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a speech generation device, the device comprising:

[0039] An acquisition module is used to acquire original speech data and training text;

[0040] A speech feature extraction module, used to extract speech features from the original speech data through a preset original speech conversion model to obtain a speech feature set; wherein the speech feature set includes reference speech features and selected speech features, and the reference speech features have the sound style of the target object;

[0041] A similarity measurement module, used for measuring the similarity between the reference speech feature and the selected speech feature to obtain feature similarity data;

[0042] A feature screening module, used for screening the selected speech features according to the feature similarity data to obtain target speech features; wherein the target speech features do not have the voice style of the target object;

[0043] A model training module, used to perform model training on a preset original speech generation model according to the target speech feature, the reference speech feature and the training text to obtain a target speech generation model;

[0044] The speech generation module is used to generate speech through the target speech generation model and the target text to obtain target speech data.

[0045] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes a computer device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the method described in the first aspect when executing the computer program.

[0046] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0047] The speech generation method and apparatus, computer equipment and storage medium proposed in the present application extract reference speech features and selected speech features of original speech data by adopting an original speech conversion model, wherein the reference speech features are speech features having the sound style of a target object. In order to reduce the influence of the sound style of a non-target object on speech generation, this embodiment extracts speech features that do not have the sound style of the target object. Therefore, by calculating the feature similarity data between the reference speech features and the selected speech features, the target speech features that do not have the sound style of the target object are extracted from the selected speech features according to the feature similarity data, and then the original speech generation model is trained in combination with the target speech features, the reference speech features and the training text to obtain a target speech generation model, and the speech data output by the target speech generation model only contains the sound style of the target object, and each speech generation does not require the aid of reference audio, making speech generation more stable and flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of the speech generation method provided in an embodiment of the present application;

[0049] Figure 2 yes Figure 1 Flow chart of step S102 in FIG.

[0050] Figure 3 yes Figure 1 Flow chart of step S104 in FIG.

[0051] Figure 4 yes Figure 3 Flow chart of step S301 in FIG.

[0052] Figure 5 is a flow chart of a speech generation method provided by another embodiment of the present application;

[0053] Figure 6 yes Figure 1 Flow chart of step S105 in FIG.

[0054] Figure 7 yes Figure 6 Flow chart of step S601 in FIG.

[0055] Figure 8 is a system framework diagram of the speech generation method provided in an embodiment of the present application;

[0056] Fig. 9 is a structural schematic diagram of a speech generating device provided in an embodiment of the present application;

[0057] Fig.10 It is a schematic diagram of the hardware structure of the computer device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0061] First, some nouns involved in this application are analyzed:

[0062] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0063] Self-supervised voice conversion model (SSVC): is a model that uses self-supervised learning techniques to process speech data. Self-supervised learning is a powerful unsupervised learning method that does not rely on labeled data, but learns the representation of data by predicting certain properties of the data itself. In the field of speech processing, this method is particularly useful because it can learn from unlabeled speech data, thereby reducing the dependence on a large amount of labeled data.

[0064] Large Language Model (LLM): It is an important branch of artificial intelligence. Through deep learning and a large amount of data training, it can understand and generate natural language text. LLM has a wide range of applications in many fields, including but not limited to chatbots, text generation, language translation, content creation, information retrieval, document classification and sentiment analysis.

[0065] WavLM model (Waveform Language Model): is a general speech pre-training model. It is self-supervised pre-trained through the Denoising Masked Speech Modeling framework, which combines masked speech prediction and denoising, so that the model can not only maintain the ability to model speech content, but also improve the potential of non-automatic speech recognition (ASR) tasks through speech denoising.

[0066] Softmax function: It is a normalization function commonly used in machine learning, especially when dealing with multi-class classification problems. The Softmax function converts a real vector into a probability distribution so that each element in the vector is between 0 and 1 and the sum of all elements is 1.

[0067] Transformer model: is a deep learning model architecture widely used in natural language processing (NLP) and other sequence-to-sequence tasks. The Transformer model introduces the self-attention mechanism, which is a key innovation that enables it to perform well when processing sequence data.

[0068] CLS (Classification) embedding: Usually used for text classification tasks, it is a special tag that is usually placed at the beginning of the input sequence. In the BERT model, the embedding vector of the CLS tag can fuse the information of the entire input sequence after being processed by a multi-layer self-attention mechanism to form a representation of the entire sentence. This representation vector can be used for sentence-level classification tasks.

[0069] VQ (Vector Quantization) layer: is a technique used for data compression and dimensionality reduction in deep learning, which discretizes the data by mapping the input data to a smaller fixed codebook. This method is particularly suitable for processing high-dimensional data, such as images or audio signals, and can significantly reduce storage requirements and computational complexity.

[0070] BigVGAN decoder: is a general neural vocoder specifically designed for synthesizing audio waveforms from Mel spectrograms. It adopts a fully convolutional architecture consisting of multiple upsampling blocks and residual dilated convolution layers.

[0071] Speech generation technology is a technology that converts text information into speech data, and there are already many speech generation systems based on large language models in the field of artificial intelligence, such as TorToiSe, VALL-E, VALL-EX, etc. It should be noted that these speech generation systems realize text-to-speech generation through the quantized token of the predictive codec (EnCodec, HiFi-Codec) or the discrete representation of VQ-VAE. However, in the related art, when the speech generation system generates speech data, it will mix in the voices of multiple speakers, resulting in unstable generated speech, affecting the sense of hearing and intelligibility. In this regard, in the related art, in order to overcome the problem of unstable speech generation, each time speech generation is performed, a reference audio needs to be input, but the setting of the reference audio will affect the flexibility of the speech generation system, and some speech generation systems also need to convert the reference audio into transcribed text, which affects the effect of speech generation.

[0072] Based on this, the embodiments of the present application provide a speech generation method and apparatus, a computer device and a storage medium, which are intended to extract target speech features that do not have the sound style of the target object and reference speech features that have the sound style of the target object from the original speech data through the original speech conversion model before speech generation, so as to train the original speech generation model according to the target speech features, the reference speech features and the training text, so as to train a target speech generation model that can accurately and stably output speech data with the sound style of the target object. Therefore, by generating the target speech data corresponding to the target text through the target speech generation model, not only stable speech data is generated, but each speech generation no longer relies on the reference audio, reducing the dependence on the reference audio and transcription, and improving the flexibility of speech generation.

[0073] The speech generation method and apparatus, computer equipment and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the speech generation method in the embodiments of the present application is described.

[0074] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0075] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0076] The speech generation method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The speech generation method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the speech generation method, etc., but is not limited to the above forms.

[0077] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer computer devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0078] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0079] Figure 1 is an optional flowchart of the speech generation method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S106.

[0080] Step S101, obtaining original speech data and training text;

[0081] Step S102, extracting speech features from the original speech data through a preset original speech conversion model to obtain a speech feature set; wherein the speech feature set includes reference speech features and selected speech features, and the reference speech features have the sound style of the target object;

[0082] Step S103, measuring the similarity between the reference speech feature and the selected speech feature to obtain feature similarity data;

[0083] Step S104, performing feature screening on the selected speech features according to the feature similarity data to obtain target speech features; wherein the target speech features do not have the sound style of the target object;

[0084] Step S105, performing model training on a preset original speech generation model according to the target speech features, the reference speech features and the training text to obtain a target speech generation model;

[0085] Step S106, performing speech generation through the target speech generation model and the target text to obtain target speech data.

[0086] In steps S101 to S106 shown in the embodiment of the present application, reference speech features and selected speech features are extracted from the original speech data through the original speech conversion model, and the reference speech features are speech features with the speaking style of the target object. Then, based on the feature similarity between the reference speech features and the selected speech features, the target speech features are screened out from the selected speech features to obtain sound features that do not have the sound style of the target object, and then the original speech generation model is trained based on the reference speech features, the target speech features and the training text to construct a target speech generation model that can generate speech stably and with a pure sound style. Therefore, the speech data generated by the target speech generation model only has the sound style of the target object, which improves the stability of speech data generation, reduces the dependence on reference audio, and makes speech generation more flexible.

[0087] In step S101 of some embodiments, the original voice data is voice data containing the sound style of the target object, and the original voice data can be obtained in real time or extracted from a pre-built training voice database. This embodiment does not impose specific restrictions on the method of obtaining the original voice data. The training text is a text used to train the original voice generation model to determine whether the semantic content in the voice data generated by the original voice generation model conforms to the training text, and the training text can be generated in real time or extracted from a pre-built text database. This embodiment does not impose specific restrictions on the method of obtaining the training text.

[0088] For example, if the application scenario is an insurance business scenario in the financial field, then voice data with multiple voice styles of insurance agents are constructed in advance, and the training text is also text related to insurance business knowledge, so that the original voice data and training text can be used to train insurance-related voice data that can accurately and stably output the voice style of insurance agents. Specifically, it can be applied to intelligent customers for insurance business Q&A to provide customers with more stable voice data for Q&A.

[0089] As disclosed above, after the original speech data is obtained, the original speech data is input into the original speech conversion model. The original speech conversion model includes: a speech encoder, a first feature extractor and a second feature extractor. The speech encoder is used to encode the original speech data, the first feature extractor is used to extract speech features with the sound style of the target object from the encoded speech features, and the second feature extractor is used to extract speech features other than the sound style of the target object from the encoded speech features.

[0090] Specifically, this embodiment uses a pre-trained WavLM model as a speech encoder. The WavLM model trains the model by predicting the masked speech part, and also includes a denoising process, which can extend the effectiveness of the WavLM model from speech recognition tasks to non-content recognition speech tasks. Therefore, by encoding the original speech data with the WavLM model, speech features of different object sound styles can be extracted.

[0091] See also Figure 2 In some embodiments, step S102 may include but is not limited to steps S201 to S203:

[0092] Step S201, extracting features from original speech data through a speech encoder to obtain candidate speech features;

[0093] Step S202, extracting speech features from the candidate speech features by a first feature extractor to obtain reference speech features;

[0094] Step S203, extracting speech features from the candidate speech features by using a second feature extractor and a reference speech feature to obtain a selected speech feature; wherein the sum of the weights of the reference speech feature and the selected speech feature is 1.

[0095] In step S201 of some embodiments, as disclosed above, the speech encoder of this embodiment is a pre-trained WavLM model, and the WavLM model is used to extract features from the original speech data, that is, to extract the hidden state of the original speech data. The WavLM model includes multiple WavLM layers, and each WavLM layer outputs a hidden state representing a different feature representation.

[0096] In step S202 of some embodiments, the first feature extractor is also called a speaker extractor, the candidate speech features include speech features with the sound style of the target object, and the first feature extractor is used to extract the speech features with the sound style of the target object. Specifically, the first feature extractor weights the candidate speech features output by each WavLM layer, and then inputs the weighted weighted speech features into the normalization layer, and maps the weighted weighted speech features to the sound style of the target object through the normalization layer to obtain reference speech features. It should be noted that the normalization layer of this embodiment selects the softmax function, and other embodiments may select a decimal calibration normalization function, a logarithmic normalization function, a power law normalization function, etc. This embodiment does not specifically limit the function of the normalization layer.

[0097] In step S203 of some embodiments, as disclosed above, the reference speech feature is obtained by weighting the candidate speech feature and inputting it into the normalization layer, so when extracting the selected speech feature, it is necessary to perform feature extraction in combination with the reference speech feature to extract the speech feature that does not have the sound style of the target object. Specifically, this embodiment determines the target object speech weight according to the reference speech feature, and then subtracts the target object speech weight from the constraint 1 to obtain the non-object speech weight. The second feature extractor weights the candidate speech feature output by each WavLM layer and inputs it into the normalization layer for normalization processing to obtain the selected speech feature.

[0098] In steps S201 to S203 shown in this embodiment, candidate speech features are extracted from the original speech data by a speech encoder, and reference speech features having the sound style of the target object are extracted from the candidate speech features in a weighted manner by a first feature extractor. Finally, a non-object weighting weight is determined based on the reference speech features by a second feature extractor, and the candidate speech features are weighted by the non-object weighting weight to obtain the selected speech features, so that the reference speech features and the selected speech features can be extracted easily.

[0099] In step S103 of some embodiments, as disclosed above, although the selected voice features are screened out from the candidate voice features according to the reference voice features, in order to ensure that the selected voice features do not have the voice features of the target object's voice style at all, it is necessary to perform feature screening on the selected voice features to screen out the target voice features that do not have the voice style of the target object. Specifically, feature similarity data is obtained by measuring the similarity of the reference voice features and the selected voice features, and the feature similarity data may be cosine similarity. In other embodiments, the feature similarity data may be Pearson correlation coefficient, Euclidean distance, log-likelihood similarity, etc. This embodiment does not impose specific restrictions on the feature similarity data.

[0100] See also Figure 3 In some embodiments, step S104 may include but is not limited to steps S301 to S302:

[0101] Step S301, performing feature quantization processing on the selected speech feature according to the preset quantization feature to obtain preliminary speech feature;

[0102] Step S302, performing feature screening on the preliminary speech features according to the feature similarity data to obtain target speech features.

[0103] In step S301 of some embodiments, the selected speech feature is first quantized, and the quantization process is a process of converting a continuous numerical signal into a discrete numerical signal. It should be noted that the preset quantized feature is a vector representation of a specified numerical value, and the quantization process of the selected speech feature by the preset quantized feature is to convert the selected speech feature into a vector representation of a fixed numerical value like the preset quantized feature to obtain the target speech feature.

[0104] In step S302 of some embodiments, the preliminary speech features are feature screened according to the feature similarity data, that is, the preliminary speech features with the smallest feature similarity data are selected as the target speech features. Alternatively, in this embodiment, the preliminary speech features are feature adjusted according to the feature similarity data to remove the speech features of the target object's voice style that the preliminary speech features have, leaving the speech features that do not have the target object's voice style as the target speech features.

[0105] In steps S301 to S302 shown in the present embodiment, the selected speech features are first converted into preliminary speech features of fixed values ​​according to preset quantitative features, and then the preliminary speech features are adjusted to target speech features that do not have the sound style of the target object according to feature similarity data, so that the speech features with the sound style of the target object and the speech features without the sound style of the target object can be known respectively through the target speech features and the reference speech features, thereby facilitating the training of a target speech generation model that only contains the sound style of the target object.

[0106] See also Figure 4 In some embodiments, step S301 may include but is not limited to steps S401 to S403:

[0107] Step S401, calculating the feature distance between the preset quantitative feature and the selected speech feature to obtain the speech feature distance;

[0108] Step S402, selecting a selected speech feature from preset quantitative features according to the speech feature distance;

[0109] Step S403, concatenating the selected speech features to obtain preliminary speech features.

[0110] It should be noted that this embodiment uses the VQ layer to quantize the selected speech features, and maps the input selected speech features to a fixed codebook through the VQ layer to achieve discretization of data. The VQ layer mainly performs the following steps: encoding, quantization and decoding, wherein the quantization operation is to find the vector closest to the selected speech feature in the codebook and use the closest vector to replace the original selected speech feature.

[0111] In step S401 of some embodiments, as disclosed above, the quantization operation needs to find the closest vector in the codebook, so the vector stored in the codebook in this embodiment is defined as a preset quantized feature, and the speech feature distance is obtained by performing feature distance calculation between each selected speech feature and each preset quantized feature. It should be noted that the distance calculation between the selected speech feature and the preset quantized feature can be calculated by Euclidean distance calculation or cosine distance calculation, and this embodiment does not impose specific restrictions on the distance calculation between the selected speech feature and the preset quantized feature.

[0112] In steps S402 to S403 of some embodiments, the minimum feature distance is selected from the speech feature distances, and the preset quantitative feature corresponding to the minimum feature distance is used as the selected speech feature, the selected speech feature is replaced by the selected speech feature, and then the selected speech features are spliced ​​together as the preliminary speech feature.

[0113] For example, the preset codebook is [1, 2, 3, 4, 5, 6, 7, 8, 9]. If the selected speech feature is 1.1, then the preset quantized feature closest to the selected speech feature is 1, and 1 is used as the preliminary speech feature; if the selected speech feature is 8.7, then the preset quantized feature closest to the selected speech feature is 9, and 9 is used as the preliminary speech feature. It should be noted that the purpose of quantizing the selected speech feature in this embodiment is to remove the local fluctuation noise in the selected speech feature. For example, if the representation of the speech "ah" should be stable (1, 1, 1, 1), rather than (1.1, 0.9, 1.2, 0.8) with timbre fluctuation noise, the selected speech feature is quantized to obtain a preliminary speech feature with a more stable timbre.

[0114] In steps S401 to S403 shown in this embodiment, the closest preset quantized feature is used as the selected speech feature of the selected speech feature, so that the selected speech feature is replaced by the selected speech feature to implement the quantization operation of the selected speech feature, eliminate the local fluctuation noise in the selected speech feature, and obtain a preliminary speech feature with more stable timbre.

[0115] It should be noted that when the target speech features are selected, in order to improve the accuracy of subsequent speech feature extraction, the original speech conversion model needs to be optimized to construct a target speech conversion model with more accurate and clean speech feature extraction.

[0116] In some embodiments, after step S104, the speech generation method may further include adjusting parameters of the original speech conversion model to obtain a target speech conversion model, thereby optimizing the model of the original speech conversion model to construct a target speech conversion model that can accurately extract speech features having the sound style of the target object.

[0117] It should be noted that the original speech conversion model is a self-supervised speech conversion model. Specifically, the SSVC model learns the high-dimensional features of the input speech by predicting the masked or changed parts, which can capture the acoustic features of the speech and map them into a compact vector space. Therefore, the SSVC model can learn contextual information without the need for reference audio.

[0118] See also Figure 5 In some embodiments, adjusting the parameters of the original speech conversion model to obtain the target speech conversion model may include but is not limited to steps S501 to S507:

[0119] Step S501, concatenating the target speech feature and the reference speech feature to obtain a concatenated speech feature;

[0120] Step S502, reconstructing the speech according to the concatenated speech features to obtain reconstructed speech data;

[0121] Step S503, performing loss calculation on the reconstructed speech data and the original speech data to obtain reconstructed loss data;

[0122] Step S504, performing loss calculation on the preset similarity threshold and the feature similarity data to obtain similarity loss data;

[0123] Step S505, performing object recognition according to the reference speech features to obtain object recognition information;

[0124] Step S506, performing loss calculation according to preset target object information and object identification information to obtain identification loss data;

[0125] Step S507, adjusting the model parameters of the original speech conversion model according to the reconstruction loss data, the similarity loss data and the recognition loss data to obtain a target speech conversion model.

[0126] It should be noted that in order to judge the feature extraction effect of the original speech conversion model, it can be judged from multiple aspects, mainly from whether the speech features input by the original speech conversion model can accurately realize speaker recognition, speech reconstruction and feature similarity calculation.

[0127] In steps S501 to S502 of some embodiments, in order to determine the feature extraction effect of the original speech conversion model, the reconstruction effect of the speech features output by the original speech conversion model can be analyzed. The target speech features and the reference speech features are spliced ​​into spliced ​​speech features, and then the spliced ​​speech features are speech reconstructed to obtain reconstructed speech data. Specifically, in this embodiment, the spliced ​​speech features are input into the BigVGAN vocoder, and the spliced ​​speech features are reconstructed by the BigVGAN vocoder to obtain reconstructed speech data. Among them, the BigVGAN vocoder is used to synthesize an audio waveform from a Mel spectrogram, and can output reconstructed speech data with better audio quality.

[0128] In step S503 of some embodiments, by calculating the loss between the reconstructed speech data and the original speech data, reconstructed loss data is obtained, and the reconstructed loss data represents whether the features extracted by the original speech conversion model can accurately represent the audio features of the original speech data. Specifically, this embodiment uses a loss function to calculate the loss data between the original speech data and the reconstructed speech data, and the loss function can be a cross entropy loss function, a mean absolute error loss function, a mean square error loss function, a log-likelihood loss function, etc. This embodiment does not specifically limit the loss function.

[0129] In step S504 of some embodiments, the feature similarity data represents the similarity between the reference speech feature and the selected speech feature. In order to determine the accuracy of the similarity calculation, it is determined whether the original speech conversion model can accurately extract speech features with the target object sound style and speech features without the target object sound style. The loss data between the similarity threshold and the feature similarity data is calculated as the similarity loss data, and the accuracy of the feature similarity calculation is determined by the similarity loss data. Specifically, the loss function used in the calculation of the similarity loss data can be any one of the cross entropy loss function, the mean absolute error loss function, the mean square error loss function, and the log-likelihood loss function. This embodiment does not impose any specific restrictions on the loss function used in the calculation of the similarity loss data.

[0130] In steps S505 to S506 of some embodiments, in order to judge the speech feature extraction effect of the original speech conversion model, it can be judged from whether the reference speech feature output by the original speech conversion model can accurately identify the target object. Specifically, the original speech conversion model extracts reference speech features that do not have the sound style of the target object from the original speech data, and the original speech data can be set to two, and the two original speech data from different speaking objects are used as negative sample data, and the two original speech data from the same speaking object are used as positive sample data. The reference speech features of the positive sample data are extracted by the original speech conversion model, and the object recognition information of the reference speech features is identified, and the two object recognition information and the two target object information are respectively calculated for loss to obtain the first loss data; at the same time, the original speech conversion model extracts the reference speech features of the negative sample data, and the object recognition information of the reference speech features and the target object information are calculated for loss to obtain the second loss data, and the first loss data and the second loss data are spliced ​​into the recognition loss data. It should be noted that the recognition loss data represents the object recognition accuracy of the reference speech features, and also reflects the effect of the original speech conversion model on speech feature extraction. Specifically, the loss function for calculating the recognition loss data may be any one of the cross entropy loss function, the mean absolute error loss function, the mean square error loss function, and the log-likelihood loss function. This embodiment does not impose any specific restrictions on the loss function used for calculating the recognition loss data.

[0131] In step S507 of some embodiments, target loss data is obtained by splicing reconstruction loss data, similarity loss data and recognition loss data, and the splicing method can be weighted summation, summation and average calculation, etc. This embodiment does not impose specific restrictions on the splicing method of target loss data. It should be noted that the parameters of the original speech conversion model are adjusted according to the target loss data, and the target loss data is continuously updated until the target loss data converges, and the original speech conversion model with adjusted parameters is used as the target speech conversion model.

[0132] In steps S501 to S507 shown in this embodiment, the reference speech features output from the original speech conversion model are subjected to reconstruction effect analysis, feature similarity evaluation and object recognition accuracy judgment, so as to adjust the parameters of the original speech conversion model from three aspects and construct a target speech conversion model with better speech feature extraction effect.

[0133] In some embodiments, the original speech generation model is a large language model, and the large language model is an LLM model. The LLM model can achieve speech generation by training a large amount of text to learn the grammar, semantics and context information of the language. The original speech generation model includes: an original text encoder and an original decoder, the original text encoder is a BERT model, and the original decoder is a BigVGAN decoder.

[0134] See also Figure 6 In some embodiments, step S105 includes but is not limited to steps S601 to S603:

[0135] Step S601, performing semantic extraction on the training text through the original text encoder to obtain text semantic information;

[0136] Step S602, converting the text semantic information into speech features through the original decoder and the reference speech features to obtain predicted speech features; wherein the predicted speech features do not have the sound style of the target object;

[0137] Step S603, adjusting the model parameters of the original speech generation model according to the predicted speech features and the target speech features to obtain the target speech generation model.

[0138] In step S601 of some embodiments, as disclosed above, the original text encoder is a BERT model. The BERT model has a bidirectional encoding feature. When processing each word, it simultaneously considers the preceding and following contextual information to obtain text semantic information containing the training text contextual information.

[0139] In step S602 of some embodiments, in order to verify whether the speech data generated by the original speech generation model is stable and has the sound style of the target object, it is necessary to first generate predicted speech features that do not have the sound style of the target object through the original speech generation model, and then train the original speech generation model based on the predicted speech features and the target speech features. Specifically, the reference speech features and text semantic information are input into the original decoder together, and the original decoder performs speech conversion on the reference speech features and the text semantic information to obtain reference speech data with the sound style of the target object, and the reference speech data is feature extracted according to the reference speech features to extract predicted speech features that do not have the sound style of the target object, so that the operation of obtaining the predicted speech features is simple.

[0140] In step S603 of some embodiments, speech loss data is obtained by performing loss calculation on predicted speech features and target speech features, and parameters of the original speech generation model are adjusted according to the speech loss data. The parameters of the original speech generation model are adjusted cyclically until the speech loss data converges to output the target speech generation model.

[0141] In steps S601 to S603 shown in the present embodiment, text semantic information in the training text is extracted by the original text encoder, and the text semantic information is converted into speech features by the original decoder and the reference speech features to obtain predicted speech features that do not have the sound style of the target object. The speech loss data between the predicted speech features and the target speech features is calculated, and the model parameters of the original speech generation model are adjusted according to the speech loss data to construct a target speech generation model that can generate stable speech data with the sound style of the target object.

[0142] In some embodiments, the original text encoder includes: a word segmentation layer, a semantic parsing layer and an encoding layer; it should be noted that the word segmentation layer is used to segment the text, the semantic parser is used for text parsing, and the encoding layer is used to extract text semantic information.

[0143] See also Figure 7 In some embodiments, step S601 may include but is not limited to steps S701 to S703:

[0144] Step S701, performing word segmentation processing on the training text through the word segmentation layer to obtain training phrases;

[0145] Step S702, performing semantic analysis on the training phrase through a semantic analysis layer to obtain semantic analysis information;

[0146] Step S703, performing semantic extraction on the training text through the coding layer and semantic parsing information to obtain text semantic information.

[0147] In step S701 of some embodiments, the training text is input into the word segmentation layer for word segmentation processing, and the specific word segmentation is BPE word segmentation, which is a word segmentation technology used in natural language processing for processing rare words and unknown words. Therefore, by performing BPE word segmentation on the training text, accurate word segmentation of the training text is achieved to obtain training phrases.

[0148] In step S702 of some embodiments, the training phrase is input into the semantic parsing layer, and the training phrase is semantically parsed by the speech parsing layer to obtain speech parsing information. It should be noted that semantic parsing is to convert the training phrase into structured data or representation that can be understood and processed, so the semantic parsing information represents the structured data or representation of the training phrase.

[0149] In step S703 of some embodiments, the encoder extracts text semantic information from the training text in combination with the semantic parsing information to obtain text semantic information that accurately represents the context information of the training text. Specifically, this embodiment extracts high-level embedding information from the training text through the encoding layer, and at the same time, uses a learnable embedding layer to obtain low-level embedding information in the training text, and splices the high-level embedding information and the low-level embedding information into text semantic information.

[0150] In steps S701 to S703 shown in this embodiment, the training text is first segmented and parsed, and then the text semantic information of the training text is extracted based on the parsed semantic analysis information, so that the text semantic information extraction operation is simple.

[0151] In step S106 of some embodiments, after the target speech generation model is constructed, the target text is first segmented by the target speech generation model to obtain the target phrase, and then the target phrase is parsed to obtain phrase parsing information, and then the target semantic information is extracted from the target text by the target speech generation model and the phrase parsing information, so as to perform speech generation according to the target semantic information to obtain the target speech data. Therefore, by training a target speech generation model that can generate a target object's voice style, the target speech data generated by the target speech generation model does not need to be input into the reference audio every time to generate speech data with the target object's voice style, which not only improves the stability of speech data generation, but also improves the flexibility of speech data generation.

[0152] Please refer to Figure 8In the embodiment of the present application, the original speech data and the training text are obtained, the original speech data is input into the WavLM model, the original speech data is subjected to feature extraction by the WavLM model to obtain candidate speech features, and then the candidate speech features are respectively input into the first feature extractor and the second feature extractor. The WavLM model includes a plurality of WavLM layers, the candidate speech features output by each WavLM layer are weighted by the first feature extractor to obtain reference speech features, the second feature extractor determines the non-object speech weight based on the reference speech features, and the candidate speech features output by each WavLM layer are weighted according to the non-object speech weight to obtain the selected speech features. Although the selected speech features are output weighted with the non-object speech weights, it is not excluded that the selected speech features still have the speech features of the target object sound style. Therefore, the selected speech features need to be further screened to screen out the target speech features that do not have the target object sound style. Specifically, the speech feature distance between the preset quantitative feature and the selected speech feature is first calculated, the preset quantitative feature with the closest speech feature distance is used as the selected speech feature, and the selected speech features are spliced ​​into preliminary speech features. Feature similarity data is obtained by calculating the similarity between the selected speech feature and the reference speech feature, and then feature adjustment is performed on the preliminary speech feature according to the feature similarity data to reduce the speech features of the target object's sound style in the preliminary speech feature, thereby obtaining the target speech feature root.

[0153] After the target speech feature is generated, the target speech feature, the reference speech feature and the training text are input into the original speech generation model. The original speech generation model is an LLM model, and the LLM model includes: an original text encoder and an original decoder, and the original text encoder includes: a word segmentation layer, a semantic analysis layer and an encoding layer. The training text is segmented into training phrases by the word segmentation layer, and the semantic analysis layer performs semantic analysis on the training phrases to obtain semantic analysis information. The text semantic information is extracted from the training text by the encoding layer and the semantic analysis information, and then the text semantic information is converted into speech features by the original decoder and the reference speech feature to obtain the predicted speech feature, and the predicted speech feature is a speech feature that does not have the sound style of the target object. The speech loss data is obtained by calculating the loss data between the predicted speech feature and the target speech feature, and the model parameters of the original speech generation model are adjusted according to the speech loss data to obtain the target speech generation model, so as to construct a target speech generation model that can stably generate speech data with the sound style of the target object.

[0154] When generating speech data, the target text is segmented by the target speech generation model to obtain the target phrase, and then the target phrase is parsed to obtain the phrase parsing information. The target semantic information is extracted from the target text by the target speech generation model and the phrase parsing information, and speech generation is performed according to the target semantic information to obtain the target speech data. Therefore, the target speech data with the target object sound style is generated by the pre-trained target speech generation model, and the speech data with non-object sound style is reduced, making the speech generation cleaner and more stable.

[0155] See also Fig. 9 The present application also provides a speech generation device that can implement the above-mentioned speech generation method. The device includes:

[0156] An acquisition module 901 is used to acquire original speech data and training text;

[0157] The speech feature extraction module 902 is used to extract speech features from the original speech data through a preset original speech conversion model to obtain a speech feature set; wherein the speech feature set includes reference speech features and selected speech features, and the reference speech features have the sound style of the target object;

[0158] A similarity measurement module 903 is used to measure the similarity between the reference speech feature and the selected speech feature to obtain feature similarity data;

[0159] A feature screening module 904 is used to screen the selected speech features according to the feature similarity data to obtain target speech features; wherein the target speech features do not have the sound style of the target object;

[0160] The model training module 905 is used to perform model training on a preset original speech generation model according to the target speech features, the reference speech features and the training text to obtain a target speech generation model;

[0161] The speech generation module 906 is used to generate speech through a target speech generation model and a target text to obtain target speech data.

[0162] The specific implementation method of the speech generation is basically the same as the specific embodiment of the above-mentioned speech generation method, and will not be repeated here.

[0163] The embodiment of the present application also provides a computer device, the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned speech generation method when executing the computer program. The computer device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0164] See also Fig.10 , Fig.10The hardware structure of a computer device according to another embodiment is shown, and the computer device includes:

[0165] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0166] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the speech generation method of the embodiment of this application;

[0167] Input / output interface 1003, used to implement information input and output;

[0168] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0169] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0170] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0171] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned speech generation method is implemented.

[0172] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0173] The speech generation method and apparatus, computer equipment and storage medium provided in the embodiments of the present application extract reference speech features having the sound style of the target object from the original speech data through the original speech conversion model, and the original speech conversion model also extracts selected speech features from the original speech data. In order to obtain speech features that do not have the sound style of the target object, the feature similarity data between the reference speech features and the selected speech features is calculated, and the target speech features that do not have the sound style of the target object are screened out from the selected speech features according to the feature similarity data. Therefore, the original speech generation model is trained in combination with the target speech features, the reference speech features and the training text to construct a target speech generation model that can generate speech data having only the sound style of the target object and reducing interference from other sound styles. Therefore, clean and stable speech data can be generated through the target speech generation model, reducing the dependence on reference audio and improving the flexibility of speech data generation.

[0174] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0175] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0176] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0177] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0178] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0179] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0180] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0181] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0182] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0183] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0184] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A speech generation method, characterized in that: The method comprises: Get raw speech data and training text; Extracting speech features from the original speech data through a preset original speech conversion model to obtain a speech feature set; wherein the speech feature set includes reference speech features and selected speech features, and the reference speech features have the sound style of the target object; Measuring the similarity between the reference speech feature and the selected speech feature to obtain feature similarity data; Performing feature screening on the selected speech feature according to the feature similarity data to obtain a target speech feature; wherein the target speech feature does not have the voice style of the target object; Performing model training on a preset original speech generation model according to the target speech feature, the reference speech feature and the training text to obtain a target speech generation model; Speech generation is performed using the target speech generation model and the target text to obtain target speech data.

2. The method according to claim 1, characterized in that The original speech conversion model includes: a speech encoder, a first feature extractor and a second feature extractor; the original speech data is subjected to speech feature extraction by the preset original speech conversion model to obtain a speech feature set, including: Extracting features from the original speech data by using the speech encoder to obtain candidate speech features; Extracting speech features from the candidate speech features by the first feature extractor to obtain the reference speech features; The candidate speech feature is subjected to speech feature extraction by the second feature extractor and the reference speech feature to obtain the selected speech feature; wherein the sum of the weights of the reference speech feature and the selected speech feature is 1.

3. The method according to claim 1, characterized in that The step of performing feature screening on the selected speech features according to the feature similarity data to obtain target speech features includes: Performing feature quantization processing on the selected speech feature according to preset quantization features to obtain preliminary speech features; The preliminary speech features are screened according to the feature similarity data to obtain the target speech features.

4. The method according to claim 3, characterized in that The step of performing feature quantization processing on the selected speech feature according to the preset quantization feature to obtain a preliminary speech feature includes: Calculating the feature distance between the preset quantitative feature and the selected speech feature to obtain a speech feature distance; Filtering out a selected speech feature from the preset quantitative features according to the speech feature distance; The selected speech features are concatenated to obtain the preliminary speech features.

5. The method according to any one of claims 1 to 4, characterized in that: After performing feature screening on the selected speech features according to the feature similarity data to obtain target speech features, the method further includes: Concatenating the target speech feature and the reference speech feature to obtain a concatenated speech feature; Perform speech reconstruction according to the concatenated speech features to obtain reconstructed speech data; Performing loss calculation on the reconstructed speech data and the original speech data to obtain reconstructed loss data; Perform loss calculation on a preset similarity threshold and the feature similarity data to obtain similarity loss data; Performing object recognition according to the reference speech features to obtain object recognition information; Calculate the loss based on the preset target object information and the object identification information to obtain identification loss data; The model parameters of the original speech conversion model are adjusted according to the reconstruction loss data, the similarity loss data and the recognition loss data to obtain a target speech conversion model.

6. The method according to any one of claims 1 to 4, characterized in that: The original speech generation model includes: an original text encoder and an original decoder; the model training is performed on the preset original speech generation model according to the target speech feature, the reference speech feature and the training text to obtain the target speech generation model, including: Performing semantic extraction on the training text by using the original text encoder to obtain text semantic information; The text semantic information is converted into speech features by using the original decoder and the reference speech features to obtain predicted speech features; wherein the predicted speech features do not have the sound style of the target object; The model parameters of the original speech generation model are adjusted according to the predicted speech features and the target speech features to obtain the target speech generation model.

7. The method according to claim 6, characterized in that The original text encoder comprises: a word segmentation layer, a semantic analysis layer and an encoding layer; the original text encoder is used to perform semantic extraction on the training text to obtain text semantic information, including: Perform word segmentation processing on the training text through the word segmentation layer to obtain training phrases; Performing semantic analysis on the training phrase through the semantic analysis layer to obtain semantic analysis information; Semantic extraction is performed on the training text through the encoding layer and the semantic parsing information to obtain the text semantic information.

8. A speech generating device, characterized in that: The device comprises: An acquisition module is used to acquire original speech data and training text; A speech feature extraction module, used to extract speech features from the original speech data through a preset original speech conversion model to obtain a speech feature set; wherein the speech feature set includes reference speech features and selected speech features, and the reference speech features have the sound style of the target object; A similarity measurement module, used for measuring the similarity between the reference speech feature and the selected speech feature to obtain feature similarity data; A feature screening module, used for screening the selected speech features according to the feature similarity data to obtain target speech features; wherein the target speech features do not have the voice style of the target object; A model training module, used to perform model training on a preset original speech generation model according to the target speech feature, the reference speech feature and the training text to obtain a target speech generation model; The speech generation module is used to generate speech through the target speech generation model and the target text to obtain target speech data.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the speech generation method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the speech generation method according to any one of claims 1 to 7 is implemented.

Images