Speech synthesis method and device based on conditional matching stream, equipment and medium

Abstract

The invention relates to the field of financial science and technology speech conversion, and discloses a speech synthesis method, device and equipment based on a conditional matching stream and a medium, and the method comprises the steps: preprocessing an input text, and converting the input text into an embedded vector; based on a stream matching model, the input text is combined with the embedded vector and mapped to the intermediate representation space, and the input text in the intermediate representation space gradually generates speech spectrum features; on the basis of a zero sample learning mechanism, generating a voice spectrum meeting requirements from the voice spectrum features; and restoring the voice spectrum into a voice signal and outputting the voice signal. By introducing a stream matching model and a zero sample learning mechanism, speech with emotions or a specific style can be directly generated without emotional data training, the dependence of a traditional speech technology on large-scale data with emotional labels is broken through, and the application range of the speech technology is greatly expanded. The method is applied to application scenes such as financial science and technology voice conversion.

Description

Technical Field

[0001] The present invention relates to the field of financial technology speech conversion, and in particular to a speech synthesis method, device, equipment and medium based on conditional matching stream. Background Art

[0002] Current text-to-speech (TTS) technology has been widely used in many fields such as voice assistants, virtual anchors, and voice navigation, but there are still many problems to be solved. The main shortcomings of existing voice technology are the lack of emotional expression: Most existing voice technologies generate neutral or mechanical voices, which cannot naturally convey emotions or express complex emotional changes. Although some models try to achieve voice emotion changes by adding emotion control tags, this method often relies on a large amount of data with emotion annotations and cannot flexibly generate diverse emotional voices. Summary of the invention

[0003] The present invention provides a speech synthesis method, device, computer equipment and medium based on conditional matching stream, so as to solve the technical problem that the existing speech technology lacks emotional expression.

[0004] In a first aspect, a speech synthesis method based on conditional matching stream is provided, comprising:

[0005] Preprocess the input text and convert it into an embedding vector;

[0006] Based on the stream matching model, the input text is combined with the embedding vector and mapped to the intermediate representation space, and the input text in the intermediate representation space is gradually generated into speech spectrum features;

[0007] Based on the zero-sample learning mechanism, the speech spectrum features are used to generate a speech spectrum that meets the requirements;

[0008] Restore the speech spectrum to the speech signal and output the speech signal.

[0009] In a second aspect, a speech synthesis device based on conditional matching stream is provided, comprising:

[0010] The conversion module is used to preprocess the input text and convert it into an embedding vector;

[0011] A classification module is used to map the input text combined with the embedding vector to the intermediate representation space based on the stream matching model, and gradually generate speech spectrum features from the input text in the intermediate representation space;

[0012] A training module is used to generate a speech spectrum that meets the requirements based on the speech spectrum features based on a zero-sample learning mechanism;

[0013] The restoration module is used to restore the speech spectrum to a speech signal and output the speech signal.

[0014] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned speech synthesis method based on conditional matching stream when executing the computer program.

[0015] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned speech synthesis method based on conditional matching stream are implemented.

[0016] In the scheme implemented by the above-mentioned speech synthesis method, device, computer equipment and storage medium based on conditional matching stream, the input text can be preprocessed and converted into an embedding vector by the conversion module; the classification module maps the input text combined with the embedding vector to the intermediate representation space based on the stream matching model, and gradually generates speech spectrum features from the input text in the intermediate representation space; based on the zero-sample learning mechanism, the training module generates a speech spectrum that meets the requirements from the speech spectrum features; the restoration module restores the speech spectrum to a speech signal and outputs the speech signal. Zero-sample emotional speech generation is achieved: by introducing the stream matching model and the zero-sample learning mechanism, the present invention can directly generate speech with emotion or a specific style without emotional data training, breaking through the dependence of traditional speech technology on large-scale emotionally labeled data, and greatly expanding the application scope of speech technology. It solves the problems of insufficient emotional expression ability and high data dependence in existing speech technology, and greatly improves the flexibility and generation effect of speech technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0018] Figure 1 It is a schematic diagram of an application environment of a speech synthesis method based on conditional matching stream in one embodiment of the present invention;

[0019] Figure 2 It is a flow chart of a speech synthesis method based on conditional matching stream in one embodiment of the present invention;

[0020] Figure 3 is a structural schematic diagram of a speech synthesis device based on conditional matching stream in one embodiment of the present invention;

[0021] Figure 4is a schematic diagram of a structure of a computer device in one embodiment of the present invention;

[0022] Figure 5 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The speech synthesis method based on conditional matching stream provided by the embodiment of the present invention can be applied in Figure 1 In the application environment, the client preprocesses the input text through the conversion module and converts the input text into an embedding vector; the classification module maps the input text combined with the embedding vector to the intermediate representation space based on the stream matching model, and gradually generates speech spectrum features from the input text in the intermediate representation space; based on the zero-sample learning mechanism, the training module generates a speech spectrum that meets the requirements from the speech spectrum features; the restoration module restores the speech spectrum to a speech signal and outputs the speech signal. Zero-sample emotional speech generation is achieved: by introducing the stream matching model and the zero-sample learning mechanism, the present invention can directly generate speech with emotion or a specific style without emotional data training, breaking through the dependence of traditional speech technology on large-scale emotionally labeled data, and greatly expanding the application scope of speech technology. It solves the problems of insufficient emotional expression ability and high data dependence in existing speech technology, and greatly improves the flexibility and generation effect of speech technology. The present invention is described in detail below through specific embodiments.

[0025] The present invention is applied to application scenarios such as financial technology speech conversion. By introducing a stream matching model and a zero-sample learning mechanism, it can directly generate speech with emotions or specific styles without emotional data training, breaking through the reliance of traditional speech technology on large-scale emotionally labeled data, and greatly expanding the application scope of speech technology. It solves the problems of insufficient emotional expression ability and high data dependence in existing speech technology, and greatly improves the flexibility and generation effect of speech technology.

[0026] See also Figure 2 As shown, Figure 2 A flow chart of a speech synthesis method based on conditional matching stream provided by an embodiment of the present invention includes the following steps:

[0027] S10: preprocess the input text and convert the input text into an embedding vector;

[0028] It should be noted that the input text includes word segmentation, character encoding, time vector, semantic information and embedded representation. Preprocess the input text and convert it into a form that the system can understand. The conversion methods include text encoding, word embedding, bag-of-words model, and sequence model. Text encoding is a method of converting characters into numbers. Each character is assigned a unique number, which is called the encoding of the character. For example, the encoding of character 'A' is 65, the encoding of character 'B' is 66, and so on; word embedding is a method of converting words into vectors. Each word is assigned a unique vector, which is called the embedding of the word. The main goal of word embedding is to capture the semantic relationship between words. For example, the embedding of "cat" and "dog" should be closer than the embedding of "cat" and "apple" because "cat" and "dog" are closer in semantics. The bag-of-words model is a method of converting text into a vector. In the bag-of-words model, each text is represented as a vector, and each element of the vector represents the frequency of a word in the text. The main disadvantage of the bag-of-words model is that it ignores the order information between words. The sequence model is a method of converting text into a vector. In the sequence model, each text is represented as a vector sequence, and each element of the vector represents the embedding of a word. The main advantage of the sequence model is that it retains the order information between words. In this embodiment, word embedding is preferably used as an input text conversion method. During the conversion process, the input text is converted into an embedding vector of a fixed size. The embedding vector can represent the semantic information and rhythmic features of the text. The embedding representation is not only used to capture basic text information, but also includes predictions of speech emotions and styles.

[0029] S20: Map the input text combined with the embedding vector to the intermediate representation space based on the stream matching model, and gradually generate speech spectrum features from the input text in the intermediate representation space;

[0030] This step uses the stream matching model to generate diverse speech signals. Stream matching is a technique for generating models that can achieve accurate matching between input and output spaces by converting complex distributions into target distributions. In the speech technology generation task, the stream matching model is used to generate the spectrogram of speech.

[0031] Flow matching includes forward flow and reverse flow. Forward flow: The forward flow is used to map the input text combined with the embedding vector along the conditional variables in the input text to the intermediate representation space. This process gradually transforms high-dimensional complex semantic information into feature representations in the speech spectrum. A conditional variable yyy is introduced on the basis of the original flow model. The conditional variable can be information such as labels and text descriptions. The goal of the conditional flow is to find a velocity field v(x(t), t, y)v(x(t), t, y)v(x(t), t, y) so that the generated text not only conforms to the target distribution but also satisfies the given conditions. Of course, a velocity field v(x(t), t)v(x(t), t)v(x(t), t) is found on the basis of the original flow model. The velocity field defines the evolution of the text point x(t)x(t)x(t) at time ttt, so that the text gradually changes from a simple distribution (such as a Gaussian distribution) to the target distribution. Flow matching describes the evolution process through differential equations and learns the velocity field by optimizing a matching loss function. The loss function measures the difference between the generated text flow path and the real path.

[0032] Based on the Gaussian probability path, the input text in the intermediate representation space is gradually evolved to generate speech spectrum features. In stream matching, different probability paths can be selected to describe the evolution of text from simple distribution to target distribution, among which the Gaussian probability path is mainly used, assuming that the text points gradually evolve from the Gaussian distribution N(0,I)\mathcal{N}(0,I)N(0,I) to the target distribution along time ttt. The Gaussian probability path can calculate the distribution of sample points at different time points through explicit formulas, thus avoiding the complex numerical solution process. This makes stream matching more efficient in practical applications.

[0033] Reverse flow: The reverse flow process is used to restore the speech signal from the spectral features. Through the generation mechanism of stream matching, the input text is gradually generated into the target speech from the intermediate representation space, and the semantic and stylistic consistency of the generated speech and the input text can be ensured.

[0034] S30: Based on the zero-sample learning mechanism, the speech spectrum features are used to generate a speech spectrum that meets the requirements;

[0035] The core goal of the zero-shot learning mechanism is to correctly identify new categories and generate speech styles that meet the requirements without ever seeing the annotated text of the new categories during training. The key to zero-shot learning is to use auxiliary information, such as the semantic description or attribute information of the category, to achieve the ability to generalize to new categories. The zero-shot learning mechanism uses auxiliary information to map new categories to the feature space of known categories to achieve classification. Common auxiliary information includes: Category attributes: Each category can be described by a set of attributes, such as the animal's fur color, size, shape, etc.; Semantic description: Text description or embedded representation of the category, for example, the category "dog" can be described by semantic information such as "having four legs, barking, and having hair".

[0036] The implementation steps of the zero-shot learning mechanism are to map the attributes or semantic descriptions of the categories into a feature space; learn a mapping function so that the input texts of known categories and their attributes or semantic embeddings in the intermediate representation space are as close as possible; for new categories, embed them into the intermediate representation space through their attributes or semantic descriptions and classify them in the intermediate representation space.

[0037] The zero-sample learning mechanism enables the generation of a speech style that meets the requirements without a specific target speech sample. In this embodiment, the zero-sample learning mechanism utilizes the following two technologies:

[0038] Style-based audio encoder: Based on the zero-shot learning mechanism, the speech spectrum features are generated into a speech style that meets the requirements; the speech style that meets the requirements is combined with the semantic information of the input text to generate a speech spectrum that meets the emotion. Based on the zero-shot learning mechanism, the abstract representation of different speech styles is learned during the training process; the speech style that meets the speech spectrum features in the abstract representation of different speech styles is extracted and generated. Specifically, by learning the abstract representation of different speech styles (such as laughter, crying, excitement, etc.) during the training process, the input text can be generated into speech with specific emotions in the zero-shot scenario. For example, the user can guide the model to generate the corresponding emotional speech through simple text input of "laughter" or "humorous tone". It further combines multimodal information, especially the bimodal fusion of emotion and text. In the process of speech technology generation, according to the input emotion or style prompts, combined with the semantic information of the text, a speech spectrum that meets the emotion is generated. For example, when the text inputs "laughter", the model can automatically adjust the pitch, rhythm and duration of the generated speech, making the generated laughter more natural and expressive.

[0039] Conditional generation based on emotion control: obtain the input emotional prompt features and the semantic information of the input text to match the emotional voice style in the required voice style; generate the voice spectrum output of the voice style that matches the emotion. Specifically, by applying emotion control conditions to the input text, the model can automatically adjust parameters such as intonation, pitch and rhythm when generating speech according to the input emotional label (such as happy, sarcastic, excited, etc.), so as to achieve dynamic matching of emotion and speech content.

[0040] Of course, there are many methods for zero-shot learning, and common methods include:

[0041] Attribute-based learning: Using the attribute description of the category, we can achieve zero-shot classification by learning the relationship between the attribute and the category. For example, we can map the attributes of animals such as "wings" and "hair" into the feature space, and then classify them according to these attributes.

[0042] Based on semantic embedding: Use the semantic description or embedding representation of the category to map the semantic information of the new category into the feature space. For example, word vectors can be used to map the category name into a vector space, and these vectors can be used for classification.

[0043] Based on generative models: Generate synthetic data of new categories and use them for classification. For example, you can use a generative adversarial network to generate samples of new categories, and use these samples for training and classification.

[0044] It should be noted that the goal of zero-shot learning is to be able to perform classification without labeled samples of new categories, but it is still necessary to learn the mapping relationship of attributes or semantic descriptions from samples of known categories. Therefore, in the training phase, data of known categories need to be used.

[0045] Zero-shot learning relies on rich auxiliary information, such as attributes or semantic descriptions of categories. This auxiliary information provides additional knowledge about new categories to achieve classification. Therefore, although there are no labeled samples for new categories, the auxiliary information actually provides a basis for classification.

[0046] The difference between the training and inference phases is that in the training phase, training is done through samples of known categories and auxiliary information. In the inference phase, when the model faces a new category, it only needs to rely on auxiliary information (such as attributes or semantic descriptions) for classification, without the need for labeled samples of the new category.

[0047] The zero-shot learning mechanism does not rely on labeled samples of new categories in the inference stage, but relies on rich auxiliary information and samples of known categories for training. Through auxiliary information, the zero-shot learning mechanism can achieve accurate recognition and classification when there are no labeled samples of new categories. The zero-shot learning mechanism provides an effective way to solve the problem of data scarcity and improve the performance and application scope of machine learning models.

[0048] S40: Restoring the speech spectrum to a speech signal and outputting the speech signal.

[0049] In this embodiment, a neural vocoder is mainly used to restore the speech spectrum to a speech signal and output the speech signal. Based on the neural vocoder, the speech spectrum is generated and output as a speech signal. After the generation of the speech spectrum is completed, the generated speech spectrum is restored to a speech signal. This process usually uses a neural vocoder (such as WaveNet or HiFi-GAN) to generate a high-fidelity speech signal. In addition, during the restoration process, the generated speech signal is also subjected to noise removal and sound quality optimization to ensure that the output speech has a high-quality auditory effect.

[0050] Of course, this step can also be achieved through reverse flow, which is used to restore the speech signal from the spectral features. Through the generation mechanism of stream matching, the input text is gradually generated from the intermediate representation space to the target speech, and the semantic and style consistency of the generated speech and the input text can be ensured.

[0051] The present invention provides a speech synthesis method based on conditional matching stream, which first pre-processes input text and converts the input text into an embedding vector; maps the input text combined with the embedding vector to an intermediate representation space based on a stream matching model, and gradually generates speech spectrum features from the input text in the intermediate representation space; based on a zero-sample learning mechanism, generates a speech spectrum that meets the requirements from the speech spectrum features; and restores the speech spectrum to a speech signal and outputs the speech signal.

[0052] Zero-sample emotional speech generation is achieved: By introducing a stream matching model and a zero-sample learning mechanism, the present invention can directly generate speech with emotion or a specific style without emotional data training, breaking through the reliance of traditional speech technology on large-scale emotionally labeled data, and greatly expanding the application scope of speech technology. Enhanced flexibility of style control and generation: Compared with existing speech technology, the present invention has stronger emotion and style control capabilities. Users can dynamically control the emotional color of speech during the generation process through simple text prompts or emotional tags, making the generated speech more diverse and personalized. Improved the naturalness and detail expression of speech generation: The stream matching model is used to ensure the stability and coherence of the generated speech. In particular, when generating speech with complex emotions (such as laughter, crying, etc.), it can capture subtle emotional changes and generate natural and expressive speech. Reduced dependence on data and improved computing efficiency: The present invention does not require a large amount of training data with emotion annotations, significantly reducing data requirements. At the same time, the stream matching model improves the efficiency of speech generation, making it more suitable for use in resource-constrained environments, such as mobile devices or real-time application scenarios. In summary, the speech synthesis method based on conditional matching stream proposed in the present invention solves the problems of insufficient emotional expression ability and high data dependence in existing speech technology, and greatly improves the flexibility and generation effect of speech technology.

[0053] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0054] In one embodiment, a speech synthesis device based on conditional matching stream is provided, and the speech synthesis device based on conditional matching stream corresponds to the speech synthesis method based on conditional matching stream in the above embodiment. Figure 3 As shown, the speech synthesis device based on conditional matching flow includes a conversion module 101, a classification module 102, a training module 103, and a restoration module 104. The detailed description of each functional module is as follows:

[0055] A conversion module 101, for preprocessing input text and converting the input text into an embedding vector;

[0056] A classification module 102, configured to map the input text combined with the embedding vector to an intermediate representation space based on a stream matching model, and gradually generate speech spectrum features from the input text in the intermediate representation space;

[0057] The training module 103 is used to generate a speech spectrum that meets the requirements based on the speech spectrum features based on the zero-sample learning mechanism;

[0058] The restoration module 104 is used to restore the speech spectrum to a speech signal and output the speech signal.

[0059] In one embodiment, the conversion module 101 is specifically configured to:

[0060] A preprocessing submodule, used for preprocessing input text, wherein the input text includes word segmentation, character encoding, time vector, semantic information and embedded representation;

[0061] The conversion submodule is used to convert the input text into a fixed-size embedding vector.

[0062] In one embodiment, the classification module 102 is specifically configured to:

[0063] A mapping submodule is used to map the input text combined with the embedding vector along the conditional variables in the input text to the intermediate representation space;

[0064] The evolution submodule is used to gradually evolve the input text in the intermediate representation space to generate speech spectrum features based on the Gaussian probability path.

[0065] In one embodiment, the training module 103 is specifically used for:

[0066] The training submodule is used to generate a speech style that meets the requirements based on the speech spectrum features based on the zero-sample learning mechanism;

[0067] The combining submodule is used to combine the required voice style with the semantic information of the input text to generate a speech spectrum that meets the emotion.

[0068] In one embodiment, the training submodule is specifically used for:

[0069] The training unit is used to learn abstract representations of different speech styles based on a zero-shot learning mechanism during the training process;

[0070] The extraction unit is used to extract and generate a speech style that meets the speech spectrum characteristics from the abstract representations of different speech styles.

[0071] In one embodiment, the training module 103 is further configured to:

[0072] The screening submodule is used to obtain the input emotional prompt features and the semantic information of the input text, and the speech style that meets the requirements; the speech style that meets the emotion is generated into a speech spectrum output.

[0073] In one embodiment, the restoration module 104 is specifically used for:

[0074] The restoration submodule is used to generate a speech signal output from the speech spectrum based on a neural vocoder.

[0075] The present invention provides a speech synthesis device based on conditional matching stream, which first pre-processes the input text through a conversion module and converts the input text into an embedding vector; the classification module maps the input text combined with the embedding vector to an intermediate representation space based on a stream matching model, and gradually generates speech spectrum features from the input text in the intermediate representation space; based on a zero-sample learning mechanism, the training module generates a speech spectrum that meets the requirements from the speech spectrum features; the restoration module restores the speech spectrum to a speech signal and outputs the speech signal. Zero-sample emotional speech generation is achieved: by introducing a stream matching model and a zero-sample learning mechanism, the present invention can directly generate speech with emotion or a specific style without emotional data training, breaking through the dependence of traditional speech technology on large-scale emotionally labeled data, and greatly expanding the application scope of speech technology. It solves the problems of insufficient emotional expression ability and high data dependence in existing speech technology, and greatly improves the flexibility and generation effect of speech technology.

[0076] For the specific definition of the speech synthesis device based on the conditional matching stream, please refer to the definition of the speech synthesis method based on the conditional matching stream above, which will not be repeated here. The various modules in the above-mentioned speech synthesis device based on the conditional matching stream can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0077] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, the functions or steps on the server side of a speech synthesis method based on conditional matching flow are implemented.

[0078] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server through a network connection. When the computer program is executed by the processor, the client-side functions or steps of a speech synthesis method based on conditional matching stream are implemented.

[0079] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0080] Preprocess the input text and convert it into an embedding vector;

[0081] Based on the stream matching model, the input text is combined with the embedding vector and mapped to the intermediate representation space, and the input text in the intermediate representation space is gradually generated into speech spectrum features;

[0082] Based on the zero-sample learning mechanism, the speech spectrum features are used to generate a speech spectrum that meets the requirements;

[0083] Restore the speech spectrum to the speech signal and output the speech signal.

[0084] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0085] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0086] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0087] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A speech synthesis method based on conditional matching stream, characterized in that: include: Preprocess the input text and convert it into an embedding vector; Based on the stream matching model, the input text is combined with the embedding vector and mapped to the intermediate representation space, and the input text in the intermediate representation space is gradually generated into speech spectrum features; Based on the zero-sample learning mechanism, the speech spectrum features are used to generate a speech spectrum that meets the requirements; Restore the speech spectrum to the speech signal and output the speech signal.

2. The speech synthesis method based on conditional matching stream according to claim 1, characterized in that: The step of preprocessing the input text and converting the input text into an embedding vector comprises: Preprocessing input text, wherein the input text includes word segmentation, character encoding, time vector, semantic information and embedded representation; Convert input text into fixed-size embedding vectors.

3. The speech synthesis method based on conditional matching stream as claimed in claim 2, characterized in that: The step of mapping the input text and the embedding vector to the intermediate representation space based on the stream matching model, and gradually generating speech spectrum features from the input text in the intermediate representation space includes: Map the input text combined with the embedding vector along the conditional variables in the input text to the intermediate representation space; Based on the Gaussian probability path, the input text in the intermediate representation space is gradually evolved to generate speech spectrum features.

4. The speech synthesis method based on conditional matching stream according to claim 2, characterized in that: The step of generating a speech spectrum that meets the requirements from the speech spectrum features based on the zero-sample learning mechanism includes: Based on the zero-sample learning mechanism, the speech spectrum features are used to generate a speech style that meets the requirements; Combine the required voice style with the semantic information of the input text to generate a speech spectrum that matches the emotion.

5. The speech synthesis method based on conditional matching stream according to claim 4, characterized in that: The step of generating a speech style that meets the requirements from speech spectrum features based on the zero-sample learning mechanism includes: Based on the zero-shot learning mechanism, abstract representations of different speech styles are learned during the training process; The speech style that conforms to the speech spectrum characteristics is extracted and generated from the abstract representations of different speech styles.

6. The speech synthesis method based on conditional matching stream according to claim 4, characterized in that: The step of combining the speech style that meets the requirements with the semantic information of the input text to generate a speech spectrum that meets the emotion includes: Acquire the input emotional prompt features and the semantic information of the input text to match the emotional voice style in the required voice style; Generate speech spectrum output that matches the speech style of the emotion.

7. The speech synthesis method based on conditional matching stream according to claim 1, characterized in that: The step of restoring the speech spectrum to a speech signal and outputting the speech signal comprises: Based on the neural vocoder, the speech spectrum is generated into a speech signal output.

8. A speech synthesis device based on conditional matching stream, characterized in that: include: The conversion module is used to preprocess the input text and convert it into an embedding vector; A classification module is used to map the input text combined with the embedding vector to the intermediate representation space based on the stream matching model, and gradually generate speech spectrum features from the input text in the intermediate representation space; A training module is used to generate a speech spectrum that meets the requirements based on the speech spectrum features based on a zero-sample learning mechanism; The restoration module is used to restore the speech spectrum to a speech signal and output the speech signal.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the speech synthesis method based on conditional matching stream as claimed in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the speech synthesis method based on conditional matching stream as claimed in any one of claims 1 to 7 are implemented.

Images