Speaker-independent pronunciation backstepping method and system based on variational auto-encoder

Abstract

The invention relates to the technical field of speech recognition, and discloses a speaker-independent pronunciation backstepping method and system based on a variational auto-encoder, and the method comprises the steps: collecting speech acoustic data and pronunciation motion data; training a voice variation auto-encoder according to the voice acoustic data, and extracting acoustic feature-based content embedded information and speaker identity embedded information from the voice acoustic data by using the voice variation auto-encoder; training a pronunciation variation auto-encoder according to the pronunciation motion data, and extracting content embedded information and speaker identity embedded information based on pronunciation motion characteristics from the pronunciation motion data by using the pronunciation variation auto-encoder; constructing a pronunciation back-stepping model, and training the pronunciation back-stepping model by using the content embedded information and the speaker identity embedded information based on the acoustic features and the content embedded information and the speaker identity embedded information based on the pronunciation motion features; and inputting the voice acoustic data into the trained pronunciation inverse model, and outputting a pronunciation movement track corresponding to the voice acoustic data.

Description

Technical Field

[0001] The present invention relates to the technical field of speech recognition, and in particular to a speaker-independent pronunciation inversion method and system based on a variational autoencoder. Background Art

[0002] Pronunciation inversion refers to inferring pronunciation posture or pronunciation movement characteristics from speech signals. Pronunciation inversion can mine the relationship between human pronunciation process and acoustic signals from data, helping to explain the causes of certain speech phenomena; compared with acoustic speech features, pronunciation movement characteristics change slowly and are insensitive to environmental noise, which is beneficial for low-bitrate speech coding, speech parameter trajectory generation and control in speech synthesis, and improving the robustness of speech recognition systems; the inverted pronunciation movement characteristics can be used to drive virtual human models to show the movement process of the pronunciation organs for foreign language learners or hearing-impaired children.

[0003] At present, the mainstream speaker-independent pronunciation inversion methods are mainly divided into model adaptive methods, interpolation methods based on typical data, and direct methods based on aggregated multi-person data. The model-based adaptive method requires a certain amount of data of the target speaker to be obtained in advance, but in actual environments it is difficult for us to obtain the target speaker's data in advance; the typical data interpolation method does not require the target speaker's data to be obtained in advance, but it is necessary to calculate the similarity between the target speaker's speech features and the speech features of a large amount of typical data and assume that the similarity distribution in the pronunciation space is the same as the similarity distribution in the acoustic space. This assumption is unreasonable and the speed of the inference stage cannot meet actual needs; the direct method based on aggregated multi-person data mixes speaker content features and personality features together to jointly model the model, and the model is not flexible enough and the accuracy does not meet the requirements.

[0004] Therefore, how to overcome the deficiencies of the existing technology and realize a speaker-independent pronunciation inversion method with high speed, strong flexibility and high accuracy is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] To achieve the purpose of the present invention, the present application provides a speaker-independent pronunciation inversion method based on a variational autoencoder, comprising:

[0006] Step S1: collecting speech acoustic data and pronunciation movement data;

[0007] Step S2: training a speech variational autoencoder according to the speech acoustic data, and using the speech variational autoencoder to extract content embedding information and speaker identity embedding information based on acoustic features from the speech acoustic data;

[0008] Step S3: training a pronunciation variational autoencoder according to the pronunciation movement data, and using the pronunciation variational autoencoder to extract content embedding information and speaker identity embedding information based on pronunciation movement features from the pronunciation movement data;

[0009] Step S4: constructing a pronunciation inversion model, and training the pronunciation inversion model using the content embedding information and speaker identity embedding information based on acoustic features and the content embedding information and speaker identity embedding information based on pronunciation movement features;

[0010] Step S5: input the speech acoustic data into the trained pronunciation inverse model, and output the pronunciation movement features corresponding to the speech acoustic data.

[0011] In some specific embodiments, step S2 and step S3 respectively optimize the parameters of the speech variational autoencoder and the pronunciation variational autoencoder according to the cost function of the standard variational autoencoder:

[0012] L=L rec +L KL

[0013]

[0014] Among them, L, L rec , L KL Represent the overall cost, reconstruction error and KL divergence respectively; z c 、z s Represent the outputs of the content feature encoder and the personal feature encoder respectively; N represents the number of speech features or pronunciation features; Indicates z c The j-th dimension component of the mean of the normal distribution it obeys, ∧ ij represents the jth component of the covariance matrix of the normal distribution; d is z c Dimension.

[0015] In some specific embodiments, step S2 includes:

[0016] Step S21: constructing a speech feature extractor, a local feature extractor, a content feature encoder and a personal feature encoder based on the trained parameters of the speech variational autoencoder, and using the speech feature extractor to extract acoustic features in the speech acoustic data;

[0017] Step S22: Based on the acoustic features, the local feature extraction, content feature encoder and personal feature encoder are used to obtain content embedding information and speaker identity embedding information based on the acoustic features.

[0018] In some specific embodiments, the pronunciation inversion model includes a content inversion network and a speaker inversion network, and step S4 includes:

[0019] Step S41: constructing the content inversion network by superimposing a multi-layer bidirectional long-short time recursive neural network with two layers of fully connected neural networks, so as to achieve mapping from content embedding information based on acoustic features to content embedding information based on pronunciation movement features;

[0020] Step S42: A one-layer linear network is used to construct the speaker inversion network to achieve mapping from speaker identity embedding information based on acoustic features to speaker identity embedding information based on pronunciation movement features.

[0021] In some specific embodiments, step S3 further includes: constructing a pronunciation decoder based on the trained parameters of the speech variational autoencoder.

[0022] In some specific embodiments, step S5 includes:

[0023] The trained pronunciation inverse model outputs content embedding information and speaker identity embedding information based on pronunciation movement features according to the speech acoustic data, and uses the pronunciation decoder to decode and obtain corresponding pronunciation movement features.

[0024] In some specific embodiments, step S4 further includes:

[0025] Step S43: training the content inversion network and the speaker inversion network according to the following formula:

[0026] L=L c +L s +L rec

[0027]

[0028]

[0029] Among them, L, L c , L s and L rec They represent the overall cost, content embedding error, speaker embedding error, and articulation motion reconstruction error respectively; is the content embedding information of the real pronunciation space corresponding to the i-th sample, is the content embedding information of the pronunciation space estimated by the content inversion network; is the speaker embedding information of the real pronunciation space corresponding to the i-th sample, is the speaker embedding information of the pronunciation space estimated by the speaker inversion network; is the true articulatory organ position vector corresponding to the i-th sample, It is the pronunciation organ position vector estimated by using the pronunciation decoder.

[0030] To achieve the same invention purpose, the present application also provides a speaker-independent pronunciation inversion system based on a variational autoencoder, comprising:

[0031] A speech acquisition module, used to acquire speech acoustic data and pronunciation movement data;

[0032] A feature extraction module, used to extract acoustic features from speech acoustic data;

[0033] An acoustic information extraction module, used for obtaining content embedding information and speaker identity embedding information based on the acoustic features from the acoustic features;

[0034] A pronunciation inversion module, comprising a content inversion network and a speaker inversion network, for mapping the content embedding information and speaker identity embedding information based on acoustic features to the content embedding information and speaker identity embedding information based on pronunciation movement features, respectively;

[0035] The pronunciation decoding module is used to decode the pronunciation movement data from the content embedded information and the speaker identity embedded information based on the pronunciation movement feature.

[0036] In some specific embodiments, the pronunciation inversion module is used to:

[0037] The content inversion network is constructed by superimposing a multi-layer bidirectional long-short time recursive neural network with two layers of fully connected neural networks to achieve mapping from content embedding information based on acoustic features to content embedding information based on pronunciation movement features;

[0038] The speaker inversion network is constructed by using a one-layer linear network to achieve mapping from speaker identity embedding information based on acoustic features to speaker identity embedding information based on pronunciation movement features.

[0039] In some specific implementations, the pronunciation inversion module is also used to train the content inversion network and the speaker inversion network according to the following formula:

[0040] L=L c +L s +L rec

[0041]

[0042] Among them, L, L c , L s and L recThey represent the overall cost, content embedding error, speaker embedding error, and articulation motion reconstruction error respectively; is the content embedding information of the real pronunciation space corresponding to the i-th sample, is the content embedding information of the pronunciation space estimated by the content inversion network; is the speaker identity embedding information of the real pronunciation space corresponding to the i-th sample, is the speaker identity embedding information of the pronunciation space estimated by the speaker inversion network; is the true pronunciation movement feature corresponding to the i-th sample, is the pronunciation motion feature estimated by the pronunciation decoder.

[0043] Beneficial effects of the above technical solution:

[0044] The present application provides a speaker-independent pronunciation inversion method based on variational autoencoder. Different from the traditional method based on aggregated multi-person data, the solution proposed by the present invention decouples the speech features to obtain content embedding and speaker identity embedding based on speech features, and then maps the content embedding and speaker identity embedding based on speech features to content embedding and speaker identity embedding based on pronunciation features, respectively. Finally, the decoder synthesizes the content embedding and speaker identity embedding to generate pronunciation motion features. The method proposed by the present invention is to perform inversion in two different information channels respectively, and then synthesize the inversion results of different channels through a decoder; while the traditional method is to splice the content embedding and speaker identity embedding obtained by decoupling the speech features into a new embedding in the feature dimension, and then use an inversion network to estimate the pronunciation features directly from the spliced ​​embedding. Compared with the traditional method, the present application can flexibly construct different mappings according to the distribution characteristics of the embeddings of different channels themselves, and obtain better results. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 A flowchart of a speaker-independent pronunciation inversion method based on a variational autoencoder provided by an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of a system framework of a speaker-independent pronunciation inversion method based on a variational autoencoder provided by an embodiment of the present invention;

[0048] Figure 3 A speech or pronunciation variational autoencoder operation framework diagram of a speaker-independent pronunciation inversion method based on a variational autoencoder provided by one embodiment of the present invention;

[0049] Figure 4 A schematic diagram of the structure of a speaker-independent pronunciation inversion system based on a variational autoencoder is provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0051] Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar symbols throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0052] Embodiment 1

[0053] An embodiment of the present invention provides a speaker-independent pronunciation inversion method based on a variational autoencoder, referring to Figure 1-Figure 3 As shown, including:

[0054] Step S1: Collect speech acoustic data and pronunciation movement data.

[0055] Specifically, an electromagnetic pronunciation recorder is used to collect speech acoustic data and pronunciation movement data.

[0056] Step S2: training a speech variational autoencoder according to the speech acoustic data, and using the speech variational autoencoder to extract content embedding information and speaker identity embedding information based on acoustic features from the speech acoustic data.

[0057] In a specific embodiment of the present invention, step S2 includes:

[0058] Step S21: constructing a speech feature extractor, a local feature extractor, a content feature encoder and a personal feature encoder based on the trained parameters of the speech variational autoencoder, and using the speech feature extractor to extract acoustic features in the speech acoustic data;

[0059] Step S22: Based on the acoustic features, the local feature extractor, the content feature encoder and the personal feature encoder are used to obtain content embedding information and speaker identity embedding information based on the acoustic features.

[0060] Step S3: training a pronunciation variational autoencoder according to the pronunciation movement data, and using the pronunciation variational autoencoder to extract content embedding information and speaker identity embedding information based on pronunciation movement features from the pronunciation movement data.

[0061] In a specific embodiment of the present invention, in step S2 and step S3, the parameters of the speech variational autoencoder and the pronunciation variational autoencoder are optimized according to the cost function of the standard variational autoencoder:

[0062] L=L rec +L KL

[0063]

[0064] Among them, L, L rec , L KL Represent the overall cost, reconstruction error and KL divergence respectively; z c 、z s Represent the outputs of the content feature encoder and the personal feature encoder respectively; N represents the number of speech features or pronunciation features; Indicates z c The j-th dimension component of the mean of the normal distribution it obeys, ∧ ij represents the jth component of the covariance matrix of the normal distribution; d is z c Dimension.

[0065] In a specific embodiment of the present invention, step S3 further includes: constructing a pronunciation decoder based on the trained parameters of the speech variational autoencoder.

[0066] Step S4: construct a pronunciation inversion model, and train the pronunciation inversion model using the content embedding information and speaker identity embedding information based on acoustic features and the content embedding information and speaker identity embedding information based on pronunciation movement features.

[0067] In a specific embodiment of the present invention, the pronunciation inversion model includes a content inversion network and a speaker inversion network, and step S4 includes:

[0068] Step S41: constructing the content inversion network by superimposing a multi-layer bidirectional long-short time recursive neural network with two layers of fully connected neural networks, so as to achieve mapping from content embedding information based on acoustic features to content embedding information based on pronunciation movement features;

[0069] Step S42: A one-layer linear network is used to construct the speaker inversion network to achieve mapping from speaker identity embedding information based on acoustic features to speaker identity embedding information based on pronunciation movement features.

[0070] Step S5: input the speech acoustic data into the trained pronunciation inverse model, and output the pronunciation movement data corresponding to the speech acoustic data.

[0071] In a specific embodiment of the present invention, step S5 includes:

[0072] The trained pronunciation inverse model outputs content embedding information and speaker identity embedding information based on pronunciation movement features according to the speech acoustic data, and uses the pronunciation decoder to decode and obtain corresponding pronunciation movement data.

[0073] In a specific embodiment of the present invention, step S4 further includes:

[0074] Step S43: training the content inversion network and the speaker inversion network according to the following formula:

[0075] L=L c +L s +L rec

[0076]

[0077] Among them, L, L c , L s and L rec They represent the overall cost, content embedding error, speaker embedding error, and articulation motion reconstruction error respectively; is the content embedding information of the real pronunciation space corresponding to the i-th sample, is the content embedding information of the pronunciation space estimated by the content inversion network; is the speaker identity embedding information of the real pronunciation space corresponding to the i-th sample, is the speaker identity embedding information of the pronunciation space estimated by the speaker inversion network; is the true pronunciation movement feature corresponding to the i-th sample, It is the pronunciation motion feature estimated by the pronunciation decoder.

[0078] Among them, the content embedding error is used to optimize the parameters of the content embedding inverse network; the speaker embedding error is used to optimize the parameters of the speaker inverse network; the pronunciation movement reconstruction error is used to comprehensively optimize the parameters of the pronunciation decoder, the content embedding inverse network, and the speaker inverse network.

[0079] Specifically, the parameters of the feature extractor, local feature extractor, content feature encoder and personal feature encoder in the speech variational autoencoder are used to initialize Figure 2 The corresponding parts shown in the following training process Figure 2 The parameters of these components remain unchanged;

[0080] Initialize the decoder parameters using the pronunciation variational autoencoder Figure 2 The decoder shown in ; in the subsequent training process Figure 2 Fine-tune the parameters of the decoder shown in ;

[0081] Random Initialization Figure 2 The parameters of the personal feature inversion network and the content feature inversion network are adjusted in the subsequent training process.

[0082] Specifically, the speech signal is analyzed and synthesized through a speech feature extractor, a local feature extractor, a content encoder, a personal feature encoder, a content inversion network, a speaker inversion network and a decoder to obtain the corresponding motion trajectory of the pronunciation organ. Among them, the speech feature extractor, the local feature extractor, the content encoder and the personal feature encoder are obtained by pre-training with speech acoustic data and pronunciation movement data, and the parameters of each component after training remain unchanged during the entire pronunciation inversion training process; the initial parameters of the decoder come from the pronunciation variational autoencoder, and are fine-tuned during the training process of the entire pronunciation inversion system; the parameters of the content inversion network and the speaker inversion network are randomly initialized, and the parameters are continuously updated during the training process of the entire pronunciation inversion system.

[0083] This application is different from the traditional method based on aggregating multi-person data. The solution proposed in the present invention extracts content embedding and speaker identity embedding based on acoustic features, and then maps the content embedding and speaker identity embedding based on acoustic features to the content embedding and speaker identity embedding based on pronunciation motion features, and finally generates pronunciation motion features by integrating the content embedding and speaker identity embedding through a pronunciation decoder.

[0084] Embodiment 2

[0085] An embodiment of the present invention provides a speaker-independent pronunciation inversion system based on a variational autoencoder, referring to Figure 4 As shown, including:

[0086] A speech acquisition module 10, used for acquiring speech acoustic data and pronunciation movement data;

[0087] A feature extraction module 20, used to extract acoustic features from speech acoustic data;

[0088] An acoustic information extraction module 30, used to obtain content embedding information and speaker embedding information based on the acoustic features from the acoustic features;

[0089] A pronunciation inversion module 40, comprising a content inversion network and a speaker inversion network, for mapping the content embedding information and speaker identity embedding information based on acoustic features to the content embedding information and speaker identity embedding information based on pronunciation movement features, respectively;

[0090] The pronunciation decoding module 50 is used to decode the pronunciation movement data from the content embedded information and the speaker embedded information based on the pronunciation movement feature.

[0091] In a specific embodiment of the present invention, the pronunciation inference module 40 is used to:

[0092] The content inversion network is constructed by superimposing a multi-layer bidirectional long-short time recursive neural network with two layers of fully connected neural networks to achieve mapping from content embedding information based on acoustic features to content embedding information based on pronunciation movement features;

[0093] The speaker inversion network is constructed by using a one-layer linear network to achieve mapping from speaker identity embedding information based on acoustic features to speaker identity embedding information based on pronunciation movement features.

[0094] In a specific embodiment of the present invention, the pronunciation inversion module 40 is further used to train the content inversion network and the speaker inversion network according to the following formula:

[0095] L=L c +L s +L rec

[0096]

[0097]

[0098] Among them, L, L c , L s and L rec They represent the overall cost, content embedding error, speaker embedding error, and articulation motion reconstruction error respectively; is the content embedding information of the real pronunciation space corresponding to the i-th sample, is the content embedding information of the pronunciation space estimated by the content inversion network; is the speaker identity embedding information of the real pronunciation space corresponding to the i-th sample, is the speaker identity embedding information of the pronunciation space estimated by the speaker inversion network; is the true articulatory organ position vector corresponding to the i-th sample, is the pronunciation organ position vector estimated by the pronunciation decoder.

[0099] Different from the traditional method based on aggregating multi-person data, the scheme proposed in the present invention extracts content embedding and speaker identity embedding based on acoustic features, and then maps the content embedding and speaker identity embedding based on acoustic features to the content embedding and speaker identity embedding based on pronunciation motion features, respectively. Finally, the pronunciation decoder is used to integrate the content embedding and speaker embedding to generate pronunciation motion features.

[0100] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

[0101] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referenced to each other. The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing terminal device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps of the functions specified in one or more boxes. Although the preferred embodiments of the embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they know the basic creative concept. Therefore, the attached claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention. Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0102] The method and device provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

[0103] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", "one specific embodiment" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A speaker-independent pronunciation inversion method based on variational autoencoder, characterized in that: include: Step S1: collecting speech acoustic data and pronunciation movement data; Step S2: training a speech variational autoencoder according to the speech acoustic data, and using the speech variational autoencoder to extract content embedding information and speaker identity embedding information based on acoustic features from the speech acoustic data; Step S3: training a pronunciation variational autoencoder according to the pronunciation movement data, and using the pronunciation variational autoencoder to extract content embedding information and speaker identity embedding information based on pronunciation movement features from the pronunciation movement data; Step S4: constructing a pronunciation inversion model, and training the pronunciation inversion model using the content embedding information and speaker identity embedding information based on acoustic features and the content embedding information and speaker identity embedding information based on pronunciation movement features; Step S5: input the speech acoustic data into the trained pronunciation inverse model, and output the pronunciation movement trajectory corresponding to the speech acoustic data.

2. The speaker-independent pronunciation inversion method based on variational autoencoder according to claim 1, characterized in that: In step S2 and step S3, the parameters of the speech variational autoencoder and the pronunciation variational autoencoder are optimized according to the cost function of the standard variational autoencoder: L=L rec +L KL Among them, L, L rec , L KL Represent the overall cost, reconstruction error and KL divergence respectively; z c 、z s Represent the outputs of the content feature encoder and the personal feature encoder respectively; N represents the number of speech features or pronunciation features; Indicates z c The j-th dimension component of the mean of the normal distribution it obeys, ∧ ij represents the jth component of the covariance matrix of the normal distribution; d is z c Dimension.

3. The speaker-independent pronunciation inversion method based on variational autoencoder according to claim 1, characterized in that: Step S2 includes: Step S21: constructing a speech feature extractor, a local feature extractor, a content feature encoder and a personal feature encoder based on the trained parameters of the speech variational autoencoder, and using the speech feature extractor to extract acoustic features in the speech acoustic data; Step S22: Based on the acoustic features, the local feature extractor, the content feature encoder and the personal feature encoder are used to obtain content embedding information and speaker identity embedding information based on the acoustic features.

4. The speaker-independent pronunciation inversion method based on variational autoencoder according to claim 1, characterized in that: The pronunciation inversion model includes a content inversion network and a speaker inversion network. Step S4 includes: Step S41: constructing the content inversion network by superimposing a multi-layer bidirectional long-short time recursive neural network with two layers of fully connected neural networks, so as to achieve mapping from content embedding information based on acoustic features to content embedding information based on pronunciation movement features; Step S42: A one-layer linear network is used to construct the speaker inversion network to achieve mapping from speaker identity embedding information based on acoustic features to speaker identity embedding information based on pronunciation movement features.

5. The speaker-independent pronunciation inversion method based on variational autoencoder according to claim 4, characterized in that: Step S3 also includes: constructing a pronunciation decoder based on the trained parameters of the speech variational autoencoder.

6. The speaker-independent pronunciation inversion method based on variational autoencoder according to claim 5, characterized in that: Step S5 includes: The trained pronunciation inverse model outputs content embedding information and speaker embedding information based on pronunciation movement features according to the speech acoustic data, and obtains corresponding pronunciation movement data by decoding using the pronunciation decoder.

7. The speaker-independent pronunciation inversion method based on variational autoencoder according to claim 5, characterized in that: Step S4 also includes: Step S43: training the content inversion network and the speaker inversion network according to the following formula: L=L c +L s +L rec Among them, L, L c , L s and L rec They represent the overall cost, content embedding error, speaker identity embedding error, and articulation motion reconstruction error respectively; is the content embedding information of the real pronunciation space corresponding to the i-th sample, is the content embedding information of the pronunciation space estimated by the content inversion network; is the speaker identity embedding information of the real pronunciation space corresponding to the i-th sample, is the speaker identity embedding information of the pronunciation space estimated by the speaker inversion network; is the true pronunciation feature corresponding to the i-th sample, is the pronunciation feature estimated by the pronunciation decoder.

8. A speaker-independent pronunciation inversion system based on variational autoencoder, characterized in that: include: A speech acquisition module, used to acquire speech acoustic data and pronunciation movement data; A feature extraction module, used to extract acoustic features from speech acoustic data; An acoustic information extraction module, used for obtaining content embedding information and speaker identity embedding information based on the acoustic features from the acoustic features; A pronunciation inversion module, comprising a content inversion network and a speaker inversion network, for mapping the content embedding information and speaker embedding information based on acoustic features to the content embedding information and speaker identity embedding information based on pronunciation movement features, respectively; The pronunciation decoding module is used to decode the pronunciation movement data from the content embedded information and the speaker identity embedded information based on the pronunciation movement feature.

9. The speaker-independent pronunciation inversion system based on variational autoencoder according to claim 8, characterized in that: The pronunciation inversion module is used for: The content inversion network is constructed by superimposing a multi-layer bidirectional long-short time recursive neural network with two layers of fully connected neural networks to achieve mapping from content embedding information based on acoustic features to content embedding information based on pronunciation movement features; The speaker inversion network is constructed by using a one-layer linear network to achieve mapping from speaker embedding information based on acoustic features to speaker embedding information based on pronunciation movement features.

10. The speaker-independent pronunciation inversion system based on variational autoencoder according to claim 9, characterized in that: The pronunciation inversion module is also used to train the content inversion network and the speaker inversion network according to the following formula: L=L c +L s +L rec Among them, L, L c , L s and L rec They represent the overall cost, content embedding error, speaker embedding error, and articulation motion reconstruction error respectively; is the content embedding information of the real pronunciation space corresponding to the i-th sample, is the content embedding information of the pronunciation space estimated by the content inversion network; is the speaker embedding information of the real pronunciation space corresponding to the i-th sample, is the speaker embedding information of the pronunciation space estimated by the speaker inversion network; is the true pronunciation feature corresponding to the i-th sample, is the pronunciation feature estimated by the pronunciation decoder.

Images