Singing feature conversion method, electronic equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a singing feature conversion method, electronic equipment and a storage medium. According to the singing feature conversion method, the source audio and the target sound identifier need to be acquired firstly, then the source audio and the target sound identifier are input into a pre-trained audio processing model, the audio processing model comprises a vocoder, a music emotion recognition network and a variational adversarial network, and acoustic feature extraction is performed on the source audio based on the vocoder; the method comprises the steps of obtaining content feature parameters, performing emotion information extraction on a source audio based on a music emotion recognition network to obtain a source emotion feature vector, and further replacing the source emotion feature vector with a target emotion feature vector through a variational adversarial network based on a target sound identifier. And finally performing reconstruction processing on the target emotion feature vector and the content feature parameter based on the variational adversarial network to obtain a target audio after singing feature conversion, and getting rid of dependence on parallel data in a singing feature conversion process.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a singing feature conversion method, electronic device, and storage medium. Background Art

[0002] Voice Conversion (VC) refers to converting the non-linguistic features of a voice while preserving its linguistic features. The linguistic features of a voice refer to the semantic content of the corresponding language pronunciation, while the non-linguistic features of a voice include accent, timbre, speaking style, etc. Singing Voice Conversion (SVC) is a technology that converts the non-linguistic features of a song from a source singer to a target singer without changing the language content of the song.

[0003] In the related art, singing feature conversion requires paired parallel data, that is, obtaining parallel data by singing the same lyrics by different people. However, for real applications, such data is difficult to obtain. Therefore, how to get rid of the dependence on parallel data in the process of realizing singing feature conversion has become a problem that needs to be solved urgently in the industry. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a singing feature conversion method, electronic device, and storage medium, which can get rid of the dependence on parallel data in the process of realizing singing feature conversion.

[0005] The singing feature conversion method according to the first aspect of the present application includes:

[0006] Obtaining source audio and target sound identification, wherein the source audio includes an audio segment of a source singer;

[0007] Inputting the source audio and the target sound identifier into a pre-trained audio processing model, wherein the audio processing model includes a vocoder, a music emotion recognition network, and a variational adversarial network;

[0008] Extracting acoustic features of the source audio based on the vocoder to obtain content feature parameters;

[0009] Extracting emotional information from the source audio based on the music emotion recognition network to obtain a source emotion feature vector;

[0010] Based on the target sound identifier, replacing the source emotion feature vector with a target emotion feature vector through the variational adversarial network;

[0011] The target emotion feature vector and the content feature parameters are reconstructed based on the variational adversarial network to obtain the target audio after the singing feature conversion.

[0012] According to some embodiments of the present application, before inputting the source audio and the target sound identifier into the pre-trained audio processing model, the method further includes pre-training the audio processing model, specifically including:

[0013] Acquire a training data set, the training data set comprising a first number of training audios, each of the training audios being configured with a one-to-one corresponding training sound identifier;

[0014] The first number of training audios are grouped to obtain a second number of training audio groups, each of which includes the training audios of different singers singing the same audio segment;

[0015] Inputting a second number of the training audio groups into the original audio processing model, and iteratively training the audio processing model based on the training audios and the training sound identifiers;

[0016] When the audio processing model meets the first preset condition in iterative training, the pre-trained audio processing model is obtained.

[0017] According to some embodiments of the present application, inputting the second number of training audio groups into the original audio processing model, and iteratively training the audio processing model based on the training audio and the training sound identifier includes:

[0018] In each round of iterative training, an iterative audio group is selected from the second number of training audio groups, and a training source audio and a training target audio are selected from the iterative audio group;

[0019] Extracting acoustic features of the training source audio based on the vocoder to obtain training content parameters;

[0020] Extracting emotion information from the training source audio based on the pre-trained music emotion recognition network to obtain an initial emotion vector;

[0021] Extracting emotion information from the training target audio based on the pre-trained music emotion recognition network to obtain an emotion replacement vector;

[0022] Determine the training sound identifier of the training target audio configuration as a sound replacement identifier, and based on the sound replacement identifier, replace the emotion initial vector with the emotion replacement vector through the variational adversarial network;

[0023] Reconstructing the emotion replacement vector and the training content parameters based on the variational adversarial network to obtain predicted audio;

[0024] Perform loss calculation based on the training source audio, the training target audio and the predicted audio to obtain prediction deviation data;

[0025] After each round of iterative training, the model parameters of the audio processing model are updated based on the prediction deviation data.

[0026] According to some embodiments of the present application, the training content parameters include training prosody parameters and training spectrum parameters, and the reconstructing the emotion replacement vector and the training content parameters based on the variational adversarial network to obtain the predicted audio includes:

[0027] Performing language feature analysis on the training spectrum parameters to obtain a training latent vector, wherein the training latent vector is used to represent the sentence content of the training source audio;

[0028] The emotion replacement vector, the training prosody parameter and the training latent vector are integrated and spliced ​​to obtain the predicted audio.

[0029] According to some embodiments of the present application, the prediction deviation data includes a loss function value, and the loss calculation is performed based on the training source audio, the training target audio, and the predicted audio to obtain the prediction deviation data, including:

[0030] Perform a first loss calculation based on the training source audio and the training latent vector to obtain a coding loss element corresponding to the music emotion recognition network;

[0031] Performing a second loss calculation based on the training source audio, the training target audio, and the predicted audio to obtain a decoding loss element corresponding to the variational adversarial network;

[0032] Perform loss construction based on the encoding loss element and the decoding loss element to obtain a prediction loss function;

[0033] Based on the prediction loss function, loss calculation is performed on the training source audio, the training target audio and the predicted audio to obtain the loss function value.

[0034] According to some embodiments of the present application, before performing loss calculation on the training source audio, the training target audio, and the predicted audio based on the prediction loss function to obtain the loss function value, the method further includes:

[0035] Performing a third loss calculation based on the training source audio, the training target audio, and the predicted audio to obtain a discriminative loss element corresponding to the variational adversarial network;

[0036] The discriminant loss element is integrated into the prediction loss function based on a preset weight factor.

[0037] According to some embodiments of the present application, extracting acoustic features from the source audio based on the vocoder to obtain content feature parameters includes:

[0038] Extracting fundamental frequency features of the source audio based on the vocoder to obtain source prosody feature parameters, wherein the source prosody feature parameters are used to characterize the prosody content of the source audio;

[0039] Extracting spectrum characteristics of the source audio based on the vocoder to obtain spectrum characteristic parameters;

[0040] The source rhythm feature parameter and the spectrum feature parameter are integrated to obtain the content feature parameter.

[0041] According to some embodiments of the present application, the target emotion feature vector and the content feature parameter are reconstructed based on the variational adversarial network to obtain the target audio after the singing feature conversion, including:

[0042] Acquire the frequency spectrum characteristic parameter and the source rhythm characteristic parameter from the content characteristic parameter;

[0043] Performing language feature analysis on the frequency spectrum feature parameters to obtain a sentence latent vector, wherein the sentence latent vector is used to represent the sentence content of the source audio;

[0044] The target emotion feature vector, the source prosody feature parameter and the sentence latent vector are integrated and spliced ​​to obtain the target audio.

[0045] In a second aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the singing feature conversion method as described in any one of the embodiments of the first aspect of the present application.

[0046] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement a singing feature conversion method as described in any one of the embodiments of the first aspect of the present application.

[0047] The singing feature conversion method, electronic device, and storage medium according to the embodiments of the present application have at least the following beneficial effects:

[0048] The singing feature conversion method of the present application needs to first obtain the source audio and the target sound identification, the source audio includes the audio segment of the source singer, and then input the source audio and the target sound identification into a pre-trained audio processing model, the audio processing model includes a vocoder, a music emotion recognition network, and a variational adversarial network. Based on the vocoder, the acoustic features of the source audio are extracted to obtain content feature parameters. Further, based on the music emotion recognition network, the emotional information of the source audio is extracted to obtain a source emotional feature vector. Further, based on the target sound identification, the source emotional feature vector is replaced with the target emotional feature vector through the variational adversarial network. Finally, the target emotional feature vector and the content feature parameters are reconstructed based on the variational adversarial network to obtain the target audio after the singing feature conversion. It should be pointed out that the singing feature conversion is realized through the audio processing model of the present application, which can get rid of the dependence on parallel data in the process of realizing the singing feature conversion.

[0049] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0051] Figure 1 An optional flow chart of a singing feature conversion method provided in an embodiment of the present application;

[0052] Figure 2 This is a schematic block diagram of an optional structure of a music emotion recognition network according to an embodiment of the present application;

[0053] Figure 3 Another optional flow chart of the singing voice feature conversion method provided in the embodiment of the present application;

[0054] Figure 4 for Figure 3 Optional flow chart of step S303;

[0055] Figure 5 This is a schematic block diagram of an optional structure of the audio processing model in the embodiment of the present application;

[0056] Figure 6 for Figure 4 Optional flow chart of step S406;

[0057] Figure 7 for Figure 4 Optional flow chart of step S407;

[0058] Figure 8Another optional flow chart of the singing voice feature conversion method provided in the embodiment of the present application;

[0059] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0060] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0061] In the description of this application, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0062] In the description of the present application, it should be understood that descriptions involving orientation, such as up, down, left, right, front, back, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0063] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0064] In the description of this application, it should be noted that, unless otherwise clearly defined, the terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in this application in combination with the specific content of the technical solution. In addition, the identification of specific steps below does not represent a limitation on the order of steps and execution logic. The execution order and execution logic between each step should be understood and inferred with reference to the contents described in the embodiment.

[0065] Voice Conversion (VC) refers to converting the non-linguistic features of a voice while preserving its linguistic features. The linguistic features of a voice refer to the semantic content of the corresponding language pronunciation, while the non-linguistic features of a voice include accent, timbre, speaking style, etc. Singing Voice Conversion (SVC) is a technology that converts the non-linguistic features of a song from a source singer to a target singer without changing the language content of the song.

[0066] In the related art, singing feature conversion requires paired parallel data, that is, different people sing the same lyrics to obtain parallel data. However, for real applications, such data is difficult to obtain. Therefore, how to get rid of the dependence on parallel data in the process of realizing singing feature conversion has become a problem that needs to be solved urgently in the industry. In addition, the current singing feature conversion tasks do not take the emotion factor into consideration. Some exemplary embodiments of the present application also model this singer-related characteristic and integrate it into the singing feature conversion.

[0067] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a singing feature conversion method, electronic device, and storage medium, which can get rid of the dependence on parallel data in the process of realizing singing feature conversion.

[0068] The singing feature conversion method is further described below in conjunction with the accompanying drawings.

[0069] Reference Figure 1 The singing feature conversion method of the first aspect embodiment of the present application shown may include, but is not limited to, the following steps S101 to S106.

[0070] Step S101, obtaining source audio and target sound identification, the source audio including an audio segment of a source singer;

[0071] Step S102, inputting the source audio and the target sound identifier into a pre-trained audio processing model, the audio processing model including a vocoder, a music emotion recognition network, and a variational adversarial network;

[0072] Step S103, extracting acoustic features of the source audio based on the vocoder to obtain content feature parameters;

[0073] Step S104, extracting emotion information from the source audio based on the music emotion recognition network to obtain a source emotion feature vector;

[0074] Step S105, based on the target sound identifier, replacing the source emotion feature vector with the target emotion feature vector through a variational adversarial network;

[0075] Step S106, reconstructing the target emotion feature vector and content feature parameters based on the variational adversarial network to obtain the target audio after the singing feature conversion.

[0076] The singing feature conversion method of the present application shown by steps S101 to S106 needs to first obtain the source audio and the target sound identification, the source audio includes the audio segment of the source singer, and then input the source audio and the target sound identification into a pre-trained audio processing model, the audio processing model includes a vocoder, a music emotion recognition network, and a variational adversarial network. Based on the vocoder, acoustic features of the source audio are extracted to obtain content feature parameters. Further, based on the music emotion recognition network, emotional information is extracted from the source audio to obtain a source emotion feature vector. Further, based on the target sound identification, the source emotion feature vector is replaced with the target emotion feature vector through the variational adversarial network. Finally, the target emotion feature vector and the content feature parameters are reconstructed based on the variational adversarial network to obtain the target audio after singing feature conversion. It should be pointed out that the singing feature conversion is realized by the audio processing model of the present application, which can get rid of the dependence on parallel data in the process of realizing singing feature conversion.

[0077] In step S101 of some embodiments, the source audio and the target sound identification are obtained, and the source audio includes the audio segment of the source singer. It should be noted that the source audio refers to the audio segment of the source singer; the target sound identification is used to identify the target singer. In some exemplary embodiments, the process of singing feature conversion is actually for the audio segment of the source singer in the source audio, from which the singing features of the source singer are converted to the singing features of the target singer. It should be pointed out that when converting singing features, the target sound identification is used as a reference for the singing feature conversion in the present application, and the audio processing model is used to realize the conversion of the singing features of the source singer to the singing features of the target singer based on the target sound identification. It should be understood that converting the singing features of the source singer to the singing features of the target singer can be converting the emotional features of the source singer to the emotional features of the target singer, or it can be converting part of the content features of the source singer to the content features of the target singer.

[0078] It should be understood that sound mainly includes three characteristics: loudness, pitch and timbre. Among them, loudness refers to the size of the sound perceived by people subjectively, which is determined by the amplitude and the distance between people and the sound source. The larger the amplitude, the louder the loudness. The smaller the distance between people and the sound source, the louder the loudness. In this application, the sound characteristics specifically refer to the attribute characteristics of the audio itself, so the distance between people and the sound source is not considered here, and the amplitude of the audio is regarded as the loudness feature; the pitch of the sound is determined by the frequency (the number of vibrations completed per unit time). The faster the object vibrates, the higher the pitch of the sound, and the slower the vibration, the lower the pitch of the sound; the timbre, also known as the sound quality, the waveform of the sound determines the timbre of the sound. The timbre itself is an abstract thing, but the waveform corresponds to the intuitive expression of this abstraction. Different timbres have different waveforms.

[0079] It should be made clear that singing voice features are a special type of sound features. In addition to analyzing the amplitude, frequency, and waveform of the sound, the analysis of singer features also includes the analysis of some characteristic parameters. These characteristic parameters may include, but are not limited to:

[0080] The fundamental frequency F0 is used to reflect a series of pulse signals emitted by the vocal cords. If this part of the pulse signal is divided into periodic signals and non-periodic signals, the rhythmic information and structural information of the singing voice can be obtained;

[0081] Spectrum Parameter (SP), also known as Spectrum Envelop. After the human voice is emitted by the vocal cords, it passes through the filter channel, that is, through the throat, mouth, lips, teeth and other parts of the vocal tract, under common modulation, humans can produce different timbres, different vowels and consonants. It should be pointed out that the throat, mouth, lips, teeth and other parts of the vocal tract together form a filter system, which describes the timbre and speech content of the singing voice, and is the frequency response of the linear time-invariant system.

[0082] In step S102 of some embodiments, the source audio and the target sound identification are input into a pre-trained audio processing model, and the audio processing model includes a vocoder, a music emotion recognition network, and a variational adversarial network. It should be noted that the audio processing model is pre-trained, and during the pre-training process, the audio processing model gradually has the ability to convert the singing characteristics of the source singer into the singing characteristics of the target singer. The audio processing model can specifically include a vocoder, a music emotion recognition network, and a variational adversarial network.

[0083] In step S103 of some embodiments, acoustic features are extracted from the source audio based on the vocoder to obtain content feature parameters. It should be noted that the vocoder is a type of speech codec that analyzes and synthesizes audio data compression, multiplexing, voice encryption or voice conversion of human voice signals; content feature parameters refer to feature parameters that reflect the rhythm information, structural information, timbre, voice content and other content information of the singing voice, for example, the fundamental frequency F0 and spectrum parameters mentioned above. It should be pointed out that the vocoder can analyze the speech signal in the source audio, extract the feature parameters of the speech signal for encoding and encryption. The parsing process of the vocoder can be carried out in the frequency domain, perform spectral analysis on the speech signal, identify the unvoiced and voiced sounds, measure the fundamental frequency of the voiced sounds, and then select the unvoiced-voiced judgment, the fundamental frequency of the voiced sounds and the spectrum envelope as feature parameters for transmission. The parsing process of the vocoder can also be carried out in the time domain, using its periodicity to extract some parameters for linear prediction, or perform correlation analysis on the speech signal. According to the working principle, vocoders can be divided into: channel vocoders, formant vocoders, pattern vocoders, linear prediction vocoders, correlation vocoders, and orthogonal function vocoders.

[0084] It should be noted that there are two types of common vocoders at present, one is based on traditional signal processing algorithms, such as STRAIGHT, Griffim Lim, WORLD; the second is the vocoder based on neural network (MelGAN, LPCNet, etc.) that has emerged in recent years. Compared with the vocoder based on neural network, the computational complexity of the WORLD vocoder is lower, and it is more suitable for low-performance computing equipment, and the WORLD vocoder does not need a large amount of data to be trained, and can be directly calculated based on the signal processing algorithm. It should be emphasized that the WORLD vocoder can realize real-time, high-quality speech synthesis. Therefore, in some preferred embodiments, the WORLD vocoder can be selected and used as the vocoder of the present application, and applied to the audio processing model.

[0085] According to some embodiments of the present application, in step S103, fundamental frequency features of the source audio may be first extracted based on a vocoder to obtain source prosody feature parameters, where the source prosody feature parameters are used to characterize the prosody content of the source audio, and then spectrum features of the source audio may be extracted based on the vocoder to obtain spectrum feature parameters, and finally the source prosody feature parameters and the spectrum feature parameters may be integrated to obtain content feature parameters.

[0086] In step S104 of some embodiments, emotional information is extracted from the source audio based on the music emotion recognition network to obtain a source emotion feature vector. It should be noted that the music emotion recognition network refers to a neural network model for extracting emotional features from singing. Music emotion recognition (MER) is a process of extracting and analyzing music features using a computer, forming a mapping relationship between music features and emotional space, and identifying the emotions expressed by the music. Music features are usually extracted from audio signals, symbolic music scores, lyrics texts, or even from biological features such as electroencephalograms. The emotional space can be represented by a finite number of discrete categories or an infinite number of points in a continuous multidimensional space. Music emotion recognition belongs to the interdisciplinary research field of music psychology, audio signal processing, and natural language processing, and music emotion recognition is a subtask of music information retrieval. Music emotion recognition can be widely used in music recommendation, retrieval, visualization, automatic composition, psychotherapy and other fields.

[0087] Reference Figure 2 In some more specific embodiments of the present application, the music emotion recognition network may specifically include a convolutional layer, a bidirectional long short-term memory layer, and an attention layer. It should be pointed out that the traditional convolutional neural network (CNN) learns the spatial information within the receptive field, and with the deepening of learning and training, the features learned by the convolutional neural network are more and more abstract, and for music emotion recognition, the abstract spatial features learned by the convolutional neural network are required, so it is set after the convolutional layer. It should be noted that the bidirectional long short-term memory layer is specifically a bidirectional long short-term memory network (Bi-directional Long Short-Term Memory), which is used to capture the long-distance dependency information of the context in the abstract spatial features, and then the features weighted by the time domain are extracted through the attention mechanism of the attention layer, and the emotional feature vector used to characterize the emotional features of the singing voice can be obtained.

[0088] It should be noted that different singers can sing different emotional effects in the same song. These emotional effects can be happy, depressed, free and unrestrained, or other types of feelings in human feelings. In view of these emotional effects, it is necessary to extract the above-mentioned emotional feature vectors from the singing voice, and based on these emotional feature vectors, the emotional effects contained in the singing voice can be quantitatively analyzed.

[0089] In step S105 of some embodiments, based on the target sound identifier, the source emotion feature vector is replaced with the target emotion feature vector through a variational adversarial network. In some exemplary embodiments of the present application, the variational adversarial network (Variational Autoencoding Wasserstein Generative Adversarial Network, VAW-GAN) refers to a generative adversarial network (GAN) based on a variational autoencoder (VAE). It should be emphasized that, in view of the abstractness of emotional features, the related technologies generally do not take the emotion factor into consideration when performing singing feature conversion tasks, and the variational adversarial network of the present application is used to replace the source emotion feature vector with the target emotion feature vector, aiming to incorporate the emotion feature conversion into the singing feature conversion process and improve the smoothness of the singing feature conversion.

[0090] In step S106 of some embodiments, the target emotional feature vector and content feature parameters are reconstructed based on the variational adversarial network to obtain the target audio after the singing feature conversion. It should be noted that the variational adversarial network is also used to reconstruct the target emotional feature vector and content feature parameters to obtain the target audio after the singing feature conversion. It should be understood that after the acoustic features of the source audio are extracted to obtain the content feature parameters, and the source emotional feature vector is replaced with the target emotional feature vector through the variational adversarial network, it is necessary to further reconstruct the target emotional feature vector and the content feature parameters so that the emotional feature vector can be replaced while retaining the content information such as the rhythmic information, structural information, timbre, and voice content in the source audio, thereby completing the overall process of singing feature conversion.

[0091] According to some embodiments of the present application, in step S106, spectral feature parameters and source prosodic feature parameters can be first obtained from the content feature parameters, and then the spectral feature parameters can be subjected to language feature analysis to obtain a sentence latent vector, where the sentence latent vector is used to characterize the sentence content of the source audio. Furthermore, the target emotion feature vector, the source prosodic feature parameters and the sentence latent vector can be integrated and spliced ​​to obtain the target audio.

[0092] Reference Figure 3 According to some embodiments of the present application, before step S102, the singing feature conversion method of the present application also includes pre-training the audio processing model, which may specifically include, but is not limited to, the following steps S301 to S304.

[0093] Step S301, obtaining a training data set, the training data set including a first number of training audios, each training audio being configured with a one-to-one corresponding training sound identifier;

[0094] Step S302, grouping the first number of training audios to obtain a second number of training audio groups, each training audio group including training audios of different singers singing the same audio segment;

[0095] Step S303, inputting a second number of training audio groups into the original audio processing model, and iteratively training the audio processing model based on the training audio and the training sound identifier;

[0096] Step S304: When the audio processing model meets the first preset condition in iterative training, a pre-trained audio processing model is obtained.

[0097] In step S301 of some embodiments, a training data set is obtained. It should be noted that the training data set includes a first number of training audios, each training audio is configured with a one-to-one corresponding training sound identifier, wherein the training sound identifier is used to identify which singer the training audio specifically corresponds to. According to some embodiments provided in the present application, the training data set can use some commonly used corpora, such as the DAMP corpus and the NUS-48E corpus, both of which contain songs sung by different singers. In some more specific embodiments, the NUS-48E corpus can be used to select 48 songs sung by 12 professional singers as the training data set.

[0098] Step S302, grouping the first number of training audios to obtain a second number of training audio groups, each training audio group including training audios of different singers singing the same audio segment. It should be noted that grouping the first number of training audios can specifically be grouping the first number of training audios based on training audios of the same audio segment, and in each training audio group, training audios of different singers singing the same audio segment are included.

[0099] Step S303, input the second number of training audio groups into the original audio processing model, and iteratively train the audio processing model based on the training audio and the training sound identifier. It should be emphasized that the audio processing model gradually has the ability to convert the source singer's singing features into the target singer's singing features during the pre-training process. It should be pointed out that the audio processing model needs to be iteratively trained according to the training audio groups. The reason is that different training audio groups correspond to one segment one by one. If the training samples of different audio segments are used to train the audio processing model, it is difficult to improve the audio processing model's ability to convert singing features for the same segment.

[0100] Reference Figure 4 According to some embodiments of the present application, step S303 may include, but is not limited to, the following steps S401 to S408.

[0101] Step S401, in each round of iterative training, selecting an iterative audio group from the second number of training audio groups, and selecting a training source audio and a training target audio from the iterative audio group;

[0102] Step S402, extracting acoustic features of the training source audio based on the vocoder to obtain training content parameters;

[0103] Step S403, extracting emotion information from the training source audio based on the pre-trained music emotion recognition network to obtain an initial emotion vector;

[0104] Step S404, extracting emotion information from the training target audio based on the pre-trained music emotion recognition network to obtain an emotion replacement vector;

[0105] Step S405, determining the training sound identifier of the training target audio configuration as the sound replacement identifier, and replacing the initial emotion vector with the emotion replacement vector through a variational adversarial network based on the sound replacement identifier;

[0106] Step S406, reconstructing the emotion replacement vector and the training content parameters based on the variational adversarial network to obtain predicted audio;

[0107] Step S407, performing loss calculation based on the training source audio, the training target audio and the predicted audio to obtain prediction deviation data;

[0108] Step S408: after each round of iterative training, the model parameters of the audio processing model are updated based on the prediction deviation data.

[0109] In step S401 of some embodiments, in each round of iterative training, an iterative audio group is selected from the second number of training audio groups, and a training source audio and a training target audio are selected from the iterative audio group. It should be noted that in one iterative round, the audio file to be replaced as the emotional feature is called the training source audio, and the audio file as the target of emotional feature replacement is called the training target audio. It should be emphasized that since the training source audio and the training target audio are selected from the same iterative audio group, the training source audio and the training target audio are actually the singing audios of two singers for the same audio segment.

[0110] In step S402 of some embodiments, acoustic features of the training source audio are extracted based on the vocoder to obtain training content parameters. It should be noted that the training content parameters are used to characterize the characteristic parameters of the vocal content of the training audio, such as the fundamental frequency F0 and spectrum parameters of the training samples. It should be emphasized that the purpose of iterative training is to allow the audio processing model to gradually have the ability to convert the singing features of the source singer to the singing features of the target singer during the pre-training process. In some exemplary embodiments, the singing features of the source singer are converted to the singing features of the target singer, specifically, the emotional features of the source singer can be converted to the emotional features of the target singer, and in this process, the content features of the audio can remain unchanged. Therefore, in order to achieve this ability of training the audio processing model, it is necessary to first extract acoustic features of the training source audio based on the vocoder to obtain training content parameters before extracting emotional features from the training audio.

[0111] From step S403 to step S404, emotion information is extracted from the training source audio based on the pre-trained music emotion recognition network to obtain an initial emotion vector, and emotion information is extracted from the training target audio based on the pre-trained music emotion recognition network to obtain an emotion replacement vector. It should be noted that the initial emotion vector is used to characterize the emotion characteristics of the training source audio, and the emotion replacement vector is used to characterize the emotion characteristics of the training target audio. It should be pointed out that the music emotion recognition network has undergone pre-training of the ability to extract music emotion features before being used to iteratively train the audio processing model. Among them, the pre-training of the music emotion recognition network is mainly carried out by supervised training with annotated audio data sets. It should be clarified that the pre-training of the music emotion recognition network is intended to enable the music emotion recognition network to have the ability to extract emotion features, or to improve the emotion feature extraction ability of the music emotion recognition network.

[0112] Step S405, the training sound identifier of the training target audio configuration is determined as the sound replacement identifier, and based on the sound replacement identifier, the initial emotion vector is replaced with the emotion replacement vector through the variational adversarial network. It should be emphasized that each training audio configuration has a one-to-one corresponding training sound identifier, so the training sound identifier can be used as a search index for the training audio. The sound replacement identifier refers to the training sound identifier of the training target audio configuration, and based on the sound replacement identifier, the initial emotion vector is replaced with the emotion replacement vector through the variational adversarial network, which means that the sound replacement identifier is used as the search index, and the initial emotion vector is replaced with the emotion replacement vector corresponding to the sound replacement identifier through the variational adversarial network. In this way, the singing feature conversion under the current iteration round can be realized.

[0113] Step S406, reconstructing the emotion replacement vector and the training content parameters based on the variational adversarial network to obtain the predicted audio. It should be noted that the emotion replacement vector and the training content parameters are reconstructed using the variational adversarial network to obtain the predicted audio.

[0114] From step S407 to step S408, loss calculation is performed based on the training source audio, the training target audio and the predicted audio to obtain prediction deviation data. After each round of iterative training, the model parameters of the audio processing model are updated based on the prediction deviation data. It should be noted that after obtaining the predicted audio, loss calculation needs to be performed based on the training source audio, the training target audio and the predicted audio to obtain prediction deviation data for measuring the accuracy of the singing feature conversion of the audio processing model. After each round of iterative training, the model parameters of the audio processing model are updated based on the prediction deviation data, so that the audio processing model gradually has the ability to convert the singing features of the source singer to the singing features of the target singer during the iterative training process, until the audio processing model meets the first preset condition in the iterative training, and a pre-trained audio processing model is obtained.

[0115] Through step S401 to step S408, the iterative training of the audio processing model is a process with many iterations. When the number of iterations is slightly less, the ability of the variational adversarial network to replace the initial emotion vector with the emotion replacement vector is crossed, and the singing feature conversion is difficult to achieve the expected effect. As the number of iterations increases, the ability of the variational adversarial network to replace the initial emotion vector with the emotion replacement vector is gradually improved, until the audio processing model in the current iteration has reached the expectations of the pre-training process, and the accuracy of its singing feature conversion has reached a level that can be applied to actual scenarios. At this time, the audio processing model in the current iteration is output to obtain the pre-trained audio processing model, thereby improving the conversion accuracy of the audio processing model in singing feature conversion, which is a more preferred embodiment.

[0116] Reference Figure 5The illustrated embodiment shows a structural block diagram of an optional audio processing model. It should be noted that after the training source audio is input into the audio processing model, the training source audio is processed by the vocoder and the music emotion recognition network respectively. Among them, based on the vocoder, the acoustic feature extraction of the training source audio can obtain the training content parameters, and the training content parameters can specifically include the training rhythm parameters and the training spectrum parameters. In addition, the pre-trained music emotion recognition network extracts the emotional information of the training source audio to obtain the emotional initial vector. It should be pointed out that in order to replace the emotional initial vector with the emotional replacement vector, first, it is necessary to use the sound replacement identifier to find the one-to-one corresponding training target audio through query, and then use the pre-trained music emotion recognition network to extract the emotional information of the training target audio to obtain the emotional replacement vector, and then the emotional initial vector can be replaced with the emotional replacement vector. Further, after obtaining the training rhythm parameters and the training spectrum parameters, and replacing the emotional initial vector with the emotional replacement vector, the training rhythm parameters, the training spectrum parameters and the emotional replacement vector can be respectively input into the variational adversarial network. It should be noted that the variational adversarial network includes an encoder, a decoder and a discriminator. In some more specific embodiments, the variational adversarial network may include an encoder, a decoder, and a discriminator. It should be noted that the encoder is used to convert the input data into a set of encoding vectors, which can be implemented by a convolutional neural network, a recurrent neural network, or other neural networks. The main function of the encoder is to convert the input data into a low-dimensional dense representation, thereby extracting important features of the data. In some embodiments, the output of the encoder is also called a "hidden vector" or "embedded vector"; the decoder is used to convert the output of the encoder back to the original data, which can be implemented by a deconvolutional neural network, a recurrent neural network, or other neural networks. The main function of the decoder is to restore the input data according to the encoding vector, and the output of the decoder is also called a "reconstructed vector"; the discriminator can be a binary classifier, which is used to determine whether the generated spectral features are real. During pre-training, the generated spectrum and the real spectrum need to be input into the discriminator for discriminative training. During discriminative training, it is necessary to calculate the loss based on the training source audio, the training target audio, and the predicted audio to obtain the predicted deviation data, and then update the model parameters of the audio processing model based on the predicted deviation data.

[0117] Figure 6 According to some embodiments of the present application, the training content parameters include training rhythm parameters and training spectrum parameters, and step S406 may include, but is not limited to, the following steps S601 to S602.

[0118] Step S601, performing language feature analysis on training spectrum parameters to obtain training latent vectors, where the training latent vectors are used to represent the sentence content of the training source audio;

[0119] Step S602, integrating and concatenating the emotion replacement vector, the training prosody parameters and the training latent vector to obtain predicted audio.

[0120] In step S601 to step S602 of some embodiments, the training spectrum parameters are first analyzed for language features to obtain a training latent vector, which is used to represent the sentence content of the training source audio. Then, the emotion replacement vector, the training prosody parameters and the training latent vector are integrated and spliced ​​to obtain the predicted audio. It should be noted that

[0121] The training prosodic parameters are used to characterize the prosodic information and structural information of the singing voice, such as the fundamental frequency F0; the training spectral parameters are used to characterize the timbre and speech content of the singing voice, such as the spectral parameter SP. It should be pointed out that since the training spectral parameters reflect the timbre and speech content of the singing voice in the training source audio, and the timbre part is to be screened out, it is necessary to extract the sentence content used to characterize the training source audio through steps S601 to S602 to obtain the training latent vector, and then further integrate and splice the training latent vector, the training prosodic parameters and the emotion replacement vector to finally obtain the predicted audio. It is possible to convert the timbre of the human voice while retaining the prosodic information and structural information in the source audio, thereby improving the accuracy and smoothness of the audio processing model in the conversion of singing voice features.

[0122] Reference Figure 7 According to some embodiments of the present application, the prediction deviation data includes a loss function value, and step S407 may include, but is not limited to, the following steps S701 to S704.

[0123] Step S701, performing a first loss calculation based on the training source audio and the training latent vector to obtain a coding loss element corresponding to the music emotion recognition network;

[0124] Step S702, performing a second loss calculation based on the training source audio, the training target audio, and the predicted audio to obtain a decoding loss element corresponding to the variational adversarial network;

[0125] Step S703, performing loss construction based on the encoding loss element and the decoding loss element to obtain a prediction loss function;

[0126] Step S704, based on the prediction loss function, loss calculation is performed on the training source audio, the training target audio and the prediction audio to obtain a loss function value.

[0127] In step S701 of some embodiments, a first loss calculation is performed based on the training source audio and the training latent vector to obtain a coding loss element corresponding to the music emotion recognition network. In some more specific embodiments, the spectral feature corresponding to the training source audio is represented by X s , the training latent vector is represented as Z, then the prior distribution of the training latent vector can be represented as pθ (Z), the posterior estimate corresponding to the predicted audio can be expressed as q φ (Z|X s ), further, a first loss calculation is performed based on the training source audio and the training latent vector to obtain the encoding loss element corresponding to the music emotion recognition network, where the encoding loss element can be specifically expressed as:

[0128] -D KL (q φ (Z|X s )||p θ (Z)

[0129] Among them, D KL Refers to the calculation of relative entropy, that is, the calculation of KL divergence, where φ refers to the model parameters of the encoder in the variational adversarial network, and θ refers to the model parameters of the decoder in the variational adversarial network.

[0130] In step S702 of some embodiments, a second loss calculation is performed based on the training source audio, the training target audio and the predicted audio to obtain a decoding loss element corresponding to the variational adversarial network. It should be noted that in some more specific embodiments, the sound replacement identifier corresponding to the training target audio can be expressed as y, and the model likelihood of the variational adversarial network can be expressed as p θ (X s |Z,Y t ), further, a second loss calculation is performed based on the training source audio, the training target audio and the predicted audio to obtain a decoding loss element corresponding to the variational adversarial network, where the decoding loss element can be specifically expressed as:

[0131]

[0132] in, Indicates the expected value of statistical significance.

[0133] In step S703 of some embodiments, loss construction is performed based on the encoding loss element and the decoding loss element to obtain a prediction loss function. It should be noted that loss construction is performed based on the encoding loss element and the decoding loss element to obtain a prediction loss function, which can be specifically expressed as:

[0134]

[0135] In step S704 of some embodiments, the loss of the training source audio, the training target audio and the predicted audio is calculated based on the prediction loss function to obtain a loss function value. It should be noted that the loss function value is used to reflect the loss of the variational encoder in the current iteration round. Since the music emotion recognition network has been pre-trained, the loss of the variational encoder in the current iteration round can also be regarded as the loss of the audio processing model in the current iteration round.

[0136] Through steps S701 to S704, the loss of the audio processing model in the current iteration round in each round of iterative training can be calculated, so that after each round of iterative training, the model parameters of the audio processing model can be updated based on the predicted deviation data, thereby pre-training the audio processing model information, so that it gradually has the ability to convert the singing features of the source singer into the singing features of the target singer, thereby improving the accuracy of the singing feature conversion of the present application.

[0137] Reference Figure 8 According to some embodiments of the present application, before step S704, the singing feature conversion method of the present application may also include, but not limited to, the following steps S801 to S802.

[0138] Step S801, performing a third loss calculation based on the training source audio, the training target audio, and the predicted audio to obtain a discriminative loss element corresponding to the variational adversarial network;

[0139] Step S802 , integrating the discrimination loss element into the prediction loss function based on a preset weight factor.

[0140] In step S801 to step S802 of some embodiments, a third loss calculation is first performed based on the training source audio, the training target audio and the predicted audio to obtain a discriminant loss element corresponding to the variational adversarial network, and then the discriminant loss element is integrated into the prediction loss function based on a preset weight factor. It should be noted that the spectral feature corresponding to the training target audio is represented by X t , then the third loss is calculated based on the training source audio, the training target audio and the predicted audio to obtain the discriminant loss element corresponding to the variational adversarial network, where the discriminant loss element can be expressed as:

[0141]

[0142] Among them, the discriminator analytical formula can be expressed as D ψ (.), the generator analysis can be expressed as G θ (.).

[0143] Furthermore, based on the preset weight factor α, the discriminant loss element is integrated into the prediction loss function, which can be expressed as:

[0144]

[0145] Through the embodiment shown in step S801 to step S802, by integrating the discriminant loss element into the prediction loss function, the training efficiency of the discriminator in the variational adversarial network can be further improved.

[0146] Step S304, when the audio processing model meets the first preset condition in the iterative training, a pre-trained audio processing model is obtained. It should be noted that when the audio processing model meets the first preset condition in the iterative training, a pre-trained audio processing model can be obtained. It should be pointed out that the audio processing model meets the first preset condition in the iterative training, which means that the audio processing model under the current iteration round has met the expectations of the pre-training process, and the accuracy of its singing feature conversion has reached a level that can be applied to actual scenarios. At this time, the audio processing model under the current iteration round is output to obtain the pre-trained audio processing model.

[0147] Fig. 9 The electronic device 900 provided in the embodiment of the present application is shown. The electronic device 900 includes: a processor 901, a memory 902, and a computer program stored in the memory 902 and executable on the processor 901, and the computer program is used to execute the above-mentioned singing feature conversion method when it is executed.

[0148] The processor 901 and the memory 902 may be connected via a bus or other means.

[0149] The memory 902 is a non-transient computer-readable storage medium that can be used to store non-transient software programs and non-transient computer executable programs, such as the singing voice feature conversion method described in the embodiment of the present application. The processor 901 implements the above-mentioned singing voice feature conversion method by running the non-transient software program and instructions stored in the memory 902.

[0150] The memory 902 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function. The data storage area may store and execute the above-mentioned singing feature conversion method. In addition, the memory 902 may include a high-speed random access memory 902, and may also include a non-transient memory 902, such as at least one storage device storage device, a flash memory device or other non-transient solid-state storage device. In some embodiments, the memory 902 may optionally include a memory 902 remotely arranged relative to the processor 901, and these remote memories 902 may be connected to the electronic device 900 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0151] The non-transient software program and instructions required to implement the above-mentioned singing voice feature conversion method are stored in the memory 902. When executed by one or more processors 901, the above-mentioned singing voice feature conversion method is executed, for example, Figure 1 Steps S101 to S104 of the method, Figure 2 Steps S201 to S205 of the method, Figure 3Steps S301 to S305 of the method, Figure 4 Steps S401 to S403 of the method, Figure 5 Steps S501 to S503 of the method, Figure 6 Steps S601 to S603 of the method, Figure 7 Steps S701 to S703 of the method, Figure 8 Method steps S801 to S802.

[0152] The embodiment of the present application also provides a computer-readable storage medium storing computer-executable instructions, and the computer-executable instructions are used to execute the above-mentioned singing feature conversion method.

[0153] In one embodiment, the computer readable storage medium stores computer executable instructions, which are executed by one or more control processors, for example, Figure 1 Steps S101 to S104 of the method, Figure 2 Steps S201 to S205 of the method, Figure 3 Steps S301 to S305 of the method, Figure 4 Steps S401 to S403 of the method, Figure 5 Steps S501 to S503 of the method, Figure 6 Steps S601 to S603 of the method, Figure 7 Steps S701 to S703 of the method, Figure 8 Method steps S801 to S802.

[0154] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, storage device storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. In addition, it is well known to those skilled in the art that communication media generally include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium. It should also be understood that the various implementations provided in the embodiments of the present application can be arbitrarily combined to achieve different technical effects.

[0156] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above implementation mode. Technical personnel familiar with the field can also make various equivalent modifications or substitutions under the shared conditions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A singing feature conversion method, comprising: Obtaining source audio and target sound identification, wherein the source audio includes an audio segment of a source singer; Inputting the source audio and the target sound identifier into a pre-trained audio processing model, wherein the audio processing model includes a vocoder, a music emotion recognition network, and a variational adversarial network; Extracting acoustic features of the source audio based on the vocoder to obtain content feature parameters; Extracting emotional information from the source audio based on the music emotion recognition network to obtain a source emotion feature vector; Based on the target sound identifier, replacing the source emotion feature vector with a target emotion feature vector through the variational adversarial network; The target emotion feature vector and the content feature parameters are reconstructed based on the variational adversarial network to obtain the target audio after the singing feature conversion.

2. According to the method of claim 1, before inputting the source audio and the target sound identifier into the pre-trained audio processing model, the method further comprises pre-training the audio processing model, specifically comprising: Acquire a training data set, the training data set comprising a first number of training audios, each of the training audios being configured with a one-to-one corresponding training sound identifier; The first number of training audios are grouped to obtain a second number of training audio groups, each of which includes the training audios of different singers singing the same audio segment; Inputting a second number of the training audio groups into the original audio processing model, and iteratively training the audio processing model based on the training audios and the training sound identifiers; When the audio processing model meets the first preset condition in iterative training, the pre-trained audio processing model is obtained.

3. The method according to claim 2, wherein the inputting the second number of the training audio groups into the original audio processing model and iteratively training the audio processing model based on the training audio and the training sound identifier comprises: In each round of iterative training, an iterative audio group is selected from the second number of training audio groups, and a training source audio and a training target audio are selected from the iterative audio group; Extracting acoustic features of the training source audio based on the vocoder to obtain training content parameters; Extracting emotion information from the training source audio based on the pre-trained music emotion recognition network to obtain an initial emotion vector; Extracting emotion information from the training target audio based on the pre-trained music emotion recognition network to obtain an emotion replacement vector; Determine the training sound identifier of the training target audio configuration as a sound replacement identifier, and based on the sound replacement identifier, replace the emotion initial vector with the emotion replacement vector through the variational adversarial network; Reconstructing the emotion replacement vector and the training content parameters based on the variational adversarial network to obtain predicted audio; Perform loss calculation based on the training source audio, the training target audio and the predicted audio to obtain prediction deviation data; After each round of iterative training, the model parameters of the audio processing model are updated based on the prediction deviation data.

4. According to the method of claim 3, the training content parameters include training prosody parameters and training spectrum parameters, and the reconstructing the emotion replacement vector and the training content parameters based on the variational adversarial network to obtain the predicted audio comprises: Performing language feature analysis on the training spectrum parameters to obtain a training latent vector, wherein the training latent vector is used to represent the sentence content of the training source audio; The emotion replacement vector, the training prosody parameter and the training latent vector are integrated and spliced ​​to obtain the predicted audio.

5. The method according to claim 4, wherein the prediction deviation data comprises a loss function value, and the loss calculation based on the training source audio, the training target audio and the predicted audio to obtain the prediction deviation data comprises: Perform a first loss calculation based on the training source audio and the training latent vector to obtain a coding loss element corresponding to the music emotion recognition network; Performing a second loss calculation based on the training source audio, the training target audio, and the predicted audio to obtain a decoding loss element corresponding to the variational adversarial network; Perform loss construction based on the encoding loss element and the decoding loss element to obtain a prediction loss function; Based on the prediction loss function, loss calculation is performed on the training source audio, the training target audio and the predicted audio to obtain the loss function value.

6. The method according to claim 5, before performing loss calculation on the training source audio, the training target audio and the predicted audio based on the prediction loss function to obtain the loss function value, the method further comprises: Performing a third loss calculation based on the training source audio, the training target audio, and the predicted audio to obtain a discriminative loss element corresponding to the variational adversarial network; The discriminant loss element is integrated into the prediction loss function based on a preset weight factor.

7. The method according to any one of claims 1 to 6, wherein extracting acoustic features of the source audio based on the vocoder to obtain content feature parameters comprises: Extracting fundamental frequency features of the source audio based on the vocoder to obtain source prosody feature parameters, wherein the source prosody feature parameters are used to characterize the prosody content of the source audio; Extracting spectrum characteristics of the source audio based on the vocoder to obtain spectrum characteristic parameters; The source rhythm feature parameter and the spectrum feature parameter are integrated to obtain the content feature parameter.

8. The method according to claim 7, wherein the target emotion feature vector and the content feature parameter are reconstructed based on the variational adversarial network to obtain the target audio after the singing feature conversion, comprising: Acquire the frequency spectrum characteristic parameter and the source rhythm characteristic parameter from the content characteristic parameter; Performing language feature analysis on the frequency spectrum feature parameters to obtain a sentence latent vector, wherein the sentence latent vector is used to represent the sentence content of the source audio; The target emotion feature vector, the source prosody feature parameter and the sentence latent vector are integrated and spliced ​​to obtain the target audio.

9. An electronic device, comprising: A memory and a processor, wherein the memory stores a computer program, and the processor implements the singing feature conversion method as described in any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the singing feature conversion method as described in any one of claims 1 to 8.

Images