Training method and system and application method and system of voice conversion model based on data distillation

Abstract

The invention discloses a training method and system and an application method and system of a voice conversion model based on data distillation. The method comprises the following steps: establishing a voice training data set, and training an original voice conversion model by using the voice training data set to obtain an initial voice conversion model; establishing a voice evaluation data set, and inputting the voice evaluation data set into the initial voice conversion model to obtain conversion voice data of each corresponding speaker; calculating a voice conversion score of each speaker; judging whether the voice conversion score of each speaker reaches a preset condition or not; if not, removing the voice data corresponding to the speaker from the voice training data set, and taking the remaining voice data as a voice database; repeatedly training; and taking the initial voice conversion model and the voice database as a trained voice conversion model and a final voice database respectively. The voice conversion model and the voice database obtained through training of the method can support conversion of various timbres, and the voice conversion effect is good.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and more specifically, to a training method and system for a speech conversion model based on data distillation and an application method and system. Background Art

[0002] Some existing methods for beautifying voices are mainly based on tuning methods of signal processing, such as de-essing, equalization, and pitch adjustment to make the voice more appealing without changing the timbre. Although these methods can beautify the voice to a certain extent, the specific sound debugging process through signal processing is cumbersome and the effect is poor. At the same time, the voice conversion model can only support a single or a small number of timbres, and it is difficult to achieve voiceprint matching, which requires a large number of timbres to be integrated in the same model. Since everyone speaks with an accent and other personal factors, a few specific timbre selections cannot guarantee that everyone's timbre conversion needs are met, nor can they guarantee good conversion quality.

[0003] The prior art discloses a method and device for voice conversion. By receiving a voice conversion instruction, the voice conversion instruction includes first voice data output by an original character and a voice conversion model identifier of a target character; loading a voice conversion model corresponding to the voice conversion model identifier of the target character, the voice conversion model is trained by a voice sequence corresponding to the target character and third voice data of at least one original training character; and according to the voice conversion instruction, the first voice data output by the original character is converted into second voice data output by the target character through the voice conversion model, so that the voice output by the user can be converted into the voice output by any target character desired by the user, meeting the voice conversion requirements of different users. The voice conversion model only converts the voices of several specific characters and adjusts according to the actual voice of the user, so the conversion effect is poor for some users. Summary of the invention

[0004] In view of the defects of low efficiency and poor effect of speech conversion in the prior art, the present invention provides a training method and system of a speech conversion model based on data distillation and an application method and system, which can support multiple timbre conversions and have good speech conversion effect.

[0005] The primary purpose of the present invention is to solve the above technical problems. The technical solution of the present invention is as follows: The training method of the speech conversion model based on data distillation includes: S101: Acquire speech data of multiple speakers to form a speech training data set; S102: training the original speech conversion model using the speech training data set until a first preset round is reached to obtain an initial speech conversion model; S103: extracting speech data of each speaker for a preset time length to obtain sample speech data of each speaker and form a speech evaluation data set; S104: inputting the speech evaluation data set into an initial speech conversion model to obtain corresponding converted speech data of each speaker; S105: Calculating a speech conversion score of each speaker according to the sample speech data and the converted speech data corresponding to each speaker; S106: Determine whether the speech conversion score of each speaker meets the preset condition; if not, remove the speech data corresponding to the speaker from the speech training data set, and use the remaining speech data as the speech database; S107: using the initial speech conversion model as a new original speech conversion model and the speech database as a new speech training data set, and repeating steps S102 to S106 until a second preset round is reached; S108: The initial speech conversion model and speech database corresponding to the second preset round are used as the trained speech conversion model and the final speech database respectively.

[0006] Furthermore, the speech data of the speaker includes: audio and speaker tag.

[0007] Further, the speech conversion model includes: a first convolution layer, a second convolution layer, a third convolution layer, a first DIT layer, a second DIT layer, a third DIT layer, a fourth DIT layer, a linear layer, a numerical integration layer, a first addition point, a speech data generation unit, a speech content extraction unit, a pitch extraction unit, and a speaker tag encoding unit; The audio input is the input end of the pitch extraction unit, the input end of the speech content extraction unit, the speaker tag is input into the input end of the speaker tag encoding unit; and the output end of the speech data generation unit outputs the converted speech data.

[0008] The output end of the pitch extraction unit is connected to the input end of the first convolutional layer, the output end of the speech content extraction unit is connected to the input end of the second convolutional layer, the output end of the speaker tag encoding unit is connected to the input end of the linear layer, the output end of the first convolutional layer, the output end of the second convolutional layer, and the output end of the linear layer are connected to the input end of the first addition point, the output end of the first addition point and the output end of the numerical integration layer are connected to the input end of the first DIT layer, the output end of the first DIT layer and the output end of the linear layer are connected to the input end of the second DIT layer, the output end of the second DIT layer and the output end of the linear layer are connected to the input end of the third DIT layer, the output end of the third DIT layer and the output end of the first convolutional layer are connected to the input end of the fourth DIT layer, the output end of the fourth DIT layer is connected to the input end of the third convolutional layer, and the output end of the third convolutional layer is connected to the input end of the speech data generation unit.

[0009] Furthermore, the loss function of the training speech conversion model is as follows:

[0010] exp represents exponential function; t represents time step; i represents sequence number; Represents the output of the third convolutional layer when the speech data of the i-th speaker is used as the input of the speech conversion model; represents the speech data of the i-th speaker; N represents the total number of speech data in the speech training data set; represents a randomly initialized 0-1 distribution.

[0011] Further, the voice conversion score of each speaker calculated in step S105 includes a voice quality score; The formula for the speech quality score is as follows:

[0012] i represents the sequence number, L represents the length of the voice data, and n represents the sequence number of the frame in the voice data. represents the converted speech data of the nth frame of the ith speaker, Represents the speech data of the nth frame of the i-th speaker.

[0013] Further, the voice conversion score calculated for each speaker in step S105 includes a self-similarity score; The formula for the self-similarity score is as follows:

[0014] i represents the sequence number, represents the converted speech data of the i-th speaker, represents the speech data of the i-th speaker, and Speaker represents the timbre feature extraction model.

[0015] Further, the voice conversion score calculated for each speaker in step S105 includes a test sample similarity score; The formula for the test sample similarity score is as follows:

[0016] i represents the serial number; Represents the voice data of a broadcaster; Indicates that the announcer's voice data When used as the speech data of the i-th speaker, it is the converted speech data; Speaker represents the timbre feature extraction model.

[0017] A training system for speech conversion models based on data distillation, including: Dataset generation module: obtains speech data from multiple speakers and builds a speech training dataset; The first training module: training the original speech conversion model using the speech training data set until a first preset round is reached to obtain an initial speech conversion model; Evaluation data extraction module: extracts the speech data of each speaker at a preset time length, obtains the sample speech data of each speaker, and forms a speech evaluation data set; Speech reasoning module: inputs the speech evaluation data set into the initial speech conversion model to obtain the corresponding converted speech data of each speaker; Evaluation module: Calculates the speech conversion score of each speaker based on the sample speech data and converted speech data corresponding to each speaker; Data distillation module: determines whether the speech conversion score of each speaker meets the preset conditions; if not, the speech data corresponding to the speaker is removed from the speech training data set, and the remaining speech data is used as the speech database; Second training module: using the initial speech conversion model as a new original speech conversion model and the speech database as a new speech training data set, and returning to the first training module until a second preset round is reached; Final model generation module: the initial speech conversion model and speech database corresponding to the second preset round are used as the trained speech conversion model and final speech database respectively.

[0018] The application method of the speech conversion model based on data distillation includes: S201: extracting the voice data of each speaker and the timbre representation of the voice data to be converted in the final voice database, and obtaining the timbre representation corresponding to each speaker and the timbre feature of the voice data to be converted; S202: Calculate the cosine similarity between the timbre feature of the speech data to be converted and the timbre representation corresponding to each speaker, and take the speaker corresponding to the timbre representation with the highest cosine similarity as the conversion target speaker; S203: Inputting the trained speech conversion model into the speech data of the conversion target person and the speech data to be converted to obtain the converted target speech data.

[0019] The application system of the speech conversion model based on data distillation includes: The timbre representation extraction module extracts the timbre representation of the speech data of each speaker and the speech data to be converted in the final speech database, and obtains the timbre representation corresponding to each speaker and the timbre characteristics of the speech data to be converted; Similarity matching module: calculates the cosine similarity between the timbre characteristics of the speech data to be converted and the timbre representation corresponding to each speaker, and takes the speaker corresponding to the timbre representation with the highest cosine similarity as the conversion target person; Target voice data generation module: input the trained voice conversion model into the voice data of the conversion target person and the voice data to be converted to obtain the converted target voice data.

[0020] Compared with the prior art, the present invention has the following beneficial effects: The present invention obtains speech data of multiple speakers to form a speech training data set; trains an original speech conversion model using the speech training data set until a first preset round is reached to obtain an initial speech conversion model; extracts speech data of each speaker for a preset time length to obtain sample speech data of each speaker to form a speech evaluation data set; inputs the speech evaluation data set into the initial speech conversion model to obtain corresponding converted speech data of each speaker; calculates a speech conversion score of each speaker according to the sample speech data and converted speech data corresponding to each speaker; determines whether the speech conversion score of each speaker meets a preset condition; if not, removes the speech data corresponding to the speaker from the speech training data set, and uses the remaining speech data as a speech database; uses the initial speech conversion model as a new original speech conversion model, and uses the speech database as a new speech training data set, and repeats the training steps until a second preset round is reached; uses the initial speech conversion model and the speech database corresponding to the second preset round as the trained speech conversion model and the final speech database respectively; and further, the trained speech conversion model and the final speech database can support multiple timbre conversions, and the speech conversion effect is good.

[0021] At the same time, the voice data of each speaker and the timbre representation of the voice data to be converted in the final voice database are extracted to obtain the timbre representation corresponding to each speaker and the timbre feature of the voice data to be converted; the cosine similarity between the timbre feature of the voice data to be converted and the timbre representation corresponding to each speaker is calculated respectively, and the speaker corresponding to the timbre representation with the highest cosine similarity is taken as the conversion target person; the voice data of the conversion target person and the voice data to be converted are input into the trained voice conversion model to obtain the converted target voice data. Thus, the voice conversion effect is good. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of the training method of the speech conversion model based on data distillation provided in Example 1.

[0023] Figure 2 This is a structural diagram of the speech conversion model provided in Example 1.

[0024] Figure 3 A structural diagram of the training system for the speech conversion model based on data distillation provided in Example 1.

[0025] Figure 4 A flowchart of the application method of the speech conversion model based on data distillation provided in Example 1.

[0026] Figure 5 A structural diagram of the application system of the speech conversion model based on data distillation provided in Example 1.

[0027] Figure 6 This is the mel spectrum diagram of the speech data to be converted provided in Example 1.

[0028] Figure 7 This is the mel spectrum diagram of the converted target speech data provided in Example 1. DETAILED DESCRIPTION

[0029] The drawings are for illustrative purposes only and should not be construed as limiting the present patent; In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0030] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0031] Example 1 like Figure 1 As shown, the training method of the speech conversion model based on data distillation includes: S101: Acquire speech data of multiple speakers to form a speech training data set; S102: training the original speech conversion model using the speech training data set until a first preset round is reached to obtain an initial speech conversion model; S103: extracting speech data of each speaker for a preset time length to obtain sample speech data of each speaker and form a speech evaluation data set; S104: inputting the speech evaluation data set into an initial speech conversion model to obtain corresponding converted speech data of each speaker; S105: Calculating a speech conversion score of each speaker according to the sample speech data and the converted speech data corresponding to each speaker; S106: Determine whether the speech conversion score of each speaker meets the preset condition; if not, remove the speech data corresponding to the speaker from the speech training data set, and use the remaining speech data as the speech database; S107: using the initial speech conversion model as a new original speech conversion model and the speech database as a new speech training data set, and repeating steps S102 to S106 until a second preset round is reached; S108: The initial speech conversion model and speech database corresponding to the second preset round are used as the trained speech conversion model and the final speech database respectively.

[0032] In a specific embodiment, the second preset rounds are 3 rounds, and the first preset rounds are 10 rounds.

[0033] In a specific embodiment, the preset time length in step S103 is 15 seconds, and the extraction is to randomly extract an audio of the speaker.

[0034] Furthermore, the speech data of the speaker includes: audio and speaker tag.

[0035] Furthermore, if Figure 2 As shown, the speech conversion model includes: a first convolution layer, a second convolution layer, a third convolution layer, a first DIT layer, a second DIT layer, a third DIT layer, a fourth DIT layer, a linear layer, a numerical integration layer, a first addition point, a speech data generation unit, a speech content extraction unit, a pitch extraction unit, and a speaker tag encoding unit; The audio input is the input end of the pitch extraction unit, the input end of the speech content extraction unit, the speaker tag is input into the input end of the speaker tag encoding unit; and the output end of the speech data generation unit outputs the converted speech data.

[0036] The output end of the pitch extraction unit is connected to the input end of the first convolutional layer, the output end of the speech content extraction unit is connected to the input end of the second convolutional layer, the output end of the speaker tag encoding unit is connected to the input end of the linear layer, the output end of the first convolutional layer, the output end of the second convolutional layer, and the output end of the linear layer are connected to the input end of the first addition point, the output end of the first addition point and the output end of the numerical integration layer are connected to the input end of the first DIT layer, the output end of the first DIT layer and the output end of the linear layer are connected to the input end of the second DIT layer, the output end of the second DIT layer and the output end of the linear layer are connected to the input end of the third DIT layer, the output end of the third DIT layer and the output end of the first convolutional layer are connected to the input end of the fourth DIT layer, the output end of the fourth DIT layer is connected to the input end of the third convolutional layer, and the output end of the third convolutional layer is connected to the input end of the speech data generation unit.

[0037] In a specific embodiment, during the training process, 30% of the data in the output results of the speech data generation unit will be replaced with random numbers to improve the robustness of the speech conversion model.

[0038] In a specific embodiment, the speech content extraction unit is a Whisper model or a Sensevoice model, the pitch extraction unit adopts an RMVPE model, the speaker tag encoding unit adopts a word2vec method, and the speech data generation unit adopts a BIGVGAN model.

[0039] It should be noted that the time step is a constant value and is input into the numerical integration layer.

[0040] Furthermore, the loss function of the training speech conversion model is as follows:

[0041] exp represents exponential function; t represents time step; i represents sequence number; Represents the output of the third convolutional layer when the speech data of the i-th speaker is used as the input of the speech conversion model; represents the speech data of the i-th speaker; N represents the total number of speech data in the speech training data set; represents a randomly initialized 0-1 distribution.

[0042] It should be noted that , Relationship satisfaction , t represents the time step.

[0043] In a specific embodiment, the relationship between the speech data distribution, the time step and the expressiveness parameter is as follows:

[0044] in, Indicates j The expressiveness parameter is in the time step t The distribution of time, Indicates j The target speech data distribution of expressive parameters, Indicates the j The distribution of input speech data with expressive parameters.

[0045] The speech data in the present invention is in a distributed form.

[0046] Further, the voice conversion score of each speaker calculated in step S105 includes a voice quality score; The formula for the speech quality score is as follows:

[0047] i represents the sequence number, L represents the length of the voice data, and n represents the sequence number of the frame in the voice data. represents the converted speech data of the nth frame of the ith speaker, Represents the speech data of the nth frame of the i-th speaker.

[0048] In a specific embodiment, when the voice quality score is greater than 2.5, it means that the voice quality score meets the preset condition.

[0049] Further, the voice conversion score calculated for each speaker in step S105 includes a self-similarity score; The formula for the self-similarity score is as follows:

[0050] i represents the sequence number, represents the converted speech data of the i-th speaker, represents the speech data of the i-th speaker, and Speaker represents the timbre feature extraction model.

[0051] In a specific embodiment, when the self-similarity score is greater than 0.8, it means that the self-similarity score meets the preset condition.

[0052] Further, the voice conversion score calculated for each speaker in step S105 includes a test sample similarity score; The formula for the test sample similarity score is as follows:

[0053] i represents the serial number; Represents the voice data of a broadcaster; Indicates that the announcer's voice data When used as the speech data of the i-th speaker, it is the converted speech data; Speaker represents the timbre feature extraction model.

[0054] In a specific embodiment, 15 seconds of speech data of two announcers, one male and one female, are used to calculate sample similarity scores respectively. When both are greater than 0.6, it means that the test sample similarity scores meet the preset conditions.

[0055] It should be noted that the voice conversion score calculated for each speaker in step S105 includes one or more scores of: voice quality score, self-similarity score, and test sample similarity score.

[0056] In a specific embodiment, in step S101, the speech data of multiple speakers are generated using the text-to-speech generation model VITS based on the data set VCTK, thereby expanding the data content and enhancing the quality and stability of speech generation.

[0057] like Figure 3 As shown, the training system of the speech conversion model based on data distillation includes: Dataset generation module: obtains speech data from multiple speakers and builds a speech training dataset; The first training module: training the original speech conversion model using the speech training data set until a first preset round is reached to obtain an initial speech conversion model; Evaluation data extraction module: extracts the speech data of each speaker at a preset time length, obtains the sample speech data of each speaker, and forms a speech evaluation data set; Speech reasoning module: inputs the speech evaluation data set into the initial speech conversion model to obtain the corresponding converted speech data of each speaker; Evaluation module: Calculates the speech conversion score of each speaker based on the sample speech data and converted speech data corresponding to each speaker; Data distillation module: determines whether the speech conversion score of each speaker meets the preset conditions; if not, the speech data corresponding to the speaker is removed from the speech training data set, and the remaining speech data is used as the speech database; Second training module: using the initial speech conversion model as a new original speech conversion model and the speech database as a new speech training data set, and returning to the first training module until a second preset round is reached; Final model generation module: the initial speech conversion model and speech database corresponding to the second preset round are used as the trained speech conversion model and final speech database respectively.

[0058] like Figure 4 As shown, the application method of the speech conversion model based on data distillation includes: S201: extracting the voice data of each speaker and the timbre representation of the voice data to be converted in the final voice database, and obtaining the timbre representation corresponding to each speaker and the timbre feature of the voice data to be converted; S202: Calculate the cosine similarity between the timbre feature of the speech data to be converted and the timbre representation corresponding to each speaker, and take the speaker corresponding to the timbre representation with the highest cosine similarity as the conversion target speaker; S203: Inputting the trained speech conversion model into the speech data of the target person to be converted and the speech data to be converted, to obtain the converted target speech data.

[0059] In a specific embodiment, the eres2netv2 model is used to extract the timbre representation.

[0060] It should be noted that if there is a situation where there are multiple speech data corresponding to one speaker, the average geometric center of the timbre representation extracted from the speech data of multiple speakers is used as the timbre representation corresponding to the speaker, and the speech data with the smallest distance to the average geometric center is used as the speech data corresponding to the speaker.

[0061] In a specific embodiment, the input pitch of the voice conversion model is the difference between the pitch of the target person and the pitch of the voice data to be converted, so as to further improve the conversion effect.

[0062] like Figure 5 As shown, the application system of the speech conversion model based on data distillation includes: The timbre representation extraction module extracts the timbre representation of the speech data of each speaker and the speech data to be converted in the final speech database, and obtains the timbre representation corresponding to each speaker and the timbre characteristics of the speech data to be converted; Similarity matching module: calculates the cosine similarity between the timbre characteristics of the speech data to be converted and the timbre representation corresponding to each speaker, and takes the speaker corresponding to the timbre representation with the highest cosine similarity as the conversion target person; Target voice data generation module: input the trained voice conversion model into the voice data of the conversion target person and the voice data to be converted to obtain the converted target voice data.

[0063] It should be noted that the purpose of similarity matching is to generate a beautiful voice, which refers to a voice that sounds like a specific user speaking but is more pleasant than the specific user speaking.

[0064] like Figure 6 , Figure 7 As shown in Figure 2, the mel spectrogram of the converted target speech data is clearer than the mel spectrogram of the speech data to be converted, especially Figure 6 , Figure 7 The part in the green box is not achieved by other speech conversion models because other speech conversion models do not directly obtain the target speech data.

[0065] The method of the present invention was applied to a data set of more than 500 people, and the speech quality score PESQ was as low as 2.9, as high as 4.0, and as high as 3.8 on average. The range of PESQ is 0-5, and 2.5 or above indicates acceptable quality to the human ear. Two male and female voices were selected as input test samples, and the timbre similarity was calculated for more than 500 target voice data. The lowest similarity was 0.69, the highest was 0.87, and the average similarity was 0.81. The range of timbre similarity is 0-1, and greater than 0.6 indicates that the timbre is relatively similar.

[0066] The same or similar reference numerals correspond to the same or similar components; The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent; Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for training a speech conversion model based on data distillation, characterized in that: include: S101: Acquire speech data of multiple speakers to form a speech training data set; S102: training the original speech conversion model using the speech training data set until a first preset round is reached to obtain an initial speech conversion model; S103: extracting speech data of each speaker for a preset time length to obtain sample speech data of each speaker and form a speech evaluation data set; S104: inputting the speech evaluation data set into an initial speech conversion model to obtain corresponding converted speech data of each speaker; S105: Calculating a speech conversion score of each speaker according to the sample speech data and the converted speech data corresponding to each speaker; S106: Determine whether the speech conversion score of each speaker meets a preset condition; If not, the speech data corresponding to the speaker is removed from the speech training data set, and the remaining speech data is used as the speech database; S107: using the initial speech conversion model as a new original speech conversion model and the speech database as a new speech training data set, and repeating steps S102 to S106 until a second preset round is reached; S108: The initial speech conversion model and speech database corresponding to the second preset round are used as the trained speech conversion model and the final speech database respectively.

2. The method for training a speech conversion model based on data distillation according to claim 1, characterized in that: The speech data of the speaker includes: audio and speaker tag.

3. The method for training a speech conversion model based on data distillation according to claim 2, characterized in that: The speech conversion model includes: a first convolution layer, a second convolution layer, a third convolution layer, a first DIT layer, a second DIT layer, a third DIT layer, a fourth DIT layer, a linear layer, a numerical integration layer, a first addition point, a speech data generation unit, a speech content extraction unit, a pitch extraction unit, and a speaker tag encoding unit; The audio input is the input end of the pitch extraction unit, the input end of the speech content extraction unit, the speaker tag is input into the input end of the speaker tag encoding unit; and the output end of the speech data generation unit outputs the converted speech data. The output end of the pitch extraction unit is connected to the input end of the first convolutional layer, the output end of the speech content extraction unit is connected to the input end of the second convolutional layer, the output end of the speaker tag encoding unit is connected to the input end of the linear layer, the output end of the first convolutional layer, the output end of the second convolutional layer, and the output end of the linear layer are connected to the input end of the first addition point, the output end of the first addition point and the output end of the numerical integration layer are connected to the input end of the first DIT layer, the output end of the first DIT layer and the output end of the linear layer are connected to the input end of the second DIT layer, the output end of the second DIT layer and the output end of the linear layer are connected to the input end of the third DIT layer, the output end of the third DIT layer and the output end of the first convolutional layer are connected to the input end of the fourth DIT layer, the output end of the fourth DIT layer is connected to the input end of the third convolutional layer, and the output end of the third convolutional layer is connected to the input end of the speech data generation unit.

4. The method for training a speech conversion model based on data distillation according to claim 3, characterized in that: The loss function for training the speech conversion model is as follows: exp represents exponential function; t represents time step; i represents sequence number; Represents the output of the third convolutional layer when the speech data of the i-th speaker is used as the input of the speech conversion model; represents the speech data of the i-th speaker; N represents the total number of speech data in the speech training data set; represents a randomly initialized 0-1 distribution.

5. The method for training a speech conversion model based on data distillation according to claim 1, characterized in that: In step S105, the voice conversion score of each speaker is calculated, including the voice quality score; The formula for the speech quality score is as follows: i represents the sequence number, L represents the length of the voice data, and n represents the sequence number of the frame in the voice data. represents the converted speech data of the nth frame of the ith speaker, Represents the speech data of the nth frame of the i-th speaker.

6. The method for training a speech conversion model based on data distillation according to claim 1, characterized in that: In step S105, the voice conversion score of each speaker is calculated including the self-similarity score; The formula for the self-similarity score is as follows: i represents the sequence number, represents the converted speech data of the i-th speaker, represents the speech data of the i-th speaker, and Speaker represents the timbre feature extraction model.

7. The method for training a speech conversion model based on data distillation according to claim 1, characterized in that: In step S105, the voice conversion score of each speaker is calculated including the test sample similarity score; The formula for the test sample similarity score is as follows: i represents the serial number; Represents the voice data of a broadcaster; Indicates that the announcer's voice data When used as the speech data of the i-th speaker, it is the converted speech data; Speaker represents the timbre feature extraction model.

8. A training system for a speech conversion model based on data distillation, applied to the training method according to any one of claims 1 to 7, characterized in that: include: Dataset generation module: obtains speech data from multiple speakers and builds a speech training dataset; The first training module: training the original speech conversion model using the speech training data set until a first preset round is reached to obtain an initial speech conversion model; Evaluation data extraction module: extracts the speech data of each speaker at a preset time length, obtains the sample speech data of each speaker, and forms a speech evaluation data set; Speech reasoning module: inputs the speech evaluation data set into the initial speech conversion model to obtain the corresponding converted speech data of each speaker; Evaluation module: Calculates the speech conversion score of each speaker based on the sample speech data and converted speech data corresponding to each speaker; Data distillation module: determines whether the speech conversion score of each speaker meets the preset conditions; If not, the speech data corresponding to the speaker is removed from the speech training data set, and the remaining speech data is used as the speech database; Second training module: using the initial speech conversion model as a new original speech conversion model and the speech database as a new speech training data set, and returning to the first training module until a second preset round is reached; Final model generation module: the initial speech conversion model and speech database corresponding to the second preset round are used as the trained speech conversion model and final speech database respectively.

9. An application method of a speech conversion model based on data distillation, characterized in that: include: S201: extracting the voice data of each speaker and the timbre representation of the voice data to be converted in the final voice database as claimed in any one of claims 1 to 7, and obtaining the timbre representation corresponding to each speaker and the timbre feature of the voice data to be converted; S202: Calculate the cosine similarity between the timbre characteristics of the speech data to be converted and the timbre representation corresponding to each speaker, and take the speaker corresponding to the timbre representation with the highest cosine similarity as the conversion target speaker; S203: Input the speech data of the target person to be converted and the speech data to be converted into the speech conversion model trained as described in any one of claims 1 to 7 to obtain the converted target speech data.

10. An application system of a speech conversion model based on data distillation, applied to the application method according to claim 9, characterized in that: include: The timbre representation extraction module extracts the timbre representation of the speech data of each speaker and the speech data to be converted in the final speech database, and obtains the timbre representation corresponding to each speaker and the timbre characteristics of the speech data to be converted; Similarity matching module: calculates the cosine similarity between the timbre characteristics of the speech data to be converted and the timbre representation corresponding to each speaker, and takes the speaker corresponding to the timbre representation with the highest cosine similarity as the conversion target person; Target voice data generation module: input the trained voice conversion model into the voice data of the conversion target person and the voice data to be converted to obtain the converted target voice data.

Images