Training method and system of speech conversion model based on data distillation, and application method and system thereof
By using a speech conversion model training method based on data distillation, speech data from multiple speakers is acquired, data that does not meet the scoring criteria is removed, and the model is repeatedly trained to generate the final model. This solves the problems of poor timbre integration and conversion effect in existing technologies, and achieves support for multiple timbre conversions and high-quality speech conversion.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JINAN UNIVERSITY
- Filing Date
- 2025-01-03
- Publication Date
- 2026-06-09
Smart Images
Figure CN119673202B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence, and more specifically, to a training method, system, and application method / system for a speech conversion model based on data distillation. Background Technology
[0002] Existing methods for enhancing voice primarily rely on signal processing techniques, such as deschisis, equalization, and pitch shifting, to make speech more engaging without altering timbre. While these methods can enhance speech to some extent, the specific voice tuning process through signal processing is cumbersome and yields poor results. Furthermore, speech conversion models can only support single or limited timbres, making it difficult to achieve voiceprint matching that requires integrating a massive number of timbres into a single model. Since everyone's speech is influenced by personal factors such as accents, selecting only a few timbres cannot guarantee that everyone's voice conversion needs will be met, nor can it guarantee good conversion quality.
[0003] Existing technology discloses a voice conversion method and apparatus. The method involves receiving a voice conversion instruction, which includes first speech data output by an original character and a voice conversion model identifier for a target character; loading a voice conversion model corresponding to the target character's voice conversion model identifier, the voice conversion model being trained from a voice sequence corresponding to the target character and third speech data from at least one original training character; and, according to the voice conversion instruction, converting the first speech data output by the original character into second speech data output by the target character using the voice conversion model. This allows the user's output voice to be converted into any desired voice from any target character, satisfying the voice conversion needs of different users. However, this voice conversion model only converts the voices of a few specific characters and adjusts them according to the user's actual voice, resulting in suboptimal conversion effects for some users. Summary of the Invention
[0004] This invention addresses the shortcomings of existing speech conversion technologies, such as low efficiency and poor performance, by providing a training method, system, and application method / system for a speech conversion model based on data distillation. This model supports multiple timbre conversions and delivers excellent speech conversion results.
[0005] The primary objective of this invention is to solve the aforementioned technical problems. The technical solution of this invention is as follows:
[0006] Training methods for speech conversion models based on data distillation include:
[0007] S101: Acquire speech data from multiple speakers and construct a speech training dataset;
[0008] S102: Train the original speech conversion model using the speech training dataset until the first preset round is reached to obtain the initial speech conversion model;
[0009] S103: Extract speech data from each speaker for a preset time length to obtain sample speech data for each speaker and build a speech evaluation dataset;
[0010] S104: Input the speech evaluation dataset into the initial speech conversion model to obtain the converted speech data for each speaker;
[0011] S105: Calculate the speech conversion score for each speaker based on the sample speech data and converted speech data corresponding to each speaker;
[0012] S106: Determine whether the speech conversion score of each speaker meets the preset conditions; if not, remove the speech data corresponding to that speaker from the speech training dataset, and use the remaining speech data as the speech database;
[0013] S107: Use the initial speech conversion model as the new original speech conversion model, and the speech database as the new speech training dataset. Repeat steps S102 to S106 until the second preset round is reached.
[0014] S108: The initial speech conversion model and speech database corresponding to the second preset round are used as the trained speech conversion model and final speech database, respectively.
[0015] Furthermore, the speaker's voice data includes: audio and speaker tags.
[0016] Furthermore, the speech conversion model includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, a first DIT layer, a second DIT layer, a third DIT layer, a fourth DIT layer, a linear layer, a numerical integration layer, a first addition point, a speech data generation unit, a speech content extraction unit, a pitch extraction unit, and a speaker tagging encoding unit.
[0017] The audio input is the input terminal of the pitch extraction unit and the speech content extraction unit; the speaker tag input is the input terminal of the speaker tag encoding unit; and the speech data generation unit outputs the converted speech data.
[0018] The output of the pitch extraction unit is connected to the input of the first convolutional layer; the output of the speech content extraction unit is connected to the input of the second convolutional layer; the output of the speaker tagging unit is connected to the input of the linear layer; the outputs of the first convolutional layer, the second convolutional layer, and the linear layer are connected to the input of the first addition point; the outputs of the first addition point and the numerical integration layer are connected to the input of the first DIT layer; the outputs of the first DIT layer and the linear layer are connected to the input of the second DIT layer; the outputs of the second DIT layer and the linear layer are connected to the input of the third DIT layer; the outputs of the third DIT layer and the first convolutional layer are connected to the input of the fourth DIT layer; the output of the fourth DIT layer is connected to the input of the third convolutional layer; and the output of the third convolutional layer is connected to the input of the speech data generation unit.
[0019] Furthermore, the loss function for training the speech conversion model is as follows:
[0020]
[0021] exp represents the exponential function; t represents the time step; i represents the index; This represents the output of the third convolutional layer when the speech data of the i-th speaker is used as input to the speech conversion model. This represents the speech data of the i-th speaker; N represents the total number of speech data in the speech training dataset; This represents a 0-1 distribution with random initialization.
[0022] Further, in step S105, the speech conversion score for each speaker is calculated, including a speech quality score;
[0023] The formula for the speech quality score is as follows:
[0024]
[0025] i represents the sequence number, L represents the length of the speech data, and n represents the sequence number of the frame in the speech data. This represents the converted speech data of the i-th speaker in the n-th frame. This represents the speech data of the i-th speaker in the n-th frame.
[0026] Further, in step S105, the speech conversion score for each speaker is calculated, including a self-similarity score;
[0027] The formula for the self-similarity score is as follows:
[0028]
[0029] i represents the sequence number. This represents the converted speech data of the i-th speaker. Let represent the speech data of the i-th speaker, and Speaker represent the timbre feature extraction model.
[0030] Further, in step S105, the speech conversion score for each speaker is calculated, including the test sample similarity score;
[0031] The formula for the similarity score of the test samples is as follows:
[0032]
[0033] i represents the sequence number; This represents a broadcaster's voice data; This indicates that the announcer's voice data will be used. When used as the speech data of the i-th speaker, it refers to the transformed speech data; Speaker represents the timbre feature extraction model.
[0034] A training system for a speech conversion model based on data distillation includes:
[0035] Dataset generation module: Acquires speech data from multiple speakers and constructs a speech training dataset;
[0036] First training module: Train the original speech conversion model using the speech training dataset until the first preset round is reached to obtain the initial speech conversion model;
[0037] Evaluation data extraction module: Extracts speech data from each speaker for a preset time period to obtain sample speech data for each speaker and builds a speech evaluation dataset;
[0038] Speech inference module: Input the speech evaluation dataset into the initial speech conversion model to obtain the converted speech data for each speaker;
[0039] Evaluation module: Calculates the speech conversion score for each speaker based on the sample speech data and converted speech data corresponding to each speaker;
[0040] Data distillation module: Determines whether the speech conversion score of each speaker meets the preset conditions; if not, removes the speech data corresponding to that speaker from the speech training dataset, and uses the remaining speech data as the speech database;
[0041] The second training module: the initial speech conversion model is used as the new original speech conversion model, the speech database is used as the new speech training dataset, and the process is repeated in the first training module until the second preset round is reached.
[0042] Final model generation module: The initial speech conversion model and speech database corresponding to the second preset round are used as the trained speech conversion model and final speech database, respectively.
[0043] Applications of speech conversion models based on data distillation include:
[0044] S201: Extract the timbre representation of each speaker's speech data and the speech data to be converted from the final speech database to obtain the timbre representation of each speaker and the timbre features of the speech data to be converted;
[0045] S202: Calculate the cosine similarity between the timbre features of the speech data to be converted and the timbre representation corresponding to each speaker, and take the speaker corresponding to the timbre representation with the highest cosine similarity as the conversion target person;
[0046] S203: Input the speech data of the target person and the speech data to be converted into the trained speech conversion model to obtain the converted target speech data.
[0047] Application systems based on data distillation speech conversion models include:
[0048] Phonogram Extraction Module: Extracts timbre representations of each speaker's speech data and the speech data to be converted from the final speech database, obtaining the timbre representation of each speaker and the timbre features of the speech data to be converted;
[0049] Similarity matching module: Calculates the cosine similarity between the timbre features of the speech data to be converted and the timbre representation of each speaker, and selects the speaker corresponding to the timbre representation with the highest cosine similarity as the conversion target person;
[0050] Target speech data generation module: Input the speech data of the target person to be converted and the speech data to be converted into the trained speech conversion model to obtain the converted target speech data.
[0051] Compared with the prior art, the beneficial effects of the present invention are:
[0052] This invention acquires speech data from multiple speakers to form a speech training dataset; trains an initial speech conversion model using this dataset until a first preset round is reached, obtaining an initial speech conversion model; extracts speech data from each speaker for a preset time period to obtain sample speech data for each speaker, forming a speech evaluation dataset; inputs the speech evaluation dataset into the initial speech conversion model to obtain converted speech data for each speaker; calculates a speech conversion score for each speaker based on their sample and converted speech data; determines whether each speaker's speech conversion score meets a preset condition; if not, removes the speaker's speech data from the speech training dataset, using the remaining data as a speech database; uses the initial speech conversion model as a new original speech conversion model and the speech database as a new speech training dataset, repeating the training process until a second preset round is reached; uses the initial speech conversion model and speech database at the second preset round as the trained speech conversion model and final speech database, respectively; thus, the trained speech conversion model and final speech database can support multiple timbre conversions, resulting in good speech conversion performance.
[0053] Simultaneously, the timbre representations of each speaker's speech data and the speech data to be converted are extracted from the final speech database to obtain the timbre representation corresponding to each speaker and the timbre features of the speech data to be converted. The cosine similarity between the timbre features of the speech data to be converted and the timbre representation corresponding to each speaker is calculated. The speaker corresponding to the timbre representation with the highest cosine similarity is selected as the conversion target speaker. The speech data of the conversion target speaker and the speech data to be converted are input into the trained speech conversion model to obtain the converted target speech data. This results in better speech conversion performance. Attached Figure Description
[0054] Figure 1 The flowchart shows the training method for the speech conversion model based on data distillation provided in Example 1.
[0055] Figure 2 The structure diagram of the speech conversion model provided in Example 1.
[0056] Figure 3 This is a structural diagram of the training system for the speech conversion model based on data distillation provided in Example 1.
[0057] Figure 4 The flowchart shows the application method of the speech conversion model based on data distillation provided in Example 1.
[0058] Figure 5 The diagram shows the structure of the application system of the speech conversion model based on data distillation provided in Example 1.
[0059] Figure 6 The MEL spectrogram of the speech data to be converted provided in Example 1.
[0060] Figure 7 The MEL spectrogram of the converted target speech data provided in Example 1. Detailed Implementation
[0061] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.
[0062] To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions;
[0063] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.
[0064] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.
[0065] Example 1
[0066] like Figure 1 As shown, the training method for the speech conversion model based on data distillation includes:
[0067] S101: Acquire speech data from multiple speakers and construct a speech training dataset;
[0068] S102: Train the original speech conversion model using the speech training dataset until the first preset round is reached to obtain the initial speech conversion model;
[0069] S103: Extract speech data from each speaker for a preset time length to obtain sample speech data for each speaker and build a speech evaluation dataset;
[0070] S104: Input the speech evaluation dataset into the initial speech conversion model to obtain the converted speech data for each speaker;
[0071] S105: Calculate the speech conversion score for each speaker based on the sample speech data and converted speech data corresponding to each speaker;
[0072] S106: Determine whether the speech conversion score of each speaker meets the preset conditions; if not, remove the speech data corresponding to that speaker from the speech training dataset, and use the remaining speech data as the speech database;
[0073] S107: Use the initial speech conversion model as the new original speech conversion model, and the speech database as the new speech training dataset. Repeat steps S102 to S106 until the second preset round is reached.
[0074] S108: The initial speech conversion model and speech database corresponding to the second preset round are used as the trained speech conversion model and final speech database, respectively.
[0075] In one specific embodiment, the second preset round is 3 rounds, and the first preset round is 10 rounds.
[0076] In one specific embodiment, the preset time length in step S103 is 15 seconds, and the extraction is to randomly select one audio clip from the speaker.
[0077] Furthermore, the speaker's voice data includes: audio and speaker tags.
[0078] Furthermore, such as Figure 2 As shown, the speech conversion model includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, a first DIT layer, a second DIT layer, a third DIT layer, a fourth DIT layer, a linear layer, a numerical integration layer, a first addition point, a speech data generation unit, a speech content extraction unit, a pitch extraction unit, and a speaker tagging encoding unit.
[0079] The audio input is the input terminal of the pitch extraction unit and the speech content extraction unit; the speaker tag input is the input terminal of the speaker tag encoding unit; and the speech data generation unit outputs the converted speech data.
[0080] The output of the pitch extraction unit is connected to the input of the first convolutional layer; the output of the speech content extraction unit is connected to the input of the second convolutional layer; the output of the speaker tagging unit is connected to the input of the linear layer; the outputs of the first convolutional layer, the second convolutional layer, and the linear layer are connected to the input of the first addition point; the outputs of the first addition point and the numerical integration layer are connected to the input of the first DIT layer; the outputs of the first DIT layer and the linear layer are connected to the input of the second DIT layer; the outputs of the second DIT layer and the linear layer are connected to the input of the third DIT layer; the outputs of the third DIT layer and the first convolutional layer are connected to the input of the fourth DIT layer; the output of the fourth DIT layer is connected to the input of the third convolutional layer; and the output of the third convolutional layer is connected to the input of the speech data generation unit.
[0081] In one specific embodiment, during the training process, 30% of the data in the output of the speech data generation unit will be replaced with random numbers to improve the robustness of the speech conversion model.
[0082] In one specific embodiment, the speech content extraction unit is a Whisper model or a Sensevoice model, the pitch extraction unit adopts the RMVPE model, the speaker tagging encoding unit adopts the word2vec method, and the speech data generation unit adopts the BIGVGAN model.
[0083] It should be noted that the time step is a fixed value and is input into the numerical integration layer.
[0084] Furthermore, the loss function for training the speech conversion model is as follows:
[0085]
[0086] exp represents the exponential function; t represents the time step; i represents the index; This represents the output of the third convolutional layer when the speech data of the i-th speaker is used as input to the speech conversion model. This represents the speech data of the i-th speaker; N represents the total number of speech data in the speech training dataset; This represents a 0-1 distribution with random initialization.
[0087] It should be noted that, , Relationship satisfaction t represents the time step.
[0088] In one specific embodiment, the relationship between the speech data distribution and the time step and expressive parameters is as follows:
[0089]
[0090] in, Indicates the first j Each expressive parameter at time step t Distribution over time Indicates the first j Distribution of target speech data for each expressive parameter Indicates the first j Distribution of input speech data for each expressive parameter.
[0091] The voice data in this invention is distributed in a distributed form.
[0092] Further, in step S105, the speech conversion score for each speaker is calculated, including a speech quality score;
[0093] The formula for the speech quality score is as follows:
[0094]
[0095] i represents the sequence number, L represents the length of the speech data, and n represents the sequence number of the frame in the speech data. This represents the converted speech data of the i-th speaker in the n-th frame. This represents the speech data of the i-th speaker in the n-th frame.
[0096] In one specific embodiment, when the voice quality score is greater than 2.5, it indicates that the voice quality score has met the preset conditions.
[0097] Further, in step S105, the speech conversion score for each speaker is calculated, including a self-similarity score;
[0098] The formula for the self-similarity score is as follows:
[0099]
[0100] i represents the sequence number. This represents the converted speech data of the i-th speaker. Let represent the speech data of the i-th speaker, and Speaker represent the timbre feature extraction model.
[0101] In one specific embodiment, when the self-similarity score is greater than 0.8, it indicates that the self-similarity score has met the preset condition.
[0102] Further, in step S105, the speech conversion score for each speaker is calculated, including the test sample similarity score;
[0103] The formula for the similarity score of the test samples is as follows:
[0104]
[0105] i represents the sequence number; This represents a broadcaster's voice data; This indicates that the announcer's voice data will be used. When used as the speech data of the i-th speaker, it refers to the transformed speech data; Speaker represents the timbre feature extraction model.
[0106] In one specific embodiment, 15s of voice data from two announcers, one male and one female, are used to calculate the sample similarity score. When both scores are greater than 0.6, it indicates that the test sample similarity score has met the preset condition.
[0107] It should be noted that the speech conversion score for each speaker calculated in step S105 includes one or more of the following: speech quality score, self-similarity score, and test sample similarity score.
[0108] In one specific embodiment, step S101 involves generating speech data from multiple speakers using the VITS text-to-speech model based on the VCTK dataset. This expands the data content and enhances the quality and stability of the generated speech.
[0109] like Figure 3 As shown, the training system for the speech conversion model based on data distillation includes:
[0110] Dataset generation module: Acquires speech data from multiple speakers and constructs a speech training dataset;
[0111] First training module: Train the original speech conversion model using the speech training dataset until the first preset round is reached to obtain the initial speech conversion model;
[0112] Evaluation data extraction module: Extracts speech data from each speaker for a preset time period to obtain sample speech data for each speaker and builds a speech evaluation dataset;
[0113] Speech inference module: Input the speech evaluation dataset into the initial speech conversion model to obtain the converted speech data for each speaker;
[0114] Evaluation module: Calculates the speech conversion score for each speaker based on the sample speech data and converted speech data corresponding to each speaker;
[0115] Data distillation module: Determines whether the speech conversion score of each speaker meets the preset conditions; if not, removes the speech data corresponding to that speaker from the speech training dataset, and uses the remaining speech data as the speech database;
[0116] The second training module: the initial speech conversion model is used as the new original speech conversion model, the speech database is used as the new speech training dataset, and the process is repeated in the first training module until the second preset round is reached.
[0117] Final model generation module: The initial speech conversion model and speech database corresponding to the second preset round are used as the trained speech conversion model and final speech database, respectively.
[0118] like Figure 4 As shown, the application method of the speech conversion model based on data distillation includes:
[0119] S201: Extract the timbre representation of each speaker's speech data and the speech data to be converted from the final speech database to obtain the timbre representation of each speaker and the timbre features of the speech data to be converted;
[0120] S202: Calculate the cosine similarity between the timbre features of the speech data to be converted and the timbre representation corresponding to each speaker, and take the speaker corresponding to the timbre representation with the highest cosine similarity as the conversion target person;
[0121] S203: Input the speech data of the target person and the speech data to be converted into the trained speech conversion model to obtain the converted target speech data.
[0122] In one specific embodiment, the timbre representation is extracted using the eres2netv2 model.
[0123] It should be noted that if there are multiple speech data for one speaker, the average geometric center of the timbre representation extracted from the speech data of multiple speakers will be taken as the timbre representation for that speaker, and the speech data with the smallest distance from the average geometric center will be taken as the speech data for that speaker.
[0124] In one specific embodiment, the input pitch of the speech conversion model is the difference between the pitch of the target speaker and the pitch of the speech data to be converted. This further improves the conversion effect.
[0125] like Figure 5 As shown, the application system based on the data distillation speech conversion model includes:
[0126] Phonogram Extraction Module: Extracts timbre representations of each speaker's speech data and the speech data to be converted from the final speech database, obtaining the timbre representation of each speaker and the timbre features of the speech data to be converted;
[0127] Similarity matching module: Calculates the cosine similarity between the timbre features of the speech data to be converted and the timbre representation of each speaker, and selects the speaker corresponding to the timbre representation with the highest cosine similarity as the conversion target person;
[0128] Target speech data generation module: Input the speech data of the target person to be converted and the speech data to be converted into the trained speech conversion model to obtain the converted target speech data.
[0129] It should be noted that the purpose of similarity matching is to generate beautiful voices. Beautiful voices refer to voices that sound like a specific user, but are more pleasant to the ear than the user's.
[0130] like Figure 6 , Figure 7 As shown, the MEL spectrogram of the converted target speech data is clearer than that of the speech data to be converted, especially... Figure 6 , Figure 7The part highlighted in the green box is something other speech conversion models cannot do. This is because other speech conversion models do not directly derive the target speech data.
[0131] The method of this invention was applied to a dataset of over 500 people. The speech quality score (PESQ) ranged from a minimum of 2.9 to a maximum of 4.0, with an average of 3.8. PESQ ranges from 0 to 5, with a score above 2.5 indicating acceptable quality to the human ear. Two voices, one male and one female, were selected as input test examples, and timbre similarity was calculated for over 500 target speech data. The lowest similarity score was 0.69, the highest was 0.87, and the average was 0.81. Timbre similarity ranges from 0 to 1, with a score greater than 0.6 indicating relatively similar timbre.
[0132] The same or similar labels correspond to the same or similar parts;
[0133] The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent.
[0134] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.
Claims
1. A training method for a speech conversion model based on data distillation, characterized by the following steps: include: S101: Acquire speech data from multiple speakers and construct a speech training dataset; S102: Train the original speech conversion model using the speech training dataset until the first preset round is reached to obtain the initial speech conversion model; S103: Extract speech data from each speaker for a preset time length to obtain sample speech data for each speaker and build a speech evaluation dataset; S104: Input the speech evaluation dataset into the initial speech conversion model to obtain the converted speech data for each speaker; S105: Calculate the speech conversion score for each speaker based on the sample speech data and converted speech data corresponding to each speaker; S106: Determine whether the speech conversion score of each speaker meets the preset conditions; If not, the speech data corresponding to that speaker is removed from the speech training dataset, and the remaining speech data is used as the speech database. S107: Use the initial speech conversion model as the new original speech conversion model, and the speech database as the new speech training dataset. Repeat steps S102 to S106 until the second preset round is reached. S108: The initial speech conversion model and speech database corresponding to the second preset round are used as the trained speech conversion model and the final speech database, respectively. The speaker's voice data includes: audio and speaker tags; The speech conversion model includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, a first DIT layer, a second DIT layer, a third DIT layer, a fourth DIT layer, a linear layer, a numerical integration layer, a first addition point, a speech data generation unit, a speech content extraction unit, a pitch extraction unit, and a speaker tagging encoding unit. The audio input pitch extraction unit is the input terminal of the audio input pitch extraction unit, the speech content extraction unit is the input terminal of the speaker tag input speaker tag encoding unit; the speech data generation unit outputs the converted speech data. The output of the pitch extraction unit is connected to the input of the first convolutional layer; the output of the speech content extraction unit is connected to the input of the second convolutional layer; the output of the speaker tagging unit is connected to the input of the linear layer; the outputs of the first convolutional layer, the second convolutional layer, and the linear layer are connected to the input of the first addition point; the outputs of the first addition point and the numerical integration layer are connected to the input of the first DIT layer; the outputs of the first DIT layer and the linear layer are connected to the input of the second DIT layer; the outputs of the second DIT layer and the linear layer are connected to the input of the third DIT layer; the outputs of the third DIT layer and the first convolutional layer are connected to the input of the fourth DIT layer; the output of the fourth DIT layer is connected to the input of the third convolutional layer; and the output of the third convolutional layer is connected to the input of the speech data generation unit.
2. The training method for the speech conversion model based on data distillation according to claim 1, characterized in that, The loss function used in training the speech conversion model is as follows: exp represents the exponential function; t represents the time step; i represents the index; This represents the output of the third convolutional layer when the speech data of the i-th speaker is used as input to the speech conversion model. This represents the speech data of the i-th speaker; N represents the total number of speech data in the speech training dataset; This represents a 0-1 distribution with random initialization.
3. The training method for the speech conversion model based on data distillation according to claim 1, characterized in that, In step S105, a speech conversion score for each speaker is calculated, including a speech quality score. The formula for the speech quality score is as follows: i represents the sequence number, L represents the length of the speech data, and n represents the sequence number of the frame in the speech data. This represents the converted speech data of the i-th speaker in the n-th frame. This represents the speech data of the i-th speaker in the n-th frame. This represents the converted speech data of the i-th speaker. This represents the speech data of the i-th speaker.
4. The training method for the speech conversion model based on data distillation according to claim 1, characterized in that, In step S105, the speech conversion score for each speaker is calculated, including the self-similarity score; The formula for the self-similarity score is as follows: i represents the sequence number. This represents the converted speech data of the i-th speaker. Let represent the speech data of the i-th speaker, and Speaker represent the timbre feature extraction model.
5. The training method for the speech conversion model based on data distillation according to claim 1, characterized in that, In step S105, the speech conversion score for each speaker is calculated, including the test sample similarity score; The formula for the similarity score of the test samples is as follows: i represents the sequence number; This represents a broadcaster's voice data; This indicates that the announcer's voice data will be used. When used as the speech data of the i-th speaker, it refers to the transformed speech data; Speaker represents the timbre feature extraction model.
6. A training system for a speech conversion model based on data distillation, using the training method described in any one of claims 1 to 5, characterized in that, include: Dataset generation module: Acquires speech data from multiple speakers and constructs a speech training dataset; First training module: Train the original speech conversion model using the speech training dataset until the first preset round is reached to obtain the initial speech conversion model; Evaluation data extraction module: Extracts speech data from each speaker for a preset time period to obtain sample speech data for each speaker and builds a speech evaluation dataset; Speech inference module: Input the speech evaluation dataset into the initial speech conversion model to obtain the converted speech data for each speaker; Evaluation module: Calculates the speech conversion score for each speaker based on the sample speech data and converted speech data corresponding to each speaker; Data distillation module: Determines whether the speech conversion score of each speaker meets the preset conditions; If not, the speech data corresponding to that speaker is removed from the speech training dataset, and the remaining speech data is used as the speech database. The second training module: the initial speech conversion model is used as the new original speech conversion model, the speech database is used as the new speech training dataset, and the process is repeated in the first training module until the second preset round is reached. Final model generation module: The initial speech conversion model and speech database corresponding to the second preset round are used as the trained speech conversion model and final speech database, respectively.
7. An application method for a speech conversion model based on data distillation, characterized in that, include: S201: Extract the timbre representation of each speaker's speech data and the speech data to be converted from the final speech database as described in any one of claims 1 to 5, to obtain the timbre representation of each speaker and the timbre features of the speech data to be converted; S202: Calculate the cosine similarity between the timbre features of the speech data to be converted and the timbre representation corresponding to each speaker, and take the speaker corresponding to the timbre representation with the highest cosine similarity as the conversion target person; S203: Input the speech data of the target person to be converted and the speech data to be converted into the speech conversion model trained by the training method described in any one of claims 1 to 5 to obtain the converted target speech data.
8. An application system based on a data distillation-based speech conversion model, using the application method described in claim 7, characterized in that, include: Phonogram Extraction Module: Extracts timbre representations of each speaker's speech data and the speech data to be converted from the final speech database, obtaining the timbre representation of each speaker and the timbre features of the speech data to be converted; Similarity matching module: Calculates the cosine similarity between the timbre features of the speech data to be converted and the timbre representation of each speaker, and selects the speaker corresponding to the timbre representation with the highest cosine similarity as the conversion target person; Target speech data generation module: Input the speech data of the target person to be converted and the speech data to be converted into the trained speech conversion model to obtain the converted target speech data.