Model learning device, model learning program, and model learning method
The model learning device trains a lightweight neural network to replicate a Transformer model's output, addressing CPU limitations and enhancing voice conversion accuracy and quality for diverse input speech qualities.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- SOFTBANK CORPORATION
- Filing Date
- 2024-11-21
- Publication Date
- 2026-06-02
AI Technical Summary
Conventional voice conversion technologies face limitations in operating environments, particularly in CPU-only settings, leading to slow operations or complete shutdowns, and struggle with maintaining voice quality when low-quality speech is input.
A model learning device that trains a second neural network with fewer hidden layers to match the output of a first model, using a Transformer model as a benchmark, enabling voice conversion on CPUs by adjusting latent variables to minimize cross-entropy or mean absolute error, and includes processes to handle telephone-quality speech.
Enables highly accurate voice conversion on CPUs without environmental constraints and improves voice quality for low-quality inputs, ensuring consistent performance across different operating conditions.
Smart Images
Figure 2026090104000001_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a model learning device, a model learning program, and a model learning method.
Background Art
[0002] For example, Non-Patent Document 1 describes a voice conversion technique using a Transformer as a Content Encoder.
Prior Art Documents
Non-Patent Documents
[0003]
Non-Patent Document 1
Summary of the Invention
Means for Solving the Problems
[0004] To solve the above problems, a model learning device according to one aspect of the present disclosure includes: a first acquisition unit that acquires a first latent variable output by inputting features of a training audio to a first model which is composed of a neural network and is constructed to output a latent variable representing the features of an audio when the features of an audio are input; a second acquisition unit that acquires a second latent variable output by inputting features of a training audio to a second model which is composed of a neural network with fewer hidden layers than the first model and is constructed to output a latent variable representing the features of an audio when the features of an audio are input; and a learning unit that trains the second model so that, based on the difference between the first latent variable and the second latent variable, the second latent variable output by the second model when the features of the training audio are input again is closer to the first latent variable than the second latent variable already acquired by the second acquisition unit.
[0005] Furthermore, a model learning program according to another aspect of the present disclosure causes a computer processor to execute: a first acquisition process to acquire a first latent variable output by inputting features of a training audio to a first model, which is composed of a neural network and constructed to output a latent variable representing the features of an audio when the features of the audio are input; a second acquisition process to acquire a second latent variable output by inputting features of a training audio to a second model, which is composed of a neural network with fewer hidden layers than the first model and constructed to output a latent variable representing the features of an audio when the features of an audio are input; and a learning process to train the second model so that, based on the difference between the first latent variable and the second latent variable, the second latent variable output by the second model when the features of the training audio are input again is closer to the first latent variable than the second latent variable already acquired in the second acquisition process.
[0006] Furthermore, a model learning method relating to another aspect of the present disclosure includes: a first acquisition step in which a computer acquires a first latent variable output by inputting features of a training audio to a first model, which is composed of a neural network and is constructed to output a latent variable representing the features of the audio when the features of the audio are input; a second acquisition step in which the computer acquires a second latent variable output by inputting features of the training audio to a second model, which is composed of a neural network with fewer hidden layers than the first model and is constructed to output a latent variable representing the features of the audio when the features of the audio are input; and a learning step in which the computer trains the second model so that, based on the difference between the first latent variable and the second latent variable, the second latent variable output by the second model when the features of the training audio are input again is closer to the first latent variable than the second latent variable acquired in the second acquisition step. [Brief explanation of the drawing]
[0007] [Figure 1] This block diagram shows an example of the functional configuration of a model learning device according to the embodiment of Disclosure 1. [Figure 2] This is a flowchart illustrating an example of the processing flow performed by the device. [Figure 3] This is a block diagram showing an example of the functional configuration of the voice conversion device according to Disclosure 2. [Figure 4] This is a flowchart illustrating an example of the processing flow performed by the device. [Modes for carrying out the invention]
[0008] <Disclosure 1> Disclosure 1 is explained below.
[0009] [Background, challenges, solutions, and principles of solutions] Prior to describing the embodiments of Disclosure 1, the background of Disclosure 1, the problems of the prior art, the means for solving the problems, and the principle of solving them by the means will be explained.
[0010] 〔background〕 One method of speech conversion is voice conversion. In typical voice conversion methods, a trained model is made to output latent variables that represent the linguistic and acoustic characteristics of the speech, and the speaker's speech characteristics are added to these latent variables to synthesize a speech waveform.
[0011] 〔assignment〕 Conventional voice conversion technologies, such as the one described in Non-Patent Document 1 at the beginning, have the advantage of high accuracy in voice conversion. However, conventional voice conversion technologies have problems such as slow operation or complete shutdown in CPU-only environments. In other words, conventional voice conversion technologies have limited operating environments.
[0012] [Solution] To solve the above problems, a model learning device 1 according to one aspect of the present disclosure includes: a first acquisition unit that acquires a first latent variable output by inputting features of a training audio to a first model which is composed of a neural network and is constructed to output a latent variable representing the features of an audio when features of an audio are input; a second acquisition unit that acquires a second latent variable output by inputting features of a training audio to a second model which is composed of a neural network with fewer hidden layers than the first model and is constructed to output a latent variable representing the features of an audio when features of an audio are input; and a learning unit that trains the second model based on the difference between the first latent variable and the second latent variable so that the second latent variable output by the second model when features of a training audio are input again is closer to the first latent variable than the second latent variable already acquired by the second acquisition unit.
[0013] [Solution Principle] The model learning device 1 configured in this way trains a second model, which is composed of a neural network with fewer hidden layers than the first model (smaller structure and lighter weight), to produce the same output as the first model. In this way, the model learning device 1 can create a second model for speech conversion that can operate on a CPU alone, even though its output for a given input is the same as that of the first model, which is difficult to operate on a CPU alone. As a result, it becomes possible to perform highly accurate voice conversion without being affected by the operating environment.
[0014] [Embodiment of Disclosure 1] Next, one embodiment of Disclosure 1 will be described in detail with reference to the drawings.
[0015] [Model used] Before describing the model learning device 1, we will explain the models used by the model learning device 1. The models used by the model learning device 1 are the first model M1 and the second model M2.
[0016] (First model M1) The first model M1 is a pre-trained model that serves as a benchmark when training the second model M2. The first model M1 is composed of a neural network. Furthermore, the first model M1 is constructed to output latent variables representing the characteristics of speech when speech features are input. The first model in this embodiment is a Transformer model.
[0017] (Second model M2) The second model M2 is the model to be trained using the model learning device 1. The second model M2 is composed of a neural network with fewer hidden layers than the first model M1. Furthermore, the second model M2 is constructed to output latent variables representing the features of speech when speech features are input. The second model in this embodiment is composed of a convolutional neural network.
[0018] [Specific configuration of model learning device 1] As shown in FIG. 1, the model learning device 1 includes an input unit 11, an output unit 12, a storage unit 13, and a calculation unit 14.
[0019] (Input Unit 11) The input unit 11 is composed of a communication module that communicates with other devices, a drive that reads information from a medium, terminals connected to other devices, and the like. The first model M1 and the second model M2 before learning are input into the input unit 11. The input first model M1 and the second model M2 before learning are stored in the storage unit 13. In addition, learning voice is input into the input unit 11. Note that the input unit 11 may be divided into an input unit for inputting a model and an input unit for inputting learning voice.
[0020] (Output Unit 12) The output unit 12 is composed of a communication module that communicates with other devices, a drive that writes information to a medium, terminals connected to other devices, and the like. The output unit 12 outputs the second model M2 after learning. Note that the output unit 12 may be integrated with the input unit 11.
[0021] (Storage Unit 13) The storage unit 13 stores a model learning program 131. The model learning program 131 is a program that causes a computer to function as the model learning device 1. In addition, the storage unit 13 may be configured to be able to store various calculation results of the calculation unit 14. The storage unit 13 according to the present embodiment is composed of a semiconductor memory, a hard disk drive, and the like.
[0022] The memory unit 13 stores a first feature generation model M3 and a second feature generation model M4. When audio waveform data is input, the first feature generation model M3 generates features to be input to the first model M1. The first feature generation model M3 can be composed of, for example, a Transformer. When audio waveform data is input, the second feature generation model M4 generates features to be input to the second model M2. The second feature generation model M4 can be composed of, for example, a convolutional neural network. If the playback time of the waveform data is longer than a first predetermined time, the first feature generation model M3 divides the waveform data into first predetermined time intervals and inputs them, and outputs features for each divided waveform data. Also, if the playback time of the waveform data is longer than a second predetermined time which is shorter than the first predetermined time, the second feature generation model M4 divides the waveform data into second predetermined time intervals and inputs them, and outputs features for each divided waveform data. In other words, the second feature generation model M4 can process a shorter length of waveform data at one time than the first feature generation model M3. On the other hand, the second feature generation model M4 generates features with the same amount of information as the first feature generation model M3, even when inputting shorter waveform data. In other words, the second feature generation model M4 requires less processing to obtain the necessary features than the first feature generation model M3, and also has a smaller delay to output. Note that the second feature generation model M4 may be the first feature generation model M3, or it may have the same weighting as the first feature generation model M3.
[0023] (Computation unit 14) The calculation unit 14 comprises a first acquisition unit 141, a second acquisition unit 142, and a learning unit 143. The calculation unit 14 according to this embodiment further comprises a calculation unit 144, a feature acquisition unit 145, and a modification unit 146. The calculation unit 14 according to this embodiment is composed of a processor and memory. In other words, the model learning device 1 is composed of a computer. Therefore, the functions of each control block 141 to 146 are realized by the calculation unit 14 executing various processes S11 to S15 in the flow shown in Figure 2 according to the model learning program 131 stored in the memory unit 13, and executing process S16 as necessary.
[0024] • Feature acquisition process S15 • Feature acquisition unit 145 In the initial feature acquisition process S15, the feature acquisition unit 145 inputs training audio to the first feature generation model M3 and acquires the features output by the first feature generation model M3. Also in the feature acquisition process S15, the feature acquisition unit 145 inputs training audio to the second feature generation model M4 and acquires the features output by the second feature generation model M4.
[0025] • First acquisition process S11 • First acquisition unit 141 After the feature acquisition process S15, the process moves to the first acquisition process S11. In the first acquisition process S11, the first acquisition unit 141 inputs the features of the training speech into the first model M1 and acquires the first latent variable output by the first model M1. The "speech features" include both linguistic and acoustic features.
[0026] • Second acquisition process S12 • Second acquisition unit 142 After the first acquisition process S11, the process moves to the second acquisition process S12. In the second acquisition process, the second acquisition unit 142 inputs the features of the training speech to the second model M2 and acquires the second latent variable output by the second model M2. The second acquisition unit 142 may execute the second acquisition process S12 before the first acquisition process S11, or it may execute it in parallel with the first acquisition process S11.
[0027] • Calculation process S14 • Calculation unit 144 After the first acquisition process S11 and the second acquisition process S12, the process moves to calculation process S14. In calculation process S14, the calculation unit 144 calculates the first probability distribution. The first probability distribution shows the probability that the first latent variable is classified into each of the multiple groups. Also in calculation process S14, the calculation unit 144 calculates the second probability distribution. The second probability distribution shows the probability that the second latent variable is classified into each of the multiple groups. Then, the calculation unit 144 further calculates the cross-entropy error using the first probability distribution as the measured value and the second probability distribution as the predicted value.
[0028] The calculation unit 144 may be configured to calculate the mean absolute error, where the first latent variable is the observed value and the second latent variable is the predicted value, rather than calculating the first and second probability distributions.
[0029] • Learning process S13 • Learning unit 143 If the magnitude of the calculated cross-entropy error is greater than or equal to a predetermined value (not zero), the process moves to learning process S13. In learning process S13, the learning unit 143 trains the second model so that the second latent variable output by the second model when the feature quantities of the training audio are input again is closer to the first latent variable than the second latent variable already acquired by the second acquisition unit 142, and more preferably matches the first latent variable. In this embodiment, the learning unit 143 trains the second model so that the cross-entropy error calculated based on the second probability distribution output by the second model when the feature quantities of the training audio are input again is closer to zero than the cross-entropy error calculated based on the second probability distribution already calculated by the calculation unit 144, and more preferably becomes zero.
[0030] Furthermore, if the calculation unit 144 is configured to calculate the mean absolute error rather than the cross-entropy error, the learning unit 143 may be configured to train the second model so that the mean absolute error calculated based on the second latent variable output by the second model when the features of the training audio are input again approaches zero, more preferably becomes zero, than the mean absolute error calculated by the second acquisition unit 142 based on the second latent variable already acquired.
[0031] • Modification treatment S16 • Modification section 146 Incidentally, conventional voice conversion models did not support telephone-quality speech. In other words, conventional voice conversion models had the problem that the quality of the output speech deteriorated significantly when low-quality speech was input. Therefore, the calculation unit 14 according to this embodiment performs a modification process S16 as needed (when it is necessary to further train the second model M2 after training with telephone speech). In the modification process S16, the modification unit 146 reduces the sound quality of the training speech. Methods for "reducing sound quality" include downsampling and noise addition. As a result, training speech that simulates telephone speech is generated. Then, the feature quantities of the training speech with reduced sound quality are input to the second model M2 after training. As a result, after calculation process S14 and training process S13, the second model M2 becomes able to handle telephone speech.
[0032] [Modified version of model learning device 1] Disclosure 1 is not limited to the embodiments described above, and various modifications are possible within the scope of the claims.
[0033] For example, the model learning device 1 would take in the first model M1 and the second model M2 (store them in the memory unit 13) and train the taken-in second model M2. However, the model learning device 1 does not have to take in at least one of the first model M1 and the second model M2 (it may obtain the first latent variable from the first model M1 stored in another device, or train the second model M2 stored in another device). In this case, the model learning device 1 does not have to store the feature generation models corresponding to the models that are not taken in, among the first feature generation model M3 and the second feature generation model M4. Also, if neither the first model M1 nor the second model M2 is taken in, the calculation unit 14 does not have to include the feature acquisition unit 145.
[0034] Furthermore, the calculation unit 14 according to the above embodiment included a calculation unit 144 and a modification unit 146. However, these functional blocks 144 and 145 may be provided by other calculation units or other devices different from the model learning device 1. In that case, the calculation unit 14 does not need to include these functional blocks 144 and 145.
[0035] Furthermore, the above model learning program may be recorded on one or more computer-readable recording media, rather than on a temporary basis. Each unit may or may not have such recording media. In the latter case, the model learning program may be supplied to each unit via any wired or wireless transmission medium.
[0036] Furthermore, some or all of the functions of each control block 141 to 146 provided by the above-mentioned arithmetic unit 14 may be realized by logic circuits. For example, an integrated circuit in which logic circuits functioning as each control block 141 to 146 are formed is also included in the scope of Disclosure 1. In addition, it is also possible to realize the functions of each control block 141 to 146 by, for example, a quantum computer.
[0037] [Effects of Model Learning Device 1] The model learning device 1 described above trains a second model, which is composed of a neural network with fewer hidden layers than the first model, to produce the same output as the first model. Therefore, the model learning device 1 makes it possible to create a second model M2 for speech conversion that can operate on a CPU alone, even though the output for a given input is the same as the output of the first model M1, which is difficult to operate on a CPU alone. As a result, it becomes possible to perform highly accurate voice conversion without being affected by the operating environment.
[0038] <Disclosure 2> Next, we will describe the second embodiment of disclosure.
[0039] [Outline configuration of the voice conversion device 2] As shown in Figure 3, the voice conversion device 2 comprises an input unit 21, an output unit 22, a storage unit 23, and an arithmetic unit 24.
[0040] (Input section 21) The input unit 21 consists of a microphone that converts sound into waveform data, a communication module that communicates with other devices, and terminals that connect to other devices. The sound to be converted in terms of voice quality is input to the input unit 21.
[0041] (Output section 22) The output unit 22 consists of a speaker that emits sound, a communication module that communicates with other devices, terminals that connect to other devices, etc. The output unit 22 then outputs sound with converted voice quality.
[0042] (Storage unit 23) The storage unit 23 stores the voice conversion program 231. The voice conversion program 231 is a program that causes the computer to function as a voice conversion device 2. The storage unit 23 may also be configured to store various calculation results from the arithmetic unit 24. In this embodiment, the storage unit 23 is composed of a semiconductor memory, a hard disk drive, or the like.
[0043] The memory unit 23 stores the feature generation model M3. The feature generation model M3 is a trained model corresponding to the first feature generation model M3 or the second feature generation model M4 according to Disclosure 1. The memory unit 23 also includes a speech representation model M5 and a voice quality conversion model M6. The speech representation model M5 is a trained model corresponding to the second model M2 after training. The voice quality conversion model M6 is constructed to output converted speech when a first latent variable and other features are input. Other features include, for example, speech frequency, volume, etc. The voice quality conversion model M6 according to this embodiment is a Transformer model.
[0044] (Computation unit 24) The calculation unit 24 comprises a feature acquisition unit 241, a latent variable acquisition unit 242, and a voice quality conversion unit 243. In this embodiment, the calculation unit 24 consists of a processor and memory. In other words, the voice conversion device 2 is composed of a computer. Therefore, the functions of each control block 241 to 243 are realized by the calculation unit 24 executing various processes S21 to S23 in the flow shown in Figure 4, according to the voice conversion program 231 stored in the memory unit 23.
[0045] • Feature acquisition process S21 • Feature acquisition unit 241 In the feature acquisition process S21, the feature acquisition unit 241 inputs the audio to be converted into the first feature generation model M3 and acquires the features output by the first feature generation model M3. In addition, in the feature acquisition process S21, the feature acquisition unit 241 extracts other features from the audio to be converted.
[0046] • Latent variable acquisition process S22, Latent variable acquisition unit 242 After the feature acquisition process S21, the process moves to the latent variable acquisition process S22. In the latent variable acquisition process S22, the latent variable acquisition unit 242 inputs the feature quantities of the speech to be converted to the speech representation model M5 and acquires the first latent variable output by the speech representation model M5.
[0047] • Voice quality conversion processing S23, voice quality conversion unit 243 After the latent variable acquisition process S22, the process moves to the voice quality conversion process S23. In the voice quality conversion process S23, the voice quality conversion unit 243 inputs the first latent variable and other features into the voice quality conversion model M6 and acquires the converted speech output by the voice quality conversion model M6.
[0048] <Summary> A model learning device according to embodiment 1 of the present invention comprises: a first acquisition unit that acquires a first latent variable output by inputting features of a training audio to a first model, which is composed of a neural network and is constructed to output a latent variable representing the features of an audio when the features of an audio are input; a second acquisition unit that acquires a second latent variable output by inputting features of a training audio to a second model, which is composed of a neural network with fewer hidden layers than the first model and is constructed to output a latent variable representing the features of an audio when the features of an audio are input; and a learning unit that trains the second model so that, based on the difference between the first latent variable and the second latent variable, the second latent variable output by the second model when the features of the training audio are input again is closer to the first latent variable than the second latent variable already acquired by the second acquisition unit.
[0049] The model learning device according to embodiment 2 of the present invention may be configured such that, in embodiment 1 above, the first model is a Transformer model and the second model is composed of a convolutional neural network.
[0050] A model learning device according to embodiment 3 of the present invention, in embodiment 1 or 2 above, further comprises a calculation unit that calculates a first probability distribution showing the probability that the first latent variable is classified into each of the plurality of groups, calculates a second probability distribution showing the probability that the second latent variable is classified into each of the plurality of groups, and further calculates a cross-entropy error using the first probability distribution as an actual value and the second probability distribution as a predicted value, wherein the learning unit is configured to train the second model such that the cross-entropy error calculated based on the second probability distribution output by the second model when the feature quantities of the learning audio are input again approaches zero more than the cross-entropy error calculated by the calculation unit based on the second probability distribution already calculated.
[0051] A model learning device according to aspect 4 of the present invention further comprises a calculation unit that calculates the mean absolute error where the first latent variable is an actual value and the second latent variable is a predicted value, in any one of aspects 1 to 3 described above, and the learning unit is configured to train the second model such that the mean absolute error calculated based on the second latent variable output by the second model when the feature quantities of the learning audio are input again is closer to zero than the mean absolute error calculated based on the second latent variable already acquired by the second acquisition unit.
[0052] The model learning device according to aspect 5 of the present invention may further include a modification unit that reduces the sound quality of the learning audio, in any one of aspects 1 to 4 described above, and the second model may be configured such that the feature quantities of the learning audio with reduced sound quality are input.
[0053] The model learning device according to aspect 6 of the present invention may further include, in any one of aspects 1 to 5 above, a first feature generation model that generates the feature quantities to be input to the first model when audio waveform data is input, and a second feature generation model that generates the feature quantities to be input to the second model when audio waveform data is input.
[0054] In the model learning apparatus according to embodiment 7 of the present invention, the second feature generation model may be configured such that, in embodiment 6 above, the second feature generation model is the first feature generation model, or has the same weighting as the first feature generation model.
[0055] A model learning program according to embodiment 8 of the present invention is configured to cause a computer processor to execute: a first acquisition process to acquire a first latent variable output by inputting features of a training audio to a first model, which is composed of a neural network and is constructed to output a latent variable representing the features of an audio when the features of an audio are input; a second acquisition process to acquire a second latent variable output by inputting features of a training audio to a second model, which is composed of a neural network with fewer hidden layers than the first model and is constructed to output a latent variable representing the features of an audio when the features of an audio are input; and a learning process to train the second model so that, based on the difference between the first latent variable and the second latent variable, the second latent variable output by the second model when the features of the training audio are input again approaches the first latent variable more closely than the second latent variable already acquired in the second acquisition process.
[0056] A model learning method according to aspect 9 of the present invention is a method comprising: a first acquisition step in which a computer acquires a first latent variable output by inputting features of a training audio to a first model constructed of a neural network that outputs a latent variable representing the features of an audio when the features of an audio are input; a second acquisition step in which the computer acquires a second latent variable output by inputting features of a training audio to a second model constructed of a neural network with fewer hidden layers than the first model that outputs a latent variable representing the features of an audio when the features of an audio are input; and a learning step in which the computer trains the second model so that, based on the difference between the first latent variable and the second latent variable, the second latent variable output by the second model when the features of the training audio are input again is closer to the first latent variable than the second latent variable acquired in the second acquisition step. [Explanation of Symbols]
[0057] 1. Model Learning Device 11 Input section 12 Output section 13 storage section 131 Model Learning Program M1 First Model M2 Second Model M3 First Feature Generation Model M4 Second Feature Generation Model 14 Arithmetic section 141 First Acquisition Department 142 Second Acquisition Department 143 Learning Department 144 Calculation Section 145 Feature acquisition unit 146 Modification section S11 First acquisition process S12 Second acquisition process S13 Learning Process S14 Calculation process S15 Feature acquisition process S16 Modification Treatment 2. Voice converter 21 Input section 22 Output section 23 Memory section 231 Voice Conversion Program M3 Feature Generation Model M5 Voice Representation Model M6 Voice Conversion Model 24 Arithmetic section 241 Feature Acquisition Unit 242 Latent Variable Acquisition Unit 243 Voice quality conversion unit S21 Feature acquisition process S22 Latent Variable Acquisition Process S23 Voice quality conversion processing
Claims
1. A first acquisition unit acquires the first latent variable output by inputting training audio features to a first model, which is composed of a neural network and is constructed to output a latent variable representing the audio features when audio features are input to the first model. A second acquisition unit acquires the second latent variable output by inputting the training audio features to a second model, which is constructed as a neural network with fewer hidden layers than the first model and is configured to output a latent variable representing the audio features when audio features are input to the second model. A learning unit that trains the second model so that, based on the difference between the first latent variable and the second latent variable, the second latent variable output by the second model when the features of the training audio are input again is closer to the first latent variable than the second latent variable already acquired by the second acquisition unit. Equipped with, Model learning device.
2. The first model mentioned above is a Transformer model, The second model described above is composed of a convolutional neural network. The model learning device according to claim 1.
3. The calculation unit further comprises a calculation unit that calculates a first probability distribution showing the probability of the first latent variable being classified into each of the multiple groups, a second probability distribution showing the probability of the second latent variable being classified into each of the multiple groups, and further calculates a cross-entropy error using the first probability distribution as an observed value and the second probability distribution as a predicted value. The learning unit trains the second model such that the cross-entropy error calculated based on the second probability distribution output by the second model when the feature quantities of the training audio are input again approaches zero, compared to the cross-entropy error calculated by the calculation unit based on the second probability distribution already calculated. The model learning device according to claim 1.
4. The system further comprises a calculation unit that calculates the mean absolute error, where the first latent variable is the measured value and the second latent variable is the predicted value. The learning unit trains the second model such that the mean absolute error calculated based on the second latent variable output by the second model when the features of the training audio are input again approaches zero, compared to the mean absolute error calculated by the second acquisition unit based on the second latent variable already acquired. The model learning device according to claim 1.
5. The system further includes a modification unit that reduces the sound quality of the aforementioned learning audio, The second model is input with feature quantities of training audio whose sound quality has been reduced. The model learning device according to claim 1.
6. A first feature generation model that generates the feature quantities to be input to the first model when audio waveform data is input, A second feature generation model that generates the feature quantities to be input to the second model when audio waveform data is input, Furthermore, The model learning device according to claim 1.
7. The second feature generation model is either the first feature generation model or has the same weighting as the first feature generation model. The model learning device according to claim 6.
8. In a computer processor, A first acquisition process involves acquiring the first latent variable output by inputting training audio features into a first model, which is constructed as a neural network and outputs a latent variable representing the audio features when audio features are input. A second acquisition process involves acquiring the second latent variable output by inputting the training audio features to a second model, which is constructed as a neural network with fewer hidden layers than the first model and is designed to output a latent variable representing the audio features when audio features are input to the second model, and the training audio features are input to the second model. A learning process that trains the second model so that, based on the difference between the first latent variable and the second latent variable, the second latent variable output by the second model when the features of the training audio are input again is closer to the first latent variable than the second latent variable already acquired in the second acquisition process. To execute Model learning program.
9. A first acquisition step involves a computer receiving training audio features as input to a first model, which is constructed as a neural network and outputs latent variables representing the audio features when audio features are input, and then acquiring the first latent variables output by the first model. The second acquisition step involves acquiring the second latent variable output by inputting the training audio features into a second model, which is constructed by a neural network with fewer hidden layers than the first model, and which outputs a latent variable representing the audio features when audio features are input into the second model. A learning step in which the computer trains the second model so that, based on the difference between the first latent variable and the second latent variable, the second latent variable output by the second model when the features of the training audio are input again is closer to the first latent variable than the second latent variable already acquired in the second acquisition step. including, Model learning methods.