Voice conversion device, voice conversion program, and voice conversion method

Abstract

Even if there are problems with the quality of the input audio, the voice quality conversion will not become unstable, and the output of audio that the listener may find unpleasant will not be produced. [Solution] The voice quality conversion device (1) includes a first conversion unit (142) that converts the voice quality of an input voice, a second conversion unit (143) that converts the sound quality of the voice using a more robust method than the first conversion unit, a determination unit (141) that determines whether the voice meets a predetermined quality based on quality information based on the voice, and an output unit (12) that outputs the voice converted by the first conversion unit if the determination unit determines that the voice meets the predetermined quality, and outputs the voice converted by the second conversion unit if the determination unit determines that the voice does not meet the predetermined quality.

Description

Technical Field

[0001] The present disclosure relates to a voice quality conversion device, a voice quality conversion program, and a voice quality conversion method.

Background Art

[0002] Conventionally, various voice quality conversion technologies have been proposed, including, for example, a method for optimizing a voice synthesis system described in Patent Document 1.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Means for Solving the Problems

[0004] In order to solve the above problems, a voice quality conversion device according to an aspect of the present disclosure includes a first conversion unit that converts the voice quality of the input voice, a second conversion unit that converts the voice quality of the voice by a method more robust than the first conversion unit, a determination unit that determines whether the voice satisfies a predetermined quality based on quality information based on the voice, and an output unit that outputs the voice whose voice quality has been converted by the first conversion unit when the determination unit determines that the voice satisfies the predetermined quality, and outputs the voice whose voice quality has been converted by the second conversion unit when the determination unit determines that the voice does not satisfy the predetermined quality.

[0005] Furthermore, a voice quality conversion program according to another aspect of the present disclosure causes a computer processor to execute at least one of a first conversion process for converting the voice quality of an input voice, and a second conversion process for converting the sound quality of the voice using a method more robust than the first conversion process; a determination process for determining whether the voice meets a predetermined quality based on quality information based on the input voice; and an output process that, if the determination process determines that the voice meets the predetermined quality, outputs the voice whose voice quality has been converted in the first conversion process, and if the determination process determines that the voice does not meet the predetermined quality, outputs the voice whose voice quality has been converted in the second conversion process.

[0006] Furthermore, a voice quality conversion method according to another aspect of the present disclosure includes at least one of a first conversion step in which a computer converts the voice quality of an input voice, and a second conversion step in which the computer converts the sound quality of the voice using a method more robust than the first conversion step; a determination step in which the computer determines whether the voice meets a predetermined quality based on quality information based on the input voice; and an output step in which, if the computer determines in the determination step that the voice meets the predetermined quality, it outputs the voice whose voice quality has been converted in the first conversion step, and if the computer determines in the determination step that the voice does not meet the predetermined quality, it outputs the voice whose voice quality has been converted in the second conversion step. [Brief explanation of the drawing]

[0007] [Figure 1] This is a block diagram showing an example of the functional configuration of a voice quality conversion device according to Embodiment 1 of this disclosure. [Figure 2] This is a flowchart illustrating an example of the processing flow performed by the device. [Figure 3] This is a block diagram showing an example of the functional configuration of a voice quality conversion device according to Embodiment 2 of the present disclosure. [Figure 4] This is a flowchart illustrating an example of the processing flow performed by the device. [Modes for carrying out the invention]

[0008] <Background, challenges, solutions, and principles of solutions> Prior to describing the embodiments of this disclosure, the background of this disclosure, the problems of the prior art, the means for solving the problems, and the principle of solving them by the means for solving the problems will be explained.

[0009] 〔background〕 One method of speech conversion is voice conversion. In typical voice conversion methods, a trained model is made to output latent variables that represent the linguistic and acoustic characteristics of the speech, and the speaker's speech characteristics are added to these latent variables to synthesize a speech waveform.

[0010] 〔assignment〕 Conventional voice conversion technologies, such as the one described in Patent Document 1 at the beginning, have enabled high-quality conversion for clearly spoken speech in a quiet environment. However, conventional voice conversion technologies have a problem in that if the quality of the input speech is poor (for example, if it contains noise), the voice conversion becomes unstable, and there is a possibility that the output will be speech that the listener will find unpleasant.

[0011] [Solution] To solve the above problems, a voice quality conversion device 1 according to one aspect of the present disclosure includes: a first conversion unit that converts the voice quality of an input voice; a second conversion unit that converts the sound quality of the voice using a method more robust than that of the first conversion unit; a determination unit that determines whether the voice meets a predetermined quality based on quality information based on the voice; and an output unit that outputs the voice converted by the first conversion unit if the determination unit determines that the voice meets the predetermined quality, and outputs the voice converted by the second conversion unit if the determination unit determines that the voice does not meet the predetermined quality.

[0012] [Solution Principle] In the voice conversion device configured in this way, if the judgment unit determines that there is a problem with the quality of the voice, the second conversion unit, which converts the voice quality using a more robust method than the first conversion unit, converts the voice quality, and the output unit outputs it. Alternatively, the output unit outputs the voice that the judgment unit determines to be free of problems in quality from among the voice converted by the first conversion unit and the voice converted by the second conversion unit. In this way, the voice conversion device according to this disclosure can prevent unstable voice quality conversion and avoid outputting voice that the listener may find unpleasant, even if there is a problem with the quality of the input voice.

[0013] <Embodiment 1> Next, one embodiment of the present disclosure will be described in detail with reference to the drawings.

[0014] [Configuration of Voice Quality Conversion Device 1] As shown in Figure 1, the voice quality conversion device 1 comprises an input unit 11, an output unit 12, a storage unit 13, and a calculation unit 14.

[0015] (Input section 11) The input unit 11 consists of a microphone that converts sound into waveform data, a communication module that communicates with other devices, and terminals that connect to other devices. The sound to be converted in terms of voice quality is input to the input unit 11.

[0016] (Output section 12) The output unit 12 consists of a speaker that emits sound, a communication module that communicates with other devices, terminals that connect to other devices, etc. The output unit 12 then outputs sound with converted voice quality.

[0017] (Storage unit 13) The storage unit 13 stores the voice conversion program 131. The voice conversion program 131 is a program that causes the computer to function as a voice conversion device 1. The storage unit 13 may also be configured to store various calculation results from the arithmetic unit 14. In this embodiment, the storage unit 13 is composed of a semiconductor memory, a hard disk drive, or the like.

[0018] The storage unit 13 stores the feature quantity generation model M1. When waveform data of voice is input, the feature quantity generation model M1 generates a feature quantity to be input to the teacher model. When the reproduction time of the waveform data is longer than a predetermined time, the feature quantity generation model M1 divides the waveform data every predetermined time and inputs it, and outputs the feature quantity for each divided waveform data.

[0019] Also, the storage unit 13 stores the voice expression model M2. The voice expression model M2 is constructed to output a latent variable representing the feature of voice when the feature quantity of voice is input. The voice expression model M2 according to the present embodiment is composed of a convolutional neural network.

[0020] The voice expression model M2 is constructed through a model learning method including, for example, a first acquisition step, a second acquisition step, and a learning step. In the first acquisition step, a computer acquires a first latent variable output when a feature amount of learning voice is input to a teacher model (not shown). The teacher model is composed of a neural network and is constructed to output a latent variable representing the feature of the voice when the feature amount of the voice is input. In the second acquisition step, a computer acquires a second latent variable output when a feature amount of learning voice is input to the voice expression model M2 before learning. The voice expression model M2 before learning is composed of a neural network with fewer hidden layer numbers than the teacher model and is constructed to output a latent variable representing the feature of the voice when the feature amount of the voice is input. In the learning step, based on the difference between the first latent variable and the second latent variable, the computer causes the voice expression model M2 to learn so that the second latent variable output when the feature amount of the learning voice is input again approaches the first latent variable more than the second latent variable acquired in the second acquisition step. The voice expression model M2 constructed through such a model learning method can be operated more easily (for example, only with a CPU) than the teacher model, even though the output for a certain input is the same as the output of the teacher model. That is, the voice quality conversion device 1 can perform highly accurate voice quality conversion without being affected by the usage environment.

[0021] Further, the storage unit 13 includes a voice quality conversion model M3. The voice quality conversion model M3 is constructed to output the converted voice when the first latent variable and other feature amounts are input. The other feature amounts include, for example, the frequency and volume of the voice. The voice quality conversion model M3 according to the present embodiment is a Transformer model.

[0022] (Arithmetic unit 14) The calculation unit 14 comprises a determination unit 141, a first conversion unit 142, and a second conversion unit 143. The calculation unit 14 according to this embodiment further comprises an output processing unit 144. The calculation unit 14 according to this embodiment is composed of a processor and memory. In other words, the voice quality conversion device 1 is composed of a computer. Therefore, the functions of each control block 141 to 144 are realized by the calculation unit 14 executing various processes S11 to S14 in the flow shown in Figure 2 according to the voice quality conversion program 131 stored in the storage unit 13.

[0023] • Decision processing S11, decision unit 141 In the initial decision process S11, the decision unit 141 determines whether the audio meets a predetermined quality based on audio quality information. The "audio quality information" includes the audio quality information input to the input unit 11 and the audio quality information after the voice quality has been converted. The decision unit 141 determines whether the audio meets a predetermined quality based on the audio quality information input to the input unit 11. The "quality information" includes acoustic features, clustering results (probability distribution and likelihood), and the variability of the output of the trained model. The "acoustic features" include the signal-to-noise ratio, volume, and fundamental frequency. The decision unit 141 makes the decision using one of the following methods (1) to (3).

[0024] Method (1) is a method that makes a determination based on acoustic features. The determination unit 141 using method (1) performs at least one of the following comparisons: comparison of the signal-to-noise ratio with a predetermined first threshold, comparison of the volume with a predetermined second threshold, and comparison of the fundamental frequency with a predetermined numerical range. The determination unit 141 then determines that the audio meets the predetermined quality if there is no quality information that exceeds the threshold or falls outside the numerical range.

[0025] Method (2) is a method that makes a determination using the speech representation model M2. When using method (2), the determination unit 141 comprises a first acquisition unit and a first calculation unit. The first acquisition unit acquires latent variables output by inputting speech-based features into the speech representation model M2. The first calculation unit calculates a score that classifies the latent variables into each of several pre-prepared groups. The "score" includes probability values ​​and log-likelihoods. The "several groups" can be created in advance, for example, using the training data used to construct the speech representation model M2. The determination unit 141 using method (2) then determines that the speech meets a predetermined quality if the highest score among the multiple scores exceeds a predetermined third threshold.

[0026] Method (3) is a method that makes a determination based on the variability of the output of the speech representation model M2. When using method (3), the determination unit 141 comprises a second acquisition unit and a second calculation unit. The second acquisition unit acquires multiple latent variables output by inputting multiple feature quantities based on multiple partial speeches, which are obtained by dividing the speech into multiple parts at predetermined time intervals, to a trained model that is constructed to output latent variables representing the characteristics of speech when speech features are input. The second calculation unit calculates the degree of variability of the multiple latent variables. Then, the determination unit 141 using method (3) determines that the predetermined quality is not met if the degree of variability exceeds a predetermined fourth threshold.

[0027] • First conversion process S12 • Second conversion process S13 After the decision process S11, the calculation unit 14 executes at least one of the first conversion process S12 and the second conversion process S13. In this embodiment, the calculation unit 14 executes either the first conversion process S12 or the second conversion process S13 according to the decision result of the decision unit 141.

[0028] • First conversion process S12, first conversion unit 142 If the determination unit 141 determines (in determination process S11) that the audio meets a predetermined quality standard, the process moves to the first conversion process S12. In the first conversion process S12, the first conversion unit 142 converts the voice quality of the input audio. In this embodiment, the first conversion unit 142 converts the voice quality of the audio using a trained model. In the first conversion process S12, the first conversion unit 142 performs, for example, a feature acquisition process, a latent variable acquisition process, and a voice quality conversion process. In the feature acquisition process, the first conversion unit 142 inputs the audio to be converted into a feature generation model M1 and acquires the features output by the feature generation model M1. In the feature acquisition process, the first conversion unit 142 also extracts other features from the audio to be converted. In the latent variable acquisition process, the first conversion unit 142 inputs the features of the audio to be converted into a speech representation model M2 and acquires the first latent variables output by the speech representation model M2. In the voice quality conversion process, the first conversion unit 142 inputs the first latent variable and other features into the voice quality conversion model M3 and obtains the converted speech output by the voice quality conversion model M3. In the first conversion process S12, the first conversion unit 142 may convert the properties of the speech using a trained model other than the speech representation model M2, or it may convert the voice quality of the speech using a method that does not use a trained model.

[0029] • Second conversion process S13, second conversion unit 143 If the determination unit 141 determines (in determination processing S11) that the audio does not meet the predetermined quality, the process moves to the second conversion processing S13. In the second conversion processing S13, the second conversion unit 143 converts the voice quality of the audio. The second conversion unit 143 converts the sound quality of the audio using a more robust method than the first conversion unit 142. In this embodiment, the second conversion unit 143 converts the voice quality of the audio by converting the pitch range. Note that the method used by the second conversion unit 143 in the second conversion processing S13 to convert the sound quality can be any method that is more robust than the first conversion processing S12 (second conversion unit 143).

[0030] Output processing S14, output processing unit 144 After the first conversion process S12 or the second conversion process S13, the process moves to the output process S14. In the output process S14, the output processing unit 144 controls the output unit 12. As described above, if the determination process S11 (the determination unit 141) determines that the audio meets the predetermined quality, the first conversion process S12 is executed. Therefore, if the determination process S11 (the determination unit 141) determines that the audio meets the predetermined quality, the output unit 12 outputs the audio whose voice quality has been converted by the first conversion unit 142 (S141). Also, as described above, if the determination process S11 (the determination unit 141) determines that the audio does not meet the predetermined quality, the second conversion process S13 is executed. Therefore, if the determination process S41 determines that the audio does not meet the predetermined quality, the output unit 12 outputs the audio whose voice quality has been converted by the second conversion unit 143 (S142).

[0031] [Effects of Voice Quality Transformation Device 1] As described above, if the voice quality conversion device 1 determines that there is a problem with the quality of the voice, the second conversion unit 143 converts the voice quality using a more robust method than the first conversion unit 142. Therefore, with the voice quality conversion device 1, even if there is a problem with the quality of the input voice, the voice quality conversion will not become unstable, and the output of voice that the listener may find unpleasant will not be produced.

[0032] <Embodiment 2> Next, other embodiments of the present disclosure will be described below. For the sake of convenience, components having the same function as those described in Embodiment 1 will be denoted by the same reference numerals, and their descriptions will not be repeated.

[0033] [Configuration of Voice Conversion Device 1A] As shown in Figure 3, the voice quality conversion device 1A according to Embodiment 2 includes an input unit 11 and an output unit 12 similar to those of the voice quality conversion device 1 according to Embodiment 1, as well as a storage unit 13A and a calculation unit 14A.

[0034] (Storage unit 13A) The storage unit 13A stores the voice conversion program 132. The voice conversion program 132 is a program that causes the computer to function as a voice conversion device 1A. The storage unit 13A may also be configured to store various calculation results from the arithmetic unit 14A. In this embodiment, the storage unit 13A is composed of semiconductor memory, a hard disk drive, or the like.

[0035] The memory unit 13A, like the memory unit 13 in the above embodiment 1, includes a feature generation model M1, a speech representation model M2, and a voice quality conversion model M3.

[0036] (Computation unit 14) The calculation unit 14A includes a first conversion unit 142 and a second conversion unit 143, similar to the calculation unit 14 in Embodiment 1 described above. The calculation unit 14A in Embodiment 2 also includes a determination unit 141A and an output processing unit 144A. The calculation unit 14A in this embodiment consists of a processor and memory. In other words, the voice quality conversion device 1A is composed of a computer. Therefore, the functions of each control block 141A, 142-144 are realized by the calculation unit 14A executing various processes S11A, S12-S14 in the flow shown in Figure 4 according to the voice quality conversion program 132 stored in the storage unit 13A.

[0037] • First conversion process S12, second conversion process S13 In the second embodiment, when audio is input to the input unit 11, the calculation unit 14A first executes the first conversion process S12 and the second conversion process S13, respectively.

[0038] • Decision processing S11A, decision unit 141A After the first conversion process S12 and the second conversion process S13, the process moves to the determination process S11A. In the determination process S11A, the determination unit 141A determines whether the audio meets a predetermined quality based on the audio quality information. As described above, the "audio quality information" includes the quality information of the input audio and the quality information of the audio whose voice quality has been converted. The determination unit 141A according to Embodiment 2 determines whether the audio meets a predetermined quality based on the audio quality information of the audio whose voice quality has been converted by at least the first conversion unit 142. The specific method of determination is the same as the method used in the determination process S11 according to Embodiment 1. If the determination unit 141A determines that the audio meets the predetermined quality, it supplies the audio whose voice quality has been converted by the first conversion unit 142 to the output processing unit 144 (output unit 12). On the other hand, if the determination unit 141A determines that the audio does not meet the predetermined quality, it supplies the audio whose voice quality has been converted by the second conversion unit 143 to the output processing unit 144 (output unit 12). The determination unit 141A may determine whether the voice meets a predetermined quality based on the voice quality information obtained by the first conversion unit 142 and the voice quality information obtained by the second conversion unit 143.

[0039] • Output processing S14A, Output processing unit 144 After the judgment process S11A, the process moves to the output process S14A. In the output process S14A, the output processing unit 144 controls the output unit 12. As described above, if the judgment process S11A (the judgment unit 141A) determines that the audio meets the predetermined quality, the audio whose voice quality has been converted by the first conversion unit 142 in the judgment process S11A (the judgment unit 141A) is supplied to the output processing unit 144. Therefore, if the judgment process S11 (the judgment unit 141) determines that the audio meets the predetermined quality, the output unit 12 outputs the audio whose voice quality has been converted by the first conversion unit 142 (S141A). Also, as described above, if the judgment process S11A (the judgment unit 141A) determines that the audio does not meet the predetermined quality, the audio whose voice quality has been converted by the second conversion unit 143 in the judgment process S11A (the judgment unit 141A) is supplied to the output processing unit 144. Therefore, if the judgment unit 141 determines that the audio does not meet the predetermined quality standards, the output unit 12 outputs audio with the voice quality converted by the second conversion unit 143 (S142A).

[0040] [Effects of Voice Conversion Device 1A] As described above, the voice quality conversion device 1A outputs, from among the voice converted by the first conversion unit 142 and the voice converted by the second conversion unit 143, which converts the sound quality of the voice using a more robust method than the first conversion unit 142, the output unit 12 outputs the voice that the judgment unit 141A has determined to have no problems with the quality of the voice. Therefore, with the voice quality conversion device 1A, similar to the voice quality conversion device 1 in Embodiment 1 described above, even if there is a problem with the quality of the input voice, the voice quality conversion will not become unstable, and the output of voice that the listener may find unpleasant will not be produced.

[0041] <Variation> This disclosure is not limited to embodiments 1 and 2 described above, and various modifications are possible within the scope of the claims.

[0042] For example, the voice conversion program described above may be recorded on one or more computer-readable recording media, rather than on a temporary basis. Each unit may or may not have such recording media. In the latter case, the voice conversion program may be supplied to each unit via any wired or wireless transmission medium.

[0043] Furthermore, some or all of the functions of each control block 141 to 144 provided by the above-mentioned arithmetic unit 14 may be realized by logic circuits. For example, an integrated circuit in which logic circuits functioning as each control block 141 to 144 are formed is also included in the scope of this disclosure. In addition, it is also possible to realize the functions of each control block 141 to 144 by, for example, a quantum computer.

[0044] <Summary> A voice quality conversion device according to embodiment 1 of the present invention comprises: a first conversion unit that converts the voice quality of an input voice; a second conversion unit that converts the sound quality of the voice using a method more robust than that of the first conversion unit; a determination unit that determines whether the voice meets a predetermined quality based on quality information based on the voice; and an output unit that outputs the voice converted by the first conversion unit if the determination unit determines that the voice meets the predetermined quality, and outputs the voice converted by the second conversion unit if the determination unit determines that the voice does not meet the predetermined quality.

[0045] The voice quality conversion device according to embodiment 2 of the present invention may be configured such that, in embodiment 1 above, the quality information includes the signal-to-noise ratio, volume, and fundamental frequency, and the determination unit performs at least one of the following comparisons: a comparison of the signal-to-noise ratio with a predetermined first threshold, a comparison of the volume with a predetermined second threshold, and a comparison of the fundamental frequency with a predetermined numerical range, and determines that the voice satisfies the predetermined quality if there is no quality information that exceeds the threshold or falls outside the numerical range.

[0046] The voice quality conversion device according to embodiment 3 of the present invention may be configured such that, in embodiment 1 or 2 above, the determination unit includes an acquisition unit that acquires latent variables output by inputting the voice-based feature quantities to a trained model constructed to output latent variables representing the features of the voice when voice feature quantities are input, and a calculation unit that calculates a score to classify the latent variables into each of a plurality of pre-prepared groups, and determines that the voice satisfies the predetermined quality when the highest score among the plurality of scores exceeds a predetermined third threshold.

[0047] A voice quality conversion device according to aspect 4 of the present invention may be configured such that, in any one of aspects 1 to 3 above, the determination unit includes an acquisition unit that acquires a plurality of latent variables output when a plurality of feature quantities based on a plurality of partial speeches obtained by dividing the speech into a plurality of parts at predetermined time intervals is input to a trained model constructed to output latent variables representing the features of the speech when a feature quantity of speech is input, and a calculation unit that calculates the degree of variation of the plurality of latent variables, and determines that the predetermined quality is not met when the degree of variation exceeds a predetermined fourth threshold.

[0048] The voice quality conversion device according to aspect 5 of the present invention may be configured such that, in any one of aspects 1 to 4 described above, the first conversion unit converts the voice quality of the speech using a learned model, and the second conversion unit converts the voice quality of the speech by converting the pitch range.

[0049] The voice quality conversion device according to embodiment 6 of the present invention may be configured such that, in any one of embodiments 1 to 5 above, the determination unit determines whether the voice meets a predetermined quality based on the voice quality information, and if the determination unit determines that the voice meets the predetermined quality, the first conversion unit converts the voice quality of the voice, and if the determination unit determines that the voice does not meet the predetermined quality, the second conversion unit converts the voice quality of the voice.

[0050] The voice quality conversion device according to embodiment 7 of the present invention may be configured such that, in any one of embodiments 1 to 5 above, the determination unit determines whether the voice meets a predetermined quality based on the quality information of the voice converted by the first conversion unit and the quality information of the voice converted by the second conversion unit, and if it is determined that the voice meets the predetermined quality, it supplies the voice converted by the first conversion unit to the output unit, and if it is determined that the voice does not meet the predetermined quality, it supplies the voice converted by the second conversion unit to the output unit.

[0051] A voice quality conversion program according to aspect 8 of the present invention is configured to cause a computer processor to execute at least one of a first conversion process for converting the voice quality of an input voice, and a second conversion process for converting the sound quality of the voice using a method more robust than the first conversion process; a determination process for determining whether the voice meets a predetermined quality based on quality information based on the input voice; and an output process for outputting the voice whose voice quality has been converted in the first conversion process if the determination process determines that the voice meets the predetermined quality, and for outputting the voice whose voice quality has been converted in the second conversion process if the determination process determines that the voice does not meet the predetermined quality.

[0052] A voice quality conversion method according to aspect 9 of the present invention is a method comprising: at least one of a first conversion step in which a computer converts the voice quality of an input voice; and a second conversion step in which the computer converts the sound quality of the voice using a method more robust than the first conversion step; a determination step in which the computer determines, based on quality information derived from the input voice, whether or not the voice meets a predetermined quality; and an output step in which, if the computer determines in the determination step that the voice meets the predetermined quality, it outputs the voice whose voice quality has been converted in the first conversion step, and if the computer determines in the determination step that the voice does not meet the predetermined quality, it outputs the voice whose voice quality has been converted in the second conversion step. [Explanation of Symbols]

[0053] 1. 1A Voice quality conversion device 11 Input section 12 Output section 13, 13A storage section 131, 132 Voice quality conversion program 14, 14A calculation section 141, 141A Judgment Department 142 First Conversion Unit 143 Second Conversion Section 144 Output Processing Unit M1 Feature Generation Model M2 Speech Representation Model M3 Voice Conversion Model S11, S11A Decision Processing S12 First conversion process S13 Second transformation process S14 Output Processing

Claims

1. A first conversion unit that converts the voice quality of the input audio, A second conversion unit converts the sound quality of the audio using a more robust method than the first conversion unit, A determination unit that determines whether the audio meets a predetermined quality based on quality information derived from the audio, If the determination unit determines that the voice meets the predetermined quality, the first conversion unit outputs the voice with the voice quality converted; if the determination unit determines that the voice does not meet the predetermined quality, the second conversion unit outputs the voice with the voice quality converted; Equipped with, Voice quality conversion device.

2. The aforementioned quality information includes signal-to-noise ratio, volume, and fundamental frequency. The unit that makes the determination said, Comparison of the signal-to-noise ratio with a predetermined first threshold. Comparison of volume and a predetermined second threshold, and comparison of the fundamental frequency with a predetermined numerical range, Perform at least one of the following comparisons: If there is no quality information that exceeds the threshold or falls outside the numerical range, it is determined that the audio meets the predetermined quality requirements. The voice quality conversion device according to claim 1.

3. The unit that makes the determination said, A first acquisition unit acquires the latent variables output by inputting the speech-based features into a pre-trained model that is constructed to output latent variables representing the speech features when speech features are input. A first calculation unit calculates a probability distribution that shows the score to which the aforementioned latent variable is classified into each of a plurality of pre-defined groups, Equipped with, If the highest score among multiple scores exceeds a predetermined third threshold, it is determined that the audio meets the predetermined quality requirements. The voice quality conversion device according to claim 1.

4. The unit that makes the determination said, A second acquisition unit acquires multiple latent variables output by inputting multiple feature quantities based on multiple partial speeches, which are obtained by dividing the speech into multiple parts at predetermined time intervals, to a pre-trained model constructed to output latent variables representing the features of the speech when speech features are input. A second calculation unit that calculates the degree of variation of the aforementioned multiple latent variables, Equipped with, If the degree of variation exceeds a predetermined fourth threshold, it is determined that the predetermined quality is not met. The voice quality conversion device according to claim 1.

5. The first conversion unit converts the voice quality of the speech using the trained model, The second conversion unit converts the voice quality of the speech by converting the pitch range. The voice quality conversion device according to claim 1.

6. The determination unit determines, based on the audio quality information, whether the audio meets a predetermined quality standard. If the determination unit determines that the voice meets the predetermined quality requirements, the first conversion unit converts the voice quality of the voice. If the determination unit determines that the aforementioned audio does not meet the predetermined quality standards, the second conversion unit converts the voice quality of the aforementioned audio. A voice quality conversion device according to any one of claims 1 to 5.

7. The unit that makes the determination said, At least based on the quality information of the voice converted by the first conversion unit, it is determined whether the voice meets a predetermined quality. If the first conversion unit determines that the audio meets the predetermined quality requirements, it supplies the audio with the converted voice quality to the output unit. If the audio is determined not to meet the predetermined quality standards, the second conversion unit supplies the converted audio to the output unit. A voice quality conversion device according to any one of claims 1 to 5.

8. In a computer processor, A first conversion process that converts the voice quality of the input audio, and at least one of a second conversion process that converts the sound quality of the audio using a method more robust than the first conversion process, A determination process that determines whether the audio meets a predetermined quality based on quality information derived from the input audio, If the judgment process determines that the aforementioned audio meets the predetermined quality, the first conversion process outputs the audio with the voice quality converted; if the judgment process determines that the aforementioned audio does not meet the predetermined quality, the second conversion process outputs the audio with the voice quality converted; To execute Voice quality conversion program.

9. A computer performs at least one of the following steps: a first conversion step in which the computer converts the voice quality of the input audio, and a second conversion step in which the computer converts the sound quality of the audio using a method that is more robust than the first conversion step. A computer makes a determination step in which it determines whether the audio meets a predetermined quality based on quality information derived from the input audio, If the computer determines in the determination step that the voice meets the predetermined quality requirements, it outputs the voice whose voice quality has been converted in the first conversion step; if the computer determines in the determination step that the voice does not meet the predetermined quality requirements, it outputs the voice whose voice quality has been converted in the second conversion step; including, Methods for changing voice quality.

Images