Learning device, learning program, and learning method

Abstract

To provide a learning device, etc., that can obtain learning results with higher accuracy in reverberation removal. [Solution] The neural network comprises a neural network that generates observed sound data to be used for training by adding a noise signal to a signal obtained by convolving an audio signal and an indoor impulse response signal, estimates the reverberation time of the indoor impulse response signal, generates a shortening window to shorten the indoor impulse response signal to a desired shortened reverberation time, generates a reverberation-shortened indoor impulse response signal in which the reverberation time is shortened by multiplying the indoor impulse response signal by the shortening window, generates a correct value corresponding to the observed sound by convolving an audio signal into the reverberation-shortened indoor impulse response signal, and outputs an inferred value based on the observed sound, obtains the inference evaluation result of the inferred value, and updates the parameters of the neural network based on the inference evaluation result.

Description

[Technical Field]

[0001] This invention relates to a learning device, a learning program, and a learning method, and can be applied, for example, to a system for learning about reverberation removal in acoustic signals. [Background technology]

[0002] In speech recognition under reverberation, the accuracy of speech recognition decreases due to the effect of reverberation, so techniques for removing reverberation without distorting the speech require high precision. Conventional reverberation removal methods include those based on statistical models and signal processing models. The common objective of these reverberation removal methods is to estimate an accurate inverse filter of the RIR signal for reverberation speech, which is generated by convolving the room impulse response (hereinafter also called "RIR") into the speech signal. For example, methods based on weighted prediction error (hereinafter called "WPE") directly estimate the inverse filter from the reverberation speech using linear prediction and apply that inverse filter to the reverberation speech to remove the reverberation. On the other hand, with the recent development of applied technologies of Deep Neural Networks (hereinafter also called "DNN"), new methods for removing noise and reverberation have emerged. One of the basic ideas of this applied technology of DNN is to estimate a nonlinear filter that generates the spectrogram features of the target speech from the spectrogram features of speech including noise and reverberation, based on supervised learning.

[0003] One method for estimating such nonlinear filters is to train a model called FullSubNet, which is a noise-removing filter, as described in Non-Patent Document 1. This model takes the spectrogram features of the observed sound as input values ​​and trains a nonlinear filter that outputs a signal with the noise removed.

[0004] Furthermore, Non-Patent Document 2 proposes a method that removes not only noise but also reverberation, in addition to the method described in Non-Patent Document 1. The model described in Non-Patent Document 1 is trained with the goal of making the inferred values ​​obtained based on the input value and the Neural Network (hereinafter also referred to as "NN") approach the sound from which only the noise has been removed, that is, speech including reverberation. On the other hand, Non-Patent Document 2 examines a method of training the inferred values ​​obtained based on the input value and the NN to approach one of the following: "speech with no reverberation at all," "early reflections containing only early reflections," or "speech with very short reverberation," and states that the method of training to approach "speech with very short reverberation" performs the best. As the reason for this, Non-Patent Document 2 states that "speech with very short reverberation" allows for the removal of reverberation while maintaining the naturalness of the speech the most, compared to "speech with no reverberation at all" or "early reflections containing only early reflections." [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Xiang Hao, Xiangdong Su, Radu Horaud, and Xiaofei Li, “Fullsubnet: A full-band and sub-band fusion model for real-time single-channel speech enhancement,” in ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 6633-6637. [Non-Patent Document 2] R. Zhou, W. Zhu, and X. Li, “Speech dereverberation with a reverberation time shortening target,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).IEEE, 2023, pp. 1-5. [Overview of the project] [Problems that the invention aims to solve]

[0006] As described above, in order to remove reverberation, it is necessary to train the input value and the inference value obtained based on the neural network so that it approaches speech with very short reverberation. However, conventional techniques have a challenge in generating speech with such very short reverberation. To generate speech with such very short reverberation, it is necessary to convolve the speech with a shortened RIR signal of the original RIR signal, and in order to shorten the original RIR signal, the reverberation time of the original RIR signal is required. In the method described in Non-Patent Document 1, the reverberation time listed in the dataset is used as the original reverberation time, but there is an error between the reverberation time listed in the dataset and the actual reverberation time, so there are cases in which the shortened RIR signal is mistakenly used to generate speech with such very short reverberation. As a result, conventional techniques may fail to shorten the reverberation time to the intended length, which can lead to a decrease in the accuracy of reverberation removal.

[0007] In light of the above-mentioned problems, there is a need for a learning device, learning program, and learning method that can obtain learning results with higher reverberation removal accuracy. [Means for solving the problem]

[0008] The first learning device of the present invention is characterized by comprising: observation sound generation means for generating observed sounds to be used as learning data by adding a noise signal to a signal obtained by convolving an audio signal and an indoor impulse response signal; reverberation time estimation means for estimating the reverberation time of the indoor impulse response signal; shortening window generation means for generating a shortening window for shortening the indoor impulse response signal to a desired shortened reverberation time; correct value generation means for generating a reverberation-shortened indoor impulse response signal whose reverberation time is the shortened reverberation time by multiplying the indoor impulse response signal by the shortening window, and generating a correct value corresponding to the observed sound by convolving the audio signal into the reverberation-shortened indoor impulse response signal; inference means comprising a neural network that outputs an inferred value based on the observed sound; inference evaluation means for obtaining an inference evaluation result of the inference value by comparing the inferred value and the correct value; and update means for updating the parameters of the neural network based on the inference evaluation result so that the observed sound approaches the correct value.

[0009] The second learning program of the present invention is a learning program characterized in that it causes a computer to function as an observation sound generation means that generates an observation sound to be used as learning data by adding a noise signal to a signal obtained by convolving an audio signal and an indoor impulse response signal; a reverberation time estimation means that estimates the reverberation time of the indoor impulse response signal; a shortening window generation means that generates a shortening window to shorten the indoor impulse response signal to a desired shortened reverberation time; a reverberation shortened indoor impulse response signal that generates a reverberation shortened indoor impulse response signal whose reverberation time is the shortened reverberation time by multiplying the indoor impulse response signal by the shortening window, and a correct value generation means that generates a correct value corresponding to the observation sound by convolving the audio signal into the reverberation shortened indoor impulse response signal; an inference means that includes a neural network that outputs an inferred value based on the observation sound; an inference evaluation means that obtains an inference evaluation result of the inference value by comparing the inferred value and the correct value; and an update means that updates the parameters of the neural network so that the observation sound approaches the correct value based on the inference evaluation result.

[0010] The third aspect of the present invention relates to a learning method performed by a learning device, comprising: observation sound generation means, reverberation time estimation means, shortening window generation means, correct value generation means, inference means, inference evaluation means, and update means, wherein the observation sound generation means generates an observation sound to be used as learning data by adding a noise signal to a signal obtained by convolving an audio signal and an indoor impulse response signal; the reverberation time estimation means estimates the reverberation time of the indoor impulse response signal; the shortening window generation means generates a shortening window to shorten the indoor impulse response signal to a desired shortened reverberation time; and the correct value generation means adds the shortening window to the indoor impulse response signal. The inference means comprises a neural network that outputs an inference value based on the observed sound. The inference evaluation means obtains an inference evaluation result of the inference value by comparing the inference value with the correct answer value. The update means updates the parameters of the neural network based on the inference evaluation result so that the observed sound approaches the correct answer value. [Effects of the Invention]

[0011] This invention provides a learning device, a learning program, and a learning method that can obtain learning results with higher reverberation removal accuracy. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing the functional configuration of the learning device according to the embodiment. [Figure 2] This is a block diagram showing the hardware configuration of the learning device according to the embodiment. [Figure 3] This is a flowchart illustrating the operation of the learning device according to the embodiment. [Modes for carrying out the invention]

[0013] (A) Main embodiment Hereinafter, an embodiment of a learning device, a learning program, and a learning method according to the present invention will be described in detail with reference to the drawings.

[0014] (A-1) Configuration of the embodiment FIG. 1 is a block diagram showing the functional configuration of a learning device 1 according to this embodiment.

[0015] The learning device 1 learns a non-linear filter that can extract only the features of the spectrogram of the target sound from the features of the spectrogram of the sound including noise and reverberation.

[0016] The learning device 1 includes an input unit 10, a dataset storage unit 20, an inference unit 30, an inference evaluation unit 40, and an update unit 50.

[0017] In this embodiment, the inference unit 30 performs learning processing and inference processing using a neural network (hereinafter also referred to as "NN"). Specifically, the inference unit 30 may have any specific configuration as long as it includes a neural network (hereinafter also denoted as "inference NN"). As shown in FIG. 1, in this embodiment, the inference unit 30 is assumed to have an inference NN 31.

[0018] The dataset storage unit 20 is a storage means for storing a dataset DS to be learned. In FIG. 1, one (one set) of the dataset DS is arranged in the dataset storage unit 20, but the number of arrangements is not limited, and a plurality of them may be arranged. The dataset DS includes learning data and data for generating the correct value of each learning data. The dataset DS of this embodiment includes an audio signal s(n), a noise signal e(n), and an RIR signal a(n).

[0019] The input unit 10 sequentially acquires an audio signal s(n), a noise signal e(n), and an RIR signal a(n) from the dataset DS in the dataset storage unit 20, generates combinations of training data and correct values ​​from each acquired signal, and sequentially supplies these combinations of training data and correct values ​​to the inference unit 30. The input unit 10 includes an observation sound generation unit 14 for generating training data, a reverberation time estimation unit 11 for generating correct values ​​for training data, a shortened window generation unit 12, and a correct value generation unit 13.

[0020] The observation sound generation unit 14 acquires the audio signal s(n), the noise signal e(n), and the RIR signal a(n) from the dataset DS, and generates the observation sound y(n) to be used as training data. The observation sound generation unit 14 may, for example, generate the observation sound y(n) using the audio signal s(n), the noise signal e(n), and the RIR signal a(n) using the following equation (1). Here, n (n=1,2,3,...) represents each time point (time series). y(n)=s(n)*a(n)+e(n)···(1)

[0021] The reverberation time estimation unit 11 acquires the RIR signal a(n) from the dataset DS and estimates the reverberation time of the RIR signal a(n). The method used by the reverberation time estimation unit 11 to estimate the reverberation time can be any method that can estimate the reverberation time. For example, as shown in Reference 1 below, the reverberation time estimation unit 11 obtains a linearly approximated linear equation using only the data of the decay curve above the noise floor level, and the time T when that linear equation is -60 dB (any value can be set, but -60 dB is preferable) 60 The reverberation time may be estimated as T. The reverberation time estimation unit 11 then estimates the reverberation time T. 60 The output is sent to the shortened window generation unit 12.

[0022] [Reference 1] Toshiki Hanyu, "Room Acoustic Indices Derived from Impulse Response," Journal of the Acoustical Society of Japan, Vol. 69, No. 6 (2013), pp. 291-296.

[0023] The shortened window generation unit 12 obtains the RIR signal a(n) from the dataset DS and the reverberation time T from the reverberation time estimation unit 11, respectively, and generates a window for shortening the RIR signal a(n) obtained from the dataset DS to a desired reverberation time T'. Next, the shortened window generation unit 12 calculates the attenuation rate q from the reverberation time T of the RIR signal a(n) obtained from the dataset DS and the desired reverberation time T' after shortening. For example, the shortened window generation unit 12 may calculate the attenuation rate q using the following equation (2). Next, the shortened window generation unit 12 generates the shortened window w(n) from the attenuation rate q. For example, the shortened window generation unit 12 may generate the shortened window w(n) using the following equation (3). Here, f represents the sampling frequency, and N1 represents the direct sound arrival time of the impulse response. 60 The shortened window generation unit 12 obtains the RIR signal a(n) from the dataset DS and the reverberation time T from the reverberation time estimation unit 11, respectively, and generates a window for shortening the RIR signal a(n) obtained from the dataset DS to a desired reverberation time T'. 60 Next, the shortened window generation unit 12 calculates the attenuation rate q from the reverberation time T of the RIR signal a(n) obtained from the dataset DS and the desired reverberation time T' after shortening. For example, the shortened window generation unit 12 may calculate the attenuation rate q using the following equation (2). Next, the shortened window generation unit 12 generates the shortened window w(n) from the attenuation rate q. For example, the shortened window generation unit 12 may generate the shortened window w(n) using the following equation (3). Here, f represents the sampling frequency, and N1 represents the direct sound arrival time of the impulse response. 60 The shortened window generation unit 12 calculates the attenuation rate q from the reverberation time T of the RIR signal a(n) obtained from the dataset DS and the desired reverberation time T' after shortening. 60 Next, the shortened window generation unit 12 generates the shortened window w(n) from the attenuation rate q. For example, the shortened window generation unit 12 may generate the shortened window w(n) using the following equation (3). Here, f represents the sampling frequency, and N1 represents the direct sound arrival time of the impulse response. s is assumed to represent the sampling frequency, and N1 is assumed to represent the direct sound arrival time of the impulse response.

Equation

[0024] The correct value generation unit 13 obtains the audio signal s(n) and the RIR signal a(n) from the dataset DS and the shortened window w(n) from the shortened window generation unit 12, respectively. Then, the correct value generation unit 13 multiplies the RIR signal a(n) by the shortened window w(n) to generate the RIR signal a 60 (n) with the desired reverberation time T'. For example, the correct value generation unit 13 may generate the RIR signal a d (n) with the desired reverberation time T' using the following equation (4). Next, the correct value generation unit 13 convolves the audio signal s(n) with the RIR signal a 60 (n) with the desired reverberation time T' to generate the correct value y'(n) corresponding to the training data. For example, the correct value generation unit 13 may generate the correct value y'(n) corresponding to the training data using the following equation (5). d (n) with the desired reverberation time T'. Next, the correct value generation unit 13 convolves the audio signal s(n) with the RIR signal a 60 (n) with the desired reverberation time T' to generate the correct value y'(n) corresponding to the training data. For example, the correct value generation unit 13 may generate the correct value y'(n) corresponding to the training data using the following equation (5). d (n) with the desired reverberation time T' to generate the correct value y'(n) corresponding to the training data. For example, the correct value generation unit 13 may generate the correct value y'(n) corresponding to the training data using the following equation (5). a d (n)=w(n)a(n) ···(4) y´(n)=s(n)*ad (n) ···(5)

[0025] The inference unit 30 obtains an inferred value based on the observed sound y(n) input from the input unit 10 and the inference NN. The inference unit 30 also stores (remembers) the weight parameters 31 used by the inference NN 31. Therefore, the inference unit 30 retrieves the stored weight parameters 31 and performs inference using the inference NN 31 based on the retrieved weight parameters 31 and the input data output from the input unit 10 to obtain an inferred value. The inference unit 30 outputs the inferred value obtained as a result of the inference to the inference evaluation unit 40.

[0026] The inference evaluation unit 40 obtains an evaluation result of the inference value (hereinafter referred to as the "inference evaluation result") based on the inference value input from the inference unit 30 and the correct value obtained by the input unit 10. Specifically, the inference evaluation unit 40 obtains the inference evaluation result by comparing the inference value input from the inference unit 30 and the correct value obtained by the input unit 10. The inference evaluation unit 40 outputs the inference evaluation result to the update unit 50.

[0027] The method by which the inference evaluation unit 40 evaluates the inference values ​​is not limited, but here we assume that a loss function corresponding to the inference values ​​input from the inference unit 30 and the correct values ​​obtained by the input unit 10 is calculated as the inference evaluation result. Here, the loss function corresponding to the inference values ​​and correct values ​​is not limited to a specific function, and may be the same as the loss function used in general neural networks. For example, the loss function corresponding to the inference values ​​and correct values ​​may be the mean squared error based on the difference between the correct values ​​and the inference values.

[0028] The update unit 50 updates the weight parameters 31 of the inference NN 31 based on the inference evaluation results input from the inference evaluation unit 40. This allows the weight parameters 31 of the inference NN 31 to be updated so that the inference values ​​output from the inference unit 30 approach the correct values. The method of updating the weight parameters 31 by the update unit 50 is not limited. For example, the update unit 50 may update the weight parameters 31 of the inference NN 31 using backpropagation. The update unit 50 may also update parameters other than the weight parameters 31 (parameters related to the neural network; for example, other parameters such as bias) for the inference NN 31.

[0029] In the learning device 1, the learning termination condition (i.e., the termination condition for updating the weight parameters 31 by the update unit 50) is not particularly limited and can be any condition indicating that the inference NN 31 has been learned to a certain extent. For example, the update unit 50 may include a condition in the learning termination condition that the value of the loss function is less than a threshold. Alternatively, the update unit 50 may include a condition in the learning termination condition that the change in the value of the loss function is less than a threshold (a condition that the value of the loss function has converged). Furthermore, the update unit 50 may include a condition in the learning termination condition that the weight parameters 31 have been updated a predetermined number of times. Moreover, if the inference evaluation unit 40 calculates accuracy (e.g., speech recognition rate) based on the correct value and the inferred value, the update unit 50 may include a condition in the learning termination condition that the accuracy exceeds a predetermined percentage (e.g., 90%).

[0030] Next, we will describe the hardware configuration of learning device 1.

[0031] Figure 2 is a block diagram showing an example of the hardware configuration of learning device 1.

[0032] Figure 2 shows an example of the hardware configuration when the learning device 1 is configured using software (computer).

[0033] The learning device 1 shown in Figure 2 has a computer 300 on which a program (including the learning program of this embodiment) is installed as a hardware component. The computer 300 may be a computer dedicated to the learning program, or it may be configured to be shared with programs for other functions. Even when the learning device 1 is composed of the computer 300, the functional configuration of the learning device 1 (learning program) can be shown as in Figure 2.

[0034] The computer 300 shown in Figure 2 includes a processor 301, a primary storage unit 302, and a secondary storage unit 303. The primary storage unit 302 is a storage means that functions as a working memory for the processor 301, and can be a high-speed memory such as DRAM (Dynamic Random Access Memory). The secondary storage unit 303 is a storage means that records various data such as the OS (Operating System) and program data (including data for the learning program according to the embodiment), and can be a non-volatile memory such as FLASH® memory, HDD (Hard Disk Drive), or SSD (Solid State Drive). In the computer 300 of this embodiment, when the processor 301 starts up, it reads the OS and program (including the learning program according to the embodiment) recorded in the secondary storage unit 303, loads them onto the primary storage unit 302, and executes them. Note that the specific configuration of the computer 300 is not limited to the configuration in Figure 2, and various configurations can be applied. For example, if the primary storage unit 302 is non-volatile memory, the configuration may exclude the secondary storage unit 303.

[0035] (A-2) Operation of the embodiment Next, the operation of the learning device 1 according to this embodiment (learning method according to this embodiment) will be described.

[0036] Figure 2 is a flowchart illustrating an example of the operation of the learning device 1 according to this embodiment.

[0037] First, the input unit 10 acquires the audio signal s(n), noise signal e(n), and RIR signal a(n) from the dataset DS, and obtains combinations of observed sounds and correct values ​​to be used as training data (S101).

[0038] Next, the inference unit 30 obtains the weight parameters 31 of the inference NN 31 (S102).

[0039] Next, the inference unit 30 performs inference based on the observed sound acquired by the input unit 10 and the inference NN 31, and outputs the inference value obtained by the inference to the inference evaluation unit 40 (S103).

[0040] Next, the inference evaluation unit 40 evaluates the inference values ​​input from the inference unit 30 based on the correct values ​​obtained by the input unit 10, obtains an inference evaluation result, and outputs the obtained inference evaluation result to the update unit 50 (S104). For example, the inference evaluation unit 40 calculates a loss function corresponding to the correct values ​​and inference values ​​as the inference evaluation result.

[0041] Next, the update unit 50 updates the weight parameters 31 of the inference NN 31 based on the inference evaluation results input from the inference evaluation unit 40 (S105). For example, the update unit 50 updates the weight parameters 31 of the inference NN 31 by backpropagation based on the inference evaluation results.

[0042] Then, each time the update unit 50 finishes updating the weight parameters 31 based on the observed sound, it determines whether the learning termination condition has been met (S106). If the update unit 50 determines that the learning termination condition has not been met (if "NO" is the result in S106), the process proceeds to step S101 described above, the input unit 10 acquires the next learning data, and the inference unit 30, the inference evaluation unit 40, and the update unit 50 each execute their respective processes based on the next observed sound. On the other hand, if the update unit 50 determines that the learning termination condition has been met (if "YES" is the result in S106), the learning device 1 terminates the learning process.

[0043] (A-3) Effects of the Embodiment This embodiment can achieve the following effects.

[0044] In this embodiment of the learning device 1, instead of using reverberation times that may have errors compared to the actual reverberation times listed in the dataset DS, the reverberation time before shortening is estimated from the RIR signal a(n), thereby improving the accuracy of the reverberation shortening of the correct values. Specifically, first, when the input unit 10 generates correct values ​​corresponding to the observed sound that will be used as learning data, it obtains the reverberation time between shortening periods from the RIR signal a(n) acquired from the dataset DS. Next, the input unit 10 calculates a shortening ratio based on the reverberation time before shortening and the desired reverberation time, and then shortens the RIR signal a(n) acquired from the dataset DS based on that shortening ratio to calculate the RIR signal a(n) with the desired reverberation time. Next, the input unit 10 calculates the correct values ​​by convolving the shortened RIR signal a(n) with the audio signal s(n) acquired from the dataset DS. As described above, in this embodiment of the learning device 1, by estimating the reverberation time before shortening from the RIR signal a(n), it is possible to calculate the RIR signal a(n) with the desired reverberation time, thereby improving the accuracy of the reverberation shortening of the correct values.

[0045] Based on the above, the learning device 1 can use such a highly accurate RIR signal a(n) for reverberation reduction to train the inference NN31 (weight parameters 32), thereby obtaining an inference NN31 (weight parameters 32) with higher reverberation removal accuracy than conventional methods.

[0046] (B) Other embodiments The present invention is not limited to the embodiments described above, and modified embodiments such as those exemplified below can also be cited.

[0047] (B-1) The inference NN31 (a neural network to which the learned weight parameters 32 have been applied) obtained as a result of learning with the learning device 1 of the above embodiment may be applied to reverberation removal of various signals. For example, in a sound collection device that collects sounds such as human voices, the inference NN31 obtained as a result of learning with the learning device 1 of the above embodiment may be applied as a preprocessing (reverberation removal process). [Explanation of Symbols]

[0048] 1...Learning device, 10...Input unit, 11...Reverberation time estimation unit, 12...Shortened window generation unit, 13...Correct value generation unit, 14...Observed sound generation unit, 20...Dataset storage unit, 30...Inference unit, 31...Weight parameters, 32...Weight parameters, 40...Inference evaluation unit, 50...Update unit, DS...Dataset

Claims

1. An observation sound generation means generates observed sounds that serve as training data by adding a noise signal to a signal obtained by convolving an audio signal and an indoor impulse response signal, Reverberation time estimation means for estimating the reverberation time of the indoor impulse response signal, A shortening window generating means for generating a shortening window to shorten the indoor impulse response signal to a desired shortened reverberation time, A means for generating a correct value corresponding to the observed sound generates a reverberation-shortened indoor impulse response signal in which the reverberation time is the shortened reverberation time by multiplying the indoor impulse response signal by the shortened window, and generates a correct value corresponding to the observed sound by convolving the sound signal into the reverberation-shortened indoor impulse response signal. An inference means comprising a neural network that outputs an inferred value based on the observed sound, An inference evaluation means that obtains an inference evaluation result of the inference value by comparing the inference value with the correct answer value, An update means that updates the parameters of the neural network so that the observed sound approaches the correct value based on the inference evaluation result. A learning device characterized by having the following features.

2. The learning device according to claim 1, characterized in that the reverberation time estimation means obtains a linearly approximated linear equation using only data of attenuation curves above the noise floor level, and estimates the time when the linear equation becomes -60 dB as the reverberation time of the room impulse response signal.

3. The learning device according to claim 1, characterized in that the shortened window generating means determines the attenuation rate from the reverberation time of the indoor impulse response signal and the shortened reverberation time, and generates the shortened window based on the attenuation rate.

4. The learning device according to claim 1, characterized in that the inference evaluation means calculates a loss function corresponding to the inferred value and the correct answer as the inference evaluation result.

5. The learning device according to claim 1, characterized in that the loss function corresponding to the observed sound and the correct value in the inference evaluation means is the mean squared error based on the difference between the correct value and the observed sound.

6. The learning device according to claim 1, characterized in that the update means updates the parameters by backpropagation.

7. Computers, An observation sound generation means generates observed sounds that serve as training data by adding a noise signal to a signal obtained by convolving an audio signal and an indoor impulse response signal, Reverberation time estimation means for estimating the reverberation time of the indoor impulse response signal, A shortening window generating means for generating a shortening window to shorten the indoor impulse response signal to a desired shortened reverberation time, A means for generating a correct value corresponding to the observed sound generates a reverberation-shortened indoor impulse response signal in which the reverberation time is the shortened reverberation time by multiplying the indoor impulse response signal by the shortened window, and generates a correct value corresponding to the observed sound by convolving the sound signal into the reverberation-shortened indoor impulse response signal. An inference means comprising a neural network that outputs an inferred value based on the observed sound, An inference evaluation means that obtains an inference evaluation result of the inference value by comparing the inference value with the correct answer value, An update means that updates the parameters of the neural network so that the observed sound approaches the correct value based on the inference evaluation result. A learning program characterized by its ability to function in this way.

8. The learning method performed by the learning device includes an observation sound generation means, a reverberation time estimation means, a shortened window generation means, a correct value generation means, an inference means, an inference evaluation means, and an update means. The aforementioned observation sound generation means generates observation sounds to be used as training data by adding a noise signal to a signal obtained by convolving an audio signal and an indoor impulse response signal. The reverberation time estimation means estimates the reverberation time of the room impulse response signal, The shortening window generating means generates a shortening window for shortening the indoor impulse response signal to a desired shortened reverberation time. The correct value generation means generates a reverberation-shortened room impulse response signal where the reverberation time is the shortened reverberation time by multiplying the room impulse response signal by the shortened window, and generates a correct value corresponding to the observed sound by convolving the audio signal into the reverberation-shortened room impulse response signal. The inference means includes a neural network that outputs an inferred value based on the observed sound. The inference evaluation means obtains the inference evaluation result of the inference value by comparing the inference value with the correct answer value. The update means updates the parameters of the neural network based on the inference evaluation result so that the observed sound approaches the correct value. A learning method characterized by the following:

Images