Learning device, learning method, and learning program

Abstract

The learning device (100) has an acquisition unit (110) that acquires a clean signal, a mixed signal, and an original domain teacher-trained model (11), an extraction unit (130) that extracts clean features using the clean signal, an estimation unit (140) that estimates a teacher vector representation using the original domain teacher-trained model (11) and the clean features, an extraction unit (150) that extracts mixed features using the mixed signal, an estimation unit (160) that estimates a student vector representation using a student learning model (12) and the mixed features, a calculation unit (170) that calculates a value based on the teacher vector representation and the student vector representation, and a learning unit (180) that learns the student learning model (12) using the value so that the estimation of the student learning model (12) approaches the estimation of the original domain teacher-trained model (11).

Description

[Technical field]

[0001] The present disclosure relates to a learning device, a learning method, and a learning program. [Background technology]

[0002] A trained model may be used in signal processing. The trained model may be trained in multiple environments. Hereinafter, the first training environment is called the original domain. The training environment after the original domain is called the application domain. A trained model obtained by training in the original domain may be retrained in the application domain. When the original domain and the application domain are different, the estimation accuracy of the trained model obtained by training in the original domain may be significantly reduced. Such a phenomenon is called catastrophic forgetting. Therefore, according to Non-Patent Document 1, the neural network is not updated as a whole, but is partially updated. This prevents catastrophic forgetting. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2016 / 143125 [Non-patent literature]

[0004] [Non-Patent Document 1] Yuki Takashima et al., "Preventing Catastrophic Forgetting by Partial Parameter Update in Continuous Training of End-to-End Speech Recognition," Onkoron Autumn, 2022 [Non-Patent Document 2] Ashish Vaswani et al. “Attention Is All You Need”, in Proc, IPS, 2017 [Non-Patent Document 3] Ryo Aihara et al. “Deep clustering-based single-channel speech separation and recent advances”, Acoust.Sci.& Tech.41,2, 2020 [Non-Patent Document 4] Ethan Perez et al. “FiLM:Visual Reasoning with a General Conditioning Layer”, in proc.AAAI, 20018 Summary of the Invention [Problem to be solved by the invention]

[0005] In the partial update proposed in Non-Patent Document 1, signals, which are learning data, are associated with labels, and supervised learning is performed. However, the task of associating the signals with labels is performed manually. This places a heavy burden on the human beings. Therefore, the method proposed in Non-Patent Document 1 cannot be said to be optimal.

[0006] The goal of this disclosure is to prevent catastrophic forgetting without using labels. [Means for solving the problem]

[0007] According to one aspect of the present disclosure, there is provided a learning device, which performs learning in an application domain, which is a subsequent learning environment of an original domain, which is a learning environment. The learning device includes an acquisition unit that acquires a first clean signal, a mixed signal that is a signal obtained by mixing the first clean signal and a noise signal, and an original domain teacher-trained model that is a trained model obtained by learning in the original domain; a first extraction unit that uses the first clean signal to extract clean features that are features of the first clean signal; a first estimation unit that uses the original domain teacher-trained model and the clean features to estimate a teacher vector representation that is a vector representation of information in which the clean features are aggregated; a second extraction unit that uses the mixed signal to extract mixed features that are features of the mixed signal; a student learning model whose initial state is the same as that of the original domain teacher-trained model, and a second estimation unit that uses the mixed features to estimate a student vector representation that is a vector representation of information in which the mixed features are aggregated; a calculation unit that calculates a value based on the teacher vector representation and the student vector representation; and a learning unit that uses the value to train the student learning model so that an estimation of the student learning model approaches an estimation of the original domain teacher-trained model. Effect of the Invention

[0008] According to the present disclosure, catastrophic forgetting can be prevented without using labels. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 illustrates a signal processing system according to a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating hardware included in the learning device of the first embodiment. [Diagram 3] 2 is a block diagram showing the functions of the learning device according to the first embodiment. FIG. [Figure 4] 4 is a flowchart showing an example of processing executed by the learning device of the first embodiment. [Diagram 5] 1 is a block diagram showing functions of an estimation device according to a first embodiment. [Figure 6] 4 is a flowchart showing an example of processing executed by the estimation device of the first embodiment. [Figure 7] FIG. 11 is a block diagram showing the functions of a learning device according to a second embodiment. [Figure 8] 11 is a flowchart showing an example of processing executed by the learning device of the second embodiment. [Figure 9] FIG. 11 is a block diagram showing the functions of a learning device according to a third embodiment. [Figure 10] 13 is a flowchart showing an example of processing executed by a learning device according to a third embodiment. [Figure 11] FIG. 11 is a block diagram showing the functions of an estimation device according to a third embodiment. [Figure 12] 13 is a flowchart showing an example of processing executed by an estimation device according to a third embodiment. [Figure 13] FIG. 13 is a block diagram showing the functions of a learning device according to a fourth embodiment. [Figure 14] 13 is a flowchart showing an example of processing executed by the learning device of the fourth embodiment. [Figure 15] FIG. 13 is a block diagram showing the functions of an estimation device according to a fourth embodiment. [Figure 16] 13 is a flowchart showing an example of processing executed by the estimation device of the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment will be described with reference to the drawings. The following embodiment is merely an example, and various modifications are possible within the scope of the present disclosure.

[0011] Embodiment 1 1 is a diagram showing a signal processing system according to embodiment 1. The signal processing system includes a learning device 100 and an estimation device 200. The learning device 100 is a device that executes a learning method.

[0012] Next, the hardware of the learning device 100 will be described. 2 is a diagram showing hardware included in the learning device of embodiment 1. The learning device 100 is a computer. The learning device 100 includes a processor 101, a volatile storage device 102, and a non-volatile storage device 103.

[0013] The processor 101 controls the entire learning device 100. For example, the processor 101 is a central processing unit (CPU) or a field programmable gate array (FPGA). The processor 101 may be a multiprocessor. Furthermore, the learning device 100 may include a processing circuit.

[0014] The volatile storage device 102 is a main storage device of the learning device 100. For example, the volatile storage device 102 is a random access memory (RAM). The nonvolatile storage device 103 is an auxiliary storage device of the learning device 100. For example, the nonvolatile storage device 103 is a hard disk drive (HDD) or a solid state drive (SSD). In addition, the storage area secured in the volatile storage device 102 or the nonvolatile storage device 103 is called a storage unit. Similar to the learning device 100, the estimation device 200 includes a processor, a volatile storage device, and a non-volatile storage device.

[0015] The learning phase and the utilization phase will be described below. In the learning phase, the learning device 100 will be described. In the utilization phase, the estimation device 200 will be described. <Learning Phase>

[0016] Next, the functions of the learning device 100 will be described. 3 is a block diagram showing the functions of the learning device of the first embodiment. The learning device 100 includes an acquisition unit 110, a mixing unit 120, an extraction unit 130, an estimation unit 140, an extraction unit 150, an estimation unit 160, a calculation unit 170, a learning unit 180, and an output unit 190. The extraction unit 130 is also referred to as a first extraction unit. The estimation unit 140 is also referred to as a first estimation unit. The extraction unit 150 is also referred to as a second extraction unit. The estimation unit 160 is also referred to as a second estimation unit.

[0017] A part or all of the acquisition unit 110, the mixing unit 120, the extraction unit 130, the estimation unit 140, the extraction unit 150, the estimation unit 160, the calculation unit 170, the learning unit 180, and the output unit 190 may be realized by a processing circuit. Also, a part or all of the acquisition unit 110, the mixing unit 120, the extraction unit 130, the estimation unit 140, the extraction unit 150, the estimation unit 160, the calculation unit 170, the learning unit 180, and the output unit 190 may be realized as a module of a program executed by the processor 101. For example, the program executed by the processor 101 is also called a learning program. For example, the learning program is recorded on a recording medium.

[0018] The acquiring unit 110 acquires the clean signal and the mixed signal. For example, the acquiring unit 110 acquires the clean signal and the mixed signal from a storage unit. Also, for example, the acquiring unit 110 acquires the clean signal and the mixed signal from an external device. Note that the external device is a device that exists outside the learning device 100. For example, the external device is a cloud server. Also, the illustration of the external device is omitted.

[0019] The clean signal is a signal that does not include a noise signal. The clean signal is also called a first clean signal. The mixed signal is a signal that is a mixture of the clean signal and the noise signal.

[0020] Here, as described below, the acquiring unit 110 may acquire the mixed signal generated by the mixing unit 120. When the acquiring unit 110 acquires the mixed signal from a storage unit or an external device, the learning device 100 does not need to include the mixing unit 120.

[0021] Moreover, the acquisition unit 110 acquires the original domain teacher-trained model 11 from a storage unit or an external device. The original domain teacher-trained model 11 is a trained model obtained by training in the original domain. The original domain teacher-trained model 11 is a neural network composed of multiple layers. The original domain teacher-trained model 11 may use a method similar to LSTM (Long Short-Term Memory), a method combining one-dimensional convolution operations, or the Transformer described in Non-Patent Document 2. The number of layers is not important.

[0022] The initial state of the student learning model 12 depicted in FIG. 3 is the same as the original domain teacher-trained model 11. As described below, the student learning model 12 learns. Through learning, the student learning model 12 becomes in a different state from the original domain teacher-trained model 11. Furthermore, learning is performed in a learning environment after the original domain. Therefore, the learning environment of the learning performed by the learning device 100 is the application domain. Furthermore, the application domain is an environment different from the original domain.

[0023] The mixer 120 generates a mixed signal by using the clean signal and the noise signal. Note that the noise signal is stored in a storage unit or an external device, for example.

[0024] The extraction unit 130 uses the clean signal to extract a clean feature that is a feature of the clean signal. For example, the extraction unit 130 extracts, as the clean feature, a time series of a power spectrum obtained by performing a Short-Term Fourier Transform (STFT) on the clean signal. The clean feature is also called a first clean feature.

[0025] The estimation unit 140 estimates a teacher vector representation by using the original domain teacher-trained model 11 and the clean feature. In detail, when the estimation unit 140 inputs the clean feature to the original domain teacher-trained model 11, the original domain teacher-trained model 11 outputs a teacher vector representation.

[0026] Here, the teacher vector representation is a vector representation of information in which clean features are aggregated. Note that aggregation may be rephrased as degeneration. Therefore, the teacher vector representation may be expressed as a vector representation of information in which clean features are degenerated.

[0027] The extraction unit 150 uses the mixed signal to extract a mixed feature that is a feature of the mixed signal. For example, the extraction unit 150 extracts, as the mixed feature, a time series of a power spectrum obtained by performing a short-time Fourier transform on the mixed signal.

[0028] The estimation unit 160 estimates a student vector representation using the student learning model 12 and the mixed feature. In detail, when the estimation unit 160 inputs the mixed feature to the student learning model 12, the student learning model 12 outputs a student vector representation. Note that the student vector representation is a vector representation of information in which the mixed feature is aggregated.

[0029] The calculation unit 170 calculates a value based on the teacher vector representation and the student vector representation. Specifically, the calculation unit 170 calculates the value L p Calculate.

[0030]

number

[0031] h N ^t is the teacher vector representation. h N ^s is the student vector representation. "^" (hat) is the symbol indicating exponentiation. When "P=1", the value L1 represents the L1 norm. When "P=2", the value L2 represents the L2 norm.

[0032] The loss function is described in Non-Patent Document 3. p may be called the error or loss.

[0033] The learning unit 180 adjusts the calculated value (i.e., the value L) so that the estimation of the student learning model 12 approaches the estimation of the original domain teacher trained model 11. p ) to train the student learning model 12. In other words, the learning unit 180 trains the student learning model 12 using the value so that the vector representation output by the student learning model 12 approaches the vector representation output by the original domain teacher trained model 11. For example, the learning unit 180 executes processing using the student learning model 12, the calculated value, and an optimization method such as Adam (Adaptive moment), and then adjusts the weight coefficient of the student learning model 12 based on the backpropagation method. Also, for example, the learning unit 180 trains the student learning model 12 so that the calculated value becomes equal to or less than a predetermined threshold.

[0034] The output unit 190 outputs the clean features, the teacher vector representation, the mixed features, the student vector representation, and the values ​​calculated by the calculation unit 170. For example, the output unit 190 outputs the clean features, etc. to a display connectable to the learning device 100. For example, the clean features, etc. are output to the display, allowing the user to recognize the learning status.

[0035] Next, the process executed by the learning device 100 will be described with reference to a flowchart. FIG. 4 is a flowchart illustrating an example of processing executed by the learning device according to the first embodiment. (Step S11) The acquisition unit 110 acquires a clean signal, a mixed signal, and an original domain supervised trained model 11. (Step S12) The extraction unit 130 extracts clean features using the clean signal. (Step S13) The estimation unit 140 estimates a teacher vector representation by using the original domain teacher-trained model 11 and the clean feature. (Step S14) The extraction unit 150 extracts a mixed feature by using the mixed signal.

[0036] (Step S15) The estimation unit 160 estimates a student vector representation using the student learning model 12 and the mixture feature. (Step S16) The calculation unit 170 calculates a value based on the teacher vector representation and the student vector representation. (Step S17) The learning unit 180 learns the student learning model 12 using the calculated value. (Step S18) The output unit 190 outputs the clean feature, the teacher vector representation, the mixed feature, the student vector representation, and the value calculated by the calculation unit 170. The execution order of steps S12 to S15 may be different from that shown in FIG.

[0037] The learning device 100 may repeat the process of FIG. 4 using a different clean signal and a different mixed signal. For example, the learning device 100 repeatedly learns the student learning model 12 until the value output by the calculation unit 170 becomes equal to or less than a predetermined threshold. In this way, the student learning model 12 is learned multiple times. For example, after the learning is completed, the learning device 100 transmits the student learning model 12 to the estimation device 200.

[0038] According to the first embodiment, the learning device 100 learns the student learning model 12 in the application domain. Therefore, the influence of the application domain is reflected in the estimation of the student learning model 12. Furthermore, the learning device 100 learns the student learning model 12 so that the estimation of the student learning model 12 approaches the estimation of the original domain teacher-trained model 11. Therefore, the estimation of the student learning model 12 is influenced by the original domain. Furthermore, the learning device 100 performs learning without using labels. Therefore, the learning device 100 can prevent catastrophic forgetting without using labels.

[0039] Next, an example of the utilization phase in the first embodiment will be described. <Utilization phase> 5 is a block diagram showing functions of the estimation device according to embodiment 1. The estimation device 200 includes an acquisition unit 210, an extraction unit 220, an estimation unit 230, and an estimation unit 240.

[0040] A part or all of the acquiring unit 210, the extracting unit 220, the estimating unit 230, and the estimating unit 240 may be realized by a processing circuit included in the estimation device 200. In addition, a part or all of the acquiring unit 210, the extracting unit 220, the estimating unit 230, and the estimating unit 240 may be realized as a program module executed by a processor included in the estimation device 200. The acquiring unit 210 acquires the mixed signal. For example, the acquiring unit 210 acquires the mixed signal from a volatile storage device or a non-volatile storage device included in the estimation device 200. Also, for example, the acquiring unit 210 acquires the mixed signal from an external device.

[0041] The acquisition unit 210 acquires the student learning model 12 from the learning device 100. Here, the student learning model 12 may be called an encoding neural network. The acquisition unit 210 acquires the trained model 21. For example, the acquisition unit 210 acquires the trained model 21 from a volatile storage device or a non-volatile storage device included in the estimation device 200. Also, for example, the acquisition unit 210 acquires the trained model 21 from an external device. Here, the trained model 21 may be called a decoding neural network.

[0042] The extraction unit 220 uses the mixed signal to extract a mixed feature that is a feature of the mixed signal. For example, the extraction unit 220 extracts, as the mixed feature, a time series of a power spectrum obtained by performing a short-time Fourier transform on the mixed signal.

[0043] The estimation unit 230 estimates a vector representation using the student learning model 12 and the mixed feature. In detail, when the estimation unit 230 inputs the mixed feature to the student learning model 12, the student learning model 12 outputs the vector representation. Note that the vector representation is a vector representation of information in which the mixed feature is aggregated.

[0044] The estimation unit 240 estimates a label using the trained model 21 and the vector representation. The label is information estimated based on information in which mixed features are aggregated. For example, when the mixed signal is a voice and the estimation device 200 performs speech recognition, the label is a character string indicating the content of the speech. Also, when the mixed signal is a voice and the estimation device 200 performs emotion estimation, the label is information indicating an emotion estimated from the voice.

[0045] Next, the process executed by the estimation device 200 will be described with reference to a flowchart. FIG. 6 is a flowchart illustrating an example of processing executed by the estimation device according to the first embodiment. (Step S21) The acquisition unit 210 acquires a mixed signal, a student learning model 12, and a trained model 21. (Step S22) The extraction unit 220 extracts a mixed feature by using the mixed signal. (Step S23) The estimation unit 230 estimates a vector representation using the student learning model 12 and the mixed feature. (Step S24) The estimation unit 240 estimates a label by using the trained model 21 and the vector representation.

[0046] Embodiment 2 Next, a description will be given of embodiment 2. In embodiment 2, differences from embodiment 1 will be mainly described. In embodiment 2, descriptions of matters common to embodiment 1 will be omitted.

[0047] <Learning Phase> 7 is a block diagram showing the functions of the learning device of the second embodiment. The acquisition unit 110 acquires the weight 13 corresponding to the noise signal included in the mixed signal. Specifically, the acquisition unit 110 acquires the weight 13 from a storage unit or an external device. For example, if the noise signal is the sound of running water, the weight 13 is a weight corresponding to the sound of running water. For example, if the noise signal is the sound of a running car, the weight 13 is a weight corresponding to the sound of a running car.

[0048] The calculation unit 170 calculates a value based on the weight 13, the teacher vector representation, and the student vector representation. Specifically, the calculation unit 170 calculates the value L p Calculate.

[0049]

number

[0050] Note that weight 13 is λ x λ x is the weight corresponding to the x-th noise signal among the Y types of noise signals. xN ^s is the student vector representation estimated using the mixture features of the mixture signal including the x-th noise signal. xN ^t is the teacher vector expression corresponding to the student vector expression. “^” (hat) is the symbol indicating exponentiation.

[0051] Next, the process executed by the learning device 100 will be described with reference to a flowchart. Fig. 8 is a flowchart showing an example of processing executed by the learning device of embodiment 2. The processing in Fig. 8 differs from the processing in Fig. 4 in that steps S11a and 16a are executed. Therefore, steps S11a and 16a will be described in Fig. 8. Descriptions of processing other than steps S11a and 16a will be omitted.

[0052] (Step S11a) The acquisition unit 110 acquires a clean signal, a mixed signal, an original domain supervised trained model 11, and weights 13. (Step S16a) The calculation unit 170 calculates a value based on the weight 13, the teacher vector representation, and the student vector representation.

[0053] According to the second embodiment, the learning device 100 uses the weight 13, thereby enabling the learning device 100 to appropriately perform learning according to the noise signal.

[0054] Embodiment 3 Next, a description will be given of embodiment 3. In embodiment 3, differences from embodiment 1 will be mainly described. In embodiment 3, descriptions of matters common to embodiment 1 will be omitted.

[0055] <Learning Phase>

[0056] 9 is a block diagram showing functions of the learning device of embodiment 3. The learning device 100 further includes an extraction unit 191 and an estimation unit 192. The extraction unit 191 is also referred to as a third extraction unit. The estimation unit 192 is also referred to as a third estimation unit. A part or all of the extraction unit 191 and the estimation unit 192 may be realized by a processing circuit. Also, a part or all of the extraction unit 191 and the estimation unit 192 may be realized as a program module executed by the processor 101.

[0057] The acquisition unit 110 acquires a noise signal from a storage unit or an external device. The noise signal is the same as the noise signal contained in the mixed signal. The acquisition unit 110 acquires the noise-learned model 14 from a storage unit or an external device.

[0058] The extraction unit 191 uses the noise signal to extract a noise feature amount that is a feature amount of the noise signal. For example, the extraction unit 191 extracts, as the noise feature amount, a time series of a power spectrum obtained by performing a short-time Fourier transform on the noise signal. The estimation unit 192 estimates a noise vector representation using the noise-learned model 14 and the noise feature amount. In detail, when the estimation unit 192 inputs the noise feature amount to the noise-learned model 14, the noise-learned model 14 outputs the noise vector representation. Note that the noise vector representation is a vector representation of information in which the noise feature amount is aggregated.

[0059] The estimation unit 160 estimates the student vector representation using the student learning model 12, the mixed feature, and the noise vector representation. In detail, when the estimation unit 160 inputs the mixed feature and the noise vector representation to the student learning model 12, the student learning model 12 outputs the student vector representation. Here, by inputting the noise vector representation, the student learning model 12 can identify which information of the mixed feature is related to the noise signal. Therefore, the student learning model 12 estimates the student vector representation while identifying information related to the noise signal.

[0060] Furthermore, the estimation unit 160 may use the method described in Non-Patent Document 4 when inputting the mixed feature amount and the noise vector representation to the student learning model 12.

[0061] Next, the process executed by the learning device 100 will be described with reference to a flowchart. Fig. 10 is a flowchart showing an example of processing executed by the learning device of embodiment 3. The processing in Fig. 10 differs from the processing in Fig. 4 in that steps S11b, 14a, 14b, and 15a are executed. Therefore, in Fig. 10, steps S11b, 14a, 14b, and 15a will be described. And, description of the processing other than steps S11b, 14a, 14b, and 15a will be omitted.

[0062] (Step S11b) The acquisition unit 110 acquires the clean signal, the mixed signal, the original domain teacher-trained model 11, the noise signal, and the noise-trained model . (Step S14a) The extraction unit 191 extracts a noise feature amount by using a noise signal. (Step S14b) The estimation unit 192 estimates a noise vector representation by using the noise-learned model 14 and the noise feature amount. (Step S15a) The estimation unit 160 estimates a student vector representation using the student learning model 12, the mixed feature, and the noise vector representation. The execution order of steps S12 to S15a may be different from that of FIG.

[0063] According to the third embodiment, the robustness of the student learning model 12 is improved. Furthermore, the student learning model 12 can estimate noise more accurately through learning.

[0064] Next, an example of the utilization phase in the third embodiment will be described. <Utilization phase> 11 is a block diagram showing functions of the estimation device of the embodiment 3. The estimation device 200 further includes a detection unit 250, an extraction unit 260, and an estimation unit 270.

[0065] A part or all of the detection unit 250, the extraction unit 260, and the estimation unit 270 may be realized by a processing circuit included in the estimation device 200. In addition, a part or all of the detection unit 250, the extraction unit 260, and the estimation unit 270 may be realized as a program module executed by a processor included in the estimation device 200.

[0066] The acquisition unit 210 further acquires the noise-learned model 14 from a volatile storage device or a non-volatile storage device included in the estimation device 200 .

[0067] The detection unit 250 detects a noise signal included in the mixed signal. For example, the detection unit 250 detects the noise signal using the method described in Patent Document 1. Also, for example, the detection unit 250 detects the noise signal using the power of the mixed signal and a threshold value.

[0068] The extraction unit 260 uses the noise signal to extract a noise feature amount that is a feature amount of the noise signal. For example, the extraction unit 260 extracts, as the noise feature amount, a time series of a power spectrum obtained by performing a short-time Fourier transform on the noise signal.

[0069] The estimation unit 270 estimates a noise vector representation using the noise-learned model 14 and the noise feature amount. In detail, when the estimation unit 270 inputs the noise feature amount to the noise-learned model 14, the noise-learned model 14 outputs the noise vector representation. Note that the noise vector representation is a vector representation of information in which the noise feature amount is aggregated.

[0070] The estimation unit 230 estimates the vector representation using the student learning model 12, the mixed feature, and the noise vector representation. In detail, when the estimation unit 230 inputs the mixed feature and the noise vector representation to the student learning model 12, the student learning model 12 outputs the vector representation.

[0071] Next, the process executed by the estimation device 200 will be described with reference to a flowchart. Fig. 12 is a flowchart showing an example of processing executed by the estimation device of the third embodiment. The processing in Fig. 12 differs from the processing in Fig. 6 in that steps S21a, 22a, 22b, 22c, and 23a are executed. Therefore, in Fig. 12, steps S21a, 22a, 22b, 22c, and 23a will be described. Then, description of the processing other than steps S21a, 22a, 22b, 22c, and 23a will be omitted.

[0072] (Step S21a) The acquisition unit 210 acquires the mixed signal, the student learning model 12, the trained model 21, and the noise trained model 14. (Step S22a) The detection unit 250 detects a noise signal contained in the mixed signal. (Step S22b) The extraction unit 260 extracts a noise feature amount by using the noise signal. (Step S22c) The estimation unit 270 estimates a noise vector representation by using the noise-learned model 14 and the noise feature amount. (Step S23a) The estimation unit 230 estimates a vector representation using the student learning model 12, the mixed feature, and the noise vector representation.

[0073] Embodiment 4 Next, a fourth embodiment will be described. In the fourth embodiment, differences from the first embodiment will be mainly described. In the fourth embodiment, descriptions of the same matters as the first embodiment will be omitted.

[0074] <Learning Phase> 13 is a block diagram showing functions of the learning device of embodiment 4. The learning device 100 further includes an extraction unit 193 and an estimation unit 194. The extraction unit 193 is also referred to as a fourth extraction unit. The estimation unit 194 is also referred to as a fourth estimation unit. A part or all of the extraction unit 193 and the estimation unit 194 may be realized by a processing circuit. Also, a part or all of the extraction unit 193 and the estimation unit 194 may be realized as a program module executed by the processor 101.

[0075] Here, the clean signal input to the extraction section 130 is called a first clean signal. The acquisition unit 110 acquires the second clean signal from a storage unit or an external device. The second clean signal is a signal different from the first clean signal. For example, the first clean signal and the second clean signal are sound signals uttered by the same speaker. The acquisition unit 110 acquires the clean trained model 15 from a storage unit or an external device.

[0076] The extraction unit 193 uses the second clean signal to extract a second clean feature that is a feature of the second clean signal. For example, the extraction unit 193 extracts, as the second clean feature, a time series of a power spectrum obtained by performing a short-time Fourier transform on the second clean signal.

[0077] The estimation unit 194 estimates a clean vector representation using the clean trained model 15 and the second clean feature. In detail, when the estimation unit 194 inputs the second clean feature to the clean trained model 15, the clean trained model 15 outputs a clean vector representation. Note that the clean vector representation is a vector representation of information in which the second clean feature is aggregated.

[0078] The estimation unit 160 estimates the student vector representation using the student learning model 12, the mixed feature, and the clean vector representation. In detail, when the estimation unit 160 inputs the mixed feature and the clean vector representation to the student learning model 12, the student learning model 12 outputs the student vector representation. Here, by inputting the clean vector representation, the student learning model 12 can identify which information of the mixed feature is related to the clean signal. Therefore, the student learning model 12 estimates the student vector representation while identifying information related to the clean signal.

[0079] Furthermore, the estimation unit 160 may use the method described in Non-Patent Document 4 when inputting the mixed feature amount and the clean vector representation to the student learning model 12.

[0080] Next, the process executed by the learning device 100 will be described with reference to a flowchart. Fig. 14 is a flowchart showing an example of processing executed by the learning device of embodiment 4. The processing in Fig. 14 differs from the processing in Fig. 4 in that steps S11c, 14c, 14d, and 15b are executed. Therefore, in Fig. 14, steps S11c, 14c, 14d, and 15b will be described. And, description of the processing other than steps S11c, 14c, 14d, and 15b will be omitted.

[0081] (Step S11c) The acquisition unit 110 acquires the first clean signal, the mixed signal, the original domain supervised trained model 11, the second clean signal, and the clean trained model 15. (Step S14c) The extraction unit 193 extracts a second clean feature by using the second clean signal. (Step S14d) The estimation unit 194 estimates a clean vector representation by using the clean trained model 15 and the second clean feature. (Step S15b) The estimation unit 160 estimates a student vector representation using the student learning model 12, the mixed feature, and the clean vector representation. The execution order of steps S12 to 15b may be different from that shown in FIG.

[0082] According to the fourth embodiment, the robustness of the student learning model 12 is improved. Furthermore, the student learning model 12 can estimate a clean signal more accurately through learning. Furthermore, the learning device 100 uses different clean signals to train the student learning model 12. Therefore, even if different speech contents are input, the student learning model 12 can easily estimate which is the clean signal.

[0083] Next, an example of the utilization phase is shown. <Utilization phase> 15 is a block diagram showing the functions of the estimation device of the embodiment 4. The estimation device 200 further includes an extraction unit 280 and an estimation unit 290.

[0084] A part or all of the extraction unit 280 and the estimation unit 290 may be realized by a processing circuit included in the estimation device 200. In addition, a part or all of the extraction unit 280 and the estimation unit 290 may be realized as a program module executed by a processor included in the estimation device 200.

[0085] The acquisition unit 210 further acquires the clean trained model 15 from a volatile storage device or a non-volatile storage device included in the estimation device 200. The acquiring unit 210 further acquires a clean signal from a volatile storage device or a non-volatile storage device included in the estimation apparatus 200. Note that the clean signal is different from the clean signal included in the mixed signal acquired by the acquiring unit 210.

[0086] The extraction unit 280 extracts a clean feature, which is a feature of the clean signal, by using a clean signal different from the clean signal contained in the mixed signal. For example, the extraction unit 280 extracts, as the clean feature, a time series of a power spectrum obtained by performing a short-time Fourier transform on the clean signal.

[0087] The estimation unit 290 estimates a clean vector representation using the clean trained model 15 and the clean features. In detail, when the estimation unit 290 inputs the clean features to the clean trained model 15, the clean trained model 15 outputs a clean vector representation. Note that the clean vector representation is a vector representation of information in which the clean features are aggregated.

[0088] The estimation unit 230 estimates the vector representation using the student learning model 12, the mixed feature, and the clean vector representation. In detail, when the estimation unit 230 inputs the mixed feature and the clean vector representation to the student learning model 12, the student learning model 12 outputs the vector representation.

[0089] Next, the process executed by the estimation device 200 will be described with reference to a flowchart. Fig. 16 is a flowchart showing an example of processing executed by the estimation device of the fourth embodiment. The processing in Fig. 16 differs from the processing in Fig. 6 in that steps S21b, 22d, 22e, and 23b are executed. Therefore, steps S21b, 22d, 22e, and 23b will be described in Fig. 16. Descriptions of processing other than steps S21b, 22d, 22e, and 23b will be omitted.

[0090] (Step S21b) The acquisition unit 210 acquires the mixed signal, the student learning model 12, the trained model 21, the clean signal, and the clean trained model 15. (Step S22d) The extraction unit 280 extracts clean features using the clean signal. (Step S22e) The estimation unit 290 estimates a clean vector representation by using the clean trained model 15 and the clean features. (Step S23b) The estimation unit 230 estimates a vector representation using the student learning model 12, the mixed feature, and the clean vector representation.

[0091] The features of each of the embodiments described above can be combined with each other as appropriate. [Explanation of symbols]

[0092] 11 original domain teacher trained model, 12 student trained model, 13 weights, 14 noise trained model, 15 clean trained model, 21 trained model, 100 learning device, 101 processor, 102 volatile storage device, 103 non-volatile storage device, 110 acquisition unit, 120 mixing unit, 130 extraction unit, 140 estimation unit, 150 extraction unit, 160 estimation unit, 170 calculation unit, 180 learning unit, 190 output unit, 191 extraction unit, 192 estimation unit, 193 extraction unit, 194 estimation unit, 200 estimation device, 210 acquisition unit, 220 extraction unit, 230 estimation unit, 240 estimation unit, 250 detection unit, 260 extraction unit, 270 estimation section, 280 extraction section, 290 estimation section.

Claims

1. A learning device that performs learning in an application domain that is a learning environment subsequent to an original domain that is a learning environment, an acquisition unit that acquires a first clean signal, a mixed signal that is a signal obtained by mixing the first clean signal and a noise signal, and an original domain teacher trained model that is a trained model obtained by training in the original domain; a first extraction unit that uses the first clean signal to extract a clean feature that is a feature of the first clean signal; a first estimation unit that estimates a teacher vector representation that is a vector representation of information in which the clean features are aggregated, using the original domain teacher-trained model and the clean features; a second extraction unit that uses the mixed signal to extract a mixed feature that is a feature of the mixed signal; a second estimation unit that estimates a student vector representation, which is a vector representation of information in which the mixed features are aggregated, by using a student learning model whose initial state is the same as the original domain teacher-trained model and the mixed features; a calculation unit for calculating a value based on the teacher vector representation and the student vector representation; a learning unit that uses the values ​​to train the student learning model so that the estimate of the student learning model approaches an estimate of the original domain teacher-trained model; A learning device having the above configuration.

2. The acquisition unit acquires a weight corresponding to the noise signal included in the mixed signal, the calculation unit calculates a value based on the weights, the teacher vector representation, and the student vector representation; The learning device according to claim 1 .

3. A third extraction unit; and A third estimation unit; and and The acquisition unit acquires the noise signal and a noise learned model, The third extraction unit extracts, using the noise signal, a noise feature amount that is a feature amount of the noise signal; the third estimation unit estimates a noise vector representation, which is a vector representation of information in which the noise features are aggregated, by using the noise-learned model and the noise features; The second estimation unit estimates the student vector representation by using the student learning model, the mixture feature, and the noise vector representation. The learning device according to claim 1 .

4. A fourth extraction unit; and A fourth estimation unit; and and The acquisition unit acquires a second clean signal and a clean trained model; the fourth extraction unit extracts, by using the second clean signal, a second clean feature that is a feature of the second clean signal; The fourth estimation unit estimates a clean vector representation, which is a vector representation of information in which the second clean features are aggregated, by using the clean trained model and the second clean features; The second estimation unit estimates the student vector representation by using the student learning model, the mixed feature, and the clean vector representation. The learning device according to claim 1 .

5. An output unit that outputs the clean feature, the teacher vector representation, the mixed feature, the student vector representation, and the value. The learning device according to any one of claims 1 to 4.

6. A learning device that performs learning in an application domain, which is a learning environment subsequent to the original domain, which is a learning environment, Obtaining a first clean signal, a mixed signal which is a signal obtained by mixing the first clean signal and a noise signal, and an original domain teacher-trained model which is a trained model obtained by training in the original domain, extracting clean features which are features of the first clean signal using the first clean signal, estimating a teacher vector representation which is a vector representation of information in which the clean features are aggregated using the original domain teacher-trained model and the clean features, extracting mixed features which are features of the mixed signal using the mixed signal, and estimating a student vector representation which is a vector representation of information in which the mixed features are aggregated using a student model whose initial state is the same as that of the original domain teacher-trained model and the mixed features; calculating a value based on the teacher vector representation and the student vector representation; training the student learning model using the values ​​such that the estimate of the student learning model approaches the estimate of the original domain teacher-trained model; How to learn.

7. A computer that performs learning in the application domain, which is the learning environment after the original domain, which is the learning environment, Obtaining a first clean signal, a mixed signal which is a signal obtained by mixing the first clean signal and a noise signal, and an original domain teacher-trained model which is a trained model obtained by training in the original domain, extracting clean features which are features of the first clean signal using the first clean signal, estimating a teacher vector representation which is a vector representation of information in which the clean features are aggregated using the original domain teacher-trained model and the clean features, extracting mixed features which are features of the mixed signal using the mixed signal, and estimating a student vector representation which is a vector representation of information in which the mixed features are aggregated using a student model whose initial state is the same as that of the original domain teacher-trained model and the mixed features; calculating a value based on the teacher vector representation and the student vector representation; training the student learning model using the values ​​such that the estimate of the student learning model approaches the estimate of the original domain teacher-trained model; A learning program that executes processing.

Images