Audio signal processing method, audio signal processing device, and program

Abstract

The present invention improves speech recognition performance. In a speech signal processing method according to an embodiment of the present invention, an output value indicating whether or not speech enhancement should be performed on an observed signal in which speech of another speaker or noise overlaps speech of a target speaker or indicating a level at which speech enhancement should be performed is acquired. Using the acquired output value, the ratio between the observed signal and an enhanced signal generated by speech enhancement is assessed under a predetermined condition to determine an input signal to be used for speech recognition.

Description

[Technical Field]

[0001] The present invention relates to a speech recognition technology, and more particularly to a technology for switching between an emphasis signal and an observation signal. [Background technology]

[0002] In recent years, advances in deep learning technology have improved the performance of speech recognition. However, situations where speech recognition is still difficult include mixed speech from multiple people (overlapping speech). To address this issue, the following technologies have been devised:

[0003] Blind source separation enables speech recognition by separating speech that is difficult to recognize as a mixed speech into the speech of each speaker (see, for example, Non-Patent Document 1).

[0004] Target speaker extraction uses pre-registered utterances from the target speaker as auxiliary information and extracts only the pre-registered speaker's voice from the mixed voice (see, for example, Non-Patent Document 2). Since the extracted voice contains only the target speaker's voice, speech recognition is possible. However, removing undesirable sounds can distort the target speaker's voice. In other words, speech enhancement can actually degrade speech recognition performance.

[0005] A method has been proposed in which the strength of speech enhancement is reduced for sections where no overlapping speech occurs (see, for example, Non-Patent Document 3). This is because, although speech enhancement is effective for overlapping speech, applying speech enhancement to non-overlapping speech (solo speech by the target speaker) is likely to degrade speech recognition. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Yu, Dong, et al. "Permutation invariant training of deep models for speaker-independent multi-talker speech separation." 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2017. [Non-patent document 2] Zmolikova, Katerina, et al. "SpeakerBeam: Speaker aware neural network for target speaker extraction in speech mixtures." IEEE Journal of Selected Topics in Signal Processing 13.4 (2019): 800-814. [Non-patent document 3] Wang, Quan, et al. "VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognition." arXiv preprint arXiv:2009.04323 (2020). Summary of the Invention [Problem to be solved by the invention]

[0007] However, the effectiveness of speech enhancement is not determined solely by the presence or absence of overlapping speech. For example, even in overlapping speech sections, if there is a large difference in volume between the target speaker and another interfering speaker, speech recognition tends to recognize only the target speaker's speech with the larger volume. In this case, it is thought that a higher speech recognition rate can be obtained by performing speech recognition on the observed signal as is without speech enhancement. Similarly, even in non-overlapping speech sections, it is also possible that a higher speech recognition rate can be obtained by using an input with speech enhancement. In view of the above-mentioned problems, the object of the present invention is to provide a technology that can improve speech recognition performance. [Means for solving the problem]

[0008] In order to solve the above problem, one aspect of the present invention provides a speech signal processing method that acquires an output value indicating whether or not speech enhancement should be performed on an observed signal in which the speech of a target speaker is overlapped with the speech of another speaker or noise, or the degree to which speech enhancement should be performed, and uses the acquired output value to determine the ratio between the observed signal and the enhanced signal generated by speech enhancement under specified conditions, thereby determining the input signal to be used for speech recognition. [Effects of the Invention]

[0009] According to the present invention, speech recognition performance can be improved. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram showing an example of a functional configuration of an audio signal processing device according to an embodiment of the present invention; [Figure 2] 1 is a diagram showing an example of a processing flow of an audio signal processing method in an audio signal processing device according to an embodiment of the present invention. [Figure 3] FIG. 2 is a diagram showing an example of the functional configuration of a voice recognition input determination unit 13. [Figure 4] FIG. 3 is a diagram showing an example of a processing flow of a method for determining a voice recognition input in the voice recognition input determination unit 13. [Figure 5] FIG. 2 is a diagram showing an example of the functional configuration of a switching model learning device. [Figure 6] FIG. 10 is a diagram showing an example of a processing flow of a method for creating a trained model in a switching model learning device. [Figure 7] FIG. 2 is a diagram showing an example of the functional configuration of a switching label creating device. [Figure 8] FIG. 10 is a diagram showing an example of a processing flow of a switching label creation method in a switching label creation device. [Figure 9] 3A and 3B are diagrams showing examples of performance results of speech recognition using the speech signal processing device 1. [Figure 10] FIG. 2 is a diagram illustrating an example of the functional configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0011] First, the notation used in this specification will be explained.

[0012] <Notation> The symbol "~" (superscript tilde) used in text should normally be written directly above the character immediately following it, but due to limitations in text notation, it is written immediately before the character in question. In mathematical formulas, these symbols are written in their proper position, i.e., directly above the character. For example, "~S" is expressed in a mathematical formula as follows:

number

number

[0013] FIG. 1 shows an example of the functional configuration of a speech signal processing device according to an embodiment of the present invention. The speech signal processing device 1 shown in FIG. 1 includes a speech enhancement unit 11, a switching model unit 12, a speech recognition input determination unit 13, and a speech recognition unit 14. The speech signal processing method of the embodiment is realized by the speech signal processing device 1 performing the processes of the steps illustrated in FIG. 2. As will be described later, one aspect of the speech signal processing device 1 switches whether to use the observed signal or the enhanced signal as input for speech recognition using the output result of the trained switching model unit 12. This makes it possible to improve speech recognition performance compared to when speech recognition is always performed after speech enhancement or when the observed signal is always recognized.

[0014] Hereinafter, with reference to FIG. 2, an audio signal processing method executed by the audio signal processing device 1 of the embodiment will be described.

[0015] In step S11, the speech enhancement unit 11 performs speech enhancement processing. That is, the speech enhancement unit 11 acquires an observed signal as input, extracts only the desired speech from the acquired observed signal using a known speech enhancement technique, and executes the speech enhancement processing. A known target speaker extraction technique, for example, can be used as a method for extracting the desired speech. As shown in FIG. 1, the target speaker extraction technique is a technique in which the speech enhancement unit 11 acquires auxiliary information about the target speaker in addition to acquiring the observed signal, thereby extracting only the speech of the target speaker from the observed signal. The auxiliary information about the target speaker can be, for example, an utterance previously registered by the target speaker. The input signal acquired by the speech enhancement unit 11 can be the speech waveform itself obtained from the observed signal, or features extracted from the observed signal. The speech enhancement unit 11 outputs the speech signal subjected to speech enhancement processing (hereinafter also referred to as the "enhanced signal") to the switching model unit 12.

[0016] In step S12, the switching model unit 12 receives the enhancement signal from the speech enhancement unit 11. The switching model unit 12 also receives an observation signal, which is a speech signal that has not been subjected to speech enhancement processing by the speech enhancement unit 11. The observation signal is configured to be input directly to the switching model unit 12, similar to the input to the speech enhancement unit 11 as shown in FIG. 1. Since the speech enhancement unit 11 acquires the observation signal in step S11, the speech enhancement unit 11 may be configured to output the observation signal that has not been subjected to speech enhancement processing to the switching model unit 12.

[0017] The switching model unit 12 is a trained model trained using known techniques such as deep neural networks. The signal received as input by the switching model unit 12 can be a waveform domain signal. Alternatively, the signal can be one that has undergone feature extraction. The switching model unit 12 receives at least one of an observed signal and an enhanced signal as input, and outputs whether or to what degree speech enhancement should be performed from the perspective of speech recognition performance. The output of the switching model unit 12, ^k, is a value (estimated value) calculated by the switching model unit 12, and can be a scalar value ranging from 0 to 1, for example, as defined by the following equation:

number

[0018] The switching model unit 12 outputs the calculated result, k, to the speech recognition input determination unit 13. The learning method of the switching model unit 12 will be described later.

[0019] In step S13, the speech recognition input determination unit 13 receives the output value ^k from the switching model unit 12 and ^S from the speech enhancement unit 11, and determines the input for speech recognition.

[0020] Here, if the input to the speech recognition unit 14 is ~S, then the input ~S to the speech recognition unit 14 is determined to be either the emphasis signal ^S or the observed signal Y, as defined by the following equation. In equation (2), λ is a preset value in the range of 0<λ<1, such as 0.5. In this embodiment, the method of determining either the emphasis signal ^S or the observed signal Y as the input ~S to the speech recognition unit 14 is referred to as the "hardware method."

number

number

number

[0021] In step S14, the speech recognition unit 14 performs speech recognition processing on the signal ∼S received from the speech recognition input determination unit 13. The speech recognition unit 14 may also receive the enhanced signal ^S obtained by the speech enhancement unit 11 and the observed signal Y containing speech from other speakers, noise, etc., and perform speech recognition processing on each of them. The speech recognition unit 14 outputs text information that is the speech recognition result corresponding to each speech signal. The speech recognition unit 14 can use known speech recognition technology.

[0022] <Processing of the voice recognition input determination unit 13> A specific processing flow of the voice recognition input determination process (FIG. 2, step S13) in the voice recognition input determination unit 13 in an embodiment of the present invention will be described. FIG. 3 is a diagram showing an example of the functional configuration of the voice recognition input determination unit 13. The voice recognition input determination unit 13 has an output acquisition unit 131, a judgment unit 132, and a determination unit 133. The voice recognition input determination unit 13 determines the input for voice recognition by performing the processing of each step exemplified in FIG. 4. Hereinafter, a voice recognition input determination method executed by the voice recognition input determination unit 13 will be described with reference to FIG. 4.

[0023] In step S131, the output acquisition unit 131 receives the output value ^k from the switching model unit 12. The output acquisition unit 131 sends the received output value ^k to the judgment unit 132. In step S132, the judgment unit 132 performs a predetermined judgment using the received output value ^k and outputs the judgment result to the decision unit 133. For example, when a hardware method is adopted, the predetermined judgment involves determining the magnitude of ^k and outputting only one of the signals ^S or Y to the decision unit 133 based on the judgment using the above equations (1) and (2). When a software method is adopted, in addition to the value of ^k, the signals ^S and Y are output to the decision unit 133. As another example, the decision unit 133 may be configured to output information indicating whether the software method or the hardware method is to be adopted, the value of ^k, and the signals ^S and Y to the decision unit 133. In step S133, the decision unit 133 determines the input signal ^S using the information received from the judgment unit 132 and the above equations (1) to (3).

[0024] <How to learn the switching model> The learning method for the switching model unit 12 in the embodiment of the present invention is performed using the switching model learning device 2 illustrated in Fig. 5. The switching model learning device 2 has a switching model unit 21 and an optimization unit 22. The switching model learning device 2 performs learning by subjecting a model created by the switching model unit 21 to optimization processing in the optimization unit 22. After learning by the switching model learning device 2, the switching model unit 21 is used as the switching model unit 12 as a trained model used in the audio signal processing device 1. The switching model learning device 2 performs processing of each step illustrated in Fig. 6, thereby realizing the learning process for the switching model. Hereinafter, the learning method for the switching model of the embodiment will be described with reference to Fig. 6.

[0025] In step S21, the switching model unit 21 receives the observed signal and the emphasis signal for learning, constructs the basic configuration of the switching model, and outputs this model (switching model under learning) to the optimization unit 22.

[0026] In step S22, the optimization unit 22 receives the model received from the switching model unit 21 and the switching label created by the switching label creation device 3, which will be described later, optimizes the parameters of the model, and returns the results to the switching model unit 21. The process between the model construction by the switching model unit 21 and the parameter optimization by the optimization unit 22 may be configured to be a loop process, with the optimization being completed by repeating these processes. In either case, when the optimization is completed and the parameters are determined, the contents are reflected in the switching model unit 21, and the switching model is completed.

[0027] The specific method of optimization by the optimization unit 22 is as follows: The optimization unit 22 calculates a loss function between a switching label k generated by the switching label creation device 3 (described later) and the output value ^k calculated by the switching model unit 21, and optimizes the model parameters included in the switching model unit 21 by minimizing the loss function.

[0028] As the loss function, for example, the well-known cross-entropy loss defined by the following equation can be used.

number

[0029] The SIR and SNR estimates of the observed signal by the switching model unit 21 are defined as ^SIR and ^SNR, respectively. That is, ^SIR is the output value of the switching model unit 21 when SIR is input as the observed signal, and ^SNR is the output value of the switching model unit 21 when SNR is input as the observed signal. If the speech of the target speaker is S, the speech of the interfering speaker is I, and the noise is N, then SIR and SNR are defined by the following equations.

number

number

number

number

number

[0030] <How to create a switching label> A method for creating switching labels in an embodiment of the present invention is performed using a switching label creation device illustrated in FIG. 7. The switching label creation device 3 has a trained speech enhancement unit 31, a trained speech recognition unit 32, a recognition performance calculation unit 33, and a switching label generation unit 34. The speech enhancement unit 31 has the same function as the speech enhancement unit 11 in FIG. 1. The speech recognition unit 32 has the same function as the speech recognition unit 14 in FIG. 1. The switching label creation device 3 generates switching labels using paired data of an observed signal, auxiliary information about the target speaker, and a transcription of the target speaker's speech. The switching label creation method of the embodiment is realized by the switching label creation device 3 performing processing of each step illustrated in FIG. 8. A method for creating matching labels used in the switching model learning device 2 will be described below with reference to FIG. 8.

[0031] In step S31, the speech enhancement unit 31 performs speech enhancement processing. That is, the speech enhancement unit 31 receives an observed signal as input, extracts only the desired speech from the obtained observed signal using a known speech enhancement technique, and performs speech enhancement processing. At this time, auxiliary information about the target speaker can be, for example, utterances registered in advance by the target speaker. The speech enhancement unit 31 outputs the enhanced signal that has been subjected to speech enhancement processing to the speech recognition unit 32.

[0032] In step S32, the speech recognition unit 32 receives an observation signal containing the speech of other speakers, noise, etc., in addition to the emphasized signal obtained from the speech enhancement unit 31. By performing speech recognition processing on each of the received observation signals, text information that is the speech recognition result corresponding to each speech signal is output to the recognition performance calculation unit 33.

[0033] In step S33, the recognition performance calculation unit 33 receives the speech recognition result corresponding to the emphasis signal received from the speech recognition unit 32, the speech recognition result for the observed signal, and the transcription of the target speaker's speech. The transcription of the target speaker's speech corresponds to correct information about the speech signal that is the target of speech recognition. The recognition performance calculation unit 33 calculates the speech recognition performance using the two speech recognition results and the transcription. The speech recognition performance can be calculated using known speech recognition performance evaluation criteria such as character error rate. The recognition performance calculation unit 33 outputs the calculated speech recognition performance result to the switching label generation unit 34.

[0034] In step S34, the switching label generation unit 34 generates a switching label k that the optimization unit 22 shown in Fig. 5 uses as a teacher label for optimizing the switching model unit 21, based on the speech recognition performance for the emphasis signal and the speech recognition performance for the observation signal acquired from the recognition performance calculation unit 33. The switching label k is a label that indicates whether the observation signal or the emphasis signal had higher speech recognition performance, and is defined by the following equation, for example.

number

[0035] The switching label k may be determined more flexibly than a binary label, as described below. That is, the speech recognition performance of the observed signal and the emphasis signal may be compared, and the switching label k may be calculated based on the performance difference. For example, the switching label k may be determined more flexibly than a binary label by using the following definition equation, where T is a temperature parameter.

number

[0036] Through the above process, paired data is generated for four types of information: observed signals, auxiliary information about the target speaker, emphasis signals, and switching labels.

[0037] <Performance results> FIG. 9 shows an example of performance results of speech recognition using the speech signal processing device 1. FIG. 9 shows results for five conditions (a) to (e) for the input target to the speech recognition unit 14. Here, condition (a) indicates an observed signal, condition (b) indicates an enhanced signal, condition (c) indicates the use of the hardware method of this embodiment and a model without multi-task learning, condition (d) indicates the use of the hardware method of this embodiment and a model with multi-task learning, and condition (e) indicates the use of the software method of this embodiment and a model with multi-task learning. In FIG. 9, SIR and SNR are evaluated for each of conditions (a) to (e) using three levels. That is, the results are shown for speech recognition processing with SIR changed to 0, 10, and 20, and SNR changed to three levels (0, 10, and 20). The performance results of speech recognition under each condition are shown using the character error rate (CER) standard, with a smaller number indicating higher speech recognition performance. In Figure 9, the same speech recognition unit was used for speech recognition, so it is possible to directly compare the character recognition results under each condition. Figure 9(f) shows the performance improvement rate for the results under condition (e) relative to the results under condition (b). Also, in Figure 9, the results under conditions (c) to (e) are compared with the results under condition (b). Results that are better than the results under condition (b) are circled (◯), results that are equivalent to the results under condition (b) are circled (△), and results that are worse than the results under condition (b) are circled (□).

[0038] As shown in Fig. 9, when the hardware method of condition (c) in this embodiment and a model without multi-task learning were used, the only case in which the results were inferior to the enhanced signal of condition (b) was when SIR = 0 and SNR = 0. The only four cases in which the results were equivalent were when SIR = 0 and SNR = 10 and 20, respectively. The remaining four cases showed better performance than the enhanced signal of condition (b). The average performance was 1.7% better than the enhanced signal of condition (b).

[0039] When the hardware method of this embodiment and a model with multitask learning were used, the only case in which the results were inferior to the enhanced signal under condition (b) was when SIR = 0 and SNR = 0. The only two cases in which the results were equivalent were when SIR = 0 and SNR = 10 and 20, and the remaining six cases were superior to the enhanced signal under condition (b). The average result was 1.9% better than the enhanced signal under condition (b).

[0040] When the software method of condition (e) in this embodiment and a model with multitask learning were used, the results were inferior to the enhanced signal of condition (b) in two cases: SIR = 0 and SNR = 10, 20. There were no cases where the results were equivalent, and the remaining seven cases produced results superior to the enhanced signal of condition (b). The average performance was 2.6% better than the enhanced signal of condition (b).

[0041] As shown in Figure 9(f), the performance improvement rate under condition (e) compared to the results under condition (b) showed a 3% performance drop when SIR = 0 and SNR was 10 or 20, but the other seven cases showed better performance results than condition (b). Specifically, when SIR was 10, an improvement of 8% to 32% was observed, and when SIR was 20, an improvement of 25% to 42% was observed. The overall average recognition rate also showed an improvement of 19%. As such, it can be seen that the performance of speech recognition is improved when the speech recognition input determination unit 13 of this embodiment is used compared to the performance of speech recognition using an emphasis signal.

[0042] The above has described a method for processing a speech signal according to an embodiment of the present invention. By using the technique of this embodiment, the present invention uses ^k output by the switching model unit 12 to selectively use the emphasis signal and the observation signal, thereby preventing performance degradation due to speech emphasis and improving speech recognition performance. This makes it possible to appropriately determine whether or not to emphasize speech when speech emphasis is not necessary even in sections where overlapping speech occurs, or when speech emphasis is necessary even in sections where no overlapping speech occurs. This makes it possible to appropriately switch between the emphasis signal and the observation signal, resulting in improved speech recognition performance.

[0043] Additionally, in the model with multitask learning for estimating SIR and SNR shown in this embodiment, higher classification performance can be obtained by taking into account SIR and SNR, which are closely related to speech enhancement.

[0044] Furthermore, by weighting and adding the emphasis signal and the observed signal using ^k, which is the output of the switching model unit 12, it becomes possible to determine the input speech taking into account the uncertainty of the identification model.

[0045] The various processes described above may not only be executed in chronological order as described, but may also be executed in parallel or individually depending on the processing capacity of the device executing the processes or as necessary. Needless to say, other modifications are possible within the scope of the present invention.

[0046] [Programs, recording media] The various processes described above can be implemented by loading a program that executes each step of the above method into the recording unit 2020 of the computer 2000 shown in Figure 10, and operating the control unit 2010, input unit 2030, output unit 2040, display unit 2050, etc.

[0047] The program describing the processing contents can be recorded on a computer-readable recording medium, which may be, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, or any other suitable recording medium.

[0048] The program may be distributed, for example, by selling, transferring, lending, etc. a portable recording medium such as a DVD or CD-ROM on which the program is recorded. Furthermore, the program may be stored in a storage device of a server computer, and then transferred from the server computer to another computer via a network, thereby distributing the program.

[0049] A computer that executes such a program may first temporarily store the program recorded on a portable recording medium or transferred from a server computer in its own storage device. Then, when executing a process, the computer reads the program stored on its own recording medium and executes the process in accordance with the read program. Alternatively, the computer may read the program directly from a portable recording medium and execute the process in accordance with the program. Furthermore, the computer may execute the process in accordance with the received program each time a program is transferred from a server computer to the computer. Alternatively, the server computer may not transfer the program to the computer, but may execute the process through a so-called ASP (Application Service Provider) service, which realizes the processing function by issuing an execution instruction and obtaining the results. In this embodiment, the program includes information used for processing by a computer that is equivalent to a program (such as data that is not a direct instruction to the computer but has properties that define computer processing).

[0050] Furthermore, in this embodiment, the device is configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware. [Explanation of symbols]

[0051] 1 Audio signal processing device 11,31 Speech enhancement unit 12,21 Switching model section 13 Voice recognition input determination unit 14,32 Speech recognition unit 2 Switching label creation device 3 Switching model learning device 22 Optimization Section 33 Recognition performance calculation section 34 Switching Label Generation Unit 131 Output Acquisition Unit 132 Judgment section 133 Decision Section

Claims

1. obtaining an output value indicating whether speech enhancement should be performed on an observed signal in which speech of a target speaker is overlapped with speech of another speaker or noise, or indicating the degree to which the speech enhancement should be performed; Using the acquired output value, a ratio between the observed signal and the enhanced signal generated by the speech enhancement is determined under a predetermined condition to determine an input signal to be used for speech recognition.

1. A method for processing an audio signal, comprising: The predetermined condition is expressed by the following equation, where the output value is ^k, the emphasis signal is ^S, the observation signal is Y, the input signal is ∼S, and λ is a value set in advance in the range of 0<λ<1. [Equation 15] [0016] is defined as the output value is an output value output by a trained model, the trained model receives at least one of the observed signal and the enhancement signal as input, and outputs whether or not the speech enhancement should be performed in terms of speech recognition performance, or a degree to which the speech enhancement should be performed; The trained model is expressed by the following equation, where L is the loss coefficient and k is the teacher label used in generating the trained model. [Equation 17] It is trained to minimize L, which is the calculation result defined by In the observed signal, the true value of the ratio between the target speaker's voice and the other speaker's voice is SIR, the true value of the ratio between the target speaker's voice and the noise is SNR, the output value of the trained model when the SIR is input is ^SIR, and the output value of the trained model when the SNR is input is ^SNR. Using parameters α and β, [Equation 18] [Equation 19] [Equation 20] The calculation result L multi defined as above is used as the loss coefficient. How audio signals are processed.

2. obtaining an output value indicating whether speech enhancement should be performed on an observed signal in which speech of a target speaker is overlapped with speech of another speaker or noise, or indicating the degree to which the speech enhancement should be performed; Using the acquired output value, a ratio between the observed signal and the enhanced signal generated by the speech enhancement is determined under a predetermined condition to determine an input signal to be used for speech recognition.

1. A method for processing an audio signal, comprising: The predetermined condition is expressed by the following equation, where the output value is ^k, the emphasis signal is ^S, the observation signal is Y, and the input signal is ∼S. [0000] is defined as the output value is an output value output by a trained model, the trained model receives at least one of the observed signal and the enhancement signal as input, and outputs whether or not the speech enhancement should be performed in terms of speech recognition performance, or a degree to which the speech enhancement should be performed; The trained model is expressed by the following equation, where L is the loss coefficient and k is the teacher label used in generating the trained model. [Equation 22] It is trained to minimize L, which is the calculation result defined by In the observed signal, the true value of the ratio between the target speaker's voice and the other speaker's voice is SIR, the true value of the ratio between the target speaker's voice and the noise is SNR, the output value of the trained model when the SIR is input is ^SIR, and the output value of the trained model when the SNR is input is ^SNR. Using parameters α and β, [Equation 23] [0000] [Equation 25] The calculation result L multi defined as above is used as the loss coefficient. How audio signals are processed.

3. an acquisition unit that acquires an output value indicating whether speech enhancement should be performed on an observed signal in which speech of a target speaker is overlapped with speech of another speaker or noise, or indicating the degree to which the speech enhancement should be performed; a determination unit that determines a ratio of the observed signal to the emphasized signal generated by the speech emphasis process under a predetermined condition using the output value acquired by the acquisition unit, and determines an input signal to be used for speech recognition; An audio signal processing device comprising: The predetermined condition is expressed by the following equation, where the output value is ^k, the emphasis signal is ^S, the observation signal is Y, the input signal is ∼S, and λ is a value set in advance in the range of 0<λ<1. [Equation 26] [0000] is defined as the output value is an output value output by a trained model, the trained model receives at least one of the observed signal and the enhancement signal as input, and outputs whether or not the speech enhancement should be performed in terms of speech recognition performance, or a degree to which the speech enhancement should be performed; The trained model is expressed by the following equation, where L is the loss coefficient and k is the teacher label used in generating the trained model. [0000] It is trained to minimize L, which is the calculation result defined by In the observed signal, the true value of the ratio between the target speaker's voice and the other speaker's voice is SIR, the true value of the ratio between the target speaker's voice and the noise is SNR, the output value of the trained model when the SIR is input is ^SIR, and the output value of the trained model when the SNR is input is ^SNR. Using parameters α and β, [0000] [Equation 30] [Equation 31] The calculation result L multi defined as above is used as the loss coefficient. Audio signal processing device.

4. an acquisition unit that acquires an output value indicating whether speech enhancement should be performed on an observed signal in which speech of a target speaker is overlapped with speech of another speaker or noise, or indicating the degree to which the speech enhancement should be performed; a determination unit that determines a ratio of the observed signal to the emphasized signal generated by the speech emphasis process under a predetermined condition using the output value acquired by the acquisition unit, and determines an input signal to be used for speech recognition; An audio signal processing device comprising: The predetermined condition is expressed by the following equation, where the output value is ^k, the emphasis signal is ^S, the observation signal is Y, and the input signal is ∼S. [Equation 32] is defined as the output value is an output value output by a trained model, the trained model receives at least one of the observed signal and the enhancement signal as input, and outputs whether or not the speech enhancement should be performed in terms of speech recognition performance, or a degree to which the speech enhancement should be performed; The trained model is expressed by the following equation, where L is the loss coefficient and k is the teacher label used in generating the trained model. [Equation 33] It is trained to minimize L, which is the calculation result defined by In the observed signal, the true value of the ratio between the target speaker's voice and the other speaker's voice is SIR, the true value of the ratio between the target speaker's voice and the noise is SNR, the output value of the trained model when the SIR is input is ^SIR, and the output value of the trained model when the SNR is input is ^SNR. Using parameters α and β, [Equation 34] [Equation 35] [Equation 36] The calculation result L multi defined as above is used as the loss coefficient. Audio signal processing device.

5. 3. A program for causing a computer to execute the signal processing method according to claim 1.

Images