Signal processing device, signal processing method, and signal processing program

Abstract

This speech recognition input assessment unit (102) has: an SIR-SNR acquisition unit (1021) that, from mixed speech in which the speech of another speaker overlaps the speech of a target speaker, acquires a signal-to-interference ratio (SIR), which is the ratio of the target speech and the interfering speaker's speech in the mixed speech, and a signal-to-noise ratio (SNR), which is the ratio of the target speech and noise, and / or acquires the mixed speech; and an assessment unit (1022) that, on the basis of the SIR and SNR, and / or on the basis of the mixed speech, assesses that speech based on at least one of the mixed speech and emphasized speech, in which the mixed speech is emphasized, is the speech to be used in speech recognition.

Description

[Technical field]

[0001] The present invention relates to a signal processing device, a signal processing method, and a signal processing program. [Background technology]

[0002] With the development of deep learning technology, the accuracy of speech recognition has improved, and the scope of application of speech recognition is gradually expanding. However, it is still difficult to recognize when multiple voices are spoken at the same time.

[0003] Therefore, various techniques have been devised to deal with overlapping voices. Blind source separation is one of these approaches, which separates a mixed sound into sounds originating from each sound source contained in the mixed sound. Blind source separation can enable speech recognition by separating a speech that is difficult to recognize as a mixed sound into the speech of each speaker (for example, see Non-Patent Document 1).

[0004] Also, a target speaker extraction technology has been proposed. The target speaker extraction technology is a technology that uses utterances preregistered by a target speaker as auxiliary information and acquires only the voice of the preregistered speaker from a mixed sound. This target speaker extraction technology has the advantage that it does not require prior information on the number of speakers contained in the mixed sound, and is a practically useful technology (for example, see Non-Patent Document 2). Since the voice extracted using the target speaker extraction technology contains only the voice of the target speaker, speech recognition is possible. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Dong Yu, et al, “PERMUTATION INVARIANT TRAINING OF DEEP MODELS FOR SPEAKER-INDEPENDENT MULTI-TALKER SPEECH SEPARATION”, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2017. [Non-Patent Document 2] Zmolikova Katerina, et al, “SpeakerBeam: Speaker Aware Neural Network for Target Speaker Extraction in Speech Mixtures”, IEEE Journal of Selected Topics in Signal Processing 13.4 (2019): 800-814. [Non-Patent Document 3] Quan Wang, et al, “VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognition”, arXiv preprint arXiv:2009.04323 (2020). Summary of the Invention [Problem to be solved by the invention]

[0006] As mentioned above, speech enhancement technology is a technology that can remove undesirable sounds and extract only the target speaker's voice when the target speaker's voice overlaps with the voices of other speakers or background noise. However, when removing undesirable sounds, the desired target speaker's voice may be distorted at the same time.

[0007] When speech recognition is performed on speech to which speech enhancement technology has been applied, this distortion of the target speaker's voice may cause degradation of speech recognition performance. Such degradation of speech recognition performance is a major problem in practical operation when target speaker extraction is performed on real speech that does not necessarily overlap with the voices of other speakers.

[0008] On the other hand, since speech recognition works with little performance degradation in the presence of a certain level of noise, depending on the level and type of noise, speech recognition performance may be better without target speaker extraction. In other words, when comparing the effect of noise reduction by applying speech enhancement technology with the adverse effect of the associated distortion, it is desirable to perform noise reduction if the former is greater, and it is desirable not to perform speech enhancement if the latter is greater.

[0009] For example, Non-Patent Document 3 proposes a method of introducing a binary classifier that distinguishes whether the type of noise is the voice of another speaker or other noise, and performing speech enhancement only when the type of noise is the voice of another speaker, based on the knowledge that speech enhancement is effective when the noise is the voice of another speaker (speech noise), while speech enhancement is highly likely to degrade speech recognition when the noise is not speech (non-speech noise). In other words, it proposes that speech enhancement is not performed when the voice of another speaker does not exist in the voice of the target speaker, and is performed only when there is overlap.

[0010] However, even when multiple speakers are speaking overlappingly, speech enhancement may be degraded if the performance degradation caused by the presence of the interfering speakers' voices is outweighed by the performance degradation caused by the distortion caused by removing them.

[0011] The present invention has been made in consideration of the above, and aims to provide a signal processing device, a signal processing method, and a signal processing program that can appropriately determine at least one of a mixed voice and an emphasized voice that emphasizes the mixed voice as an input voice that can improve the accuracy of voice recognition when the voice of a target speaker is overlapped with the voice of another speaker. [Means for solving the problem]

[0012] In order to solve the above-mentioned problems and achieve the object, the signal processing device of the present invention is characterized by having an acquisition unit that acquires at least one of SIR (Signal to Interference Ratio), which is the ratio of target speech to interfering speaker speech in the mixed speech, and SNR (Signal to Noise Ratio), which is the ratio of target speech to noise in the mixed speech, from a mixed speech in which the speech of a target speaker is overlapped with the speech of another speaker, and the mixed speech, and a determination unit that determines, based on at least one of the SIR, SNR, and the mixed speech, a speech based on at least one of the mixed speech and an emphasized speech that emphasizes the mixed speech, as a speech to be used for speech recognition. Effect of the Invention

[0013] According to the present invention, when the voice of a target speaker overlaps with the voice of another speaker, it is possible to appropriately determine at least one of a mixed voice and an emphasized voice that emphasizes the mixed voice as an input voice that can improve the accuracy of voice recognition. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a signal processing device according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of the voice recognition input determination unit illustrated in FIG. [Diagram 3] FIG. 3 is a diagram showing speech recognition performance for emphasized speech obtained by emphasizing mixed speech. [Figure 4] FIG. 4 is a diagram showing the speech recognition performance for mixed speech. [Diagram 5] FIG. 5 is a flowchart showing a processing procedure of signal processing according to the embodiment. [Figure 6] FIG. 6 is a flowchart illustrating the procedure of the voice recognition input determination process shown in FIG. [Figure 7]FIG. 7 is a diagram showing a speech recognition result when the signal processing device according to the embodiment is applied. [Figure 8] FIG. 8 is a diagram showing a performance improvement rate of the speech recognition result using the signal processing device according to the embodiment, compared to the speech recognition result using emphasized speech. [Figure 9] FIG. 9 is a diagram illustrating an example of a computer that realizes a signal processing device by executing a program. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are indicated by the same reference numerals.

[0016] [Embodiment Mode] Even when the voice of a target speaker overlaps with the voice of another speaker, there may be cases where it is better not to perform voice enhancement. For example, when the amplitude of the voice of the interfering speaker is smaller than that of the target speaker, speech recognition may be able to recognize the voice of the target speaker to a certain extent without voice enhancement. In addition, it has been confirmed that when non-voice noise exists in addition to the voice of the interfering speaker, the effect of distortion caused by voice enhancement tends to be large. In such cases, the adverse effect of voice enhancement is large, and it may be more advantageous for speech recognition to recognize the original mixed voice.

[0017] Therefore, in the signal processing device according to the embodiment, a mixed voice in which the voice of a target speaker overlaps with the voice of another speaker is targeted, and a voice based on at least one of the mixed voice and an emphasized voice obtained by emphasizing the mixed voice is determined as an input voice that can improve the accuracy of voice recognition. In the signal processing device according to the embodiment, a voice based on at least one of the mixed voice and the emphasized voice is determined based on at least one of an SNR (Signal to Noise Ratio) which is a ratio of the target voice to noise in the mixed voice, an SIR (Signal to Interference Ratio) which is a ratio of the target voice to an interfering speaker voice in the mixed voice, and the mixed voice. In the signal processing device according to the embodiment, a voice based on at least one of the determined mixed voice and the emphasized voice is performed to improve the accuracy of voice recognition.

[0018] [Signal processing device] Next, a signal processing device according to an embodiment will be described below. Fig. 1 is a diagram illustrating an example of the configuration of a signal processing device according to an embodiment.

[0019] The signal processing device 100 is realized, for example, by loading a predetermined program into a computer including a ROM (Read Only Memory), a RAM (Random Access Memory), a CPU (Central Processing Unit), etc., and executing the predetermined program by the CPU. The signal processing device 100 also has a communication interface for transmitting and receiving various information to and from other devices connected via a wired connection or a network, etc. The signal processing device 100 has a voice emphasizing unit 101, a voice recognition input determining unit 102, and a voice recognition unit 103.

[0020] The speech enhancement unit 101 extracts a desired speech from the input mixed speech by using a known speech enhancement technique. A known sound source separation technique or a known target speaker extraction technique can be applied as the processing of the speech enhancement unit 101. In the sound source separation technique, the input mixed speech is separated into each speaker, and the speech of each speaker is independently provided to the speech recognition input judgment unit 102. In the target speaker extraction technique, auxiliary information regarding the target speaker must also be provided as an input. As the auxiliary information regarding the target speaker, for example, an utterance registered in advance by the target speaker is used. The auxiliary information regarding the target speaker may be held by the signal processing device 100, or may be input from an external device that can communicate with the signal processing device 100.

[0021] The voice emphasizing unit 101 extracts emphasized voice by emphasizing the mixed voice. For example, when a mixed voice of the voices of a target speaker and other interfering speakers is input to the voice emphasizing unit 101 as the mixed voice, the voice emphasizing unit 101 extracts the voice of the target speaker who uttered the pre-registered utterance from the mixed voice. The voice emphasizing unit 101 inputs the extracted voice as emphasized voice to an SIR-SNR acquisition unit 1021 (described later) of the voice recognition input judgment unit 102. The input mixed voice and emphasized voice may be voice that has been subjected to some conversion such as feature extraction, or may be a voice waveform.

[0022] The voice recognition input determination unit 102 determines, based on at least one of the SNR of the mixed voice, the SIR of the mixed voice, and the mixed voice, a voice based on at least one of the mixed voice and an emphasized voice obtained by emphasizing the mixed voice, as a voice to be used for voice recognition. The voice recognition input determination unit 102 inputs the determined voice based on at least one of the mixed voice and the emphasized voice to the voice recognition unit 103. The voice recognition input determination unit 102 determines, for an utterance to be input to voice recognition, a voice based on at least one of the emphasized voice output from the voice emphasis unit 101 and the mixed voice input to the signal processing device 100, on an utterance basis or on a fixed section basis such as a frame.

[0023] Specifically, the voice recognition input determination unit 102 performs voice determination by hardware. For example, the voice recognition input determination unit 102 inputs only one of the emphasized voice output from the voice emphasis unit 101 and the mixed voice input to the signal processing device 100 to the voice recognition unit 103 by "1" or "0".

[0024] The voice recognition input determination unit 102 can also perform voice determination using software. For example, the voice recognition input determination unit 102 weights one of the emphasized voice output from the voice emphasizing unit 101 and the mixed voice input to the signal processing device 100 by, for example, “0.3”, and weights the other by “0.7”, and inputs the weighted mixed voice and emphasized voice to the voice recognition unit 103.

[0025] The voice recognition unit 103 performs voice recognition processing using the voice based on at least one of the mixed voice and the emphasized voice determined by the voice recognition input determination unit 102. A known voice recognition technology can be applied to the voice recognition processing of the voice recognition unit 103.

[0026] [SIR, SNR] Here, the calculation formula for SIR is shown in formula (1), and the calculation formula for SNR is shown in formula (2). In formulas (1) and (2), S is the voice of the target speaker, I is the voice of the interfering speaker, and N is noise.

[0027]

number

[0028]

number

[0029] Fig. 3 is a diagram showing speech recognition performance for emphasized speech obtained by emphasizing mixed speech. Fig. 4 is a diagram showing speech recognition performance for mixed speech. In both Fig. 3 and Fig. 4, the character error rate was evaluated as the speech recognition performance.

[0030] As shown in Figures 3 and 4, it has been experimentally proven that whether the input of the speech recognition is more effective, that is, the emphasized speech obtained by emphasizing the mixed speech or the original mixed speech, depends on the SIR and SNR. Specifically, the larger the value obtained by subtracting the SNR from the SIR, the lower the character error rate becomes when the mixed speech is used as the input rather than the emphasized speech.

[0031] For this reason, the voice recognition input determination unit 102 estimates the SIR and SNR from the input voice, selects the emphasized voice for the SNR and SIR range in which the voice recognition accuracy can be improved in Fig. 3, and selects the mixed voice for other ranges, and inputs them to the voice recognition unit 103. Alternatively, the voice recognition input determination unit 102 weights the emphasized voice more heavily than the mixed voice for the SNR and SIR range in which the voice recognition accuracy can be improved in Fig. 3, and weights the mixed voice more heavily than the emphasized voice for other ranges, and inputs the weighted mixed voice and emphasized voice to the voice recognition unit 103.

[0032] [Voice recognition input judgment section] Next, there will be described the voice recognition input determination unit 102. Fig. 2 is a diagram showing an example of the configuration of the voice recognition input determination unit 102 shown in Fig. 1.

[0033] As shown in FIG. 2, the voice recognition input determination unit 102 includes an SIR-SNR acquisition unit 1021 (acquisition unit), a determination unit 1022, and a switching unit 1023.

[0034] The SIR-SNR acquisition unit 1021 acquires at least one of the SIR in the mixed voice, the SNR in the mixed voice, and the mixed voice from the mixed voice input to the system. The SIR-SNR acquisition unit 1021 may acquire the SIR and the SIR in a fixed time unit such as a frame, or may acquire them in units of utterances.

[0035] The SIR-SNR acquisition unit 1021 estimates the SIR and SNR using, for example, an estimation model for estimating the SIR and SNR. The estimation model is configured by a neural network or the like. The estimation model is previously trained on pair data of the voice and the SIR and SNR.

[0036] In addition, when the voice of the target speaker and the voices of other interfering speakers are extracted by the voice enhancement unit 101, the SIR-SNR acquisition unit 1021 may calculate the SIR and SIR based on the extracted voice of the target speaker and the voices of other interfering speakers using equations (1) and (2).

[0037] The determination unit 1022 determines, as a speech to be used for speech recognition, a speech based on at least one of the mixed speech and an emphasized speech obtained by emphasizing the mixed speech, based on at least one of the SIR and SNR acquired by the SIR-SNR acquisition unit 1021 and the mixed speech. The determination unit 1022 determines, based on at least one of the SIR and SNR estimated or calculated by the SIR-SNR acquisition unit 1021 and the mixed speech, which of the mixed speech and the emphasized speech obtained by emphasizing the mixed speech is more advantageous for speech recognition.

[0038] For example, the determination unit 1022 determines a voice based on at least one of the mixed voice and the emphasized voice as a voice to be used for voice recognition using a predetermined rule using SIR and SNR. When performing the voice determination in a hard manner, the determination unit 1022 uses a rule that selects the emphasized voice when the following formula (3) is satisfied.

[0039]

number

[0040] In addition, based on the fact that whether the mixed voice or the emphasized voice is more advantageous as an input for voice recognition depends on the SIR and SNR (see Figures 3 and 4), the judgment unit 1022 may make a judgment using a rule that pre-sets a weighting value for the mixed voice and a weighting value for the emphasized voice depending on the combination of SIR and SNR.

[0041] When the determination unit 1022 performs voice determination softly, it may use an identification model that identifies voice based on at least one of the mixed voice and the emphasized voice as input to the voice recognition unit 103, and determine that the voice based on at least one of the mixed voice and the emphasized voice is the voice to be used for voice recognition.

[0042] The discrimination model is composed of a neural network, etc. The discrimination model is trained in advance using at least one of the features obtained from the mixed voice, to which a teacher label indicating whether the mixed voice or the emphasized voice is more advantageous as a voice to be used for speech recognition, and the SNR and SIR as inputs.

[0043] When the discrimination model takes SNR and SIR as input, the discrimination model is trained based on teacher labels indicating whether mixed speech or emphasized speech is more advantageous for use in speech recognition, and on paired data of estimated or calculated SNR and SIR.

[0044] The discrimination model can also directly estimate which of the mixed speech and the emphasized speech is more advantageous for speech recognition by using the features obtained from the mixed speech as input. In this case, the discrimination model is trained based on the teacher label indicating which of the mixed speech and the emphasized speech is more advantageous for speech recognition, and the pair data of the features extracted from the mixed speech.

[0045] When the discriminative model inputs both the SNR and SIR and mixed speech, the discriminative model is trained based on paired data of these and teacher labels that indicate which of the mixed speech and the emphasized speech is more advantageous for use in speech recognition.

[0046] The discrimination model outputs weights indicating the advantages of the mixed speech and the emphasized speech as speech to be used for speech recognition. The discrimination model may also output a determination result that selects either the mixed speech or the emphasized speech as a speech that is advantageous for use in speech recognition.

[0047] Based on the determination result of the determination unit 1022, the switching unit 1023 inputs a voice based on at least one of the mixed voice and the emphasized voice to the voice recognition unit 103. When the voice determination is performed in a hard manner as shown in FIG. 4, the switching unit 1023 inputs the emphasized voice or the mixed voice to the voice recognition unit 103 based on the selection result of the determination unit 1022. When the voice determination is performed in a soft manner (not shown), the switching unit 1023 weights the mixed voice and the emphasized voice according to the weight output by the determination unit 1022, and outputs the weighted mixed voice and the emphasized voice to the voice recognition unit 103.

[0048] [Signal processing procedure] Next, there will be described signal processing executed by the signal processing device 100. Fig. 5 is a flowchart showing a processing procedure of signal processing according to the embodiment.

[0049] When receiving input of a mixed voice (step S1), the voice enhancement unit 101 performs a voice enhancement process in which the voice of the target speaker is extracted from the input mixed voice and the extracted voice is input to the voice recognition input judgment unit 102 as an enhanced voice that enhances the mixed voice (step S2).

[0050] The voice recognition input determination unit 102 performs a voice recognition input determination process to determine, based on at least one of the SNR of the mixed voice, the SIR of the mixed voice, and the mixed voice, that a voice based on at least one of the mixed voice and an emphasized voice that emphasizes the mixed voice is the voice to be used for voice recognition (step S3).

[0051] The voice recognition unit 103 performs voice recognition processing using a voice based on at least one of the mixed voice and the emphasized voice determined by the voice recognition input determination unit 102 (step S4), and outputs the voice recognition result (step S5).

[0052] [Processing procedure for voice recognition input judgment] Next, a description will be given of the voice recognition input determination process (step S3) in Fig. 5. Fig. 6 is a flowchart showing the procedure of the voice recognition input determination process shown in Fig. 5.

[0053] As shown in FIG. 6, the SIR-SNR acquisition unit 1021 performs an acquisition process to acquire at least one of the SIR and SNR of the mixed sound, and the mixed sound from the mixed sound input to the system (step S11).

[0054] The determination unit 1022 performs a determination process based on at least one of the SIR and SNR acquired by the SIR-SNR acquisition unit 1021 and the mixed sound, to determine a sound based on at least one of the mixed sound and the emphasized sound obtained by emphasizing the mixed sound as a sound to be used for speech recognition (step S12). The switching unit 1023 inputs the sound based on at least one of the mixed sound and the emphasized sound to the speech recognition unit 103 as a sound to be used for speech recognition, based on the determination result of the determination unit 1022 (step S13).

[0055] [Effects of the embodiment] In this way, in the signal processing device 100, based on the fact that whether the mixed voice or the emphasized voice is more advantageous as an input for voice recognition depends on the SIR and SNR, the signal processing device 100 determines, based on at least one of the SIR and SNR, and the mixed voice, a voice based on at least one of the mixed voice and the emphasized voice as the voice to be used for voice recognition.

[0056] Fig. 7 is a diagram showing an example of a speech recognition result when the signal processing device 100 according to the embodiment is applied. Fig. 8 is a diagram showing an example of a performance improvement rate of a speech recognition result using the signal processing device 100 according to the embodiment, compared to a speech recognition result when emphasized speech is used. Figs. 7 and 8 show speech recognition results when a speech to be input to the speech recognition unit 103 is selected using a rule that emphasized speech is selected in the case of formula (3) from among emphasized speech and mixed speech. Note that in both Figs. 7 and 8, the character error rate was evaluated as the speech recognition performance.

[0057] The regions of frame W1 shown in Fig. 7 and frame W11 shown in Fig. 8 are regions where the input signal is thought to be selected as mixed speech based on the estimated SIR and SNR. From the speech recognition results of frame W1 shown in Fig. 7 and frame W11 shown in Fig. 8, it was found that the signal processing device 100 that selects a signal to the speech recognition unit 103 using the rule that emphasized speech is selected in the case of formula (3) has higher speech recognition accuracy than the case where only emphasized speech is used.

[0058] It was found that, on average, the signal processing device 100 had higher speech recognition accuracy than the case where only the emphasized speech was used across all combinations of SIR and SNR. Furthermore, it was found that in areas where the SIR value was relatively large (see, for example, frames W11 and W12), the signal processing device 100 had particularly higher speech recognition accuracy than the case where only the emphasized speech was used.

[0059] As a result, even when the voice of a target speaker overlaps with the voice of another speaker, the signal processing device 100 appropriately determines, as an input voice that can improve the accuracy of voice recognition, a voice based on at least one of the mixed voice and the emphasized voice obtained by emphasizing the mixed voice. By appropriately determining an input voice that can improve the accuracy of voice recognition, the signal processing device 100 can prevent performance degradation due to distortion in voice emphasis and improve the accuracy of voice recognition.

[0060] In this embodiment, when the mixed voice is selected as the input voice that can improve the accuracy of the voice recognition, the voice emphasizing unit 101 does not need to perform the voice emphasizing process. In this way, the voice emphasizing unit 101 performs the voice emphasizing process only for the section where the voice emphasizing process is required, that is, the section where the voice recognition input determining unit 102 selects the emphasized voice as the input voice that can improve the accuracy of the voice recognition.

[0061] [System configuration of the embodiment] Each component of the signal processing device 100 is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of the functions of the signal processing device 100 is not limited to that shown in the figure, and all or part of it can be functionally or physically distributed or integrated in any unit according to various loads, usage conditions, etc.

[0062] Further, all or any part of each process performed in the signal processing device 100 may be realized by a CPU, a GPU (Graphics Processing Unit), and a program analyzed and executed by the CPU and the GPU. Further, each process performed in the signal processing device 100 may be realized as hardware using wired logic.

[0063] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the process procedures, control procedures, specific names, and information including various data and parameters described above and shown in the drawings can be changed as appropriate unless otherwise specified.

[0064] [program] 9 is a diagram showing an example of a computer in which a program is executed to realize a signal processing device 100. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0065] The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0066] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the signal processing device 100 is implemented as a program module 1093 in which code executable by the computer 1000 is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, the program module 1093 for executing the same process as the functional configuration in the signal processing device 100 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced with an SSD (Solid State Drive).

[0067] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1090. Then, CPU 1020 reads out program module 1093 or program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as necessary, and executes the program.

[0068] Note that the program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or wide area network (WAN)). The program module 1093 and the program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0069] Although the embodiment of the invention made by the present inventor has been described above, the present invention is not limited by the description and drawings that form a part of the disclosure of the present invention according to the present embodiment. In other words, other embodiments, examples, operation techniques, etc. made by those skilled in the art based on the present embodiment are all included in the scope of the present invention. [Explanation of symbols]

[0070] 100 Signal processing device 101 Voice Enhancement Unit 102 Voice recognition input judgment unit 103 Voice Recognition Unit 1021 SIR-SNR acquisition section 1022 Judgment section 1023 Switching section

Claims

1. An acquisition unit that acquires at least one of a signal to interference ratio (SIR) that is a ratio of a target voice to an interference speaker voice in a mixed voice in which a voice of a target speaker overlaps with a voice of another speaker, a signal to noise ratio (SNR) that is a ratio of a target voice to noise in the mixed voice, and the mixed voice; a determination unit that determines, based on at least one of the SIR, the SNR, and the mixed sound, a sound based on at least one of the mixed sound and an emphasized sound obtained by emphasizing the mixed sound as a sound to be used for speech recognition; having the determination unit determines, as the speech to be used for the speech recognition, a speech based on at least one of the mixed speech and the emphasized speech, using an identification model trained using as input at least one of features acquired from the mixed speech and the SNR and the SIR, to which a teacher label indicating which of the mixed speech and the emphasized speech is more advantageous as a speech to be used for speech recognition, the speech based on at least one of the mixed speech and the emphasized speech.

2. An acquisition unit that acquires at least one of a signal to interference ratio (SIR) that is a ratio of a target voice to an interference speaker voice in a mixed voice in which a target speaker's voice is overlapped with a voice of another speaker, a signal to noise ratio (SNR) that is a ratio of a target voice to noise in the mixed voice, and the mixed voice; a determination unit that determines, based on at least one of the SIR, the SNR, and the mixed sound, a sound based on at least one of the mixed sound and an emphasized sound obtained by emphasizing the mixed sound as a sound to be used for speech recognition; having The signal processing device according to claim 1, wherein the determination unit determines that the emphasized voice is to be used for voice recognition when a value obtained by subtracting the SNR from the SIR is smaller than a predetermined value.

3. An acquisition unit that acquires at least one of a signal to interference ratio (SIR) that is a ratio of a target voice to an interference speaker voice in the mixed voice, a signal to noise ratio (SNR) that is a ratio of a target voice to noise in the mixed voice, and the mixed voice from a mixed voice in which the target speaker's voice is overlapped with another speaker's voice; a determination unit that determines, based on at least one of the SIR, the SNR, and the mixed sound, a sound based on at least one of the mixed sound and an emphasized sound obtained by emphasizing the mixed sound as a sound to be used for speech recognition; having The signal processing device is characterized in that the determination unit determines a voice to be used for voice recognition using a rule in which a weighting value for the mixed voice and a weighting value for the emphasized voice are preset in accordance with a combination of the SIR and the SNR.

4. The signal processing device according to any one of claims 1 to 3, further comprising a voice recognition unit that performs voice recognition processing using a voice based on at least one of the mixed voice and the emphasized voice determined by the determination unit.

5. 5. The signal processing device according to claim 1, further comprising a speech emphasizing unit that extracts the speech of the target speaker from the mixed speech and inputs the extracted speech to the acquisition unit as the emphasized speech.

6. A signal processing method executed by a signal processing device, comprising: A step of acquiring at least one of a signal to interference ratio (SIR) which is a ratio of a target voice to an interference speaker voice in the mixed voice, a signal to noise ratio (SNR) which is a ratio of a target voice to noise in the mixed voice, and the mixed voice from a mixed voice in which a target speaker's voice is overlapped with a voice of another speaker; determining, based on at least one of the SIR, the SNR, and the mixed sound, a sound based on at least one of the mixed sound and an enhanced sound obtained by enhancing the mixed sound as a sound to be used for speech recognition; Including, The signal processing method is characterized in that the determining step determines a speech based on at least one of the mixed speech and the emphasized speech as a speech to be used for the speech recognition using a discrimination model trained using as input at least one of features obtained from the mixed speech and the SNR and the SIR, to which a teacher label indicating which of the mixed speech and the emphasized speech is more advantageous as a speech to be used for speech recognition is assigned.

7. A signal processing method executed by a signal processing device, comprising: A step of acquiring at least one of a signal to interference ratio (SIR) which is a ratio of a target voice to an interference speaker voice in the mixed voice, a signal to noise ratio (SNR) which is a ratio of a target voice to noise in the mixed voice, and the mixed voice from a mixed voice in which a target speaker's voice is overlapped with a voice of another speaker; determining, based on at least one of the SIR, the SNR, and the mixed sound, a sound based on at least one of the mixed sound and an enhanced sound obtained by enhancing the mixed sound as a sound to be used for speech recognition; Including, The signal processing method according to claim 1, wherein said determining step determines that said emphasized speech is to be used for speech recognition when a value obtained by subtracting said SNR from said SIR is smaller than a predetermined value.

8. A signal processing method executed by a signal processing device, comprising: A step of acquiring at least one of a signal to interference ratio (SIR) which is a ratio of a target voice to an interference speaker voice in the mixed voice, a signal to noise ratio (SNR) which is a ratio of a target voice to noise in the mixed voice, and the mixed voice from a mixed voice in which a target speaker's voice is overlapped with a voice of another speaker; determining, based on at least one of the SIR, the SNR, and the mixed sound, a sound based on at least one of the mixed sound and an enhanced sound obtained by enhancing the mixed sound as a sound to be used for speech recognition; Including, The signal processing method is characterized in that the determining step determines a voice to be used for voice recognition using a rule in which a weighting value for the mixed voice and a weighting value for the emphasized voice are preset according to a combination of the SIR and the SNR.

9. A step of acquiring at least one of a signal to interference ratio (SIR) which is a ratio of a target voice to an interference speaker voice in the mixed voice, a signal to noise ratio (SNR) which is a ratio of a target voice to noise in the mixed voice, and the mixed voice from a mixed voice in which a voice of a target speaker overlaps with a voice of another speaker; determining, based on at least one of the SIR, the SNR, and the mixed sound, a sound based on at least one of the mixed sound and an enhanced sound obtained by enhancing the mixed sound as a sound to be used for speech recognition; on the computer, The determining step uses a discrimination model trained using as input at least one of features obtained from the mixed speech and the SNR and the SIR, to which a teacher label indicating which of the mixed speech and the emphasized speech is more advantageous as a speech to be used for speech recognition, to determine a speech based on at least one of the mixed speech and the emphasized speech as a speech to be used for the speech recognition. The discrimination model is a signal processing program that determines whether a speech based on the mixed speech or the emphasized speech is more advantageous as a speech to be used for speech recognition.

10. A step of obtaining at least one of a signal to interference ratio (SIR) which is a ratio of a target voice to an interference speaker voice in a mixed voice in which a target speaker's voice is overlapped with a voice of another speaker, a signal to noise ratio (SNR) which is a ratio of a target voice to noise in the mixed voice, and the mixed voice; determining, based on at least one of the SIR, the SNR, and the mixed sound, a sound based on at least one of the mixed sound and an enhanced sound obtained by enhancing the mixed sound as a sound to be used for speech recognition; on the computer, The signal processing program includes a step of determining that the emphasized speech is to be used for speech recognition when a value obtained by subtracting the SNR from the SIR is smaller than a predetermined value.

11. A step of obtaining at least one of a signal to interference ratio (SIR) which is a ratio of a target voice to an interference speaker voice in a mixed voice in which a target speaker's voice is overlapped with a voice of another speaker, a signal to noise ratio (SNR) which is a ratio of a target voice to noise in the mixed voice, and the mixed voice; determining, based on at least one of the SIR, the SNR, and the mixed sound, a sound based on at least one of the mixed sound and an enhanced sound obtained by enhancing the mixed sound as a sound to be used for speech recognition; on the computer, The determining step is a signal processing program that determines a voice to be used for voice recognition using a rule that pre-sets a weighting value for the mixed voice and a weighting value for the emphasized voice according to a combination of the SIR and the SNR.

Images