Sound collection device, sound collection program, and sound collection method

Abstract

To suppress erroneous extraction of interference sounds after the end of utterance of a target sound while supporting a hangover function in area sound collection processing. [Solution] The present invention relates to a sound collection device. The sound collection device of the present invention is characterized by having a means for extracting a target area sound based on a beamformer output of a plurality of microphone arrays, a means for acquiring a target area sound judgment value, a means for acquiring a non-target area sound judgment value, a means for controlling a hangover length based on a comparison result between the target area sound judgment value and the non-target area sound judgment value, a means for performing a target area sound section judgment process for judging whether or not it is a target area sound section based on the target area sound judgment value, and if it is a target area sound section, a means for determining that the result of the target area sound section judgment process is a target area sound section from that point in time for the hangover length, and a means for outputting the target area sound only while the input signal is judged to be a target area sound section in the most recently performed target area sound section judgment process.

Description

[Technical field]

[0001] The present invention relates to a sound collection device, a program, and a method, and can be applied to, for example, a sound collection device that emphasizes only sounds emitted from a specific area and suppresses sounds from other areas. [Background technology]

[0002] In speech recognition under noisy conditions, the accuracy of speech recognition decreases due to the influence of noise, so high accuracy is required for voice activity detection, which excludes noise periods when no speech is occurring and extracts only the user's speech. Many conventional methods for voice activity detection have been introduced, including methods that distinguish based on the power ratio between speech and noise, and methods that distinguish based on speech-like characteristics such as the shape of the spectrum. Note that the method that distinguishes based on the power ratio between speech and noise is difficult to extract for low-power speech such as unvoiced consonants, so as a countermeasure, a method that compensates by performing a process that determines the presence of speech as present for several seconds after determining that speech is present regardless of the result of the determination (processing corresponding to the hangover function) is often used.

[0003] However, while these methods can distinguish between voice and noise, they cannot distinguish between the desired voice (hereafter referred to as "target voice") and other unwanted voices (hereafter referred to as "interfering voice"). On the other hand, a method that uses coherence, which means the direction of arrival of the input signal, has also been introduced as a method for distinguishing between the desired voice and the interfering voice and detecting the voice activity. Coherence is the correlation between the signal arriving from the right and the signal arriving from the left, so it becomes small when the direction of arrival is significantly biased to either the left or right, or when there is no bias but the signal has little clear regularity, such as noise, while it becomes large when the direction of arrival is from the front. This coherence is used to determine whether the direction of arrival of the input signal is from the front or the side, and to distinguish between the desired voice arriving from the front and the interfering voice arriving from the side. However, when the target sound and the interfering sound are spoken simultaneously (hereinafter referred to as "double talk"), if the interfering sound arrives from the side, the coherence becomes smaller, so the hangover effect during the target sound section cannot be obtained. Conversely, if the interfering sound arrives from the front, the coherence becomes larger, so the hangover effect continues to work even after the target sound section ends.

[0004] In response to this, Patent Document 1 proposes a method of controlling the length of the hangover (hereinafter referred to as "hangover length") by focusing on the movement of coherence when an interfering sound arrives from the side or the front. Specifically, when an interfering sound arrives from the side, the magnitude of the coherence frequently fluctuates, whereas when an interfering sound arrives from the front, the magnitude of the coherence does not fluctuate much. By focusing on this point, when it is determined that an interfering sound arrives from the side, the hangover length is lengthened to prevent the voice from being interrupted, and when it is determined that an interfering sound arrives from the front, the hangover length is shortened to prevent the interfering sound from being erroneously extracted after the utterance of the target sound has ended.

[0005] Meanwhile, methods such as those disclosed in Patent Documents 2 and 3 have been proposed as methods for detecting voice segments by acquiring a target direction signal based on the beamformer output of a multiple microphone array, extracting non-target area sounds by performing spectral subtraction processing on the acquired target direction signal from another target direction signal, extracting target area sounds by performing spectral subtraction of the non-target area sounds from the target direction signal, and detecting voice segments based on whether or not the target area sounds are included in the input signal.

[0006] Patent Document 2 points out the problem that when processing to extract sounds in an area is performed in an environment with strong background noise, musical noise remains, and is particularly harsh to the ear in sections where there is no sound in the area. In response to this problem, this patent proposes judging whether or not the input signal contains target area sound from the relationship between the power ratio of the input signal to the target area sound and a threshold value, and not outputting the processing result if the target area sound is not present.

[0007] On the other hand, although Patent Document 2 was able to deal with musical noise in an environment with strong background noise, the SNR remains poor. Therefore, the problem is raised that the target sound is buried in the noise and cannot be detected, especially when the power is originally small, such as unvoiced consonants. In response to this, Patent Document 3 proposes a method of re-determining the sound in an area for each frequency component or each frequency band, even if Patent Document 2 determines that there is no sound.

[0008] Here, the background technology of Patent Document 3 will be described. In an environment where multiple sound sources exist, a beam former (hereinafter also referred to as "BF") using a microphone array is a technology for separating and collecting only sounds from a specific direction. BF is a technology for forming directivity by utilizing the time difference between sounds arriving at each microphone. Conventionally, BF is broadly divided into two types: additive and subtractive. In particular, subtractive BF has the advantage that directivity can be formed with a smaller number of microphones than additive BF.

[0009] FIG. 6 is a block diagram showing the configuration of a subtraction-type BF 200 when the number of microphones M is two.

[0010] FIG. 7 is an explanatory diagram showing an example of a directional filter formed by a subtractive BF 200 using two microphones M1 and M2.

[0011] The subtractive BF 200 first calculates the time difference between the arrival of a target sound in a desired direction at each of the microphones M1 and M2 using the delay unit 210, and aligns the phase of the target sound by adding a delay. The above-mentioned time difference can be calculated using the following formula (1).

[0012] Here, d is the distance between the microphones M1 and M2, c is the speed of sound, and τL is the delay amount. Also, θL is the angle from the perpendicular direction to the line connecting each microphone M (M1, M2) to the target direction.

[0013] In addition, when the blind spot exists in the direction of microphone M1 with respect to the center of microphones M1 and M2, delay unit 210 performs delay processing on input signal x1(t) of microphone M1. After that, in subtraction type BF 200, subtractor 220 performs processing (subtraction processing) according to the following formula (2).

[0014] The subtractive BF200 process can be performed in the frequency domain as well. In that case, equation (2) is modified to the following equation (3): τ L =(dsinθ L ) / c ···(1) m(t) = x2(t) - x1(t L ) ···(2) M(ω)=X2(ω)-e -jωL X1(ω) (3)

[0015] Here, θ L In the case of "θ = ±π / 2", the directivity formed by the subtractive BF 200 is a cardioid type unidirectional pattern as shown in FIG. L= 0, π”, the directivity formed by the subtractive BF200 is a figure-8 shaped bidirectional pattern as shown in FIG. 7(b).

[0016] Hereinafter, a filter that forms a unidirectional pattern from an input signal will be referred to as a "unidirectional filter," and a filter that forms a bidirectional pattern will be referred to as a bidirectional filter.

[0017] In addition, the subtractor 220 can form a directivity that is strong in the blind spots of the bidirectional beam by using a Spectral Subtraction (hereinafter, simply referred to as "SS"). The directivity by SS is formed in all frequencies or in a specified frequency band according to the following formula (4).

[0018] In the following formula (4), the input signal X1 of the microphone M1 is used, but the same effect can be obtained with the input signal X2 of the microphone M2. Here, β is a coefficient for adjusting the strength of the SS. Furthermore, in the subtractor 220, if the value becomes negative during subtraction, a flooring process is performed to replace the original value with 0 or a value that is smaller than the original value. In the above-described processing method of the subtractive BF200, sounds existing in directions other than the target direction (hereinafter referred to as "non-target sounds") are extracted using the bidirectional characteristics, and the amplitude spectrum of the extracted non-target sounds is subtracted from the amplitude spectrum of the input signal, thereby emphasizing the target sound. Y(n) = X1(n) - βM(n) (4)

[0019] However, when it is desired to collect only sounds present within a certain area (hereinafter referred to as "target area sounds"), using only a subtractive BF may result in the collection of sounds from sound sources present around the area (hereinafter referred to as "non-target area sounds"). Therefore, Patent Document 2 proposes a method (hereinafter referred to as "area sound collection") for collecting target area sounds by using multiple microphone arrays, directing the directivities toward the target area from different directions and crossing the directivities at the target area. In area sound collection, first, the ratio of the amplitude spectrum of the target area sound contained in the BF output of each microphone array is estimated, and this is used as a correction coefficient.

[0020] FIG. 8 is a diagram showing an example of area sound collection processing of target area sound having a target area as a sound source, using two microphone arrays MA1 and MA2.

[0021] For example, as shown in FIG. 8, when two microphone arrays MA1 and MA2 (2ch microphone arrays having microphones M1 and M2) are used, the correction coefficient of the target area sound amplitude spectrum can be calculated by a combination of the following equations (5) and (6), or a combination of the following equations (7) and (8). Here, Y 1k (n) is the amplitude spectrum of the BF output of the first microphone array MA1, and Y 2k (n) is the amplitude spectrum of the BF output of the second microphone array MA2, N is the total number of frequency bins, and k is the frequency. Also, α1(n) and α2(n) are amplitude spectrum correction coefficients for each BF output. Furthermore, here, mode represents the mode, and median represents the median.

number

[0022] Through the above processing, the subtractor 220 obtains the correction coefficients α1(n) and α2(n), corrects each BF output with the obtained correction coefficient, and extracts the non-target area sound present in the target area direction by performing SS. Furthermore, the subtractor 220 can extract the target area sound by SSing the extracted non-target area sound from the output of each BF.

[0023] When extracting non-target area sound N1(n) existing in the target area direction as seen from the first microphone array, the subtraction type BF200 performs SS on the first microphone array BF output Y1(n) multiplied by the amplitude spectrum correction coefficient α2 to the second microphone array BF output Y2(n) as shown in equation (9), for example. Similarly, the subtraction type BF200 extracts non-target area sound N2(n) existing in the target area direction as seen from the second microphone array according to the following equation (10).

[0024] Thereafter, the subtraction type BF200 extracts the target area sound by SS of the non-target area sound from each BF output according to the following formula (11) or formula (12). Note that the following formula (11) shows the process when the target area sound is extracted based on the first microphone array. Also, the following formula (12) shows the process when the target area sound is extracted based on the second microphone array. Here, γ1(n) and γ2(n) are coefficients for changing the strength during SS. N1(n)=Y1(n)-α2(n)Y2(n) ···(9) N2(n)=Y2(n)-α1(n)Y1(n) ···(10) Z1(n)=Y1(n)-γ1(n)N1(n) ···(11) Z2(n)=Y2(n)-γ2(n)N2(n) ···(12)

[0025] When the background noise is strong, some of the noise remains in the target area sounds Z1(n) and Z2(n), which can cause unpleasant sounds such as musical noise. Therefore, the input signal of the microphone array is compared with the extracted target area sound to determine whether or not the target area sound is present, and if the target area sound does not exist, no sound or a low sound is output. Specifically, the sum R of the amplitude ratios for each frequency is calculated using equations (13) and (14). If R is equal to or greater than a predetermined threshold, it is determined that the target area sound is present, and if R is less than the threshold, the amplitude ratio R for each frequency in equation (13) is calculated. k and a predefined threshold, R k If is greater than or equal to a predetermined threshold, it is determined that sound exists in the target area, and Rk If X is less than the threshold, it is determined that the destination area sound does not exist. In addition, when it is determined that the destination area sound exists, a process (a process corresponding to a hangover function) may be added to the determination of the presence or absence of the destination area sound, regardless of the result of the determination, for the next few seconds. 1k is the amplitude spectrum of frequency k in the input signal X1 of the microphone array MA1 (the input signal of the first microphone M1), and Z 1k is the amplitude spectrum of frequency k in the target area sound. Also, m is the lower limit of the frequency to be processed, and n is the upper limit of the frequency.

[0026] The sum of the amplitude ratios for each frequency, R, is the amplitude ratio R k up to frequencies k = m, . . . , n.

number

[0027] As described above, the area sound collection technology described in Patent Document 3 is composed of multiple microphone arrays, each microphone array forms a directivity in the direction of the target area by a beamformer, the power is corrected so that the power of the target area sound contained in all beamformer outputs is equal, non-target area sound is extracted by performing spectral subtraction on each beamformer output, and the target area sound is extracted by performing spectral subtraction on the non-target area sound from each beamformer output. As a result, the area sound collection technology described in Patent Document 3 judges whether or not there is an output for all frequency components and each frequency component based on the relationship between the volume of the target area sound contained in the input sound, and if it is judged that the target area sound exists, the target area sound is output, and if it is judged that the target area sound does not exist, no sound or a small sound is output.

[0028] Conventional area sound collection technology is a very effective method for suppressing noise generated outside the area, but as described in the background technology of Patent Document 3, when the level of background noise present in the surroundings is high, harsh abnormal sounds such as musical noise may occur. Patent Document 3 attempts to improve this by comparing the input signal of the microphone array with the extracted target area sound to determine the presence or absence of the target area sound, and outputting no sound or a low sound if the target area sound does not exist. Another technology for improving musical noise in area sound collection is described in Patent Document 4.

[0029] Patent Document 4 proposes a method (signal mixing area sound collection method) of mixing an input signal and an estimated noise signal with the output of area sound collection to mask abnormal sounds such as musical noise as a means of improving the sound quality of area sound collection. Hereinafter, the method of mixing a predetermined sound (for example, an input signal or estimated noise) with the output of area sound collection to mask abnormal sounds such as musical noise as in the technology described in Patent Document 4 will be referred to as "mixing area sound collection." [Prior art documents] [Patent documents]

[0030] [Patent Document 1] JP 2014-56112 A [Patent Document 2] JP 2016-127457 A [Patent Document 3] JP 2018-164156 A [Patent Document 4] JP 2017-183902 A Summary of the Invention [Problem to be solved by the invention]

[0031] As described above, the conventional mixing area sound collection is a method to significantly improve the sound quality of area sound collection, but since it is a method of mixing the input signal and estimated noise, its effect is somewhat weakened from the viewpoint of noise suppression. For example, in double talk, when the target sound is within the area and the interfering sound is outside the area, the input signal is mixed into the mixing area sound collection, so the interfering sound of the input signal is also mixed in, and the accuracy of voice recognition is reduced.

[0032] Incidentally, in the case of mixing area sound collection, like normal area sound collection, hangover is used to improve the determination performance of the target sound section, preventing sound interruption of low-power sounds such as unvoiced consonants. On the other hand, when hangover is used, there is a risk that an interfering sound may be mistakenly extracted during double talk even though the target sound has ended. Therefore, it is desirable to set an appropriate hangover length for the area sound collection technology using the coherence of Patent Document 1, which can control the hangover length depending on whether or not there is an interfering sound. However, while coherence detects the voice section based on the arrival direction of the sound source, the area sound collection technology detects the voice section based on the position of the sound source, so this is not applicable.

[0033] Here, the reason why coherence cannot be applied to area sound collection technology will be described. Area sound collection technology is composed of multiple microphone arrays, each microphone array forms a directivity in the direction of the target area by a beamformer, the power is corrected so that the power of the target area sound contained in all beamformer outputs is equal, non-target area sound is extracted by performing spectral subtraction on each beamformer output, the target area sound is extracted by performing spectral subtraction on the non-target area sound from each beamformer output, and the section in which the target area sound exists is detected from the relationship of the magnitude of the target area sound contained in the input sound. On the other hand, the coherence of the method of Patent Document 1 is calculated using the input signals of two microphones and indicates the arrival direction of the input signal. In area sound collection technology, the coherence calculated from the input signals of two microphones that make up a microphone array is large when the sound source is located closer to the front. Therefore, even if the interfering sound is outside a certain area and arrives from the front direction, it becomes large, and the interfering sound is erroneously extracted.

[0034] In view of the above problems, there is a need for a sound collection device, a sound collection program, and a sound collection method that can suppress erroneous extraction of an interfering sound after the speech of a target sound is terminated while supporting a hangover function in area sound collection processing. [Means for solving the problem]

[0035] The first sound collection device of the present invention includes a target area sound acquisition means for acquiring a beamformer output with directivity directed toward a target area from an input signal of a plurality of microphone arrays, and extracting a target area sound having the target area as a sound source based on each of the beamformer outputs, a target area sound judgment value acquisition means for acquiring a target area sound judgment value for the target area sound, a non-target area sound judgment value acquisition means for acquiring a non-target area sound judgment value for a non-target area sound having a sound source other than the target area, a hangover length control means for controlling a hangover length based on a result of comparison between the target area sound judgment value and the non-target area sound judgment value, and a target area sound judgment value acquisition means for acquiring a target area sound judgment value for the target area sound. a target area sound section detection means for performing a target area sound section determination process based on a constant value to determine whether the input signal is a target area sound section containing a target area sound component equal to or greater than a predetermined standard, and if the input signal is determined to be a target area sound section in the target area sound section determination process, the target area sound section detection means determines that the result of the target area sound section determination process is a target area sound section from that point onwards for the hangover length controlled by the hangover length control means, and an output means for outputting the target area sound only while the input signal is determined to be a target area sound section in the most recently performed target area sound section determination process.

[0036] A second sound collection program of the present invention includes a computer, a target area sound acquisition means for acquiring a beamformer output with directivity directed toward a target area from an input signal of a plurality of microphone arrays, and extracting a target area sound having the target area as a sound source based on each of the beamformer outputs, a target area sound judgment value acquisition means for acquiring a target area sound judgment value for the target area sound, a non-target area sound judgment value acquisition means for acquiring a non-target area sound judgment value for a non-target area sound having a sound source other than the target area, a hangover length control means for controlling a hangover length based on a result of a comparison between the target area sound judgment value and the non-target area sound judgment value, and a target area sound section detection means for determining, based on a sound judgment value, whether the input signal is a target area sound section containing a target area sound component equal to or greater than a predetermined standard, and, if the input signal is determined to be a target area sound section in the target area sound section judgment process, determining that the result of the target area sound section judgment process is a target area sound section from that point onwards for the hangover length controlled by the hangover length control means, and functioning as an output means for outputting the target area sound only while the input signal is determined to be a target area sound section in the most recently performed target area sound section judgment process.

[0037] The third aspect of the present invention is a sound collection method performed by a sound collection device, the sound collection device having a target area sound acquisition means, a target area sound judgment value acquisition means, a non-target area sound judgment value acquisition means, a hangover length control means, a target area sound section detection means and an output means, the target area sound acquisition means acquires a beamformer output with directivity oriented in the direction of the target area from an input signal of a plurality of microphone arrays, and extracts a target area sound having the target area as a sound source based on each of the beamformer outputs, the target area sound judgment value acquisition means acquires a target area sound judgment value for the target area sound, the non-target area sound judgment value acquisition means acquires a non-target area sound judgment value for non-target area sound having a sound source other than the target area, and the hangover length control means detects the target area sound and outputs the target area sound. a hangover length is controlled based on a comparison result between a target area sound judgment value and the non-target area sound judgment value, and the target area sound section detection means performs a target area sound section judgment process to judge whether the input signal is a target area sound section containing a target area sound component equal to or greater than a predetermined standard based on the target area sound judgment value, and when the input signal is judged to be a target area sound section in the target area sound section judgment process, from that point onwards for the hangover length controlled by the hangover length control means, the result of the target area sound section judgment process is that the target area sound section, and the output means outputs the target area sound only while the input signal is judged to be a target area sound section in the most recently performed target area sound section judgment process. Effect of the Invention

[0038] According to the present invention, it is possible to provide a sound collection device, a sound collection program, and a sound collection method that can suppress erroneous extraction of an interfering sound after the speech of a target sound is terminated while supporting a hangover function in area sound collection processing. [Brief description of the drawings]

[0039] [Figure 1] 1 is a block diagram showing a functional configuration of a sound collecting device according to a first embodiment. [Diagram 2]2 is a block diagram showing an example of the internal configuration of a target area sound acquisition unit according to the first embodiment. FIG. [Diagram 3] 1 is a block diagram showing an example of a hardware configuration of a sound collecting device according to a first embodiment. [Figure 4] FIG. 11 is a diagram showing, in a table format, maximum hangover count variables (hangover lengths) corresponding to judgment value ratios according to the first embodiment. [Diagram 5] 5 is a flowchart showing an example of an operation performed by a target area sound section detection unit according to the first embodiment when performing target area sound section detection. [Figure 6] FIG. 1 is a block diagram showing a configuration of a conventional subtraction beamformer BF when the number of microphones M is two. [Figure 7] FIG. 1 is an explanatory diagram showing an example of a directional filter formed by a conventional subtractive beamformer using two microphones. [Figure 8] FIG. 13 is a diagram showing an example of area sound collection processing of target area sound whose sound source is the target area using two conventional microphone arrays. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0040] (A) First embodiment Hereinafter, a first embodiment of a sound collection device, a sound collection program, and a sound collection method according to the present invention will be described in detail with reference to the drawings.

[0041] (A-1) Configuration of the First Embodiment Fig. 1 is a block diagram showing the functional configuration of the sound collection device of this embodiment. Note that in Fig. 1, the reference numerals in parentheses are used only in the second embodiment.

[0042] The sound collection device performs target area sound collection processing to collect target area sound from a sound source in the target area using two microphone arrays MA (MA1, MA2).

[0043] The microphone arrays MA1 and MA2 are arranged at any location in the space in which the target area exists. The positions of the microphone arrays MA1 and MA2 relative to the target area may be anywhere as long as the directivities overlap only in the target area, and may be arranged opposite each other across the target area. Each microphone array MA is composed of two or more microphones M, and each microphone M picks up an acoustic signal. In this embodiment, it is described that each microphone array MA is arranged with two microphones M (M1, M2) that pick up an acoustic signal. That is, each microphone array MA constitutes a 2ch microphone array. Note that the number of microphone arrays MA is not limited to two, and when there are multiple target areas, it is necessary to arrange the number of microphone arrays MA that can cover all the areas.

[0044] Next, the internal configuration of the sound collection device 10 will be described.

[0045] The sound collection device 10 has a data input unit 101, a target area sound acquisition unit 102, a target area sound judgment value acquisition unit 103, a non-target area sound judgment value acquisition unit 104, a hangover length control unit 105, a target area sound section detection unit 106, and an output unit 107.

[0046] FIG. 2 is a block diagram showing an example of the functional configuration of the target area sound acquisition unit 102 of this embodiment.

[0047] In this embodiment, the target area sound acquisition unit 102 is assumed to have a noise suppression unit 201, a directivity forming unit 202, a delay correction unit 203, a spatial coordinate holding unit 204, a correction coefficient calculation unit 205, and a target area sound extraction unit 206.

[0048] FIG. 3 is a block diagram showing an example of a hardware configuration of the sound collection device 10. As shown in FIG.

[0049] FIG. 3 shows an example of a hardware configuration when the sound collection device 10 is configured using software (computer).

[0050] 3 includes, as a hardware component, a computer 300 in which a program (including the sound collection program of the embodiment) is installed. The computer 300 may be a computer dedicated to the sound collection program, or may be configured to be shared with programs of other functions.

[0051] The computer 300 shown in FIG. 3 includes a processor 301, a primary storage unit 302, and a secondary storage unit 303. The primary storage unit 302 is a storage unit that functions as a working memory (work memory) for the processor 301, and can be, for example, a high-speed memory such as a dynamic random access memory (DRAM). The secondary storage unit 303 is a storage unit that records various data such as an operating system (OS) and program data (including data of a sound collection program according to the embodiment), and can be, for example, a non-volatile memory such as a FLASH (registered trademark) memory, HDD, or SSD. In the computer 300 of this embodiment, when the processor 301 starts up, the OS and programs (including the sound collection program according to the embodiment) recorded in the secondary storage unit 303 are read, deployed on the primary storage unit 302, and executed.

[0052] Note that the specific configuration of the computer 300 is not limited to the configuration in Fig. 3, and various configurations can be applied. For example, if the primary storage unit 302 is a non-volatile memory (e.g., a flash memory, etc.), the secondary storage unit 303 may be excluded.

[0053] (A-2) Operation of the First Embodiment Next, the operation of the sound collection device 10 of the first embodiment having the above-mentioned configuration will be described.

[0054] The data input unit 101 converts the acoustic signals collected by each microphone array from analog signals to digital signals and inputs them. The data input unit 101 then converts the digital signals from the time domain to the frequency domain using, for example, a fast Fourier transform.

[0055] In this embodiment, the acquisition of the destination area sound by the destination area sound acquisition unit 102 is described as being realized, for example, by the configuration shown in FIG. 2, but a configuration for acquiring the destination area sound using another method may also be applied.

[0056] The operation of each component of the target area sound acquisition unit 102 shown in FIG. 2 will be described below.

[0057] The noise suppressor 201 estimates and suppresses background noise components contained in the sound acquired from the data input unit 101. For noise suppression by the noise suppressor 201, for example, SS or Wiener filtering can be used.

[0058] The directivity forming unit 202 forms directivity in the direction of the target area by BF according to equation (4) for the signal in which background noise has been suppressed by the noise suppressing unit 201 for each microphone array.

[0059] The delay correction unit 203 calculates and corrects the delay that occurs due to the difference in distance between the target area and each microphone array. The delay correction unit 203 first obtains the position of the target area and the position of each microphone array from the spatial coordinate storage unit 204, and calculates the difference in arrival time of the target area sound to each microphone array. Next, using the microphone array located at the farthest position from the target area as a reference, a delay is added so that the target area sound arrives at all microphone arrays at the same time.

[0060] The spatial coordinate holding unit 204 holds position information of all target areas and the microphones constituting each microphone array. The method in which the spatial coordinate holding unit 204 holds the position information of each microphone in each microphone array and the specific format of the position information held by the spatial coordinate holding unit 204 are not limited, and various data formats can be applied.

[0061] The correction coefficient calculation unit 205 calculates correction coefficients for making the amplitude spectra of the target area sound components included in each BF output the same, according to equations (5) and (6) or (7) and (8).

[0062] The target area sound extraction unit 206 extracts non-target area sounds existing in the target area direction by subjecting each BF output data corrected by the correction coefficient calculated by the correction coefficient calculation unit 205 to SS according to equation (9) or (10). Furthermore, the target area sound extraction unit 206 extracts the target area sound by subjecting the extracted noise from the output of each BF to SS according to equation (11) or (12).

[0063] As described above, the destination area sound acquisition unit 102 acquires the destination area sound.

[0064] The target area sound judgment value acquisition unit 103 calculates a judgment value for the target area sound acquired from the target area sound acquisition unit 102 (hereinafter referred to as a "target area sound judgment value" or a "target area sound evaluation value"). Here, the target area sound judgment value will be specifically described. First, for each frequency, the input signal X acquired from the data input unit 101 is 1k and the target area sound Z acquired from the target area sound acquisition unit 102 1k Using the above, the amplitude ratio R k Specifically, the amplitude ratio for each frequency is calculated based on equation (13).

[0065] The target area sound judgment value is the amplitude ratio R k is the sum R of the amplitude ratios for frequencies k=m, . . . , n. Specifically, for example, the target area sound determination value acquisition unit 103 may calculate the sum of the amplitude ratios for each frequency based on equation (14).

[0066] The target area sound judgment value acquisition unit 103 calculates a judgment value for the non-target area sound contained in the input signal (hereinafter referred to as a "non-target area sound judgment value" or a "non-target area sound evaluation value"). Here, the non-target area sound judgment value will be specifically described. For example, the non-target area sound judgment value acquisition unit 104 may calculate the non-target area sound judgment value by dividing a signal calculated by subtracting the target area sound acquired from the target area sound acquisition unit 102 from the input signal acquired from the data input unit 101 by the target area sound or the input signal. Here, the non-target area sound judgment value acquisition unit 104 calculates the amplitude ratio by dividing by the input signal. First, the non-target area sound judgment value acquisition unit 104 calculates the amplitude ratio for the input signal X acquired from the data input unit 101 for each frequency. 1k and the target area sound Z acquired from the target area sound acquisition unit 102 1k Using the amplitude ratio Rd k Specifically, the non-target area sound determination value acquisition unit 104 may calculate the amplitude ratio for each frequency based on, for example, equation (15).

[0067] The non-target area sound judgment value Rd is the amplitude ratio Rd k Alternatively, the sum of the amplitude ratios for frequencies k=m, . . . , n may be calculated. Specifically, the non-target area sound determination value acquisition unit 104 may calculate the sum of the amplitude ratios for each frequency based on, for example, equation (16).

number

[0068] The hangover length control unit 105 determines an appropriate hangover length based on the target area sound judgment value acquired from the target area sound judgment value acquisition unit 103 and the non-target area sound judgment value acquired from the non-target area sound judgment value acquisition unit 104, and outputs a corresponding maximum hangover count variable Maxcount. In addition, the hangover length control unit 105 divides the target area sound judgment value R by the non-target area sound judgment value Rd based on the formula (17), thereby obtaining a judgment value ratio R jud The hangover length control unit 105 calculates the judgment value ratio R judYou may also select the maximum hangover count variable Maxcount corresponding thereto.

[0069] FIG. 4 is a diagram showing in tabular form the maximum hangover count variable Maxcount (hangover length) corresponding to the determination value ratio R jud For example, when the determination value ratio R

[0070] For example, the hangover length control unit 105, according to the table in FIG. 4, when the determination value ratio R jud is calculated to be 0.0 or more and less than α (~α), sets the maximum hangover count variable Maxcount to Count_A in order to set the hangover length to Length_A. Also, the hangover length control unit 105, according to the table in FIG. 4, when the determination value ratio R jud is calculated to be α or more and less than β (α~β), sets the maximum hangover count variable Maxcount to Count_B in order to set the hangover length to Length_B. Further, the hangover length control unit 105, according to the table in FIG. 4, when the determination value ratio R jud is calculated to be β or more (β~), sets the maximum hangover count variable Maxcount to Count_C in order to set the hangover length to Length_C.

[0071] In the example of FIG. 4, the relationship is Count_A < Count_B < Count_C, α < β. That is, the hangover length control unit 105 sets the maximum hangover count variable Maxcount to a larger value (sets a longer hangover length) as the value of the determination value ratio R jud is larger. Regarding Count_A, Count_B, Count_C, α, and β, as long as the above relationship holds, the specifically set values are not limited, but it is desirable to set suitable values in advance by experiments or the like for the hangover length control unit 105. Also, in the table shown in FIG. 4, the settable hangover lengths are in three stages, but the number of stages may be further increased.

Number

[0072] The hangover length control unit 105 stores a maximum hangover count variable Maxcount for each frame. The hangover length control unit 105 compares the maximum hangover count variable Maxcount stored in the previous frame with the maximum hangover count variable Maxcount stored in the previous frame, and assigns the result obtained by subtracting the maximum hangover count variable Maxcount stored in the previous frame from the maximum hangover count variable Maxcount of the current frame to a maximum hangover count change variable diff_Maxcount.

[0073] The target area sound section detection unit 106 judges whether or not the current frame (the input signal currently being processed) is a section (hereinafter referred to as a "target area sound section") that includes a target area sound component (for example, a speaker's speech) that satisfies a predetermined criterion. Specifically, the target area sound section detection unit 106 makes a judgment based on the target area sound judgment value acquired by the target area sound judgment value acquisition unit 103, and performs target area sound section detection based on the judgment result obtained there and the hangover length acquired by the hangover length control unit 105. In other words, even if the result of the judgment based on the target area sound judgment value is not a target area sound section, the target area sound section detection unit 106 performs processing to regard the hangover period (the period during which a hangover counter, described later, is counting down) as a target area sound section.

[0074] FIG. 5 is a flowchart showing an example of the operation of the target area sound section detection unit 106 when performing target area sound section detection.

[0075] In the following, Vad ase is a judgment result variable for holding the judgment result of the target area sound section, and is set to either 1 indicating that the current frame is a target area sound section, or 0 indicating that the current frame is not a target area sound section.

[0076] First, the target area sound section detection unit 106 holds the target area sound judgment value of the current frame acquired by the target area sound judgment value acquisition unit 103, and the maximum hangover count variable Maxcount and maximum hangover count change variable diff_Maxcount corresponding to the target area sound judgment value of the current frame (S101).

[0077] Next, the target area sound section detection unit 106 compares the acquired target area sound judgment value with a predetermined threshold value Th (S102), and if the target area sound judgment value is equal to or greater than the threshold value Th, the operation starts from step S105 described later, otherwise the operation starts from step S103 described later. The value of the threshold value Th is not limited, but it is desirable to set a suitable value in advance in the target area sound section detection unit 106 through experiments or the like.

[0078] In step S102, if the target area sound judgment value is equal to or greater than the threshold value Th, the target area sound section detection unit 106 judges the current frame to be a target area sound section and sets the judgment result variable Vad ase The CPU 101 sets 1 to the hangover counter, sets the Maxcount acquired from the hangover length control unit 105 to the hangover counter (S105), and ends the process.

[0079] On the other hand, if the destination area sound judgment value is smaller than the threshold value Th in step S102, the destination area sound section detection unit 106 changes the hangover counter by the maximum hangover count change variable diff_Maxcount, and reflects the change in the maximum hangover count variable Maxcount in the hangover counter (S103).

[0080] Next, the target area sound section detection unit 106 determines whether the hangover counter is greater than 0 (S104), and if the hangover counter is greater than 0, the operation proceeds from step S106 described below, and if not, the operation proceeds from step S107 described below.

[0081] In the above-mentioned step S104, if it is determined that the hangover counter is greater than 0 (i.e., if the hangover period is continuing), the target area sound section detection unit 106 regards the current frame as a target area sound section even if the target area sound judgment value is smaller than the threshold value Th, and sets the judgment result variable Vad ase The CPU 11 assigns 1 to the hangover counter, decrements the hangover counter by 1 (S106), and ends the process.

[0082] On the other hand, in the above-mentioned step S104, if the hangover counter is equal to or less than 0, the target area sound section detection unit 106 determines that the current frame is not a target area sound section because the area sound judgment value is smaller than the threshold value Th and the hangover counter is equal to or less than 0. ase Then, 0 is assigned to (S107), and the process ends.

[0083] As described above, the target area sound section detection unit 106 determines whether or not a section is a target area sound section, taking into account the hangover period.

[0084] The output unit 107 outputs the target area sound acquired by the target area sound acquisition unit 102 and the determination result variable Vad ase (i.e., the result of the most recent target area sound section detection). ase If is 1, output the target area sound, and Vad ase If it is 0, no sound or a small sound (for example, a specified dummy sound) is output.

[0085] (A-3) Effects of the First Embodiment According to the first embodiment, the following effects can be obtained.

[0086] In the sound collection device 10 of the first embodiment, the hangover length is controlled by calculating the judgment value ratio between the extracted target sound and the interfering sound. Specifically, in the sound collection device 10, if the judgment value ratio is large, that is, if the judgment value indicating the target area sound is large, the hangover length is lengthened to prevent sound interruption, and on the other hand, if the judgment value ratio is small, that is, if the judgment value indicating the non-target area sound is large, the hangover length is shortened to suppress erroneous extraction of the interfering sound after the speech of the target sound is finished.

[0087] (B) Second embodiment Hereinafter, a second embodiment of a sound collection device, a sound collection program, and a sound collection method according to the present invention will be described in detail with reference to the drawings.

[0088] (B-1) Configuration of the second embodiment The configuration of the sound collection device 10A of the second embodiment can be shown using the above-mentioned Figures 1 and 2, similar to the first embodiment. In Figures 1 and 2, the reference characters in parentheses are used only in the second embodiment.

[0089] The following describes the configuration of a sound collection device 10A of the second embodiment, focusing on the differences from the first embodiment.

[0090] The sound collection device 10A of the second embodiment is different from the first embodiment in that the hangover length control unit 105 is replaced with a hangover length control unit 105A.

[0091] In the first embodiment, the hangover length control unit 105 determines an appropriate hangover length based on the target area sound judgment value acquired from the target area sound judgment value acquisition unit 103 and the non-target area sound judgment value acquired from the non-target area sound judgment value acquisition unit 104, and performs a process of selecting a corresponding maximum hangover count variable Maxcount for each frame. Meanwhile, the target area sound judgment value becomes small in a frame in which a sound with low power such as an unvoiced consonant exists. Therefore, the judgment value ratio calculated based on the target area sound judgment value also becomes small, and the maximum hangover count variable Maxcount selected by referring to this judgment value ratio also becomes small. Therefore, in the target area sound section detection unit 106, even though it is a section in which the target area sound is extracted, a small maximum hangover count variable Maxcount is mistakenly set to the hangover counter counter, and as a result, there is a possibility that sound interruption occurs in the middle of the utterance of the target area sound.

[0092] Therefore, the hangover length control unit 105A of the sound collection device 10A of the second embodiment differs from the first embodiment in that it does not set a maximum hangover count variable Maxcount corresponding to the judgment value ratio for each frame, but instead sets a maximum hangover count variable Maxcount selected using a judgment value ratio acquired during a certain period. In other words, the hangover length control unit 105A of the sound collection device 10A of the second embodiment differs from the first embodiment in that it does not review the hangover length for each frame of the input signal, but reviews the hangover length for each fixed period corresponding to two or more frames of the input signal.

[0093] (B-2) Operation of the Second Embodiment Next, the operation of the sound collecting device 10A of the second embodiment having the above configuration will be described. In the sound collecting device 10A of the second embodiment, only the hangover length control unit 105A is different from the first embodiment, so the following description will focus on the operation of the hangover length control unit 105A.

[0094] As described above, the hangover length control unit 105A of the sound collection device 10A of the second embodiment changes to an operation of setting the maximum hangover count variable Maxcount selected using the judgment value ratio acquired during a certain fixed period.

[0095] Specifically, the hangover length control unit 105A calculates all the judgment value ratios R jud (k) (k=1, 2, . . . T) for each section of the judgment value ratio in Table 1, and select the maximum hangover count variable Maxcount corresponding to the section of the judgment value ratio that is the most frequent value. For example, the hangover length control unit 105A selects the maximum hangover count variable Maxcount corresponding to the section of the judgment value ratio R jud (k) (k=1, 2, . . . T) is counted for each section of the judgment value ratio, and when α to β are the most frequent values, the hangover length is set to Length_A, so that the maximum hangover count variable Maxcount is set to Count_A. jud (k) (k=1, 2, . . . T) is counted for each section of the judgment value ratio, and when α to β are the most frequent values, the hangover length is set to Length_B, so that the maximum hangover count variable Maxcount is set to Count_B. Furthermore, the hangover length control unit 105A counts the judgment value ratio R jud(k) (k=1, 2, ...T) is counted for each section of the judgment value ratio, and when β~ is the mode, the maximum hangover count variable Maxcount is set to Count_C in order to set the hangover length to Length_C. Note that the hangover length control unit 105A does not select the maximum hangover count variable Maxcount corresponding to the judgment value ratio when the frame is other than the period (k=1, 2, ...T). Therefore, when the frame is other than the period (k=1, 2, ...T-1), the maximum hangover count variable Maxcount is set to the maximum hangover count variable Maxcount selected in the previous period. Therefore, in the hangover length control unit 105A, the calculation process of the maximum hangover count change variable diff_Maxcount can be carried out in the same manner as in the first embodiment. For example, the hangover length control unit 105A can obtain the maximum hangover count change variable diff_Maxcount by comparing the maximum hangover count variable Maxcount stored in the previous frame with the maximum hangover count variable Maxcount stored in the current frame and subtracting the maximum hangover count variable Maxcount stored in the previous frame from the maximum hangover count variable Maxcount of the current frame.

[0096] In the above, the hangover length control unit 105A has been described as changing the operation to set the maximum hangover count variable Maxcount selected using the judgment value ratio acquired during a certain period, instead of setting the maximum hangover count variable Maxcount corresponding to the judgment value ratio for each frame, but the judgment value ratio may be replaced by another method as long as it is possible to avoid the judgment value ratio being erroneously reduced. For example, the hangover length control unit 105A may take the average value of the judgment value ratios acquired during a certain period, and set the maximum hangover count variable Maxcount selected using the average value. Instead of the average value, the hangover length control unit 105A may use a forgetting coefficient to increase the influence of current information and decrease the influence of past information.

[0097] Furthermore, in the description so far, the hangover length control unit 105A uses the judgment value ratio acquired during a certain period, but the target area sound judgment value and the non-target area sound judgment value for calculating the judgment value ratio may be acquired during a certain period, and the corresponding maximum hangover count variable Maxcount may be set using a value calculated by dividing the average value of the target area sound judgment values ​​acquired during the certain period by the average value of the non-target area sound judgment values. Also, the hangover length control unit 105A may use a value using a forgetting coefficient for the target area sound judgment value and a value using a forgetting coefficient for the non-target area sound judgment value instead of the average value of the target area sound judgment value and the average value of the non-target area sound judgment value acquired during the certain period.

[0098] (B-3) Effects of the Second Embodiment According to the second embodiment, in addition to the effects of the first embodiment, the following effects can be achieved.

[0099] In the sound collection device 10A of the second embodiment, when the target area sound contains a low-power sound such as an unvoiced consonant, the hangover length is not set based on the judgment value ratio for each frame, so as to prevent sound interruption from occurring in the middle of the target area sound's speech by setting the hangover length based on the judgment value ratio that occurs most frequently among the acquired judgment value ratios for each certain period (a predetermined period corresponding to two or more frames), rather than setting the hangover length based on the judgment value ratio for each frame. As a result, in the sound collection device 10A of the second embodiment, sound interruption is less likely to occur in the middle of the speech than in the first embodiment, even if the target area sound contains a low-power sound such as an unvoiced consonant.

[0100] (C) Other embodiments The present invention is not limited to the above-described embodiments, and may include modified embodiments such as those exemplified below.

[0101] (C-1) In each of the above embodiments, the target area sound judgment value acquisition unit 103 and the non-target area sound judgment value acquisition unit 104 each calculate an amplitude ratio Rk , Rd k The sums R and Rd are used as the judgment values ​​(target area sound judgment value and non-target area sound judgment value), but may be replaced with a power ratio in the time domain. For example, if the input signal (power) in the time domain that is the source of the input signal X1 is x1(t) and the signal (power) in the time domain of the target area sound Z1 is z1(t), the target area sound judgment value R may be z1(t) / x1(t) and the non-target area sound judgment value Rd may be {x1(t)-z1(t)} / x1(t).

[0102] (C-2) In each of the above embodiments, the non-target area sound judgment value acquisition unit 104 receives the input signal X 1k and destination area sound Z 1k Specifically, in each of the above-described embodiments, the non-target area sound judgment value acquisition unit 104 calculates the non-target area sound judgment value based on (X 1k -Z 1k ) is applied, but it may be replaced with the calculation result of the non-target area sound itself (for example, the non-target area sound N1 calculated by the above formula (9)). For example, the frequency k component of the non-target area sound N1 may be replaced with the non-target area sound N 1k Then, (X 1k -Z 1k ) to N 1k It may be replaced by: [Explanation of symbols]

[0103] 10, 10A...sound collection device, 101...data input unit, 102...target area sound acquisition unit, 103...target area sound judgment value acquisition unit, 104...non-target area sound judgment value acquisition unit, 105...hangover length control unit, 105A...hangover length control unit, 106...target area sound interval detection unit, 107...output unit, 201...noise suppression unit, 202...directivity formation unit, 203...delay correction unit, 204...spatial coordinate holding unit, 205...correction coefficient calculation unit, 206...target area sound extraction unit, M, M1, M2...microphone, MA, MA1, MA2...microphone array

Claims

1. a target area sound acquisition means for acquiring beamformer outputs having directivity toward a target area from input signals of a plurality of microphone arrays, and extracting target area sounds having the target area as a sound source based on the respective beamformer outputs; a target area sound judgment value acquisition means for acquiring a target area sound judgment value for the target area sound; a non-target area sound judgment value acquisition means for acquiring a non-target area sound judgment value for a non-target area sound having a sound source other than the target area; a hangover length control means for controlling a hangover length based on a comparison result between the target area sound judgment value and the non-target area sound judgment value; a target area sound section detection means for performing a target area sound section determination process for determining whether the input signal is a target area sound section including a target area sound component equal to or greater than a predetermined standard based on the target area sound determination value, and when the input signal is determined to be a target area sound section in the target area sound section determination process, the target area sound section detection means determines that the result of the target area sound section determination process is a target area sound section from that point onwards for the hangover length controlled by the hangover length control means; an output means for outputting the target area sound only while the input signal is determined to be a target area sound section in the most recently performed target area sound section determination process; A sound collecting device comprising:

2. The sound collection device according to claim 1, characterized in that the hangover length control means compares the magnitude of the target area sound judgment value and the non-target area sound judgment value, and controls the hangover length to be longer as the target area sound judgment value is larger, and controls the hangover length to be shorter as the non-target area sound judgment value is larger.

3. The sound collection device according to claim 1 , wherein the non-target area sound judgment value acquisition means acquires the non-target area sound judgment value based on a signal obtained by performing spectrum subtraction of the target area sound from the input signal.

4. 2. The sound pickup device according to claim 1, wherein the hangover length control means controls the hangover length at regular intervals corresponding to two or more frames of the input signal.

5. The sound pickup device according to claim 4, characterized in that the hangover length control means controls the hangover length using a value obtained by weighting and adding the target area sound judgment value and the non-target area sound judgment value acquired during the constant period using an average value or a forgetting coefficient.

6. Computer, a target area sound acquisition means for acquiring beamformer outputs having directivity toward a target area from input signals of a plurality of microphone arrays, and extracting target area sounds having the target area as a sound source based on the respective beamformer outputs; a target area sound judgment value acquisition means for acquiring a target area sound judgment value for the target area sound; a non-target area sound judgment value acquisition means for acquiring a non-target area sound judgment value for a non-target area sound having a sound source other than the target area; a hangover length control means for controlling a hangover length based on a comparison result between the target area sound judgment value and the non-target area sound judgment value; a target area sound section detection means for performing a target area sound section determination process for determining whether the input signal is a target area sound section including a target area sound component equal to or greater than a predetermined standard based on the target area sound determination value, and when the input signal is determined to be a target area sound section in the target area sound section determination process, the target area sound section detection means determines that the result of the target area sound section determination process is a target area sound section from that point onwards for the hangover length controlled by the hangover length control means; an output means for outputting the target area sound only while the input signal is determined to be a target area sound section in the most recently performed target area sound section determination process; A sound recording program characterized by functioning as follows.

7. In the sound collection method performed by the sound collection device, The sound collection device includes a target area sound acquisition means, a target area sound judgment value acquisition means, a non-target area sound judgment value acquisition means, a hangover length control means, a target area sound section detection means, and an output means, The target area sound acquisition means acquires beamformer outputs with directivity toward the target area from input signals of a plurality of microphone arrays, and extracts target area sounds having the target area as a sound source based on the respective beamformer outputs; The target area sound judgment value acquisition means acquires a target area sound judgment value for the target area sound, The non-target area sound judgment value acquisition means acquires a non-target area sound judgment value for a non-target area sound whose sound source is other than the target area, The hangover length control means controls a hangover length based on a comparison result between the target area sound judgment value and the non-target area sound judgment value, The target area sound section detection means performs a target area sound section determination process to determine whether the input signal is a target area sound section containing a target area sound component equal to or greater than a predetermined standard based on the target area sound determination value, and when the input signal is determined to be a target area sound section in the target area sound section determination process, the result of the target area sound section determination process is determined to be a target area sound section from that point onwards for the hangover length controlled by the hangover length control means, The output means outputs the target area sound only while the input signal is determined to be a target area sound section in the most recently performed target area sound section determination process. A sound collection method comprising:

Images