Sound source direction detection method, program, sound source direction detection device, and autonomous mobile body

Abstract

To quickly and accurately detect the direction in which a sound source is located.SOLUTION: A sound source direction detection method is intended for detecting a direction in which a sound source is located. The method includes: an audio waveform acquisition step S20 of acquiring, using multiple microphones 11 to 14, multiple audio waveforms related to the sound source; a spectrogram acquisition step S30 of acquiring multiple spectrograms on the basis of the multiple audio waveforms; a first distribution acquisition step S40 of acquiring one or more first likelihood distributions, which are likelihood distributions of the position of the sound source, on the basis of the multiple spectrograms; and a detection step S70 of detecting the direction in which the sound source is located on the basis of the one or more first likelihood distributions.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] The present invention relates to a sound source direction detection method, a program, a sound source direction detection device, and an autonomous moving body. [Background technology]

[0002] Currently, research into the autonomous driving of moving objects such as automobiles is being actively conducted. One of the technologies required for the autonomous driving of moving objects is surrounding environment recognition, and autonomous vehicles recognize the surrounding environment using cameras, LiDAR (Light Detection and Ranging), and the like. Emergency vehicles such as ambulances, police cars, and fire engines are traveling on public roads. When an emergency vehicle approaches, autonomous vehicles must perform driving actions such as giving way so as not to impede the emergency vehicle's travel, and therefore it is necessary for autonomous vehicles to quickly detect the position of the emergency vehicle. When such an emergency vehicle is hidden behind a building, another vehicle, or the like, it cannot be detected by sensors such as cameras and LiDAR. Therefore, a technology has been proposed for detecting the position of an emergency vehicle based on its siren sound (for example, Patent Document 1, etc.).

[0003] The invention described in Patent Document 1 attempts to detect the position of an emergency vehicle from audio signals and images acquired from the surrounding environment using a neural network. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2022-48199 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the invention described in Patent Document 1, training of the neural network takes time and effort, and the load of calculation processing in detecting the position of an emergency vehicle is large.

[0006] Therefore, an object of the present invention is to quickly and accurately detect the direction in which a sound source is located. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, a sound source direction detection method according to one embodiment of the present invention is a sound source direction detection method for detecting the direction in which a sound source is located, and includes an audio waveform acquisition step of acquiring multiple audio waveforms related to the sound source using multiple microphones, a spectrogram acquisition step of acquiring multiple spectrograms based on the multiple audio waveforms, a first distribution acquisition step of acquiring one or more first likelihood distributions, which are likelihood distributions of the position of the sound source, based on the multiple spectrograms, and a detection step of detecting the direction in which the sound source is located based on the one or more first likelihood distributions.

[0008] In order to achieve the above object, a program according to one aspect of the present invention is a program for causing a computer to execute the above sound source direction detection method.

[0009] In addition, in order to achieve the above-mentioned object, a sound source direction detection device according to one embodiment of the present invention is a sound source direction detection device that detects the direction in which a sound source is located, and includes an audio waveform acquisition unit that acquires multiple audio waveforms related to the sound source using multiple microphones, a spectrogram acquisition unit that acquires multiple spectrograms based on the multiple audio waveforms, a first distribution acquisition unit that acquires one or more first likelihood distributions, which are likelihood distributions of the position of the sound source, based on the multiple spectrograms, and a detection unit that detects the direction in which the sound source is located based on the one or more first likelihood distributions.

[0010] In order to achieve the above object, an autonomous moving body according to one aspect of the present invention includes the above sound source direction detection device and the multiple microphones.

[0011] In addition, these comprehensive or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a non-transitory computer-readable recording medium such as a CD-ROM, or may be realized by any combination of a system, a method, an integrated circuit, a computer program, and a recording medium. Effect of the Invention

[0012] According to the present invention, the direction in which a sound source is located can be detected quickly and accurately. [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram showing an example of the configuration of a sound source direction detecting device and an autonomous moving body according to a first embodiment. [Diagram 2] FIG. 2 is a schematic plan view showing an arrangement of a plurality of microphones according to the first embodiment. [Diagram 3] 4 is a graph showing an example of a sound waveform detected by each microphone according to the first embodiment. [Figure 4] 4 is a graph showing an example of a spectrogram acquired by a spectrogram acquisition unit according to the first embodiment. [Diagram 5] 1 is a schematic diagram showing an overview of a method for detecting the direction in which a sound source is located by using a phase difference between sounds; [Figure 6] 4 is a first diagram showing a first likelihood distribution obtained from two spectrograms corresponding to the microphones in the first embodiment. FIG. [Figure 7] 4 is a second diagram showing a first likelihood distribution obtained from two spectrograms corresponding to the microphones in the first embodiment. FIG. [Figure 8] 11 is a third diagram showing a first likelihood distribution obtained from two spectrograms corresponding to the microphones in the first embodiment. FIG. [Figure 9] FIG. 4 is a fourth diagram showing a first likelihood distribution obtained from two spectrograms corresponding to the microphones in the first embodiment. [Figure 10]FIG. 5 is a fifth diagram showing a first likelihood distribution obtained from two spectrograms corresponding to the microphones in the first embodiment. [Figure 11] FIG. 6 is a sixth diagram showing a first likelihood distribution obtained from two spectrograms corresponding to the microphones in the first embodiment. [Figure 12] FIG. 4 is a diagram showing an example of an integrated likelihood distribution according to the first embodiment. [Figure 13] 4 is a flowchart showing the flow of a sound source direction detection method according to the first embodiment. [Figure 14] FIG. 11 is a block diagram showing an example of the configuration of a sound source direction detecting device and an autonomous moving body according to the second embodiment. [Figure 15] 11 is a first diagram showing a second likelihood distribution and a third likelihood distribution acquired from two voice waveforms detected by a microphone in accordance with the second embodiment. FIG. [Figure 16] 13 is a second diagram showing a second likelihood distribution and a third likelihood distribution acquired from two voice waveforms detected by a microphone in accordance with the second embodiment. FIG. [Figure 17] 13 is a third diagram showing a second likelihood distribution and a third likelihood distribution acquired from two voice waveforms detected by a microphone in accordance with the second embodiment. FIG. [Figure 18] 13 is a fourth diagram showing a second likelihood distribution and a third likelihood distribution acquired from two voice waveforms detected by a microphone in accordance with the second embodiment. FIG. [Figure 19] 13 is a fifth diagram showing a second likelihood distribution and a third likelihood distribution acquired from two voice waveforms detected by a microphone in accordance with the second embodiment. FIG. [Figure 20] 13 is a sixth diagram showing a second likelihood distribution and a third likelihood distribution acquired from two voice waveforms detected by a microphone in accordance with the second embodiment. FIG. [Figure 21] FIG. 11 is a diagram showing an example of a fourth likelihood distribution according to the second embodiment. [Figure 22] FIG. 11 is a diagram showing an example of a fourth likelihood distribution corresponding to a priori probability distribution according to the second embodiment. [Figure 23]FIG. 11 is a diagram showing an example of a fourth likelihood distribution corresponding to a posterior probability distribution according to the second embodiment. [Figure 24] 10 is a flowchart showing the flow of a sound source direction detection method according to the second embodiment. [Diagram 25] 13 is a graph showing the accuracy of a detection result obtained by the sound source direction detection method according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0015] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component arrangement and connection forms, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in an independent claim showing a top concept are described as optional components.

[0016] In addition, each drawing is a schematic diagram and is not necessarily a precise illustration. In addition, in each drawing, the same components are denoted by the same reference numerals.

[0017] (Embodiment 1) A sound source direction detecting method, a sound source direction detecting device, and an autonomous moving body according to a first embodiment will be described.

[0018] [1-1. Configuration of sound source direction detection device and autonomous mobile body] The configuration of a sound source direction detecting device and an autonomous moving body according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of a sound source direction detecting device 100 and an autonomous moving body V10 according to this embodiment.

[0019] The autonomous moving body V10 is an autonomously driven moving body, and includes a plurality of microphones 11-14 and a sound source direction detection device 100. In this embodiment, the autonomous moving body V10 further includes a travel control unit 80 and a storage unit 90. The autonomous moving body V10 is not particularly limited as long as it is a moving body. In this embodiment, the autonomous moving body V10 is a vehicle that performs autonomous driving. Hereinafter, the autonomous moving body V10 is also referred to as the host vehicle.

[0020] The microphones 11 to 14 are sound receivers that detect a plurality of sound waveforms (i.e., time waveforms of sound intensity) related to a sound source. In this embodiment, the number of the microphones 11 to 14 is four, but the number of the microphones is not limited to four and may be two or more. For example, the microphones may include three or more microphones. The arrangement of the microphones 11 to 14 according to this embodiment will be described with reference to FIG. 2. FIG. 2 is a schematic plan view showing the arrangement of the microphones 11 to 14 according to this embodiment. FIG. 2 shows a plan view of the autonomous moving body V10 viewed from above the autonomous moving body V10. FIG. 2 also shows mutually perpendicular x-axis and y-axis indicating coordinates in the horizontal direction. As shown in FIG. 2, the direction from the rear to the front of the autonomous moving body V10 is the direction from negative to positive in the x-axis direction, and the direction from the right side to the left side of the autonomous moving body V10 is the direction from negative to positive in the y-axis direction.

[0021] As shown in Fig. 2, in this embodiment, the multiple microphones 11-14 include four microphones 11-14 respectively arranged at positions corresponding to the four vertices of a rectangle R0. In other words, one microphone is arranged at each vertex of the rectangle R0. The multiple microphones 11-14 are arranged, for example, on the roof of the autonomous moving body V10. When the multiple microphones 11-14 are arranged outside the vehicle in this manner, each of the multiple microphones 11-14 may be fitted with a sponge-like microphone cover to suppress detection of wind noise that becomes noise.

[0022] In this embodiment, the phase difference between two sound waveforms detected by two microphones is used to detect the direction of the sound source. In such a detection method, three or more microphones do not need to be arranged on the same line. This allows the sound waveforms detected by each microphone to be effectively used to accurately detect the direction of the sound source.

[0023] The sound source direction detection device 100 is a device that detects the direction in which a sound source is located. The sound source direction detection device 100 according to this embodiment detects the direction in which a sound source is located using a sound source direction detection method described later. The sound source is not particularly limited as long as it is a sound source. For example, an emergency vehicle, a railroad crossing, etc. are assumed as the sound source. In the following, as an example, a case in which the sound source is an ambulance sounding a siren will be described. The sound source direction detection device 100 has a voice waveform acquisition unit 20, a spectrogram acquisition unit 30, a first distribution acquisition unit 40, and a detection unit 70.

[0024] The voice waveform acquisition unit 20 is a processing unit that acquires a plurality of voice waveforms related to a sound source using a plurality of microphones 11 to 14. In this embodiment, the voice waveform acquisition unit 20 acquires four voice waveforms that correspond one-to-one to the plurality of microphones 11 to 14. The voice waveform acquisition unit 20 acquires a voice waveform, for example, every 0.1 seconds. The sampling frequency of the voice waveform in the voice waveform acquisition unit 20 is not particularly limited as long as it is large enough to sufficiently reproduce the voice waveform. The sampling frequency of the voice waveform in the voice waveform acquisition unit 20 is, for example, 44.1 kHz or 48 kHz. Here, an example of a voice waveform acquired by the voice waveform acquisition unit 20 will be described with reference to FIG. 3. FIG. 3 is a graph showing an example of a voice waveform detected by each microphone according to this embodiment. The horizontal axis of FIG. 3 indicates time, and the vertical axis indicates the amplitude of the sound. The four voice waveforms acquired using each of the plurality of microphones 11 to 14 have similar shapes and are out of phase with each other.

[0025] The spectrogram acquisition unit 30 is a processing unit that acquires a plurality of spectrograms based on a plurality of audio waveforms acquired by the audio waveform acquisition unit 20. In this embodiment, the spectrogram acquisition unit 30 acquires four spectrograms that correspond one-to-one to the four audio waveforms. A spectrogram is a graph that represents the relationship between the frequency and intensity of a sound and time. The spectrogram acquisition unit 30 acquires a spectrogram, for example, by performing a short-time Fourier transform on the audio waveform. The spectrogram will be described with reference to FIG. 4. FIG. 4 is a graph showing an example of a spectrogram acquired by the spectrogram acquisition unit 30 according to this embodiment. The horizontal axis of FIG. 4 indicates time, and the vertical axis indicates the frequency of the sound. In FIG. 4, the intensity of the sound is shown in gray scale. In FIG. 4, the intensity of the sound increases as the color approaches white. As shown in FIG. 4, the frequency (i.e., the pitch) of an ambulance siren fluctuates periodically. More specifically, the frequency of an ambulance siren oscillates periodically in a step shape. In other words, an ambulance siren alternates between periods of a low pitch at a nearly constant pitch and periods of a high pitch at a nearly constant pitch.

[0026] The frequency range of the ambulance siren detected by each microphone is 650 Hz to 1137 Hz, taking into account the frequency change caused by the Doppler effect. Therefore, in this embodiment, the frequency range for calculating the spectrogram is set to 600 Hz to 1200 Hz. The time width of each spectrogram may be 0.65 seconds to 1 second. This allows each spectrogram to include at least one edge. In this embodiment, the time width of each spectrogram is 0.8 seconds.

[0027] The first distribution acquisition unit 40 is a processing unit that acquires one or more first likelihood distributions, which are likelihood distributions of the positions of sound sources, based on the multiple spectrograms acquired by the spectrogram acquisition unit 30. In this embodiment, in the first distribution acquisition unit 40, each of the one or more first likelihood distributions is acquired based on two spectrograms out of the multiple spectrograms. Moreover, the one or more first likelihood distributions include two or more first likelihood distributions.

[0028] In addition, since the time resolution of a spectrogram is lower than that of a speech waveform, it is difficult to obtain the correlation between two spectrograms. Therefore, in this embodiment, the first distribution acquisition unit 40 detects multiple edges from multiple spectrograms. Here, an edge means a portion (vertical edge in FIG. 4) where the time derivative of the frequency in the spectrogram (horizontal derivative in FIG. 4) changes suddenly. The first distribution acquisition unit 40 detects one or more edges in each spectrogram by differentiating each of the multiple spectrograms with respect to time. The first distribution acquisition unit 40 acquires one first likelihood distribution based on the phase difference between the edges of two spectrograms among the multiple spectrograms. Note that, when a lot of noise is included in the spectrogram, the edges may be detected after performing gamma correction and smoothing using a Gaussian filter on the spectrogram.

[0029] Here, an overview of a method for acquiring the likelihood of the direction in which a sound source is located using the phase difference of sound will be described with reference to FIG. 5. FIG. 5 is a schematic diagram showing an overview of a method for detecting the direction in which a sound source is located using the phase difference of sound. FIG. 5 shows microphones 11 and 12, and an ambulance SS, which is an example of a sound source. As shown in FIG. 5, the inclination of a straight line passing through the ambulance SS and the microphone 12 with respect to the arrangement direction in which the microphones 11 and 12 are arranged is θ. Here, it is assumed that the inclination of a straight line passing through the ambulance SS and the microphone 11 with respect to the arrangement direction can also be regarded as θ. In other words, it is assumed that the distance between the ambulance SS and the microphones 11 and 12 is sufficiently long with respect to the distance d12 between the microphones 11 and 12. In this case, the inclination θ can be calculated from the phase of the sound from the ambulance SS arriving at the microphone 12, the phase difference φ12 of the sound from the ambulance SS arriving at the microphone 11, and the distance d12. In this embodiment, the likelihood of the direction in which the sound source is located is obtained using the phase difference between edges detected in each of the two spectrograms.

[0030] The first distribution acquisition unit 40 acquires one or more first likelihood distributions based on the cross-correlation of a plurality of edges detected in a plurality of spectrograms. In this embodiment, normalized cross-correlation is used as an example of the cross-correlation. The normalized cross-correlation is one of the methods for evaluating the similarity of two signals. The larger the correlation value of the normalized cross-correlation, the higher the likelihood (probability of the existence of a sound source). In this embodiment, the first distribution acquisition unit 40 acquires six first likelihood distributions from four spectrograms. Hereinafter, the first likelihood distribution according to this embodiment will be described with reference to Figs. 6 to 11. Fig. 6 is a diagram showing a first likelihood distribution acquired from two spectrograms corresponding to the microphones 11 and 12 according to this embodiment. Fig. 7 is a diagram showing a first likelihood distribution acquired from two spectrograms corresponding to the microphones 12 and 13 according to this embodiment. Fig. 8 is a diagram showing a first likelihood distribution acquired from two spectrograms corresponding to the microphones 13 and 14 according to this embodiment. Fig. 9 is a diagram showing a first likelihood distribution acquired from two spectrograms corresponding to the microphones 14 and 11 according to this embodiment. FIG. 10 is a diagram showing a first likelihood distribution obtained from two spectrograms corresponding to the microphones 11 and 13 according to the present embodiment. FIG. 11 is a diagram showing a first likelihood distribution obtained from two spectrograms corresponding to the microphones 12 and 14 according to the present embodiment. The center of each of FIGS. 6 to 11 corresponds to the center of the autonomous moving body V10 (the intersection of the x-axis and the y-axis shown in FIG. 2). The horizontal and vertical directions of FIGS. 6 to 11 correspond to the x-axis and y-axis directions shown in FIG. 2, respectively. In this embodiment, each first likelihood distribution shows the likelihood in the ranges of -100m≦x≦100m and -100m≦y≦100m. In each first likelihood distribution, the probability of existence of a sound source at points plotted at 2m intervals in each of the x-axis and y-axis directions is shown in grayscale. In FIGS. 6 to 11, the likelihood increases as the color approaches white. In each first likelihood distribution, a direction with high likelihood is shown.

[0031] The detection unit 70 is a processing unit that detects the direction in which the sound source is located based on one or more first likelihood distributions acquired by the first distribution acquisition unit 40. In this embodiment, the first distribution acquisition unit 40 acquires two or more first likelihood distributions. The detection unit 70 detects the direction in which the sound source is located based on an integrated likelihood distribution that integrates two or more first likelihood distributions. Here, the integrated likelihood distribution will be described with reference to FIG. 12. FIG. 12 is a diagram showing an example of the integrated likelihood distribution according to this embodiment.

[0032] In this embodiment, the integrated likelihood distribution shown in Fig. 12 is obtained by multiplying the likelihood (probability of existence of the sound source) of each point in each first likelihood distribution shown in Fig. 6 to Fig. 11. In each of the first likelihood distributions shown in Fig. 6 to Fig. 11, the direction in which the sound source has a high probability of existence does not necessarily match, but by integrating these, one direction can be detected as the direction in which the sound source has a high probability of existence, as shown in Fig. 12. Note that when the direction in which the sound source has a high probability of existence spans a predetermined range, the center of the range may be detected as the direction in which the sound source is located.

[0033] An example of a specific method for calculating the integrated likelihood distribution will be described. When calculating the integrated likelihood distribution based on the six patterns of first likelihood distributions as shown in Figs. 6 to 11, the coordinates in the first likelihood distribution of the nth pattern (1≦n≦6) are represented as (x, y)=(i, j), and the correlation value at the coordinates is represented as C n,(i,j) (-1≦C n,(i,j) ≦1). In this case, the correlation value C at the coordinate (i,j) is expressed by the following formula: n,(i,j) Let P be the weight 1,n,(i,j) Convert to.

[0034]

number

[0035] Here, a and b are constants. In this embodiment, a=1 and b=2 are set. In this case, the weight P 1,n,(i,j) The minimum value of is 0 and the maximum value is 1. In the first likelihood distribution of six patterns (i.e., 1≦n≦6), this weight P 1,n,(i,j)The integration weight P is calculated by multiplying (i,j) Calculate.

[0036]

number

[0037] This integration weight P 1,n,(i,j) By calculating for all coordinates, it is possible to calculate an integrated likelihood distribution, which is indicated by a likelihood in the same range as the probability value range from 0 to 1.

[0038] The driving control unit 80 is a processing unit that controls the driving of the vehicle. In this embodiment, the driving control unit 80 controls the driving so as to avoid a collision with an emergency vehicle or the like, based on the direction in which a sound source such as an emergency vehicle detected by the sound source direction detection device 100 is located. The driving control unit 80 controls the traveling direction of the vehicle, for example, based on the position of the vehicle, map information stored in the storage unit 90, and the like, in addition to information on the direction in which the sound source is located. The driving control unit 80 may also use information on the surrounding environment acquired by a camera, LiDAR, and the like.

[0039] The storage unit 90 stores information for estimating the position of the vehicle. For example, the storage unit 90 stores map information including information on the road on which the vehicle is driving. The storage unit 90 may also store information for automatic driving.

[0040] [1-2. Sound source direction detection method] The sound source direction detection method according to this embodiment will be described with reference to Fig. 13. Fig. 13 is a flowchart showing the flow of the sound source direction detection method according to this embodiment.

[0041] The sound source direction detection method according to this embodiment is a method for detecting the direction in which a sound source is located. In the sound source direction detection method according to this embodiment, first, as shown in FIG. 13, the voice waveform acquisition unit 20 acquires a plurality of voice waveforms related to the sound source using the plurality of microphones 11-14 (voice waveform acquisition step S20). The plurality of microphones 11-14 may include three or more microphones. In this embodiment, the plurality of microphones 11-14 includes four microphones that are respectively arranged at positions corresponding to the four vertices of a rectangle.

[0042] Next, the spectrogram acquisition unit 30 acquires a plurality of spectrograms based on the plurality of voice waveforms acquired in the voice waveform acquisition step S20 (spectrogram acquisition step S30).

[0043] Next, the first distribution acquisition unit 40 acquires one or more first likelihood distributions, which are likelihood distributions of the positions of the sound sources, based on the multiple spectrograms (first distribution acquisition step S40). Each of the one or more first likelihood distributions is acquired based on two spectrograms out of the multiple spectrograms. In the first distribution acquisition step S40 according to this embodiment, the one or more first likelihood distributions include two or more first likelihood distributions. As described above, the first distribution acquisition unit 40 detects multiple edges from the multiple spectrograms and acquires one or more first likelihood distributions based on the cross-correlation of the multiple edges. In this embodiment, the multiple edges are detected by time-differentiating the multiple spectrograms. In addition, normalized cross-correlation is used as an example of cross-correlation.

[0044] Next, the detection unit 70 detects the direction in which the sound source is located based on the one or more first likelihood distributions acquired in the first distribution acquisition step S40 (detection step S70). In this embodiment, the one or more first likelihood distributions include two or more first likelihood distributions, and in the detection step S70, the direction in which the sound source is located is detected based on an integrated likelihood distribution in which the one or more first likelihood distributions are integrated.

[0045] In the present embodiment, the direction in which the sound source is located can be accurately detected by the above-described sound source direction detection method. In addition, the sound source direction detection method according to the present embodiment does not require complicated calculations, so the load of calculation processing is small and quick detection is possible.

[0046] [1-3. Effects] The effects of the sound source direction detecting method (and sound source direction detecting device 100) according to this embodiment will be described.

[0047] The sound source direction detection method of this embodiment is a method for detecting the direction in which a sound source is located, and includes an audio waveform acquisition step S20 of acquiring multiple audio waveforms related to the sound source using multiple microphones, a spectrogram acquisition step S30 of acquiring multiple spectrograms based on the multiple audio waveforms, a first distribution acquisition step S40 of acquiring one or more first likelihood distributions, which are likelihood distributions of the position of the sound source, based on the multiple spectrograms, and a detection step S70 of detecting the direction in which the sound source is located based on the one or more first likelihood distributions.

[0048] As a result, even if the sound source is hidden behind another vehicle, the direction of the sound source can be accurately detected. Furthermore, the sound source direction detection method according to the embodiment does not require complex calculations, so rapid detection is possible.

[0049] In the first distribution acquisition step of the sound source direction detection method according to this embodiment, a plurality of spectrograms may be differentiated over time to detect a plurality of edges, and one or more first likelihood distributions may be acquired based on the cross-correlation of the plurality of edges.

[0050] In this way, the correlation between two spectrograms can be easily calculated using the edges of the spectrograms, and the phase difference between the edges detected in each of the two spectrograms can be easily calculated using, for example, normalized cross-correlation.

[0051] In the sound source direction detection method of this embodiment, the multiple microphones include three or more microphones, the one or more first likelihood distributions include two or more first likelihood distributions, and in the first distribution acquisition step S40, each of the one or more first likelihood distributions is acquired based on two spectrograms out of the multiple spectrograms, and in the detection step S70, the direction in which the sound source is located may be detected based on an integrated likelihood distribution that integrates the one or more first likelihood distributions.

[0052] By using the integrated likelihood distribution acquired based on two or more first likelihood distributions in this manner, the direction in which the sound source is located can be detected more accurately.

[0053] In the sound source direction detecting method according to the present embodiment, the plurality of microphones may include four microphones 11 to 14 arranged at positions corresponding to the four vertices of a rectangle.

[0054] In this way, by arranging the four microphones 11 to 14, it is possible to avoid arranging three or more microphones on the same line. Therefore, the sound waveforms detected by each microphone can be effectively used to accurately detect the direction in which the sound source is located.

[0055] The sound source direction detection device 100 of this embodiment is a device that detects the direction in which a sound source is located, and includes an audio waveform acquisition unit 20 that acquires multiple audio waveforms related to the sound source using multiple microphones, a spectrogram acquisition unit 30 that acquires multiple spectrograms based on the multiple audio waveforms, a first distribution acquisition unit 40 that acquires one or more first likelihood distributions, which are likelihood distributions of the position of the sound source, based on the multiple spectrograms, and a detection unit 70 that detects the direction in which the sound source is located based on the one or more first likelihood distributions.

[0056] This provides the same effects as the sound source direction detection method described above.

[0057] An autonomous moving body V10 according to this embodiment includes a sound source direction detecting device 100 and a plurality of microphones.

[0058] This provides the same effects as the sound source direction detection method described above.

[0059] The sound source direction detecting device 100 according to this embodiment also provides the same effects as those of the above-described sound source direction detecting method.

[0060] (Embodiment 2) A sound source direction detection method, a sound source direction detection device, and an autonomous moving body according to embodiment 2 will be described. The sound source direction detection method according to this embodiment is different from the first likelihood distribution according to embodiment 1 in that the sound source direction detection method detects the direction in which the sound source is located by using not only the first likelihood distribution according to embodiment 1 but also the second likelihood distribution acquired based on a voice waveform. The sound source direction detection method according to this embodiment will be described below, focusing on the differences from the sound source direction detection method according to embodiment 1.

[0061] [2-1. Configuration of sound source direction detection device and autonomous mobile body] The configuration of a sound source direction detecting device and an autonomous moving body according to this embodiment will be described with reference to Fig. 14. Fig. 14 is a block diagram showing an example of the configuration of a sound source direction detecting device 200 and an autonomous moving body V20 according to this embodiment.

[0062] The autonomous moving body V20 is an autonomous moving body that performs automatic driving, and includes a plurality of microphones 11-14 and a sound source direction detection device 200. In this embodiment, the autonomous moving body V20 further includes a travel control unit 80 and a storage unit 90.

[0063] The sound source direction detecting device 200 according to this embodiment detects the direction in which a sound source is located by using a sound source direction detecting method described later. The sound source direction detecting device 200 includes a speech waveform acquiring unit 20, a spectrogram acquiring unit 30, a first distribution acquiring unit 40, a second distribution acquiring unit 250, and a detecting unit 270.

[0064] The first distribution acquisition unit 40 according to this embodiment acquires two or more first likelihood distributions, similarly to the embodiment 1. Furthermore, each of the two or more first likelihood distributions is acquired based on two spectrograms among the multiple spectrograms.

[0065] The second distribution acquisition unit 250 is a processing unit that acquires one or more second likelihood distributions, which are likelihood distributions of the position of a sound source, based on a plurality of voice waveforms acquired by the voice waveform acquisition unit 20. In this embodiment, the one or more second likelihood distributions include two or more second likelihood distributions, and each of the one or more second likelihood distributions is acquired based on two voice waveforms among the plurality of voice waveforms. The second distribution acquisition unit 250 acquires one or more second likelihood distributions based on the cross-correlation of the plurality of voice waveforms. In this embodiment, the second distribution acquisition unit 250 acquires six second likelihood distributions from four voice waveforms detected by the four microphones 11 to 14. In addition, normalized cross-correlation is used as an example of cross-correlation. The second likelihood distribution according to this embodiment will be described below with reference to Figs. 15 to 20. Fig. 15 is a diagram showing the second likelihood distribution and the third likelihood distribution acquired from two voice waveforms detected by the microphones 11 and 12 according to this embodiment. The third likelihood distribution will be described later. FIG. 16 is a diagram showing the second likelihood distribution and the third likelihood distribution acquired from two voice waveforms detected by the microphones 12 and 13 according to the present embodiment. FIG. 17 is a diagram showing the second likelihood distribution and the third likelihood distribution acquired from two voice waveforms detected by the microphones 13 and 14 according to the present embodiment. FIG. 18 is a diagram showing the second likelihood distribution and the third likelihood distribution acquired from two voice waveforms detected by the microphones 14 and 11 according to the present embodiment. FIG. 19 is a diagram showing the second likelihood distribution and the third likelihood distribution acquired from two voice waveforms detected by the microphones 11 and 13 according to the present embodiment. FIG. 20 is a diagram showing the second likelihood distribution and the third likelihood distribution acquired from two voice waveforms detected by the microphones 12 and 14 according to the present embodiment. The distribution diagram (a) shown in each of FIG. 15 to FIG. 20 shows the second likelihood distribution, and the distribution diagram (b) shows the third likelihood distribution.

[0066] The center of each distribution diagram (a) and each distribution diagram (b) in Figs. 15 to 20 corresponds to the center of the autonomous moving body V20 (see the intersection of the x-axis and y-axis shown in Fig. 2). The horizontal and vertical directions of each distribution diagram (a) and each distribution diagram (b) in Figs. 15 to 20 correspond to the x-axis and y-axis directions shown in Fig. 2, respectively. In this embodiment, each second likelihood distribution shows likelihoods in the ranges of -100m≦x≦100m and -100m≦y≦100m. Each second likelihood distribution shows, in grayscale, the existence probability of a sound source at points plotted at 2m intervals in each of the x-axis and y-axis directions.

[0067] As shown in FIGS. 15 to 20, in the second likelihood distribution, the likelihood is high at positions corresponding to a plurality of directions in a striped pattern.

[0068] The detection unit 270 detects the direction in which the sound source is located based on both one or more first likelihood distributions and one or more second likelihood distributions. In this embodiment, the detection unit 270 acquires one or more third likelihood distributions by integrating an integrated likelihood distribution obtained by integrating one or more first likelihood distributions with each of one or more second likelihood distributions, and detects the direction in which the sound source is located based on a fourth likelihood distribution acquired by integrating one or more third likelihood distributions. For example, the detection unit 270 multiplies the second likelihood distribution shown in the distribution diagram (a) of FIG. 15 by the integrated likelihood distribution as shown in FIG. 12 to acquire the third likelihood distribution as shown in the distribution diagram (b) of FIG. 15. Similarly, the detection unit 270 multiplies the second likelihood distribution shown in each distribution diagram (a) of FIG. 16 to FIG. 20 by the integrated likelihood distribution to acquire the third likelihood distribution shown in each distribution diagram (b) of FIG. 16 to FIG. 20. Furthermore, the detection unit 270 obtains a fourth likelihood distribution by multiplying the third likelihood distribution shown in each distribution diagram (b) of FIG. 15 to FIG.

[0069] The integration weight P (i,j) In this embodiment, the correction is performed based on the following formula.

[0070]

number

[0071] This results in the corrected integrated weight P' (i,j) The range of X min For example, the uncorrected integrated weight P (i,j) If the maximum value of is less than 0.5, the integrated likelihood distribution tends not to show the correct direction. For this reason, for example, the integrated weight P (i,j) If the maximum value of is less than 0.5, then X min By setting to 0.9, the corrected integration weight P' (i,j) is converted to a range of 0.9 to 1. This makes it possible to reduce the influence of the integrated likelihood distribution that does not indicate the correct direction on the third likelihood distribution and the fourth likelihood distribution.

[0072] Furthermore, in the calculation of the third likelihood distribution and the fourth likelihood distribution, the likelihood value may become too small overall due to repeated multiplication of likelihoods between 0 and 1. In order to prevent the likelihood value from becoming too small in this way, for example, an integrated weight P (i,j) If any of the values ​​exceeds 0.5, X min By setting to 0.7, the corrected integration weight P' (i,j) The range may be converted to 0.7 to 1.

[0073] Next, the calculation of the third likelihood distribution will be described in detail. The weight P 2,n,(i,j) (Likelihood) is the weight P in the first likelihood distribution 1,n,(i,j) The weight P 2,n,(i,j) is the corrected integration weight P' (i,j) Using the following formula, the weight P in the third likelihood distribution is 3,n,(i,j) is converted to

[0074]

number

[0075] By performing this transformation on all coordinates of the second likelihood distribution, the third likelihood distribution can be calculated. Note that the weights P 3,n,(i,j) may be corrected by the following formula:

[0076]

number

[0077] This results in the corrected weight P' 3,n,(i,j) The range of X min The weight P in the third likelihood distribution 3,n,(i,j) If is less than 0.6, there is little correlation between the two voice waveforms, i.e., the sound from the sound source (such as an ambulance siren) is not recognized, or the joint likelihood distribution does not indicate the correct direction. Therefore, the weight P 3,n,(i,j) If the maximum value of is less than 0.6, then X min By setting to 0.9, the corrected weight P' 3,n,(i,j) is converted to a range of 0.9 to 1. This makes it possible to reduce the influence of the second likelihood distribution and the integrated likelihood distribution, which do not indicate the correct direction, on the third likelihood distribution and the fourth likelihood distribution.

[0078] In addition, in order to prevent the likelihood value from becoming too small, for example, the weight P 3,n,(i,j) If any of the values ​​exceeds 0.6, X min By setting to 0.6, the corrected weight P' 3,n,(i,j) The range may be converted to 0.6 or more and 1 or less.

[0079] The fourth likelihood distribution will be described with reference to Fig. 21. Fig. 21 is a diagram showing an example of the fourth likelihood distribution according to the present embodiment. Fig. 21 shows the fourth likelihood distribution acquired at a certain time point t. Fig. 21 also shows the direction in which the sound source is actually located by a dashed arrow.

[0080] The weight P at the coordinate (i,j) of the fourth likelihood distribution 4,(i,j) is the weight P' as shown in the following formula: 3,n,(i,j) It is calculated by multiplying

[0081]

number

[0082] The direction in which the sound source is located can be detected based on the area closest to white in the fourth likelihood distribution shown in Fig. 21. The detected direction is almost the same as the actual direction in which the sound source is located, as indicated by the dashed arrow in Fig. 21. In this way, the sound source direction detection device 200 according to this embodiment can accurately detect the direction in which the sound source is located.

[0083] Furthermore, detection unit 270 according to this embodiment acquires a posterior probability distribution by performing time series processing of a probability distribution related to one or more first likelihood distributions. In this embodiment, time series processing of a fourth likelihood distribution is performed as a probability distribution related to one or more first likelihood distributions. The time series processing according to this embodiment will be described with reference to Figs. 21 to 23. Figs. 22 and 23 are diagrams showing examples of a fourth likelihood distribution corresponding to the prior probability distribution and the posterior probability distribution according to this embodiment, respectively.

[0084] When the detection unit 270 acquires a fourth likelihood distribution as shown in Fig. 21 at a certain time t, the detection unit 270 acquires a posterior probability distribution as shown in Fig. 23 using the fourth likelihood distribution acquired at time t and the fourth likelihood distribution (corresponding to the prior probability distribution at time t and the posterior probability distribution at time t-1) at time t-1 immediately before time t (i.e., the time when the fourth likelihood distribution was acquired immediately before time t) as shown in Fig. 22. As a result, even if the direction detected based on the fourth likelihood distribution acquired at a certain time significantly differs from the direction in which the sound source is actually located due to the influence of noise, etc., the influence of such noise can be reduced by using the prior probability distribution.

[0085] In this embodiment, the specific method of the time series processing is not particularly limited. In this embodiment, a BBF (Binary Bayes Filter) is used in the time series processing. An example of the time series processing according to this embodiment will be described below. A fourth likelihood distribution z t The probability p estimated from the fourth likelihood distribution weight P 4,(i,j) Using this, it is expressed by the following formula:

[0086]

number

[0087] Here, since the sound source direction detection according to the present embodiment is performed over a long period of time, it is necessary to prevent the logarithmic odds value from becoming excessively large. 4,(i,j) The range of X using formula (7) min More than X max Probability P corrected to the following range (i,j) (p|z t In this embodiment, X used in formula (7) min For example, 0.4 is used as X max For example, 0.6 is used.

[0088] The posterior probability expressed as the log odds at time t is L t,(i,j) (p), the posterior probability up to the time point t-1 immediately before time point t is L t-1,(i,j) However, when the relative speed between the autonomous moving body V20 and the sound source is greater than zero, the direction detected from the fourth likelihood distribution may change at each time point, and therefore, a deviation may occur in the detected direction in each of the fourth likelihood distributions at time point t-1 and time point t. In order to correct this deviation, the posterior probability L t-1,(i,j) A Gaussian filter is applied to (p). This can increase the detection response speed. The posterior probability L' t-1,(i,j) Using (p), the posterior probability L at time t t,(i,j) (p) is expressed by the following formula.

[0089]

number

[0090] As shown in equation (7), in order to prevent the posterior probability value from becoming excessively large, the posterior probability L' t-1,(i,j) (p) is multiplied by 0.95. The posterior probability L expressed in log odds t,(i,j) (p) is the posterior probability P (i,j) (p|z 1:t ) can be converted to

[0091]

number

[0092] For all coordinates, the posterior probability P (i,j) (p|z 1:t ), the posterior probability distribution at time t as shown in FIG. 23 can be calculated.

[0093] [2-2. Sound source direction detection method] The sound source direction detection method according to this embodiment will be described with reference to Fig. 24. Fig. 24 is a flowchart showing the flow of the sound source direction detection method according to this embodiment.

[0094] In the sound source direction detection method according to the present embodiment, as shown in FIG. 24, a voice waveform acquisition step S20, a spectrogram acquisition step S30, and a first distribution acquisition step S40 are executed, similarly to the sound source direction detection method according to the first embodiment.

[0095] Next, the second distribution acquisition unit 250 of the sound source direction detection device 200 acquires one or more second likelihood distributions, which are likelihood distributions of the position of the sound source, based on the multiple voice waveforms (second distribution acquisition step S250). In this embodiment, in the second distribution acquisition step S250, one or more second likelihood distributions are acquired based on the cross-correlation of the multiple voice waveforms. In this embodiment, normalized cross-correlation is used as an example of the cross-correlation. Moreover, the one or more second likelihood distributions include two or more second likelihood distributions, and in the second distribution acquisition step S250, each of the one or more second likelihood distributions is acquired based on two voice waveforms among the multiple voice waveforms.

[0096] Next, the detection unit 270 detects the direction in which the sound source is located based on both the one or more first likelihood distributions and the one or more second likelihood distributions (detection step S270). In this embodiment, in the detection step S270, an integrated likelihood distribution obtained by integrating the one or more first likelihood distributions is integrated with each of the one or more second likelihood distributions to obtain one or more third likelihood distributions, and the direction in which the sound source is located is detected based on a fourth likelihood distribution obtained by integrating the one or more third likelihood distributions.

[0097] In addition, in the detection step S270, a posterior probability distribution is obtained by performing time series processing of the probability distribution related to one or more first likelihood distributions. In this embodiment, for example, a fourth likelihood distribution can be used as the probability distribution related to the one or more first likelihood distributions.

[0098] In the time series processing used in the detection step S270, for example, BBF can be used.

[0099] In this embodiment, the direction in which the sound source is located can be accurately detected by the above-mentioned sound source direction detection method. Also, like the sound source direction detection method according to the first embodiment, the sound source direction detection method according to this embodiment does not require complicated calculations, so that rapid detection is possible.

[0100] [2-3. Effects] The effects of the sound source direction detecting method (and sound source direction detecting device 200) according to this embodiment will be described.

[0101] The sound source direction detection method of this embodiment is a sound source direction detection method for detecting the direction in which a sound source is located, similar to the sound source direction detection method of embodiment 1, and includes an audio waveform acquisition step S20 for acquiring multiple audio waveforms related to the sound source using multiple microphones, a spectrogram acquisition step S30 for acquiring multiple spectrograms based on the multiple audio waveforms, a first distribution acquisition step S40 for acquiring one or more first likelihood distributions, which are likelihood distributions of the position of the sound source, based on the multiple spectrograms, and a detection step S270 for detecting the direction in which the sound source is located based on the one or more first likelihood distributions.

[0102] The sound source direction detection method of this embodiment further includes a second distribution acquisition step S250 for acquiring one or more second likelihood distributions, which are likelihood distributions of the position of the sound source, based on multiple sound waveforms, and in a detection step S270, the direction in which the sound source is located is detected based on both the one or more first likelihood distributions and the one or more second likelihood distributions.

[0103] The effect of the sound source direction detection method according to this embodiment will be described with reference to FIG. 25. FIG. 25 is a graph showing the accuracy of the detection result by the sound source direction detection method according to this embodiment. The horizontal axis shows the distance from the sound source direction detection device 200 (or the sound source direction detection device 100) to the sound source, and the vertical axis shows the average accuracy of the detected direction. Note that FIG. 25 shows the accuracy in four ranges of distance: a range of more than 0 m and less than 50 m, a range of more than 50 m and less than 100 m, a range of more than 100 m and less than 150 m, and a range of more than 150 m and less than 200 m. FIG. 25 also shows the accuracy of the detection result by the sound source direction detection method according to the first embodiment, and the accuracy when detection is performed based only on the likelihood distribution based on the voice waveform as shown in FIG. 15 to FIG. 20.

[0104] As shown in Fig. 25, the sound source direction detection method according to the first embodiment can improve accuracy compared to the case where only the voice waveform is used, particularly in a distance range of 150 m or less. In this embodiment, the accuracy can be improved in a distance range of 200 m or less compared to both the case where only the voice waveform is used and the case where the sound source direction detection method according to the first embodiment is used.

[0105] In the sound source direction detecting method according to the present embodiment, in the second distribution acquiring step, one or more second likelihood distributions are acquired based on the cross-correlation of a plurality of voice waveforms.

[0106] This makes it easy to calculate the phase difference between the two audio waveforms.

[0107] In the sound source direction detection method according to the present embodiment, the multiple microphones include three or more microphones, the one or more first likelihood distributions include two or more first likelihood distributions, and the one or more second likelihood distributions include two or more second likelihood distributions. In the first distribution acquisition step S40, each of the one or more first likelihood distributions is acquired based on two spectrograms among the multiple spectrograms. In the second distribution acquisition step S250, each of the one or more second likelihood distributions is acquired based on two audio waveforms among the multiple audio waveforms. In the detection step S270, an integrated likelihood distribution obtained by integrating the one or more first likelihood distributions is integrated with each of the one or more second likelihood distributions to acquire one or more third likelihood distributions, and the direction in which the sound source is located is detected based on a fourth likelihood distribution acquired by integrating the one or more third likelihood distributions.

[0108] By using the integrated likelihood distribution acquired based on two or more first likelihood distributions in this manner, the direction in which the sound source is located can be detected more accurately.

[0109] In the detection step S270 of the sound source direction detection method according to this embodiment, a posterior probability distribution is obtained by performing time series processing of the probability distribution related to one or more first likelihood distributions.

[0110] This makes it possible to suppress momentary deviations in the detection results caused by noise, etc.

[0111] In the detection step S270 of the sound source direction detection method according to the present embodiment, BBF is used in the time series processing.

[0112] This allows likelihoods (probabilities) to be treated additively.

[0113] The sound source direction detecting device 200 according to this embodiment also provides the same effects as those of the above-mentioned sound source direction detecting method.

[0114] (Variations, etc.) Although the sound source direction detection method according to one aspect of the present invention has been described based on the embodiment, the present invention is not limited to the embodiment. As long as it does not deviate from the gist of the present invention, various modifications conceived by a person skilled in the art may be included in the scope of the present invention.

[0115] For example, in each of the above embodiments, the sound source is an ambulance, but the sound source is not limited to an ambulance. For example, the sound source may be a moving object such as an emergency vehicle other than an ambulance, or may be a sound source other than a moving object such as a railroad crossing.

[0116] In each of the above embodiments, in the first distribution acquisition step S40, the first distribution acquisition unit 40 detects the multiple edges by time-differentiating the multiple spectrograms. However, the method of detecting the multiple edges is not limited to time differentiation, and any method capable of detecting the multiple edges may be adopted as the method of detecting the multiple edges.

[0117] Moreover, the time series processing employed in the detection section 270 and detection step S270 according to the second embodiment may also be employed in the detection section 70 and detection step S70 according to the first embodiment.

[0118] The following forms may also be included within the scope of one or more aspects of the present disclosure.

[0119] (1) Some of the components included in the above sound source direction detection device may be a computer system consisting of a microprocessor, ROM, RAM, a hard disk unit, a display unit, a keyboard, a mouse, etc. A computer program is stored in the RAM or hard disk unit. The microprocessor operates according to the computer program to achieve its functions. Here, the computer program is composed of a combination of multiple instruction codes that indicate commands for a computer to achieve a specified function.

[0120] (2) Some of the components included in the sound source direction detection device may be configured from one system LSI (Large Scale Integration). The system LSI is an ultra-multifunctional LSI manufactured by integrating multiple components on one chip, and specifically, is a computer system including a microprocessor, ROM, RAM, etc. A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating in accordance with the computer program.

[0121] (3) Some of the components included in the sound source direction detection device may be composed of an IC card or a standalone module that can be attached to each device. The IC card or the module is a computer system composed of a microprocessor, ROM, RAM, etc. The IC card or the module may include the ultra-multifunctional LSI. The microprocessor operates according to a computer program, causing the IC card or the module to achieve its functions. The IC card or the module may be tamper-resistant.

[0122] (4) Furthermore, some of the components included in the sound source direction detection device may be the computer program or the digital signal recorded on a computer-readable recording medium, such as a flexible disk, a hard disk, a CD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, a BD (Blu-ray (registered trademark) Disc), a semiconductor memory, etc. Also, the components may be the digital signal recorded on such a recording medium.

[0123] In addition, some of the components included in the above-mentioned sound source direction detection device may transmit the computer program or the digital signal via a telecommunications line, a wireless or wired communication line, a network such as the Internet, data broadcasting, etc.

[0124] (5) The present disclosure may be the sound source direction detection method described above. It may also be a computer program for causing a computer to realize the sound source direction detection method, or a digital signal consisting of the computer program. Furthermore, the present disclosure may be realized as a non-transitory computer-readable recording medium, such as a CD-ROM, on which the computer program is recorded.

[0125] (6) The present disclosure may also provide a computer system having a microprocessor and a memory, the memory storing the computer program, and the microprocessor operating in accordance with the computer program.

[0126] (7) The program or the digital signal may also be implemented by another independent computer system by recording it on the recording medium and transferring it, or by transferring the program or the digital signal via the network, etc.

[0127] (8) The above-described embodiments and modifications may be combined with each other.

[0128] (Additional Note) Furthermore, based on the above description, the following techniques are disclosed.

[0129] (Technology 1) A sound source direction detection method for detecting the direction in which a sound source is located, comprising: an audio waveform acquisition step for acquiring multiple audio waveforms related to the sound source using multiple microphones; a spectrogram acquisition step for acquiring multiple spectrograms based on the multiple audio waveforms; a first distribution acquisition step for acquiring one or more first likelihood distributions, which are likelihood distributions of the position of the sound source, based on the multiple spectrograms; and a detection step for detecting the direction in which the sound source is located based on the one or more first likelihood distributions.

[0130] (Technology 2) The sound source direction detection method according to Technology 1 further includes a second distribution acquisition step of acquiring one or more second likelihood distributions, which are likelihood distributions of the position of the sound source, based on the multiple sound waveforms, and in the detection step, detecting the direction in which the sound source is located based on both the one or more first likelihood distributions and the one or more second likelihood distributions.

[0131] (Technology 3) The sound source direction detection method according to Technology 1 or 2, in which, in the first distribution acquisition step, a plurality of edges are detected from the plurality of spectrograms, and the one or more first likelihood distributions are acquired based on the cross-correlation of the plurality of edges.

[0132] (Technology 4) The sound source direction detection method according to Technology 2, wherein in the second distribution acquisition step, the one or more second likelihood distributions are acquired based on cross-correlation of the multiple voice waveforms.

[0133] (Technology 5) A sound source direction detection method according to any one of Technologies 1 to 4, wherein the plurality of microphones include three or more microphones, the one or more first likelihood distributions include two or more first likelihood distributions, in the first distribution acquisition step, each of the one or more first likelihood distributions is acquired based on two spectrograms out of the plurality of spectrograms, and in the detection step, the direction in which the sound source is located is detected based on an integrated likelihood distribution that integrates the one or more first likelihood distributions.

[0134] (Technology 6) The sound source direction detection method according to Technology 2 or 4, wherein the multiple microphones include three or more microphones, the one or more first likelihood distributions include two or more first likelihood distributions, and the one or more second likelihood distributions include two or more second likelihood distributions, wherein in the first distribution acquisition step, each of the one or more first likelihood distributions is acquired based on two spectrograms among the multiple spectrograms, and in the second distribution acquisition step, each of the one or more second likelihood distributions is acquired based on two audio waveforms among the multiple audio waveforms, and in the detection step, an integrated likelihood distribution obtained by integrating the one or more first likelihood distributions is integrated with each of the one or more second likelihood distributions to acquire one or more third likelihood distributions, and the direction in which the sound source is located is detected based on a fourth likelihood distribution acquired by integrating the one or more third likelihood distributions.

[0135] (Technology 7) The sound source direction detecting method according to any one of Techniques 1 to 6, wherein the plurality of microphones includes four microphones arranged at positions corresponding to the four vertices of a rectangle.

[0136] (Technology 8) The sound source direction detection method according to any one of Techniques 1 to 7, wherein in the detection step, a posterior probability distribution is obtained by performing time series processing of the probability distribution related to the one or more first likelihood distributions.

[0137] (Technology 9) The sound source direction detection method according to Technology 8, in which BBF is used in the time series processing.

[0138] (Technology 10) A program for causing a computer to execute the sound source direction detection method according to any one of techniques 1 to 9.

[0139] (Technology 11) A sound source direction detection device that detects the direction in which a sound source is located, comprising: an audio waveform acquisition unit that acquires multiple audio waveforms related to the sound source using multiple microphones; a spectrogram acquisition unit that acquires multiple spectrograms based on the multiple audio waveforms; a first distribution acquisition unit that acquires one or more first likelihood distributions, which are likelihood distributions of the position of the sound source, based on the multiple spectrograms; and a detection unit that detects the direction in which the sound source is located based on the one or more first likelihood distributions.

[0140] (Technology 12) An autonomous moving body comprising the sound source direction detection device according to Technology 11 and the plurality of microphones. [Industrial Applicability]

[0141] The sound source direction detection method according to one aspect of the present invention can be applied to vehicles for autonomous driving, for example. [Explanation of symbols]

[0142] 11, 12, 13, 14 Microphones 20 Voice waveform acquisition unit 30 Spectrogram acquisition section 40 First distribution acquisition part 70, 270 Detector 80 Driving control unit 90 Memory section 100, 200 Sound source direction detection device 250 Second distribution acquisition part R0 rectangle SS Ambulance V10, V20 autonomous mobile

Claims

1. A sound source direction detection method for detecting the direction in which a sound source is located, comprising the steps of: a sound waveform acquiring step of acquiring a plurality of sound waveforms related to the sound source using a plurality of microphones; a spectrogram acquisition step of acquiring a plurality of spectrograms based on the plurality of speech waveforms; a first distribution acquisition step of acquiring one or more first likelihood distributions, which are likelihood distributions of the positions of the sound sources, based on the plurality of spectrograms; and detecting a direction in which the sound source is located based on the one or more first likelihood distributions. Sound source direction detection method.

2. The sound source direction detection method includes: The method further includes a second distribution acquisition step of acquiring one or more second likelihood distributions, which are likelihood distributions of the positions of the sound sources, based on the plurality of sound waveforms; In the detection step, a direction in which the sound source is located is detected based on both the one or more first likelihood distributions and the one or more second likelihood distributions. The sound source direction detection method according to claim 1 .

3. In the first distribution acquisition step, a plurality of edges are detected from the plurality of spectrograms, and the one or more first likelihood distributions are acquired based on a cross-correlation of the plurality of edges. The sound source direction detection method according to claim 1 or 2.

4. In the second distribution acquisition step, the one or more second likelihood distributions are acquired based on a cross-correlation between the plurality of speech waveforms. The sound source direction detection method according to claim 2.

5. the plurality of microphones includes three or more microphones; the one or more first likelihood distributions include two or more first likelihood distributions; In the first distribution acquisition step, each of the one or more first likelihood distributions is acquired based on two spectrograms of the plurality of spectrograms; In the detection step, a direction in which the sound source is located is detected based on an integrated likelihood distribution obtained by integrating the one or more first likelihood distributions. The sound source direction detection method according to claim 1 or 2.

6. the plurality of microphones includes three or more microphones; the one or more first likelihood distributions include two or more first likelihood distributions; the one or more second likelihood distributions include two or more second likelihood distributions; In the first distribution acquisition step, each of the one or more first likelihood distributions is acquired based on two spectrograms of the plurality of spectrograms; In the second distribution acquisition step, each of the one or more second likelihood distributions is acquired based on two speech waveforms among the plurality of speech waveforms; In the detection step, an integrated likelihood distribution obtained by integrating the one or more first likelihood distributions is integrated with each of the one or more second likelihood distributions to obtain one or more third likelihood distributions, and a direction in which the sound source is located is detected based on a fourth likelihood distribution obtained by integrating the one or more third likelihood distributions. The sound source direction detection method according to claim 2 or 4.

7. The plurality of microphones includes four microphones arranged at positions corresponding to the four vertices of a rectangle. The sound source direction detection method according to claim 1 or 2.

8. In the detection step, a posterior probability distribution is obtained by performing time series processing of a probability distribution related to the one or more first likelihood distributions. The sound source direction detection method according to claim 1 or 2.

9. In the time series processing, a BBF (Binary Bayes Filter) is used. The sound source direction detection method according to claim 8.

10. A method for causing a computer to execute the sound source direction detection method according to any one of claims 1 to 4. program.

11. A sound source direction detection device that detects the direction in which a sound source is located, a voice waveform acquisition unit that acquires a plurality of voice waveforms related to the sound source using a plurality of microphones; a spectrogram acquisition unit that acquires a plurality of spectrograms based on the plurality of voice waveforms; a first distribution acquisition unit that acquires one or more first likelihood distributions, which are likelihood distributions of the positions of the sound sources, based on the plurality of spectrograms; a detection unit that detects a direction in which the sound source is located based on the one or more first likelihood distributions. Sound source direction detection device.

12. A sound source direction detecting device according to claim 11, The plurality of microphones. Autonomous mobile body.

Images