Sound source location estimation device and sound source location estimation method

Abstract

The sound source location can be easily determined based on the direction of arrival of direct and reflected sound measured by a microphone array. [Solution] The sound source location estimation device 10 includes one or more microphone arrays MA1 to MA4 installed in the monitoring space, a sound wave path calculation unit 23 that calculates multiple sound wave paths from each of the multiple grid points set in the monitoring space to the microphone array based on spatial information of the monitoring space, a sound wave path selection unit 25 that selects a first predetermined number of sound wave paths from the sound wave path calculation unit based on at least one of the sound wave shielding, propagation distance, number of reflections or elevation angle of the direction of arrival to the microphone array in the multiple sound wave paths, and a sound source determination unit 26 that determines whether or not there is a sound source at any of the grid points based on the acoustic power detected by the microphone array in the direction of arrival of the sound wave propagating along the sound wave path selected by the sound wave path selection unit.

Description

Technical Field

[0001] The present invention relates to a sound source position estimation device and a sound source position estimation method.

Background Art

[0002] In Patent Document 1 below, the arrival direction of sound arriving at a sound sensor array is specified, and based on the specified arrival direction of the sound and three-dimensional space map information, the sound wave paths of direct sound and reflected sound arriving from the sound source to the sound sensor array are respectively estimated, and a sound source position estimation device that estimates the intersection position of these sound wave paths as the position of the sound source is described.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When estimating the sound source position using the sound wave paths of reflected sounds that are reflected multiple times in the sound source position estimation device described in Patent Document 1, the number of combinations of straight lines subject to intersection determination significantly increases, leading to a sharp increase in the calculation amount. The present invention has been made in view of the above problems, and an object thereof is to more easily specify the sound source position based on the arrival directions of direct sound and reflected sound measured by a microphone array.

Means for Solving the Problems

[0005] A sound source location estimation device according to one embodiment of the present invention comprises: one or more microphone arrays installed in a monitoring space; a sound wave path calculation unit that calculates multiple sound wave paths from a grid point to a microphone array for each combination of the microphone array and a plurality of grid points predetermined in the monitoring space, based on spatial information of the monitoring space; a sound wave path selection unit that selects a first predetermined number of sound wave paths from the sound wave path calculation unit based on at least one of the presence or absence of sound wave shielding in the plurality of sound wave paths, propagation distance, number of reflections, or elevation angle of the direction of arrival to the microphone array; and a sound source determination unit that determines whether or not there is a sound source at any of the plurality of grid points based on the acoustic power detected by the microphone array in the direction of arrival of the sound wave propagating along the sound wave path selected by the sound wave path selection unit. [Effects of the Invention]

[0006] According to the present invention, the sound wave path from the grid point to the microphone array, where sound waves arrive directly, and the sound wave path where sound waves are reflected along the way before arriving at the microphone array can be calculated in advance. Therefore, by detecting the acoustic power in the direction of arrival of these sound wave paths, it is possible to determine whether or not a sound source exists at the grid point. For this reason, the sound source location can be more easily identified based on the direction of arrival of the direct sound and reflected sound measured by the microphone array. [Brief explanation of the drawing]

[0007] [Figure 1] This is a schematic diagram showing an example of the configuration of the sound source location estimation device according to the embodiment. [Figure 2] This is a block diagram showing an example of the functional configuration of the control unit of the first embodiment. [Figure 3] (a) and (b) are explanatory diagrams of the spatial spectrum representing the intensity of sound waves for each direction of arrival at the microphone array. [Figure 4] This is a schematic diagram of an example of a sound wave path calculated by the sound wave path calculation unit. [Figure 5] (a) and (b) are illustrative diagrams illustrating examples of occluding objects that are set to represent a person detected within the surveillance space. [Figure 6] This figure shows an example of sound wave path information generated for each grid point. [Figure 7] This is an explanatory diagram illustrating an example of the selection of an effective sound wave path by the sound wave path selection unit. [Figure 8] (a) to (c) are illustrative diagrams illustrating an example of updating the list of effective acoustic paths when a moving object is detected within the surveillance space. [Figure 9] This is a flowchart of the sound source location estimation method according to the first embodiment. [Figure 10] This is a block diagram of an example of the functional configuration of the control unit of the second embodiment. [Figure 11] (a) is a schematic diagram of an example of a sound wave path from a grid point to a microphone array, and (b) is a schematic diagram illustrating an example of an estimated sound wave path calculated for the peak direction detected near the direction of arrival of the direct sound wave path. [Figure 12] This is a schematic diagram illustrating an example of an estimated sound wave path calculated for the peak direction detected near the direction of arrival of the reflected sound wave path. [Figure 13] This is an explanatory diagram illustrating an example of the criteria for determining intersections of estimated sound wave paths. [Figure 14] This is a flowchart of the sound source location estimation method according to the second embodiment. [Modes for carrying out the invention]

[0008] Embodiments of the present invention will be described below with reference to the drawings. The embodiments of the present invention described below are illustrative examples of devices and methods for realizing the technical concept of the present invention, and the technical concept of the present invention is not limited to the structure, arrangement, etc., of the components described below. Various modifications can be made to the technical concept of the present invention within the technical scope defined by the claims described in the patent claims.

[0009] (First Embodiment) (composition) FIG. 1 is a schematic diagram showing an example of the configuration of the sound source position estimation device according to the embodiment. The sound source position estimation device 10 estimates the position of the sound source of the sound generated in the monitoring space 1 based on the acoustic signal collected in the monitoring space 1, which is a space (closed space) with, for example, walls, floors, ceilings, etc. In the monitoring space 1, in addition to the walls, floors, and ceilings that define the monitoring space 1, there may be a person 2, furniture 3 other than walls and ceilings, etc.

[0010] For example, the sound source position estimation device 10 may estimate the position of a specific type of sound source (that is, the type of detection target sound to be detected as the sound source may be specified). For example, the sound source position estimation device 10 may detect the position where the dropping sound of the lost item (such as a key) of the person 2 in the monitoring space 1 occurs, or may detect the position where the breaking sound of the window glass of the sound source position estimation device 10 occurs.

[0011] The sound source position estimation device 10 includes a plurality of microphone arrays MA1 to MA4, a camera CM, a control unit 11, and an output unit 12. In the following description, the microphone arrays MA1 to MA4 may be collectively referred to as "microphone array MA". FIG. 1 illustrates the case where four microphone arrays MA1 to MA4 are provided, but the microphone array MA may be two or three, or five or more. Also, the microphone array MA may be one.

[0012] The microphone array MA is installed in the monitoring space 1 to collect sound waves propagating in the monitoring space 1. For example, the microphone array MA may be installed on the ceiling surface of the monitoring space 1. The camera CM generates an image of the inside of the monitoring space 1. Only a single camera CM is shown in FIG. 1, but the sound source position estimation device 10 may include a plurality of cameras CM.

[0013] The control unit 11 is an electronic control unit or a computer that estimates the position of the sound source of the sound generated in the monitoring space 1 based on the output signal (observation signal) of the microphone array MA. For example, it is installed indoors in the same building as the monitoring space 1. However, a cloud-type system in which the functions of the control unit 11 are installed in a remote server connected to the microphone array MA and the output unit 12 via a network may also be used.

[0014] The control unit 11 includes a processor 11a and peripheral components such as a storage device 11b. The processor 11a may be, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or a MCU (Micro Control Unit). The storage device 11b may include a semiconductor storage device, a magnetic storage device, an optical storage device, or the like. The output unit 12 may include a display device such as a liquid crystal display or a CRT (Cathode Ray Tube) display that outputs the estimation result of the sound source position by the control unit 11, a printing device such as a printer, and a communication device.

[0015] FIG. 2 is a block diagram of an example of the functional configuration of the control unit 11 according to the first embodiment. The control unit 11 includes a spatial spectrum calculation unit 20, a spatial information storage unit 21, a moving body detection unit 22, a sound wave path calculation unit 23, a path information storage unit 24, a sound wave path selection unit 25, and a sound source determination unit 26.

[0016] The functions of the spatial spectrum calculation unit 20, the moving body detection unit 22, the sound wave path calculation unit 23, the sound wave path selection unit 25, and the sound source determination unit 26 in FIG. 2, and the peak direction detection unit 27, the sound wave path estimation unit 28, and the sound source position determination unit 29 in FIG. 10 are realized, for example, when the processor 11a in FIG. 1 executes a computer program stored in the storage device 11b. Also, the spatial information stored in the spatial information storage unit 21 and the sound wave path information stored in the path information storage unit 24 may be stored in a storage area prepared in the storage device 11b.

[0017] The spatial spectrum calculation unit 20 estimates the direction of arrival of sound waves arriving at each of the multiple microphone arrays MA1 to MA4 based on the array output signals of each of the multiple microphone arrays MA1 to MA4 when each of the multiple microphone arrays MA1 to MA4 collects sound generated in the monitoring space 1. The spatial spectrum calculation unit 20 calculates a spatial spectrum representing the received intensity (sound pressure level or acoustic power) of the sound waves for each direction of arrival at the microphone array MA.

[0018] For example, the spatial spectrum calculation unit 20 may calculate the spatial spectrum of an incoming sound wave in the audible frequency band. For example, the spatial spectrum calculation unit 20 may calculate the spatial spectrum of a sound wave with a frequency range from several hundred Hz to several tens of kHz. Figures 3(a) and 3(b) are explanatory diagrams of the spatial spectra representing the intensity of sound waves for each direction of arrival at the microphone array MA.

[0019] As shown in Figure 3(a), the direction of arrival Di of the incoming sound wave as seen from the microphone array MA is defined by the combination of the azimuth angle θ and the elevation angle φ (θ,φ). The spatial spectrum calculation unit 20 measures the received intensity of the incoming sound wave for each direction of arrival (θ,φ) by processing the array output signal of the microphone array MA using an existing sound source estimation method.

[0020] For example, the spatial spectrum calculation unit 20 may calculate the received intensity of incoming sound waves for each direction of arrival (θ,φ) using beamformer methods such as the delay sum method or the minimum variance method, or subspace methods such as the MUSIC (Multiple Signal Classification) method, as a sound source estimation method. The spatial spectrum calculation unit 20 generates a two-dimensional spatial spectrum image in which the received intensity of incoming sound waves for each direction of arrival (θ,φ) is represented by pixel values ​​at positions where the azimuth angle θ and elevation angle φ are the x and y coordinates, respectively. Figure 3(b) shows an example of a spatial spectrum image. The spatial spectrum image represents the difference in received intensity by at least one of the brightness, saturation, and color of the pixels. In the following description, the spatial spectrum image generated by the spatial spectrum calculation unit 20 will be simply referred to as "spatial spectrum".

[0021] Refer to Figure 2. The spatial information storage unit 21 stores spatial information, which is information about elements that define the space within the monitoring space 1 in three dimensions. The spatial information may include planar information of the planes that form the outlines of structures (walls, ceilings, floors, windows, doors, etc.) that define the inside of the monitoring space 1, and stationary objects (e.g., furniture 3, etc.) placed within the monitoring space 1. The spatial information may be information expressed using modeling methods such as BIM (Building Information Model), which includes geometric information of the surfaces of these structures and objects, and mathematical formula information of the surface shape. For example, the planar information may include the identifier, type (wall, floor, window, etc.), normal direction, and area of ​​the plane (e.g., a set of vertex coordinates of a polygon) for each plane. The spatial information may also include material information and sound wave reflection coefficient (reflectance) information of the planes that form these structures and outlines.

[0022] The moving object detection unit 22 performs image recognition processing on images captured by the camera CM within the surveillance space 1 to detect people 2 and other moving objects present within the surveillance space 1. In the following description, moving objects other than people and people may be collectively referred to as "moving objects." For example, when the moving object detection unit 22 recognizes a person 2 from the image captured by the camera CM, it may recognize the position, height, and posture (e.g., standing, flexed, lying down, etc.) of person 2 through image recognition processing.

[0023] For example, if the moving object detection unit 22 recognizes a moving object other than person 2 from the image captured by camera CM, it may recognize the area of ​​the plane forming the outline of the moving object and its normal direction. Note that the sensor used by the moving object detection unit 22 to detect moving objects in the monitoring space 1 is not limited to camera CM; the moving object detection unit 22 may also detect moving objects present in the monitoring space 1 based on point cloud information detected by, for example, Lidar (Light Detection and Ranging).

[0024] The sound wave path calculation unit 23 calculates multiple sound wave paths from a grid point to each of the microphone arrays MA1 to MA4 for each combination of a plurality of grid points pre-set in the monitoring space 1 and a plurality of microphone arrays MA1 to MA4, based on the static spatial information stored in the spatial information storage unit 21. For example, the sound wave path calculation unit 23 may calculate sound wave paths based on the spatial information stored in the spatial information storage unit 21 and the dynamic detection results of a moving object by the moving object detection unit 22.

[0025] Figure 4 is a schematic diagram of an example of a sound wave path calculated by the sound wave path calculation unit 23. In Figure 4, the multiple white circular plots drawn at the intersections of the grid indicated by dashed lines represent multiple grid points that have been pre-set within the monitoring space 1. In Figure 4, the sound wave path drawn with a dashed line represents the sound wave path from one of the grid points, G1, to the microphone arrays MA1 to MA4. The sound wave path calculation unit 23 calculates the sound wave paths to the microphone arrays MA1 to MA4 for the other grid points besides grid point G1 in the same manner as described later.

[0026] In the following explanation, the sound wave path from grid point G1 to microphone array MA may be referred to as the "sound wave path of grid point G1". Additionally, the sound wave paths from any of the grid points to microphone arrays MA1 through MA4 may be referred to as the "sound wave path of microphone array MA1", the "sound wave path of microphone array MA2", the "sound wave path of microphone array MA3", and the "sound wave path of microphone array MA4", respectively.

[0027] In Figure 4, the dashed lines p1d, p2d, p3d, and p4d represent the sound wave paths of direct sound arriving directly from grid point G1 to microphone arrays MA1 to MA4, respectively. Furthermore, the sound wave path calculation unit 23 identifies the positions of reflective surfaces on the surfaces of structures and objects present in the monitoring space 1 where sound waves are reflected, based on static spatial information stored in the spatial information storage unit 21 and the dynamic detection results of moving objects by the moving object detection unit 22, and calculates the sound wave path of the reflected sound reflected by the reflective surfaces on the way from grid point G1 to microphone array MA1 to MA4.

[0028] In Figure 4, the dashed lines p1rA, p2rA, p3rA, and p4rA represent the sound wave paths of the reflected sound from the sound wave emitted from grid point G1, after it is reflected by wall A, to the microphone arrays MA1 to MA4. For example, the sound wave path calculation unit 23 may read information about the normal direction and region of wall A from the spatial information storage unit 21, and calculate the sound wave path of the reflected sound from the sound wave emitted from grid point G1, based on the coefficients of the plane equation of wall A and the region of the surface obtained from these. In Figure 4, wall A is used as an example of a sound reflection surface from grid point G1. However, the sound wave path calculation unit 23 similarly calculates the sound wave path of reflected sound for reflection surfaces other than wall A within the monitoring space 1, based on static spatial information stored in the spatial information storage unit 21 and the dynamic detection results of moving objects by the moving object detection unit 22.

[0029] This allows the sound wave path calculation unit 23 to pre-calculate multiple sound wave paths from a grid point to a microphone array MA, for each combination of multiple grid points and multiple microphone arrays MA1 to MA4, including the paths through which direct sound and reflected sound propagate. Therefore, when the sound wave path calculation unit 23 calculates sound wave paths based on spatial information stored in the spatial information storage unit 21, and the object detection result by the moving object detection unit 22 is used to determine whether or not the detected object is obstructing the sound wave path, as described later, the sound wave path calculation unit 23 does not need to perform sound wave path calculation processing in real time.

[0030] Furthermore, the sound wave path calculation unit 23 determines whether there is a line of sight in each of these sound wave paths p1d~p4d and p1rA~p4rA (i.e., whether the sound wave path is blocked by an obstructing object) based on the static spatial information stored in the spatial information storage unit 21 and the dynamic detection results of moving objects by the moving object detection unit 22. For example, in Figure 4, the direct sound wave path p3d from grid point G1 to microphone array MA3 is blocked by the shielding object Obt, and the sound wave path calculation unit 23 determines that there is no line of sight (NLOS) for the sound wave path p3d.

[0031] Furthermore, if the moving object detection unit 22 detects a person 2 present in the monitoring space 1 as an obstruction to the sound wave path, strictly determining the presence or absence of obstruction based on the shape of person 2 would increase the computational load. Therefore, the sound wave path calculation unit 23 may place an object with a simpler shape as an obstruction that mimics the person 2 detected in the monitoring space 1.

[0032] Figure 5(a) shows a first example of a shielding object set as a person 2 detected in the monitoring space 1. For example, the sound wave path calculation unit 23 may place a shielding object where a rectangular first plate with width w and height h and a rectangular second plate with depth d and height h intersect at the horizontal coordinate Pd where person 2 is detected. The first plate and the second plate intersect at the center of the width direction and the center of the depth direction, respectively, such that the angles between the sides along the width direction and the sides along the depth direction are right angles.

[0033] For example, the width w and depth d may be set to fixed values ​​(e.g., 0.5 m) according to the standard size of the human body. Alternatively, the height h may be set by multiplying the height by a posture coefficient (e.g., "1" for standing, "0.5" for flexed position, and "0" for lying down) determined according to the posture of the person 2 detected by the moving body detection unit 22. Figure 5(b) shows a second example of an occlusion object set as a person 2 detected within the monitoring space 1. For example, the sound wave path calculation unit 23 may place a rectangular occlusion object with width w, depth d, and height h on the horizontal plane coordinate Pd where person 2 is detected.

[0034] Refer to Figure 2. The sound wave path calculation unit 23 stores the sound wave path information, which is information about the calculated sound wave paths p1d~p4d and p1rA~p4rA, in the path information storage unit 24. The moving object detection unit 22 may detect moving objects within the monitoring space 1 at regular intervals. The sound wave path calculation unit 23 may, each time the detection result of the moving object detection unit 22 is updated, regenerate sound wave path information based on the updated detection result of the moving object and the spatial information of the spatial information storage unit 21, and update the sound wave path information stored in the path information storage unit 24.

[0035] Furthermore, the sound wave path calculation unit 23 does not need to calculate the sound wave path reflected by the moving object detected by the moving object detection unit 22, but only needs to use the moving object detected by the moving object detection unit 22 to determine whether there is a line of sight in the sound wave path. In this case, even if the moving object detected by the moving object detection unit 22 is updated, the sound wave path calculation unit 23 does not need to recalculate the sound wave path itself, but only needs to re-determine whether there is a line of sight in the already calculated sound wave path.

[0036] Figure 6 shows an example of sound wave path information generated for each grid point. Figure 6 illustrates the sound wave path information for grid point G1 in Figure 4, and the sound wave path calculation unit 23 creates similar sound wave path information for other grid points other than grid point G1 and stores it in the path information storage unit 24.

[0037] The sound wave path information may include, for each combination of grid point and microphone array, the propagation distance of the sound wave path, the direction of arrival of the sound wave propagating through the sound wave path to the microphone array, the number of reflections of the sound wave in the sound wave path, and information on whether or not there is a line of sight in the sound wave path, for each sound wave path including the sound wave paths of the direct sound and reflected sound from the grid point to the microphone array. The sound wave path information may include information that identifies the location of the sound wave path. For example, the information that identifies the location of the sound wave path may be the coordinates of the start and end points of the line segments that form the sound wave path.

[0038] Alternatively, the sound wave path information relating to the sound wave path of reflected sound may include information that identifies the reflective surface to which the sound wave propagating along the sound wave path reflects. The information that identifies the reflective surface may include an identifier of the planar information stored in the spatial information storage unit 21 as information about the reflective surface, and may also include information about the normal direction and region of the reflective surface itself, as well as information about the reflection coefficient. For example, the sound wave path calculation unit 23 may obtain information that identifies the reflective surface from the spatial information stored in the spatial information storage unit 21.

[0039] For example, Figure 6 shows the sound wave path information for the combination of grid point G1 and microphone array MA1, specifically the direct sound wave path p1d and the reflected sound wave path p1rA from grid point G1 to microphone array MA1. Similarly, for the combination of grid point G1 and microphone array MA2, it shows the sound wave path information for the direct sound wave path p2d and the reflected sound wave path p2rA from grid point G1 to microphone array MA2.

[0040] Refer to Figure 2. The sound wave path selection unit 25 reads sound wave path information for multiple sound wave paths generated by the sound wave path calculation unit 23 for each combination of multiple grid points and multiple microphone array MAs from the path information storage unit 24. The sound wave path selection unit 25 extracts only the sound wave paths with a line of sight (i.e., sound wave paths without NLOS) from the sound wave paths read from the path information storage unit 24 and groups them by grid point.

[0041] Figure 7 shows a table of sound wave paths extracted from the sound wave paths at grid point G1 that have a line of sight (i.e., a list of sound wave paths with a line of sight grouped with respect to grid point G1). For example, the list in Figure 7 includes the direct sound wave paths p4d, p2d, and p1d from grid point G1 to microphone arrays MA4, MA2, and MA1, respectively, as well as the reflected sound wave paths p4rA, p3rA, p2rA, etc., originating from grid point G1, reflecting off wall A, and reaching microphone arrays MA4, MA3, MA2, etc. However, the direct sound wave path p3d from grid point G1 to microphone array MA3 is excluded from the list because it has no line of sight.

[0042] The sound wave path selection unit 25 prioritizes the sound wave paths, which are grouped by grid point, for each grid point. In the example in Figure 7, the sound wave paths p4d, p2d, p1d, p4rA, p3rA, and p2rA are assigned priorities 1 to 6, respectively. For example, the sound wave path selection unit 25 may prioritize sound wave paths based on at least one of the number of reflections or the propagation distance. For example, the sound wave path selection unit 25 may prioritize sound wave paths with fewer reflections over sound wave paths with more reflections (for example, prioritizing paths with fewer reflections), or sound wave paths with shorter propagation distances over sound wave paths with longer propagation distances (for example, prioritizing paths with shorter propagation distances).

[0043] For example, the sound wave path selection unit 25 may prioritize sound wave paths in the order of the number of reflections and the propagation distance. Alternatively, the sound wave path selection unit 25 may estimate the amount of attenuation due to reflection in the sound wave path based on the reflection coefficient information included in the spatial information. The sound wave path selection unit 25 may estimate the amount of attenuation in the sound wave path by adding the amount of attenuation due to reflection and the amount of attenuation based on the propagation distance, and determine a priority based on the estimated amount of attenuation. For example, a sound wave path with less attenuation may be prioritized over a sound wave path with greater attenuation.

[0044] Furthermore, when determining priority based on propagation distance, instead of determining priority based on the propagation distance itself, priority may be determined based on a score corresponding to the propagation distance. For example, since the estimation accuracy of the received intensity (sound pressure level or acoustic power) directly below the microphone array MA may deteriorate, a penalty may be added when determining priority for sound wave paths with a propagation distance less than a threshold, and a score may be set accordingly.

[0045] For example, for sound wave paths with a propagation distance less than a threshold, a fixed value (e.g., 20m) may be added to the propagation distance to determine the score. For sound wave paths with a propagation distance greater than or equal to the threshold, the propagation distance itself may be used as the score, and priority may be determined so that sound wave paths with smaller scores are given priority over sound wave paths with larger scores.

[0046] The sound wave path selection unit 25 selects a list consisting of the top-priority first predetermined number M (M=5 in the example of Figure 7) sound wave paths from among the prioritized sound wave paths as the effective sound wave path list Lvsp. In the example of Figure 7, sound wave paths p4d, p2d, p1d, p4rA, and p3rA are included in the effective sound wave path list Lvsp. The sound wave path selection unit 25 similarly selects an effective sound wave path list Lvsp for grid points other than grid point G1. In other words, the sound wave path selection unit 25 selects an effective sound wave path list Lvsp for each of the multiple grid points.

[0047] As described above, when the detection results of the moving object detection unit 22 are updated at regular intervals, the sound wave path information stored in the path information storage unit 24 is also updated. Each time the sound wave path information is updated, the sound wave path selection unit 25 re-selects the valid sound wave path list Lvsp based on the updated sound wave path information.

[0048] Refer to Figures 8(a) to 8(c). Figure 8(a) is a schematic diagram showing the presence of person 2 that obstructs the sound wave path p4d through which sound directly propagates from grid point G1 to microphone array MA4. Figure 8(b) is a diagram showing an example of the effective sound wave path list Lvsp selected before the detection of person 2. Figure 8(c) is a diagram showing an example of the effective sound wave path list Lvsp selected after the detection of person 2.

[0049] Because Person 2 obstructs the line of sight of sound wave path p4d, sound wave path p4d, which was included in the effective sound wave path list Lvsp in Figure 8(b), is excluded from the effective sound wave path list Lvsp in Figure 8(c). In addition, the rank of sound wave path p4rA, which had the lowest priority in the effective sound wave path list Lvsp in Figure 8(b), is moved up in the effective sound wave path list Lvsp in Figure 8(c), so that sound wave path p3rA, which has the next priority after sound wave path p4rA, is added to the effective sound wave path list Lvsp in Figure 8(c).

[0050] Refer to Figure 2. The sound source determination unit 26 obtains information on the direction of arrival of sound waves that propagate along sound wave paths p4d, p2d, p1d, p4rA, and p3rA included in the effective sound wave path list Lvsp and arrive at microphone arrays MA4, MA2, MA1, MA4, and MA3, respectively, from the sound wave path information.

[0051] The sound source determination unit 26 obtains the received intensity (sound pressure level or acoustic power) Pw4d, Pw2d, Pw1d, Pw4r, and Pw3r in the direction of arrival of the sound waves that are propagating along these sound wave paths p4d, p2d, p1d, p4rA, and p3rA from the spatial spectrum calculated by the spatial spectrum calculation unit 20. In the following explanation, the received signal strength in the direction of arrival of a sound wave propagating through each sound wave path may simply be referred to as the "received signal strength along the sound wave path."

[0052] The sound source determination unit 26 determines that there is a sound source at grid point G1 if, among these first predetermined number M received intensities Pw4d, Pw2d, Pw1d, Pw4r, and Pw3r, a second predetermined number N (for example, N=3) or more received intensities exceed a predetermined threshold. For example, the sound source determination unit 26 may estimate the position of grid point G1 as the sound source position.

[0053] If the number of received signals exceeding a predetermined threshold out of a first predetermined number of M received signals is less than a second predetermined number of N, it is determined that there is no sound source at grid point G1. The sound source determination unit 26 similarly determines the presence or absence of a sound source for other grid points besides grid point G1. In other words, the sound source determination unit 26 determines the presence or absence of a sound source for each of the multiple grid points.

[0054] For example, if the second predetermined number N=3, the sound source determination unit 26 may determine that there is a sound source at grid point G1 if four received intensities Pw4d, Pw2d, Pw4r, and Pw3r exceed the threshold, as shown in Figure 7. On the other hand, if only two received intensities Pw4d and Pw4r exceed the threshold, the sound source determination unit 26 may determine that there is no sound source at grid point G1. The sound source detection unit 26 outputs the result of determining whether or not a sound source is present from the output unit 12. For example, the sound source detection unit 26 may output the position of the grid point where it determined that a sound source is present as the sound source position from the output unit 12.

[0055] Furthermore, when the sound source determination unit 26 determines whether the received intensity of the sound wave path exceeds the threshold, it may change the threshold according to the magnitude of the ambient noise in the monitoring space 1. For example, a larger threshold may be set when the ambient noise in the monitoring space 1 is louder than when it is quieter. For example, a larger threshold may be set the louder the ambient noise. Furthermore, the sound source determination unit 26 may change the threshold according to the magnitude of the sound to be detected as a sound source. For example, a larger threshold may be set for louder sounds (e.g., the sound of a window breaking) compared to quieter sounds (e.g., the sound of something falling). For example, the louder the sound to be detected, the larger the threshold may be set.

[0056] Furthermore, for example, when the sound source determination unit 26 determines whether the received intensity of the reflected sound wave path exceeds a threshold, it may change the threshold according to the reflection coefficient of the reflective surface on the sound wave path. For example, a larger threshold may be set when the reflection coefficient is large compared to when it is small. For example, a larger threshold may be set the larger the reflection coefficient.

[0057] Alternatively, the sound source determination unit 26 may, for example, normalize the received intensity according to the propagation distance of the sound wave path and the attenuation due to reflections along the sound wave path when determining whether the received intensity of the sound wave path exceeds a threshold, and then determine whether the normalized received intensity exceeds the threshold. For example, if the sound source is estimated to be a point source, the received intensity will attenuate by 6 dB when the propagation distance doubles. Therefore, based on the propagation distance information of each sound wave path, the received intensity obtained from the spatial spectrum may be corrected to the received intensity when the propagation distance is a predetermined value (e.g., 1 m), and then it may be determined whether the corrected received intensity exceeds the threshold.

[0058] (operation) Figure 9 is a flowchart of the sound source location estimation method according to the first embodiment. The processing shown in the flowchart in Figure 9 is performed for each of a plurality of grid points that are pre-set within the monitoring space 1. In the following description, the grid points that are the target of the processing shown in the flowchart in Figure 9 will be referred to as "target grid points". In step S1, the sound wave path calculation unit 23 calculates multiple sound wave paths from the target grid point to the microphone array MA1 to MA4 for each of the multiple microphone arrays MA1 to MA4, based at least on the spatial information of the monitoring space 1.

[0059] In step S2, the sound wave path selection unit 25 selects a first predetermined number M of sound wave paths from among the sound wave paths calculated by the sound wave path calculation unit 23, based on at least one of the presence or absence of sound wave shielding in a plurality of sound wave paths, propagation distance, or number of reflections. In step S3, the spatial spectrum calculation unit 20 calculates spatial spectra representing the acoustic power of sound waves for each direction of arrival at the microphone arrays MA1 to MA4.

[0060] In step S4, the sound source determination unit 26 obtains the acoustic power in the direction of arrival of sound waves propagating along the sound wave path selected by the sound wave path selection unit 25 from the spatial spectrum calculated by the spatial spectrum calculation unit 20. The sound source determination unit 26 determines whether the number of sound wave paths in which the acoustic power exceeds the threshold is greater than or equal to a second predetermined number N. If the number of sound wave paths in which the acoustic power exceeds the threshold is greater than or equal to a second predetermined number N (step S4: Y), the process proceeds to step S6. If the number of sound wave paths in which the acoustic power exceeds the threshold is not greater than or equal to a second predetermined number N (step S4: N), the process proceeds to step S5.

[0061] In step S5, the sound source detection unit 26 determines that there is no sound source at the target grid point. The process then terminates. In step S6, the sound source detection unit 26 determines that there is a sound source at the target grid point. The process then ends.

[0062] (Second Embodiment) In the sound source location estimation device 10 of the second embodiment, when the sound source determination unit 26 determines that there is a sound source at any of the grid points, the spatial spectrum calculation unit 20 uses the peak of the received intensity appearing in the spatial spectrum to confirm whether there is a sound source at or near the grid point where the sound source was determined to be located.

[0063] Figure 10 is a block diagram of an example of the functional configuration of the control unit 11 of the second embodiment. In addition to the components of the control unit 11 of the first embodiment shown in Figure 2, the control unit 11 of the second embodiment includes a peak direction detection unit 27, a sound wave path estimation unit 28, and a sound source position determination unit 29. The functions of the components which are the same as those of the control unit 11 of the first embodiment will not be described.

[0064] The peak direction detection unit 27 determines whether there is a direction of arrival of a sound wave where the received intensity peaks on the spatial spectrum calculated by the spatial spectrum calculation unit 20, near the direction of arrival of a sound wave that propagates through multiple sound wave paths included in the effective sound wave path list Lvsp from a grid point where the sound source has been determined to be present by the sound source determination unit 26 to reach the microphone array MA. In the following description, the direction of arrival of a sound wave that propagates through a sound wave path to reach the microphone array MA may be simply referred to as the "direction of arrival of the sound wave path," and the direction of arrival of a sound wave where the received intensity peaks on the spatial spectrum may be referred to as the "peak direction."

[0065] In other words, the peak direction detection unit 27 detects the peak direction of the received intensity near the direction of arrival of multiple sound wave paths included in the effective sound wave path list Lvsp. Figure 11(a) is a schematic diagram of an example of a sound wave path from the grid point G1, where the sound source determination unit 26 has determined that a sound source is present, to the microphone array MA. To simplify the explanation, the following describes an example where the sound source location estimation device 10 is equipped with three microphone arrays MA1 to MA3.

[0066] In Figure 11(a), the sound wave paths of direct sound arriving directly from grid point G1 to microphone arrays MA1 to MA3 are denoted as p1d, p2d, and p3d, respectively, and the directions of arrival of sound wave paths p1d, p2d, and p3d are denoted as (θ1, φ1), (θ2, φ2), and (θ3, φ3), respectively.

[0067] The peak direction detection unit 27 determines whether or not a peak direction exists in the vicinity of each of the incoming directions (θ1, φ1), (θ2, φ2), and (θ3, φ3). For example, the peak direction detection unit 27 may determine that a peak direction exists near the direction of arrival if the angle between the direction of arrival and the peak direction of the sound wave path is less than a threshold, and may determine that a peak direction does not exist near the direction of arrival if the angle between the direction of arrival and the peak direction of the sound wave path is greater than or equal to the threshold.

[0068] For example, the peak direction detection unit 27 may determine that a peak direction exists near the direction of arrival if the difference Δθ between the azimuth angle of the direction of arrival of the sound wave path and the azimuth angle of the peak direction is less than a threshold, and the difference Δφ between the elevation angle of the direction of arrival of the sound wave path and the elevation angle of the peak direction is less than a threshold. Conversely, if at least one of the difference Δθ or Δφ is greater than or equal to the threshold, it may determine that a peak direction does not exist near the direction of arrival.

[0069] For example, the peak direction detection unit 27 may determine whether or not there is a peak direction near the direction of arrival of all or some of the sound wave paths included in the effective sound wave path list Lvsp. Alternatively, for example, the peak direction detection unit 27 may determine whether or not there is a peak direction near the direction of arrival of only the sound wave paths included in the effective sound wave path list Lvsp whose received intensity has been determined by the sound source determination unit 26 to exceed a predetermined threshold.

[0070] Refer to Figure 10. When a peak direction exists near the arrival direction of multiple sound wave paths included in the effective sound wave path list Lvsp, the sound wave path estimation unit 28 estimates the sound wave path through which the sound wave arriving from the detected peak direction to the microphone array propagates. In the following description, the sound wave path estimated by the sound wave path estimation unit 28 will be referred to as the "estimated sound wave path".

[0071] For example, the sound wave path estimation unit 28 calculates the estimated sound wave path as an extension line extending from the position of the microphone array in the direction of the peak detected near the direction of arrival of the direct sound wave path. Figure 11(b) is a schematic diagram illustrating an example of an estimated sound wave path calculated for the peak direction detected near the direction of arrival of the direct sound wave path.

[0072] The peak direction detection unit 27 detects the peak directions (θ1^, φ1^), (θ2^, φ2^), and (θ3^, φ3^) near the arrival directions (θ1, φ1), (θ2, φ2), and (θ3, φ3) of the sound wave paths p1d to p3d. The sound wave path estimation unit 28 calculates estimated sound wave paths EL1, EL2, and EL3, which extend in the peak directions (θ1^, φ1^), (θ2^, φ2^), and (θ3^, φ3^), starting from positions P1 to P3 of the microphone array MA1 to MA3 where sound waves propagating along sound wave paths p1d to p3d arrive.

[0073] Furthermore, the sound wave path estimation unit 28 calculates the reflection position of the sound wave arriving from the peak direction for the peak direction detected near the direction of arrival of the reflected sound wave path, and calculates the estimated sound wave path as an extension line extending in the direction in which the peak direction is reversed from the reflection position. That is, it traces back the reflection path of the sound wave arriving from the peak direction and calculates the estimated sound wave path as an extension line extending in the direction in which the peak direction is reversed from the first reflection position.

[0074] Figure 12 is a schematic diagram illustrating an example of an estimated sound wave path calculated for the peak direction detected near the direction of arrival of the reflected sound wave path. The sound wave path from grid point G1 to microphone array MA3 is blocked by an obstruction, and the effective sound wave path list Lvsp includes the sound wave path p3rA of the reflected sound that originates from grid point G1, reflects off wall A, and reaches microphone array MA3.

[0075] In this case, the peak direction detection unit 27 detects the peak direction (θ3^, φ3^) near the sound wave path p3rA of the reflected sound. The sound wave path estimation unit 28 calculates the reflection position Pr at which sound waves arriving at the microphone array MA3 from the peak direction (θ3^, φ3^) are reflected by wall A. For example, the sound wave path estimation unit 28 may read information that identifies the reflective surface of wall A to which sound waves are reflected on sound wave path p3rA from the sound wave path information of sound wave path p3rA (for example, an identifier of the planar information stored in the spatial information storage unit 21, or the information itself of the normal direction and region of the reflective surface), and calculate the reflection position Pr based on the coefficients of the planar equation of wall A and the region of the surface obtained from these, as well as the position and peak direction (θ3^, φ3^) of the microphone array MA3.

[0076] The sound wave path estimation unit 28 calculates the direction (θ3^', φ3^') obtained by reversing the peak direction (θ3^, φ3^) at the reflection position Pr. Here, "the direction obtained by reversing the peak direction at the reflection position Pr" is the direction that is symmetric to the peak direction with respect to a straight line extending in the direction normal to the reflecting surface through the reflection position Pr. The sound wave path estimation unit 28 calculates the estimated sound wave path EL3 as an extension line extending in the direction (θ3^', φ3^') obtained by reversing the peak direction (θ3^, φ3^) with the reflection position Pr as the starting point P3.

[0077] Furthermore, if the extension line from the position of the microphone array MA3 in the peak direction (θ3^, φ3^) does not intersect with the area of ​​the reflective surface of wall A (for example, if the reflection position Pr is outside the area of ​​the reflective surface of wall A), the sound wave path estimation unit 28 does not calculate the estimated sound wave path EL3 in the peak direction (θ3^, φ3^). In this case, the sound source position determination unit 29, described later, does not determine whether there is an intersection between the estimated sound wave path from which the sound wave arrives from the peak direction (θ3^, φ3^) and the other estimated sound wave paths EL1 and EL3 (it does not determine whether the predetermined intersection determination conditions are met).

[0078] Furthermore, if the sound wave path p3rA is a sound wave path involving multiple reflections, the sound wave path estimation unit 28 may obtain information from the sound wave path information of the sound wave path p3rA to identify each of the multiple reflective surfaces to which the sound waves propagating along the sound wave path p3rA reflect.

[0079] The sound wave path estimation unit 28 estimates the reflection surfaces obtained from the sound wave path information of the sound wave path p3rA as multiple reflection surfaces that reflect sound waves arriving at the microphone array MA3 from the peak direction (θ3^, φ3^), and may also estimate the reflection surface to which sound waves propagating along the sound wave path p3rA from grid point G1 first reflect, as the first reflection surface of sound waves arriving at the microphone array MA3 from the peak direction (θ3^, φ3^).

[0080] The sound wave path estimation unit 28 may estimate the propagation path of sound waves arriving at the microphone array MA3 from the peak direction (θ3^, φ3^) by sequentially tracking the reflection position and incident direction of each of these multiple reflective surfaces in the opposite direction to the propagation direction of the sound waves. In this case, if the reflection position at any of these multiple reflective surfaces is outside the region of the reflective surface, the sound wave path estimation unit 28 does not calculate the estimated sound wave path EL3 for the peak direction (θ3^, φ3^).

[0081] The sound wave path estimation unit 28 estimates the reflection position Pr at which the sound wave propagating along the estimated propagation path is reflected by the first reflection surface. The sound wave path estimation unit 28 may use the reflection position Pr as the starting point P3 and calculate the estimated sound wave path EL3 as an extension line extending in the opposite direction to the incident direction in which the sound wave enters the reflection position Pr.

[0082] Refer to Figure 10. If a peak direction exists near the arrival direction of multiple sound wave paths included in the effective sound wave path list Lvsp, the sound source location determination unit 29 determines that the intersection point of the multiple estimated sound wave paths calculated by the sound wave path estimation unit 28 is the location of the sound source, based on whether or not there is an intersection point among the multiple estimated sound wave paths. If there is no intersection point among the multiple estimated sound wave paths, the sound source location determination unit 29 determines that there is no sound source at the grid point that the sound source determination unit 26 has determined to be the sound source location.

[0083] Here, since it is rare for the multiple estimated sound wave paths calculated by the sound wave path estimation unit 28 to intersect exactly, the sound source position determination unit 29 may, for example, determine whether the multiple estimated sound wave paths calculated by the sound wave path estimation unit 28 satisfy a predetermined intersection determination condition, and if the predetermined intersection determination condition is met, calculate an approximate position of the intersection of the multiple estimated sound wave paths and determine that the approximate position is the position of the sound source. If the predetermined intersection determination condition is not met, the sound source position determination unit 29 may determine that there is no sound source at the grid point that the sound source determination unit 26 has determined to be the sound source position.

[0084] Figure 13 is an explanatory diagram illustrating an example of the intersection determination condition for estimated sound wave paths EL1 to EL3. The sound source position determination unit 29 may determine the validity of the intersection determination condition for two estimated sound wave paths or four or more estimated sound wave paths using the same method. The dashed lines EL1 to EL3 indicate multiple estimated sound wave paths calculated by the sound wave path estimation unit 28. The sound source position determination unit 29 calculates the proximity points Q12 and Q13 where estimated sound wave path EL1 is closest to estimated sound wave paths EL2 and EL3, respectively; the proximity points Q23 and Q21 where estimated sound wave path EL2 is closest to estimated sound wave paths EL3 and EL1, respectively; and the proximity points Q31 and Q32 where estimated sound wave path EL3 is closest to estimated sound wave paths EL1 and EL2, respectively.

[0085] The sound source position determination unit 29 calculates the distance between proximity points as the distance between the proximity point where one of the pair of estimated sound wave paths comes closest to the other estimated sound wave path and the proximity point where the other estimated sound wave path comes closest to the first estimated sound wave path, for all combinations of selecting a pair of estimated sound wave paths (i.e., two estimated sound wave paths) from the multiple estimated sound wave paths calculated by the sound wave path estimation unit 28.

[0086] For example, the sound source position determination unit 29, for all combinations (EL1, EL2), (EL2, EL3), and (EL3, EL1) in which a pair of estimated sound waves are extracted from the three estimated sound wave paths EL1 to EL3 calculated by the sound wave path estimation unit 28, determines the proximity point between the proximity point Q12 where one estimated sound wave path EL1 is closest to the other estimated sound wave path EL2 and the proximity point Q21 where the other estimated sound wave path EL2 is closest to the one estimated sound wave path EL1. The distance, the distance between approach points Q23 where one estimated sound wave path EL2 is closest to the other estimated sound wave path EL3 and Q32 where the other estimated sound wave path EL3 is closest to one estimated sound wave path EL2, and the distance between approach points Q31 where one estimated sound wave path EL3 is closest to the other estimated sound wave path EL1 and Q13 where the other estimated sound wave path EL1 is closest to one estimated sound wave path EL3 are calculated.

[0087] Furthermore, the sound source position determination unit 29 calculates the midpoint of the approach point for all combinations of selecting a pair of estimated sound wave paths from the multiple estimated sound wave paths calculated by the sound wave path estimation unit 28. This midpoint is the midpoint of the line segment connecting the approach point where one of the pair of estimated sound wave paths is closest to the other estimated sound wave path, and the approach point where the other estimated sound wave path is closest to the first estimated sound wave path.

[0088] For example, the sound source position determination unit 29, for all combinations (EL1, EL2), (EL2, EL3), and (EL3, EL1) in which a pair of estimated sound waves are extracted from the three estimated sound wave paths EL1 to EL3 calculated by the sound wave path estimation unit 28, determines the midpoint R of the approach points Q12 where one estimated sound wave path EL1 is closest to the other estimated sound wave path EL2, and Q21 where the other estimated sound wave path EL2 is closest to the one estimated sound wave path EL1. Calculate 12, the midpoint R23 of the approach points Q23 where one estimated sound wave path EL2 is closest to the other estimated sound wave path EL3 and Q32 where the other estimated sound wave path EL3 is closest to one estimated sound wave path EL2, and the midpoint R31 of the approach points Q31 where one estimated sound wave path EL3 is closest to the other estimated sound wave path EL1 and Q13 where the other estimated sound wave path EL1 is closest to one estimated sound wave path EL3.

[0089] Furthermore, the sound source position determination unit 29 calculates the centroid position CG of all the midpoints R12, R23, and R31 of the calculated proximity points. The sound source position determination unit 29 may determine that the intersection determination condition is met if all calculated proximity point distances are less than or equal to threshold th1, and the respective distances between all calculated proximity point midpoints R12, R23, and R31 and the centroid position CG are all less than or equal to threshold th2, and may determine the centroid position CG as the sound source position.

[0090] Furthermore, if there is no peak direction near the arrival direction of any of the multiple sound wave paths included in the effective sound wave path list Lvsp (for example, if there is no peak direction near the arrival direction of any of the sound wave paths included in the effective sound wave path list Lvsp, or if there is a peak direction near the arrival direction of only one sound wave path), the sound source position determination unit 29 may determine that there is no sound source at the grid point that the sound source determination unit 26 has determined to be the sound source position.

[0091] For example, the sound source location determination unit 29 may determine whether there is an intersection point among the multiple estimated sound wave paths calculated by the sound wave path estimation unit 28 only if, among the sound wave paths included in the effective sound wave path list Lvsp, the sound source determination unit 26 has determined that the received intensity exceeds a predetermined threshold, and for all of these sound wave paths, there is a peak direction near the direction of arrival of the sound wave path. If there is an intersection point, the sound source location determination unit 29 may determine that the intersection point of the estimated sound wave paths is the location of the sound source. (For example, if a predetermined intersection determination condition is met, the approximate location of the intersection point of the multiple estimated sound wave paths may be determined to be the location of the sound source.)

[0092] For example, in all sound wave paths where the sound source determination unit 26 has determined that the received intensity exceeds a predetermined threshold, the sound wave path estimation unit 28 calculates an estimated sound wave path for the peak direction detected near the direction of arrival of the sound wave path. Only in this case, the sound wave path estimation unit 28 determines whether there is an intersection point among the multiple estimated sound wave paths it has calculated, and if there is an intersection point, it may determine that the intersection point of the estimated sound wave paths is the location of the sound source. (For example, if a predetermined intersection determination condition is met, it may be determined that the approximate location of the intersection point of the multiple estimated sound wave paths is the location of the sound source.)

[0093] On the other hand, if the sound source determination unit 26 determines that the received intensity exceeds a predetermined threshold, and there is no peak direction near the direction of arrival of the sound wave path in any of the sound wave paths, the sound source position determination unit 29 may determine that there is no sound source at the grid point that the sound source determination unit 26 has determined to be the sound source position. Furthermore, if the sound wave path estimation unit 28 does not calculate an estimated sound wave path for any sound wave path in which the sound source determination unit 26 has determined that the received intensity exceeds a predetermined threshold, and the sound wave path estimation unit 28 does not calculate an estimated sound wave path for a peak direction detected near the direction of arrival of the sound wave path, the sound source location determination unit 29 may determine that there is no sound source at the grid point that the sound source determination unit 26 has determined to be the sound source location.

[0094] The sound source location determination unit 29 outputs the result of determining the presence or absence of a sound source from the output unit 12. For example, the sound source location determination unit 29 may output the determined sound source location, such as the centroid position CG in Figure 13, as the sound source location from the output unit 12.

[0095] Figure 14 is a flowchart of the sound source location estimation method of the second embodiment. The processing in steps S11 to S15 is the same as the processing in steps S1 to S5 in Figure 9. If in step S14 the number of sound wave paths in which the acoustic power exceeds the threshold is greater than or equal to a second predetermined number N, the process proceeds to step S16 in step S14:Y).

[0096] In step S16, the peak direction detection unit 27 detects the peak direction of the received intensity near the direction of arrival of the sound wave path from the grid point where the sound source determination unit 26 has determined to have a sound source to the microphone array MA. In step S17, the sound wave path estimation unit 28 calculates the estimated sound wave path, which is the path through which sound waves arriving from the peak direction to the microphone array position propagate.

[0097] In step S18, the sound source position determination unit 29 determines whether the multiple estimated sound wave paths calculated by the sound wave path estimation unit 28 satisfy predetermined intersection determination conditions. If the predetermined intersection determination conditions are not met, the process proceeds to step S15 in step S18:N). If the predetermined intersection determination conditions are met, the process proceeds to step S19 in step S18:Y). In step S19, the sound source location determination unit 29 determines the intersection point of the multiple estimated sound wave paths calculated by the sound wave path estimation unit 28 as the sound source location. The process then ends.

[0098] (modified version) (1) In the above description, the sound wave path selection unit 25 selected a first predetermined number M of sound wave paths based on at least one of the presence or absence of sound wave shielding in a plurality of sound wave paths, propagation distance, or number of reflections. In addition, the sound wave path selection unit 25 may also select a first predetermined number M of sound wave paths based on the elevation angle φ of the direction of arrival of the sound waves to the microphone array MA in these sound wave paths.

[0099] As the elevation angle φ increases, the resolution of the microphone array MA deteriorates. Therefore, for example, the sound wave path selection unit 25 may prioritize sound wave paths with a smaller elevation angle φ over sound wave paths with a larger elevation angle φ (for example, prioritizing the smaller the elevation angle φ), or it may select only sound wave paths where the elevation angle φ is less than a threshold (for example, 75 degrees).

[0100] (2) In areas of the monitoring space 1 where a large number of objects are placed and there is a risk of complicating the reflection path, the sound wave path reflected in this area does not need to be used in the process of determining whether or not there is a sound source. For example, the sound wave path reflected in an area where the density of the number of objects is greater than or equal to a threshold does not need to be used in determining whether or not there is a sound source at a grid point.

[0101] For example, information about objects in areas where many objects are located (for example, areas where the density of objects is above a threshold) or information about walls adjacent to these areas may be deleted from the spatial information stored in the spatial information storage unit 21. As a result, these areas are treated as open spaces where no walls exist, and the sound wave path calculation unit 23 no longer calculates sound wave paths that reflect from these areas and reach the microphone array MA.

[0102] For example, when calculating the path through which sound waves propagate from a grid point, the sound wave calculation unit 23 may determine whether or not the sound wave will be reflected within an area where many objects are located, and may interrupt the path calculation if it determines that the sound wave will be reflected within that area. Furthermore, for example, the sound wave path selection unit 25 does not have to select a sound wave path in which sound waves are reflected in an area where many objects are placed as a sound wave path in the effective sound wave path list Lvsp.

[0103] (3) When there are many objects or people within the monitoring section 1, the line of sight is lost in many sound wave paths, and the sound wave path selection unit 25 may not be able to select a first predetermined number of M sound wave paths from a certain grid point to any of the microphone array MA. In such cases, for example, the sound source determination unit 26 may output that it is not possible to determine whether or not there is a sound source at the grid point.

[0104] For example, if the number M' of non-NLOS sound wave paths (line-of-sight sound wave paths) from the grid point to any microphone array MA is greater than or equal to a second predetermined number N and less than a first predetermined number M, the presence or absence of a sound source is determined using the M' non-NLOS sound wave paths in the same manner as described above. If it is determined that a sound source is present, the sound source determination unit 26 may output a determination result indicating that a sound source is present at the grid point, along with a statement that the reliability of the determination result is low.

[0105] For example, if the number M' of non-NLOS sound wave paths from the grid point to any of the microphone array MA is less than the second predetermined number N, the sound source determination unit 26 may output that it cannot determine whether or not there is a sound source at the grid point. Alternatively, the presence or absence of a sound source may be determined by replacing the first predetermined number M and the second predetermined number N in the above description with the above number M', and if it is determined that there is a sound source, the sound source determination unit 26 may output the determination result that there is a sound source at the grid point, along with a statement that the reliability of the determination result is low.

[0106] (Effects of the embodiment) (1) The sound source location estimation device 10 comprises one or more microphone arrays MA installed in the monitoring space, a sound wave path calculation unit 23 that calculates multiple sound wave paths from a grid point to the microphone array MA for each combination of the microphone array MA and a plurality of grid points set in advance in the monitoring space based on spatial information of the monitoring space, a sound wave path selection unit 25 that selects a first predetermined number of sound wave paths from the sound wave path calculation unit 23 based on at least one of the presence or absence of sound wave shielding in the plurality of sound wave paths, propagation distance, number of reflections, or elevation angle of the direction of arrival to the microphone array, and a sound source determination unit 26 that determines whether or not there is a sound source at any of the plurality of grid points based on the acoustic power detected by the microphone array MA in the direction of arrival of the sound waves propagating along the sound wave path selected by the sound wave path selection unit 25.

[0107] This allows the sound wave path to be calculated in advance. In other words, the sound wave path calculation unit 23 does not need to perform sound wave path calculation processing in real time when estimating the sound source position. The sound source determination unit 26 can easily estimate the sound source position by calculating the acoustic power in the direction of arrival of these sound wave paths.

[0108] (2) The sound wave path selection unit 25 may prioritize a sound wave path with a shorter propagation distance over a sound wave path with a longer propagation distance, a sound wave path with fewer reflections over a sound wave path with many reflections, or a sound wave path with a smaller elevation angle over a sound wave path with a larger elevation angle in the direction of arrival of the sound wave. This makes it possible to estimate the sound source position based on a sound wave path with high accuracy in estimating the direction of arrival of the sound wave.

[0109] (3) The sound source determination unit 26 may determine that there is a sound source at a grid point of the first predetermined number of sound wave paths if the acoustic power detected in the direction of arrival of a sound wave propagating through a second predetermined number or more of the first predetermined number of sound wave paths selected by the sound wave path selection unit 25 exceeds a threshold. This makes it possible to determine whether or not a sound source exists at a grid point. (4) The threshold may be set based on at least one of the ambient noise in the monitoring space, the sound to be detected as a sound source, or the reflection coefficient of an object that reflects sound waves in the monitoring space. This can suppress deterioration of estimation accuracy due to setting an inappropriate threshold.

[0110] (5) The system may also include: a peak direction detection unit 27 that detects the peak direction, which is the direction in which the sound wave arrives when the acoustic power detected by the microphone array MA is at its peak, near the direction of arrival of the sound wave that propagates from the grid point where the sound source has been determined to be present by the sound source determination unit 26 along the sound wave path selected by the sound wave path selection unit 25 to the microphone array MA; a sound wave path estimation unit 28 that estimates the sound wave path through which the sound wave arriving from this peak direction to the position of the microphone array propagates; and a sound source position determination unit 29 that determines whether or not there is an intersection point in the sound wave path estimated by the sound wave path estimation unit 28, and that the position of the intersection point of the sound wave path is the position of the sound source.

[0111] This allows the sound source determination unit 26 to narrow down the candidate sound source locations estimated based on the acoustic power in the direction of arrival of the pre-calculated sound wave path, based on the peak direction of the acoustic power calculated by the microphone array MA. As a result, the accuracy of sound source location estimation can be improved.

[0112] (6) The sound source determination unit 26 may determine that there is a sound source at a grid point of the first predetermined number of sound wave paths if the acoustic power detected in the direction of arrival of a sound wave propagating through a second predetermined number or more of the first predetermined number of sound wave paths selected by the sound wave path selection unit 25 exceeds a threshold. The peak direction detection unit 27 may detect the peak direction in the vicinity of the direction of arrival where the acoustic power exceeds the threshold. This improves the accuracy of estimating the sound source position based on the peak direction.

[0113] (7) The sound source location estimation device 10 may include a moving object detection unit 22 for detecting people or moving objects present in the monitoring space. The sound wave path calculation unit 23 may determine at least one of sound wave shielding or reflection in the sound wave path based on the detection result of people or moving objects by the moving object detection unit 22. This makes it possible to reflect the effects of sound wave shielding and reflection by moving objects present in the monitoring space in the sound source location estimation result, thereby improving the accuracy of sound source location estimation. [Explanation of Symbols]

[0114] 1...Surveillance space, 2...Person, 3...Furniture, 10...Sound source location estimation device, 11...Control unit, 11a...Processor, 11b...Storage device, 12...Output unit, 20...Spatial spectrum calculation unit, 21...Spatial information storage unit, 22...Moving object detection unit, 23...Sound wave path calculation unit, 24...Path information storage unit, 25...Sound wave path selection unit, 26...Sound source determination unit, 27...Peak direction detection unit, 28...Sound wave path estimation unit, 29...Sound source location determination unit, CG...Center of gravity position, CM...Camera, MA1~MA4...Microphone array

Claims

1. One or more microphone arrays installed within the monitoring space, Based on the spatial information of the monitoring space, a sound wave path calculation unit calculates multiple sound wave paths from the grid points to the microphone array for each combination of the microphone array and a plurality of grid points pre-set in the monitoring space. A sound wave path selection unit selects a first predetermined number of sound wave paths from among the sound wave paths calculated by the sound wave path calculation unit, based on at least one of the presence or absence of sound wave shielding in the plurality of sound wave paths, propagation distance, number of reflections, or elevation angle of the direction of arrival to the microphone array. A sound source determination unit determines whether or not there is a sound source at any of the plurality of grid points based on the acoustic power detected by the microphone array in the direction of arrival of sound waves propagating along the sound wave path selected by the sound wave path selection unit, A sound source location estimation device characterized by comprising the following features.

2. The sound source position estimation device according to claim 1, characterized in that the sound wave path selection unit prioritizes a sound wave path with a shorter propagation distance over a sound wave path with a longer propagation distance, a sound wave path with fewer reflections over a sound wave path with many reflections, or a sound wave path with a smaller elevation angle over a sound wave path with a larger elevation angle in the direction of arrival of the sound wave.

3. The sound source determination unit determines that there is a sound source at the grid point of the first predetermined number of sound wave paths when the acoustic power detected in the direction of arrival of a sound wave propagating through a second predetermined number or more of the first predetermined number of sound wave paths selected by the sound wave path selection unit exceeds a threshold, characterized in that the sound source location estimation device according to claim 1.

4. The sound source position estimation device according to claim 3, characterized in that the threshold is set based on at least one of the ambient sound in the monitoring space, the sound to be detected as the sound source, or the reflection coefficient of an object that reflects sound waves in the monitoring space.

5. A peak direction detection unit detects the peak direction, which is the direction in which the sound wave reaches the microphone array, near the direction of arrival of the sound wave that propagates from the grid point where the sound source has been determined to be present by the sound source determination unit along the sound wave path selected by the sound wave path selection unit, to the microphone array. A sound wave path estimation unit estimates the sound wave path through which sound waves arriving from the peak direction to the position of the microphone array propagate. A sound source location determination unit determines, based on whether or not there is an intersection in the sound wave path estimated by the sound wave path estimation unit, that the location of the intersection in the sound wave path is the location of the sound source. The sound source location estimation device according to claim 1, characterized by comprising the following:

6. The sound source determination unit determines that there is a sound source at the grid point of the first predetermined number of sound wave paths when the acoustic power detected in the direction of arrival of a sound wave propagating through a second predetermined number or more of the first predetermined number of sound wave paths selected by the sound wave path selection unit exceeds a threshold, The peak direction detection unit detects the peak direction in the vicinity of the incoming direction where the acoustic power exceeds the threshold. The sound source location estimation device according to feature 5.

7. The system includes a moving object detection unit that detects a person or moving object present within the aforementioned monitoring space. The sound wave path calculation unit determines at least one of sound wave shielding or reflection in the sound wave path based on the detection result of a person or moving object by the moving object detection unit. The sound source location estimation device according to feature 1.

8. The sound source position estimation device according to claim 1, characterized in that the spatial information of the monitoring space does not include information about objects in a region where the density of the number of objects placed in the monitoring space is greater than or equal to a threshold, and information about walls adjacent to such objects, or the sound wave path calculation unit does not calculate the reflection path of sound waves reflected in the region where the density of the number of objects placed in the monitoring space is greater than or equal to a threshold, or the sound wave path selection unit does not select a sound wave path in which sound waves are reflected in the region where the density of the number of objects placed in the monitoring space is greater than or equal to a threshold.

9. For each combination of one or more microphone arrays installed in the monitoring space and a plurality of grid points pre-set in the monitoring space, a plurality of sound wave paths from the grid point to the microphone array are calculated based on the spatial information of the monitoring space. Based on at least one of the above-mentioned plurality of sound wave paths, such as the presence or absence of sound wave shielding, propagation distance, number of reflections, or elevation angle of the direction of arrival to the microphone array, a first predetermined number of sound wave paths are selected from the above-mentioned plurality of sound wave paths. Based on the acoustic power detected by the microphone array in the direction of arrival of sound waves propagating through the selected first predetermined number of sound wave paths, it is determined whether or not there is a sound source at any of the plurality of grid points. A method for estimating the location of a sound source, characterized by the features described above.

Images