Abnormal sound detection method and abnormal sound detection device

Abstract

A controller (20) extracts a subject frequency component corresponding to the frequency characteristics of an abnormal sound from sound data that are sound pressures in the time domain and computes an exponential moving average obtained by weighted averaging of the extracted subject frequency component on the basis of weighting coefficients adjusted according to a time constant. The controller (20) identifies a time at which the sound pressure level becomes not less than a first threshold on the basis of sound pressure level data computed from the exponential moving average and performs frequency analysis for the sound data on the basis of the identified time and the subject frequency. The controller (20) identifies, as an abnormal sound, a frequency component with which the sound pressure level becomes not less than a second threshold on the basis of the result of the frequency analysis and estimates the position of the abnormal sound by utilizing the characteristics of the differences in time taken for the sound to reach a plurality of microphones.

Description

Abnormal Sound Detection Method and Abnormal Sound Detection Device

[0001] The present invention relates to an abnormal sound detection method and an abnormal sound detection device.

[0002] Patent Document 1 discloses an abnormal sound determination device for a vehicle that determines whether specific abnormal sounds are included in vehicle running sounds. The abnormal sound determination device performs frequency analysis on the sound data of the vehicle running sound to obtain a frequency-sound pressure level waveform in a predetermined frequency range. The abnormal sound determination device calculates a line connecting representative values of each frequency for the frequency-sound pressure level waveform as an estimated value of background noise, and sets the estimated value of background noise as a threshold level by offsetting it by a predetermined offset amount on the higher side of the sound pressure level in the frequency-sound pressure level waveform. The abnormal sound determination device calculates the area of the portion exceeding the threshold level in the frequency-sound pressure level waveform as an excess area corresponding to the abnormal sound area, and compares the excess area with a preset determination value to determine the presence or absence of abnormal sounds.

[0003] Japanese Patent No. 6089948

[0004] However, Patent Document 1 only determines the presence or absence of abnormal sounds, and has a problem that the position of the abnormal sounds cannot be known.

[0005] An object of the present invention is to provide an abnormal sound detection method and an abnormal sound detection device capable of estimating the position of abnormal sounds in a target space.

[0006] An abnormal sound detection method according to an aspect of the present invention includes extracting a target frequency component corresponding to the frequency characteristics of an abnormal sound from sound data that is the sound pressure in the time domain, and calculating an exponentially weighted moving average obtained by weighted averaging of the extracted target frequency components based on a weighting coefficient adjusted according to a time constant. Further, the abnormal sound detection method includes specifying the time when the sound pressure level becomes equal to or higher than a first threshold value based on the sound pressure level data calculated from the exponentially weighted moving average, and performing frequency analysis on the sound data based on the specified time and the target frequency. The abnormal sound detection method includes specifying a frequency component whose sound pressure level becomes equal to or higher than a second threshold value as an abnormal sound based on the result of the frequency analysis, and estimating the position of the abnormal sound using the characteristics of the time difference until the sound reaches a plurality of microphones.

[0007] According to one aspect of the present invention, the location of an abnormal sound in a target space can be estimated with high accuracy.

[0008] Figure 1 is a diagram showing the configuration of the abnormal sound detection device according to this embodiment. Figure 2 is a flowchart showing the abnormal sound detection method according to this embodiment. Figure 3 is a diagram showing sound data in the target space. Figure 4 is a diagram showing sound data from which the target frequency components have been extracted. Figure 5 is a diagram showing sound pressure level data and thresholds. Figure 6 is a diagram showing a spectrogram showing the results of frequency analysis. Figure 7 is a diagram showing a spectrogram from which the highest sound pressure levels have been extracted.

[0009] The abnormal noise detection method and abnormal noise detection device according to this embodiment will be described below with reference to the drawings.

[0010] The abnormal noise detection device 1 shown in Figure 1 is a device that detects abnormal noises generated in a target space. In this embodiment, the abnormal noise detection device 1 is applied to a vehicle and detects abnormal noises generated in the vehicle's interior space when the vehicle is in motion. The abnormal noise detection device 1 may detect abnormal noises while the vehicle is actually in motion, or it may detect abnormal noises while the vehicle is simulated to be in motion by vibrating the vehicle's wheels using a vibrator prepared in a laboratory.

[0011] The abnormal noise detection device 1 comprises a microphone array 10, a camera unit 15, a controller 20, a storage device 30, and a display device 35.

[0012] The microphone array 10 is a sound collection device equipped with multiple microphones arranged spatially. For example, the microphone array 10 is configured by distributing multiple microphones on the surface of a spherical housing of a predetermined radius. Each microphone outputs an analog signal corresponding to the sound pressure. The microphone array 10 is located in the center of the vehicle interior, for example, between the driver's seat headrest and the passenger seat headrest. The analog signal output from the microphone array 10 is converted into a digital signal by the AD converter 11 and input to the controller 20.

[0013] The camera unit 15 generates images of the area surrounding the microphone array 10. For example, the camera unit 15 has multiple cameras. The multiple cameras are distributed around the housing of the microphone array 10, and by each camera imaging a different area, the area around the microphone array 10 can be imaged in a 360-degree range. Each camera is equipped with a solid-state image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). The images generated by the camera unit 15 are input to the controller 20.

[0014] The controller 20 performs various processes to detect abnormal noises occurring inside the vehicle cabin. The controller 20 consists of a hardware processor, various types of memory, input / output interfaces, etc. The hardware processor may be, for example, a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit). Various programs are installed on the controller 20, and the controller 20 can perform various processes by executing these programs.

[0015] The storage device 30 stores sound data corresponding to a certain period of time of digital signals acquired from the microphone array 10. The storage device 30 stores multiple sound data corresponding to multiple microphones. In addition, the storage device 30 also stores images acquired from the camera unit 15.

[0016] The display device 35 is a display that shows the results of abnormal noise detection.

[0017] Of the elements constituting the abnormal noise detection device 1, the controller 20, the storage device 30, and the display device 35 can be configured using a personal computer. When detecting abnormal noise, the controller 20, the storage device 30, and the display device 35 may be placed inside the vehicle or outside the vehicle.

[0018] The abnormal noise detection method according to this embodiment will now be described with reference to Figure 2. The process shown in the flowchart of Figure 2 is executed by the controller 20.

[0019] In step S10, the controller 20 acquires an image from the camera unit 15.

[0020] In step S11, the controller 20 turns on the microphone array 10 and records the sound inside the vehicle. The analog signal output from the microphone array 10 is converted into a digital signal by the AD converter 11 and acquired by the controller 20. The controller 20 stores (records) sound data in the storage device 30 based on the digital signal acquired from the microphone array 10 for a certain period of time. The sound data is sound pressure data in the time domain, and as shown in Figure 3, it is a sound pressure waveform with time on the horizontal axis and sound pressure on the vertical axis. Multiple sound data collected by each of the multiple microphones equipped in the microphone array 10 are stored in the storage device 30.

[0021] The processing from step S12 to step S18 is performed on arbitrary sound data extracted from multiple sound data recorded by the microphone array 10. The sound data to be processed may be one sound data or two or more sound data. If two or more sound data are used, the processing from step S12 to step S18 is performed for each of the two or more sound data.

[0022] First, in step S12, the controller 20 applies a bandpass filter to the sound data. The frequency band of the bandpass filter is pre-set to correspond to the frequency characteristics of the unwanted sound, so that it is the band with a good signal-to-noise ratio within the sound data. By applying the bandpass filter, sound data is generated in which the target frequency components corresponding to the frequency characteristics of the unwanted sound are extracted from the sound data. Figure 4 shows an example of sound data in which the target frequency components have been extracted.

[0023] In step S13, the controller 20 generates sound pressure level data. Specifically, the controller 20 generates sound pressure level data by calculating an exponential moving average obtained by weighting and averaging the sound data of the target frequency components. By applying the exponential moving average, a weight that decays over time is given to the change in sound pressure. This allows the fluctuation of sound pressure to be represented as a smooth sound pressure level. The weight coefficients of the exponential moving average are adjusted in advance according to the time constant targeted by the exponential moving average. The sound pressure level data is sound pressure level data in the time domain, and as shown in Figure 5, it is a sound pressure level waveform with time on the horizontal axis and sound pressure level on the vertical axis.

[0024] In step S14, the controller 20 sets a threshold. As shown in Figure 5, the controller 20 sets a base threshold Thb for the sound pressure level data. The base threshold Thb is a threshold for separating background noise and is set based on the sound pressure level data. For example, the controller 20 sets the base threshold Thb using the average of the bottom 10% of the sound pressure level data as background noise. Next, the controller 20 sets the sound pressure level obtained by adding a pre-set adjustment level to the base threshold Thb as the first threshold Th1. The adjustment level is, for example, 6 dB, but is not limited to this.

[0025] In step S14, the controller 20 recognizes the region in the sound pressure level data where the sound pressure level is equal to or greater than the first threshold Th1 as an abnormal sound, and identifies the time (time period) in which it occurs. If multiple regions are recognized as abnormal sounds, the controller 20 performs the processing described later for each region of abnormal sound.

[0026] In step S15, the controller 20 performs frequency analysis. Specifically, the controller 20 performs frequency analysis on the sound data based on the specified time and the target frequency of the bandpass filter. For example, short-time FFT processing can be used for frequency analysis. Short-time FFT allows for the analysis of frequency components by performing a Fourier transform on the time-varying sound pressure level at short time intervals. Figure 6 shows an example of a spectrogram obtained by frequency analysis of sound data.

[0027] The controller 20 performs frequency analysis over a time period (time zone) that includes a small margin beyond the specified time. As mentioned above, since an exponential moving average is used to generate sound pressure level data, the sound pressure level data may contain a delay corresponding to the time constant compared to the sound data. Therefore, it is possible to target a large range including the time of the abnormal noise for frequency analysis and to perform frequency analysis appropriately for the abnormal noise.

[0028] In step S16, the controller 20 identifies the time and frequency at which the sound pressure level exceeds the second threshold as abnormal noise, based on the results of the frequency analysis. As described above, the frequency analysis process in step S15 analyzes a wide range including abnormal noise, so the results of the frequency analysis also include data that is noise. Therefore, the controller 20 excludes unnecessary data by using a second threshold. The second threshold is set at a position several dB (for example, 3 dB) above the maximum sound pressure level. Figure 7 is the spectrogram of the finally extracted abnormal noise.

[0029] In step S17, the controller 20 identifies the location of the abnormal noise. In this step S17, multiple sound data collected by multiple microphones of the microphone array 10 are used. The controller 20 estimates the direction (location) of the abnormal noise by utilizing the characteristics of the time difference until the sound reaches the multiple microphones. Techniques such as delayed-sum beamforming can be used to estimate the direction (location) of the abnormal noise. For example, the controller 20 filters the multiple sound data by the frequency band of the abnormal noise, performs beamforming on the sound data in that frequency band, and estimates the direction in which the signal strength is maximized as the direction of the abnormal noise.

[0030] In step S18, the controller 20 reads the surrounding image from the storage device 30. Then, based on the direction of the abnormal noise, the controller 20 creates a superimposed image by superimposing the location of the abnormal noise onto the surrounding image. The controller 20 displays the superimposed image on the display device 35.

[0031] Thus, the abnormal noise detection method of this embodiment includes acquiring sound data from the microphone array 10 and extracting target frequency components corresponding to the frequency characteristics of the abnormal noise from the sound data. With this method, since only data in the region where the abnormal noise is clearly audible can be used, the accuracy of abnormal noise detection can be improved.

[0032] Furthermore, the abnormal noise detection method includes generating sound pressure level data by calculating an exponential moving average obtained by weighting the target frequency components based on weighting coefficients adjusted according to a time constant. This method allows for the appropriate setting of the time constant for properly detecting the target abnormal noise, thereby improving sensitivity to abnormal noise.

[0033] Furthermore, the abnormal noise detection method identifies the time during which the sound pressure level exceeds a first threshold set based on sound pressure level data. Because this method sets the first threshold based on sound pressure level data, the same algorithm can be used to accurately detect abnormal noises in different test environments.

[0034] In addition, the abnormal noise detection method includes frequency analysis of sound data based on the identified time and target frequency, identifying frequency components whose sound pressure level exceeds a second threshold as abnormal noises, and estimating the location of the abnormal noises by utilizing the characteristics of the time difference until the sound reaches the microphone array 10. According to this method, even if the signal-to-noise ratio of the abnormal noise is poor, noise components can be removed, so the location of the abnormal noises can be estimated regardless of the environment.

[0035] Thus, according to the noise detection method of this embodiment, the location of an abnormal noise within the vehicle cabin can be estimated with high accuracy.

[0036] The abnormal noise detection method of this embodiment acquires an ambient image captured by the camera unit 15 around the microphone array 10, and displays the location of the abnormal noise on the ambient image based on the location of the abnormal noise.

[0037] This method overlays the location of the abnormal sound onto the surrounding image, making it easier to identify the source of the abnormal sound.

[0038] In the abnormal noise detection method of this embodiment, the process for determining time involves setting a base threshold for separating background noise based on sound pressure level data, and setting the sound pressure level obtained by adding a pre-set adjustment level to the base threshold as the first threshold.

[0039] This method sets the first threshold based on sound pressure level data, allowing for accurate detection of abnormal noises using the same algorithm in different test environments.

[0040] Furthermore, the abnormal sound detection device 1 described above is also included as one or more embodiments of the present invention. The abnormal sound detection device 1 comprises a microphone array 10 (sound collection device) including a plurality of microphones installed in the target space, and a controller that detects abnormal sounds occurring in the target space based on sound data, which is sound pressure in the time domain, acquired by the microphone array 10. This controller executes each of the processes of the abnormal sound detection method described above. With this abnormal sound detection device 1, the location of abnormal sounds in the target space can be estimated with high accuracy.

[0041] As described above, embodiments of the present invention have been presented, but the statements and drawings that constitute part of this disclosure should not be understood as limiting the invention. Various alternative embodiments, examples, and operational techniques will become apparent to those skilled in the art from this disclosure.

[0042] 1: Abnormal noise detection device, 10: Microphone array, 15: Camera unit, 20: Controller, 30: Storage device, 35: Display device

Claims

1. A method for detecting abnormal noises that occur in a target space, performed by a controller, comprising: acquiring sound data, which is sound pressure in the time domain, from a sound collection device including a plurality of microphones spatially arranged in the target space; extracting target frequency components corresponding to the frequency characteristics of the abnormal noise from the sound data; generating sound pressure level data, which is the sound pressure level in the time domain, by calculating an exponential moving average obtained by weighting the extracted target frequency components based on a weighting coefficient adjusted according to a time constant; identifying the time at which the sound pressure level becomes equal to or greater than a first threshold set based on the sound pressure level data; performing frequency analysis on the sound data based on the identified time and the target frequency; identifying the frequency component at which the sound pressure level becomes equal to or greater than a second threshold as the abnormal noise based on the results of the frequency analysis; and estimating the location of the abnormal noise by utilizing the characteristics of the time difference until the sound reaches the plurality of microphones.

2. The method for detecting abnormal noise according to claim 1, further comprising acquiring an ambient image of the area around the sound collecting device by capturing the area with a camera, and displaying the location of the abnormal noise on the ambient image based on the location of the abnormal noise.

3. The abnormal noise detection method according to claim 1 or 2, wherein the process for determining the time includes setting a base threshold for separating background noise based on the sound pressure level data, and setting a sound pressure level obtained by adding a preset adjustment level to the base threshold as the first threshold.

4. An abnormal sound detection device comprising: a sound collection device including a plurality of microphones installed in a target space; and a controller for detecting abnormal sounds occurring in the target space based on sound data, which is sound pressure in the time domain, acquired by the sound collection device, wherein the controller extracts target frequency components corresponding to the frequency characteristics of the abnormal sound from the sound data, generates sound pressure level data, which is the sound pressure level in the time domain, by calculating an exponential moving average of the extracted target frequency components based on a weighting coefficient adjusted according to a time constant, identifies the time at which the sound pressure level becomes equal to or greater than a first threshold set based on the sound pressure level data, performs frequency analysis on the sound data based on the identified time and the target frequency, identifies the frequency components at which the sound pressure level becomes equal to or greater than a second threshold as the abnormal sound based on the results of the frequency analysis, and estimates the location of the abnormal sound by utilizing the characteristics of the time difference until the sound reaches the plurality of microphones.

Images