Sound source detection method and sound source detection device

Abstract

To accurately estimate a sound source position of sound generated from a vehicle being inspected.SOLUTION: A sound source detection device comprises: a sound acquisition unit 10 by which sound generated from a vehicle being inspected is acquired and converted into an electric signal; a sound source position estimation unit 20 for calculating a sound source position from the electric signal; an air state measuring unit 40 for measuring an air state of a space from a position, in which sound generated from the measurement object is acquired, to the vehicle being inspected; a sound field state estimation unit 50 for estimating a sound field state of the space from the air state; and a sound source position correction unit 60 which calculates displacement of the sound source position from the sound field state and corrects the sound source position.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a sound source detection method and a sound source detection device. [Background technology]

[0002] For example, Patent Document 1 discloses a technology for displaying on a display screen a visual image of a sound source in which an audio heat map generated based on sounds picked up by a microphone array is superimposed on an image captured by an omnidirectional camera. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2018 / 20841 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, when generating an audio heat map, which is map data that visually indicates the location of a sound source, no consideration is given to the deviation of the sound source position measured by the microphone array.

[0005] That is, there is room for further improvement in detecting the sound source position in order to improve the accuracy of sound source measurement. [Means for solving the problem]

[0006] The sound source detection of the present invention involves acquiring sound generated from a measurement object and converting it into an electrical signal, calculating the sound source position from this electrical signal, measuring the air condition in the space from the position where the sound generated from the measurement object is acquired to the measurement object, estimating the sound field state of the space from the air condition, calculating the deviation of the sound source position from the sound field state, and correcting the sound source position, An image and temperature information of the measurement object are acquired, and a sound pressure heat map generated based on the corrected sound source position and a temperature heat map of the measurement object are superimposed on the image of the measurement object and displayed, the sound pressure heat map of the sound source position displays differences in sound pressure using contour lines, and the temperature heat map of the measurement object displays differences in temperature using different colors. It is characterized by the following. [Effects of the Invention]

[0007] According to the present invention, the state of the medium through which the sound propagates can be reflected, thereby enabling the sound source position to be accurately estimated. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a functional block diagram of a first embodiment to which the present invention is applied. [Figure 2] 4 is a flowchart showing the flow of processing in the first embodiment. [Figure 3] FIG. 3 is an explanatory diagram showing a specific display example in the first embodiment. [Figure 4] 10A and 10B are explanatory diagrams showing specific display examples in the first embodiment, in which FIG. 10A shows a display in which only the heat map of sound pressure is superimposed, and FIG. 10B shows a display in which only the heat map of air condition is superimposed. [Figure 5] FIG. 10 is an explanatory diagram showing a specific display example in the first embodiment, where a shows various heat maps superimposed with a spectrogram showing the temporal change in the sound pressure heat map, and b shows an enlarged view of the spectrogram showing the temporal change in the sound pressure heat map. [Figure 6] FIG. 10 is a functional block diagram of a second embodiment to which the present invention is applied. [Figure 7] 10 is a flowchart showing the flow of processing in a second embodiment. [Figure 8] FIG. 10 is a functional block diagram of a third embodiment to which the present invention is applied. [Figure 9] 10 is a flowchart showing the flow of processing according to a third embodiment. [Figure 10] FIG. 11 is an explanatory diagram showing a specific display example in the third embodiment. [Figure 11] FIG. 11 is an explanatory diagram showing a specific display example in the third embodiment. [Figure 12] 10A and 10B are explanatory diagrams showing specific display examples in the third embodiment, in which (a) shows the names of abnormal noise determination items and results superimposed in text, and the vehicle status superimposed in a time series graph, and (b) shows an enlarged view of the superimposed time series graph showing the vehicle status. [Figure 13] FIG. 10 is an explanatory diagram showing an outline of a system configuration of a fourth embodiment to which the present invention is applied. DETAILED DESCRIPTION OF THE INVENTION

[0009] An embodiment of the present invention will be described in detail below with reference to the drawings. An embodiment in which the present invention is applied to the detection of abnormal noise from a vehicle will be described below. This embodiment of abnormal noise detection is implemented, for example, to detect the location of a sound source generated from a completed vehicle, which is the object of measurement, during the completed vehicle inspection process, which is the final stage of an automobile production line. Generally, during the completed vehicle inspection process, an operator (inspector) test drives the completed vehicle to be inspected on free rollers, and inspects a number of items, including the engine, meters, brakes, etc.

[0010] During this inspection, sounds generated from the completed vehicle, which is the measurement object, are detected, and it is determined which part of the completed vehicle the sound is generated from.

[0011] Hereinafter, the device of this embodiment will also be referred to as a "sound source detection device," and this sound source detection device is configured as a part of an inspection device in the finished vehicle inspection process.

[0012] The completed vehicles to be inspected are not limited to vehicles powered solely by gasoline engines (in other words, spark ignition internal combustion engines) or diesel engines (in other words, compression ignition internal combustion engines), but may also be so-called hybrid vehicles or electric vehicles.

[0013] 1 shows a functional block diagram of a sound source detection device of Example 1. The sound source detection device of Example 1 includes a sound acquisition unit 10, a sound source position estimation unit 20, an image acquisition unit 30, an air condition measurement unit 40, a sound field condition estimation unit 50, a sound source position correction unit 60, a synthetic image generation unit 70, a display control unit 80, and a display unit 90.

[0014] The sound acquisition unit 10 acquires sounds generated from a vehicle under inspection, which is the measurement object, and converts the sounds into electrical signals, that is, sound data.

[0015] The sound acquisition unit 10 includes a microphone array having a plurality of microphones arranged in an array that acquires sounds generated by the vehicle under inspection and converts them into sound data, and a recording unit that temporarily stores the sound data from the microphone array. The microphone array is placed outside the vehicle so that it can collect sounds such as the engine sound of the vehicle under inspection running on free rollers. The directivity and frequency characteristics of the microphones are selected according to the position of the vehicle under inspection that is the measurement target and the frequency band of the engine sound. The microphones in the microphone array may, for example, have a single directivity toward the vehicle under inspection.

[0016] Instead of a microphone array, the sound acquisition unit 10 may, for example, move a single microphone to acquire sound from a sound source at multiple different positions and use the acquired sound as sound data. Instead of a microphone array, the sound acquisition unit 10 may provide multiple sound reflectors in space and use a single microphone to acquire the sound as sound data. The sound acquisition unit 10 is not limited to a microphone array, and may use any microphone that can collect sound from a sound source at multiple positions and use the collected sound as sound data. The microphone used in the sound acquisition unit 10 may also be, for example, an existing microphone installed to collect horn sounds in the finished vehicle inspection process.

[0017] The sound source position estimation unit 20 measures the time difference between the multiple sounds measured by the sound acquisition unit 10, and estimates the sound source position from the amount of time difference.

[0018] More specifically, the sound source position estimation unit 20 measures the difference Δt in the time it takes for sound to reach multiple microphones from the vehicle under inspection, based on the sound data acquired by the sound acquisition unit 10, and estimates the sound source position using this value. A specific method for estimating the sound source position is to estimate the sound source position using an angle θ that can be expressed using the time difference ΔT between sound arriving at microphone A and microphone B, which are separated by a distance d (m), at a sound speed of C, and trigonometry. The angle θ can be expressed using Equation 1: θ=sin-1 (C×ΔT / d) (1) The sound source position estimation unit 20 may estimate the sound source position with higher accuracy by using the delay sum of the sound or the plane wave / spherical wave characteristics of the sound.

[0019] Furthermore, it is also possible to extract only sounds of specific frequencies using techniques such as FFT (Fast Fourier Transform) or wavelet analysis, and calculate the time shift of sounds of each frequency to estimate the sound source position with high accuracy.

[0020] The video acquisition unit 30 is a video camera or the like that indicates the position of the sound source calculated by the sound source position estimation unit 20 and the sound source position correction unit 60 and indicates the position of the sound on the vehicle under inspection. The video acquisition unit 30 acquires images or videos.

[0021] The air condition measuring unit 40 measures the temperature of the vehicle under inspection and the state of the air that serves as the medium for sound in the space from the vehicle under inspection to the sound acquiring unit 10. In other words, the air condition measuring unit 40 measures the spatial state that affects the sound from the vehicle under inspection.

[0022] The air condition measuring unit 40 in the first embodiment is an infrared camera capable of non-contact temperature measurement. The infrared camera acquires the temperature conditions of the vehicle under inspection and the temperature conditions from the vehicle under inspection to the sound acquisition unit 10. The air condition measuring unit 40 may be a measuring device other than an infrared camera that can measure the temperature conditions, such as a space temperature measuring device that uses ultrasound.

[0023] Note that spatial conditions that affect sound, i.e., air conditions related to sound, include, for example, air temperature, humidity, air pressure, direction and strength of air flow, as well as gas coefficients, the mixture ratio, specific heat ratio, molecular weight, etc. Therefore, the air condition measuring unit 40 may be a device that measures these air conditions related to sound.

[0024] The sound field state estimation unit 50 estimates the sound field state of the space from the vehicle under inspection to the sound acquisition unit 10 based on the data acquired by the air condition measurement unit 40 .

[0025] For example, the speed of sound C is expressed by Equation 2: C=(kRT / M) 0.5 …(2). Here, k is the specific heat ratio of air, R is the gas constant, T is the temperature of air, and M is the molecular weight of air.

[0026] The effect of air humidity on sound absorption is described, for example, in ISO 9613-1: Acoustics - Attenuation of sound during propagation outdoors - Part 1: Calculation of the absorption of sound by the atmosphere.

[0027] The wind direction from the temperature distribution is from low temperature areas to high temperature areas. The wind speed can be estimated based on the volume of the heat source, and can be calculated using, for example, TIV (Thermal Image Velocimetry).

[0028] When the air condition measuring unit 40 is an infrared camera, if there are high and low heat sources, the sound field state estimating unit 50 estimates that the temperature of the space nearby is close to that of the heat source, estimates the sound field state due to temperature changes, estimates the speed of sound from the temperature up to each microphone of the sound acquiring unit 10, and calculates the time lag Δtt. Similarly, if the temperature distribution in the space is known by the air condition measuring unit 40, the sound field state estimating unit 50 may estimate the speed of sound up to each microphone in the same manner as above and calculate the time lag Δtt.

[0029] The sound source position correction unit 60 corrects the position of the sound source calculated by the sound source position estimation unit 20 when the sound field state estimation unit 50 detects a change in the sound propagation state.

[0030] The sound source position is corrected using factors such as the speed of sound, which affects the time lag of the sound for each microphone of the sound acquisition unit 10 used when identifying the sound source. For example, in the above-mentioned formula 1, if the temperature T is 30°C higher, the sound speed C becomes about 8% faster. The sound source position is corrected using these characteristics and calculation results. Specifically, the sound source position estimated by the sound source position estimation unit 20 is corrected using the value Δtt calculated by the sound field state estimation unit 50.

[0031] The composite image generation unit 70 generates a heat map like a thermograph using sound pressure as a parameter for the sound source position obtained by the sound source position correction unit 60, and superimposes it on the image or video obtained by the video acquisition unit 30 by performing processing such as semi-transparency. In other words, the composite image generation unit 70 uses the results calculated by the sound source position correction unit 60 to generate an image or video in which the sound source position is displayed semi-transparently as a heat map of the sound pressure at the sound source position and is superimposed on the image or video of the vehicle under inspection obtained by the video acquisition unit 30.

[0032] In addition, the composite image generation unit 70 generates a heat map of the air condition from the temperature information measured by the infrared camera, which is the air condition measurement unit 40, and generates an image or video that is superimposed semi-transparently on the image or video of the vehicle under inspection acquired by the video acquisition unit 30.

[0033] The display control unit 80 controls and displays a plurality of types of display, such as switching between the image (first composite image) or video (first composite video) generated by the composite image generation unit 70 and the heat map image (second composite image) or heat map video (second composite video) acquired by the infrared sensor of the air condition measurement unit 40. The display control unit 80 processes the image (first composite image) or video (second composite video) generated by the composite image generation unit 70 so that it is easy to view.

[0034] Specifically, since it is difficult to understand when both the sound source position information and the air information are superimposed, the heat map of the sound pressure at the sound source position can be displayed as separate superimposed images, with only the contour lines displayed and the semi-transparent areas filled in with color and made transparent.

[0035] The contour display is not limited to a heat map of sound pressure at the sound source position, but may also be applied to a heat map of air condition such as temperature information. Furthermore, when the heat map of sound pressure at the sound source position and the heat map of air condition are superimposed on an image or video of the vehicle under inspection, characters, pictograms, images, etc. that indicate the displayed content may be added.

[0036] The display control unit 80 may have the same functions as the composite image generation unit 70 and may perform the same processing.

[0037] The display unit 90 displays sound source position information and air information to relevant parties around the vehicle under inspection, such as workers (including the vehicle driver), managers, and data scientists who utilize the data. The display unit 90 is, for example, a liquid crystal display or an organic EL display. In addition, if the display unit 90 is accompanied by sound, it is a sound source, an amplifier, a speaker, etc. for generating and emitting sound.

[0038] FIG. 2 is a flowchart showing the flow of processing in the sound source detection device of the first embodiment.

[0039] In step S1, the microphone of the sound acquisition unit 10 collects sounds generated by the vehicle under test running on the free rollers, and acquires the collected sounds as sound data.

[0040] In step S2, the sound source position is estimated using the acquired sound data (sound source position estimation unit 20).

[0041] In step S3, an image or video is acquired from the infrared camera.

[0042] In step S4, the sound field state of the space is estimated (sound field state estimation unit 50).

[0043] In step S5, the sound source position estimated in step S2 is corrected in accordance with the sound field state of the space (sound source position correcting unit 60). In step S6, a thermography-like heat map is generated with sound pressure as a parameter for the sound source position estimated in step S2, and is superimposed (first composite image or first composite video) on the image or video acquired by video acquisition unit 30 after processing such as making it semi-transparent. Also in step S6, a heat map of the air condition is generated from the data acquired in step S3 (temperature information measured by the infrared camera), and is superimposed (second composite image or second composite video) on the image or video of the vehicle under inspection acquired by video acquisition unit 30 after processing such as making it semi-transparent.

[0044] In step S7, processing is performed so that the first composite image or first composite video generated in step S6 and the second composite image or second composite video generated in step S6 are displayed alternately on the display unit 90.

[0045] In step S8, it is determined whether or not a decision has been made to stop the display on the display unit 90. If a decision has been made to stop the display on the display unit 90, the current routine is terminated. If a decision has not been made to stop the display on the display unit 90, the process proceeds to step S1. The decision on whether or not to stop the display on the display unit 90 is made, for example, by a person involved, such as a worker, operating a switch that stops the display function to determine whether or not to turn the display on the display unit 90 on / off.

[0046] Specific display examples in the first embodiment described above are shown in Figures 3 to 5. The attached Figures 3 to 5 are black and white images, but in actual images, the temperatures are color-coded, for example, with high temperatures being red, medium temperatures being yellow, and low temperatures being blue, and the temperature is indicated by varying the brightness of each color.

[0047] The first display example shown in Figure 3 is a composite video or composite image in which a heat map of sound pressure at the sound source position and a heat map of temperature (air condition) generated from temperature information measured by an infrared camera are superimposed on an image or video of a vehicle under inspection acquired by the video acquisition unit 30.

[0048] In this first display example, the heat map of sound pressure at the sound source position is shown as contour lines only, making it easier to distinguish from the heat map of temperature (air condition) generated from temperature information measured by an infrared camera.

[0049] In addition, for the temperature heat map, pictograms and legends for each measurement quantity (e.g., red for high temperatures and blue for low temperatures on a thermometer) have been added.

[0050] The second display example shown in Figure 4 is a method in which Figures 4(a) and 4(b) are not always displayed, but are alternately displayed over time. This display has the advantage that the display for each measurement quantity is organized and the differences between them are easily seen the moment the display is switched. Figure 4(a) is a composite video or image in which a heat map of sound pressure at the sound source position is superimposed as contour lines only on an image or video of the vehicle under inspection acquired by the video acquisition unit 30. Figure 4(b) is a composite video or image in which a heat map of air conditions (temperature) generated from data acquired by the air condition measurement unit 40 (infrared camera) is superimposed on an image or video of the vehicle under inspection acquired by the video acquisition unit 30.

[0051] The third display example shown in FIG. 5 displays the time-varying contour line display of the sound source as a spectrogram, allowing the user to see the changes in the frequency characteristics and sound pressure of the sound.

[0052] A spectrogram is three-dimensional data that includes time. Here, in accordance with a common format, the horizontal axis represents time and the vertical axis represents frequency. The total time length is, for example, 20 seconds, and the frequency range includes, for example, up to around 1000 Hz.

[0053] The brightness and color of each point represent the power (dBA) (or sound pressure or amplitude) of a certain frequency at a certain point in time. The attached drawing is a black and white image, but in an actual spectrogram display, for example, high power is colored red, medium power is yellow-green, and low power is blue, and the power is indicated by changes in brightness within each color.

[0054] Figure 5(a) is a composite video or image in which a heat map of sound pressure at the sound source position is superimposed as contour lines only, a spectrogram showing the change in the contour line display of the sound source over time, and a heat map of air conditions (temperature) generated from data acquired by the air condition measuring unit 40 (infrared camera) are superimposed on an image or video of a vehicle under inspection acquired by the video acquisition unit 30.

[0055] FIG. 5(b) shows an enlarged spectrogram showing the time-varying contour representation of the sound source.

[0056] Displays such as the first to third display examples are controlled by the display control unit 80 so as to be displayed at appropriate timing, for appropriate duration, and under appropriate display conditions (brightness, color, etc.).

[0057] In the second display example, the sound source and temperature are displayed cyclically over time. This makes it easier to see the difference between the sound source and the temperature, and makes it easier to compare the two displays, such as when they are displayed side by side.

[0058] The display processed in this way is then presented as the measurement results to the inspector, manager, and other relevant parties on a display device such as a liquid crystal display or organic EL of the display unit 90. Note that the sound source monitoring results may be presented as sound or voice to the inspector, manager, and other relevant parties as the measurement results using an audio presentation device such as a speaker.

[0059] In addition, the results of the sound source monitoring may be visually displayed using video or images, and the results of the sound source monitoring may be announced using sound or voice, and presented visually and audibly as measurement results to relevant parties such as inspection workers and managers.

[0060] After performing this process, sound source monitoring continues unless an operation such as a stop button is pressed to stop sound source monitoring. If sound source monitoring continues, sound data is collected again as in the above process, and sound source monitoring is performed. If sound source monitoring is to be terminated, a message indicating termination is displayed on the display unit 90. A buzzer or voice may be used to notify the end of the monitoring.

[0061] As explained above, in the first embodiment described above, the sound acquisition unit 10 acquires sound generated from the vehicle under inspection and converts it into an electric signal, the sound source position estimation unit 20 calculates the sound source position from this electric signal, the air condition measurement unit 40 measures the air condition of the vehicle under inspection and the space from the sound acquisition unit 10 to the vehicle under inspection, the sound field condition estimation unit 50 estimates the sound field condition of the space from the air condition, and the sound source position correction unit 60 calculates the deviation of the sound source position from the sound field condition and corrects the sound source position. Therefore, the sound source detection device of the first embodiment can estimate the sound source position more accurately than conventional methods by reflecting the state of the medium through which the sound propagates.

[0062] In the first embodiment described above, the sound field state of the space is estimated using the sound propagation speed from the sound source position calculated using a calculation formula with parameters being the physical quantities of the air in the space from the sound acquisition unit 10 to the vehicle under inspection. Therefore, the sound source detection device of the first embodiment can measure and estimate the state of the air, which is the medium of sound, at high speed and in a short time with a low calculation load.

[0063] In the first embodiment described above, the image acquisition unit 30 acquires an image of the vehicle under inspection, and the composite image generation unit 70 superimposes a heat map of sound pressure generated based on the corrected sound source position on the image of the vehicle under inspection to generate a composite image or a composite video, and the composite image or the composite video is displayed on the screen of the display unit 90. Therefore, the sound source detection device of the first embodiment can display the accurately estimated sound source position in an easy-to-understand manner.

[0064] In the first embodiment described above, a heat map of the air condition generated by acquiring information on the air condition in the space from the sound acquisition unit 10 to the vehicle under inspection is superimposed on a video or image of the vehicle under inspection to generate a composite video or composite image, and the display control unit 80 processes the sound pressure heat map and the air condition heat map in this composite video or composite image so that they are easy to see, and displays them on the screen of the display unit 90. Therefore, the sound source detection device of the first embodiment can clearly display the accurately estimated sound source position, and can also clearly display the factors that corrected the sound source position.

[0065] In the first embodiment described above, the air condition in the space from the sound acquisition unit 10 to the vehicle under inspection is measured in a non-contact manner. Therefore, the sound source detection device of the first embodiment can measure and estimate the temperature condition of the sound medium inexpensively and simply.

[0066] In the first embodiment described above, the display of the heat map of the sound source position and sound pressure is corrected based on the temperature distribution in the space. Therefore, the sound source detection device of the first embodiment can easily estimate the sound source position with high accuracy and display it in an easy-to-understand manner.

[0067] In the first embodiment described above, the higher the temperature in the space, the farther the sound source position is, and the acquired sound pressure may be corrected to decrease, thereby correcting the display of the sound pressure heat map. In this case, the sound source detection device can easily estimate the sound source position with high accuracy and display it in an easy-to-understand manner.

[0068] In the first embodiment described above, the propagation speed of sound from the sound source position may be corrected so that it increases as the temperature at the sound source position increases. In this case, the sound source detection device can easily estimate the sound source position with high accuracy and display it in an easy-to-understand manner.

[0069] In the first embodiment described above, the video or image (composite video or composite image) of the vehicle under inspection is displayed on a visual display device such as a liquid crystal display. Information can be presented in an easy-to-understand manner in an appropriate modal depending on the situation at the site, the characteristics of the informant, etc. Therefore, the sound source detection device of the first embodiment can present information in an easy-to-understand manner in an appropriate modal depending on the situation at the site, the characteristics of the informant, etc.

[0070] In the second display example of the first embodiment described above, a first composite video or first composite image is generated by superimposing a heat map of sound pressure generated based on the corrected sound source position on a video or image of the vehicle under inspection, and a second composite video or second composite image is generated by acquiring information on the air condition in the space from the sound acquisition unit 10 to the vehicle under inspection and superimposing the heat map on the video or image of the vehicle under inspection, and the first composite video or first composite image and the second composite video or second composite image are displayed alternately for a certain period of time. Therefore, in the second display example of the first embodiment, it is not necessary to forcibly change the display content to generate a single composite image, and the spatial difference with respect to the measurement target becomes clear the moment the switch is made, making it possible to present information in an easy-to-understand manner.

[0071] In the first and third display examples of the first embodiment described above, a heat map of sound pressure generated based on the corrected sound source position and a heat map of air conditions generated by acquiring information on the air conditions in the space from the sound acquisition unit 10 to the vehicle under inspection are displayed superimposed in parallel on a video or image of the vehicle under inspection. Therefore, in the first and third display examples of the first embodiment, information can be presented in an easy-to-understand manner.

[0072] In the first embodiment described above, the heat map of sound pressure generated based on the corrected sound source position is not colored but is represented only by contour lines. Therefore, in the first embodiment, even if different items (sound pressure at the sound source position and the air condition in the space between the sound acquisition unit 10 and the vehicle under inspection) are displayed on the heat map, the viewer can easily distinguish them without getting confused.

[0073] In the first embodiment described above, the phase of the sound generated from the vehicle under inspection may be used when calculating the sound source position.

[0074] Another embodiment of the present invention will be described below. Note that the same components as those in the first embodiment described above are given the same reference numerals, and redundant explanations will be omitted.

[0075] A sound source detection device according to a second embodiment of the present invention will be described with reference to Figures 6 and 7. Figure 6 shows a functional block diagram of the sound source detection device according to the second embodiment. Figure 7 is a flowchart showing the flow of processing in the sound source detection device according to the second embodiment.

[0076] The sound source detection device of the second embodiment has substantially the same configuration as the sound source detection device of the first embodiment described above, but the air condition measuring unit 41 is an anemometer using ultrasonic waves (ultrasonic anemometer) instead of an infrared camera.

[0077] In the second embodiment, the air condition measuring unit 41 measures the state of the air that serves as a medium for sound in the space from the vehicle under inspection to the sound acquiring unit 10. In other words, the air condition measuring unit 41 measures the spatial state that affects the sound from the vehicle under inspection.

[0078] The air condition measuring unit 41 of the second embodiment measures the wind direction and wind speed of the airflow in the space from the vehicle under inspection to the sound acquiring unit 10 and in the space around the vehicle under inspection in a non-contact manner. Note that the air condition measuring unit 41 may use a measuring means other than an ultrasonic anemometer as long as it can measure the state of the airflow.

[0079] In the second embodiment, when the air condition measuring unit 41 is an ultrasonic anemometer, the sound field state estimating unit 50 estimates the sound speed and the direction and amount of sound movement using the wind speed magnitude F and wind directions Fx, Fy, and Fz, and calculates the time delay Δtt between the sound source position and the arrival of the sound at each microphone of the sound acquiring unit 10. The sound speed C can be expressed as in Equation 3, where C0 is the speed of sound when the wind speed is zero, and F is the wind speed: C=C0+F...(3). F, which represents the wind speed, is positive when it is in the same direction as the sound, and negative when it is opposite the sound's direction of travel. The influence of wind direction on the sound's direction of travel can be calculated in the same way as wind speed, using the direction and magnitude of a vector in three-dimensional space.

[0080] In the second embodiment, the sound source position correction unit 60 corrects the sound source position calculated by the sound source position estimation unit 20 based on the sound field state acquired by the sound field state estimation unit 50.

[0081] In the second embodiment, the synthetic image generation unit 70 generates a heat map like a thermograph using sound pressure as a parameter for the sound source position obtained by the sound source position correction unit 60, and superimposes it on the image or video obtained by the video acquisition unit 30 by performing processing such as semi-transparency. In other words, the synthetic image generation unit 70 uses the results calculated by the sound source position correction unit 60 to generate an image or video in which the sound source position is superimposed on the image or video of the vehicle under inspection obtained by the video acquisition unit 30, with the display of the sound source position being semi-transparent.

[0082] In addition, in the second embodiment, the composite image generation unit 70 uses information on the air flow measured by the ultrasonic measuring meter, which is the air condition measurement unit 41, to represent the wind direction and wind speed of the air flow at each point around the vehicle under inspection using the direction and length of an arrow, and generates an image or video by superimposing this on the image or video of the vehicle under inspection acquired by the video acquisition unit 30.

[0083] The flow of processing in the sound source detection device of the second embodiment will be described with reference to FIG.

[0084] In step S21, the microphone of the sound acquisition unit 10 collects sounds generated by the vehicle under test running on the free rollers, and acquires the collected sounds as sound data.

[0085] In step S22, the sound source position is estimated using the acquired sound data (sound source position estimation unit 20).

[0086] In step S23, the measurement results of the ultrasonic anemometer are acquired.

[0087] In step S24, the sound field state of the space is estimated (sound field state estimation unit 50).

[0088] In step S25, the sound source position estimated in step S22 is corrected in accordance with the sound field state of the space (sound source position correcting unit 60).

[0089] In step S26, a thermography-like heat map is generated using the sound pressure corrected in step S25 as a parameter for the sound source position corrected in step S25, and the heat map is superimposed (first composite image or first composite video) on the image or video acquired by video acquisition unit 30 after processing such as semi-transparency. Also in step S26, the data acquired in step S23 (wind direction and wind speed of airflow measured by the ultrasonic anemometer) is converted into a pictogram and superimposed (second composite image or second composite video) on the image or video of the vehicle under inspection acquired by video acquisition unit 30 after processing such as semi-transparency.

[0090] In step S27, processing is performed so that the first composite image or first composite video generated in step S26 and the second composite image or second composite video generated in step S26 are displayed alternately on the display unit 90.

[0091] In step S28, it is determined whether or not a decision has been made to stop the display on the display unit 90. If a decision has been made to stop the display on the display unit 90, the current routine is ended. If a decision has not been made to stop the display on the display unit 90, the process proceeds to step S21.

[0092] The sound source detection device of the second embodiment can achieve substantially the same effects as the sound source detection device of the first embodiment described above.

[0093] In the second embodiment, the display of the heat map of the sound source position and sound pressure is corrected based on the wind direction and wind speed of the airflow in the space. Therefore, the sound source detection device of the first embodiment can easily estimate the sound source position with high accuracy and display it in an easy-to-understand manner.

[0094] A sound source detection device according to a third embodiment of the present invention will be described with reference to Figures 8 and 9. Figure 8 shows a functional block diagram of the sound source detection device according to the third embodiment. Figure 9 is a flowchart showing the flow of processing in the sound source detection device according to the third embodiment.

[0095] The sound source detection device of the third embodiment has substantially the same configuration as the sound source detection device of the first embodiment described above, but in addition to the air condition measurement unit 40 which is an infrared camera, it has an air condition measurement unit 41 which is an ultrasonic anemometer.

[0096] That is, the sound source detection device of the third embodiment measures temperature and air flow as spatial conditions that affect the sound from the vehicle under inspection.

[0097] Furthermore, the sound source detection device of the third embodiment has a vehicle state acquisition unit 100, and is capable of displaying the state of the vehicle under inspection.

[0098] Furthermore, the sound source detection device of the third embodiment has an abnormal sound determination unit 110, which determines whether the sound of the measured sound source is normal or abnormal, and can display the result.

[0099] In the third embodiment, the sound field state estimation unit 50 calculates the time lag Δtt between the sound arriving at each microphone of the sound acquisition unit 10 from the sound source position, taking into account the temperature measured by the air state measurement unit 40 and the state of the air flow measured by the air state measurement unit 41.

[0100] In the third embodiment, the sound source position correction unit 60 corrects the sound source position calculated by the sound source position estimation unit 20 based on the sound field state acquired by the sound field state estimation unit 50, i.e., based on the temperature and airflow.

[0101] The vehicle state acquisition unit 100 acquires information for associating the state of the vehicle under inspection with the sound measured by the sound acquisition unit 10. The vehicle state acquisition unit 100 acquires, for example, the vehicle speed of the vehicle under inspection, the engine speed of the internal combustion engine mounted on the vehicle, the accelerator opening of the vehicle under inspection, the amount of depression of the brake pedal of the vehicle under inspection, the steering angle of the vehicle under inspection, the acceleration of the vehicle under inspection, the gear shift position of the vehicle under inspection, etc.

[0102] The vehicle condition information acquired by the vehicle condition acquisition unit 100 is not limited to the above, but may be any information related to the vehicle being inspected, such as the coolant temperature of the internal combustion engine installed in the vehicle being inspected, the operating status and set temperature of the air conditioning system (air conditioner) of the vehicle being inspected, and the number of occupants in the vehicle being inspected.

[0103] The abnormal sound detection unit 110 determines whether the sound from the sound source corrected by the sound source position correction unit 60 is a normal sound. The abnormal sound detection unit 110 has an abnormal sound detection algorithm modeled, for example, with an unsupervised learning machine learning model, which is one of the machine learning techniques. As an unsupervised learning machine learning model, for example, an unsupervised learning model that only learns normal data and does not learn abnormal data can be applied.

[0104] The abnormal sound detector 110 can also use, for example, a supervised learning machine learning model that learns and determines both normal and abnormal sound data.

[0105] Furthermore, the abnormal sound detection unit 110 is not limited to machine learning, but may determine whether something is normal or abnormal by setting thresholds for the physical quantities of sound, such as sound pressure, frequency characteristics, and time modulation of sound, or may determine whether something is normal or abnormal using the MT method used in quality engineering.

[0106] In the third embodiment, the synthetic image generation unit 70 generates a heat map like a thermograph using sound pressure as a parameter for the sound source position obtained by the sound source position correction unit 60, and superimposes it on the image or video obtained by the video acquisition unit 30 by performing processing such as semi-transparency. In other words, the synthetic image generation unit 70 uses the results calculated by the sound source position correction unit 60 to generate an image or video in which the sound source position is superimposed on the image or video of the vehicle under inspection obtained by the video acquisition unit 30, with the display of the sound source position being semi-transparent.

[0107] In addition, in the third embodiment, the composite image generation unit 70 uses temperature information formed by the infrared camera, which is the air condition measurement unit 40, and air flow information measured by the ultrasonic measuring instrument, which is the air condition measurement unit 41, to represent the temperature at each point around the vehicle under inspection with color, and the wind direction and wind speed of the air flow with the direction and length of an arrow, and generates an image or video by superimposing this on the image or video of the vehicle under inspection acquired by the video acquisition unit 30.

[0108] Furthermore, in the third embodiment, the composite image generation unit 70 displays the results of the abnormal noise determination and the state of the vehicle under inspection superimposed on an image or video of the vehicle under inspection, in which a heat map of the sound pressure of the sound source, temperature information, and air volume information are superimposed on the image or video of the vehicle under inspection.

[0109] The flow of processing in the sound source detection device of the third embodiment will be described with reference to FIG.

[0110] In step S31, the microphone of the sound acquisition unit 10 collects sounds generated by the vehicle under test running on the free rollers, and acquires the collected sounds as sound data.

[0111] In step S32, the sound source position is estimated using the acquired sound data (sound source position estimation unit 20).

[0112] In step S33, an image or video is acquired from the infrared camera.

[0113] In step S34, the measurement results of the ultrasonic anemometer are acquired.

[0114] In step S35, the sound field state of the space is estimated (sound field state estimation unit 50).

[0115] In step S36, the sound source position estimated in step S22 is corrected in accordance with the sound field state of the space (sound source position correcting unit 60).

[0116] In step S37, a thermography-like heat map is generated using the sound pressure corrected in step S36 as a parameter for the sound source position corrected in step S36, and the heat map is superimposed (first composite image or first composite video) on the image or video acquired by video acquisition unit 30 after processing such as semi-transparency. Also in step S37, the data acquired in step S34 (wind direction and wind speed of airflow measured by the ultrasonic anemometer) is converted into a pictogram and superimposed (second composite image or second composite video) on the image or video of the vehicle under inspection acquired by video acquisition unit 30 after processing such as semi-transparency.

[0117] In step S38, processing is performed so that the first composite image or first composite video generated in step S37 and the second composite image or second composite video generated in step S37 are displayed alternately on the display unit 90.

[0118] In step S39, it is determined whether or not a decision has been made to stop the display on the display unit 90. If a decision has been made to stop the display on the display unit 90, the current routine is ended. If a decision has not been made to stop the display on the display unit 90, the process proceeds to step S31.

[0119] Specific display examples in the third embodiment described above are shown in Figures 10 to 12. The attached Figures 10 to 12 are black and white images, but in actual images, the temperatures are color-coded, for example, with high temperatures being red, medium temperatures being yellow, and low temperatures being blue, and the temperature is indicated by varying the brightness of each color.

[0120] The fourth display example shown in Figure 10 is a composite video or composite image in which a heat map of the sound pressure at the sound source location, symbols (circles) which are predetermined shapes that display the temperature at each point with their colors, and symbols (arrows) that display the wind direction (direction) and wind speed (magnitude) of the air flow at each point are superimposed on an image or video of the vehicle under inspection acquired by the video acquisition unit 30.

[0121] In the fourth display example shown in Fig. 10, the heat map of sound pressure at the sound source position is displayed as contour lines only. Also, in the fourth display example shown in Fig. 10, the direction and length of the arrow indicate the wind direction and wind speed of the airflow at each point. That is, the direction of the arrow indicates the wind direction of the airflow at that point, and the length of the arrow indicates the wind speed at that point.

[0122] In other words, in the fourth display shown in Figure 10, a circular symbol indicating the temperature and an arrow protruding from the circular symbol indicating the direction (wind direction) and magnitude (wind speed) of the air flow are generated for each of the vehicle under inspection and multiple points around the vehicle under inspection, with the color of the circle representing the temperature at that point, the direction of the arrow representing the direction (wind direction) of the air flow at that point, and the length or thickness of the arrow representing the magnitude (wind speed) of the air flow at that point.

[0123] 10 is just an example, and the wind speed at each point may be represented by the thickness of the arrow. Also, the way in which the temperature and air flow are represented is not limited to that shown in Fig. 10, and the temperature at each point may be represented by a symbol other than a circle (for example, a rectangle), and the air flow at each point may be represented by a symbol other than an arrow (for example, a line diagram with a fluttering flag).

[0124] For example, the wind speed at each point may be represented by the color of the arrow or the size of the circle in Figure 10. The temperature at each point may be represented by the color depth of the temperature symbol, the thickness of the line, the number of lines, etc. In short, the temperature and air flow may be represented in any way as long as the respective physical quantities are displayed graphically.

[0125] The fifth display example shown in FIG. 11 is obtained by superimposing the item names and results of the abnormal noise determination in text on the first display example described above.

[0126] The sixth display example shown in FIG. 12 is an example in which the names and results of the abnormal noise determination are superimposed in text on the first display example described above, and the vehicle state is superimposed in a time series graph.

[0127] Fig. 12(a) is a composite video or image in which the names of abnormal noise determination items and results are superimposed in text on the above-mentioned display example 1, and a time series graph showing the vehicle status is superimposed on it. Fig. 12(b) is an enlarged view of the superimposed time series graph showing the vehicle status.

[0128] The sound source detection device of the third embodiment can achieve substantially the same effects as the sound source detection devices of the first and second embodiments described above.

[0129] Furthermore, in the third embodiment described above, a circular symbol indicating the temperature and an arrow protruding from the circular symbol indicating the direction (wind direction) and magnitude (wind speed) of the airflow are generated for each of the vehicle under inspection and multiple points around the vehicle, the color of the circle represents the temperature at that point, the direction of the arrow represents the direction of the airflow at that point, and the length or thickness of the arrow represents the magnitude (wind speed) of the airflow at that point, and the circle and arrow are superimposed on a video or image of the vehicle under inspection and displayed on the screen. Therefore, the sound source detection device of the third embodiment can display the state of the air, which is the medium of sound, in an easy-to-understand manner in the composite image generation unit 70.

[0130] In the third embodiment described above, the sound from the sound source position is judged to be normal / abnormal, and the judgment result is superimposed on the video or image of the vehicle under inspection. Therefore, the sound source detection device of the third embodiment can detect the location from which the abnormal sound is coming with high accuracy and display it in an easy-to-understand manner.

[0131] In the third embodiment described above, information about the condition of the vehicle under inspection is superimposed on a video or image of the vehicle under inspection. As a result, the sound source detection device of the third embodiment can detect with high accuracy the locations from which abnormal sounds are being generated in relation to the vehicle condition, and can display these locations in an easy-to-understand manner.

[0132] A sound source detection device according to a fourth embodiment of the present invention will be described with reference to Fig. 13. Fig. 13 is an explanatory diagram showing an outline of the system configuration of the sound source detection device according to the fourth embodiment.

[0133] The sound source detection device of the fourth embodiment has substantially the same configuration as the sound source detection device of the first embodiment described above, but further includes an oscillator 120, speakers 121a and 121b, and an LED 122 in order to check the accuracy of the sound field measurement and to calibrate the measurement equipment.

[0134] The sound source detection device of the fourth embodiment generates a reference sound on the vehicle under inspection, which is the measurement object, in order to check the accuracy of the device and to calibrate the device. Can.

[0135] The oscillator 120 generates sound of a predetermined frequency and sound pressure, and may also include an amplifier so that sound can be output from the speakers 121a and 121b.

[0136] The speakers 121a and 121b are installed on the vehicle under inspection, which is the object to be measured, to confirm the accuracy and calibrate the device, and generate reference sounds. The sound source detection device of the fourth embodiment has two speakers 121a and 121b, but in order to estimate the sound source position, it is necessary to have at least two or more speakers so that sounds can be generated from multiple points simultaneously. Note that, when simultaneously measuring in the three axial directions of up, down, left, right, and depth, it is sufficient to install six or more speakers that generate reference sounds.

[0137] The LED (light emitting diode) 122 can record the frequency and sound pressure of the sound from the speaker in both the image acquisition unit 30 and the infrared camera (air condition measurement unit 40), and also collects and checks information from the oscillator 120. The LED 122 is a light source that changes intensity and color according to the physical properties of the reference sound source. Note that the sound source detection device of the fourth embodiment may use a lighting fixture whose brightness and color can be changed instead of the LED 122.

[0138] The calibration of the sound source detection device of the fourth embodiment is performed in a booth such as an anechoic chamber where the temperature and airflow are controlled so that no changes occur in the sound field. If such an environment cannot be prepared, walls made of sound-absorbing material may be erected to prevent influence from the surroundings.

[0139] The speakers 121a and 121b emit sounds for a predetermined duration in response to a signal of a predetermined frequency and sound pressure sent from the oscillator 120. In other words, the speakers 121a and 121b are multiple reference sound sources that emit reference sounds.

[0140] This sound is simultaneously captured by microphones 10a and 10b that make up sound capture unit 10, and the time difference between them is measured to estimate the sound source position, and it is confirmed whether it matches the position of speaker 121a or speaker 121b. If there is a positional difference, the device can be adjusted or a correction coefficient can be added to address the problem.

[0141] Since the characteristics of measuring instruments vary depending on the frequency of the sound, the above measurements are carried out while changing the frequency of the oscillator 120, and adjustments are made each time.

[0142] After the calibration is completed as described above, the accuracy (performance) of the sound field measurement is confirmed in an actual measurement environment. Here, a case where an infrared camera (air condition measuring unit 40) is used will be described.

[0143] As in the calibration, the oscillator 120 generates a sound with a predetermined frequency, sound pressure, and duration, and the sound is emitted from the speakers 121a and 121b.

[0144] This sound is measured by microphones 10a and 10b of sound acquisition unit 10, and temperature is measured by an infrared camera (air condition measurement unit 40) for measuring the sound field state at that time. Furthermore, an image is taken by a camera (image acquisition unit 30) for superimposed display.

[0145] In order to visually record the sound state on the video, a display such as LED122 is used to project the frequency, sound pressure, etc. onto the camera image using the color and brightness of LED122, text display, etc.

[0146] Then, the sound source position actually corrected as obtained in the first to third embodiments described above is compared with the speaker position to confirm the performance of the sound field measurement.

[0147] The sound source detection device of the fourth embodiment can achieve substantially the same effects as the sound source detection device of the first embodiment described above.

[0148] In the above-described fourth embodiment, the process of calculating and correcting the sound source position is calibrated using multiple reference sound sources that emit reference sounds and a light source that changes intensity and color according to the physical properties of the reference sound source. Therefore, the sound source detection device of the fourth embodiment can simply and efficiently measure the sound field state, evaluate the performance of measuring equipment, and calibrate using only sound and video, without acquiring information about the reference sound source.

[0149] While specific embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments and various modifications are possible without departing from the spirit of the present invention. The above-described embodiments relate to a sound source detection method and a sound source detection device. [Explanation of symbols]

[0150] 10...Sound acquisition section 10a...Microphone 10b...Microphone 20…Sound source position estimation unit 30...Video acquisition unit 40...Air condition measuring unit 41...Air condition measuring unit 50...Sound field state estimation unit 60...Sound source position correction section 70...synthetic image generation unit 80...Display control unit 90...Display section 100...Vehicle status acquisition unit 110...abnormal noise detection unit 120...Oscillator 121a...Speaker 121b...Speaker 122...LED

Claims

1. The sound generated by the vehicle, which is the object of measurement, is acquired and converted into an electrical signal. The sound source position is calculated from this electrical signal, measuring the air condition in the space from a position where sound generated from the measurement object is acquired to the measurement object; Estimating the sound field state of the space from the air state; A sound source detection method for calculating a deviation of the sound source position from the sound field state and correcting the sound source position, Acquire the image and temperature information of the object to be measured, a heat map of sound pressure generated based on the corrected sound source position and a heat map of temperature of the measurement object are superimposed on an image of the measurement object, and displayed; The sound pressure heat map of the sound source position shows the difference in sound pressure with contour lines. A sound source detection method characterized in that the heat map of the temperature of the measurement object is displayed in different colors to indicate differences in temperature.

2. 2. The sound source detection method according to claim 1, wherein the sound field state of the space is estimated using the propagation velocity of sound from the sound source position calculated using a calculation formula with physical quantities of air as parameters.

3. superimposing a heat map of the air condition generated by acquiring information on the air condition in the space up to the measurement object onto an image of the measurement object; The sound source detection method according to claim 1, characterized in that the sound pressure heat map and the air condition heat map superimposed on the image of the measurement object are processed and displayed so as to be easily visible.

4. The sound source detection method according to claim 3, wherein the air condition in the space from a position where the sound generated from the measurement object is acquired to the measurement object is measured in a non-contact manner.

5. 5. The sound source detection method according to claim 1, wherein the display of the heat map of the sound source position and sound pressure is corrected based on the temperature distribution or air flow in the space.

6. The sound source detection method according to any one of claims 1 to 4, wherein the higher the temperature of the space, the farther the sound source position is corrected, and the acquired sound pressure is corrected to be smaller, thereby correcting the display of the sound pressure heat map.

7. 7. The sound source detection method according to claim 2, wherein the propagation speed of the sound from the sound source position is corrected so that it becomes faster as the temperature of the sound source position increases.

8. The sound source detection method according to claim 1, characterized in that a predetermined graphic indicating temperature and arrows protruding from the graphic indicating the direction and magnitude of air flow are generated for each of the measurement object and a plurality of points around the measurement object, the color of the graphic indicates the temperature at that point, the direction of the arrow indicates the direction of the air flow at that point, and the length or thickness of the arrow indicates the magnitude of the air flow at that point, and the graphic and arrows are superimposed on an image of the measurement object.

9. The sound source detection method according to any one of claims 1 to 8, characterized in that a process for calculating a sound source position and correcting the sound source position is calibrated using a plurality of reference sound sources that emit reference sounds and a light source that changes intensity or color according to physical properties of the reference sound sources.

10. 10. The sound source detection method according to claim 1, further comprising determining whether the sound from the sound source position is normal or abnormal, and superimposing the determination result on the image of the object to be measured.

11. The sound source detection method according to any one of claims 1 to 10, wherein information about the state of the measurement object is superimposed on an image of the measurement object.

12. 12. The sound source detection method according to claim 1, wherein the image of the object to be measured is displayed on a visual display device.

13. 13. The sound source detection method according to claim 2, wherein the phase of the sound generated from the object to be measured is utilized when calculating the sound source position.

14. a sound acquisition unit that acquires sound generated from a vehicle as a measurement object and converts it into an electrical signal; a sound source position estimation unit that calculates the sound source position from the electrical signal; an air condition measuring unit that measures the air condition in the space from a position where sound generated from the measurement object is acquired to the measurement object; a sound field state estimation unit that estimates a sound field state of the space from the air state; a sound source position correction unit that calculates a deviation of the sound source position from the sound field state and corrects the sound source position; an image acquisition unit that acquires an image of the measurement object; a state measurement unit that acquires temperature information of the measurement object; a display control unit that displays a heat map of sound pressure generated based on the corrected sound source position and a heat map of the temperature of the measurement object superimposed on an image of the measurement object, A sound source detection device characterized in that a heat map of sound pressure at a sound source position displays differences in sound pressure using contour lines, and a heat map of the temperature of the object being measured displays differences in temperature using different colors.

Images