Acoustic measurement device, acoustic measurement system, acoustic measurement method, and program

Abstract

The present invention provides an acoustic measurement device comprising: an observation unit that observes sound-induced vibrations of a subject on the basis of event information indicating luminance changes in individual pixels of an image in which the subject is captured, and outputs vibration information indicating the observed vibrations; an analysis unit that analyzes the vibration direction of the subject on the basis of the vibration information, and outputs vibration direction information indicating the analyzed vibration direction; a signal generation unit that generates a continuous signal from the vibration information, taking into account the vibration direction indicated by the vibration direction information, and outputs the generated continuous signal; and a measurement unit that measures acoustics by performing signal analysis on the continuous signal.

Description

Acoustic measurement device, acoustic measurement system, acoustic measurement method and program

[0001] The present invention relates to an acoustic measurement device, an acoustic measurement system, an acoustic measurement method, and a program.

[0002] Visual microphone technology is an optical acoustic measurement technology based on image processing (see, for example, Non-Patent Document 1). Visual microphone technology is a technology that restores ambient sounds from video images.

[0003] A. Davis, M. Rubinstein, N. Wadhwa, GJ Mysore, F. Durand and WT Freeman, "The Visual Microphone: Passive Recovery of Sound from Video," 2014, [Retrieved August 10, 2023], Internet (URL: https: / / people.csail.mit.edu / mrub / papers / VisualMic_SIGGRAPH2014.pdf)

[0004] Visual microphone technology requires expensive high-speed cameras. Mid-range models of high-speed cameras have a sampling rate of approximately 10 kHz. This sampling rate is lower than the 44.1 kHz sampling rate of typical microphones. Therefore, the measurable sound range of mid-range models may be limited. Furthermore, visual microphone technology requires wavelet transforms on large amounts of image data, which requires a significant amount of computation time. Therefore, visual microphone technology has the drawback of being difficult to perform in real time.

[0005] In view of the above circumstances, an object of the present invention is to provide an acoustic measurement device, an acoustic measurement system, an acoustic measurement method, and a program that can realize high-resolution, high-speed optical acoustic measurement while keeping costs down.

[0006] One aspect of the present invention is an acoustic measurement device that includes an observation unit that observes vibrations of a subject caused by sound based on event information that indicates a change in brightness for each pixel in a video of the subject, and outputs vibration information that indicates the observed vibrations; an analysis unit that analyzes the vibration direction of the subject based on the vibration information, and outputs vibration direction information that indicates the analyzed vibration direction; a signal generation unit that generates a continuous signal from the vibration information, taking into account the vibration direction indicated by the vibration direction information, and outputs the generated continuous signal; and a measurement unit that measures acoustics by performing signal analysis on the continuous signal.

[0007] Another aspect of the present invention is an acoustic measurement system that includes an event camera that captures an image of a subject and outputs event information that indicates a change in brightness for each pixel in an image of the subject; an observation unit that observes vibrations of the subject caused by sound based on the event information and outputs vibration information that indicates the observed vibrations; an analysis unit that analyzes the vibration direction of the subject based on the vibration information and outputs vibration direction information that indicates the analyzed vibration direction; a signal generation unit that generates a continuous signal from the vibration information, taking into account the vibration direction indicated by the vibration direction information, and outputs the generated continuous signal; and a measurement unit that measures acoustics by performing signal analysis on the continuous signal.

[0008] Another aspect of the present invention is an acoustic measurement method having an observation step of observing vibrations of a subject caused by sound based on event information indicating a change in brightness for each pixel in an image of the subject, and outputting vibration information indicating the observed vibrations; an analysis step of analyzing a vibration direction of the subject based on the vibration information, and outputting vibration direction information indicating the analyzed vibration direction; a signal generation step of generating a continuous signal from the vibration information, taking into account the vibration direction indicated by the vibration direction information, and outputting the generated continuous signal; and a measurement step of measuring acoustics by performing signal analysis on the continuous signal.

[0009] Another aspect of the present invention is an acoustic measurement method having an output step in which an event camera captures an image of a subject and outputs event information indicating a change in brightness for each pixel in an image of the subject; an observation step in which an acoustic measurement device observes vibrations of the subject caused by sound based on the event information and outputs vibration information indicating the observed vibrations; an analysis step in which the acoustic measurement device analyzes the vibration direction of the subject based on the vibration information and outputs vibration direction information indicating the analyzed vibration direction; a signal generation step in which the acoustic measurement device generates a continuous signal from the vibration information, taking into account the vibration direction indicated by the vibration direction information, and outputs the generated continuous signal; and a measurement step in which the acoustic measurement device measures sound by performing signal analysis on the continuous signal.

[0010] Another aspect of the present invention is a program for causing a computer to execute the following steps: an observation step of observing vibrations of a subject caused by sound based on event information indicating a change in brightness for each pixel in a video of the subject, and outputting vibration information indicating the observed vibrations; an analysis step of analyzing the vibration direction of the subject based on the vibration information, and outputting vibration direction information indicating the analyzed vibration direction; a signal generation step of generating a continuous signal from the vibration information, taking into account the vibration direction indicated by the vibration direction information, and outputting the generated continuous signal; and a measurement step of measuring sound by performing signal analysis on the continuous signal.

[0011] The present invention makes it possible to realize high-resolution, high-speed optical acoustic measurement while keeping costs down.

[0012] Fig. 1 is a block diagram showing the functional configuration of an acoustic measurement system 1 according to a first embodiment of the present invention. Fig. 2 is a flowchart showing the operation of the acoustic measurement system 1 according to the first embodiment of the present invention. Fig. 3 is a block diagram showing the functional configuration of an acoustic measurement system 1a according to a second embodiment of the present invention. Fig. 4 is a flowchart showing the operation of the acoustic measurement system 1a according to the second embodiment of the present invention.

[0013] An acoustic measurement device, an acoustic measurement system, an acoustic measurement method, and a program according to an embodiment of the present invention will be described below.

[0014] The acoustic measurement system in each embodiment described below uses an event camera to observe vibrations of a subject caused by sound as event information. The event camera is a specialized camera that outputs the brightness change of each pixel in a video of the subject as an event. Typically, the maximum sampling rate of an event camera is 1 MHz, which provides much higher time resolution than a high-speed camera (in the mid-range). Furthermore, an event camera is characterized by excellent data efficiency because it outputs data asynchronously for each pixel. Additionally, event camera devices are less expensive than high-speed cameras.

[0015] The acoustic measurement system in each embodiment described below analyzes the vibration direction of a subject by calculating optical flow for observed event information. Optical flow is a vector representation of the movement of an object in a visual representation (generally, a time-series digital image). The acoustic measurement system generates a continuous signal based on the analyzed vibration direction and event information. The acoustic measurement system performs frequency analysis, such as a short-time Fourier transform, on the generated continuous signal to estimate the frequencies contained in the vibration and the phase of the frequency signal. The acoustic measurement system also estimates the intensity of each frequency signal by estimating the amplitude based on the optical flow calculated in the vibration direction analysis or by using the number of events occurring per unit time. The acoustic measurement system adds up the frequency signals taking into account the estimated intensity and phase.

[0016] With this configuration, the acoustic measurement system in each embodiment can measure acoustics, thereby realizing high-resolution, high-speed optical acoustic measurement while reducing costs such as device costs and maintenance costs.

[0017] In the acoustic measurement systems of the embodiments described below, components having the same functions are denoted by the same reference numerals, and repeated explanations of the functions may be omitted.

[0018] First Embodiment An acoustic measurement system 1 according to a first embodiment of the present invention will be described in detail below with reference to the drawings.

[0019] [Functional Configuration of Acoustic Measurement System] The functional configuration of the acoustic measurement system 1 will be described below. Fig. 1 is a block diagram showing the functional configuration of the acoustic measurement system 1 according to a first embodiment of the present invention. As shown in Fig. 1, the acoustic measurement system 1 includes an acoustic measurement device 10 and an event camera 20. Also, as shown in Fig. 1, the acoustic measurement device 10 includes a vibration information acquisition unit 11, a vibration direction analysis unit 12, a continuous signal generation unit 13, and a signal analysis unit 14. As shown in Fig. 1, the vibration direction analysis unit 12, the continuous signal generation unit 13, and the signal analysis unit 14 function as an acoustic analysis unit.

[0020] The event camera 20 captures an image of a subject, generates event information indicating a change in luminance for each pixel in the captured image of the subject, and outputs the generated event information to the sound measuring device 10.

[0021] The event information acquired by the acoustic measuring device 10 is not limited to event information generated by capturing images with the event camera 20, but may also be event information generated by a simulator that converts general moving images, etc. into event information.

[0022] The vibration information acquisition unit 11 of the acoustic measurement device 10 acquires event information output from the event camera 20. The vibration information acquisition unit 11 observes vibrations of the subject caused by sound based on the acquired event information. The vibration information acquisition unit 11 outputs information indicating the observed vibrations (hereinafter referred to as "vibration information") to the vibration direction analysis unit 12.

[0023] The vibration information acquisition unit 11 generates vibration information by, for example, extracting (extracting) event information corresponding to vibrations caused by sound from the acquired event information. Any method can be used to extract (extract) event information corresponding to vibrations caused by sound. Note that the vibration information acquisition unit 11 may be configured to generate vibration information by processing the acquired event information. In this case, any method can be used to process the event information.

[0024] In an environment where only vibrations caused by sound can be directly observed by the event camera 20, the vibration information acquisition unit 11 is not required. In this case, the event information output by the event camera 20 is information equivalent to vibration information. An environment where only vibrations caused by sound can be observed is, for example, an environment where only a subject affected by sound (which vibrates due to sound) is photographed in a stationary state.

[0025] The vibration direction analysis unit 12 acquires the vibration information output from the vibration information acquisition unit 11. The vibration direction analysis unit 12 analyzes the vibration direction of the subject by calculating an optical flow for the acquired vibration information. While general conventional technology can be used to calculate the optical flow, further improvement in measurement accuracy can be expected by calculating the optical flow assuming vibration. The vibration direction analysis unit 12 outputs information indicating the analyzed vibration direction of the subject (hereinafter referred to as "vibration direction information") and the vibration information to the continuous signal generation unit 13.

[0026] The continuous signal generation unit 13 acquires the vibration direction information and the vibration information output from the vibration direction analysis unit 12. The continuous signal generation unit 13 generates a continuous signal from the acquired vibration information, taking into account the vibration direction based on the acquired vibration direction information.

[0027] More specifically, the continuous signal generation unit 13 generates a pulse signal in response to the occurrence of an event at a certain time in a single pixel. The positive and negative polarities of the generated pulse signals are determined according to the direction of vibration. The continuous signal generation unit 13 generates a continuous signal by adding up the generated pulse signals in the time direction. The continuous signal generation unit 13 outputs the generated continuous signal to the signal analysis unit 14.

[0028] The signal analysis unit 14 acquires the continuous signal output from the continuous signal generation unit 13. The signal analysis unit 14 measures the sound by performing signal analysis on the acquired continuous signal.

[0029] More specifically, the signal analysis unit 14 performs frequency analysis such as a short-time Fourier transform on the acquired continuous signal. The signal analysis unit 14 estimates the frequencies contained in the vibration and the phases of the frequency signals. The signal analysis unit 14 also estimates the intensity of each frequency signal by calculating the amplitude based on the optical flow calculated by the vibration direction analysis unit 12. The signal analysis unit 14 measures the sound by adding up the frequency signals in consideration of the estimated intensity and phase.

[0030] The signal analysis unit 14 outputs information indicating the measured sound (hereinafter referred to as "measured sound information") to an external device. The external device here is, for example, a storage device that stores the measured sound information, or a speaker that outputs sound based on the measured sound information.

[0031] [Operation of Acoustic Measurement System] An example of the operation of the acoustic measurement system 1 will now be described. Fig. 2 is a flowchart showing the operation of the acoustic measurement system 1 according to the first embodiment of the present invention. The operation of the acoustic measurement system 1 shown in the flowchart of Fig. 2 is started, for example, when the event camera 20 receives an instruction to photograph a subject.

[0032] The event camera 20 captures an image of a subject (step S01). The event camera 20 generates event information indicating a change in luminance for each pixel in the captured image of the subject, and outputs the generated event information to the sound measuring device 10 (step S02).

[0033] The vibration information acquisition unit 11 of the acoustic measurement device 10 observes vibrations of the object caused by the sound based on the acquired event information (step S03). The vibration information acquisition unit 11 outputs vibration information indicating the observed vibrations to the vibration direction analysis unit 12.

[0034] The vibration direction analysis unit 12 calculates an optical flow for the acquired vibration information to analyze the vibration direction of the subject (step S04). While general conventional techniques can be used to calculate the optical flow, further improvement in measurement accuracy can be expected by calculating the optical flow assuming vibration. The vibration direction analysis unit 12 outputs information indicating the analyzed vibration direction of the subject (hereinafter referred to as "vibration direction information") and the vibration information to the continuous signal generation unit 13.

[0035] The continuous signal generator 13 generates a pulse signal in response to an event occurring at a certain time in a single pixel, taking into account the vibration direction (step S05). The continuous signal generator 13 generates a continuous signal by adding up the generated pulse signals in the time direction (step S06). The continuous signal generator 13 outputs the generated continuous signal to the signal analyzer 14.

[0036] The signal analysis unit 14 performs frequency analysis such as a short-time Fourier transform on the acquired continuous signal. The signal analysis unit 14 estimates the frequencies contained in the vibration and the phases of the frequency signals (step S07). The signal analysis unit 14 also estimates the strength of each frequency signal by calculating the amplitude based on the optical flow calculated by the vibration direction analysis unit 12 (step S08). The signal analysis unit 14 measures the sound by adding up the frequency signals taking into account the estimated strength and phase (step S09).

[0037] The signal analysis unit 14 outputs the measured acoustic information to an external device (step S10). This completes the operation of the acoustic measurement system 1 shown in the flowchart of FIG.

[0038] As described above, the sound measurement system 1 in the first embodiment generates event information using the event camera 20 and observes vibrations of a subject caused by sound. The sound measurement system 1 analyzes the vibration direction of the subject based on the vibration information. The sound measurement system 1 generates a continuous signal from the vibration information, taking the vibration direction into consideration. The sound measurement system 1 measures sound by performing signal analysis on the generated continuous signal.

[0039] As described above, the acoustic measurement system 1 in the first embodiment uses the event camera 20, and therefore does not require an expensive high-speed camera, making it possible to reduce costs such as device costs and maintenance costs.

[0040] Furthermore, because the sound measurement system 1 in the first embodiment uses the event camera 20, it can ensure a maximum sampling rate of 1 MHz, achieving extremely high time resolution compared to a high-speed camera. This value is sufficient compared to the 44.1 kHz sampling rate of a typical sound signal. Furthermore, the event camera 20 provides excellent data efficiency because it asynchronously outputs signals from each pixel. As a result, the sound measurement system 1 in the first embodiment is not limited in the range of sounds that can be measured.

[0041] Furthermore, the acoustic measurement system 1 according to the first embodiment does not need to perform processes requiring a large amount of time calculations, such as wavelet transforms, on large amounts of image data, thereby suppressing an increase in the amount of time calculations. As a result, the acoustic measurement system 1 according to the first embodiment also facilitates real-time measurement.

[0042] With this configuration, the acoustic measurement system 1 according to the first embodiment of the present invention can achieve high-resolution and high-speed optical acoustic measurement while keeping costs down.

[0043] Second Embodiment An acoustic measurement system 1a according to a second embodiment of the present invention will be described in detail below with reference to the drawings.

[0044] [Functional Configuration of Acoustic Measurement System] The functional configuration of the acoustic measurement system 1a will be described below. Fig. 3 is a block diagram showing the functional configuration of the acoustic measurement system 1a according to a second embodiment of the present invention. As shown in Fig. 3, the acoustic measurement system 1a includes an acoustic measurement device 10a and an event camera 20. Also, as shown in Fig. 3, the acoustic measurement device 10a includes a vibration information acquisition unit 11, a vibration direction analysis unit 12, a continuous signal generation unit 13a, and a signal analysis unit 14a. As shown in Fig. 3, the vibration direction analysis unit 12, the continuous signal generation unit 13a, and the signal analysis unit 14a function as an acoustic analysis unit.

[0045] The configuration of the event camera 20 shown in Fig. 3 is similar to the configuration of the event camera 20 of the acoustic measurement system 1 in the first embodiment shown in Fig. 1 described above, and therefore a description thereof will be omitted. Also, the configurations of the vibration information acquisition unit 11 and the vibration direction analysis unit 12 of the acoustic measurement device 10 shown in Fig. 3 are similar to the configurations of the vibration information acquisition unit 11 and the vibration direction analysis unit 12 of the acoustic measurement device 10 in the acoustic measurement system 1 in the first embodiment shown in Fig. 1 described above, and therefore a description thereof will be omitted.

[0046] The continuous signal generating unit 13a acquires the vibration direction information and the vibration information output from the vibration direction analyzing unit 12. The continuous signal generating unit 13a generates a continuous signal from the acquired vibration information, taking into account the vibration direction based on the acquired vibration direction information.

[0047] More specifically, the continuous signal generator 13a calculates the number of occurrences of events for each section in a single pixel, taking into account the vibration direction, in accordance with the sampling rate of the sound being measured. The continuous signal generator 13a generates a continuous signal by integrating the number of occurrences of events calculated for each section across all sections. Note that calculations can also be performed for multiple pixels in order to increase the number of samples. The continuous signal generator 13a outputs the generated continuous signal to the signal analyzer 14a.

[0048] The signal analysis unit 14a acquires the continuous signal output from the continuous signal generation unit 13a, and measures the sound by performing signal analysis on the acquired continuous signal.

[0049] More specifically, the signal analysis unit 14a performs frequency analysis such as a short-time Fourier transform on the acquired continuous signal. The signal analysis unit 14a estimates the frequency contained in the vibration and the phase of the frequency signal. Furthermore, based on the assumption that a large amplitude indicates a large number of event occurrences, the signal analysis unit 14a estimates the intensity obtained from the generated continuous signal as the intensity of the frequency signal. The signal analysis unit 14a measures the sound by adding up the frequency signals taking into account the estimated intensity and phase.

[0050] The signal analyzer 14a outputs measured acoustic information indicating the measured acoustic to an external device. The external device here is, for example, a storage device that stores the measured acoustic information, or a speaker that outputs sound based on the measured acoustic information.

[0051] [Operation of Acoustic Measurement System] An example of the operation of the acoustic measurement system 1a will be described below. Fig. 4 is a flowchart showing the operation of the acoustic measurement system 1a according to the second embodiment of the present invention. The operation of the acoustic measurement system 1a shown in the flowchart of Fig. 4 is started, for example, when the event camera 20 receives an instruction to photograph a subject.

[0052] The operation of the acoustic measurement system 1a from step S11 to step S14 shown in FIG. 4 is similar to the operation of the acoustic measurement system 1 in the first embodiment from step S01 to step S04 shown in FIG. 2 described above, and therefore will not be described again.

[0053] The continuous signal generator 13a calculates the number of occurrences of events for each section in accordance with the sampling rate of the sound being measured, taking into account the vibration direction, for a single pixel (step S15). The continuous signal generator 13a generates a continuous signal by integrating the number of occurrences of events calculated for each section across all sections (step S16). The continuous signal generator 13a outputs the generated continuous signal to the signal analyzer 14a.

[0054] The signal analysis unit 14a performs frequency analysis such as a short-time Fourier transform on the acquired continuous signal. The signal analysis unit 14a estimates the frequency included in the vibration and the phase of the frequency signal (step S17). The signal analysis unit 14a also estimates the intensity obtained from the generated continuous signal as the intensity of the frequency signal (step S18). The signal analysis unit 14a measures the sound by adding up the frequency signals taking into account the estimated intensity and phase (step S19).

[0055] The signal analysis unit 14a outputs the measured acoustic information to an external device (step S20). This completes the operation of the acoustic measurement system 1a shown in the flowchart of FIG.

[0056] As described above, the sound measurement system 1a in the second embodiment generates event information using the event camera 20 and observes vibrations of a subject caused by sound. The sound measurement system 1a analyzes the vibration direction of the subject based on the vibration information. The sound measurement system 1a generates a continuous signal from the vibration information, taking the vibration direction into consideration. The sound measurement system 1a measures sound by performing signal analysis on the generated continuous signal.

[0057] As described above, the acoustic measurement system 1a in the second embodiment uses the event camera 20, and therefore does not require an expensive high-speed camera, making it possible to reduce costs such as device costs and maintenance costs.

[0058] Furthermore, since the sound measurement system 1a in the second embodiment uses the event camera 20, it can ensure a maximum sampling rate of 1 MHz, achieving extremely high time resolution compared to a high-speed camera. This value is sufficient compared to the 44.1 kHz sampling rate of a typical sound signal. Furthermore, the event camera 20 provides asynchronous output for each pixel, resulting in excellent data efficiency. As a result, the sound measurement system 1a in the second embodiment is not limited in the range of sounds that can be measured.

[0059] Furthermore, the acoustic measurement system 1a according to the second embodiment does not need to perform processes requiring a large amount of time calculations, such as wavelet transforms, on large amounts of image data, thereby suppressing an increase in the amount of time calculations required. As a result, the acoustic measurement system 1a according to the second embodiment also facilitates real-time measurement.

[0060] With this configuration, the acoustic measurement system 1a according to the second embodiment of the present invention can achieve high-resolution and high-speed optical acoustic measurement while keeping costs down.

[0061] According to the above-described embodiment, the acoustic measurement device includes an observation unit, an analysis unit, a signal generation unit, and a measurement unit. For example, the acoustic measurement device is the acoustic measurement device 10 or 10a in the embodiment, the observation unit is the vibration information acquisition unit 11 in the embodiment, the analysis unit is the vibration direction analysis unit 12 in the embodiment, the signal generation unit is the continuous signal generation unit 13 or 13a in the embodiment, and the measurement unit is the signal analysis unit 14 or 14a in the embodiment.

[0062] The observation unit observes vibrations of the subject caused by sound based on event information indicating brightness changes for each pixel in a video of the subject, and outputs vibration information indicating the observed vibrations. The analysis unit analyzes the vibration direction of the subject based on the vibration information, and outputs vibration direction information indicating the analyzed vibration direction. The signal generation unit generates a continuous signal from the vibration information, taking into account the vibration direction indicated by the vibration direction information, and outputs the generated continuous signal. The measurement unit measures sound by performing signal analysis on the continuous signal.

[0063] In the above acoustic measuring device, the analysis unit may analyze the vibration direction by calculating an optical flow for the vibration information.

[0064] In the above-described acoustic measuring device, the signal generating unit may generate a pulse signal in response to the occurrence of an event at a certain time in a single pixel, determine whether the pulse signal is positive or negative depending on the vibration direction, and generate a continuous signal by adding up the generated pulse signals in the time direction.

[0065] In the above-described acoustic measuring device, the signal generating unit may calculate the number of occurrences of events for each section in accordance with the sampling rate of the sound to be measured while taking into account the vibration direction for a single pixel, and may generate a continuous signal by integrating the number of occurrences of events for each section across all sections.

[0066] Furthermore, according to the above-described embodiment, the acoustic measurement system includes an event camera, an observation unit, an analysis unit, a signal generation unit, and a measurement unit. For example, the acoustic measurement system is the acoustic measurement system 1 or 1a in the embodiment, the event camera is the event camera 20 in the embodiment, the observation unit is the vibration information acquisition unit 11 in the embodiment, the analysis unit is the vibration direction analysis unit 12 in the embodiment, the signal generation unit is the continuous signal generation unit 13 or 13a in the embodiment, and the measurement unit is the signal analysis unit 14 or 14a in the embodiment.

[0067] The event camera captures an image of a subject and outputs event information indicating brightness changes for each pixel in the image of the subject. The observation unit observes vibrations of the subject caused by sound based on the event information and outputs vibration information indicating the observed vibrations. The analysis unit analyzes the vibration direction of the subject based on the vibration information and outputs vibration direction information indicating the analyzed vibration direction. The signal generation unit generates a continuous signal from the vibration information, taking into account the vibration direction indicated by the vibration direction information, and outputs the generated continuous signal. The measurement unit measures sound by performing signal analysis on the continuous signal.

[0068] Part or all of the configuration of the acoustic measuring device 10, 10a in the above-described embodiments may be implemented by a computer. In this case, a program for implementing the functions may be recorded on a computer-readable recording medium, and the program may be loaded into a computer system and executed. Note that the term "computer system" as used herein includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. Furthermore, the term "computer-readable recording medium" may also include media that dynamically store programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or telephone lines, or media that store programs for a fixed period of time, such as volatile memory within a computer system that serves as a server or client. The program may be for implementing part of the functions described above, or may be capable of implementing the functions in combination with a program already stored in the computer system, or may be implemented using a programmable logic device such as an FPGA (Field Programmable Gate Array).

[0069] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the present invention that do not deviate from the gist of the present invention.

[0070] 1, 1a... Acoustic measurement system, 10, 10a... Acoustic measurement device, 11... Vibration information acquisition unit, 12... Vibration direction analysis unit, 13, 13a... Continuous signal generation unit, 14, 14a... Signal analysis unit, 20... Event camera

Claims

1. An acoustic measurement device comprising: an observation unit that observes vibrations of a subject caused by sound based on event information indicating brightness changes for each pixel in an image of the subject, and outputs vibration information indicating the observed vibrations; an analysis unit that analyzes a vibration direction of the subject based on the vibration information, and outputs vibration direction information indicating the analyzed vibration direction; a signal generation unit that generates a continuous signal from the vibration information, taking into account the vibration direction indicated by the vibration direction information, and outputs the generated continuous signal; and a measurement unit that measures acoustics by performing signal analysis on the continuous signal.

2. The acoustic measuring device according to claim 1, wherein the analysis unit analyzes the vibration direction by calculating an optical flow for the vibration information.

3. The acoustic measuring device according to claim 1 or 2, wherein the signal generating unit generates a pulse signal in response to the occurrence of an event at a certain time in a single pixel, determines whether the pulse signal is positive or negative in response to the vibration direction, and generates the continuous signal by adding up the generated pulse signals in the time direction.

4. The acoustic measuring device according to claim 1 or 2, wherein the signal generating unit calculates the number of occurrences of events for each section in accordance with a sampling rate of the acoustic being measured while taking into account the vibration direction for a single pixel, and generates the continuous signal by integrating the number of occurrences of the events for each section over the entire section.

5. An acoustic measurement system comprising: an event camera that captures an image of a subject and outputs event information indicating a change in brightness for each pixel in an image of the subject; an observation unit that observes vibrations of the subject caused by sound based on the event information and outputs vibration information indicating the observed vibrations; an analysis unit that analyzes a vibration direction of the subject based on the vibration information and outputs vibration direction information indicating the analyzed vibration direction; a signal generation unit that generates a continuous signal from the vibration information taking into account the vibration direction indicated by the vibration direction information and outputs the generated continuous signal; and a measurement unit that measures acoustics by performing signal analysis on the continuous signal.

6. An acoustic measurement method comprising: an observation step of observing vibrations of a subject caused by sound based on event information indicating a luminance change for each pixel in an image of the subject, and outputting vibration information indicating the observed vibrations; an analysis step of analyzing a vibration direction of the subject based on the vibration information, and outputting vibration direction information indicating the analyzed vibration direction; a signal generation step of generating a continuous signal from the vibration information, taking into account the vibration direction indicated by the vibration direction information, and outputting the generated continuous signal; and a measurement step of measuring acoustics by performing signal analysis on the continuous signal.

7. An acoustic measurement method comprising: an output step in which an event camera captures an image of an object and outputs event information indicating a luminance change for each pixel in an image of the object; an observation step in which an acoustic measurement device observes vibrations of the object caused by sound based on the event information and outputs vibration information indicating the observed vibrations; an analysis step in which the acoustic measurement device analyzes the vibration direction of the object based on the vibration information and outputs vibration direction information indicating the analyzed vibration direction; a signal generation step in which the acoustic measurement device generates a continuous signal from the vibration information, taking into account the vibration direction indicated by the vibration direction information, and outputs the generated continuous signal; and a measurement step in which the acoustic measurement device measures sound by performing signal analysis on the continuous signal.

8. A program for causing a computer to execute the following steps: an observation step of observing vibrations of a subject caused by sound based on event information indicating brightness changes for each pixel in an image of the subject, and outputting vibration information indicating the observed vibrations; an analysis step of analyzing the vibration direction of the subject based on the vibration information, and outputting vibration direction information indicating the analyzed vibration direction; a signal generation step of generating a continuous signal from the vibration information, taking into account the vibration direction indicated by the vibration direction information, and outputting the generated continuous signal; and a measurement step of measuring sound by performing signal analysis on the continuous signal.