Noise reduction method, noise reduction device, program, and noise reduction system

Abstract

Provided are a noise reduction method, a noise reduction device, and a program for outputting an antiphase sound without delay without using a reference microphone. The noise reduction method: acquires optical information obtained by imaging vibration caused by noise; estimates, from the optical information, noise in an object to be imaged; and generates a cancel sound from the estimated value of the noise.

Description

Noise reduction method, noise reduction device, program, and noise reduction system

[0001] The present invention relates to a noise reduction technology that reduces target noise by using cancellation with an antiphase sound (canceling sound) called Active Noise Control (ANC).

[0002] Non-Patent Document 1 is known as prior art for ANC.

[0003] There are two main types of ANC control methods. The first is called feedforward control, which calculates the corresponding antiphase sound by directly measuring the noise signal with a reference microphone. The second is called feedback control, which calculates the corresponding antiphase sound by estimating the noise signal from the error signal observed with an error microphone without using a reference microphone.

[0004] Masaharu Nishimura, Yoshi Kajikawa, "2 Group (Image, Sound, Language) 6 Volumes (Acoustic Signal Processing) Chapter 6 Active Noise Control," [Retrieved May 9, 2024], Internet<https: / / www.ieice-hbkb.org / files / ad_base / view_pdf.html?p= / files / 02 / 02gun_06hen_06.pdf> .

[0005] Feedforward control uses a reference microphone to measure noise sources far from the control point in advance, and the time it takes for sound waves to propagate can be used to calculate the antiphase sound. However, due to the cost of installing the reference microphone, or if there is a noise source other than the installation location, it may not be possible to effectively reduce noise.

[0006] On the other hand, feedback control uses the error signal observed at the control point (the location where the error microphone is installed) to calculate the antiphase sound, so there is a delay in the output of the antiphase sound due to the amount of calculation processing.

[0007] An object of the present invention is to provide a noise reduction method, a noise reduction device, and a program that output an antiphase sound without delay without using a reference microphone.

[0008] In order to solve the above problem, according to one aspect of the present invention, a noise reduction method acquires optical information obtained by photographing vibrations caused by noise, estimates noise in the photographed object from the optical information, and generates a cancellation sound from the estimated noise value.

[0009] According to the present invention, it is possible to output antiphase sound without delay without using a reference microphone.

[0010] Fig. 1 is a functional block diagram of a noise reduction system according to a first embodiment. Fig. 2 is a diagram showing an example of a processing flow of the noise reduction system according to the first embodiment. Fig. 3 is a functional block diagram of a noise reduction device according to the first embodiment. Fig. 4 is a diagram showing an example of the functional configuration of a computer.

[0011] Hereinafter, embodiments of the present invention will be described in detail. Components having the same functions will be assigned the same numbers, and duplicate explanations will be omitted. In the following description, unless otherwise specified, processing performed on each element of a vector or matrix will be applied to all elements of that vector or matrix.

[0012] <Key Points of the First Embodiment> In this embodiment, optical acoustic measurement technology (see References 1 and 2) is used to observe noise sources. (Reference 1) Kohei Yatabe, Kenji Ishikawa, Risako Tanigawa, Yasuhiro Oikawa, "Optical Acoustic Measurement," Institute of Electronics, Information and Communication Engineers, Fundamentals Review, Vol. 12, No. 4, pp. 259-268, 2019. (Reference 2) Abe Davis et al., "The Visual Microphone: Passive Recovery of Sound from Video," [Retrieved May 9, 2024], Internet<https: / / people.csail.mit.edu / mrub / papers / VisualMic_SIGGRAPH2014.pdf> Optical acoustic measurement is a measurement method that uses optical information, so it is possible to obtain acoustic information earlier than the propagation of sound waves. In addition, optical measuring instruments such as image sensors that measure optical information can also obtain spatial information.

[0013] Therefore, by using optical acoustic measurement technology to estimate the propagation signal to the control point before the sound wave propagates, the time it takes for the sound wave to propagate can be used to calculate the antiphase sound, just as in the case of feedforward control. Furthermore, since a reference microphone is not used, there is no cost for installing one. Furthermore, by using spatial information acquired by optical measuring instruments such as image sensors, the primary path can be flexibly set, which solves the problem of ineffective noise reduction when there is a noise source other than the location where the reference microphone is installed.

[0014] First Embodiment FIG. 1 is a functional block diagram of a noise reduction system according to a first embodiment, and FIG. 2 shows the processing flow thereof.

[0015] The noise reduction system includes an optical measurement device 91 , an error sensor 93 , a secondary sound source 95 , and a noise reduction device 100 .

[0016] <Optical Measuring Instrument 91> The optical measuring instrument 91 captures images of vibrations caused by noise, obtains optical information (S91), and outputs the information.

[0017] The optical measuring device 91 is a high-speed camera, an event camera, a rolling shutter camera, etc., and captures images of the noise source or the surrounding vibrating body (hereinafter also referred to as the photographed object). If it is a high-speed camera or a rolling shutter camera, time-series image data is used as optical information, and if it is an event camera, changes in brightness of the photographed object are detected and event data such as time, brightness change, location, etc. are obtained as optical information.

[0018] <Noise Reduction Device 100> FIG. 3 shows a functional block diagram of the noise reduction device 100. As shown in FIG.

[0019] Noise reduction device 100 receives optical information and error signal e(n) as input, calculates and outputs signal Y(n) (hereinafter also referred to as antiphase sound signal) that is an estimate of the antiphase sound of the noise, where n is an index indicating the sampling time.

[0020] The noise reduction device 100 includes an optical acoustic measurement unit 110 , a depth estimation unit 120 , a primary path model generation unit 130 , a secondary path model application unit 140 , a coefficient update unit 150 , and a noise control filter unit 160 .

[0021] Each part will be explained below.

[0022] <Optical Acoustic Measuring Unit 110> The optical acoustic measuring unit 110 receives optical information as input, estimates the noise in the object to be photographed from the optical information (S110), and calculates an estimated value X(n)=[x(n) x(n-1) ... x(n-i+1) ... x(n-N+1)] T where T denotes transposition.

[0023] For example, the optical acoustic measurement unit 110 estimates noise by performing signal processing such as wavelet transform and optical flow estimation on time-series image data or optical information such as event data on time, brightness change, location, etc. For example, the method described in Reference 2 can be considered as an existing method using a high-speed camera as the optical measuring instrument 91.

[0024] <Depth Estimation Unit 120> The depth estimation unit 120 receives optical information as input, estimates the depth of the object to be photographed from the acquired optical information (S120), and outputs the estimated value. Note that depth refers to the distance from the photographing device, such as a camera, of the optical measuring device 91 to the object to be photographed. The depth estimation unit 120 can calculate the depth using an algorithm corresponding to the data format of the optical information output by the optical measuring device 91. (Example of Combination of Optical Acoustic Measurement and Depth Estimation) When a high-speed camera or a rolling shutter camera is used as the optical measuring device 91, the optical measuring device 91 outputs time-series image data (optical information), and the optical acoustic measurement unit 110 detects vibrations from pixel variations and color fluctuations between frames to acquire sound signals. Depth can also be estimated using existing distance estimation technology. For example, methods using a monocular depth estimation machine learning model or a distance image sensor are conceivable.

[0025] For example, if an event camera is used as the optical measuring device 91, the optical measuring device 91 outputs event data such as time, brightness change, and location, and the optical acoustic measuring unit 110 detects vibrations from the brightness change and acquires sound signals. Depth can be estimated using existing distance estimation techniques, such as stereo matching using multiple cameras or motion compensation.

[0026] For example, if a laser Doppler vibrometer (LDV) or a polarized high-speed camera is used as the optical measuring device 91, the optical measuring device 91 outputs an electrical signal (optical information) representing the vibration velocity of the object being imaged, and the optical acoustic measuring unit 110 processes the signal obtained by A / D converting and sampling the electrical signal, regarding the signal as a sound signal. Note that the air may be regarded as a vibrating body surrounding the noise source, and sound waves emitted into the air may be calculated based on the phase change of diffracted light caused by changes in the refractive index of the air. Depth can be estimated using existing distance estimation technology. For example, a method using a distance image sensor may be considered.

[0027] The object to be photographed may be identified manually, or may be automatically identified by the system from optical information or estimated noise values ​​using an existing noise identification method.

[0028] <Primary Path Model Generator 130> The primary path model generator 130 receives an estimated depth value as input, uses the estimated depth value to estimate a transfer function from the object to be photographed to the error sensor 93 (S130), and outputs the estimated transfer function as a primary path model. For example, the position of the error sensor 93 and the position and orientation of the optical measuring instrument 91 are provided to the primary path model generator 130 in advance. The direction of the object to be photographed relative to the optical measuring instrument 91 is identified from the orientation of the optical measuring instrument 91, and the position of the object to be photographed is estimated from the identified direction and the estimated depth value. The distance from the object to the error sensor 93 is calculated from the estimated value of the position of the object to be photographed and the position of the error sensor 93, and the transfer function is estimated from the calculated distance. Because the position of the object to be photographed can be sequentially estimated from the optical information and the estimated depth value, it is possible to sequentially generate an appropriate primary path model even if the position of the object to be photographed changes. Furthermore, when time-series image data (optical information) is used, even if the image data contains multiple photographic objects, the transfer function from each photographic object to the error sensor 93 can be estimated, making it possible to deal with multiple noise sources using a single device.

[0029] <Secondary path model application unit 140> The secondary path model application unit 140 receives as input the noise estimate X(n) obtained by the optical acoustic measurement unit 110, and filters it using a secondary path model including a transfer function from the secondary sound source 95 to the error sensor 93 according to the following equation (S140), to obtain a filtered reference signal R(n)=[r(n) r(n-1) ... r(n-i+1) ... r(n-N+1)] T Calculate and output R(n)=C T X(n) where C=[c0 c1 ... c i ...c N-1 ] Tis a filter coefficient vector of the secondary path model. It is assumed that the positions of the secondary sound source 95 and the error sensor 93 are predetermined, and C is calculated in advance from the distance from the secondary sound source 95 to the error sensor 93 before calculating the filtered reference signal R(n). Note that other conventional techniques (e.g., Non-Patent Document 1) or methods may be used to estimate and apply the secondary path model. Furthermore, other conventional techniques (e.g., Non-Patent Document 1) or methods may also be used for the subsequent processes S150, S160, S95, and S93.

[0030] <Coefficient update unit 150> The coefficient update unit 150 receives the filtered reference signal R(n) and the error signal e(n) as input, and calculates the filter coefficient vector W(n) of the noise control filter using the following equation: W(n)=[w0(n) w1(n) ... w i (n) ... w N-1 (n)] T (S150), and outputs the updated filter coefficient vector W(n+1). W(n+1)=W(n)+μR(n)E(n) E(n)=[e(n) e(n-1) ... e(n-i+1) ... e(n-N+1)] T where μ is a step size parameter, which adjusts the convergence speed and estimation accuracy of the adaptive operation.

[0031] <Noise control filter unit 160> The noise control filter unit 160 receives the primary path model, the updated filter coefficient vector W(n+1), and the noise estimate X(n) as input, and performs filtering using the primary path model generated by the primary path model generation unit 130 or the filter coefficient vector W(n+1) updated by the coefficient update unit 150 according to the following equation (S160), to obtain an antiphase sound signal Y(n) for the noise: Y(n)=[y(n) y(n-1) ... y(n-i+1) ... y(n-N+1)] T Calculate and output Y(n)=W(n). T X(n) If there is a change in the primary path (the path from the object to be photographed to the error sensor 93), the primary path model generated by the primary path model generation unit 130 is used, and if there is no change in the primary path model, the filter coefficient vector W(n+1) updated by the coefficient update unit 150 is used.

[0032] <Secondary Sound Source 95> The secondary sound source 95 is, for example, a speaker, and receives the antiphase sound signal Y(n) as input and reproduces it (S95).

[0033] <Error Sensor 93> The error sensor 93 is made up of, for example, a microphone, and observes and outputs the error signal e(n) of the control point (S93).

[0034] With this configuration, the noise and the antiphase sound cancel each other out at the control point. <Effects> With the above configuration, the problems of the cost of installing a reference microphone in feedforward control and the inability to effectively reduce noise when there is a noise source other than the location where the reference microphone is installed do not occur. Furthermore, compared to feedback control, the antiphase sound can be output without delay. Furthermore, when time-series image data is used as optical information, it is possible to deal with multiple noise sources using a single device. [Variation 1] If the distance from the secondary sound source 95 to the error sensor 93 is short and the secondary path does not need to be considered, the secondary path model application unit 140 does not need to be provided. In this case, the coefficient update unit 150 receives the noise estimate X(n) and the error signal e(n) as input, and calculates the filter coefficient vector W(n) of the noise control filter using the following equation: W(n)=[w0(n) w1(n) ... w i (n) ... w N-1 (n)] Tand outputs the updated filter coefficient vector W(n+1). W(n+1)=W(n)+μX(n)E(n) [Variation 2] In this embodiment, the depth estimation process (S120) and the primary path model generation process (S130) are performed on the assumption that the position of the object to be photographed may change. However, the position of the object to be photographed may be fixed. In this case, the depth estimation process (S120) and the primary path model generation process (S130) can be omitted, and a transfer function from the object to be photographed, which is in a fixed position, to the error sensor 93 can be estimated in advance, and the estimated transfer function can be set in the noise control filter unit 160. This configuration can achieve the same effects as in the first embodiment. [Other Variations] Furthermore, a device (terminal) for using the device, system, or method of the present invention via a network (telecommunications line) may also be provided. The "device (terminal) for use" may be equipped with functions (e.g., control function, decoding function, restoration function, input / output function, etc.) necessary to achieve the effects achieved by implementing the device, system, or method of the present invention. A configuration including a device (terminal) for using the device of the present invention or the method of the present invention via a network (telecommunication line) is also called a noise reduction system.

[0035] The present invention is not limited to the above-described embodiments and modifications. For example, the various processes described above may not only be executed in chronological order as described, but may also be executed in parallel or individually as needed depending on the processing capabilities of the device executing the processes. Other appropriate modifications are possible without departing from the spirit of the present invention. [Processor, Program, Recording Medium] The functions realized by the components described in this specification may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (Central Processing Units), conventional circuits, and / or combinations thereof, programmed to realize the described functions. A processor includes transistors and other circuits and is considered to be circuitry or processing circuitry. A processor may also be a programmed processor that executes a program stored in memory.

[0036] In this specification, a circuitry, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions.

[0037] If the hardware is a processor considered to be a type of circuitry, the circuitry, means, or unit is a combination of the hardware and software used to configure the hardware and / or processor.

[0038] The various processes described above can be implemented by loading a program that executes each step of the above method into the recording unit 2020 of the computer 2000 shown in Figure 4, and operating the control unit 2010, input unit 2030, output unit 2040, display unit 2050, etc.

[0039] The program describing the processing contents can be recorded on a computer-readable recording medium, which may be, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, or any other suitable recording medium.

[0040] The program may be distributed by, for example, selling, transferring, lending, etc. portable recording media such as DVDs and CD-ROMs on which the program is recorded. Furthermore, the program may be stored in a storage device of a server computer, and then transferred from the server computer to other computers via a network, thereby distributing the program.

[0041] A computer that executes such a program may first temporarily store the program recorded on a portable recording medium or transferred from a server computer in its own storage device. Then, when executing a process, the computer reads the program stored on its own recording medium and executes the process in accordance with the read program. Alternatively, the computer may read the program directly from a portable recording medium and execute the process in accordance with the program. Furthermore, the computer may execute the process in accordance with the program each time a program is transferred from a server computer to the computer. Alternatively, the server computer may not transfer the program to the computer, but may instead execute the process through a so-called ASP (Application Service Provider) service, which realizes the processing function by issuing an execution instruction and obtaining the results. Furthermore, the server computer may execute the process at the terminal using a so-called SaaS (Software as a Service) service, which allows users to use part of a server computer along with the program. In this embodiment, the program includes information used for processing by an electronic computer that is equivalent to a program (such as data that is not a direct instruction to a computer but has properties that dictate computer processing).

[0042] Furthermore, in this embodiment, the device is configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware.

Claims

1. A noise reduction method comprising: acquiring optical information obtained by photographing vibrations caused by noise; estimating the noise at an object being photographed from the optical information; and generating a cancellation sound from the estimated noise.

2. A noise reduction method according to claim 1, comprising: estimating a transfer characteristic from the object to be photographed to a control point using the optical information; and generating the cancellation sound from the estimated value of the noise and the estimated value of the transfer characteristic.

3. A noise reduction method according to claim 1, comprising the steps of: estimating the depth to the object to be photographed from the optical information; generating a primary path model using the estimated depth to estimate a transfer function from the object to be photographed to a control point; updating a coefficient of a noise control filter using an error signal at the control point and a signal based on the estimated value of the noise; and filtering the estimated value of the noise using the estimated value of the transfer function estimated in the generating primary path model or the filter coefficient updated in the coefficient updating step.

4. A noise reduction device comprising: acquiring optical information obtained by photographing vibrations caused by noise; estimating the noise at an object being photographed from the optical information; and generating a cancellation sound from the estimated noise value.

5. A program for causing a computer to execute the noise reduction method of claim 1.

6. A noise reduction system comprising the noise reduction device of claim 4 and a device for using said noise reduction device via a network.

Images