Signal Processing Device, Signal Processing Method, and Program

Abstract

A signal processing device includes: a reverberant sound signal generation unit that generates a reverberant sound signal based on the sound source position of a virtual sound source and the distance from a reference point; and a drive signal generation unit that generates a drive signal for a speaker array through a wavefront synthesis filter. The drive signal generation unit generates the drive signal based on: a signal obtained by performing wavefront synthesis filtering on a signal obtained by convolving the reverberant sound signal with the signal of the virtual sound source and / or a signal obtained by performing wavefront synthesis filtering on the reverberant sound signal to make the reverberant sound signal a virtual sound source.

Description

Technical Field

[0001] The present disclosure relates to a signal processing device, a signal processing method, and a program. Background Art

[0002] In a general environment such as a room, the sound heard reverberates due to the sound wave generated from a sound source being repeatedly reflected on the floor, ceiling, or wall. Since the frequency characteristics and duration of the sound are changed due to this reverberation, the listener can perceive the sense of distance and reverberation of the sound through these changes. A technique for adding such reverberation is described in Patent Document 1 below.

[0003] Citation List

[0004] Patent Document

[0005] Patent Document 1: WO 2013 / 057948 A. Summary of the Invention

[0006] Problems to be Solved by the Invention

[0007] However, in the technique described in Patent Document 1, only the delay of the simulated reverberation is operated as a control target, and therefore, it is insufficient to control the sense of reverberation caused by sound reflection or sound duration.

[0008] An object of the present disclosure is to provide, for example, a signal processing device, a signal processing method, and a program capable of adding appropriate reverberation to a signal of a virtual sound source.

[0009] Solutions to the Problems

[0010] For example, the present disclosure provides,

[0011] A signal processing device, comprising:

[0012] A reverberant sound signal generation unit that generates a reverberant sound signal according to a sound source position of a virtual sound source and a distance to a reference point; and

[0013] A drive signal generation unit that generates a drive signal for a speaker array through a wavefront synthesis filter, wherein

[0014] The drive signal generation unit generates a drive signal based on a signal obtained by performing wavefront synthesis filtering on a signal obtained by convolving the reverberant sound signal with the signal of the virtual sound source and / or a signal obtained by performing wavefront synthesis filtering on the reverberant sound signal so that the reverberant sound signal becomes the virtual sound source.

[0015] For example, the present disclosure provides,

[0016] A signal processing method, comprising:

[0017] The reverberant sound signal generation unit generates a reverberant sound signal based on the sound source position of the virtual sound source and the distance from the reference point; and

[0018] The drive signal generation unit generates a drive signal for the speaker array through a wavefront synthesis filter, where

[0019] the drive signal generation unit generates a drive signal based on a signal obtained by performing wavefront synthesis filtering on a signal obtained by convolving the reverberant sound signal with the signal of the virtual sound source and / or a signal obtained by performing wavefront synthesis filtering on the reverberant sound signal so that the reverberant sound signal becomes a virtual sound source.

[0020] For example, the present disclosure provides

[0021] a program for causing a computer to execute a signal processing method, the signal processing method including:

[0022] The reverberant sound signal generation unit generates a reverberant sound signal based on the sound source position of the virtual sound source and the distance from the reference point; and

[0023] The drive signal generation unit generates a drive signal for the speaker array through a wavefront synthesis filter, where

[0024] the drive signal generation unit generates a drive signal based on a signal obtained by performing wavefront synthesis filtering on a signal obtained by convolving the reverberant sound signal with the signal of the virtual sound source and / or a signal obtained by performing wavefront synthesis filtering on the reverberant sound signal so that the reverberant sound signal becomes a virtual sound source. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a diagram for describing the outline of the wavefront synthesis technique.

[0026] Figure 2 is a diagram for describing the outline of the first embodiment.

[0027] Figure 3 A of Figure 3 and B of

[0028] Figure 4 are diagrams each showing an example of a reverberation waveform measured using a microphone.

[0029] Figure 5 is a diagram showing a reproduction system according to the first embodiment.

[0030] Figure 6 is a diagram for describing an example of the configuration of a signal processing device according to the first embodiment.

[0031] Figure 7 is a flowchart referred to when describing an operation example of the signal processing apparatus according to the first embodiment.

[0032] Figure 8 is a diagram referred to when describing a modification example of the first embodiment.

[0033] Figure 9 is a diagram referred to when describing a modification example of the first embodiment.

[0034] Figure 10 is a diagram referred to when describing a modification example of the first embodiment.

[0035] Figure 11 is a diagram for describing an overview of the second embodiment.

[0036] Figure 12 is a diagram for describing a configuration example of the signal processing apparatus according to the second embodiment.

[0037] Figure 13 is a diagram for describing a configuration example of the signal processing apparatus according to a modification example of the second embodiment.

[0038] Figure 14 is a diagram referred to when describing an operation example of the signal processing apparatus according to a modification example of the second embodiment.

[0039] Figure 15 is a diagram referred to when describing a variation example of the second embodiment.

[0040] Figure 16 is a diagram referred to when describing a variation example of the second embodiment.

[0041] Figure 17 is a diagram for describing an overview of the third embodiment.

[0042] Figure 18 is a diagram for describing a configuration example of the signal processing apparatus according to the third embodiment.

[0043] Figure 19 is a diagram referred to when describing the processing of the rear reverberation signal generation unit according to the third embodiment. DETAILED DESCRIPTION

[0044] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that the description will be made in the following order.

[0045] <Problems to be Considered in the Present Invention>

[0046] <First Embodiment>

[0047] <Second Embodiment>

[0048] <Third Embodiment>

[0049] <Modification>

[0050] The embodiments described below are preferred specific examples of the present disclosure, and the content of the present disclosure is not limited to these embodiments and the like.

[0051] <Problems to be Considered in the Present Invention>

[0052] First, for the sake of easy understanding of the present disclosure, the background of the present disclosure and the problems to be considered in the present disclosure will be described.

[0053] In recent years, high-resolution audio recorded at high bitrates and high sampling rates and virtual surround audio reproduced through headphones have become widespread. These provide new value to conventional digital audio. At the same time, research and development of surround audio reproduced by multi-channel speakers are also underway, and efforts have begun to achieve a realistic listening experience by increasing the resolution not only in the time direction but also in the space direction. A technique has been proposed to faithfully reproduce the original wavefront by using far more speakers than the conventional 2- to 5.1-channel (ch) surround system to reproduce the wavefront that propagates in space with high spatial resolution. This technique is generally referred to as "wavefront synthesis," and various research and developments are being carried out in academic and industrial fields.

[0054] Figure 1 is a diagram for describing the outline of wavefront synthesis. As Figure 1 shown, wavefront synthesis is a system in which musical instruments as virtual sound sources (object sound sources) are arranged at any position in space, and the wavefronts of the sounds generated from the virtual sound sources are reproduced from a plurality of speakers. Physically reproducing the wavefronts generated from musical instruments enables the provision of a sound environment as if real musical instruments were present on site. In addition, each virtual sound source can be moved or arranged closer to the listener than the speaker array for reproducing the sounds for realizing wavefront synthesis, which enables music reproduction with unprecedented freedom. It should be noted that although musical instruments are shown in Figure 1 these show the types of virtual sound sources in an easy-to-understand manner and do not represent the setting of actual musical instruments. In addition, the type of each virtual sound source is not limited to musical instruments and can be the sounds of animals or environmental sounds.

[0055] Meanwhile, the sound heard in a general environment reverberates due to the repeated reflection of sound waves generated from a sound source on the floor, ceiling, or walls. Since the frequency characteristics and duration of the sound are changed due to this reverberation, the listener can perceive the sense of distance and reverberation of the sound through these changes. However, in wavefront synthesis, the effects such as reflection are not physically reproduced. Therefore, even if the virtual sound source is set to accurately reproduce the wavefront, it may be difficult for the listener to perceive the sense of distance and reverberation of the sound source. Therefore, even in wavefront synthesis, it is considered that appropriately adding reverberation enables improvement of the realism caused by the sound field to be reproduced. In view of the above, embodiments of the present disclosure will be described in detail.

[0056] <First Embodiment>

[0057] [Overview]

[0058] The present disclosure is a technique for adding reverberation in a system applying wavefront synthesis (hereinafter, such a system will be appropriately abbreviated as a "wavefront synthesis system"). Figure 2 is a diagram for describing the outline of the first embodiment. As Figure 2 shown, the wavefront of the sound generated from a predetermined virtual sound source VS is reproduced from a speaker array SA including a plurality of speaker units SU. Therefore, the listener L listens to the sound of the virtual sound source VS, where the sound source such as an instrument is located. The position of the virtual sound source VS can be changed. The first embodiment is an embodiment in which virtual reverberation VR is added to the virtual sound source VS itself.

[0059] Here, a specific example of adding reverberation to the virtual sound source itself in the wavefront synthesis system will be described. First, the characteristics of reverberation will be described in terms of the distance between the sound source and the sound receiving point.

[0060] Figure 3 Shows an example of a reverberation (impulse response) waveform measured using a microphone. Figure 3 A of shows an example of a reverberation waveform in the case where the distance between a predetermined sound source and the sound receiving point (the installation position of the microphone) is short, and Figure 3 B of shows an example of a reverberation waveform in the case where the distance between a predetermined sound source and the sound receiving point is long. Reverberation can be divided into three parts: direct sound, reflected sound (also called initial reflected sound), and late reverberation (also called late reverberation sound). It should be noted that the reflected sound and the late reverberation are distinguished according to the time taken for the reference direct sound to reach the sound receiving point, the presence or absence of the perceptibility of the direction (for example, the arrival direction of the reflected sound can be perceived, while the arrival direction of the late reverberation cannot be perceived), etc.

[0061] When comparing two direct sounds, Figure 3 the amplitude value of the direct sound shown in A of is greater than Figure 3The amplitude value of the direct sound shown in B. This indicates that the distances from the position of the sound source to the sound reception point are different, and as the distance increases, the sound pressure decreases due to the attenuation with distance. In addition, each of the reflected sound and the late reverberation is the sound that reflects and collects the sound radiated at all angles on the wall, and no large difference in amplitude caused by the difference in the distance from the sound source to the sound reception point is observed. Therefore, it can be considered that the energy ratio of the "direct sound" to the "reflected sound and late reverberation" changes according to the distance. In addition, in the case of normalizing the sound pressure of the direct sound using the maximum amplitude, it can be considered that the longer the distance to the sound source, the relatively higher the ratio of the late reverberation, and thus the longer the time to hear the late reverberation. In the present embodiment, the reverberation control according to the distance is the technical basis.

[0062] Figure 4 FIG. is a diagram showing a reproduction system according to the first embodiment. Specifically, a wavefront synthesis system in which a virtual sound source VS is arranged in space, and a listener L listens to the sound of the virtual sound source VS in front of the speaker array SA. The distance (shortest distance) between the virtual sound source VS and the reference point is set to the distance r. The reference point according to the present embodiment is the sound reception position with respect to the sound source position of the virtual sound source VS (i.e., the position of the listener L). The distance r is obtained by using a distance measuring camera (depth camera), a time-of-flight (ToF) sensor, a light detection and ranging (LiDAR), a beacon, etc. that can measure the distance to the listener L. It should be noted that the position of the listener L can change between the positions PO1, PO2, and PO3, as shown in Figure 4 FIG. The new distance r is obtained according to the change in the position of the listener L. When the position of the virtual sound source VS changes, the distance r also changes.

[0063] The level and length of the reverberation are adjusted according to the distance r to generate a reverberation sound signal. The reverberation sound signal is convolved with the signal of the virtual sound source VS and then processed by a wavefront synthesis filter to generate a drive signal, and the drive signal is reproduced from the speaker array SA so that the listener L can listen to the sound with added reverberation. As the reverberation convolved with the virtual sound source VS, the reverberation pre-measured in a concert hall or the like can be used, or the reverberation created by an analog algorithm using a method such as mirroring can be used.

[0064] The reverberation added according to the distance r is determined according to, for example, the reverberation control function. Figure 5 FIG. shows an example of such a reverberation control function. In Figure 5 FIG., the horizontal axis represents the distance r, and the vertical axis represents the level of the reverberation or the reverberation time. Applying Figure 5The reverberation control function shown in enables control such that at least one of the reverberation level and the reverberation time becomes longer as the distance increases. Maintaining the reverberation control function or obtaining the reverberation control function from an external device enables following and processing of the distance change accompanying the movement of the virtual sound source VS or the listener L. That is, the distance r is measured in advance so that the reverberation reproduced by the wavefront synthesis system can be automatically calculated. In addition, except for a system that performs wavefront synthesis offline, the distance r can be measured in real time, and the reverberation according to the distance change can be calculated and reproduced in real time.

[0065] Note that the user can adjust the above-described reverberation control function. For example, the curvature of the reverberation control function can be adjusted by the user. A configuration can be adopted in which the user can freely change the curvature of the reverberation control function by displaying the reverberation control function on the editing software for wavefront synthesis and performing a drag-and-drop operation on the reverberation control function with a mouse or the like.

[0066] <Configuration Example of Signal Processing Device>

[0067] Figure 6 is a diagram for describing a configuration example of a signal processing device (signal processing device 1) according to the first embodiment. In Figure 6 the thin arrows indicate the flow of audio signals, the thick arrows indicate the flow of drive signals (wavefront synthesis signals (multichannel signals including signals of eight channels in this example)), and the dashed arrows indicate the flow of parameters.

[0068] A signal of the virtual sound source VS (hereinafter referred to as the virtual sound source signal D0) is input to the signal processing device 1. For example, the virtual sound source signal D0 is a monophonic signal of one channel and is an object audio signal corresponding to a predetermined object. The signal of the virtual sound source VS can be read from an appropriate memory or distributed from a network such as the Internet. Note that in the wavefront synthesis system, information indicating the position of the virtual sound source VS is also input to the signal processing device 1, but the description of this point is omitted.

[0069] In addition, distance information I(r) indicating the distance r measured by a distance measuring device or the like and Figure 5 the reverberation control function FU shown in are input to the signal processing device 1. Note that the signal processing device 1 may have a configuration including the above-described distance measuring device or may have a configuration in which the reverberation control function FU is stored in advance in a memory.

[0070] When the signal processing device 1 performs processing, a drive signal for driving the speaker array SA is generated. The drive signal includes signals of channels (ch) corresponding to the number of speaker units SU included in the speaker array SA. In Figure 6In the illustrated example, since the speaker array SA includes eight speaker units SU, the signal processing device 1 outputs a drive signal including signals of eight channels to be provided to the speaker units SU. Note that in an actual wavefront synthesis system, a larger number of speaker units SU are usually used, but in this example, for the sake of convenience of description, it will be assumed that the speaker array SA includes eight speaker units SU for description.

[0071] The signal processing device 1 includes, for example, a reverberation reading unit 11, a reverberation correction unit 12, a reverberation convolution unit 13, a wavefront synthesis filter 14, and a digital-to-analog (DA) conversion / amplification unit 15. The reverberation reading unit 11 reads a signal corresponding to reverberation from an appropriate memory. The signal corresponding to reverberation may be distributed via a network such as the Internet, and in this case, the reverberation reading unit 11 serves as a communication unit connectable to the network. The reverberation reading unit 11 outputs the signal RDA corresponding to the read reverberation to the reverberation correction unit 12.

[0072] As an example of a reverberant sound signal generation unit, the reverberation correction unit 12 specifies the level of reverberation corresponding to the distance information I(r) using, for example, the reverberation control function FU. Then, the reverberation correction unit 12 generates a reverberant sound signal RDB by adjusting the level of the signal RDA corresponding to reverberation based on the specified level. The reverberation correction unit 12 outputs the generated reverberant sound signal RDB to the reverberation convolution unit 13. That is, in this embodiment, the reverberation correction unit 12 corresponds to the reverberant sound signal generation unit. Note that the reverberation time may be adjusted instead of the level of reverberation, or the reverberation time may be adjusted together with the level of reverberation.

[0073] The reverberation convolution unit 13 performs a process of convolving the reverberant sound signal RDB with the virtual sound source signal D0 to generate a convolution signal D1. Since convolution processing is usually performed by converting a signal into a frequency domain signal, the reverberation convolution unit 13 may include a fast Fourier transform (FFT) or the like. The reverberation convolution unit 13 outputs the convolution signal D1 to the wavefront synthesis filter 14.

[0074] The wavefront synthesis filter 14, which is an example of the drive signal generation unit, generates a drive signal D2 by performing wavefront synthesis filtering on the convolution signal D1. For example, the wavefront synthesis filter 14 converts the convolution signal D1 into a multi-channel signal (in this example, eight channels), and appropriately adjusts the phase, gain, delay, etc. of each signal included in the multi-channel signal to reproduce such a wavefront where the virtual sound source signal D0 is positioned at a predetermined position. As a result, the drive signal D2 including a signal of eight channels is output from the wavefront synthesis filter 14. As an algorithm for the processing performed by the wavefront synthesis filter 14, any algorithm can be used according to the processing power of the calculator, the arrangement of the speakers, etc. Specifically, a higher-order high-fidelity stereo (HOA) method, a weighted pattern matching method, a spectral division method, etc. can be applied.

[0075] The DA conversion / amplification unit 15 converts the drive signal D2 in digital format into a drive signal D3 in analog format and amplifies the drive signal D3. As a result, the drive signal D3 in analog format is generated. The signal of each channel included in the drive signal D3 is supplied to the corresponding speaker unit in the speaker unit SU and reproduced. Note that the components including the wavefront synthesis filter 14 and the DA conversion / amplification unit 15 can be components corresponding to the drive signal generation unit.

[0076]

Operation Example of the Signal Processing Device

[0077] Next, an operation example of the signal processing device 1 will be described with reference to Figure 7 the flowchart shown in. In step ST11, the virtual sound source signal D0 is read. The read virtual sound source signal D0 is supplied to the reverberation convolution unit 13. Then, the process proceeds to step ST12.

[0078] In step ST12, the reverberation reading unit 11 reads the signal RDA corresponding to the reverberation. Note that the signal RDA corresponding to the reverberation can be generated. Then, the process proceeds to step ST13.

[0079] In step ST13, the reverberation correction unit 12 reads the distance information I(r) and the reverberation control function FU. Then, the process proceeds to step ST14.

[0080] In step ST14, the reverberation correction unit 12 performs reverberation correction processing. Through this processing, the reverberation sound signal RDB is generated. Specifically, the signal RDA corresponding to the reverberation is corrected based on the distance information I(r) and the reverberation control function FU, thereby generating the reverberation sound signal RDB. The generated reverberation sound signal RDB is output to the reverberation convolution unit 13. Then, the process proceeds to step ST15.

[0081] In step ST15, the reverberation convolution unit 13 performs a process of convolving the virtual sound source signal D0 and the reverberant sound signal RDB. Through this process, a convolution signal D1 is generated. Then, the process proceeds to step ST16.

[0082] In step ST16, the wavefront synthesis filter 14 performs a filtering process. Through this process, a drive signal D2 in digital format is generated. Then, the process proceeds to step ST17.

[0083] In step ST17, the DA conversion / amplification unit 15 performs a process. Through this process, an analog format drive signal D3 is generated and the drive signal D3 is amplified. The signal of each channel of the drive signal D3 is supplied to the corresponding speaker unit in the speaker unit SU, and sound is reproduced from the speaker unit SU.

[0084] Note that the order of the above-described process flow can be appropriately changed, or multiple processes can be executed in parallel.

[0085] According to the above-described present embodiment, reverberation independent of the position of the listener L can be added in the wavefront synthesis system. In addition, even when the position of the listener L changes, appropriate reverberation can be added.

[0086] It should be noted that in the case where there are multiple virtual sound sources VS, that is, in the case where there are multiple virtual sound source signals D0, the signal corresponding to the reverberation can be the same signal for all virtual sound source signals D0, or can be different signals for each virtual sound source signal D0. In addition, in the case where there are multiple signals corresponding to the reverberation, the reverberation control function FU can be the same function for the signals, or can be different functions for each signal.

[0087] [Modification Example of the First Embodiment]

[0088] In the above description, an example in which there is one listener for one or more virtual sound sources is assumed, but there can be multiple listeners. Figure 8 is a diagram schematically showing such a situation. As Figure 8 shown, for example, virtual sound source VS A and virtual sound source VS B exist as virtual sound sources. In addition, there are listeners L A and listener L B as listeners. The distance from listener L A to virtual sound source VS A is defined as distance r A1 , and the distance from listener L A to virtual sound source VS B is defined as distance r B1。From listener L B to the virtual sound source VS A The distance is defined as distance r A2 and from listener L B to the virtual sound source VS B The distance is defined as distance r B2 。

[0089] For example, a reference listener L will be considered A towards the virtual sound source VS A and the virtual sound source VS B Add reverberation. At this time, the relationship of the distances from the listener L A to each virtual sound source is r A1 < r B1 and therefore, in the case of performing processing similar to that in the embodiment, the reverberation added to the virtual sound source VS B is set to be larger and longer than the reverberation added to the virtual sound source VS A As described above, this setting represents the phenomenon that the reverberation of a distant sound source is relatively large through wavefront synthesis. However, when the listener L B listens to the sound with reverberation added as described above, although the relationship of the distances from the listener L B to each virtual sound source is r A2 > r B2 the reverberation of the virtual sound source VS B is heard louder.

[0090] One of the advantages of the wavefront synthesis system is that multiple people can listen to the same wavefront simultaneously, and it is very likely that there are multiple listeners. In the case of multiple listeners, using the shortest distance between the sound source and the listener optimizes the reverberation only for the listener. Therefore, for example, in the case where the presence of multiple listeners is detected by a distance measuring device or the like, or in the case of setting a predetermined mode, the distance that minimizes the distance between the virtual sound source and the speaker array SA is set as the distance r.

[0091] Specifically, as Figure 9 shown, the shortest distance from the virtual sound source VS A to the speaker array SA is set as the distance r A and the shortest distance from the virtual sound source VS B to the speaker array SA is set as the distance r B 。The shortest distance to the speaker array SA is defined as the shortest distance to a predetermined one of the speaker units SU included in the speaker array SA, but can be defined with reference to another position (e.g., the sound emission surface) of the speaker array SA. Each having a corresponding distance r A and distance r BThe horizontal or temporal reverberation segments of one of them are respectively added to the virtual sound source VS A and the virtual sound source VS B The processing is the same as the above processing, so redundant descriptions will be omitted.

[0092] Therefore, it is possible to avoid the situation where the reverberation is different for each listener as shown in Figure 8 and achieve the consistency of the reverberation of the virtual sound source throughout the space where the speaker array SA is installed. Therefore, it is possible to prevent the number of listeners causing the above inconveniences or the movement of each listener.

[0093] Note that although the speaker array SA with the speaker units SU arranged in a straight line is described as an example above, the speaker units SU can be arranged in a rectangular shape (four directions), for example, as shown in Figure 10 or can be arranged along a circular shape or any other shape. It should be noted that in the example shown in Figure 10 since there are multiple listeners within the speaker array SA, the shortest distance from each virtual sound source to the speaker array SA is set to the distance r.

[0094] <Second Embodiment>

[0095] Next, the second embodiment will be described. It should be noted that in the description of the second embodiment, the same or similar components in the above description are denoted by the same reference numerals, and redundant descriptions will be appropriately omitted. In addition, unless otherwise specified, the content described in the first embodiment can be applied to the second embodiment.

[0096] [Overview]

[0097] In the first embodiment, reverberation can be added without significantly changing the framework of the general wavefront synthesis system. The second embodiment is an embodiment that regenerates the reflection component of the reverberation as a virtual sound source, and more effective reverberation can be added.

[0098] Reverberation includes reflected sounds following the direct sound. This reflected sound is observed when sound waves reflected from the ceiling, floor, or wall reach the sound reception point. That is, it can be considered that the reflected sound does not reach from the position of the sound source, but from the reflection point. Therefore, assuming that the reflection point is the sound source that generates the reflected sound, wavefront synthesis can be performed while setting the reflected sound as a virtual sound source.

[0099] Figure 11 is a diagram for describing the outline of the second embodiment. In this embodiment, the reflected sound of the reverberation is made into a virtual sound source, and then wavefront synthesis is performed. For example, the reflected sound of a single reflection is actually reverberated as the virtual reverberation V R1In addition, the reflected sound of the secondary reflection is actually reverberated into the virtual reverberation V R2 Note that although the reflection occurs multiple times at different positions, for simplicity, only the first reflection and the secondary reflection are shown in Figure 11 . By placing the virtual sound source at the final reflection point of the route through which the reflected sound reaches the listener L (in this instance, near the wall), the reflected sound can be made into a virtual sound source. Since the reflection occurs not only on the wall but also on the ceiling, the floor, the objects at the scene, etc., the virtual sound source can be placed at those locations. Because of the need to arrange the positions of the reflected sounds to be virtual sound sources, the virtual space information is input to the signal processing device. The virtual space information is, for example, information preset by the creator or the like.

[0100] Figure 12 is a diagram for describing a configuration example of the signal processing device (signal processing device 1A) according to the second embodiment. The signal processing device 1A includes a virtual sound source signal wavefront synthesis filter 21, a virtual reverberation signal generation unit 22, a virtual reverberation signal wavefront synthesis filter 23, an adder 24, an adder 25, and a DA conversion / amplification unit 15. The reflection can occur at N positions instead of at one position, and thus N signals (hereinafter, referred to as virtual reverberation signals as appropriate) of the reflected sound that becomes a virtual sound source are generated. Therefore, the virtual reverberation signal generation unit 22 includes N virtual reverberation signal generation units (virtual reverberation signal generation unit 22 1 to virtual reverberation signal generation unit 22 N ). For similar reasons, the virtual reverberation signal wavefront synthesis filter 23 also includes N virtual reverberation signal wavefront synthesis filters (virtual reverberation signal wavefront synthesis filter 23 1 to virtual reverberation signal wavefront synthesis filter 23 N ). It should be noted that when it is not necessary to distinguish each virtual reverberation signal generation unit, the virtual reverberation signal generation units are appropriately collectively referred to as the virtual reverberation signal generation unit. In addition, when it is not necessary to distinguish the virtual reverberation signal wavefront synthesis filters, the virtual reverberation signal wavefront synthesis filters are appropriately collectively referred to as the virtual reverberation signal wavefront synthesis filter 23.

[0101] A virtual sound source signal D0 is provided to the signal processing device 1A. The virtual sound source signal D0 is branched and also provided to the virtual reverberation signal generation unit 22. In addition, virtual wall position information I(VW) indicating the position of the virtual wall relative to the predetermined virtual sound source VS is provided to the virtual reverberation signal generation unit 22, and the virtual wall position information I(VW) serves as an example of the virtual space information.

[0102] The virtual sound source signal wavefront synthesis filter 21 filters the virtual sound source signal D0 through the wavefront synthesis filter. Wavefront synthesis filtering is performed to generate the output signal D5. For example, the virtual sound source signal wavefront synthesis filter 21 converts the virtual sound source signal D0 into a multi-channel signal (in this example, eight channels), and appropriately adjusts the phase, gain, delay, etc. of each signal included in the multi-channel signal to reproduce the wavefront where the virtual sound source signal D0 is located at a predetermined position. As a result, the output signal D5 including eight channel signals is output from the virtual sound source signal wavefront synthesis filter 21. As the algorithm for the processing performed by the virtual sound source signal wavefront synthesis filter 21, any algorithm can be used according to the processing power of the calculator, the arrangement of the speakers, etc. Specifically, the HOA method, the weighted pattern matching method, the spectral division method, etc. can be applied.

[0103] The virtual reverberation signal generation unit 22 acquires the virtual sound source signal D0, the sound source position of the virtual sound source signal D0, and the virtual wall position information I(VW), and generates a virtual reverberation signal D6 (D6 1 ~D6 N ). For example, the virtual reverberation signal generation unit 22 performs processing to specify the reflection position of the virtual reverberation signal D6, in other words, processing to specify the position where the virtual reverberation signal D6 is located. For example, by using a method such as mirroring to simulate sound propagation, the reflection position of the virtual reverberation signal D6 can be calculated. For example, the virtual reverberation signal generation unit 22 refers to the virtual wall position information I(VW) and sets the reflection position of the once-reflected virtual reverberation signal D6 1 at the position of the wall closest to the reproduction position of the virtual sound source signal D0. As described above, in the present embodiment, according to the sound source position of the virtual sound source VS and the distance to the surface reflecting the virtual sound source signal D0, a virtual reverberation signal D6 (D6 1 to D6 N ), which is a reverberation sound signal in the present embodiment, is generated.

[0104] The virtual reverberation signal wavefront synthesis filter 23 performs wavefront synthesis filtering on the virtual reverberation signal D6 to generate a virtual reverberation signal D6' (D6' 1 to D6' N ) obtained by making the virtual reverberation signal D6 a virtual sound source. For example, the virtual reverberation signal wavefront synthesis filter 23 1 converts the virtual reverberation signal D6 1 into a multi-channel signal (in this example, eight channels), and adjusts the phase, gain, delay, etc. of each signal included in the multi-channel signal so that the virtual reverberation signal D6 1 is located at a predetermined position (along the wall). Therefore, from the virtual reverberation signal wavefront synthesis filter 23 1The output is a virtual reverberation signal D6’ including signals of eight channels 1 . As an algorithm for the process executed by the virtual reverberation signal wavefront synthesis filter 23 1 , any algorithm can be used according to the processing ability of the calculator, the arrangement of the speakers, etc. Specifically, the HOA method, the weighted pattern matching method, the spectral division method, etc. can be applied

[0105] The adder 24 adds the signals of the corresponding channels included in the virtual reverberation signal D6’ obtained by making the virtual reverberation signal D6 a virtual sound source (specifically, the virtual reverberation signal D6’ 1 to D6’ N ). As a result of the addition process of the adder 24, an output signal D7 including signals of eight channels is output from the adder 24

[0106] Drive signals for each of the speaker units SU are generated based on the virtual reverberation signal D6’ obtained by making the virtual reverberation signal D6 a virtual sound source. Specifically, the adder 25 adds, for each corresponding channel, the output signal D5 obtained by performing wavefront synthesis filtering on the virtual sound source signal D0 and the output signal D7 output from the adder 24. Through the addition process of the adder 25, a drive signal D8 including signals of eight channels in digital format is generated. The generated drive signal D8 is output from the adder 25

[0107] The DA conversion / amplification unit 15 converts the drive signal D8 in digital format into a signal in analog format, and then amplifies the converted signal to generate and output a drive signal D9. The signal of each channel of the drive signal D9 in analog format is provided to the corresponding one of the speaker units SU, and sound is reproduced from the speaker units SU

[0108] As described above, in the present embodiment, for example, components including the virtual reverberation signal generation unit 22 and the virtual reverberation signal wavefront synthesis filter 23 correspond to the reverberant sound signal generation unit. In addition, for example, components including the virtual sound source signal wavefront synthesis filter 21, the adder 24, and the adder 25 (which may include the DA conversion / amplification unit 15) correspond to the drive signal generation unit

[0109] According to the second embodiment described above, making the reverberation a virtual sound source enables the reverberation to be arranged at any position. Therefore, the reverberation can be reproduced appropriately

[0110] Note that in the second embodiment, the number of reflections (primary reflection, secondary reflection, and tertiary reflection) can be set to any number. In addition, since the attenuation rate of reflections varies depending on the material and shape of the wall, virtual reverberation can be generated in consideration of these factors. Generating and reproducing virtual reverberation in consideration of the material and shape of the wall enables a higher sense of realism to be given to the listener.

[0111] [Modification Example of the Second Embodiment]

[0112] Next, a modification example of the second embodiment will be described. In the wavefront synthesis system that makes reverberation a virtual sound source described above, as the reflection order increases, the wavefront synthesis filter requires a larger number of calculations, which increases the computational amount. Such a problem is disadvantageous in applications such as an online processing wavefront synthesis system.

[0113] A normal wavefront synthesis filter is designed based on the number and arrangement of the installed speakers and the order of the wavefront to be reproduced. Reducing the number of speakers and lowering the reproduction order makes it possible to reduce the computational amount of the wavefront synthesis filtering process. When the order is lowered, the resolution of the wavefront to be reproduced decreases. Therefore, a trade-off occurs between sound quality and computational amount. However, generally, the gain of the reflection component is smaller than that of the direct sound, and the listener does not strictly listen only to the reflection. Therefore, it is considered that performing wavefront synthesis on the reverberation component with a low order helps to improve the sense of reality without problems. Therefore, a modification example of the second embodiment will be described.

[0114] (First Modification Example)

[0115] First, Modification Example 1 will be described. Figure 13 is a diagram for describing a configuration example of a signal processing device (signal processing device 1B) according to this modification example. The signal processing device 1B is different from the signal processing device 1A in that the signal processing device 1B includes a virtual reverberation signal low-order wavefront synthesis filter 27 (virtual reverberation signal low-order wavefront synthesis filter 27 1 ~27 N ) to replace the virtual reverberation signal wavefront synthesis filter 23, and includes a signal duplication unit 28 (signal duplication unit 28 1 ~28 N ). Note that N is a value corresponding to the reflection order in the second embodiment.

[0116] The operations of the virtual reverberation signal low-order wavefront synthesis filter 27 and the signal duplication unit 28 will be described with reference to Figure 14 . Note that in Figure 14 , the virtual reverberation signal low-order wavefront synthesis filter 27 1 and the signal duplication unit 28 1, but other low-order wavefront synthesis filters for virtual reverberation signals and signal duplication units operate similarly.

[0117] Low-order wavefront synthesis filter 27 for virtual reverberation signals 1 Performs wavefront synthesis filtering on the virtual reverberation signal D6 1 , but outputs a signal with sparse channels. For example, as Figure 14 shown, the low-order wavefront synthesis filter 27 for virtual reverberation signals 1 outputs a signal D10 obtained by making the virtual reverberation signal D6 1 into a virtual sound source. 1 . The number of channels of the signal D10 1 (e.g., 4 channels) is less than the number of channels of the speaker array SA (8 channels in this example). Then, the signal duplication unit 28 1 interpolates the signal so that sound is reproduced from all speaker units SU included in the speaker array SA. As Figure 14 shown, for example, for a speaker with sparse signals, the signal duplication unit 281 duplicates the drive signals of nearby speakers (more specifically, adjacent speakers). Through such processing, an output signal D11 including an 8-channel signal is generated 1 , and the generated output signal D11 is output 1 . Since the operations of the adders 24 and 25 have been described, redundant descriptions will be omitted.

[0118] Note that an interpolation process can be performed in which the signals of the speakers on both sides of the speaker without a signal are averaged to generate a signal for the speaker without a signal. In addition, the output of only the low-order wavefront synthesis filter 27 for virtual reverberation signals can be reproduced without performing signal duplication, and the sparse speakers are left as they are. As described above, the computational amount can be reduced.

[0119] (Second modified example)

[0120] A configuration can be adopted in which wavefront synthesis is not performed on the reflection component and the reflection component is not made into a virtual sound source, so that the computational amount can be reduced. As Figure 15 shown, the virtual reverberation generation unit 22 designates the position of the virtual reverberation VR corresponding to the first reflection A . In addition, the position of the listener L can also be determined by imaging with a camera or the like. For example, the position of the virtual reverberation VR can be located by connecting a straight line A and the position of the listener L, and sound can be reproduced in a state where a reverberation component is added to the speaker unit located on the straight line (in Figure 15 , the speaker unit SU A ). Not only the virtual reverberation VR corresponding to the first reflection is locatedA position, and the position of the virtual reverberation VR corresponding to the secondary reflection is located B The position can also be connected to the position of the listener L by a straight line, and the reverberation component can be added to the speaker unit located on the straight line (in Figure 15 , the speaker unit SU B ) and reproduce the sound. Even with this processing, the computational amount can be reduced.

[0121] In addition, the reverberation speaker can be separately installed from the speaker array SA for wavefront synthesis, and the reverberation can be reproduced from the speaker. For example, as Figure 16 shown, the speaker SP A for reproducing the virtual reverberation corresponding to the primary reflection and the speaker SP B for reproducing the virtual reverberation corresponding to the secondary reflection can be arranged so that the reverberation can be reproduced from the speaker SP A and the speaker SP B . According to the method shown in Figure 16 , the reverberation can also be reproduced from the side and the back, where the wavefront synthesis speaker is not installed, so that a higher feeling of being surrounded by sound can be given to the listener L.

[0122] <Third Embodiment>

[0123] Next, the third embodiment will be described. Note that in the descriptions of the first embodiment and the second embodiment, the same or similar components in the above descriptions are denoted by the same reference numerals, and the redundant descriptions will be appropriately omitted. In addition, unless otherwise specified, the contents described in the first embodiment and the second embodiment can be applied to the third embodiment.

[0124] In the first embodiment and the second embodiment, the reflection of the reverberation has been focused on. However, other features of the reverberation include the feeling of being surrounded by sound and the duration of the sound. Figure 3 In, the entire space where the listener is located emits sound and the sound that lasts for a long time (i.e., the result of countless reflections occurring everywhere) is represented as a component of the rear reverberation. This embodiment is an embodiment regarding a method for effectively reproducing the rear reverberation.

[0125] Figure 17 is a diagram for describing the outline of the third embodiment. In the third embodiment, the rear reverberation is reproduced by using the speaker array SA. In the rear reverberation, since the entire space emits sound as described above, the entire widely installed speaker array AR is used to represent the sound in the entire space.

[0126] Figure 18This is a diagram for describing a configuration example of a signal processing device (signal processing device 1C) according to the third embodiment. In addition to the reverberation reading unit 11, the wavefront synthesis filter 14, and the DA conversion / amplification unit 15 included in the signal processing device 1 according to the first embodiment, the signal processing device 1C further includes a rear reverberation signal generation unit 31, a reverberation convolution unit 32, a multichannel processing unit 33, and an adder 34.

[0127] The wavefront synthesis filter 14 outputs a signal D15 by performing wavefront synthesis filtering on the virtual sound source signal D0.

[0128] A signal RDA corresponding to the reverberation output from the reverberation reading unit 11 is supplied to the rear reverberation signal generation unit 31. The rear reverberation signal generation unit 31 generates a rear reverberation signal D16 based on the signal RDA corresponding to the reverberation. For example, as Figure 19 schematically shown, the rear reverberation signal generation unit 31 generates the rear reverberation signal D16 by removing the direct sound and the reflected sound of the impulse response.

[0129] The reverberation convolution unit 32 performs a process of convolving the rear reverberation signal D16 with the virtual sound source signal D0 to generate a convolution signal D17. Then, the reverberation convolution unit 32 outputs the generated convolution signal D17 to the multichannel processing unit 33.

[0130] The multichannel processing unit 33 performs a process of converting the convolution signal D17 into a multichannel signal such that the converted signal corresponds to the number of channels of the speaker array SA (in this example, eight channels). The multichannel processing unit 33 performs a process to generate a multichannel rear reverberation sound (hereinafter, the multichannel rear reverberation signal D18). The multichannel rear reverberation signal D18 is supplied to the adder 34.

[0131] The adder 34 adds the signal D15 output from the wavefront synthesis filter 14 and the multichannel rear reverberation signal D18 for each corresponding channel. The adder 34 performs a process to generate a drive signal D19 in digital format.

[0132] The DA conversion / amplification unit 15 converts the drive signal D19 in digital format into a signal in analog format, and then amplifies the converted signal to generate and output a drive signal D20. Each channel signal of the drive signal D20 in analog format is supplied to a corresponding one of the speaker units SU, and sound is reproduced from the speaker units SU.

[0133] According to the present embodiment described above, rear reverberation can be reproduced in a wide range, and a listener can be given a feeling of being surrounded by sound. In addition, the rear reverberation component can be reproduced as if there is sound throughout the space, without performing calculations through the wavefront synthesis filter.

[0134] Note that in the actual space, the waveform of the impulse response varies depending on the observation point. Considering this, different segments of the rear reverberation can be used for each position of the speaker unit SU, and a signal obtained by convolving the segment of the rear reverberation with the virtual sound source signal D0 can be reproduced. In addition, the rear reverberation signal generation unit 31 can generate a pseudo rear reverberation signal, and the generated pseudo rear reverberation signal can be convolved with the virtual sound source signal D0.

[0135] <Modification Example>

[0136] Although multiple embodiments of the present disclosure have been specifically described above, the content of the present disclosure is not limited to the above embodiments, and various modifications based on the technical concept of the present disclosure are possible.

[0137] The configurations, methods, steps, shapes, materials, numerical values, etc. described in the above embodiments and modification examples are merely examples, and as needed, configurations, methods, steps, shapes, materials, numerical values, etc. different from those above can be used, or known ones can be used instead of the above configurations, methods, steps, shapes, materials, numerical values, etc. In addition, the configurations, methods, steps, shapes, materials, numerical values, etc. in the embodiments and modification examples can be combined with each other within the range where no technical contradiction occurs.

[0138] It should be noted that the content of the present disclosure is not construed as being limited by the effects of the examples in this specification.

[0139] The present disclosure can also adopt the following configurations. (1)

[0141] A signal processing device, comprising:

[0142] A reverberant sound signal generation unit that generates a reverberant sound signal based on the sound source position of a virtual sound source and the distance to a reference point; and

[0143] A drive signal generation unit that generates a drive signal for a speaker array through a wavefront synthesis filter, where

[0144] The drive signal generation unit generates a drive signal based on a signal obtained by performing wavefront synthesis filtering on a signal obtained by convolving the reverberant sound signal with the signal of the virtual sound source and / or a signal obtained by performing wavefront synthesis filtering on the reverberant sound signal so that the reverberant sound signal becomes a virtual sound source. (2)

[0146] The signal processing device according to (1), wherein

[0147] The drive signal generation unit generates a drive signal by adding a signal obtained by performing wavefront synthesis filtering on the signal of a virtual sound source and a signal obtained by making a reverberant sound signal into a virtual sound source. (3)

[0149] The signal processing device according to (1) or (2), wherein

[0150] The reference point is the position of the listener relative to the sound source position of the virtual sound source. (4)

[0152] The signal processing device according to (1) or (2), wherein

[0153] The reference point is the position where the speaker array has the shortest distance from the sound source position of the virtual sound source. (5)

[0155] The signal processing device according to (1) or (2), wherein

[0156] The reference point is the surface that reflects the signal of the virtual sound source. (6)

[0158] The signal processing device according to any one of (1) to (5), wherein

[0159] By performing wavefront synthesis filtering processing on the reverberant sound signal, a signal obtained by making the reverberant sound signal into a virtual sound source is output, and the number of channels of the obtained signal is less than the number of channels of the speaker array. (7)

[0161] The signal processing device according to (6), wherein

[0162] A process of interpolating a signal obtained by making a reverberant sound signal into a virtual sound source is performed, and sound is output from all speaker units included in the speaker array. (8)

[0164] The signal processing device according to (7), wherein

[0165] The signal reproduced from a predetermined speaker included in the speaker array is copied to generate a signal reproduced from a speaker adjacent to the predetermined speaker. (9)

[0167] The signal processing device according to any one of (1) to (8), wherein

[0168] Generate a rear reverberation sound signal, convert the generated rear reverberation sound signal into a multi-channel signal corresponding to a speaker array, and generate a driving signal by adding the multi-channel signal to the output of a wavefront synthesis filter. (10)

[0170] A signal processing method, comprising:

[0171] A reverberation sound signal generation unit generates a reverberation sound signal based on the sound source position of a virtual sound source and the distance from a reference point; and

[0172] A driving signal generation unit generates a driving signal for a speaker array through a wavefront synthesis filter, where

[0173] The driving signal generation unit generates a driving signal based on a signal obtained by performing wavefront synthesis filtering on a signal obtained by convolving the reverberation sound signal with the signal of the virtual sound source and / or a signal obtained by performing wavefront synthesis filtering on the reverberation sound signal to make the reverberation sound signal a virtual sound source. (11)

[0175] A program for causing a computer to execute a signal processing method, the signal processing method comprising:

[0176] A reverberation sound signal generation unit generates a reverberation sound signal based on the sound source position of a virtual sound source and the distance from a reference point; and

[0177] A driving signal generation unit generates a driving signal for a speaker array through a wavefront synthesis filter, where

[0178] The driving signal generation unit generates a driving signal based on a signal obtained by performing wavefront synthesis filtering on a signal obtained by convolving the reverberation sound signal with the signal of the virtual sound source and / or a signal obtained by performing wavefront synthesis filtering on the reverberation sound signal to make the reverberation sound signal a virtual sound source.

[0179] List of reference numerals

[0180] 1, 1A, 1B, 1C Signal processing device

[0181] 12 Reverberation correction unit

[0182] 14 Wavefront synthesis filter

[0183] 15 DA conversion / amplification unit

[0184] 21 Virtual sound source signal wavefront synthesis filter

[0185] 22 Virtual reverberation signal generation unit

[0186] 23 Virtual Reverberation Signal Wavefront Synthesis Filter

[0187] 24, 24, 34 Adder

[0188] 33 Multi-Channel Processing Unit

[0189] SU Speaker Unit

[0190] SA Speaker Array

Claims

1. A signal processing device, comprising: a reverberant sound signal generation unit that generates a reverberant sound signal based on the sound source position of a virtual sound source and the distance to a reference point; and a drive signal generation unit that generates a drive signal for a speaker array through a wavefront synthesis filter, where the drive signal generation unit generates the drive signal based on a signal obtained by performing wavefront synthesis filtering on a signal obtained by convolving the reverberant sound signal with the signal of the virtual sound source and / or a signal obtained by performing wavefront synthesis filtering on the reverberant sound signal so that the reverberant sound signal becomes the virtual sound source; wherein, by performing wavefront synthesis filtering on the reverberant sound signal, a signal obtained by making the reverberant sound signal become the virtual sound source is output, and the number of channels of the obtained signal is less than the number of channels of the speaker array.

2. The signal processing device according to claim 1, wherein, the drive signal generation unit generates the drive signal by adding a signal obtained by performing wavefront synthesis filtering on the signal of the virtual sound source and a signal obtained by making the reverberant sound signal become the virtual sound source.

3. The signal processing device according to claim 1, wherein, the reference point is the position of the listener relative to the sound source position of the virtual sound source.

4. The signal processing device according to claim 1, wherein, the reference point is the position where the speaker array has the shortest distance from the sound source position of the virtual sound source.

5. The signal processing device according to claim 1, wherein, the reference point is a surface that reflects the signal of the virtual sound source.

6. The signal processing device according to claim 1, wherein, a process of interpolating the signal obtained by making the reverberant sound signal become the virtual sound source is performed so that sound is output from all speaker units included in the speaker array.

7. The signal processing device according to claim 6, wherein, the signal reproduced from a predetermined speaker included in the speaker array is copied to generate a signal reproduced from a speaker adjacent to the predetermined speaker.

8. The signal processing device according to claim 1, wherein, a rear reverberant sound signal is generated, the generated rear reverberant sound signal is converted into a multi-channel signal corresponding to the speaker array, and the drive signal is generated by adding the multi-channel signal and the output of the wavefront synthesis filter.

9. A signal processing method, comprising: a reverberant sound signal generation unit generates a reverberant sound signal based on the sound source position of a virtual sound source and the distance from a reference point; and a drive signal generation unit generates a drive signal for a speaker array through a wavefront synthesis filter, where The driving signal generating unit generates the driving signal based on a signal obtained by performing wavefront synthesis filtering on a signal obtained by convolving the reverberant sound signal with the signal of the virtual sound source and / or a signal obtained by performing wavefront synthesis filtering on the reverberant sound signal to make the reverberant sound signal a virtual sound source; wherein, by performing wavefront synthesis filtering on the reverberant sound signal, a signal obtained by making the reverberant sound signal the virtual sound source is output, and the number of channels of the obtained signal is less than the number of channels of the speaker array.

10. A computer-readable storage medium storing a program for causing a computer to execute a signal processing method, the signal processing method comprising: A reverberant sound signal generating unit generates a reverberant sound signal according to the sound source position of the virtual sound source and the distance from a reference point; and A driving signal generating unit generates a driving signal for a speaker array through a wavefront synthesis filter, wherein The driving signal generating unit generates the driving signal based on a signal obtained by performing wavefront synthesis filtering on a signal obtained by convolving the reverberant sound signal with the signal of the virtual sound source and / or a signal obtained by performing wavefront synthesis filtering on the reverberant sound signal to make the reverberant sound signal a virtual sound source; wherein, by performing wavefront synthesis filtering on the reverberant sound signal, a signal obtained by making the reverberant sound signal the virtual sound source is output, and the number of channels of the obtained signal is less than the number of channels of the speaker array.

Images