Headphone playback device and program
The headphone playback device addresses discontinuity and noise in binaural reproduction by dividing sound source signals and interpolating direction changes, providing a continuous and accurate sound image localization.
Patent Information
- Application Number
- JP2020203004
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-12-07
- Publication Date
- 2024-11-14
- Estimated Expiration
- 2040-12-07
AI Technical Summary
Existing binaural reproduction methods experience discontinuity and noise generation due to inaccuracies in convolution processing when the sound source or listener's head moves, as the temporal resolution of external sensors does not match the movement, leading to mismatched direction assignments in buffer units.
A headphone playback device that divides sound source signals into smaller units, interpolates direction changes based on external sensor data, and convolves head impulse responses with these signals to generate a continuous and accurate reproduction signal.
The solution alleviates discontinuity in the reproduction signal and suppresses noise generation by ensuring accurate direction alignment with movement, resulting in a more natural sound image localization.
Smart Images

Figure 0007583594000005 
Figure 0007583594000006 
Figure 0007583594000007
Abstract
Description
[Technical field]
[0001] The present invention relates to a headphone playback device and a program for presenting a desired acoustic signal (sound image) at a predetermined position by headphone playback. In particular, the present invention relates to a technology for reproducing a sound source signal when the timing of inputting the sound source signal does not coincide with the timing of the sound source movement or head movement in a binaural playback method that convolves a head impulse response into the sound source signal by following the movement of the sound source or the head movement when a sound source signal such as streaming is input sequentially. [Background technology]
[0002] Conventionally, a binaural reproduction method using a head-related transfer function is known as one of the stereophonic reproduction methods (sound image localization technology) using headphones.
[0003] 10 is a diagram for explaining the head-related transfer function. The head-related transfer function is a transfer function (left ear head-related transfer function and right ear head-related transfer function) from the center position of both ears when the head of the listener 101 is not present in the free sound field, through the position of the sound source 100, to the entrance of the left and right ear canals or the position of the eardrum when the head is present.
[0004] As shown in Fig. 10, the head-related transfer function is often approximately defined as a transfer function from the position of a sound source 100 to the entrance of the ear canal or the position of the eardrum of a listener 101. Since the head-related transfer function is a transfer function from the sound source 100 to both ears, it is considered to contribute to the perception of the sense of direction of a sound image. The time domain expression of the head-related transfer function is called a head impulse response.
[0005] The binaural reproduction method is realized by convolving the head impulse response with the sound source signal. Figure 11 is a diagram explaining the binaural reproduction method. The sound source signal output from the sound source 100 reaches the entrance of the right ear canal or the eardrum of the listener 101 through the convolvers 102-1 and 103-1, and also reaches the entrance of the left ear canal or the eardrum through the convolvers 102-2 and 103-2.
[0006] The convolver 102-1 receives the right ear head impulse response X r The convolver 102-2 convolves the left ear head impulse response X l is convolved with the source signal.
[0007] Convolver 103-1 convolves the inverse characteristics of the right-ear headphone into the acoustic signal output from convolver 102-1, and convolver 103-2 convolves the inverse characteristics of the left-ear headphone into the acoustic signal output from convolver 102-2.
[0008] The acoustic signals output from the convolvers 103-1 and 103-2 reach the right and left ear canal entrances or eardrums of the listener 101. The convolvers 103-1 and 103-2 can cancel the effect of headphones when the listener 101 hears sounds.
[0009] For example, referring to FIG. 10, the head impulse response in the left diagonal front direction with respect to the listener 101 is expressed as the right ear head impulse response X r and left ear head impulse response X l If this is the case, then the binaural reproduction method shown in Figure 11 allows a person to perceive a sound image diagonally forward to the left.
[0010] When the sound source 100 moves, the head impulse response is switched according to the positional relationship between the sound source 100 and the head of the listener 101 as the sound source 100 moves, thereby achieving a binaural reproduction method.
[0011] 12 is a diagram for explaining a case where the sound source 100 moves. D1 and D2 respectively indicate the directions from the center position of both ears of the listener 101 to the sound source 100. It is assumed that the position of the listener 101 is fixed and the sound source 100 moves from the direction D1 at time T1 to the direction D2 at time T2.
[0012] In this case, the head impulse response in direction D1 is convolved with the sound source signal from time T1 to just before time T2, and the head impulse response in direction D2 is convolved with the sound source signal at time T2.
[0013] Similarly, when the listener 101 moves, the binaural reproduction method is realized by switching the head impulse response according to the positional relationship between the sound source 100 and the head of the listener 101 as the listener 101 moves.
[0014] 13 is a diagram for explaining a case where the listener 101 moves. D1 indicates the direction from the center position of both ears to the sound source 100 when the listener 101 is at a1, and D2 indicates the direction from the center position of both ears to the sound source 100 when the listener 101 is at a2. It is assumed that the position of the sound source 100 is fixed, and the listener 101 moves from the location a1 where the listener 101 was at time T1 to the location a2 where the listener 101 is at time T2.
[0015] In this case, the head impulse response in direction D1 is convolved with the sound source signal from time T1 to just before time T2, and the head impulse response in direction D2 is convolved with the sound source signal at time T2.
[0016] Incidentally, a method has been proposed for generating a playback signal that smoothly tracks the movement of a sound source 100 even when the sound source 100 moves (see, for example, Patent Document 1). This method detects the position of the sound source 100 by performing head-related transfer function processing in parallel on the sound source signal, and generates a playback signal by interpolating the acoustic signal that has been processed by the head-related transfer function based on the detection result. [Prior art documents] [Patent documents]
[0017] [Patent Document 1] Japanese Patent Application Publication No. 07-248255 Summary of the Invention [Problem to be solved by the invention]
[0018] Conventionally, when the sound source signal and the head impulse response are digital signals, the convolution process is performed in units of samples or in units of a fixed size (256 samples, 1024 samples, etc.). In addition, position parameters that indicate the relative positional relationship due to the movement of the sound source 100 or the listener 101 are acquired as data at a fixed period from an external sensor or the like.
[0019] A device (headphone playback device) that generates a playback signal for presenting a desired sound image at a predetermined position by headphone playback sequentially inputs a sound source signal from a sound source 100 and also inputs a position parameter at a constant period from an external sensor. The headphone playback device then generates a playback signal by convolving a head impulse response in a direction D indicated by the position parameter with a sound source signal in a buffer unit of a predetermined size.
[0020] In this case, the direction D of the head impulse response to be convolved with the sound source signal is switched at the timing when the sound source signal of a buffer unit of a predetermined size is input or at the timing when the position parameter is input.
[0021] When the sound source 100 or the listener 101 moves, the time resolution depends on the resolution of the external sensors that generate their position parameters.
[0022] Referring to FIG. 12, at time T1, the headphone playback device inputs a first position parameter from an external sensor to determine direction D1, and at time T2, as the sound source 100 continues to move, inputs a second position parameter from the external sensor to determine direction D2.
[0023] In this case, even though the position of the sound source 100 changes continuously from time T1 to just before time T2, the convolution process from time T1 to just before time T2 is performed using direction D1.
[0024] 14 is a diagram for explaining time T, direction D, sound source signal buffer B (sound source signal in buffer units), and the assigned direction D in the conventional technology. Time T is set at equal intervals. Direction D is data calculated based on position parameters input from an external sensor, and sound source signal buffer B is a sound source signal in buffer units of a predetermined size input from sound source 100.
[0025] Regarding direction D, directions D1, D5, D9, D13, D17, D21, D25, etc. are obtained based on the respective position parameters input from external sensors at times T1, T5, T9, T13, T17, T21, T25, etc.
[0026] As for the sound source signal, the sound source signal input from sound source 100 from time T1 to time T6 (just before time T7) is buffered as the sound source signal in buffer B1 at time T6. The sound source signal from time T7 to time T12 (just before time T13) is buffered as the sound source signal in buffer B2 at time T12. The sound source signal from time T13 to time T18 (just before time T19) is buffered as the sound source signal in buffer B3 at time T18. The sound source signal from time T19 to time T24 (just before time T25) is buffered as the sound source signal in buffer B4 at time T24.
[0027] In the convolution process, the head impulse response corresponding to the direction D5 stored at the time point is convolved with the sound source signal in the buffer B1 buffered at the timing of the time T6. Therefore, the direction D5 is assigned to the sound source signal in the buffer B1, that is, the sound source signal from the time T1 to the time T6.
[0028] Furthermore, the head impulse response corresponding to direction D9 stored at the time point is convolved with the sound source signal in buffer B2 buffered at the timing of time T12. Therefore, the direction D9 is added to the sound source signal in buffer B2, that is, the sound source signal from time T7 to time T12.
[0029] Furthermore, the head impulse response corresponding to direction D17 stored at that time is convolved with the sound source signal in buffer B3 buffered at the timing of time T18. Therefore, the direction D17 is added to the sound source signal in buffer B3, that is, the sound source signal from time T13 to time T18.
[0030] Furthermore, the head impulse response corresponding to direction D21 held at that time point is convolved with the sound source signal in buffer B4 buffered at the timing of time T24. Therefore, the direction D21 is added to the sound source signal in buffer B4, that is, the sound source signal from time T19 to time T24.
[0031] However, in reality, during the time period between times T1, T5, T9, T13, etc., when the direction D is acquired from the external sensor, for example, the sound source 100 moves continuously between directions D1, D5, D9, D13, etc. Therefore, the direction D assigned to the time period between times T1, T5, T9, T13, etc. is not accurate. For example, the direction D5 assigned to the sound source signal at time T5 is accurate, but the direction D5 assigned to the sound source signal at times T1, T2, T3, T4, T6 is not accurate.
[0032] Furthermore, in a sound source signal in a buffer unit of a predetermined size, the larger the predetermined size (buffer size), the greater the fluctuation in the direction D assigned to the sound source signal. This causes the playback signal generated by the headphone playback device to become discontinuous, i.e., the sound image to become discontinuous, and there is a possibility that unnecessary noise will occur at the seams (changes in the sound source signals in buffer units).
[0033] Therefore, the present invention has been made to solve the above-mentioned problems, and its object is to provide a headphone playback device and program that can mitigate discontinuity in the playback signal and suppress noise generation in a binaural playback method that performs convolution by following the movement of the sound source or the movement of the head. [Means for solving the problem]
[0034] In order to solve the above problem, a headphone playback device according to claim 1 generates a playback signal for headphone playback by convolving a head impulse response from a position of a sound source to a position of a listener with a sound source signal input from the sound source, the headphone playback device comprising: a sound source signal input unit that inputs the sound source signal from the sound source and stores a sound source signal of a buffer unit of a predetermined size in an input buffer; a position parameter input unit that inputs position parameters indicating the positional relationship from an external sensor that measures the positional relationship between the sound source and the listener, calculates a direction from the position of the listener to the position of the sound source based on the position parameters, and stores the direction in the input buffer; a sound source signal division unit that divides the sound source signal of the buffer unit stored in the input buffer into sound source signals of divided buffer units of a predetermined division size; A given number of 2 or more The direction to perform interpolation, extrapolation, or both By , minutes Direction of the buffer unit of and a convolution unit that generates the playback signal by convolving the predetermined head impulse response corresponding to the direction of the divided buffer unit generated by the directional interpolation unit with the sound source signal of the divided buffer unit divided by the sound source signal division unit.
[0036] Furthermore, claims 2 The program of claim 1 to The present invention is characterized in that it functions as the headphone playback device described above. Effect of the Invention
[0037] As described above, according to the present invention, in a binaural reproduction method in which convolution is performed by following the movement of a sound source or the movement of the head, discontinuity in the reproduction signal can be mitigated and the occurrence of noise can be suppressed. [Brief description of the drawings]
[0038] [Figure 1] 1 is a block diagram showing an example of the configuration of a headphone playback device according to an embodiment of the present invention. [Diagram 2] 11 is a diagram for explaining a transition of the units, etc., of a sound source signal stored in an input buffer. FIG. [Diagram 3] 11 is a flowchart showing an example of a reproduction signal generation process performed by a reproduction unit for a sound source signal in buffer units. [Figure 4] FIG. 11 is a diagram illustrating a first example of the interpolation process (step S302). [Diagram 5] FIG. 11 is a diagram illustrating a second example of the interpolation process (step S302). [Figure 6] 1 is a diagram illustrating time T, direction D, buffer B of a sound source signal, divided buffers Bn of a sound source signal, and a given direction D in an embodiment of the present invention. FIG. [Figure 7] FIG. 4 illustrates an example of a data configuration of a memory. [Figure 8] FIG. 13 is a diagram illustrating an example of convolution processing (step S304). [Figure 9] FIG. 13 is a diagram for explaining an example of addition processing of convolution results (step S305). [Figure 10] FIG. 1 is a diagram illustrating a head-related transfer function. [Figure 11] FIG. 1 is a diagram illustrating a binaural reproduction method. [Figure 12] FIG. 13 is a diagram illustrating a case where a sound source moves. [Figure 13] FIG. 13 is a diagram for explaining a case where a listener moves. [Figure 14] FIG. 1 is a diagram for explaining time T, direction D, buffer B of a sound source signal, and the direction D to be added in the prior art. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0039] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. The present invention divides a buffer-based sound source signal into divided buffer units, obtains a direction from the head to the sound source based on a position parameter indicating the positional relationship between the sound source and the head input from an external sensor or the like, and interpolates the direction so as to correspond to the sound source signal of the divided buffer unit. The present invention then generates a playback signal by convolving the head impulse response corresponding to the interpolated direction with the sound source signal of the divided buffer unit.
[0040] This allows the sound image to be perceived as being localized in a more natural direction and reduces unnecessary noise. In other words, in the binaural reproduction method, which performs convolution by following the movement of the sound source or the head movement, discontinuity in the reproduction signal can be mitigated and the generation of noise can be suppressed.
[0041] [Headphone playback device] A headphone playback device according to an embodiment of the present invention will be described below. Fig. 1 is a block diagram showing an example of the configuration of a headphone playback device according to an embodiment of the present invention.
[0042] The headphone playback device 1 includes a sound source signal input unit 10, a position parameter input unit 11, an input buffer 12, a playback unit 13, a memory 14, a playback buffer 15, and a playback signal output unit 16. The playback unit 13 includes a sound source signal division unit 20, a directional interpolation unit 21, and a convolution unit 22.
[0043] The sound source signal input unit 10 sequentially inputs sound source signals such as streaming sound source signals from the sound source 2. The sound source signal input unit 10 waits for the input of a sound source signal in a buffer unit of a predetermined size, and stores the sound source signal in the input buffer 12 in that buffer unit.
[0044] The position parameter input unit 11 inputs position parameters indicating the positional relationship from an external sensor 3 that measures the positional relationship between a sound source 2 (not shown) and the head of the listener 101. Then, the position parameter input unit 11 calculates directional information (hereinafter simply referred to as "direction") from the position of the head of the listener 101 to the position of the sound source 2 based on the position parameters, and stores the direction in an input buffer 12.
[0045] Here, the direction is a vector indicating the angle and distance from the position of the head of the listener 101 to the position of the sound source 2 on two predetermined orthogonal axes. The input position parameters are assumed to change with the movement of the sound source 2 or the movement of the head, and the calculated direction is also assumed to change. Furthermore, the timing at which the sound source signal input unit 10 inputs the sound source signal is assumed to not coincide with the timing at which the sound source 2 moves or the head moves. In other words, the input timing of the sound source signal is assumed to not coincide with the timing at which the external sensor 3 updates and outputs the position parameters (the timing at which the position parameter input unit 11 inputs the position parameters).
[0046] 6, which will be described later, the input buffer 12 stores a direction D at a predetermined time interval, a buffer B of an audio source signal, and a divided buffer Bn of an audio source signal for each time T and an assigned direction D. The buffer B of the audio source signal is an audio source signal in buffer units of a predetermined size, and the divided buffer Bn of an audio source signal for each time T is an audio source signal in divided buffer units of a predetermined division size.
[0047] Fig. 2 is a diagram for explaining the transition of the units of the sound source signal stored in the input buffer 12. As shown in Fig. 2(1), the sound source signal input unit 10 stores the sound source signal from time T1 to time T6 (sound source signal in buffer units of a predetermined size) at the timing of time T=T6. It is assumed that time T is divided at equal intervals. The same applies to Figs. 4, 5 and 6 described later.
[0048] Furthermore, as shown in FIG. 2(1), the position parameter input unit 11 stores direction D=D1 at time T1, and stores direction D5 at time T5.
[0049] Returning to Fig. 1, the reproduction unit 13 divides the sound source signal in units of buffers stored in the input buffer 12, and interpolates the direction D stored in the input buffer 12. The reproduction unit 13 then convolves the head impulse response corresponding to the direction D after the interpolation into the sound source signal in units of divided buffers, thereby generating a reproduction signal for headphone reproduction. Here, interpolation refers to various processes such as interpolation, extrapolation, or both interpolation and extrapolation.
[0050] Specifically, the reproduction unit 13 divides the sound source signal of a buffer unit of a predetermined size stored in the input buffer 12 into sound source signals of divided buffer units of a predetermined division size. Also, the reproduction unit 13 interpolates the direction D stored in the input buffer 12 so that it corresponds to the sound source signal of the divided buffer unit.
[0051] The reproduction unit 13 reads out the head impulse response corresponding to the direction D after the interpolation from the memory 14, convolves the head impulse response with the sound source signal in units of divided buffers, and stores the convolution result in the reproduction buffer 15 by adding the result. The reproduction signal is stored in the reproduction buffer 15. The reproduction unit 13 and the memory 14 will be described in detail later.
[0052] The reproduction signal output unit 16 reads the reproduction signal from the reproduction buffer 15 and outputs the reproduction signal.
[0053] This allows the listener 101 to receive a sound source signal from the sound source 2 in the direction D and play it back as a playback signal through the headphones.
[0054] (Processing of playback unit 13) Next, a detailed description will be given of the processing of the reproduction unit 13 shown in Fig. 1. Fig. 3 is a flowchart showing an example of a reproduction signal generation process performed by the reproduction unit 13 for a sound source signal in buffer units.
[0055] The reproduction unit 13 performs processing of steps S301 to S307 described later each time a buffer unit of a predetermined size of sound source signal is stored in the input buffer 12 (at each of times T6, T12, T18, T24, . . . in FIG. 6 described later).
[0056] (Sound source signal division processing) When the sound source signal division unit 20 of the playback unit 13 determines that a sound source signal of a buffer unit of a predetermined size has been stored in the input buffer 12, it divides the sound source signal into sound source signals of divided buffer units of a predetermined division size, which is the smallest unit (step S301).
[0057] Here, the predetermined size of the sound source signal in the buffer unit of the predetermined size is arbitrarily set in advance. Also, the predetermined division size of the sound source signal in the divided buffer unit of the predetermined division size is arbitrarily set in advance, and may be smaller than the predetermined size, or may be the size of one sample.
[0058] As shown in FIG. 2(2), it is assumed that a buffer-unit sound source signal (buffer B1) is stored in the input buffer 12 at time T6 (that is, a buffer-unit sound source signal is input from the sound source 2).
[0059] Then, as shown in Fig. 2(3), the buffer-based sound source signal (buffer B1) is divided into divided buffer-based sound source signals (divided buffers B1-1,..., B1-6) by the process of step S301. Here, the predetermined size is divided into 6 equal parts, which is the predetermined division size.
[0060] As a result, the input buffer 12 stores the divided buffer unit excitation signal (divided buffer B1-1) at time T1 and the divided buffer unit excitation signal (divided buffer B1-2) at time T2. The input buffer 12 also stores the divided buffer unit excitation signal (divided buffer B1-3) at time T3, ..., and the divided buffer unit excitation signal (divided buffer B1-6) at time T6.
[0061] (Interpolation process in direction D) 3, the direction interpolation unit 21 proceeds from step S301 to read the direction D from the input buffer 12, and performs interpolation using the direction D to obtain the direction D corresponding to the sound source signal in units of divided buffers. Then, the direction interpolation unit 21 stores the direction D after the interpolation in the input buffer 12 (step S302).
[0062] 2(4), in step S302, an interpolation process is performed using directions D1, D2, etc., to obtain directions D2', D3', D4', and D6' at times T2, T3, T4, and T6. The directions D1, D2', . . ., D5, and D6' at times T1, T2, . . ., T5, and T6 correspond to divided buffers B1-1, B1-2, . . ., B1-5, and B1-6, which are sound source signals in divided buffer units, respectively.
[0063] This allows the trajectory of direction D to be interpolated so that the sound source 2 moves smoothly and at a more accurate position as seen by the listener 101 in accordance with the movement of the sound source 2 or the movement of the head, and it is possible to obtain the direction D corresponding to the sound source signal of each divided buffer unit.
[0064] (Example of interpolation processing) Next, an example of the interpolation process in step S302 will be described.
[0065] 4 is a diagram for explaining a first example of the interpolation process (step S302), and shows examples of the interpolation and extrapolation processes. It is assumed that the input buffer 12 stores a direction D13 at time T13, a direction D17 at time T17, and a buffer B3, which is a buffer-based sound source signal, at time T18. It is also assumed that the buffer B3, which is a buffer-based sound source signal, is divided into divided buffers B3-1, B3-2, . . . , B3-6, which are divided buffer-based sound source signals.
[0066] Then, at time T18, the direction interpolation unit 21 assigns directions D14', D15', and D16' at times T14, T15, and T16 by an interpolation process using the directions D13 and D17 at the two previous times T13 and T17. Also, the direction interpolation unit 21 assigns a direction D18' at time T18 by an extrapolation process using the directions D13 and D17.
[0067] Specifically, the direction D13 is expressed as a vector D → 13 , direction D17 is vector D → 17 Then, the direction interpolation unit 21 calculates the vectors D14', D15', D16', and D18' in the directions D14', D15', D16', and D18' using the following equation: → 14 ,D → 15 ,D → 16 ,D → 18 Request.
number
[0068] 5 is a diagram for explaining a second example of the interpolation process (step S302), and shows an example of the interpolation process. It is assumed that direction D13 is stored in the input buffer 12 at time T13, direction D17 is stored at time T17, and buffer B3, which is a buffer-based sound source signal, is stored at time T18. It is also assumed that buffer B3, which is a buffer-based sound source signal, is divided into divided buffers B3-1, B3-2, . . . , B3-6, which are divided buffer-based sound source signals. It is also assumed that direction D21 is stored at time T21.
[0069] Then, after time T18, the directional interpolation unit 21 waits for the direction D21 at time T21 to be stored in the input buffer 12. Then, the directional interpolation unit 21 assigns directions D14', D15', D16', D18', D19', and D20' at times T14, T15, T16, T18, T19, and T20 by an interpolation process using the directions D13, D17, and D21 at the past three times T13, T17, and T21.
[0070] In other words, at time T18, the direction interpolation unit 21 assigns directions D14', D15', and D16' by interpolation processing using directions D13 and D17, and at time T21, assigns directions D18', D19', and D20' by interpolation processing using directions D17 and D21.
[0071] Specifically, the direction D13 is expressed as a vector D → 13 , direction D17 is vector D → 17 , direction D21 is vector D → 21 The direction interpolation unit 21 calculates vectors D in the directions D14', D15', D16', D18', D19', and D20' using the following formula: → 14 ,D → 15 ,D → 16 ,D → 18 ,D → 19 ,D → 20 Request.
number
[0072] As described above, at time T18, directions D14’, D15’, and D16’ are obtained by interpolation processing using directions D13 and D17, and at time T21, directions D18’, D19’, and D20’ are obtained by interpolation processing using directions D17 and D21. Therefore, since the interpolation processing of directions D18’, D19’, and D20’ is performed after waiting for time T21 when direction D21 is stored in the input buffer 12, a processing delay occurs.
[0073] Incidentally, the direction interpolation unit 21 may perform interpolation by linear prediction using a plurality of past directions D according to the following formula.
Equation
[0074] In the formula (3), the direction interpolation unit 21 compares the direction D stored in the input buffer 12 in the past with the direction D obtained by the interpolation processing, and the error e i is minimized so that the weight ω i-j is updated. The error e i in this case is calculated by, for example, the following formula when using the least squares error.
Equation
[0075] Alternatively, the direction interpolation unit 21 may perform spline interpolation using the past direction D.
[0076] In this way, by the interpolation processing of step S302 in FIG. 3 by the direction interpolation unit 21, the input buffer 12 is in a state where the direction D corresponding to each sound source signal in units of divided buffers (for each time T) is stored.
[0077] 6 is a diagram for explaining time T, direction D, buffer B of the sound source signal (sound source signal in units of buffers), divided buffers Bn of the sound source signal (sound source signal in units of divided buffers), and the assigned direction D in the embodiment of the present invention. The time T, direction D, etc. shown in FIG. 6 correspond to the time T, direction D, etc. in the conventional technology shown in FIG.
[0078] For direction D, directions D1, D5, D9, D13, D17, D21, D25, etc. are stored in the input buffer 12 at times T1, T5, T9, T13, T17, T21, T25, etc., as in FIG.
[0079] As for the sound source signal, buffers B1, B2, B3, B4, etc., which are sound source signals in units of buffers, are stored in the input buffer 12 at times T6, T12, T18, T24, etc., as in FIG.
[0080] In the embodiment of the present invention, the sound source signal in buffer units is divided into sound source signals in divided buffer units by the sound source signal division unit 20. As a result, as shown in Fig. 6, the input buffer 12 stores divided buffer B1-1 at time T1, divided buffer B1-2 at time T2, ..., divided buffer B5-2 at time T26, etc.
[0081] Moreover, in the embodiment of the present invention, the direction D is interpolated by the direction interpolation unit 21. As a result, as shown in Fig. 6, the direction D1 at time T1, the direction D2' at time T2, ..., the direction D26' at time T26, etc. are stored.
[0082] (Convolution processing of head impulse response) Returning to FIG. 3, the convolution unit 22 proceeds from step S302 and sets (time T in) the first divided buffer unit of the excitation signal in the buffer unit excitation signal as the processing target (step S303).
[0083] In the example of FIG. 2(4), in the buffer B1 which is a buffer unit of the sound source signal from time T1 to time T6, the divided buffer B1-1 which is the first divided buffer unit of the sound source signal is set as the processing target.
[0084] The convolution unit 22 reads out the divided buffer Bn which is the sound source signal of the divided buffer unit to be processed from the input buffer 12, and also reads out the direction D corresponding to the sound source signal of the divided buffer unit to be processed. The convolution unit 22 also reads out the head impulse response corresponding to the direction D from the memory 14. Then, the convolution unit 22 convolves the head impulse response corresponding to the direction D into the divided buffer Bn which is the sound source signal of the divided buffer unit (step S304).
[0085] That is, for each sound source signal in divided buffer units, the convolution unit 22 convolves the head impulse response corresponding to the direction D at the time T into the sound source signal in divided buffer units at the time T.
[0086] 7 is a diagram showing an example of the data configuration of the memory 14. The memory 14 stores a direction D and a head impulse response corresponding to the direction D. It is assumed that the head impulse response corresponding to the direction D is measured in advance and stored in the memory 14. The head impulse response is a right ear head impulse response X r and left ear head impulse response X l The convolution unit 22 reads out the head impulse response corresponding to the direction D.
[0087] 3, the convolution unit 22 adds the convolution result obtained in step S304 to the data (stored data) already stored in the playback buffer 15 (step S305). As a result, the convolution result is stored in the playback buffer 15. As such a process is performed for each sound source signal in units of divided buffers, a playback signal is stored in the playback buffer 15.
[0088] FIG. 8 is a diagram for explaining an example of the convolution process (step S304), and FIG. 9 is a diagram for explaining an example of the addition process of the convolution results (step S305).
[0089] When the sound source signal and the head impulse response are digital signals, the convolution process is performed in sample units of a predetermined division size or in divided buffer units of a fixed size (256 samples, 1024 samples, etc.).
[0090] 8, in step S304, the convolution unit 22 convolves the left ear head impulse response corresponding to direction D13 at time T13 into divided buffer B3-1, which is the sound source signal in divided buffer units at time T13, to obtain a left ear convolution result.
[0091] Furthermore, in step S304, the convolution unit 22 convolves the right ear head impulse response corresponding to the direction D13 at time T13 into the divided buffer B3-1, which is the sound source signal in divided buffer units at time T13, to obtain a right ear convolution result.
[0092] Here, the divided buffer B3-1 is a sound source signal of N samples, and the left ear head impulse response and the right ear head impulse response are each a response of M samples. In this case, the number of samples of the left ear convolution result and the right ear convolution result is (M+N-1).
[0093] Then, in step S305, the convolution unit 22 adds the left ear convolution result to the already stored data based on the position in the playback buffer 15 that corresponds to divided buffer B3-1 at time T13. By performing such processing for each sound source signal in divided buffer units, the left ear playback signal is stored in the playback buffer 15.
[0094] In step S305, the convolution unit 22 adds the right ear convolution result to the already stored data based on the position in the playback buffer 15 that corresponds to the divided buffer B3-1 at time T13. By performing such processing for each sound source signal in divided buffer units, the right ear playback signal is stored in the playback buffer 15.
[0095] The process of adding the convolution results will be described with reference to Fig. 9. Although the left ear convolution results and the right ear convolution results are not specifically shown in Fig. 9, the process is applied to both.
[0096] In step S304 for divided buffer B3-1, which is the sound source signal in divided buffer units at time T13, the convolution unit 22 convolves the head impulse response corresponding to direction D13 into divided buffer B3-1 to obtain a convolution result C1.
[0097] Then, in step S305 for divided buffer B3-1, the convolution unit 22 adds the convolution result C1 to the already stored data based on position k in the playback buffer 15. As a result, the result of the addition process using the convolution result C1 is stored in the area from position k to position β in the playback buffer 15.
[0098] In step S304 for divided buffer B3-2, which is the sound source signal in divided buffer units at time T14, the convolution unit 22 convolves the head impulse response corresponding to direction D14 into divided buffer B3-2 to obtain a convolution result C2.
[0099] Then, in step S305 for divided buffer B3-2, the convolution unit 22 adds the convolution result C2 to the already stored data based on the position (k+N) in the playback buffer 15. As a result, the result of the addition process using the convolution result C2 is stored in the area in the playback buffer 15 from the position (k+N) to the position α.
[0100] Here, the process of step S305 for split buffer B3-2 will be described in detail. It is assumed that the results of the addition process using the convolution result C1 are stored in the area from position k to position β in the playback buffer 15. In other words, it is assumed that the addition results for split buffer units up to split buffer B3-1 are stored in the area from position o to position β in the playback buffer 15.
[0101] With the playback buffer 15 in this state, the convolution unit 22 adds the convolution result C2 of (M+N-1) samples to the stored data in the area from position (k+N) to position α, with position (k+N) as the reference position.
[0102] Specifically, the convolution unit 22 adds signals of (M-1) samples in the front part on the time axis out of the convolution result C2 of (M+N-1) samples to stored data of (M-1) samples in the area from position (k+N) to position β in the playback buffer 15. The convolution unit 22 also stores signals of N samples in the rear part on the time axis out of the convolution result C2 of (M+N-1) samples in the area from position β+1 to position α in the playback buffer 15 (the area from position β+1 to position α does not store data, but it is assumed that null data is stored, so the process of storing signals of N samples in the front part on the time axis in the area from position β+1 to position α is equivalent to the process of adding signals of the N samples to stored data (null data) from position β+1 to position α).
[0103] Returning to FIG. 3, the convolution unit 22 proceeds from step S305 and determines whether or not the processes of steps S304 and S305 have been completed for the sound source signals of all divided buffer units (step S306).
[0104] When it is determined in step S306 that the processing has not been completed for all the divided buffer units of the sound source signals (step S306: N), the convolution unit 22 sets the next divided buffer unit of the sound source signal as the processing target (step S307). Then, the convolution unit 22 proceeds to step S304 and performs the processing of steps S304 and S305.
[0105] On the other hand, if it is determined in step S306 that the processing has been completed for all the divided buffer-unit excitation signals (step S306: Y), the convolution unit 22 ends the processing for the buffer-unit excitation signal.
[0106] In this way, the process of steps S301 to S307 is performed for each excitation signal in a buffer unit, and the process of steps S304 to S307 is performed for each excitation signal in a divided buffer unit. The process of steps S304 to S307, which is repeatedly performed for the excitation signal in a divided buffer unit, is performed the same number of times as the number of the excitation signals in divided buffer units divided in step S301.
[0107] As described above, according to the headphone playback device 1 of the embodiment of the present invention, the sound source signal input unit 10 stores the sound source signal input from the sound source 2 in buffer units in the input buffer 12. In addition, the position parameter input unit 11 calculates the direction D from the position of the listener 101 to the position of the sound source 2 based on the position parameter input from the external sensor 3 and indicating the positional relationship between the sound source 2 and the head of the listener 101, and stores the direction D in the input buffer 12.
[0108] The sound source signal division unit 20 of the playback unit 13 divides the sound source signal of a buffer unit stored in the input buffer 12 into sound source signals of divided buffer units, and the direction interpolation unit 21 interpolates the direction D stored in the input buffer 12 so that it corresponds to the sound source signal of the divided buffer unit.
[0109] By interpolating the direction D, it is possible to obtain the direction D corresponding to the sound source signal in units of divided buffers so that the sound source 2 moves smoothly and at a more accurate position as seen by the listener 101 in accordance with the movement of the sound source 2 or the movement of the head.
[0110] The convolution unit 22 convolves the interpolated head impulse response corresponding to direction D with the sound source signal in units of divided buffers to generate a playback signal, which is stored in the playback buffer 15. The playback signal output unit 16 reads out the playback signal from the playback buffer 15 and outputs it.
[0111] This makes it possible to mitigate discontinuity in the playback signal and suppress the generation of noise even if the timing of sequential input of the sound source signal does not coincide with the timing of movement of the sound source 2 or the head movement, i.e., the timing of discretely inputting position parameters to calculate the direction D.
[0112] Moreover, the headphone playback device 1 according to the embodiment of the present invention is premised on using the direction D stored in the memory 14 and the head impulse response corresponding to the direction D.
[0113] The head impulse response corresponding to this direction D is a response from the position of the sound source 2 to the position of the head of the listener 101, and is measured in advance, for example, by placing a microphone at the position of the head and automatically and continuously moving the microphone position. Therefore, the head impulse response corresponding to the direction D can be measured as data corresponding to various directions D, and various directions D and the head impulse responses corresponding to the directions D can be stored in the memory 14.
[0114] On the other hand, when the head impulse response cannot be measured from various directions D, it is necessary to interpolate the measured head impulse response. Since the head impulse response interpolation process needs to take into account the level difference and time difference for each frequency (see Patent Document 1 mentioned above), it has a higher processing load and requires more processing time than the interpolation process for direction D (angle and distance) according to the embodiment of the present invention.
[0115] Therefore, the headphone playback device 1 according to the embodiment of the present invention uses head impulse responses measured in advance, that is, head impulse responses corresponding to various directions D stored in the memory 14. In other words, the headphone playback device 1 does not interpolate head impulse responses corresponding to the direction D, but instead interpolates the direction D and then reads out the head impulse response corresponding to the interpolated direction D from the memory 14.
[0116] This makes it possible to effectively utilize head impulse responses corresponding to various directions D obtained by pre-measurement without having to perform interpolation of head impulse responses, which is a heavy load, and enables practical operation of the headphone playback device 1 that realizes the binaural playback method.
[0117] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments and can be modified in various ways without departing from the technical concept thereof.
[0118] For example, in the above embodiment, the position parameter input unit 11 of the headphone playback device 1 is configured to input the position parameters from the external sensor 3. The present invention is not limited to inputting the position parameters from the external sensor 3, and the position parameters may be input from a device other than the external sensor 3.
[0119] A normal computer can be used as the hardware configuration of the headphone playback device 1 according to the embodiment of the present invention. The headphone playback device 1 is configured by a computer equipped with a CPU, a volatile storage medium such as a RAM, a non-volatile storage medium such as a ROM, an interface, etc.
[0120] The functions of the sound source signal input unit 10, position parameter input unit 11, input buffer 12, playback unit 13, memory 14, playback buffer 15 and playback signal output unit 16 provided in the headphone playback device 1 are each realized by having a CPU execute a program that describes these functions.
[0121] These programs are stored in the storage medium and are read and executed by the CPU. These programs can also be distributed by storing them in a storage medium such as a magnetic disk (floppy disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), or a semiconductor memory, and can also be transmitted and received via a network. [Industrial Applicability]
[0122] The headphone playback device 1 and the program according to the embodiment of the present invention are useful in the technical field related to sound field reproduction by headphones, and in particular as a technique related to binaural playback using a head-related transfer function (head-related impulse response). [Explanation of symbols]
[0123] 1 Headphone playback device 2,100 sources 3 External Sensors 10. Audio signal input section 11 Position parameter input section 12 Input Buffer 13 Playback Department 14. Memory 15 Playback Buffer 16 Playback signal output section 20 Sound source signal division section 21 Directional Interpolation Section 22 Convolution section 101 listeners 102-1, 102-2, 103-1, 103-2 Convolver T time B Buffer Bn Split buffer D direction
Claims
1. 1. A headphone playback device that generates a playback signal for headphone playback by convolving a head impulse response from a position of a sound source to a position of a listener with a sound source signal input from the sound source, a sound source signal input unit that receives the sound source signal from the sound source and stores the sound source signal in a buffer unit of a predetermined size in an input buffer; a position parameter input unit that receives a position parameter indicating the positional relationship from an external sensor that measures the positional relationship between the sound source and the listener, calculates a direction from the listener's position to the sound source's position based on the position parameter, and stores the direction in the input buffer; an excitation signal division unit that divides the buffer-unit excitation signal stored in the input buffer into divided buffer-unit excitation signals of a predetermined division size; a direction interpolation unit that generates a direction for each divided buffer by performing interpolation, extrapolation, or both interpolation and extrapolation using a predetermined number of the directions stored in the input buffer, the direction interpolation unit being configured to generate a direction for each divided buffer; a convolution unit that generates the playback signal by convolving the predetermined head impulse response corresponding to the direction of the divided buffer unit generated by the direction interpolation unit with the sound source signal of the divided buffer unit divided by the sound source signal division unit; A headphone playback device comprising:
2. A program for causing a computer to function as the headphone playback device according to claim 1.
Citation Information
Patent Citations
Method and apparatus for forming stereophonic image
JP1995248255A
Three-dimensional moving sound generating method and its apparatus
JP2003078999A
Acoustic processing apparatus and method, and program
JP2016039568A
Information processing apparatus and information processing method
JP2018152669A
