Binaural reproduction device and program

Abstract

To present a sound image with a higher sense of presence and realism by considering the acoustic propagation of sound waves in each direction arriving at a listening position when a sound source object has radiation characteristics.SOLUTION: A harmonic expansion unit 10 of a binaural reproduction device 1 performs circular harmonic expansion of a pre-measured binaural head transfer function to obtain expansion coefficients An(L)(ω), An(R)(ω). A harmonic expansion unit 11 performs a circular harmonic expansion of sound signals of a source object 100 having sound collection radiation characteristics and obtains an expansion coefficient An(1)(ω). A conversion unit 12 converts the expansion coefficient An(1)(ω) to an expansion coefficient An(2)(ω) at a listening position by the additive theorem of the Bessel function. A binaural signal calculation unit 13 calculates the binaural signals using the expansion coefficients An(L)(ω), An(R)(ω), An(2)(ω), etc., by the Kirchhoff-Helmholtz integral equation.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to a reproduction technique for presenting, using headphones, an audio signal when a sound source object has radiation characteristics, and in particular to a binaural reproduction device and program for stereophonic sound using a head-related transfer function. [Background technology]

[0002] Conventionally, in audiovisual content, a sound source may be treated as an object (sound source object) that exists at a certain position. As a method for stereoscopically reproducing the sound emitted by this sound source object through headphones, there is a binaural reproduction method using a head-related transfer function (see, for example, Patent Document 1 and Non-Patent Document 1).

[0003] 12 is a diagram for explaining an outline of the binaural reproduction method. In a propagation path from a sound source position 101 of a sound source object 100 to a head position 103 of a listener 102, an arrow from the sound source position 101 to a left ear 104 of the listener 102 indicates a head-related transfer function of the left ear 104. Also, an arrow from the sound source position 101 to a right ear 105 of the listener 102 indicates a head-related transfer function of the right ear 105.

[0004] The head-related transfer function represents the acoustic propagation characteristics from the sound source position 101 to the eardrum positions of the left ear 104 and the right ear 105. The spectrum of the audio signal is multiplied by this head-related transfer function, or the time domain representation of the audio signal is convolved with a head impulse response, which is a time domain representation of the head-related transfer function, to obtain the resulting audio signal. The audio signal is then presented to the left ear 104 and right ear 105 by headphones or earphones, allowing the listener 102 to perceive a sound image localized at the sound source position 101.

[0005] In such a binaural reproduction method, it is assumed that the sound waves radiated from the sound source object 100 have omnidirectional characteristics that do not change depending on the direction. Therefore, the position of the sound source object 100 is given as a point, and only the head-related transfer function from the sound source position 101 at that point to the listening positions of the left ear 104 and the right ear 105 is multiplied by the audio signal to generate a signal to be presented to the left ear 104 and the right ear 105, that is, a so-called binaural signal.

[0006] In other words, when the sound source object 100 has omnidirectional characteristics, the binaural signals to be presented to the left ear 104 and right ear 105 are calculated taking into account acoustic propagation from the sound source position 101 to the listening positions of the left ear 104 and right ear 105. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Patent Publication No. 2022-90546 [Non-patent literature]

[0008] [Non-Patent Document 1] Ohga, Yamazaki, Kaneda, "Acoustic Systems and Digital Processing", Institute of Electronics, Information and Communication Engineers, 1995 Summary of the Invention [Problem to be solved by the invention]

[0009] However, when the sound waves emitted by the sound source object 100 change depending on the direction, i.e., when the sound source object 100 has a radiation characteristic, the binaural reproduction method used when the sound source object 100 has an omnidirectional characteristic cannot be directly applied.

[0010] This is because the binaural signal needs to be calculated taking into account not only the acoustic propagation from the sound source position 101 to the listening positions of the left ear 104 and right ear 105, but also the acoustic propagation of sound waves in each direction that are radiated into space based on the radiation characteristics and arrive at the listening positions.

[0011] Therefore, the present invention has been made to solve the above-mentioned problems, and its object is to provide a binaural playback device and program that can present a more realistic and real sound image by taking into account the acoustic propagation of sound waves from each direction arriving at the listening position when a sound source object has radiation characteristics. [Means for solving the problem]

[0012] In order to solve the above problem, the binaural playback device of claim 1 is a binaural playback device that calculates an audio signal as a binaural signal when a sound source object has radiation characteristics, and is characterized in that it comprises: a first harmonic expansion unit that inputs a head transfer function of both ears of a listener that has been measured in advance, and obtains expansion coefficients of the head transfer function by performing a circular harmonic expansion or a spherical harmonic expansion of the head transfer function; a conversion unit that inputs an expansion coefficient of a sound wave radiated from the sound source object that has been measured in advance, and converts the expansion coefficients of the sound waves radiated from the sound source object into expansion coefficients of sound waves arriving at the listening position of the listener according to the addition theorem of Bessel functions; and a binaural signal calculation unit that calculates the binaural signal using the expansion coefficients of the head transfer function obtained by the first harmonic expansion unit and the expansion coefficients of the sound waves arriving at the listening position of the listener converted by the conversion unit based on the Kirchhoff-Helmholtz integral equation.

[0013] Furthermore, the binaural playback device of claim 2 is the binaural playback device of claim 1, further comprising a second harmonic expansion unit that inputs an audio signal of the collected sound waves radiated from the sound source object and obtains the expansion coefficients of the sound waves radiated from the sound source object by performing a circular harmonic expansion or a spherical harmonic expansion of the audio signal, and the conversion unit converts the expansion coefficients of the sound waves radiated from the sound source object obtained by the second harmonic expansion unit into the expansion coefficients of sound waves arriving at the listening position of the listener, using the addition theorem of Bessel functions.

[0014] Furthermore, the binaural playback device of claim 3 is characterized in that, in the binaural playback device of claim 1 or 2, the binaural signal calculation unit calculates the binaural signal using a predetermined equation derived by substituting the expansion coefficients of the head-related transfer function and the gradients of the expansion coefficients, as well as the sound pressure and sound pressure gradient on a circular boundary centered on the listener's head, which are obtained from the expansion coefficients of the sound waves arriving at the listener's listening position, into the Kirchhoff-Helmholtz integral equation.

[0015] In addition, the binaural reproduction device of claim 4 is the binaural reproduction device according to claim 2, wherein when the circular harmonic expansion is performed by the first harmonic expansion section and the second harmonic expansion section, the binaural signal calculation section defines ω as an angular frequency, TIFF2025038778000002.tif8170 is the position centered on the listener's head, r is the distance from the center of the head, and p (L) (ω,r) is the signal presented to the left ear, p (R) (ω,r) is the signal presented to the right ear, k is the wave number, N is the finite expansion order in the circular harmonic expansion, A n (2) (ω) is the expansion coefficient of the sound wave arriving at the listening position of the listener, A n (L) (ω) is the expansion coefficient of the head-related transfer function of the left ear, A n (R) (ω) is the expansion coefficient of the head-related transfer function of the right ear, H n (2) is the Hankel function of the second kind, J n is the Bessel function, and the following equation: The binaural signal is calculated using TIFF2025038778000003.tif35170TIFF2025038778000004.tif35170.

[0016] Furthermore, a program according to claim 5 causes a computer to function as the binaural reproduction device according to claim 1. Effect of the Invention

[0017] As described above, according to the present invention, when a sound source object has radiation characteristics, a more realistic and real sound image can be presented by taking into account the acoustic propagation of sound waves arriving at the listening position from each direction. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 is a diagram illustrating a polar coordinate system (spherical coordinate system). [Diagram 2] FIG. 2 is a diagram illustrating a spherical harmonic function. [Diagram 3] FIG. 2 is a diagram illustrating a cylindrical coordinate system. [Figure 4] FIG. 1 is a diagram illustrating a circular harmonic function. [Diagram 5] 1A and 1B are diagrams illustrating conversion of expansion coefficients based on the addition theorem of Bessel functions. [Figure 6] FIG. 1 is a conceptual diagram illustrating the Kirchhoff-Helmholtz integral equation. [Figure 7] 1 is a diagram illustrating the center Oq of the listener's head position, the measurement distance rh, the measurement azimuth angle φh, the boundary radius rq, and the boundary azimuth angle φq. [Figure 8] 1 is a block diagram showing an example of the configuration of a binaural reproduction device according to an embodiment of the present invention. [Figure 9] 13 is a flowchart illustrating an example of processing performed by the binaural reproduction device. [Figure 10] FIG. 13 is a diagram showing an example of picking up a sound signal from a sound source object having radiation characteristics when microphones are arranged two-dimensionally. [Figure 11] FIG. 13 is a diagram showing an example of picking up a sound signal from a sound source object having radiation characteristics when microphones are arranged three-dimensionally. [Figure 12] FIG. 1 is a diagram for explaining an overview of a binaural reproduction method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Summary of the Invention First, an overview of the present invention will be described. The present invention is characterized in that a sound pressure distribution on a boundary set in the vicinity of a listening position away from a sound source object is estimated based on the radiation characteristics of a sound source object, and the sound pressure distribution on the boundary is used to calculate the sound pressure at the binaural positions, i.e., the binaural signal that is an audio signal presented to both ears, based on the Kirchhoff-Helmholtz integral equation.

[0020] More specifically, the present invention obtains the radiation characteristics of a sound source object based on data (metadata) related to the radiation of a collected sound signal or a transmitted sound wave, estimates sound waves arriving at a boundary set near a listening position by the sound waves radiated from the sound source object, and estimates the sound pressure distribution on the boundary. Then, the binaural signal is calculated by multiplying each arriving sound wave by a head-related transfer function from each point on the boundary to the eardrum position of both ears and integrating on the boundary. These processes are performed in the spherical harmonic domain or the circular harmonic domain, and the integration on the boundary is replaced by a product-sum operation (see, for example, equations (13) and (14) described later).

[0021] In this case, a head-related transfer function is used as a Green's function that represents the propagation characteristics of sound waves from each point on the boundary to the position of each ear.

[0022] This makes it possible to present a more realistic and real sound image compared to the general binaural reproduction method in which the head-related transfer function from the sound source position to the binaural positions is convolved with the audio signal.

[0023] Next, the technical elements used to calculate the binaural signal of the present invention when the sound source object has radiation characteristics will be described.

[0024] (Spherical harmonic expansion and circular harmonic expansion) First, the spherical harmonic expansion and the circular harmonic expansion will be explained. From a physical perspective, sound waves follow the wave equation. The spherical harmonic expansion and the circular harmonic expansion correspond to the spatial Fourier transform based on the solution of the wave equation in the polar coordinate system (spherical coordinate system) and the cylindrical coordinate system.

[0025] Figure 1 is a diagram explaining the polar coordinate system (spherical coordinate system). In contrast to the Cartesian coordinate system of space represented by the xyz axes, the polar coordinate system is a method of expressing the radius (radius vector, distance) of a sphere centered at the origin as r, the azimuth angle as φ, and the elevation angle as θ.

[0026] In this polar coordinate system, a sound source (sound source object) is assumed to be inside the radius r, and there is no sound source outside. When expressing the sound field p(ω,r,φ,θ) outside the radius r, the solution is expressed by the following equation.

number

[0027] where ω is the angular frequency, k r is the radial wave number (k r =ω / c), c is the speed of sound. n,m are the expansion orders, A n,m is the expansion coefficient, h n (2) is the second kind of spherical Hankel function, Y n m is a spherical harmonic function.

[0028] FIG. 2 is a diagram for explaining the spherical harmonic function. As shown in FIG. 2, the spherical harmonic function Y n m is a function that expresses the shape of a sphere, ellipsoid, etc., using expansion orders n and m (n is the order and m is the order).

[0029] In the above formula (1), the infinite-order addition is usually approximated by a finite order N. The above formula (1) is expressed by the following formula.

number

[0030] This equation (2) expresses the sound field p(ω,r,φ,θ) formed by the sound waves emitted from the sound source as the basis spherical harmonic function Y n m It is shown that the basis components are the second kind of spherical Hankel functions h n (2) This shows that the decay rate is proportional to the

[0031] Fig. 3 is a diagram explaining a cylindrical coordinate system. The cylindrical coordinate system is a method of expressing the radius of a cylinder (radius vector, distance) as r and the azimuth angle as φ in an orthogonal coordinate system of space represented by the xyz axes.

[0032] In this cylindrical coordinate system, there is a sound source inside the radius r, and there is no sound source outside the radius r. When expressing the sound field p(ω,r,φ,z) outside the radius r, the solution is expressed by the following equation.

number

[0033] where k z is the wave number in the z-axis direction, A n is the expansion coefficient, H n (2) is the Hankel function of the second kind, e jnφ is a circular harmonic function.

[0034] FIG. 4 is a diagram for explaining the circular harmonic function. As shown in FIG. 4, the circular harmonic function e jnφ is a function that represents a shape such as a circle or an ellipse, determined by the expansion order n.

[0035] In order to simplify the above equation (3), the target sound field p(ω,r,φ,z) is limited to two dimensions in the horizontal plane (xy plane) of the cylindrical coordinate system shown in Figure 3, and the z-axis dependency is considered to be uniform. z and z are omitted. Then, the above formula (3) is expressed as follows:

number

[0036] In the above formula (4), the infinite-order addition is usually approximated by a finite order N. In this case, the above formula (4) is expressed by the following formula.

number

[0037] This equation (5) expresses the sound field p(ω,r,φ) formed by the sound wave radiated from the sound source, under the assumption that the dependence on the z axis is uniform, as a function of the circular harmonic functions e jnφ It is shown that the basis components are the second kind Hankel functions H n (2) This shows that the decay rate is proportional to the

[0038] Thus, the sound field p(ω,r,φ,θ) in the above formula (2) is expressed by the second kind of spherical Hankel function h n (2) , a spherical harmonic function Y with respect to azimuth angle φ and elevation angle θ n m , and its expansion coefficient A n,m The sound field p(ω,r,φ) in the above formula (5) can be expressed as the second kind Hankel function H n (2) , the circular harmonic function e with respect to the azimuth angle φ jnφ , and its expansion coefficient A n It can be described separately as follows.

[0039] Given sound fields p(ω,r,φ,θ),p(ω,r,φ) due to sound waves emitted from a sound source object, describing these sound fields p(ω,r,φ,θ),p(ω,r,φ) separately is called a spherical harmonic expansion or a circular harmonic expansion.

[0040] Specifically, the sound field p(ω,r,φ,θ) is expressed as two kinds of spherical Hankel functions h n (2) , spherical harmonic function Y n m and expansion coefficient A n,mThe description of the sound field p(ω,r,φ) in terms of the second kind Hankel function H n (2) , the circular harmonic function e jnφ and expansion coefficient A n Separating it into these and describing it is called a circular harmonic expansion.

[0041] (Addition theorem of Bessel functions) Next, the addition theorem of the Bessel function will be described. Fig. 5 is a diagram for explaining the conversion of the expansion coefficients based on the addition theorem of the Bessel function.

[0042] As shown in FIG. 5, according to the addition theorem of Bessel functions, the expansion coefficient A (1) (=A (1) n Or A (1) n,m ) is the expansion coefficient A obtained by expanding the sound wave arriving at another position 106 (head position 103 of the listener 102) into a spherical harmonic expansion or a circular harmonic expansion. (2) (=A (2) n Or A (2) n,m ) can be converted to

[0043] The expansion coefficients of the sound wave radiated from the sound source position 101 in the spherical harmonic expansion are A n,m (1) , the coefficients of the spherical harmonic expansion of the sound wave arriving at another position 106 are A n,m (2) If the distance between the sound source position 101 and another position 106 is r1, the azimuth angle which is the direction they form is φ1, and the elevation angle is θ1, then the relationship is expressed by the following function.

number

[0044] In addition, the expansion coefficients of the sound wave emitted from the sound source position 101 in the circular harmonic expansion are A n (1)The coefficients of the circular harmonic expansion of the sound wave arriving at another position 106 are A n (2) If the distance between the sound source position 101 and another position 106 is r1 and the azimuth angle therebetween is φ1, the relationship is expressed by the following function.

number

[0045] Incidentally, the functions f and F are known processes for converting the expansion coefficients according to the addition theorem of Bessel functions, and therefore a detailed description thereof will be omitted here.

[0046] (Kirchhoff-Helmholtz integral equation) Next, the Kirchhoff-Helmholtz integral equation will be described. Fig. 6 is a conceptual diagram for explaining the Kirchhoff-Helmholtz integral equation.

[0047] As shown in Fig. 6, the position inside the region Ω that does not contain a sound source is In the following, variables that represent vectors are written in bold in the formulas, and in the text, the variable name is written in parentheses after the vector (e.g., position r(vector)). Also, the sound pressure at position r(vector) is Let's say TIFF2025038778000013.tif10170.

[0048] Then, the sound pressure p(r(vector)) is expressed by the following integral equation on the boundary ∂Ω:

number

[0049] Where: TIFF2025038778000015.tif8170(r'(vector)) is a point (position) on the boundary ∂Ω. TIFF2025038778000016.tif7170(n'(vector)) is a vector in the normal direction of the boundary ∂Ω (normal vector). TIFF2025038778000017.tif15170 is the normal derivative (sound pressure gradient) of sound pressure p(r' (vector)) at position r' (vector). TIFF2025038778000018.tif9170 is the Green's function that represents how sound waves propagate from position r' (vector) to position r (vector), TIFF2025038778000019.tif14170 is the derivative (gradient) in the normal direction.

[0050] The Kirchhoff-Helmholtz integral equation shown in equation (8) above indicates that the sound pressure p(r(vector)) inside the region Ω can be obtained by using the sound pressure p(r'(vector)) at each point on the boundary ∂Ω (first data), the sound pressure gradient in the normal direction ∂p(r'(vector)) / ∂n'(vector) (second data), the Green's function G(r(vector)|r'(vector)) from each point on the boundary ∂Ω to an internal position (third data), and the gradient in the normal direction ∂G(r(vector)|r'(vector)) / ∂n'(vector) (fourth data).

[0051] In the present invention, the Green's function G(r(vector)|r'(vector)) is used as the head-related transfer function, so that the audio signal is treated as a binaural signal.

[0052] (Derivation of the binaural signal calculation formula) The inventor has found that when a sound source object has radiation characteristics, a binaural signal, which is an audio signal to be presented to both ears as shown in Equation (13) and Equation (14) described below, can be derived based on the above-mentioned technical elements (spherical harmonic expansion and circular harmonic expansion, the additive theorem of Bessel functions, and the Kirchhoff-Helmholtz integral equation), and by using these equations, a more realistic and real sound image can be presented.

[0053] Incidentally, the formulas (13) and (14) described later are applied in the case of circular harmonic expansion that approximately targets only a two-dimensional (horizontal plane) sound field. When approximately targeting only a two-dimensional sound field, the formulas (13) and (14) described later are derived by circular harmonic expansion, and when targeting a three-dimensional sound field, the corresponding formulas are derived by spherical harmonic expansion, but the derivation procedure is the same.

[0054] Below, the procedure for deriving Equation (13) and Equation (14) (procedures A, B, C, and D) described later will be explained for the case where only a two-dimensional sound field using a circular harmonic expansion is approximately targeted.

[0055] FIG. 7 shows the center O of the head position 103 of the listener 102. q , measurement distance r h , measured azimuth angle φ h , boundary radius r q and boundary azimuth φ q FIG.

[0056] The center of the head position 103 of the listener 102 is O q The center O of the head position 103 q The distance from the origin to the measurement point is called the measurement distance r h , measure the azimuth angle of the measurement point between a given axis and the azimuth angle φ h Let us assume that.

[0057] Also, the center O of the head position 103 q Let us set a circular boundary around the boundary, and let the radius of the circular boundary be r q , the azimuth angle of a point on the boundary between a given axis and the circle is the boundary azimuth angle φ q Let us assume that.

[0058] <Step A: Head-related transfer function and its gradient (third data, fourth data)> The basic form of the circular harmonic expansion is expressed by the above formula (5). With reference to FIG. q With the origin as the measurement distance r h and measurement azimuth angle φ h The head-related transfer function G of the left ear at (L) (ω,rh ,φ h The circular harmonic expansion of

number

[0059] Also, the gradient in the normal direction, i.e., the radial gradient ∂G (L) (ω,r h ,φ h ) / ∂r h is expressed by the following formula:

number

[0060] In addition, the head-related transfer function of the right ear is G (R) (ω,r h ,φ h ), and the gradient in the normal direction is ∂G (R) (ω,r h ,φ h ) / ∂r h Let us assume that.

[0061] Here, in the Kirchhoff-Helmholtz integral equation of the above formula (8), the head-related transfer function G (L) (ω,r h ,φ h ) and the head-related transfer function of the right ear is G (R) (ω,r h ,φ h ) corresponds to the third data. In addition, the gradient ∂G (L) (ω,r h ,φ h ) / ∂r h and the normal gradient of the right ear head-related transfer function ∂G (R) (ω,r h ,φ h ) / ∂r h corresponds to the fourth data.

[0062] <Step B: Conversion of expansion coefficients> Referring to FIG. 5, the expansion coefficients obtained by circularly harmonic expanding the sound wave radiated from the sound source position 101 (the expansion coefficients of the radiation characteristics of the sound source object 100) are expressed as A n (1) (ω), and the expansion coefficients (expansion coefficients at the listening position) obtained by circularly expanding the sound waves arriving at another position 106 are A n (2) (ω). The expansion coefficient A of the listening position n (2) (ω) is the expansion coefficient obtained by circularly harmonic expanding the sound waves arriving from outside onto a circular boundary set near the listening position.

[0063] Then, according to the addition theorem of the Bessel function, as shown in the above formula (7), the expansion coefficient A of the radiation characteristic of the sound source object 100 is n (1) (ω) is the expansion coefficient A of the listening position n (2) (ω).

[0064] <Procedure C: Sound pressure and sound pressure gradient on the circular boundary of head position 103 (first data, second data)> Referring to Figure 7, the sound pressure on the circular boundary is p (q) (ω,r q ,φ q ) and the gradient in the normal direction on the circular boundary, i.e., the gradient in the radial direction (sound pressure gradient), is ∂p (q) (ω,r q ,φ q ) / ∂r q Let us assume that.

[0065] boundary radius r q and boundary azimuth φ q The sound pressure at each point on the circular boundary p (q) (ω,r q ,φ q ) and the sound pressure gradient ∂p (q) (ω,r q ,φ q ) / ∂r q is the expansion coefficient A of the listening position as shown in the following formula. n (2) (ω) can be calculated.

number

number

[0066] Here, in the Kirchhoff-Helmholtz integral equation of the above formula (8), the sound pressure p (q) (ω,r q ,φ q ) corresponds to the first data. Also, the sound pressure gradient ∂p (q) (ω,r q ,φ q ) / ∂r q corresponds to the second data.

[0067] <Procedure D: Binaural Signal> The head-related transfer function G of the left ear obtained by the above procedure A (L) (ω,r h ,φ h ) and the right ear head-related transfer function G (R) (ω,r h ,φ h ) (corresponding to the third data), the normal gradient of the left ear head-related transfer function ∂G (L) (ω,r h ,φ h ) / ∂r h and the normal gradient of the right ear head-related transfer function ∂G (R) (ω,r h ,φ h ) / ∂r h (corresponding to the fourth data), and the sound pressure p (q) (ω,r q ,φ q ) (corresponding to the first data) and the sound pressure gradient ∂p (q) (ω,r q ,φ q ) / ∂r q (corresponding to the second data) can be used to obtain a binaural signal, which is a signal presented to both ears, according to the Kirchhoff-Helmholtz integral equation of equation (8).

[0068] In FIG. 7, the center O of the head position 103, which is the center of the circular boundary, q That is, the position of the origin is The file is TIFF2025038778000024.tif7170 (not shown). In the following, when a variable represents a vector, the variable name is written in bold in the formula, and in the text, the word "vector" is written in parentheses after the variable name. h =r q = r, where r is the center O of the head position 103 q Furthermore, φ h =φ q =φ, where φ represents the direction.

[0069] By substituting the above equations (9), (10), (11), and (12) into the Kirchhoff-Helmholtz integral equation of the above equation (8), the audio signal p presented to the left ear is given by the following equation: (L) (ω,r(vector)) and the audio signal p presented to the right ear (R) A binaural signal with a magnitude of (ω,r (vector)) is obtained.

number

number

[0070] As shown in the above equations (13) and (14), the binaural signal is expressed by the expansion coefficients A n (L) (ω),A n (R) (ω) and the expansion coefficient A of the listening position n (2) (ω) can be obtained. In this case, the expansion coefficient A n (2) (ω) is the expansion coefficient A of the radiation characteristic of the sound source object 100 n (1)For this conversion, it is necessary to specify the distance r1 from the sound source position 101 to the head position 103 and the azimuth angle φ1 to determine the head position 103, as shown in FIG.

[0071] [Binaural playback device] Next, a binaural reproduction device according to an embodiment of the present invention will be described. Fig. 8 is a block diagram showing an example of the configuration of a binaural reproduction device according to an embodiment of the present invention, and Fig. 9 is a flowchart showing an example of the processing of the binaural reproduction device. In this example, a binaural signal is obtained by the above-mentioned formulas (13) and (14) when only a two-dimensional sound field using circular harmonic expansion is approximately targeted.

[0072] The binaural reproduction device 1 includes harmonic expansion sections 10 and 11, a conversion section 12, and a binaural signal calculation section 13.

[0073] The harmonic development unit 10 receives the head-related transfer function of the left ear 104 measured in advance for each direction centered on the head position 103 of the listener 102, and also receives the head-related transfer function of the right ear 105 (step S901).

[0074] The harmonic expansion unit 10 performs circular harmonic expansion on the head-related transfer function of the left ear 104 and the head-related transfer function of the right ear 105, respectively, and obtains the expansion coefficients A n (L) (ω),A n (R) (ω) is calculated (step S902).

[0075] Specifically, the harmonic development unit 10 calculates the head-related transfer function G (L) (ω,r h ,φ h ) as shown in the above equation (9), the expansion coefficient A n (L) (ω), the Hankel function of the second kind H n (2) (kr h ) and the circular harmonic function e jnφh(h is a subscript of φ) n (L) (ω) is calculated. The head-related transfer function G (R) (ω,r h ,φ h ) similarly, the expansion coefficient A n (R) Find (ω).

[0076] The harmonic expansion unit 10 expands the head-related transfer function coefficients A n (L) (ω),A n (R) (ω) is output to the binaural signal calculation unit 13.

[0077] The harmonic development unit 11 receives an input of a sound signal of the sound source object 100 having a radiation characteristic picked up by the multiple microphones 110 (step S903).

[0078] Fig. 10 is a diagram showing an example of picking up a sound signal of a sound source object having a radiation characteristic when microphones are arranged two-dimensionally. As shown in Fig. 10, microphones 110 are arranged two-dimensionally so as to surround a sound source object 100-1. The sound signal is picked up by microphone 110 and input to harmonic development section 11.

[0079] Returning to FIG. 8 and FIG. 9, the harmonic development unit 11 calculates the distance r between the sound source object 100-1 and each of the microphones 110. m and its azimuth angle φ m The harmonic expansion unit 11 then performs circular harmonic expansion of the input audio signals, and obtains the expansion coefficients A n (1) (ω) is calculated (step S904).

[0080] Specifically, the harmonic development unit 11 calculates the distance of each microphone 110 from the sound source object 100-1 as r m , azimuth angle is φ m As the sound field of the input audio signal p(ω,r m,φ m ) as shown in the above formula (4), the expansion coefficient A n (ω), the Hankel function of the second kind H n (2) (k rm r m )(k rm m is a subscript of r) and the circular harmonic function e jnφm (m is a subscript of φ) n (1) (ω)=A n (ω) is calculated. In the above formula (4), r = r m , φ=φ m , k r =k rm (m is a subscript of r), p(ω,r,φ)=p(ω,r m ,φ m )

[0081] The harmonic expansion unit 11 calculates the expansion coefficient A of the radiation characteristic of the sound source object 100. n (1) (ω)(=A n (ω) is output to the conversion unit 12.

[0082] The conversion unit 12 receives the expansion coefficients A of the radiation characteristics of the sound source object 100 from the harmonic expansion unit 11. n (1) (ω)(=A n (ω)), as well as the distance r1 between the sound source position 101 and the head position 103 and the azimuth angle φ1 thereof are input.

[0083] The conversion unit 12 calculates the expansion coefficient A by the addition theorem of Bessel functions based on the distance r1 and the azimuth angle φ1 that define the head position 103. n (1) (ω)(=A n (ω)) as the expansion coefficient A n (2) (ω) (step S905).

[0084] Incidentally, since the process of converting the expansion coefficients according to the addition theorem of the Bessel function is known, a detailed description thereof will be omitted here.

[0085] The conversion unit 12 converts the expansion coefficient A n (2) (ω) is output to the binaural signal calculation unit 13.

[0086] The binaural signal calculation unit 13 receives the expansion coefficients A of the head-related transfer functions of both ears from the harmonic expansion unit 10. n (L) (ω),A n (R) (ω) is input, and the expansion coefficient A of the listening position is input from the conversion unit 12. n (2) Enter (ω).

[0087] The binaural signal calculation unit 13 calculates the expansion coefficients A of the head-related transfer functions of both ears by the above-mentioned equations (13) and (14) derived from the Kirchhoff-Helmholtz integral equation. n (L) (ω),A n (R) (ω) and the expansion coefficient A of the listening position n (2) (ω) etc., binaural signal p (L) (ω,r(vector)),p (R) (ω, r (vector)) is calculated and output (step S906).

[0088] Specifically, the binaural signal calculation unit 13 calculates the expansion coefficient A n (L) (ω),A n (2) (ω), wave number k, second kind Hankel function H n (2) and Bessel function J n Using the above equation (13), the audio signal p (L) Calculate (ω,r (vector)).

[0089] In addition, the binaural signal calculation unit 13 calculates the expansion coefficient A n (R) (ω),A n (2) (ω), wave number k, second kind Hankel function H n (2)and Bessel function J n Using the above equation (14), the audio signal p (R) Calculate (ω,r (vector)).

[0090] The binaural reproduction device 1 determines whether the input of the audio signal of the sound source object 100 continues (step S907). If the binaural reproduction device 1 determines in step S907 that the input of the audio signal of the sound source object 100 continues (step S907: Y), the binaural reproduction device 1 proceeds to step S903 and performs the processes of steps S903 to S906.

[0091] On the other hand, if it is determined in step S907 that the input of the sound signal of the sound source object 100 is not continuing (step S907: N), the binaural reproduction device 1 ends the process.

[0092] As described above, according to the binaural reproduction device 1 of the embodiment of the present invention, the harmonic expansion unit 10 inputs the head-related transfer functions of both ears for each direction that have been measured in advance, and performs circular harmonic expansion of the head-related transfer functions as shown in the above equation (9), and obtains the expansion coefficients A n (L) (ω),A n (R) Find (ω).

[0093] The harmonic expansion unit 11 inputs the sound signal of the sound source object 100 having the collected radiation characteristic, and performs circular harmonic expansion of the sound signal as shown in the above formula (4), and obtains the expansion coefficient A n (1) Then, the conversion unit 12 calculates the expansion coefficient A based on the distance r1 and the azimuth angle φ1 that define the head position 103, using the addition theorem of the Bessel function. n (1) (ω)(=A n (ω)) as the expansion coefficient A n (2) Convert to (ω).

[0094] The binaural signal calculation unit 13 calculates the expansion coefficients A n (L) (ω),A n (R) (ω) and the expansion coefficient A of the listening position n (2) (ω), etc., the binaural signal p is calculated by the above-mentioned equations (13) and (14) derived from the Kirchhoff-Helmholtz integral equation. (L) (ω,r(vector)),p (R) Calculate and output (ω,r(vector)).

[0095] Thus, in an embodiment of the present invention, when the sound source object 100 has radiation characteristics, the sound waves in each direction that are radiated into space based on the radiation characteristics and arrive at the listening position are multiplied by a head-related transfer function and presented to both ears as a binaural signal.

[0096] Specifically, by using the addition theorem of the Bessel function, the expansion coefficient A of the circular harmonic expansion or spherical harmonic expansion that represents the radiation characteristics of the sound wave radiated from the sound source object 100 is n (ω),A n,m (ω) is the expansion coefficient A of the sound wave arriving at the listening position n (2) (ω),A n,m (2) (ω), and the binaural signal p presented to both ears is calculated by taking into account the acoustic propagation of sound waves arriving at the listening position from each direction. (L) (ω,r(vector)),p (R) It is now possible to calculate (ω,r (vector)).

[0097] This allows the sound of the sound source object 100 to be accurately delivered to the listener 102. Therefore, when the sound source object 100 has radiation characteristics, a more realistic and real sound image can be presented by taking into consideration the acoustic propagation of sound waves arriving at the listening position from each direction.

[0098] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments and can be modified in various ways without departing from the technical concept thereof.

[0099] For example, as shown in FIG. 8, the binaural reproduction device 1 performs circular harmonic expansion of the sound signal of a sound source object 100 having a radiation characteristic, and obtains an expansion coefficient A n (1) Although the harmonic expansion section 11 for determining (ω) is provided, the harmonic expansion section 11 may not be provided.

[0100] In this case, the binaural reproduction device 1 includes a harmonic expansion unit 10, a conversion unit 12, and a binaural signal calculation unit 13. n (1) Assuming that (ω) has already been acquired, the conversion unit 12 converts the expansion coefficient A n (1) (ω) is input, and by the addition theorem of Bessel functions, this is expanded to the listening position coefficient A n (2) Convert to (ω).

[0101] 8 and 9, the binaural reproduction device 1 obtains a binaural signal by the above-mentioned formulas (13) and (14) when only a two-dimensional sound field using a circular harmonic expansion is approximately targeted. On the other hand, when a three-dimensional sound field using a spherical harmonic expansion is targeted, the binaural reproduction device obtains a binaural signal by the same configuration and processing as the binaural reproduction device 1, but with the circular harmonic expansion replaced by a spherical harmonic expansion.

[0102] An example of collecting audio signals in a three-dimensional sound field is shown in Fig. 11. Fig. 11 is a diagram showing an example of collecting audio signals from a sound source object having radiation characteristics in a three-dimensional arrangement of microphones. As shown in Fig. 11, microphones 110 are arranged three-dimensionally so as to surround a sound source object 100-2. Audio signals are collected by the multiple microphones 110 and input to a harmonic development unit 11 in a three-dimensional sound field.

[0103] A typical computer can be used as the hardware configuration of the binaural reproduction device 1 according to the embodiment of the present invention. The binaural reproduction device 1 is configured by a computer including a CPU, a volatile storage medium such as a RAM, a non-volatile storage medium such as a ROM, and an interface.

[0104] The functions of the harmonic development units 10 and 11, the conversion unit 12 and the binaural signal calculation unit 13 included in the binaural reproduction device 1 are each realized by causing a CPU to execute a program describing these functions.

[0105] These programs are stored in the storage medium and are read and executed by the CPU. These programs can also be distributed by storing them in a storage medium such as a magnetic disk (floppy disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), or a semiconductor memory, and can also be transmitted and received via a network. [Industrial Applicability]

[0106] The present invention is used in technology related to object audio in games, AR, VR, etc., and is particularly used for headphone playback of sound sources that are objects. [Explanation of symbols]

[0107] 1. Binaural playback device 10,11 Harmonic expansion part 12 Conversion section 13 Binaural signal calculation section 100 Sound Source Objects 101 Sound source position 102 listeners 103 Head position 104 Left Ear 105 Right ear 106 Another Position 110 Microphone

Claims

1. In a binaural reproduction device that calculates an audio signal as a binaural signal when a sound source object has a radiation characteristic, a first harmonic expansion unit that inputs a head-related transfer function of both ears of a listener that has been measured in advance, and performs a circular harmonic expansion or a spherical harmonic expansion on the head-related transfer function to obtain an expansion coefficient of the head-related transfer function; a conversion unit that receives as input expansion coefficients of sound waves radiated from the sound source object, the expansion coefficients being measured in advance, and converts the expansion coefficients of the sound waves radiated from the sound source object into expansion coefficients of sound waves arriving at the listening position of the listener, using the addition theorem of Bessel functions; A binaural signal calculation unit that calculates the binaural signal using the expansion coefficients of the head related transfer function obtained by the first harmonic expansion unit and the expansion coefficients of the sound waves arriving at the listening position of the listener that are converted by the conversion unit based on the Kirchhoff-Helmholtz integral equation; A binaural reproduction device comprising:

2. 2. The binaural reproduction device according to claim 1, a second harmonic expansion unit that inputs an audio signal of the collected sound waves emitted by the sound source object and performs a circular harmonic expansion or a spherical harmonic expansion of the audio signal to obtain expansion coefficients of the sound waves emitted by the sound source object; The conversion unit is A binaural playback device characterized in that the expansion coefficients of the sound waves radiated by the sound source object obtained by the second harmonic expansion unit are converted into expansion coefficients of sound waves arriving at the listening position of the listener, using the addition theorem of the Bessel function.

3. 3. The binaural reproduction device according to claim 1, The binaural signal calculation unit A binaural reproduction device, characterized in that the binaural signal is calculated by a predetermined equation derived by substituting the expansion coefficients of the head transfer function and the gradients of the expansion coefficients, as well as the sound pressure and sound pressure gradient on a circular boundary centered on the head of the listener, which are obtained from the expansion coefficients of the sound waves arriving at the listening position of the listener, into the Kirchhoff-Helmholtz integral equation.

4. 3. The binaural reproduction device according to claim 2, The binaural signal calculation unit When the circular harmonic expansion is performed by the first harmonic expansion unit and the second harmonic expansion unit, ω is an angular frequency, is the position centered on the listener's head, r is the distance from the center of the head, and p (L) (ω, r) is the signal presented to the left ear, p (R) (ω, r) is the signal presented to the right ear, k is the wave number, N is the finite expansion order in the circular harmonic expansion, A n (2) (ω) is the expansion coefficient of the sound wave arriving at the listening position of the listener, A n (L) (ω) is the expansion coefficient of the head-related transfer function of the left ear, A n (R) (ω) is the expansion coefficient of the head-related transfer function of the right ear, H n (2) is the Hankel function of the second kind, J n is the Bessel function, The following formula: A binaural reproduction device, comprising:

5. A program for causing a computer to function as the binaural reproduction device according to claim 1.

Images