Object-based audio rendering device and program

The object-based sound rendering device combines WFS and VBAP to generate drive signals that accurately represent sound source distance, addressing scalability and quality issues in existing technologies.

JP7583638B2Active Publication Date: 2024-11-14NIPPON HOSO KYOKAI
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2021023419
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-02-17
Publication Date
2024-11-14
Estimated Expiration
2041-02-17

AI Technical Summary

Technical Problem

Existing object-based sound rendering technologies face challenges in expressing the distance of sound sources accurately without increasing system scale, particularly when using Wave Field Synthesis (WFS), and suffer from quality deterioration when speakers are not positioned optimally.

Method used

An object-based sound rendering device that generates drive signals using a combination of Wave Field Synthesis (WFS) and Vector Based Amplitude Panning (VBAP), incorporating high-pass filters and attenuation coefficients to approximate speaker positions, allowing for accurate distance representation without expanding the system's physical scale.

Benefits of technology

The solution enables precise sound source distance representation while maintaining system scalability, ensuring high-quality sound reproduction across various speaker arrangements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007583638000026
    Figure 0007583638000026
  • Figure 0007583638000027
    Figure 0007583638000027
  • Figure 0007583638000028
    Figure 0007583638000028
Patent Text Reader

Abstract

To generate a driving signal expressing the perspective of the distance of a sound source based on WFS without expanding the scale of the system.SOLUTION: A signal amplifier 9-l of an object-based acoustic rendering device 1 amplifies a sound source signal S, and HPF 10-l performs filtering processing on the amplified signal by using HPFh(rl,rS). An attenuation addition unit 11-l multiplies the filtered signal by an attenuation coefficient 1 / ||rl-rS||2. An area element adder 12-l multiplies the signal multiplied by the attenuation adder 11-l by an area element ΔSl. An angle weight adder 13-l sets an angle weight wC(θl) based on an intersection angle θl between a vector (rl-rS) connecting the position of a virtual sound source 104 to the position of a speaker 106-1 and a boundary surface inward normal vector n(rl), and multiplies the signal multiplied by the area element adder 12-l by the angle weight wC(θl) to generate a driving signal d(rl).SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an object-based audio reproduction device, and more particularly to an object-based audio rendering device and program that calculates speaker drive signals based on the concept of WFS (Wave Field Synthesis). [Background technology]

[0002] In recent years, object-based audio technology has been gaining attention, especially in the field of movie sound. Conventionally, channel-based audio, in which multiple audio materials are mixed down to a specified channel format and then recorded, was the mainstream.

[0003] In object-based audio, each piece of audio material is recorded separately as an audio object, and rendering is performed based on audio metadata that describes the level, coordinates, etc. of the audio object.

[0004] Hereinafter, such an object-based audio playback device will be referred to as an object-based audio rendering device. The object-based audio rendering device renders audio objects based on audio metadata and the positions of multiple speakers arranged for the audio objects, and generates drive signals according to the positions of each speaker, thereby playing back audio according to the listening environment. Therefore, even if the content is played back with a speaker arrangement different from that at the time of its creation, playback that is adapted to the listening environment is possible.

[0005] [VBAP] Conventionally, an algorithm called Vector Based Amplitude Panning (VBAP) has been widely used for rendering object-based audio (see, for example, Non-Patent Document 1).

[0006] In VBAP, the playback space is divided into a triangular region consisting of three speakers, and weights are calculated for the speakers located at each vertex of the triangle including the sound source coordinates.Then, amplitude panning is performed by distributing the sound source signal by the calculated weights.

[0007] However, in VBAP, the speakers are arranged on a sphere with a specific radius, and the sound source is processed as if it were located on the sphere, so it is not possible to express the sense of perspective based on the distance of the sound source.

[0008] When using VBAP to represent a sound field that includes both near and far, distance attenuation and reverberation may be added (see, for example, Non-Patent Document 2), but these are methods based on psychoacoustics and are not intended to physically reproduce the sound field.

[0009] Furthermore, according to the VBAP weight calculation algorithm, the weights are calculated assuming listening in the sweet spot, so playback quality is not guaranteed when listening from a position away from that sweet spot.

[0010] [WFS] On the other hand, a method called WFS is known as a method for expressing a sound field based on wave acoustics. WFS is a sound field reproduction method supported by wave acoustics using the Kirchhoff-Helmholtz integral equation expressed by the following formula (for example, see Non-Patent Document 3).

number

[0011] Here, p(r) denotes the sound pressure at any point r in a region V enclosed by the boundary ∂V, and G(r|r o ) is the boundary ∂V at point r o Denotes the Green's function from to any point r within the boundary ∂V. o ) is the point r o p(r o ) is the boundary ∂V at point ro denotes the sound pressure at ∂p(r o ) / ∂n is the boundary ∂V at point r o Unit vector n(r o ) sound pressure gradient vector in the direction of the sound pressure. O is an infinitesimal area on the boundary ∂V.

[0012] In addition, when the boundary has an arbitrary shape, the boundary condition at the boundary ∂V is ∂G(r|r o ) / ∂n=0. Then, set the Green function assuming the Neumann condition. o Average sound intensity I(r o ) and unit vector n(r o ) is determined by the following window function a(r o ) will be introduced.

number

[0013] This results in the following equality:

number

[0014] G o (r|r o ) is the point r o The free-field Green's function from to point r is shown. The Kirchhoff-Helmholtz integral equation describes the fact that when a sound source is present outside a domain, the sound field inside the domain is determined by the sound pressure gradient distribution in the boundary normal direction at the boundary surface.

[0015] In WFS, speakers are densely placed on the boundary surfaces of the area where the sound field is to be reproduced, and the coordinates of the sound source (assumed to be a point source) are given, and the driving signals for the speakers are determined based on the Kirchhoff-Helmholtz integral equation. [Prior art documents] [Non-patent literature]

[0016] [Non-Patent Document 1] VlLLE PULKKI, “Virtual Sound Source Positioning Using Vector Base Amplitude Panning”, J. Audio Eng. Soc., Vol. 45, No.6, 1997 June [Non-Patent Document 2] Recommendation ITU-R.BS.2127-0 [Non-Patent Document 3] Ahrens Jens, Rabenstein Rudolph, Spors Sascha, “The Theory of wave field synthesis revisited”, in 124th Conv. Audio Eng. Soc., 2008 Summary of the Invention [Problem to be solved by the invention]

[0017] In this way, by using WFS, it is possible to resynthesize wavefronts inside the reproduction area, so unlike VBAP, there is no problem of degradation of reproduction quality at points away from the sweet spot.

[0018] However, in order to precisely synthesize wavefronts using WFS, it is necessary to place speakers densely on the boundary surfaces, and it is also necessary to add elements such as filters with specific frequency characteristics and delays. Therefore, when using WFS, there is an issue that the system size becomes larger than that of VBAP.

[0019] Therefore, the present invention has been made to solve the above-mentioned problems, and its object is to provide an object-based audio rendering device and program that can generate drive signals that express the distance of a sound source based on WFS without expanding the scale of the system. [Means for solving the problem]

[0020] In order to solve the above problem, the object-based audio rendering device of claim 1 generates a drive signal for each of a plurality of speakers arranged in a listening environment by performing rendering on a sound source object arranged at a position described in audio metadata, the number of the plurality of speakers is L, the speaker number is l (=1, . . . , L), the lth speaker is the lth speaker, and the coordinates of a virtual sound source from which a sound source signal is output are virtual sound source coordinates r S , the coordinates of the l-th speaker are speaker coordinates r l , the inward normal vector of the boundary surface at the position of the l-th speaker is n(r l ), and the area element centered on the position of the l-th speaker is ΔS l , the driving signal for the l-th speaker is d(r l ) for each of the plurality of speakers, S The speaker coordinate r of the l-th speaker from the position l and the inward normal vector n(r l ) and the angle θ l and a signal amplifier, a HPF (high pass filter), an attenuation adding unit, an area element adding unit and a first angle weighting adding unit, each of which corresponds to each of the plurality of speakers. The signal amplifier amplifies the sound source signal by a constant, and the HPF calculates the virtual sound source coordinate r S and the speaker coordinate r of the l-th speaker l Based on this, HPFh(r l ,r S ), and the HPFh(r l ,r S ), and the attenuation adding unit performs filtering using the virtual sound source coordinate r S The speaker coordinate r of the l-th speaker from the position l The absolute value of the vector up to the position is calculated, and the inverse of the result of squaring the absolute value is taken as the attenuation coefficient 1 / ||r l -r S ||2 and apply the attenuation coefficient 1 / ||r l -r S || 2 and the area element adding unit multiplies the area element ΔS with respect to the signal multiplied by the attenuation adding unit. l and the first angle weighting unit multiplies the angle θ of the first speaker calculated by the angle calculation unit. l Based on this, the angle θ l The closer the absolute value of is to 0°, the closer it is to 1, and the angle θ l The closer the absolute value of is to 90°, the closer it is to 0, and the angle θ l The angle weight w takes a value of 0 when the absolute value of is less than 0° or greater than 90°. C (θ l ) is set, and the angle weight w C (θ l ) to obtain the driving signal d(r l ) is generated.

[0021] The object-based audio rendering apparatus of claim 2 is the object-based audio rendering apparatus of claim 1, further comprising: S The unit vector of the virtual sound source direction to the speaker coordinate r l Based on the unit vector of the speaker direction to the target speaker, the three speakers including the virtual sound source are identified as three speakers subject to VBAP (Vector Based Amplitude Panning), and the unit vector r n1 ,r n2 ,r n3 and obtain the virtual sound source coordinate r S The unit vector of the virtual sound source direction to the VBAP target speaker and the unit vector r n1 ,r n2 ,r n3 Based on the above, coefficients g corresponding to the three speakers of the VBAP target are calculated.l =g n1 ,g n2 ,g n3 Then, among all the speakers, the coefficient g corresponding to the speakers other than the three speakers targeted by the VBAP is calculated. l = 0, and the coefficient g l a VBAP coefficient calculation unit that outputs a VBAP coefficient g l and the second angle weighting unit multiplies the angle θ of the first speaker calculated by the angle calculation unit. l Based on this, the angle θ l The closer the absolute value of is to 0°, the closer it is to 0, and the angle θ l The closer the absolute value of is to 90°, the closer it is to 1. l The angle weight w takes a value of 0 when the absolute value of is less than 0° or greater than 90°. S (θ l ) is set, and the angle weight w S (θ l ) to generate a second drive signal, and the first angle weighting unit multiplies the angle weight w C (θ l ) is set, and the angle weight w C (θ l ), a first drive signal is generated, and the adder adds the first drive signal and the second drive signal to generate the drive signal d(r l ) is generated.

[0022] The object-based acoustic rendering apparatus of claim 3 is the object-based acoustic rendering apparatus of claim 2, wherein the VBAP coefficient multiplier inputs the sound source signal instead of the signal multiplied by the surface element adding unit, and applies the coefficient g to the sound source signal. lThe present invention is characterized in that:

[0023] Further, the object-based acoustic rendering apparatus of claim 4 is the object-based acoustic rendering apparatus according to any one of claims 1 to 3, wherein the HPF is calculated by the following formula, where α is a preset parameter, N is a preset filter order, and n=0, 1, . . . , N: In TIFF0007583638000004.tif20170, HPF coefficient h n (r l ,r S ) and Let x[t] and y[t] be the input and output of the HPF at time t, respectively, and the input / output characteristics are expressed by the following equation: TIFF0007583638000005.tif15170, the HPF coefficient h n (r l ,r S ) is used to perform filtering.

[0024] Further, the object-based acoustic rendering apparatus according to claim 5 is the object-based acoustic rendering apparatus according to any one of claims 1 to 4, wherein the attenuation adding unit calculates the attenuation coefficient β1 and β2 by the following equation: Attenuation coefficient g(r l ,r S ) and apply the attenuation coefficient g(r l ,r S ) is multiplied.

[0025] Further, a program according to claim 6 causes a computer to function as the object-based audio rendering device according to any one of claims 1 to 5. Effect of the Invention

[0026] As described above, according to the present invention, a drive signal that expresses the distance of a sound source based on a WFS can be generated without increasing the scale of the system. [Brief description of the drawings]

[0027] [Figure 1] 1 is a block diagram showing an example of the configuration of an object-based acoustic rendering device according to a first embodiment. [Diagram 2] 4 is a pseudo code showing an example of processing of the object-based audio rendering device of the first embodiment. [Diagram 3] 4 is a block diagram showing an example of the configuration of an angle calculation unit; FIG. [Figure 4] FIG. 13 is a diagram illustrating a case where speakers are sparsely arranged. [Diagram 5] FIG. 11 is a block diagram showing an example of the configuration of an object-based acoustic rendering device according to a second embodiment. [Figure 6] 11 is a pseudo code showing an example of processing of the object-based audio rendering device of the second embodiment. [Figure 7] 11 is a block diagram showing an example of the configuration of a VBAP coefficient calculation unit. [Figure 8] FIG. 11 is a block diagram showing an example of the configuration of an object-based acoustic rendering device according to a third embodiment. [Figure 9] 13 is a pseudo code showing an example of processing of the object-based audio rendering device of the third embodiment. [Figure 10] FIG. 1 is a conceptual diagram for explaining VBAP. [Figure 11] FIG. 1 is a conceptual diagram for explaining WFS. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Hereinafter, the embodiments of the present invention will be described in detail with reference to the drawings. An object-based audio rendering device according to a first embodiment of the present invention generates a drive signal expressing the distance of a sound source based on a WFS without expanding the scale of the system by simplifying the configuration of the WFS through approximation. Also, an object-based audio rendering device according to second and third embodiments incorporates a VBAP configuration into the configuration of the first embodiment, thereby solving the problem of the drive signal becoming small when speakers are sparsely arranged.

[0029] Before describing the object-based acoustic rendering apparatus according to the first, second and third embodiments of the present invention, an overview of the sound field reproduction method using VBAP and WFS will be given.

[0030] [Outline of VBAP] First, we will explain the outline of VBAP. As mentioned above, VBAP is an amplitude panning technique that divides the playback space into triangular regions consisting of three speakers, calculates weights for the speakers located at each vertex of the triangle including the sound source coordinates, and distributes the sound source signal to perform amplitude panning.

[0031] 10 is a conceptual diagram for explaining VBAP. In the xyz space, the sound receiving point 100 is set as the origin, and the unit vector in the direction of the virtual sound source 101 is expressed as r nS =(x nS ,y nS ,z nS ) T In addition, in the divided triangular region, the unit vectors in the directions of the three speakers 102-1, 102-2, and 102-3 including the virtual sound source 101 inside are respectively defined as r n1 =(x n1 ,y n1 ,z n1 ) T ,r n2 =(x n2 ,y n2 ,z n2 ) T ,r n3 =(x n3 ,y n3 ,z n3 ) TLet us assume that.

[0032] At this time, the unit vector r nS is the unit vector r in the direction of speakers 102-1, 102-2, and 102-3. n1 ,r n2 ,r n3 and coefficient vector g = (g n1 ,g n2 ,g n3 ) T It is expressed as follows using

number

[0033] The coefficient vector g can be calculated by the following formula.

number

[0034] The coefficient vector g calculated by the above formula (5) = (g n1 ,g n2 ,g n3 ) T Then, by playing the weighted signals from the three speakers 102-1, 102-2, and 102-3, the listener perceives the unit vector r nS The weights of the speakers forming a triangle that does not include the virtual sound source 101 inside are all set to 0.

[0035] [Outline of WFS] Next, an overview of WFS will be explained. As mentioned above, WFS is a sound field reproduction method that reproduces the sound field inside the boundary surface by densely arranging speakers on the boundary surface of the area where the sound field is to be reproduced, and reproducing the normal direction sound pressure gradient of the boundary surface in the original sound field, which is the sound field to be reproduced, at the speaker position based on the Kirchhoff-Helmholtz integral equation of the above formula (1).

[0036] FIG. 11 is a conceptual diagram for explaining WFS, where (1) shows the original sound field and (2) shows the reproduced sound field.

[0037] The original sound field of (1) is located at the coordinate r outside the area C1 in the xyz space. S =(x S ,y S ,z S ) T The sound is generated by a sound wave emitted from the sound source 103 located at the point of coordinate r on the boundary surface of the region C1. l =(x l ,y l ,z l ) T The sound pressure p at is expressed by the following formula: l (El) is the coordinate r l The numbers of the points correspond to the numbers (system numbers) of the speakers 106-l described later.

number

[0038] Furthermore, the coordinate r on the boundary surface of region C1 l =(x l ,y l ,z l ) T The unit vector in the boundary normal direction (boundary inward normal vector) at the point is n(r l ) and the coordinate r S From the point of coordinate r l The vector (r l -r S ) and the inward normal vector n(r l) is the angle θ l Coordinate r l The sound pressure gradient vector in the normal direction at the point is expressed by the following equation.

number

[0039] Therefore, according to the Kirchhoff-Helmholtz integral equation, the speaker array 105 discretely arranged in the reproduced sound field of (2) can be expressed as follows: l The driving signal d(r l ) to drive the coordinate r S A sound field in which a point sound source of the virtual sound source 104 is placed at this point can be reproduced inside the area C2.

number

[0040] Here, ΔS l is the area of ​​the l-th element (area element) when the boundary surface of area C2 is divided into area elements centered on the speaker position, and the more densely the speaker elements are arranged, the smaller the value becomes, and the more sparsely they are arranged, the larger the value becomes.

[0041] In addition, in the xyz space, r S are the coordinates of the virtual sound source 104 (virtual sound source coordinates), and r l are the coordinates (speaker coordinates) of the l-th speaker element that constitutes the speaker array 105.

[0042] Also, w C (θ l ) is a function defined by the following equation:

number

[0043] [Object-based audio rendering device] Next, the object-based audio rendering apparatuses according to the first, second and third embodiments will be described.

[0044] In the first embodiment, the speaker drive signal d(r l In the above formula (8) showing the virtual sound source coordinate r S A virtual sound source 104 placed at the position of the speaker coordinate r l This represents the time delay due to the distance between the speaker and the -jk||rl-rs|| Ignoring the term simplifies the system at the expense of exact wavefront reconstruction.

[0045] In Example 1, jk||r in the above formula (8) l -r S The term ||+1 has a high-pass characteristic that changes depending on the distance between the virtual sound source 104 and the speaker. This characteristic is filtered by a low-order high-pass filter h(r l ,r S )(HPF).

[0046] As a result, in the first embodiment, by simplifying the configuration of the WFS through approximation, it is possible to generate a drive signal that expresses the distance of a sound source based on the WFS without increasing the scale of the system.

[0047] In addition, in the second and third embodiments, by incorporating a VBAP configuration into the configuration of the first embodiment, in addition to the effects of the first embodiment, the problem of the drive signal becoming small when the speakers are sparsely arranged is solved.

[0048] Example 1 First, an object-based audio rendering apparatus according to the first embodiment will be described. As described above, the first embodiment uses a speaker drive signal d(r l In the above equation (8) which generates -jk||rl-rs|| Ignoring the term jk||r l -r SThe term ||+1 is treated as an HPF, and the high-pass characteristics that change depending on the distance between the virtual sound source 104 and the speaker are filtered using a low-order high-pass filter h(r l ,r S )(HPF).

[0049] FIG. 1 is a block diagram showing an example of the configuration of an object-based audio rendering apparatus according to the first embodiment, and FIG. 2 is a pseudo code showing an example of the processing of the object-based audio rendering apparatus according to the first embodiment.

[0050] This object-based acoustic rendering device 1 includes signal amplifiers 9-1 to 9-L, HPFs (high pass filters) 10-1 to 10-L, attenuation adding sections 11-1 to 11-L, area element adding sections 12-1 to 12-L, angle weight adding sections 13-1 to 13-L, and an angle calculation section 20. L is the number of speakers 106-1 to 106-L and is also the number of systems.

[0051] Hereinafter, in the object-based acoustic rendering device 1, the components corresponding to the l-th speaker 106-l (l-th system) are respectively defined as a signal amplifier 9-l, an HPF 10-l, an attenuation adding unit 11-l, an area element adding unit 12-l, and an angle weight adding unit 13-l. That is, the object-based acoustic rendering device 1 includes L system signal amplifiers 9-l, HPFs 10-l, attenuation adding units 11-l, area element adding units 12-l, and angle weight adding units 13-l, and further includes an angle calculation unit 20. l=1,...,L.

[0052] The overall process of the object-based audio rendering device 1 will be described with reference to Fig. 2. The object-based audio rendering device 1 receives a sound source signal S and calculates virtual sound source coordinates r S (Step S201).

[0053] The object-based audio rendering device 1 calculates the speaker coordinate r of the speaker 106-l for each of the speakers 106-1 to 106-L (for the speaker 106-l system). l, the area element ΔS of the speaker 106-l l , and the inward normal vector n(r l ) (step S202).

[0054] Here, the speaker coordinate r l , area element ΔS l and the inward normal vector n(r l The subscript "l" in each of the subscripts indicates the number (system number) of the speakers 106-1 to 106-L.

[0055] The sound source signal S is, for example, a signal broadcast from a broadcasting station, and the virtual sound source coordinates r S is data included in the audio metadata broadcast by a broadcasting station. l , area element ΔS l and the inward normal vector n(r l ) is data included in speaker layout information that is preset by a user who operates the object-based acoustic rendering device 1, for example.

[0056] In this case, the object-based audio rendering device 1 receives a sound source signal S transmitted from a broadcast station and virtual sound source coordinates r S and receives the speaker coordinate r l , area element ΔS l and the inward normal vector n(r l )

[0057] The object-based audio rendering device 1 calculates the virtual sound source coordinates r S , and the speaker coordinate r input in step S202 l Based on this, the HPF h(r l ,r S ) is set (step S203).

[0058] The object-based audio rendering device 1 calculates the virtual sound source coordinates r S , and the speaker coordinate r input in step S202 l Based on this, the attenuation coefficient 1 / ||r of the attenuation adding unit 11-l according to the distance between the virtual sound source 104 and the speaker 106-l is l -r S || 2 is set (step S204).

[0059] That is, the object-based audio rendering device 1 calculates the absolute value ||r l -r S || is calculated and the absolute value is squared, resulting in ||r l -r S || 2 The inverse of this is the damping coefficient 1 / ||r l -r S || 2 Set as.

[0060] The object-based audio rendering device 1 calculates the virtual sound source coordinates r S , and the speaker coordinate r l and the inward normal vector n(r l ), a vector (r l -r S ) and the inward normal vector n(r l ) and the angle θ l (Step S205). l is calculated by the angle calculation unit 20.

[0061] The object-based audio rendering device 1 calculates the angle θ l Absolute value of |θ l Determine whether | is greater than 0° and less than 90°, and use the angle weight wC (θ l ) is set (step S206).

[0062] Specifically, the object-based audio rendering device 1 determines the angle |θ l If it is determined that | is greater than 0° and smaller than 90° (cosθ l >0), angle weight w C (θ l )=cosθ l On the other hand, the object-based audio rendering device 1 sets the angle |θ l is determined to be 0° or less or 90° or more (cosθ l > 0), angle weight w C (θ l ) = 0. This angle weight w C (θ l ) is used in the angle weighting unit 13-l.

[0063] Angle weight w C (θ l ) ranges from 0 to 1, and the angle |θ l | is 0°<|θ l In the range of |<90°, the angle |θ l The closer | is to 0°, the closer it is to 1; the closer it is to 90°, the closer it is to 0; the angle |θ l |θ l |≦0° or 90°≦|θ l In the range |, the value is 0.

[0064] The object-based audio rendering device 1 calculates, for the speaker 106-l, a constant u (for example, u=1 / 2π, which may be a different value for each system), the sound source signal S input in step S201, and the virtual sound source coordinates r S , the speaker coordinate r input in step S202 l and area element ΔS l , HPFh(r l ,r S), the damping coefficient 1 / ||r set in step S204 l -r S || 2 and the angle weight w set in step S206 C (θ l ) to calculate the drive signal d(r l ) is calculated and output (step S207).

number

[0065] Next, the processing of the components of the object-based acoustic rendering device 1 will be described with reference to Fig. 1. The signal amplifier 9-l (9-1 to 9-L) receives a sound source signal S, multiplies the sound source signal S by a constant u (e.g., 1 / 2π), thereby amplifying the sound source signal S, and outputs the amplified signal to the HPF 10-l.

[0066] HPF10-l (10-1 to 10-L) is the virtual sound source coordinate r S and speaker coordinate r l Then, HPF10-l inputs the virtual sound source coordinate r S and speaker coordinate r l Based on this, as in step S203, a HPF h(r l ,r S The HPF 10-l receives the amplified signal from the signal amplifier 9-l and applies HPFh(r l ,r S ) and outputs the filtered signal to the attenuation adding section 11-l.

[0067] The attenuation adding unit 11-l (11-1 to 11-L) is a virtual sound source coordinate r S and speaker coordinate r l Input the virtual sound source coordinate r S and speaker coordinate rl Based on this, as in step S204, the damping coefficient 1 / ||r l -r S || 2 The attenuation adding unit 11-l receives the filtered signal from the HPF 10-l and sets an attenuation coefficient 1 / ||r l -r S || 2 By multiplying by , the signal is attenuated according to the distance between the virtual sound source 104 and the speaker 106-l. The attenuation adding unit 11-l outputs the attenuated signal (the signal to which attenuation has been added) to the surface element adding unit 12-l.

[0068] The area element adding unit 12-l (12-1 to 12-L) adds an area element ΔS l Enter the area element ΔS l takes a small value when the speakers 106-l are densely arranged, and takes a large value when the speakers 106-l are sparsely arranged.

[0069] The surface area element adding unit 12-l receives the signal to which attenuation has been added from the attenuation adding unit 11-l and adds an surface area element ΔS l The area element adding unit 12-l outputs the signal to which the area element has been added to the angle weighting unit 13-l.

[0070] The angle calculation unit 20 calculates the virtual sound source coordinate r S , speaker coordinates r1~r L and the inward normal vector n(r1)~n(r L ) is input. Then, as in step S205, the angle calculation unit 20 calculates a vector (r l -r S ) and calculate the vector (r l -r S ) and the inward normal vector n(r l ) and the angle θl The angle calculation unit 20 calculates the angle θ l (θ1~θ L ) to the corresponding angle weighting unit 13-l (13-1 to 13-L).

[0071] 3 is a block diagram showing an example of the configuration of the angle calculation unit 20. The angle calculation unit 20 includes calculation units 30-1 to 30-L. The calculation unit 30-1 is a component that calculates an angle θ1 formed with respect to the speaker 106-1, and calculates virtual sound source coordinates r S , as well as the l=1-th speaker coordinate r1 and the boundary surface inward normal vector n(r1). The calculation unit 30-1 then calculates a vector (r1-r S ) and calculate the vector (r1-r S Calculation unit 30-1 calculates angle θ1 between vector n(r1) facing inwardly toward the boundary surface at the position of speaker 106-1 and vector n(r1). Calculation unit 30-1 outputs angle θ1 to angle weighting unit 13-1.

[0072] Similarly, the calculation unit 30-L calculates the angle θ L A component that calculates the virtual sound source coordinates r S , and the l=Lth speaker coordinate r L and the inward normal vector n(r L ) is input to the calculation unit 30-L. Then, the calculation unit 30-L calculates a vector (r L -r S ) and calculate the vector (r L -r S ) and the inward normal vector n(r L ) and the angle θ L The calculation unit 30-L calculates the angle θ L to the angle weighting unit 13-L.

[0073] The same process is performed for the calculation units 30-2 to 30-(L-1), and the vectors (r2 to r L-1-r S ) and the inward normal vectors n(r2) to n(r L-1 ) and the angle θ2~θ L-1 are calculated respectively.

[0074] Returning to FIG. 1, the angle weighting unit 13-l (13-1 to 13-L) calculates the angle θ l (θ1~θ L ) and enter the angle θ l Based on this, in step S206, the angle weight w C (θ l The angle weighting unit 13-l receives the signal to which the area element has been added from the area element adding unit 12-l and assigns the angle weight w C (θ l ) to get the angle θ l A signal reflecting the

[0075] Angle θ l The signal reflecting the angle θ l The closer to 0°, the closer the signal is to the area element. l The signal reflecting the angle θ l The closer to 90°, the closer to a zero signal.

[0076] The angle weighting unit 13-l calculates the angle θ l The signal reflecting the drive signal d(r l ) to the speaker 106-l.

[0077] 1 and 2 show the configuration and processing of object-based audio rendering for the sound source signal S of one audio object, but in reality, the processing is performed for the sound source signals S of multiple audio objects. In this case, the driving signal d(r l ) are added for each speaker 106-l, and the driving signal resulting from the addition is output to the speaker 106-l. The same applies to Figs. 5 and 6 (embodiment 2) and Figs. 8 and 9 (embodiment 3) described later.

[0078] As described above, according to the object-based audio rendering device 1 of the first embodiment, the driving signal d(r l In the above equation (8) which generates -jk||rl-rs|| Ignoring the term jk||r l -r S The term ||+1 is filtered by a low-order high-pass filter h(r l ,r S )(HPF).

[0079] Specifically, the signal amplifier 9-l amplifies the sound source signal S by multiplying the sound source signal S by a constant u (which may be a different value for each system). S and speaker coordinate r l Based on this, the HPF h(r l ,r S ) is set to the HPF 10-l, and the signal amplified by the signal amplifier 9-l is subjected to the HPFh(r l ,r S ) is used to perform filtering.

[0080] The attenuation adding unit 11-l is a virtual sound source coordinate r S and speaker coordinate r l Based on the damping factor 1 / ||r l -r S || 2 Then, the attenuation adding unit 11-l sets the attenuation coefficient 1 / ||r l -r S || 2 By multiplying by , a signal is generated that is attenuated according to the distance between the virtual sound source 104 and the speaker 106-l.

[0081] The area element adding unit 12-l adds an area element ΔS of the speaker 106-l to the attenuated signal. l By multiplying by , a signal with an area element is generated.

[0082] The angle calculation unit 20 calculates a vector (r l -r S ) and the inward normal vector n(r l ) and the angle θ l Calculate.

[0083] The angle weighting unit 13-l calculates the angle θ l Based on this, the angle weight w C (θ l Then, the angle weighting unit 13-l assigns an angle weight w C (θ l ) to get the angle θ l The signal reflecting the drive signal d(r l ) to the speaker 106-l.

[0084] This allows e, which represents the time delay in equation (8), to be calculated. -jk||rl-rs|| The system can be simplified by ignoring the term jk||r. l -r S HPFh(r l ,r S The system can be simplified by using the HPF10-l.

[0085] Therefore, by simplifying the configuration of the WFS by approximation, it is possible to obtain a driving signal d(r l ) can be generated.

[0086] (Another example of HPF10-l) In addition, the HPF 10-l (10-1 to 10-L) of the object-based audio rendering device 1 shown in FIG. 1 is expressed by jk∥r in the above formula (8). l -r S HPFh(r l ,r S) is used to perform filtering on the signal amplified by the signal amplifier 9-l (9-1 to 9-L).

[0087] For example, in the above formula (8), jk||r l -r S The term ||+1 is replaced with a high-pass filter expressed by the following equation.

number

[0088] x[t] and y[t] are the input and output of the HPF 10-l at time t. n (r l ,r S ) is expressed by the following formula, where n = 0, 1, . . . , N.

number

[0089] In other words, HPF10-l is the virtual sound source coordinate r S and speaker coordinate r l The HPF 10-1 inputs the virtual sound source coordinates r S , speaker coordinate r l Based on the parameter α and the filter order N, HPF coefficients having the above equation (12) as coefficients are set.

[0090] The HPF 10-l receives the signal amplified by the signal amplifier 9-l (9-1 to 9-L) and applies the HPF coefficient h n (r l ,r S) to perform filtering so that the input / output characteristics of the HPF 10-l satisfy the above-mentioned formula (11). That is, the HPF 10-l is configured by an HPF having the input / output characteristics of the above-mentioned formula (11).

[0091] This allows the object-based audio rendering device 1 as a whole to be simplified without increasing the system scale.

[0092] (Another example of the attenuation adding portion 11-l) In addition, the attenuation adding unit 11-l (11-1 to 11-L) of the object-based sound rendering device 1 shown in FIG. S and speaker coordinate r l Based on the damping factor 1 / ||r l -r S || 2 and apply a damping factor of 1 / ||r l -r S || 2 By multiplying the signal by , a signal attenuated according to the distance between the virtual sound source 104 and the speaker 106-l is generated.

[0093] However, the damping coefficient 1 / ||r l -r S || 2 is the virtual sound source coordinate r S and speaker coordinate r l If the attenuation coefficient 1 / ||r l -r S || 2 The damping coefficient g(r l ,r S ) is used.

number

[0094] β1 is a positive real number for adjusting the signal amplitude, and β2 is the attenuation coefficient g(r l ,rS ) is a positive real number to suppress the divergence of the parameter β1 and β2. The values ​​of these parameters β1 and β2 are determined arbitrarily by the user. In particular, the value of the parameter β2 is l ,r S ) is set by the user to a small value to suppress the divergence of

[0095] That is, the attenuation adding unit 11-l calculates the absolute value ||r l -r S || is calculated, the absolute value is squared, and the parameter β2 is added to the result. l -r S || 2 +β2) and add the parameter β1 (||r l -r S || 2 +β2), the damping coefficient g(r l ,r S ) is required.

[0096] Then, the attenuation adding unit 11-l applies g(r l ,r S ) to generate a signal that is attenuated according to the distance between the virtual sound source 104 and the speaker 106-l.

[0097] The smaller the value of parameter β2, the smaller the damping coefficient g(r l ,r S ) behaves with a damping coefficient of 1 / ||r l -r S || 2 On the other hand, the virtual sound source coordinate r S is the speaker coordinate r l As the signal approaches , the signal amplitude increases dramatically.

[0098] This makes it possible to suppress signal divergence, and the user can adjust the fluctuation in signal amplitude when the position of the sound object changes according to the values ​​of parameters β1 and β2 that the user sets.

[0099] In addition, when the user sets the parameters β1 and β2 to β1=β2, the virtual sound source coordinate r S and speaker coordinate r l When (r S =r l ), damping coefficient g(r l ,r S )=1.

[0100] Therefore, the user can use the values ​​of these parameters β1 and β2 as a guide for adjustment so as not to unnecessarily increase or decrease the signal.

[0101] In addition, the user sets the values ​​of parameters β1 and β2 as β1=β2 / ΔS l When set to , the virtual sound source coordinate r S and speaker coordinate r l When (r S =r l ), damping coefficient g(r l ,r S )ΔS l =1, and even h(r l ,r S )g(r l ,r S )ΔS l w C (θ l )=1. h(r l ,r S )=1 and w C (θ l )=1.

[0102] Therefore, the virtual sound source coordinate r S and speaker coordinate r l When these values ​​match, the angle weighting unit 13-l outputs a driving signal d(r l ) is the same as the signal amplified by the signal amplifier 9-l. That is, the speaker coordinate r l From the speaker 106-l located at , an amplified signal is reproduced for the sound source signal S inputted by the object-based sound rendering device 1.

[0103] In this way, the damping coefficient g(r l ,r S ) to obtain the virtual sound source coordinate r S and speaker coordinate r l Even if the two coincide with each other, the signal will not diverge to infinity, and therefore the problem of being unable to reproduce the signal can be eliminated.

[0104] Example 2 Next, an object-based audio rendering device according to a second embodiment will be described. As described above, the second embodiment incorporates the configuration of VBAP into the configuration of the first embodiment, and in addition to the effects of the first embodiment, the second embodiment can reduce the driving signal d(r l In more detail, the second embodiment solves the problem that the driving signal d(r l )

[0105] In general, when using the WFS sound field reproduction method, it is assumed that many speakers 106-l are densely arranged. However, in home audio, it is difficult to densely arrange many speakers 106-l, and it is common to arrange the speakers 106-l sparsely.

[0106] 4 is a diagram for explaining a case where the speakers 106-l are sparsely arranged. As shown in FIG. 4, in the reproduced sound field, the speakers 106-l are sparsely arranged, and the position of the virtual sound source 104 (virtual sound source coordinates r S ) is close to the boundary surface of the area where you want to reproduce the sound field.

[0107] In this case, for all the speakers 106-1 (106-1 to 106-L), a vector (r l -r S ) and the inward normal vector n(r l ) and the angle θ lwill be close to or exceed 90°.

[0108] For this reason, the angle weight w used in the angle weight adding unit 13-l C (θ l ) is close to 0 or is 0 according to the above formula (9). Then, the driving signal d(r l ) becomes a small value or 0, causing a problem that the reproduced signal becomes small even though the virtual sound source 104 is close to the boundary surface.

[0109] In order to solve this problem, in the second embodiment, in the object-based audio rendering device 1 of the first embodiment shown in FIG. 1, the angle θ l A VBAP configuration is added so that the signal distribution law is continuously and seamlessly switched to the VBAP signal distribution law depending on the input.

[0110] In the object-based audio rendering device 1 of the first embodiment, which is a simplified WFS shown in FIG. l ,r S ) is a term related to frequency correction. Also, the attenuation coefficient 1 / ||r l -r S || 2 is expressed as a term relating to distance attenuation due to the position of the virtual sound source 104, and an area element ΔS l is interpreted as a term related to the correction by discretizing the speaker 106-l. Furthermore, the angle weight w C (θ l ) is interpreted as a term relating to the distribution side of the signal to each speaker 106-l.

[0111] In the second embodiment, the angle weight w of the angle weight adding unit 13-l in the WFS C (θ l A term relating to the distribution law of the signal for each speaker 106-l determined by VBAP is connected in parallel to a term relating to the distribution law of the signal for each speaker 106-l determined by VBAP.

[0112] FIG. 5 is a block diagram showing an example of the configuration of an object-based audio rendering apparatus according to the second embodiment, and FIG. 6 is a pseudo code showing an example of the processing of the object-based audio rendering apparatus according to the second embodiment.

[0113] This object-based acoustic rendering device 2 includes signal amplifiers 9-1 to 9-L, HPFs 10-1 to 10-L, attenuation adding units 11-1 to 11-L, area element adding units 12-1 to 12-L, angle weighting adding units 13-1 to 13-L, VBAP coefficient multipliers 14-1 to 14-L, angle weighting adding units 15-1 to 15-L, adders 16-1 to 16-L, an angle calculation unit 20, and a VBAP coefficient calculation unit 21.

[0114] Hereinafter, the components corresponding to the l-th speaker 106-l (l-th system) in the object-based acoustic rendering device 2 are defined as a signal amplifier 9-l, HPF 10-l, attenuation adding unit 11-l, area element adding unit 12-l, angle weight adding unit 13-l, VBAP coefficient multiplier 14-l, angle weight adding unit 15-l, and adder 16-l. In other words, the object-based acoustic rendering device 2 includes signal amplifiers 9-l, HPF 10-l, attenuation adding unit 11-l, area element adding unit 12-l, angle weight adding unit 13-l, VBAP coefficient multiplier 14-l, angle weight adding unit 15-l, and adder 16-l for L systems, and further includes an angle calculation unit 20 and a VBAP coefficient calculation unit 21.

[0115] The signal amplifier 9-l, HPF 10-l, attenuation unit 11-l, area element unit 12-l, and angle weight unit 13-l are components of the WFS. The VBAP coefficient multiplier 14-l and angle weight unit 15-l are components of the VBAP, and are connected in parallel to the angle weight unit 13-l of the WFS.

[0116] Comparing the object-based acoustic rendering device 1 of the first embodiment shown in FIG. 1 with the object-based acoustic rendering device 2 of the second embodiment, both object-based acoustic rendering devices 1 and 2 have in common the fact that they are equipped with a signal amplifier 9-l, a HPF 10-l, an attenuation adding unit 11-l, an area element adding unit 12-l, an angle weighting adding unit 13-l, and an angle calculation unit 20.

[0117] On the other hand, the object-based acoustic rendering device 2 differs from the object-based acoustic rendering device 1, which does not have these components, in that it is equipped with a VBAP coefficient multiplication unit 14-l, an angle weighting unit 15-l, an addition unit 16-l, and a VBAP coefficient calculation unit 21.

[0118] The overall process of the object-based audio rendering device 2 will be described with reference to Fig. 6. The object-based audio rendering device 2 receives the sound source signal S as an input, and also receives the virtual sound source coordinates r of the virtual sound source 104, in the same manner as in step S201 of Fig. 2. S (Step S601).

[0119] The object-based audio rendering device 2 calculates the speaker coordinate r l , area element ΔS l and the inward normal vector n(r l ) (step S602).

[0120] The object-based audio rendering device 2 calculates the virtual sound source coordinates r S and speaker coordinate r l Based on this, HPFh(r l ,r S ) is set (step S603).

[0121] The object-based audio rendering apparatus 2 calculates the virtual sound source coordinates r S and speaker coordinate r lBased on the damping coefficient 1 / ||r l -r S || 2 is set (step S604).

[0122] The object-based audio rendering device 2 calculates the virtual sound source coordinates r S , speaker coordinate r l and the inward normal vector n(r l ) based on the angle θ l is calculated (step S605).

[0123] The object-based audio rendering device 2 defines an angle θ l Absolute value of |θ l Determine whether | is greater than 0° and less than 90°, and use the WFS angle weight w C (θ l ), and the VBAP angle weight w S (θ l ) is set (step S606).

number

[0124] Specifically, the object-based audio rendering device 2 determines the angle |θ l If it is determined that | is greater than 0° and smaller than 90° (in the above formulas (9) and (14), cosθ l >0), angle weight w C (θ l )=cosθ l and angle weight w S (θ l )=|1-cosθ l On the other hand, the object-based audio rendering device 2 sets the angle |θ l is determined to be 0° or less or 90° or more (cosθ l > 0), angle weight w C (θl )=0 and angle weight w S (θ l ) = 0. This angle weight w C (θ l ) is used by the angle weighting unit 13-l, and the angle weight w S (θ l ) is used in the angle weighting unit 15-l.

[0125] Angle weight w C (θ l ) ranges from 0 to 1, and the angle |θ l | is 0°<|θ l In the range of |<90°, the angle |θ l The closer to 0° the closer 1(cosθ l ) and the closer it is to 90°, the closer it is to 0. l |θ l |≦0° or 90°≦|θ l In the range |, the value is 0.

[0126] Angle weight w S (θ l ) is the angle |θ l | is 0°<|θ l In the range of |<90°, the angle |θ l The closer | is to 0°, the closer it is to 0, and the closer it is to 90°, the closer it is to 1(|1-cosθ l |) and the angle |θ l |θ l |≦0° or 90°≦|θ l In the range |, the value is 0.

[0127] The object-based audio rendering device 2 calculates the virtual sound source coordinates r from a given sound receiving point (the position of the listener). S The unit vector of the virtual sound source direction to the speaker coordinate r l Based on the unit vector of the speaker direction to , the three speakers including the virtual sound source 104 therein are identified as the three speakers 106-n1, 106-n2, and 106-n3 targeted for VBAP.

[0128] The object-based audio rendering device 2 calculates the unit vector r n1 ,r n2 ,r n3 (Step S607). The unit vector r n1 ,r n2 ,r n3 can be obtained based on the coordinates (speaker coordinates) of a predetermined sound receiving point and the speakers 106-n1, 106-n2, and 106-n3, respectively. n1, n2, and n3 are indexes of the three speakers 106-n1, 106-n2, and 106-n3 targeted for VBAP.

[0129] The object-based audio rendering device 2 is a virtual sound source coordinate r S The unit vector of the virtual sound source direction to the speaker direction r n1 ,r n2 ,r n3 Based on this, the coefficient g corresponding to the speakers 106-n1, 106-n2, and 106-n3 is calculated using the following formula: l =g n1 ,g n2 ,g n3 is calculated (step S608).

number

[0130] In addition, the object-based audio rendering device 2 calculates the coefficient g of the speaker 106-l other than the speakers 106-n1, 106-n2, and 106-n3 among the speakers 106-1 to 106-L. l Set = 0. These coefficients g l =g n1 ,g n2 ,g n3 , 0 are calculated and set by the VBAP coefficient calculation unit 21.

[0131] The object-based audio rendering device 2 calculates the constant u (for example, u=1 / 2π), the sound source signal S input in step S601, and the virtual sound source coordinates r for the speakers 106-n1, 106-n2, and 106-n3 with indexes l=n1, n2, and n3. S , the speaker coordinate r input in step S602 l and area element ΔS l , HPFh(r l ,r S ), the damping coefficient 1 / ||r set in step S604 l -r S || 2 and the angle weight w set in step S606 C (θ l ) and angle weight w S (θ l ) based on the following formula, the drive signal d(r l ) is calculated and output (step S609).

number

[0132] In this formula, the coefficient g l =g n1 ,g n2 ,g n3 In addition, * indicates convolution. The driving signal d(r l ) is calculated by a signal amplifier 9-l, a HPF 10-l, an attenuation adding section 11-l, an area element adding section 12-l, an angle weighting adding section 13-l, a VBAP coefficient multiplier 14-l, an angle weighting adding section 15-l and an adder 16-l.

[0133] On the other hand, the object-based audio rendering device 2 calculates a constant u, a sound source signal S, a virtual sound source coordinate r for the index l other than n1, n2, and n3, that is, for the speaker 106-l other than the speakers 106-n1, 106-n2, and 106-n3. S , speaker coordinate r l and area element ΔS l , HPFh(r l ,r S), damping coefficient 1 / ||r l -r S || 2 and angle weight w C (θ l ) based on the following formula, the drive signal d(r l ) is calculated and output (step S610).

number

[0134] This driving signal d(r l ) is calculated by the signal amplifier 9-l, the HPF 10-l, the attenuator 11-l, the area element adder 12-l and the angle weighter 13-l.

[0135] Next, the processing of the components of the object-based acoustic rendering device 2 will be described with reference to Fig. 5. The signal amplifiers 9-l (9-1 to 9-L), HPFs 10-l (10-1 to 10-L), attenuation adding units 11-l (11-1 to 11-L), area element adding units 12-l (12-1 to 12-L), angle weight adding units 13-l (13-1 to 13-L) and angle calculation unit 20 are the same as the components shown in Fig. 1, so their description will be omitted.

[0136] Here, the angle weighting unit 13-l calculates the angle θ l The WFS drive signal reflecting the angle θ is output to the corresponding adder 16-l as a first drive signal. l to the corresponding angle weighting units 13-l and 15-l.

[0137] The VBAP coefficient calculation unit 21 calculates the virtual sound source coordinate r S and speaker coordinates r1~r L Then, in steps S607 and S608, the VBAP coefficient calculation unit 21 calculates the virtual sound source coordinate r S and the unit vector of the virtual sound source direction to the speaker coordinates r1 to r2 from the predetermined sound receiving points in all the speakers 106-1 to 106-L. LBased on the unit vector of the speaker direction to the virtual sound source 104, the coefficient g corresponding to the three speakers 106-n1, 106-n2, and 106-n3 of the VBAP target including the virtual sound source 104 therein is calculated. n1 ,g n2 ,g n3 Calculate.

[0138] The VBAP coefficient calculation unit 21 calculates the coefficient g l =g n1 ,g n2 ,g n3 to the VBAP coefficient multipliers 14-n1, 14-n2, and 14-n3 corresponding to the speakers 106-n1, 106-n2, and 106-n3, respectively. l =0 to the VBAP coefficient multiplication units 14-l corresponding to the speakers 106-l other than the speakers 106-n1, 106-n2, and 106-n3.

[0139] 7 is a block diagram showing an example of the configuration of the VBAP coefficient calculation unit 21. The VBAP coefficient calculation unit 21 includes a speaker determination unit 31 and a calculation unit 32. The speaker determination unit 31 determines a virtual sound source coordinate r S and the speaker coordinates r1 to r of all speakers 106-1 to 106-L. L Then, the speaker determination unit 31 inputs the virtual sound source coordinates r S and speaker coordinates r1~r L Based on this, three speakers 106-n1, 106-n2, and 106-n3 that are VBAP targets and that include the virtual sound source 104 therein are identified.

[0140] The speaker determination unit 31 determines the unit vector r n1 ,r n2 ,r n3 and obtain the virtual sound source coordinate r S and unit vector r in the speaker direction n1 ,r n2 ,r n3 is output to the calculation unit 32.

[0141] The calculation unit 32 receives the virtual sound source coordinates r S and unit vector r in the speaker direction n1 ,r n2 ,r n3 Then, the calculation unit 32 inputs the virtual sound source coordinates r S The unit vector of the virtual sound source direction to the speaker direction r n1 ,r n2 ,r n3 Based on this, the coefficient g corresponding to the speakers 106-n1, 106-n2, and 106-n3 in the above equation (15) is n1 ,g n2 ,g n3 Then, the calculation unit 32 calculates the coefficient g n1 ,g n2 ,g n3 to the corresponding VBAP coefficient multiplication units 14-n1, 14-n2, and 14-n3.

[0142] In addition, the calculation unit 32 calculates a coefficient g corresponding to the speaker 106-l other than the speakers 106-n1, 106-n2, and 106-n3. l =0 and outputs it to the corresponding VBAP coefficient multiplication unit 14-l.

[0143] Returning to FIG. 5, the VBAP coefficient multiplication unit 14-l (14-1 to 14-L) receives the coefficient g l (g n1 ,g n2 ,g n3 , 0) is input to the VBAP coefficient multiplier 14-l. The VBAP coefficient multiplier 14-l inputs the signal to which the area element has been added from the area element adding unit 12-l, and applies a coefficient g l By multiplying by the coefficient g l Generates a signal that reflects the

[0144] The VBAP coefficient multiplication unit 14-l calculates the coefficient g l The signal reflecting the above is output to the angle weighting unit 15-l.

[0145] The angle weighting unit 15-l (15-1 to 15-L) calculates the angle θ l (θ1~θL ) and enter the angle θ l Based on this, in step S606, the angle weight w S (θ l Then, the angle weighting unit 15-l sets the coefficient g l The signal reflecting the angle weight w S (θ l ) to get the angle θ l Generates a signal that reflects the

[0146] The angle θ l The signal reflecting the angle θ l The closer to 90°, the greater the coefficient g l On the other hand, the angle θ l The sound source signal S reflecting the angle θ l The closer to 0° the signal is, the closer it is to 0. In addition, in the angle weighting unit 15-l corresponding to the speaker 106-l other than the three speakers 106-n1, 106-n2, and 106-n3 including the virtual sound source 104 therein, the angle θ l The signal reflecting this is 0.

[0147] The angle weighting unit 15-l calculates the angle θ l The signal reflecting this is output to the adder 16-l as the second drive signal.

[0148] The addition unit 16-l (16-1 to 16-L) calculates the angle θ l The first drive signal reflecting the angle θ l The adder 16-l receives the second drive signal reflecting the angle θ l The first drive signal reflecting the angle θ l The second drive signal reflecting the above is added to the drive signal d(r l ) to the speaker 106-l.

[0149] As described above, according to the object-based audio rendering device 2 of the second embodiment, the VBAP configuration is incorporated into the WFS configuration of the first embodiment, and a part of the WFS configuration and the VBAP configuration are seamlessly switched according to the angle weighting, thereby reducing the driving signal d(r l ) is now generated.

[0150] Specifically, the VBAP coefficient calculation unit 21 calculates the virtual sound source coordinate r S Unit vector of the virtual sound source direction and speaker coordinates r1 to r L Based on the unit vector of the speaker direction to the virtual sound source 104, the coefficient g n1 ,g n2 ,g n3 Calculate.

[0151] The VBAP coefficient multiplication unit 14-l receives the coefficient g from the VBAP coefficient calculation unit 21. l (g n1 ,g n2 ,g n3 , 0) to the signal obtained by the signal amplifier 9-l, the HPF 10-l, the attenuation adding unit 11-l, and the area element adding unit 12-l. l By multiplying by the coefficient g l A signal reflecting the

[0152] The angle weighting unit 15-l calculates the angle θ l Based on this, the angle weight w S (θ l ) is set. Then, the angle weighting unit 15-l sets the coefficient g l The signal reflecting the angle weight w S (θ l ) to get the angle θ l A second drive signal reflecting the above is generated.

[0153] The adder 16-l adds the angle θ 2 , which is obtained by processing the WFS consisting of the signal amplifier 9-l, the HPF 10-l, the attenuation adding unit 11-l, the area element adding unit 12-l, and the angle weighting adding unit 13-l. l A first drive signal reflecting the above, and an angle θ l The second drive signal reflecting the above is added to the drive signal d(r l ) to the speaker 106-l.

[0154] In this way, the angle weight w S (θ l The angle weighting unit 15-l using the coefficient g l The VBAP coefficient multiplier 14-l and the angle weighting unit 15-l of the VBAP are connected in series to the angle weighting unit 13-l of the WFS. C (θ l ) and the angle weight w used in the angle weighting unit 15-l S (θ l ) is in a trade-off relationship in the range of 0 to 1.

[0155] Therefore, the position of the virtual sound source 104 (virtual sound source coordinates r S ) is away from the boundary surface of the speaker 106-l, and the angle θ l If is approaching 0°, the angle weight w C (θ l ) is a large value close to 1, but the angle weight w S (θ l ) is a small value close to 0. In this case, the drive signal d(r l ) will be dominated by the distribution side through WFS.

[0156] On the other hand, the position of the virtual sound source 104 (virtual sound source coordinates r S ) is close to the boundary surface of the speaker 106-l and forms an angle θ l If is approaching 90°, the angle weight w C (θ l ) is a small value close to 0, but the angle weight w S (θ l) is a large value close to 1. In this case, the driving signal d(r l ) the distribution side based on VBAP will dominate.

[0157] That is, depending on the position of the virtual sound source 104 relative to the speaker 106-l, the distribution side based on WFS and the distribution side based on VBAP are continuously and seamlessly switched over.

[0158] Therefore, the object-based audio rendering device 2 of the second embodiment can achieve the same effects as those of the first embodiment. In addition, in an environment in which the speakers 106-l are sparsely arranged, the position of the virtual sound source 104 (virtual sound source coordinates r S ) is close to the boundary surface of the speaker 106-l and forms an angle θ l As the angle weight of WFS becomes larger, the angle weight of VBAP becomes larger, and the rendering rule is switched continuously. Therefore, even if the playback signal becomes smaller due to the smaller angle weight of WFS, the problem of the playback signal becoming extremely small can be solved by compensating for it with the angle weight of VBAP.

[0159] The HPF 10-l (10-1 to 10-L) of the object-based audio rendering device 2 of the second embodiment is similar to the first embodiment shown in FIG. l -r S ||+1 is the HPF coefficient h n (r l ,r S ) may be used in place of the high-pass filter of the above-mentioned equation (11).

[0160] 1. Also, the attenuation adding unit 11-l (11-1 to 11-L) of the object-based acoustic rendering device 2 of the second embodiment has an attenuation coefficient 1 / ||r l -r S || 2 Instead of the damping coefficient g(r l ,r S ) may be used. In other words, the attenuation adding unit 11-l may use the attenuation coefficient g(rl ,r S ) is used to generate a signal that is attenuated according to the distance between the virtual sound source 104 and the speaker 106-l.

[0161] Example 3 Next, a third embodiment will be described. As described above, the third embodiment, like the second embodiment, incorporates the configuration of VBAP into the configuration of the first embodiment, and in addition to the effects of the first embodiment, achieves the following effects: l In more detail, the third embodiment solves the problem that the driving signal d(r l )

[0162] FIG. 8 is a block diagram showing an example of the configuration of an object-based audio rendering apparatus according to the third embodiment, and FIG. 9 is a pseudo code showing an example of the processing of the object-based audio rendering apparatus according to the third embodiment.

[0163] This object-based acoustic rendering device 3 includes signal amplifiers 9-1 to 9-L, HPFs 10-1 to 10-L, attenuation adding units 11-1 to 11-L, area element adding units 12-1 to 12-L, angle weighting adding units 13-1 to 13-L, VBAP coefficient multipliers 14-1 to 14-L, angle weighting adding units 15-1 to 15-L, adders 16-1 to 16-L, an angle calculation unit 20, and a VBAP coefficient calculation unit 21.

[0164] The signal amplifier 9-l, HPF 10-l, attenuation adding unit 11-l, area element adding unit 12-l and angle weight adding unit 13-l are components of the WFS, and the VBAP coefficient multiplier 14-l and angle weight adding unit 15-l are components of the VBAP. The components of the WFS and the components of the VBAP are connected in parallel as the object-based acoustic rendering device 3 as a whole.

[0165] Comparing the object-based acoustic rendering device 2 of the second embodiment shown in FIG. 2 with the object-based acoustic rendering device 3 of the third embodiment, both the object-based acoustic rendering devices 2 and 3 have the same components in common.

[0166] On the other hand, the object-based acoustic rendering device 3 differs from the object-based acoustic rendering device 2 in that the VBAP components, that is, the VBAP coefficient multiplication unit 14-l and the angle weighting unit 15-l, are connected in parallel to the entire WFS components, in that they are connected in parallel to only a part of the WFS, the angle weighting unit 13-l.

[0167] The overall processing of the object-based acoustic rendering device 3 will be described with reference to Fig. 9. The processing of steps S901 to S908 shown in Fig. 9 is the same as the processing of steps S601 to S608 by the object-based acoustic rendering device 2 of the second embodiment shown in Fig. 6, so the description thereof will be omitted here.

[0168] The object-based audio rendering device 3 uses the speaker coordinate r acquired in step S907. l When the index of is l=n1, n2, n3, for the speakers 106-n1, 106-n2, and 106-n3, a constant u (for example, u=1 / 2π), the sound source signal S input in step S901, and the virtual sound source coordinate r S , the speaker coordinate r input in step S902 l and area element ΔS l , HPFh(r l ,r S ), the damping coefficient 1 / ||r set in step S904 l -r S || 2 and the angle weight w set in step S906 C (θ l ) and angle weight w S (θ l ) based on the following formula, the drive signal d(r l ) is calculated and output (step S909).

number

[0169] In this formula, the coefficient g l =g n1 ,g n2 ,g n3 In addition, * indicates convolution. The driving signal d(r l ) is calculated by a signal amplifier 9-l, a HPF 10-l, an attenuation adding section 11-l, an area element adding section 12-l, an angle weighting adding section 13-l, a VBAP coefficient multiplier 14-l, an angle weighting adding section 15-l and an adder 16-l.

[0170] On the other hand, when the index is other than n1, n2, n3, the object-based audio rendering device 3 calculates the constant u, the sound source signal S, the virtual sound source coordinate r for the speaker 106-l other than the speakers 106-n1, 106-n2, and 106-n3. S , speaker coordinate r l and area element ΔS l , HPFh(r l ,r S ), damping coefficient 1 / ||r l -r S || 2 and angle weight w C (θ l ) based on the following formula, the drive signal d(r l ) is calculated and output (step S910).

number

[0171] This driving signal d(r l ) is calculated by the signal amplifier 9-l, the HPF 10-l, the attenuator 11-l, the area element adder 12-l and the angle weighter 13-l.

[0172] In the calculation of the above formula (18), a delay occurs due to filtering between the calculation result of the WFS configuration, which is the first term on the right side, and the calculation result of the VBAP configuration, which is the second term. For this reason, in the angle weighting units 13-l and 15-l, it is necessary to add a delay to the VBAP side to adjust the phase, but this is not shown here.

[0173] In addition, in the formulas (18) and (19), the convolution* may be calculated by inversely transforming the product in the frequency domain using FFT (Fast Fourier Transform). The same applies to the formulas (16) and (17).

[0174] Referring to Figure 8, the processing of each component of the object-based acoustic rendering device 3 is the same as the processing of each component of the object-based acoustic rendering device 2 of the embodiment 2 shown in Figure 5, so the explanation of the processing of the components is omitted.

[0175] The input data of the VBAP coefficient multiplier 14-l (14-1 to 14-L) differs between the object-based acoustic rendering devices 2 and 3. The VBAP coefficient multiplier 14-l of the object-based acoustic rendering device 2 shown in Fig. 5 inputs a signal to which an area element has been added from the area element adding unit 12-l. In contrast, the VBAP coefficient multiplier 14-l of the object-based acoustic rendering device 3 shown in Fig. 8 inputs a sound source signal S.

[0176] Specifically, the VBAP coefficient multiplier 14-l of the object-based audio rendering device 3 shown in FIG. 8 receives the coefficient g l (g n1 ,g n2 ,g n3 , 0) is input to the VBAP coefficient multiplier 14-l, which inputs the sound source signal S and multiplies the sound source signal S with a coefficient g l By multiplying by the coefficient g l Generates a signal that reflects the

[0177] As described above, according to the object-based audio rendering device 3 of the third embodiment, the VBAP configuration is incorporated into the WFS configuration of the first embodiment, and the entire WFS configuration and the VBAP configuration are seamlessly switched according to the angle weighting, thereby reducing the driving signal d(r l )

[0178] Specifically, the VBAP coefficient multiplication unit 14-l receives the coefficient g l (g n1 ,g n2 ,g n3 , 0) is input to the VBAP coefficient multiplier 14-l. Then, the VBAP coefficient multiplier 14-l multiplies the sound source signal S by a coefficient g l By multiplying by the coefficient g l A signal reflecting the

[0179] The angle weighting unit 15-l calculates the angle θ l Based on this, the angle weight w S (θ l ) is set. Then, the angle weighting unit 15-l sets the coefficient g l The signal reflecting the angle weight w S (θ l ) to get the angle θ l A second drive signal reflecting the above is generated.

[0180] The adder 16-l adds the angle θ 2 , which is obtained by processing the WFS consisting of the signal amplifier 9-l, the HPF 10-l, the attenuation adding unit 11-l, the area element adding unit 12-l, and the angle weighting adding unit 13-l. l A first drive signal reflecting the above, and an angle θ l The second drive signal reflecting the above is added to the drive signal d(r l ) to the speaker 106-l.

[0181] In this way, the angle weight w S (θ l The angle weighting unit 15-l using the coefficient g lThe VBAP coefficient multiplier 14-l and angle weighting unit 15-l of the VBAP are connected in series to the signal amplifier 9-l, HPF 10-l, attenuation unit 11-l, area element adding unit 12-l and angle weighting unit 13-l of the WFS. The angle weight w C (θ l ) and the angle weight w used in the angle weighting unit 15-l S (θ l ) is in a trade-off relationship in the range of 0 to 1.

[0182] For this reason, as in the second embodiment, θ l When approaches 0°, the drive signal d(r l ) is dominated by the distribution side by WFS, and θ l When the driving signal d(r l ) the distribution side based on VBAP will dominate.

[0183] That is, the distribution side between WFS and VBAP is switched continuously and seamlessly depending on the position of the virtual sound source 104 with respect to the speaker 106-l.

[0184] Therefore, the object-based audio rendering device 3 of the third embodiment can achieve the same effect as that of the first embodiment. Also, as in the second embodiment, when rendering by WFS is performed in an environment where the speakers 106-l are sparsely arranged, if the virtual sound source 104 is placed close to a boundary surface, the rendering is switched to rendering by VBAP continuously and seamlessly. This solves the problem of the playback signal becoming small.

[0185] The HPF 10-l (10-1 to 10-L) of the object-based audio rendering device 3 of the third embodiment is similar to the first embodiment shown in FIG. 1 and the second embodiment shown in FIG. 5 in that jk∥r l -r S ||+1 is the HPF coefficient h n (r l ,r S) may be used in place of the high-pass filter of the above-mentioned equation (11).

[0186] 5. The attenuation coefficient 1 / ||r l -r S || 2 Instead of the damping coefficient g(r l ,r S ) may be used. In other words, the attenuation adding unit 11-l may use the attenuation coefficient g(r l ,r S ) is used to generate a signal that is attenuated according to the distance between the virtual sound source 104 and the speaker 106-l.

[0187] However, when the virtual sound source 104 is placed close to the speaker 106-l, the signal amplitude may differ significantly between the simplified WFS configuration (HPF 10-l to angle weighting unit 13-l) and the VBAP configuration (VBAP coefficient multiplication unit 14-l and angle weighting unit 15-l). For this reason, in order to prevent the signal amplitude from differing too much, the attenuation coefficient g(r l ,r S The parameters β1 and β2 used to calculate β1, β2 need to be appropriately adjusted.

[0188] Although the present invention has been described above with reference to the first, second and third embodiments, the present invention is not limited to the first, second and third embodiments and can be modified in various ways without departing from the technical concept thereof.

[0189] A normal computer can be used as the hardware configuration of the object-based acoustic rendering devices 1, 2, and 3 according to the first, second, and third embodiments of the present invention. The object-based acoustic rendering devices 1, 2, and 3 are configured by a computer equipped with a CPU, a volatile storage medium such as a RAM, a non-volatile storage medium such as a ROM, and an interface, etc.

[0190] The functions of the HPFs 10-1 to 10-L, the attenuation adding units 11-1 to 11-L, the area element adding units 12-1 to 12-L, the angle weighting adding units 13-1 to 13-L and the angle calculation unit 20 provided in the object-based acoustic rendering device 1 are each realized by causing a CPU to execute a program in which these functions are written.

[0191] Furthermore, the functions of HPFs 10-1 to 10-L, attenuation adding units 11-1 to 11-L, area element adding units 12-1 to 12-L, angle weighting adding units 13-1 to 13-L, VBAP coefficient multiplier units 14-1 to 14-L, angle weighting adding units 15-1 to 15-L, adder units 16-1 to 16-L, angle calculation unit 20 and VBAP coefficient calculation unit 21 provided in the object-based acoustic rendering devices 2 and 3 are each realized by having a CPU execute a program in which these functions are described.

[0192] These programs are stored in the storage medium and are read and executed by the CPU. These programs can also be distributed by storing them in a storage medium such as a magnetic disk (floppy disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), or a semiconductor memory, and can also be transmitted and received via a network. [Explanation of symbols]

[0193] 1,2,3 Object-based acoustic rendering system 9 Signal Amplification Section 10 HPF (High Pass Filter) 11 Damping section 12 Addition of area element 13,15 Angle weighting section 14 VBAP coefficient multiplication section 16 Addition section 20 Angle calculation section 21 VBAP coefficient calculation section 30,32 Calculation section 31 Speaker determination unit 100 receiving points 101,104 Virtual Sound Source 102,106 Speakers 103 Sound Source 105 Speaker Array r nS Unit vector in the direction of the virtual sound source r n1 ,r n2 ,r n3 Unit vector in the speaker direction g l ,g n1 ,g n2 ,g n3 coefficient r S Virtual sound source coordinates r l Speaker Coordinates C1,C2 area S sound source signal L Number of speakers l Speaker number (system number) ΔS l Speaker area element n(r l ) Normal vector inward of the speaker boundary surface k wavenumber θ l The angle (vector (r l -r S ) and the inward normal vector n(r l ) and the angle h(r l ,r S ) HPF w C (θ l ),w S (θ l ) Angle weight 1 / ||r l -r S || 2 ,g(r l ,r S ) damping coefficient d(r l ) Drive signal

Claims

1. 1. An object-based audio rendering device that generates drive signals for a plurality of speakers arranged in a listening environment by performing rendering on a sound source object arranged at a position described in audio metadata, The number of the plurality of speakers is L, the number of the speakers is l (=1, . . . , L), the lth speaker is the lth speaker, and the coordinates of the virtual sound source from which the sound source signal is output are virtual sound source coordinates r S , the coordinates of the first speaker are speaker coordinates r l , the boundary surface inward normal vector at the position of the first speaker is n(r l ), the area element centered on the position of the first speaker is ΔS l , the driving signal for the first speaker is d(r l ) as For each of the plurality of speakers, the virtual sound source coordinates r S The speaker coordinate r of the first speaker from the position l and the boundary surface inward normal vector n(r l ) and the angle θ l An angle calculation unit that calculates a signal amplifier, a high pass filter, an attenuation adding unit, an area element adding unit, and a first angle weighting unit, each of which corresponds to one of the plurality of speakers; The signal amplifier unit includes: amplifying the sound source signal by a constant; The HPF is The virtual sound source coordinate r S and the speaker coordinate r of the first speaker. l Based on this, HPFh (r l , r S ) is set, and the HPFh(r l , r S ) to filter the The attenuation adding unit is The virtual sound source coordinate r S The speaker coordinate r of the first speaker from the position l The absolute value of the vector up to the position is calculated, and the reciprocal of the result of squaring the absolute value is used as the attenuation coefficient 1 / ||r l -r S || 2 and applying the attenuation coefficient 1 / ||r l -r S || 2 Multiply by The surface area element adding unit The area element ΔS for the signal multiplied by the attenuation adding unit l Multiply by The first angle weighting unit The angle θ of the first speaker calculated by the angle calculation unit l Based on this, the angle θ l The closer the absolute value of is to 0°, the closer it is to 1, and the angle θ l The closer the absolute value of is to 90°, the closer it is to 0, and the angle θ l The angle weight w takes a value of 0 when the absolute value of is less than 0° or greater than 90°. C (θ l ) is set, and the angle weight w C (θ l ) to obtain the driving signal d(r l ) to generate an object-based audio rendering apparatus.

2. 2. The object-based audio rendering apparatus of claim 1, Furthermore, the virtual sound source coordinate r S , and the speaker coordinates r l Based on the unit vector of the speaker direction to the target speaker, the three speakers including the virtual sound source therein are identified as three speakers subject to VBAP (Vector Based Amplitude Panning), and the unit vector r n1 , r n2 , r n3 and obtain the virtual sound source coordinate r S The unit vector of the virtual sound source direction to the VBAP target speaker and the unit vector r n1 , r n2 , r n3 Based on the above, the coefficients g corresponding to the three speakers of the VBAP target are calculated. l = g n1 , g n2 , g n3 and obtain a coefficient g corresponding to the speakers other than the three VBAP target speakers among all the speakers. l = 0, and the coefficient g l A VBAP coefficient calculation unit that outputs a VBAP coefficient multiplier, a second angle weighting unit, and an adder, each corresponding to one of the plurality of speakers; The VBAP coefficient multiplication unit The coefficient g output by the VBAP coefficient calculation unit for the signal multiplied by the area element addition unit l Multiply by The second angle weighting unit is The angle θ of the first speaker calculated by the angle calculation unit l Based on this, the angle θ l The closer the absolute value of is to 0°, the closer the value to 0 is. l The closer the absolute value of is to 90°, the closer the value is to 1. l The angle weight w takes a value of 0 when the absolute value of is less than 0° or greater than 90°. S (θ l ) is set, and the angle weight w S (θ l ) to generate a second drive signal; The first angle weighting unit The angle weight w C (θ l ) is set, and the angle weight w C (θ l ) to generate a first drive signal, and the adder adds the first drive signal and the second drive signal to generate the drive signal d(r l ) to generate an object-based audio rendering apparatus.

3. 3. The object-based audio rendering apparatus of claim 2, The VBAP coefficient multiplication unit Instead of the signal multiplied by the area element adding unit, the sound source signal is input, and the coefficient g l and multiplying the first and second inputs by a first factor of the audio signal.

4. An object-based acoustic rendering apparatus according to any one of claims 1 to 3, The HPF is With a preset parameter as α, a preset filter order as N, and n=0, 1, . . . , N, the following equation is used: In, the HPF coefficient h n (r l , r S ) and Let x[t] and y[t] be the input and output of the HPF at time t, and the input / output characteristics are expressed by the following formula: The HPF coefficient h n (r l , r S ) for performing filtering.

5. An object-based acoustic rendering apparatus according to any one of claims 1 to 4, The attenuation adding unit is The preset parameters are 1 , β 2 As the following formula: The damping coefficient g (r l , r S ) and calculate the attenuation coefficient g(r l , r S ) multiplying the input image by 1.

6. A program for causing a computer to function as the object-based acoustic rendering device according to any one of claims 1 to 5.

Citation Information

Patent Citations

  • ITBS.2127-0

  • Device and method of calculating drive coefficient of speaker in speaker facility based on audio signal related to virtual sound source, and device and method of supplying drive signal of speaker in speaker facility

    JP2014090504A

  • Sound signal processing device, sound reproduction apparatus and sound signal processing program

    JP2014220614A

  • Sound image generating apparatus

    JP2019016871A