Device, method, and sound system

Abstract

The present disclosure relates to an apparatus comprising a circuit configured to control a speaker arrangement comprising at least one virtual speaker and at least one real speaker to generate at least one virtual sound source, wherein the at least one virtual sound source is generated based on contributions of the at least one virtual speaker and the at least one real speaker, and wherein a sound field modulation function configured to generate an acoustic impression for a user is used to modulate the sound field emitted by the virtual sound source so that the acoustic impression is generated at a predetermined position.

Description

Technical Field

[0001] The present disclosure generally relates to an apparatus, method, and sound system for generating a height impression of sound for a user. Background Art

[0002] Generally, known audio systems may use a large number of loudspeakers, for example due to an increasing demand for hardware such as new amplifier channels and the loudspeakers themselves.

[0003] In order to render the impression of height, which is now required in formats such as Dolby Atmos, DTS-X, Auro-3D and NHK or 360RA at the bottom (i.e. "negative" height impression), the current method is to use additional speakers placed at a certain height, ceiling reflections or speakers on the floor. These methods require wiring and fixings and may not be optimal from an aesthetic point of view.

[0004] Other approaches, such as acoustic pressure bars, typically reduce the system to a frontal system. However, such systems can have a limited sweet spot, which is primarily located on an axis orthogonal to the device. Summary of the invention

[0005] It is therefore generally desirable to provide an improved apparatus and method for providing audio output.

[0006] According to a first aspect, the present disclosure provides an apparatus comprising a circuit configured to control a speaker arrangement comprising at least one virtual speaker and at least one real speaker to generate at least one virtual sound source, wherein the at least one virtual sound source is generated based on contributions of the at least one virtual speaker and the at least one real speaker, and wherein a sound field modulation function configured to generate an acoustic impression for a user is used to modulate the sound field emitted by the virtual sound source so that the acoustic impression is generated at a predetermined position.

[0007] According to a second aspect, the present disclosure provides a method, comprising: controlling a speaker arrangement including at least one virtual speaker and at least one real speaker to generate at least one virtual sound source, wherein the at least one virtual sound source is generated based on contributions of the at least one virtual speaker and the at least one real speaker, and wherein a sound field modulation function configured to generate an acoustic impression for a user is used to modulate the sound field emitted by the virtual sound source so that an acoustic impression is generated at a predetermined position. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Embodiments are described by way of example with reference to the accompanying drawings, in which:

[0009] Figure 1 is an explanatory diagram for explaining an angle of a head-related transfer function associated with a listener;

[0010] Figure 2 is an overview of a system for generating a speaker of a virtual sound source according to an embodiment of the present disclosure;

[0011] Figure 3 is a coordinate system diagram showing different expansion factors according to an embodiment of the present disclosure;

[0012] Figure 4 shows the generation of a virtual speaker with horizontal panning between two speakers;

[0013] Figure 5 shows the generation of a virtual loudspeaker with a head-related transfer function;

[0014] Figure 6 An embodiment of an arrangement of speakers for generating a virtual speaker is shown;

[0015] Figure 7 An embodiment showing how to generate a virtual height speaker by applying a head-related transfer function to a virtual speaker;

[0016] Figure 8 An embodiment of a system including a speaker, a virtual speaker, and a moving virtual sound source is shown;

[0017] Fig. 9 is a diagram of a method for determining a function to be applied to a speaker to generate a virtual speaker;

[0018] Fig.10 is a diagram of a method for generating a set of virtual loudspeakers;

[0019] Fig.11 An electronic device for an audio system according to an embodiment of the present disclosure is depicted;

[0020] Fig.12 An implementation method for 3D audio rendering based on a digital monopole synthesis algorithm is provided;

[0021] Fig.13 is a diagram of a method for generating a set of virtual loudspeakers and for moving a virtual sound source; and

[0022] Fig.14 is a diagram of a method according to the present disclosure. DETAILED DESCRIPTION

[0023] In reference Figure 6 Before the following detailed description of the embodiments, some general remarks are given.

[0024] Some embodiments of the present disclosure relate to an apparatus comprising a circuit configured to control a speaker arrangement comprising at least one virtual speaker and at least one real speaker to generate at least one virtual sound source, wherein the at least one virtual sound source is generated based on contributions of the at least one virtual speaker and the at least one real speaker, and wherein a sound field modulation function configured to generate an acoustic impression for a user is used to modulate the sound field emitted by the virtual sound source so that the acoustic impression is generated at a predetermined position.

[0025] The device may be any device suitable for controlling a speaker arrangement, such as a processor, an amplifier (such as an electronic amplifier, such as a single-sided amplifier, a bilateral amplifier, an inverting amplifier, a non-inverting amplifier, a servo amplifier, a linear amplifier, a non-linear amplifier, a broadband amplifier, a radio frequency amplifier, an audio amplifier, a resistor-capacitor coupled amplifier (RC), an inductor-capacitor coupled amplifier (LC), a transformer-coupled amplifier, a direct-coupled amplifier, etc. The device may also be a 3D audio rendering system, such as an ambisonics system, a wave field synthesis system, a surround sound system, etc. The device (circuit) may be configured to generate a signal to perform control as described herein, wherein the control signal may be applied to a speaker arrangement, etc.

[0026] 3D audio rendering operations can be based on wave field synthesis, which can be used to generate a sound field that gives the impression that audio point sources are located within a predefined space. This impression can be achieved by using a wave field synthesis method that drives an array of speakers, thereby generating the impression of virtual sound sources.

[0027] In some implementations, 3D audio rendering operations may be based on monopole synthesis.

[0028] The theoretical background of the present technology used in some embodiments is described in more detail in patent application US 2016 / 0037282 A1, which is incorporated herein by reference.

[0029] The technique implemented in the embodiment of US 2016 / 0037282 A1 is conceptually similar to wave field synthesis, which uses a finite number of acoustic enclosures to generate a defined sound field. However, the basic basis of the generation principle of the embodiment is specific, because the synthesis does not attempt to accurately simulate the sound field, but is based on a least squares method.

[0030] According to some embodiments, the virtual sound source has a directivity pattern, for example, the directivity is achieved by superimposing multiple monopoles, wherein the directivity can describe the change in the frequency response of the loudspeaker at off-axis angles.

[0031] The circuit of the device (e.g., electronic device) may include a processor, memory (RAM, ROM, etc.), storage and / or storage, interface, etc. The circuit may include or may be connected to input devices (mouse, keyboard, camera, etc.), output devices (display (e.g., liquid crystal, (organic) light emitting diode, etc.)), speakers, etc., (wireless) interfaces, etc., such as commonly known electronic devices (computers, smart phones, etc.). In addition, the circuit may include or may be connected to sensors for sensing still images or video image data (image sensors, camera sensors, video sensors, etc.), for sensing environmental parameters (e.g., radar, humidity, light, temperature, etc.), etc.

[0032] The control signal may be generated by any part of the circuit and may include electromagnetic, electronic, acoustic, optical signals (such as infrared, laser, visible light, etc.). The control signal may be based on or applied based on wired technology, such as optical fiber technology, electronic technology, etc.; it may be based on wireless technology, such as Bluetooth, Wi-Fi, wireless LAN (local area network), infrared, etc.

[0033] The speaker arrangement can be a plurality of at least two individual speakers, wherein these individual speakers can be arbitrarily distributed in a room, several rooms; outside a room; outside a house; in a car; in a headset; in a sound bar; in a television; in a radio; in a sound system, such as a stereo system, a surround system, a high-fidelity stereo system, a 3D audio rendering system, a sound field generation system, etc.

[0034] In some embodiments, the speaker arrangement includes at least one real (physical) speaker and / or at least one virtual speaker, wherein the at least one virtual speaker can be generated together with the at least one real speaker using known methods (such as amplitude panning, delay, etc.). The at least one virtual speaker can also be generated using other real speakers that do not contribute to the generation of at least one virtual sound source, or a mixture of real speakers that contribute to the virtual sound source and real speakers that do not contribute to the virtual sound source.

[0035] The control of the speaker arrangement may have the result that at least one individual speaker of the speaker arrangement emits a sound (or in some cases also a sound signal or a sound wave). The sound may be emitted instantaneously after the speaker receives, for example, a control signal or at a predetermined point in time. The predetermined point in time may in this context be part of a signal or part of an intrinsic programming of at least one individual speaker.

[0036] The generation of at least one virtual sound source may be based on sound field synthesis techniques. The virtual sound source may be a sound field that gives the impression that the sound source is located in a predefined space. For example, the use of virtual sound sources may allow the generation of spatially confined audio signals. The generation of virtual sound sources may be considered as a form of generating virtual speakers throughout the three-dimensional space, including, for example, behind, above or below the listener.

[0037] The contribution of at least one virtual speaker and at least one real speaker may be at least one sound signal emitted by the respective speaker. Furthermore, in the case of multiple virtual speakers and real speakers, only a subset of the multiple virtual speakers and / or real speakers may contribute to the generation of the virtual sound source, wherein the subset may also be zero, i.e., the virtual sound source may be generated only by the real speakers, etc.

[0038] The sound field modulation function may be any function that affects a sound field parameter, such as amplitude, frequency, phase, wave number, gain, phase, etc. It may be a function that is transmitted by an electrical signal or an acoustic signal and causes interference with the acoustic signal and the generated sound field. For example, the sound field modulation function may modulate the physical parameters of the signal to give the listener the impression that the generated sound originates from another direction than it actually is. For example, if the position of the virtual sound source is in front of the listener, applying the sound field modulation function may result in the listener's acoustic perception that the sound originates from a predetermined position, such as above, below or behind the listener, etc. (although there is no (real) speaker there).

[0039] Specifically, in some embodiments, the sound field modulation function may include a head-related transfer function (HRTF), which can simulate the composite filtering effect of the auricle, wherein, for simplicity, in some embodiments, an artificial auricle is used to create the HRTF. The HRTF itself is known. In addition, in some embodiments, the HRTF is obtained / created by measuring the filtering of the auricle of a single user or by averaging multiple HRTFs of multiple users. Moreover, an artificial (fake) head including at least one artificial auricle can be used to obtain the HRTF in one or more measurements. The HRTF can be obtained based on at least one of creating a three-dimensional model of the auricle, computer simulation, trial and error, etc.

[0040] In some implementations, to simplify the HRTF, a finite impulse response (FIR) quotient filter may be applied to the virtual sound source to create the perception of height.

[0041] In this document, virtual height may refer to a vertical direction, where the height may be positive above a certain reference direction and negative below a certain reference direction.

[0042] The FIR filter can be derived by taking the quotient between the altitude HRTF and the corresponding HRTF at the same azimuth in the horizontal plane. Due to symmetry reasons, the HRTF can be derived for only one ear of the listener or a modeled head and applied with minor modifications to the second ear.

[0043] In some implementations, an infinite impulse response (HR) quotient filter may be applied to the virtual sound source in order to create the perception of height.

[0044] In some embodiments, the sound field modulation function comprises at least one height-dependent cue, which is used as a filter for generating an acoustic impression at a predetermined height, for example by means of a HRTF or the like.

[0045] In some embodiments, HRTF may refer to the perceived angle, that is, the listener's perception of the height from which the sound originates, which will be referenced Figure 1 Provide explanation.

[0046] Figure 1 It is an explanatory diagram for explaining the angle of the head related transfer function associated with the listener 3.

[0047] Six arrows -60, -30, 0, 30, 60 and 90 point to the listener 3. The reference numerals -60, -30, 0, 30, 60 and 90 indicate the angles at which the arrows are inclined relative to a central horizontal axis parallel to the viewing axis of the listener 3, i.e., arrow 0 (-60, -30, 30, 60, 90) has an angle of 0 (-60, -30, 30, 60, 90) degrees relative to the viewing axis of the listener 3. Arrows -60, -30, 0, 30, 60 and 90 indicate (or are associated with) HRTFs that cause the listener 3 to perceive the sound as originating from the angle associated with the corresponding arrow, i.e., at 30 (-60, -30, 0, 60, 90) degrees, it can be understood that the sound is perceived as originating from the corresponding angle. However, the present disclosure is not limited to simulating HRTF angles of 0, 30, 60 or 90 degrees, but in principle, every other angle between 0 and 360 degrees can be achieved, for example 180 degrees, simulating sounds behind the listener 3, etc., where negative angles can correspond to positive angles, for example, as is generally known, an angle of -30 degrees can correspond to a positive angle of 330 degrees.

[0048] Thus, in some embodiments, the angle relative to the position of the listener 3 corresponds to a height-related cue, which is used as a filter for generating an acoustic impression at a predetermined height. Moreover, in some embodiments, more than one height-related cue may be used, for example for generating multiple virtual sound sources.

[0049] Furthermore, in some embodiments, the acoustic impression at a predetermined height corresponds to a positioning of a virtual sound source at a predetermined height, ie the listener 3 (or any other listener) may have the acoustic impression that the sound source is located at a predetermined height.

[0050] In some embodiments, the circuit is further configured to move the virtual sound source to improve height perception. As described above, the movement can be generated by the device by (re)positioning the virtual sound source at a position different from the predetermined position. The movement can be perceived or generated as continuous in a predetermined pattern, random pattern, etc., or at discrete positions, as will be further discussed below.

[0051] In some embodiments, the head-related transfer function is specific to an individual loudspeaker in the loudspeaker arrangement.The parameters for determining the head-related transfer function may be given by a manufacturer or the like, or determined by a calibration process.

[0052] In some embodiments, the sound field modulation function depends on the posture of the listener relative to the virtual sound source. For example, if the desired acoustic impression is that the sound is generated from behind the listener, the sound field modulation function may be a different function if the listener is facing the virtual sound source or if the virtual sound source is actually behind the listener's head. In the latter case, in some embodiments, the sound field modulation function may not necessarily be applied to the virtual sound source to generate the acoustic impression that the sound originates from behind the listener, but may also be applied to generate the acoustic impression that the sound originates from in front of the listener. For example, the sound field modulation function may be applied to the virtual sound source in order to generate the acoustic impression that the sound comes from the side, below, above, etc. of the listener, which may also depend on the posture of the listener.

[0053] In some embodiments, the modulation of the sound field depends on the position of the listener relative to the virtual sound source. For example, if the sound is generated one meter away from the listener, the sound field modulation function may be a different function if the distance between the virtual sound source and the listener is two meters, and a different function if the distance between the virtual sound source and the listener is three meters, etc.

[0054] Any other distance between the virtual sound source and the listener can be realized. In addition, any other acoustic impression distance besides 1 meter is also possible, such as 0 meters, 50 centimeters, 2 meters, etc.

[0055] In some embodiments, the circuit is further configured to generate additional virtual sound sources, wherein the position of the additional virtual sound sources depends on the position of at least one individual speaker of the speaker arrangement relative to the position of the listener. For example, in order to provide acoustic uniformity, it may be useful to generate a virtual sound source located between two speakers. In this case, several virtual sound sources may be generated to provide acoustic uniformity so that the listener can move within the speaker arrangement and is not fixed to a certain point.

[0056] In some embodiments, additional virtual sound sources are generated by horizontal / amplitude panning, whereby so-called phantom speakers can be created to fill the sound hole and provide acoustic uniformity, e.g., using vector-based amplitude panning, multi-directional amplitude panning, etc.

[0057] In some embodiments, the circuit is further configured to adjust the signal gain for operation of an individual speaker of the speaker arrangement. The adjustment of the gain can be based on the directional information of the speaker. The gain can be a factor or parameter for modulating the amplitude of the sound field, for modulating the amplitude or intensity of only certain frequencies of the sound emitted by the individual speaker (such as treble frequencies, bass frequencies, midrange frequencies, etc.).

[0058] In some implementations, the gain is adjusted depending on the distance between the listener's position and the virtual sound source.

[0059] For example, in some embodiments, if the listener is farther (closer) from the virtual sound source, the gain may be higher (lower). On the other hand, in some embodiments, if the listener is closer (farther) from the virtual sound, the gain may be higher (lower).

[0060] In the latter case, if two sound sources (e.g., a separate speaker or a virtual sound source or a separate speaker and a virtual sound source), especially in the case where one of two different types of speakers (e.g., a subwoofer and a tweeter) is closer to the listener than the other, the gain of the sound source closer to the listener can be increased in order to create a pleasing sound impression for the listener.

[0061] In some embodiments, the circuit is also configured to control a separate speaker of the speaker arrangement according to the distance between the separate speaker and the virtual sound source. For example, as described herein, a separate speaker can be controlled to generate a virtual sound source. However, the separate speaker can also be controlled so that the sound generated by the separate speaker is modulated according to the distance between the separate speaker and the virtual sound source. For example, the gain of the sound generated by the separate speaker is reduced at a short distance to the virtual sound source and increased at a long distance to the virtual sound source, or vice versa, without limiting the present disclosure in this regard. In addition, as described herein, other parameters such as amplitude, wave number, etc. can be modulated according to the distance from the separate speaker to the virtual sound source. In some embodiments, the circuit is also configured to determine the time point at which the separate speaker generates sound to generate a virtual sound source, wherein the time point can depend on the distance between the separate speaker of the speaker arrangement and the listener's position. As described above, the time point can be determined based on a delay.

[0062] Some embodiments relate to a method comprising: controlling a speaker arrangement comprising at least one virtual speaker and at least one real speaker to generate at least one virtual sound source, wherein the at least one virtual sound source is generated based on contributions of the at least one virtual speaker and the at least one real speaker, and wherein a sound field modulation function configured to generate an acoustic impression for a user is used to modulate the sound field emitted by the virtual sound source so that the acoustic impression is generated at a predetermined position, as described herein.

[0063] In some embodiments, the method is performed on a device as described above or by any other device, apparatus, processor, circuit, etc.

[0064] In some embodiments, the sound field modulation function comprises a head related transfer function, as described herein.

[0065] In some embodiments, the sound field modulation function includes at least one height-dependent cue, as described herein, which is used as a filter for generating an acoustic impression at a predetermined height.

[0066] In some implementations, the acoustic impression at a predetermined height corresponds to the positioning of another virtual sound source at a predetermined height, as described herein.

[0067] In some embodiments, the method further comprises moving additional virtual sound sources, as described herein.

[0068] In some embodiments, the head-related transfer functions are specific to individual speakers of a speaker arrangement, as described herein.

[0069] In some embodiments, the head-related transfer function is obtained by averaging a plurality of head-related transfer functions, wherein each of the plurality of head-related transfer functions corresponds to a dedicated head-related transfer function for an individual listener, as described herein.

[0070] In some embodiments, the head-related transfer functions are obtained by measuring individual head-related transfer functions of individual users, as described herein.

[0071] In some embodiments, the generation of at least one virtual sound source at a horizontal position includes amplitude panning of the sound field and / or delaying the sound field, as described herein.

[0072] In some implementations, the sound field modulation function depends on the posture of the listener relative to the virtual sound source, as described herein.

[0073] In some implementations, modulation of the sound field depends on the position of the listener relative to the virtual sound sources, as described herein.

[0074] In some embodiments, the method comprises generating a further virtual sound source depending on the position of at least one individual speaker of the speaker arrangement relative to the position of the listener, as described herein.

[0075] In some implementations, additional virtual sound sources are generated by horizontal panning, as described herein.

[0076] In some implementations, the method includes adjusting signal gain for operation of an individual speaker of a speaker arrangement, as described herein.

[0077] In some implementations, the gain is adjusted based on the distance between the listener's position and the virtual sound source, as described herein.

[0078] In some embodiments, the method includes controlling individual speakers of the speaker arrangement as a function of a distance between the individual speakers and a virtual sound source, as described herein.

[0079] In some embodiments, the method includes determining a point in time at which an individual speaker of the speaker arrangement generates a sound to generate a virtual sound source based on a distance between the individual speakers of the speaker arrangement and the position of the listener, as described herein.

[0080] In some embodiments, the method described herein is also implemented as a computer program, which, when executed on a computer and / or a processor, causes the computer and / or the processor to perform the method. In some embodiments, a non-transitory computer-readable recording medium is also provided, in which a computer program product is stored, which, when executed by a processor such as the above-mentioned processor, causes the method described herein to be performed.

[0081] In the following, an overview of basic implementations is given, with reference to Figure 2 , Figure 3 and Figure 4 To illustrate the generation of horizontal speakers, refer to Figure 5 Describes the generation of virtual height speakers.

[0082] Figure 2 is an overview of a system 100 comprising a virtual sound source 2 and a speaker arrangement comprising loudspeakers 4, 5, 6, 7, wherein a listener 3 is depicted for illustration purposes.

[0083] Arrows 32, 34, 35, 36, 37, 42, 52, 62, 72 indicate vectors, wherein the reference numerals of the arrows indicate the start and end of the respective vectors, such that an exemplary vector XY, wherein X and Y are selected from a reference numeral library 2, 3, 4, 5, 6, 7, starts at an element with reference numeral X and ends at an element with reference numeral Y. For example, arrow 32 shows a vector starting at user 3 and ending at virtual sound source 2, arrow 35 shows a vector starting at user 3 and ending at speaker 5, arrow 62 shows a vector starting at speaker 6 and ending at virtual sound source 2, and so on.

[0084] The virtual sound source 2 is depicted as an extended object. However, this is for illustration purposes only, and in the present embodiment, it is assumed that the virtual sound source is a point source. Therefore, vectors 32, 42, 52, 62, 72 are considered to end at the same point, although they are depicted as ending at different points.

[0085] Furthermore, for illustrative purposes, a two-dimensional arrangement of the elements 2 to 7 is depicted. However, the embodiment is not limited to a two-dimensional arrangement. In general, a three-dimensional arrangement should be considered.

[0086] Furthermore, for the purpose of illustration, only one virtual sound source is depicted. However, the present disclosure is not limited to one virtual sound source. Other embodiments may involve any number of virtual sound sources greater than 1.

[0087] Therefore, in this embodiment, a plurality of virtual sound sources are assumed.

[0088] The number of speakers is not limited to 4. It can also be 2, 3 or any number greater than 4.

[0089] To illustrate the formula, the variables and how to exemplarily retrieve them are described below.

[0090] First, the distance between the virtual sound source 2 and the respective loudspeaker 4 to 7 is determined by determining the norm of the associated vector X2, where X in this case is the element of {4, 5, 6, 7}, resulting in the distance r:

[0091]

[0092] Among them, index n refers to virtual sound source 2 among multiple virtual sound sources; l refers to speakers 4 to 7 of the speaker arrangement; m refers to the vector of virtual sound source 2; X refers to the vector of speakers 4 to 7 of the speaker arrangement; and indices x, y and z refer to the x, y and z coordinates of the vector in three-dimensional space, respectively.

[0093] For example, 2,5 It can refer to the distance between the virtual sound source 2 and the speaker 5; m 2,x Can refer to the x coordinate of virtual sound source 2; x 5,y It can refer to the y coordinate of speaker 5, etc.

[0094] Second, determine the gain G of each speaker relative to the virtual sound source according to the equation

[0095]

[0096] However, the present disclosure is not limited to determining the gain in this manner, and any other manner of determining the gain is possible. For example, the gain value may be a characteristic of a smaller size or have other dimensions. Depending on, for example, the speaker type of the speaker 4 to 7, another manner of determining the gain may also be used that is different from other speakers 4 to 7 in the same system.

[0097] Third, determine the delay D of each loudspeaker 4 to 7 relative to the virtual sound source 2 according to the equation

[0098]

[0099] Among them, c0 refers to the speed of sound, T s Refers to the sampling period. However, the present disclosure is not limited to determining the delay in this manner, and any other manner of determining the delay is possible. For example, the delay may not be a rounded value, and the delay may be a dimension of time, space, etc. Depending on, for example, the speaker type of the speakers 4 to 7, another manner of determining the delay that is different from other speakers 4 to 7 in the same system may also be used.

[0100] These first three steps may be repeatedly performed for each speaker 4 to 7 and for each sound source 2. However, they may be performed only for one speaker (e.g., speaker 4) and one virtual sound source (e.g., virtual sound source 2), or for a subset of speakers 4 to 7 and a subset of sound sources 2. These first three steps may also be performed in another order, such as exchanging the second and third steps, without limiting the present disclosure in this regard.

[0101] The fourth step may comprise determining, for each virtual sound source 2, a minimum distance r between a virtual sound source 2 (eg, n=2) and a loudspeaker 4 to 7 of the loudspeaker arrangement. n,min and the maximum distance r n,max .

[0102] The fifth step can be to calculate the spread factor using the formula

[0103]

[0104] where σ n is the expansion factor. In some embodiments, the expansion factor may have a positive value.

[0105] The sixth step can be applied to γ n,l The conditions include:

[0106] If γ n，l >0, then G n，l =γ n，l *G n，l , otherwise G n，l =0(5)

[0107] The fifth and sixth steps may be performed repeatedly for each loudspeaker 4 to 7 or for an individual loudspeaker 4 or for a subset of the loudspeakers 4 to 7 .

[0108] For the expansion factor σ n = 1, the result is the expansion factor γ n,l From γ n,min =1 to γ n,max = 0 at the nearest speaker γ n,min and the farthest γ n,max With σ n →∞, the larger expansion coefficient converges to the identity (γ n,l =1), and has σ n = 0, the smaller expansion factor increases the directivity (γ n,l →-∞). In the latter case, only the speaker closest to the source produces the sound (γ n,min =1,γ n,l≠min →-∞,G n,l≠min =0).

[0109] Figure 3 is a graph showing different types of expansion factors γ as a function of the normalized distance (abscissa) n,l (ordinate) of the coordinate system 200, where γ min Corresponding to the distance of 0, γ max corresponds to a distance of 1.

[0110] These functions include an identity function 201, a linear reduction function 202, an indicator function 203 in the case of an expansion factor of 0.5, and a cardioid function 204. These functions are not limited to the functions shown in the context. Any other function for the expansion factor, such as an omnidirectional function, a directional function, a super directional function, a bidirectional function, a figure-of-eight function, a subcardioid function, a cardioid function, a unidirectional function, a supercardioid function, a supercardioid directional function, etc., can also be derived and implemented.

[0111] Figure 4 The generation of a virtual loudspeaker 300 with (real) loudspeakers 4 and 5 is shown. The virtual loudspeaker 300 simulates the loudspeaker to the left of the listener 3.

[0112] The virtual loudspeaker 300 is generated by horizontal panning, i.e. applying gain and delay to the sound emitted by loudspeakers 4 and 5, thereby creating a virtual sound source, as already described with reference to Figure 1 and Figure 2 As described.

[0113] Figure 5 It is shown that a virtual speaker 301 is generated using the speakers 4 and 6. The virtual speaker 301 simulates a virtual speaker above the listener 3. The virtual speaker 301 can also be placed below the listener 3, behind the listener 3, or in any other position. The virtual speaker 301 is generated by applying an associated head-related transfer function to the sound emitted by the speakers 4 and 6, so that the sound perceived at the position of the listener 3 causes a predefined impression (which is associated with the corresponding head-related transfer function). Specifically, a finite impulse response (FIR) quotient filter is applied to the virtual sound source to generate a height perception of the listener 3.

[0114] Figure 6 An embodiment of an installation 350 of loudspeakers 4, 5, 6, 7 is shown. The loudspeakers 4 to 7 are arranged in an arbitrary space shown by a coordinate system 351 .

[0115] Loudspeakers 4 to 7 are panned horizontally; multiple virtual sound sources are generated, also referred to herein as virtual loudspeakers 8, 9, 10, 11, 12, 13 and 14, by calculating gains, delays and applying expansion factors, as described herein. The virtual loudspeakers generated by horizontal panning are also referred to as phantom loudspeakers. The main function of phantom loudspeakers 8 to 14 is to provide spatial uniformity of the created sound field, especially where "sound holes" occur, such as where phantom loudspeakers 8, 11 and 13 are placed. By providing spatial uniformity, such as Figure 1 , 2 , 3 or 4 listeners 3 are able to perceive the same acoustic impression at different locations in the space rather than just at one sweet spot.

[0116] Furthermore, the speakers 4 to 7 generate a plurality of virtual speakers at positions above the horizontal plane on which the speakers 4 to 7 are placed by applying the relevant head-related transfer functions (HRTFs) to the phantom speakers 9, 10, 12, and 14. The plurality of virtual speakers generated in this manner are also referred to as virtual height speakers 15, 16, 17, 18. Assuming that the listener is located at the center of the four speakers 4 to 7, the virtual height speakers simulate the acoustic impression of sound emitted from above at an angle of 60 degrees.

[0117] Speakers 4 to 7 generate another virtual speaker 19 (also called a top virtual height speaker) by applying HRTF to a virtual sound source at the center of the four speakers 4 to 7, simulating the acoustic impression of sound emitted at an angle of 90 degrees from above a listener assumed to be at the center of the four speakers 4 to 7, that is, above the listener's head.

[0118] Figure 6 The principle of the embodiment can also be applied to the case where the number of loudspeakers is lower or higher than 4, as well as different numbers of phantom loudspeakers, virtual height loudspeakers and top virtual height loudspeakers. For example, in another embodiment, there can be seven (five) loudspeakers, which are common in systems on the market. Using the principle of generating phantom loudspeakers, virtual height loudspeakers (also at angles different from 60 degrees) and top virtual height loudspeakers, it is possible to fill the space with sound as desired.

[0119] Figure 7 An embodiment of the system 500 of the present disclosure is shown with a virtual loudspeaker 501 and two virtual loudspeakers 502 and 503 generated by applying an HRTF to the virtual loudspeaker 501 , wherein a listener 3 is depicted for illustration purposes.

[0120] The virtual speaker 502 simulates a sound originating from the right reference direction of the listener 3 , and the virtual speaker 503 simulates a sound originating from the left reference direction of the listener 3 .

[0121] Figure 8 An embodiment of a system 510 of the present disclosure is shown. The system 510 includes a virtual sound source 511 generated by applying an HRTF quotient filter to speakers 512 and 513, two speakers 512 and 513, and two virtual speakers 514 and 515, wherein a listener 3 is depicted for illustrative purposes. The virtual sound source 511 is moving (indicated by an arrow), thereby affecting the listener's perception of the direction of the sound originating from the virtual sound source. The movement of the virtual sound source affects the sound generated by the two virtual speakers 514 and 515.

[0122] When the virtual sound source 511 moves, the HRTF quotient filter is excited with different gains and delays generated by the current position of the virtual sound source 511, thereby providing different binaural height cues to the listener 3. In some embodiments, the amplitude and speed changes of the movement can be variable and depend on the number of virtual speakers. In some embodiments, the amplitude and speed changes of the movement can be random, while in other embodiments spatial continuity can be achieved. In other embodiments, the virtual sound source 511 can move around a central position in the shape of a disk or circle, for example to simulate a head rotation. In other embodiments, the position of the virtual sound source 511 can be selected from a table of predefined positions.

[0123] For example, in this embodiment, the movement of the head of the listener 3 may be simulated by moving the monopole source 511. The perception of the movement of the listener 3 by an object, such as the flight of a bird, etc., may also be simulated.

[0124] refer to Fig. 9 A method 519 for controlling a loudspeaker arrangement is described. The method is based on a monopole synthesis algorithm 520, as will be described with reference to Fig.11 As described.

[0125] At 521, 522 and 523, the positions of the top virtual height speakers, virtual height speakers and phantom speakers are illustrated, as shown in reference Figure 6 As described.

[0126] In addition, the HRTF quotient of the determined position of the top virtual height speaker is determined at 521 , and the HRTF quotient of the determined position of the virtual height speaker is determined at 522 .

[0127] At 524 , a delay is determined based on the determined positions of 521 to 523 .

[0128] At 525 , a gain is determined based on the determined positions of 521 and 523 .

[0129] At 526 , the parameters determined at 520 to 525 are applied to the individual speakers 527 , 528 , 529 , 530 .

[0130] In this embodiment, the relevant delays and gains for generating the top virtual height speakers are applied to all individual speakers 527 to 530, the relevant delays and gains for generating the virtual height speakers are applied to speaker 528, and the relevant delays and gains for generating the phantom speakers are applied to speakers 529 and 530. This means that the top virtual height speakers are generated by all speakers 527 to 530, the virtual height speakers are generated by speakers 528 and 529, and the phantom speakers are generated by speakers 529 and 530.

[0131] In other embodiments, the top virtual height speakers, virtual height speakers, and phantom speakers are generated by any suitable combination of speakers 527 to 530.

[0132] Other embodiments provide for generating more than four speakers or less than four speakers simultaneously generating other combinations of virtual speakers.

[0133] In the following, reference Fig.10 The illustrated flow chart describes a method 540 for generating a set of virtual speakers.

[0134] At 541 , positions of speakers of a speaker arrangement are determined.

[0135] At 542 , the position of the virtual speaker is determined based on the position determined at 541 .

[0136] At 543, phantom speakers are generated by horizontal panning (ie, calculating associated delays, gains, and expansion factors), as described herein.

[0137] At 544, a virtual height speaker is generated by applying the HRTF to the phantom speaker and / or speakers.

[0138] At 545, a top virtual height speaker is generated as a virtual sound source by applying the HRTF to the phantom speaker and / or the speaker.

[0139] Reference below Fig.11 Embodiments of the device, Fig.11 A block diagram of an apparatus implemented as an audio system 400 (or alternatively as an electronic device 401) is shown. The apparatus may be included in a car, a smart phone, a sound system, or the like.

[0140] The audio system 400 comprises an electronic device 401 connected to a microphone arrangement 410, a speaker arrangement 411, a user interface 412 and a sensor 413. In the present embodiment, the electronic device 401 is a 3D sound rendering system.

[0141] The electronic device 401 has a CPU 402 as a processor, a data storage 403, and a data memory 404 (here, RAM).

[0142] The data memory 404 is arranged to temporarily store or cache data and / or computer instructions for processing by the processor 402 .

[0143] The data store 403 is used to store recorded sensor data obtained from, for example, the microphone arrangement 410 .

[0144] The electronic device 401 is configured to execute software for 3D audio rendering operations that place sound sources virtually anywhere inside a room, including behind, above, or below the listener, such as Figure 1 of listeners 3.

[0145] The electronic device 401 has a WLAN interface 405, a Bluetooth interface 406, and an Ethernet interface 407. These interfaces 405, 406, 407 are used as I / O interfaces for data communication with external devices.

[0146] For example, a smartphone may be connected to the 3D sound rendering system through the Bluetooth interface 406 and / or the WLAN interface 405. Additional speakers, microphones and video cameras with Ethernet, WLAN or Bluetooth connections may be coupled to the electronic device 401 via these wireless / wired interfaces 405, 406 and 407.

[0147] For example, microphone arrangement 410 may consist of one or more microphones distributed around a listener.

[0148] The user interface 412 is connected to the processor 402. The user interface 412 acts as a human-machine interface and allows a dialog between an administrator and the audio system 400.

[0149] Sensor 413 is connected to processor 402. Sensor 413 includes a temperature sensor and a video camera. In other embodiments, the sensor includes a GPS sensor or other positioning sensor, and / or an acceleration sensor, etc. Sensor 413 is configured to obtain the presence and position of one or more listeners and the position and orientation of the listener's head. Moreover, sensor 413 is configured to obtain the position and orientation of speaker arrangement 411. These video cameras can be distributed over a predefined space, or a separate camera can be used to obtain images.

[0150] In some embodiments, the sensors further include at least one external microphone placed in at least one ear of the user and / or a camera configured to obtain at least one picture of at least one ear of the user to at least determine parameters for determining the HRTF.

[0151] The audio system 400 receives audio from a speaker arrangement 411 and at least one virtual sound source (eg, a virtual sound source 2, Figure 1 ) in order to monitor the generated virtual sound source (e.g., virtual sound source 2, Figure 1 ) and, if necessary, adjusting the speaker arrangement 411 to influence the generated virtual sound source.

[0152] In some embodiments, 3D audio rendering based on a digital monopole synthesis algorithm is implemented, as described below with reference to Fig.12 described.

[0153] The theoretical background of the present technology used in some embodiments is described in more detail in patent application US 2016 / 0037282 A1, which is incorporated herein by reference.

[0154] The technique implemented in the embodiment of US 2016 / 0037282 A1 is conceptually similar to wave field synthesis, which uses a finite number of acoustic enclosures to generate a defined sound field. However, the basic basis of the generation principle of the embodiment is specific, because the synthesis does not attempt to accurately simulate the sound field, but is based on a least squares method.

[0155] The target sound field is modeled as at least one target monopole placed at a defined target position. In one embodiment, the target sound field is modeled as a single target monopole. In other embodiments, the target sound field is modeled as multiple target monopoles placed at corresponding defined target positions. For example, each target monopole can represent a noise cancellation source included in a set of multiple noise cancellation sources located at a specific position in space. The position of the target monopole can be moving. For example, the target monopole can adapt to the movement of the noise source to be attenuated. If multiple target monopoles are used to represent the target sound field, the method of synthesizing the sound of the target monopole based on a set of defined synthetic monopoles as described below can be applied independently to each target monopole, and the contribution of the synthetic monopole obtained for each target monopole can be added to reconstruct the target sound field.

[0156] The source signal x(n) is fed to the -np The delay unit and the amplifier unit are marked as a P , where p=1, ..., N is the index of each synthetic monopole used to synthesize the target monopole signal. The delay and amplification unit according to this embodiment can apply equation (117) of reference US 2016 / 0037282 A1 to calculate the result signal y for synthesizing the target monopole signal p (n) = s p (n). The resulting signal S P (n) is power amplified and fed to the speaker S p .

[0157] In this embodiment, the synthesis is therefore performed in the form of delayed and amplified components of the source signal x.

[0158] According to this embodiment, the delay n of the synthetic monopolar index p P Corresponding to the target monopole r o With the generator r PThe Euclidean distance between p0 =|r p -r o |Sound travel time.

[0159] In addition, according to this embodiment, the magnification factor and the distance r=R P0 Inversely proportional.

[0160] In an alternative embodiment of the present system, a modified amplification factor according to equation (118) of reference US 2016 / 0037282 A1 may be used.

[0161] Fig.13 A method 600 according to an embodiment of the present disclosure is shown. Generally, the method 600 is based on the method 540, as described herein. Therefore, repeated descriptions of reference numerals 541 to 545 are omitted.

[0162] After 545, the method 600 further comprises moving the virtual sound source in 546. In this embodiment, the movement is an oscillating movement of the virtual sound source, making it more natural for the listener and thus improving the (height) perception of the sound emitted by the top virtual height speakers.

[0163] Fig.14 A method 700 for controlling a speaker arrangement according to the present disclosure is shown.

[0164] In 701, the method includes controlling a speaker arrangement including virtual speakers and real speakers to generate at least one virtual sound source, wherein the at least one virtual sound source is generated based on contributions of the virtual speakers and the real speakers, and wherein a sound field modulation function configured to generate an acoustic impression for a user is used to modulate the sound field emitted by the virtual sound source so that an acoustic impression is generated at a predetermined position, as described herein.

[0165] In this embodiment, the sound field modulation function comprises a head-related transfer function as described herein. The head-related transfer function also comprises a height-related cue, which is used as a filter for generating the acoustic impression at a predetermined height. This corresponds to the positioning of a virtual sound source at a predetermined height, as described herein. Moreover, the head-related transfer function is specific to an individual speaker of the speaker arrangement.

[0166] As already discussed herein, in some embodiments the head-related transfer function is obtained by measuring an individual head-related transfer function of the user or the fake head. However, in this embodiment the head-related transfer function is obtained by averaging a plurality of (known) head-related transfer functions, wherein each of the plurality of head-related transfer functions corresponds to a dedicated head-related transfer function of an individual listener.

[0167] Furthermore, the generating of at least one virtual sound source at the horizontal position comprises (at least one of) an amplitude shift of the sound field and a delay of the sound field.

[0168] At 702, the method further includes moving the virtual sound source, as described herein.

[0169] At 703 , the method further comprises adjusting a signal gain for operation of an individual speaker of the speaker arrangement, as described herein.

[0170] It should be appreciated that these embodiments describe methods with an exemplary order of method steps. However, the specific order of method steps is given for illustrative purposes only and should not be construed as binding. For example, the order of steps may be interchanged. Fig. 9 In the embodiment of the present invention, the order of 524 and 525 can be changed. Fig.10 Other variations in the order of the method steps will be apparent to those skilled in the art.

[0171] Please note that the division of the electronic device into units 403 to 407 is for illustration purposes only, and the present disclosure is not limited to any particular division of functions in a particular unit.For example, control 401 may be implemented by a respective programmed processor, field programmable gate array (FPGA), or the like.

[0172] If not stated otherwise, all units and entities described in this specification and claimed in the appended claims may be implemented as integrated circuit logic, for example on a chip, and if not stated otherwise, the functions provided by such units and entities may be implemented by software.

[0173] To the extent that the above disclosed embodiments are implemented at least in part using software controlled data processing apparatus, it should be understood that providing such software controlled computer programs and transmission, storage or other media providing such computer programs are contemplated as aspects of the present disclosure.

[0174] Note that the present technology may also be configured as described below.

[0175] (1) An apparatus comprising a circuit configured to control a speaker arrangement comprising at least one virtual speaker and at least one real speaker to generate at least one virtual sound source, wherein the at least one virtual sound source is generated based on contributions of the at least one virtual speaker and the at least one real speaker, and wherein a sound field modulation function configured to generate an acoustic impression for a user is used to modulate the sound field emitted by the virtual sound source so that the acoustic impression is generated at a predetermined position.

[0176] (2) The apparatus according to (1), wherein the sound field modulation function includes a head-related transfer function.

[0177] (3) The apparatus according to (1) or (2), wherein the sound field modulation function includes at least one height-dependent cue used as a filter for generating an acoustic impression at a predetermined height.

[0178] (4) The apparatus according to any one of (1) to (3), wherein the acoustic impression at the predetermined height corresponds to the positioning of the virtual sound source at the predetermined height.

[0179] (5) The apparatus of any one of (1) to (4), wherein the circuit is further configured to move the virtual sound source to improve height perception.

[0180] (6) The apparatus according to any one of (1) to (5), wherein the head-related transfer function is specific to an individual speaker of the speaker arrangement.

[0181] (7) An apparatus according to any one of (1) to (6), wherein the head-related transfer function is obtained by averaging a plurality of head-related transfer functions, wherein each of the plurality of head-related transfer functions corresponds to an individual head-related transfer function of an individual listener.

[0182] (8) The apparatus according to any one of (1) to (7), wherein the head-related transfer function is obtained by measuring individual head-related transfer functions of individual users.

[0183] (9) The apparatus according to any one of (1) to (8), wherein the generation of at least one virtual sound source at a horizontal position includes at least one of amplitude shifting of the sound field and delaying the sound field.

[0184] (10) The apparatus of any one of (1) to (9), wherein the circuit is further configured to adjust a signal gain for operation of an individual speaker of the speaker arrangement.

[0185] (11) A method comprising:

[0186] Controlling a speaker arrangement comprising at least one virtual speaker and at least one real speaker to generate at least one virtual sound source, wherein the at least one virtual sound source is generated based on contributions of the at least one virtual speaker and the at least one real speaker, and wherein a sound field modulation function configured to generate an acoustic impression for a user is used to modulate the sound field emitted by the virtual sound source so that the acoustic impression is generated at a predetermined position.

[0187] (12) The method according to (11), wherein the sound field modulation function includes a head-related transfer function.

[0188] (13) A method according to any one of (11) or (12), wherein the sound field modulation function includes at least one height-dependent cue that is used as a filter for generating an acoustic impression at a predetermined height.

[0189] (14) The method according to any one of (11) to (13), wherein the acoustic impression at the predetermined height corresponds to the positioning of the virtual sound source at the predetermined height.

[0190] (15) The method according to any one of (11) to (14), further comprising moving the virtual sound source to improve height perception.

[0191] (16) The method according to any one of (11) to (15), wherein the head-related transfer function is specific to an individual speaker of the speaker arrangement.

[0192] (17) A method according to any one of (11) to (16), wherein the head-related transfer function is obtained by averaging a plurality of head-related transfer functions, wherein each of the plurality of head-related transfer functions corresponds to a dedicated head-related transfer function for an individual listener.

[0193] (18) The method according to any one of (11) to (17), wherein the head-related transfer function is obtained by measuring individual head-related transfer functions of individual users.

[0194] (19) The method according to any one of (11) to (18), wherein the generation of at least one virtual sound source at a horizontal position includes at least one of amplitude shifting of the sound field and delaying the sound field.

[0195] (20) The method according to any one of (11) to (19), further comprising:

[0196] The signal gain is adjusted for operation of an individual speaker of the speaker arrangement.

[0197] (21) A computer program comprising a program code, which, when executed on a computer, causes the computer to execute the method according to any one of (11) to (20).

[0198] (22) A non-transitory computer-readable recording medium storing a computer program product which, when executed by a processor, causes execution of the method according to any one of (11) to (20).

Claims

1. An apparatus for generating a height perception of sound for a user, comprising a circuit configured to control a speaker arrangement comprising at least one virtual speaker and at least one real speaker to generate at least one virtual sound source, wherein: The at least one virtual sound source is generated based on contributions of the at least one virtual loudspeaker and the at least one real loudspeaker, and wherein a sound field modulation function configured to generate an acoustic impression for a user is used to modulate the sound field emitted by the virtual sound source so that the acoustic impression is generated at a predetermined position, The circuit is further configured to move the virtual sound source to improve height perception, wherein the amplitude and speed of the movement are variable and depend on the number of the virtual speakers.

2. The device according to claim 1, wherein: The sound field modulation function includes a head related transfer function.

3. The device according to claim 1, wherein: The sound field modulation function comprises at least one height-dependent cue, which is used as a filter for generating the acoustic impression at a predetermined height.

4. The device according to claim 3, wherein: The acoustic impression at a predetermined height corresponds to a positioning of the virtual sound source at the predetermined height.

5. The device according to claim 2, wherein: The head related transfer function is specific to an individual loudspeaker in the loudspeaker arrangement.

6. The device according to claim 2, wherein: The head related transfer function is obtained by averaging a plurality of head related transfer functions, wherein each of the plurality of head related transfer functions corresponds to an individual head related transfer function of an individual listener.

7. The device according to claim 2, wherein: The head related transfer functions are obtained by measuring individual head related transfer functions of individual users.

8. The device according to claim 1, wherein: The generating of the at least one virtual sound source at a horizontal position comprises at least one of an amplitude shift of the sound field and a delay of the sound field.

9. The device according to claim 1, wherein: The circuit is further configured to adjust signal gain for operation of individual speakers of the speaker arrangement.

10. A method of generating a heightened perception of sound for a user, comprising: controlling a speaker arrangement comprising at least one virtual speaker and at least one real speaker to generate at least one virtual sound source, wherein the at least one virtual sound source is generated based on contributions of the at least one virtual speaker and the at least one real speaker, and wherein a sound field modulation function configured to generate an acoustic impression for a user is used to modulate the sound field emitted by the virtual sound source such that the acoustic impression is generated at a predetermined position, and The virtual sound source is moved to improve height perception, wherein the magnitude and speed of the movement are variable and depend on the number of the virtual speakers.

11. The method according to claim 10, wherein: The sound field modulation function includes a head related transfer function.

12. The method according to claim 10, wherein: The sound field modulation function comprises at least one height-dependent cue, which is used as a filter for generating the acoustic impression at a predetermined height.

13. The method according to claim 12, wherein: The acoustic impression at a predetermined height corresponds to a positioning of the virtual sound source at the predetermined height.

14. The method according to claim 11, wherein: The head related transfer function is specific to an individual loudspeaker in the loudspeaker arrangement.

15. The method according to claim 11, wherein: The head related transfer function is obtained by averaging a plurality of head related transfer functions, wherein each of the plurality of head related transfer functions corresponds to a dedicated head related transfer function of an individual listener.

16. The method according to claim 11, wherein: The head related transfer functions are obtained by measuring individual head related transfer functions of individual users.

17. The method according to claim 10, wherein: The generating of the at least one virtual sound source at a horizontal position comprises at least one of an amplitude shift of the sound field and a delay of the sound field.

18. The method of claim 10, further comprising adjusting signal gain for operation of individual speakers of the speaker arrangement.

Images