Device and method

Abstract

The present disclosure relates to an apparatus comprising a circuit, which is configured to: determine a speaker-dependent expansion factor of at least one individual speaker of a speaker arrangement, wherein the speaker-dependent expansion factor depends on specifications of the at least one individual speaker; and control outputs of each speaker of the speaker arrangement based on the speaker-dependent expansion factor of the at least one individual speaker to generate at least one virtual sound source.

Description

Technical Field

[0001] The present disclosure generally relates to apparatus and methods for operating spatial audio techniques. Background Art

[0002] Current systems for generating spatial sound fields, such as wave field synthesis, usually require a relatively large amount of acoustic equipment, which is mainly available in the form of a set of loudspeakers. The equations used to derive such systems are funded by the desire to reproduce the sound field as accurately as possible.

[0003] For example, known systems are so-called 5.1 or 7.1 systems, which consist of 5 or 7 loudspeakers and one or two additional subwoofers designed to reproduce the low frequency range of the sound with higher energy. However, it is known that such systems may be limited in their ability to generate a desired sound field with a well-balanced timbre, so that the listener must be placed in a relatively central area.

[0004] For example, in an automotive environment where speakers are placed at different heights, such as a woofer placed at the bottom of the car and a tweeter placed on the dashboard, the wave field generated (e.g., a monopole sound source) may be unbalanced depending on its position relative to the speakers. For example, if the monopole sound source is placed at a higher position, high frequencies may be dominant, while if the sound source is placed at a lower position, low frequencies may be dominant, and the balance of frequencies may be achieved only at the predetermined position of the monopole sound source.

[0005] Furthermore, other systems are known that attempt to physically reproduce the sound field in the same way as a real sound source would exist, such as the so-called wave field synthesis already introduced above. Here, the reproduction of the sound field is based on the Huygens principle and the sound field is approximated with a number of loudspeakers. Such methods may involve a relatively high computational complexity and, therefore, may provide approximations, such as monopole synthesis, which, however, may lead to inaccuracies in the generated wave fields. Summary of the invention

[0006] While techniques exist for monopole synthesis, it is generally desirable to provide an apparatus and method for generating a perceptually well-balanced timbre of a desired sound field.

[0007] According to a first aspect, the present disclosure provides an apparatus comprising a circuit, wherein the circuit is configured to determine a speaker-dependent expansion factor of at least one individual speaker of a speaker arrangement, wherein the speaker-dependent expansion factor depends on specifications of the at least one individual speaker; and to control outputs of the speakers of the speaker arrangement based on the speaker-dependent expansion factor of the at least one individual speaker to generate at least one virtual sound source.

[0008] According to a second aspect, the present disclosure provides a method, comprising: determining a speaker-related expansion factor of at least one individual speaker of a speaker device, wherein the speaker-related expansion factor depends on specifications of the at least one individual speaker; and controlling outputs of each speaker of the speaker device based on the speaker-related expansion factor of the at least one individual speaker to generate at least one virtual sound source.

[0009] Further aspects are set out in the dependent claims, the following description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Embodiments are described by way of example with reference to the accompanying drawings, in which:

[0011] Figure 1 A speaker system for generating a virtual sound source according to an embodiment of the present disclosure is depicted;

[0012] Figure 2 is a diagram of a coordinate system including different expansion factors according to an embodiment of the present disclosure;

[0013] Figure 3 is a polar coordinate system diagram including different expansion factors according to an embodiment of the present disclosure;

[0014] Figure 4 The situation addressed by the present disclosure is shown;

[0015] Figure 5 An electronic device for controlling an audio system according to an embodiment of the present disclosure is depicted;

[0016] Figure 6 A method for generating a virtual sound source according to an embodiment of the present disclosure is described; and

[0017] Figure 7 Embodiments of 3D audio rendering based on a digital monopole synthesis algorithm are provided. DETAILED DESCRIPTION

[0018] In giving reference Figure 5 Before describing the embodiments in detail, some general remarks are made.

[0019] As mentioned at the outset, known techniques may be limited in their ability to generate a desired sound field with a perceptually well-balanced timbre, and therefore, some embodiments are directed to improving a listener's perception of timbre in monopole synthesis applications.

[0020] Therefore, some embodiments relate to an apparatus comprising a circuit configured to generate a signal to determine a speaker-related expansion factor of at least one individual speaker of a speaker arrangement, wherein the speaker-related expansion factor depends on specifications of the at least one individual speaker; and to control outputs of each speaker of the speaker arrangement based on the speaker-related expansion factor of the at least one individual speaker to generate at least one virtual sound source.

[0021] The circuit configured to control the speaker device (or a device suitable for controlling the speaker device) may include any one of an electronic device, a processor, a computer, an electronic amplifier (such as a single-sided amplifier, a bilateral amplifier, an inverting amplifier, a non-inverting amplifier, a servo amplifier, a linear amplifier, a non-linear amplifier, a broadband amplifier, a radio frequency amplifier, an audio amplifier, a resistor-capacitor coupled amplifier (RC), an inductor-capacitor coupled amplifier (LC), a transformer-coupled amplifier, a direct-coupled amplifier, etc.). The device may also be or include a 3D or spatial audio rendering system that performs 3D or spatial audio rendering operations, such as an ambient stereo and sound field synthesis system, a surround sound system, etc. In addition, the device may be independent, or it may be integrated in another device / equipment.

[0022] For example, 3D audio rendering operations are based on wave field synthesis, where wave field synthesis techniques can be used to generate a sound field that gives the impression that audio point sources are located within a predetermined space.

[0023] This impression can be achieved by using a monopole synthesis method that drives an array of loudspeakers to generate the impression of a virtual sound source.

[0024] According to some embodiments, the 3D audio rendering operation is based on monopole synthesis.

[0025] The theoretical background of this technology used in some embodiments is described in more detail in patent application US 2016 / 0037282A1, which is incorporated herein by reference.

[0026] The technology implemented in the embodiments of US 2016 / 0037282A1 is conceptually similar to wave field synthesis, which uses a limited number of acoustic enclosures to generate a defined sound field. However, the basic basis of the generation principle of the embodiments is specific, because the synthesis does not attempt to accurately model the sound field, but is based on a least squares method.

[0027] According to an embodiment, the virtual sound source is associated with specifications of (at least one) individual loudspeaker, such as directivity pattern, frequency range, etc. Directivity may be achieved by superimposing multiple monopoles, and directivity may describe the variation of the frequency response of a loudspeaker, wherein the frequency and / or the frequency response may depend on the angle of the loudspeaker.

[0028] The circuitry of the device may include: a processor (or multiple processors), memory (RAM, ROM, etc.), storage and / or memory, interfaces, etc. The circuitry may include or may be connected to input means (mouse, keyboard, camera, etc.), output means (display (e.g. liquid crystal, (organic) light emitting diodes, etc.)), speakers, etc., (wireless) interfaces, etc., as is generally known for electronic devices (computers, smartphones, etc.). Furthermore, the circuitry may include or may be connected to sensors for sensing still or video image data (image sensors, camera sensors, video sensors, etc.), for sensing environmental parameters (e.g. radar, humidity, light, temperature, etc.), etc.

[0029] The determination of the expansion factor associated with the speaker may include determining characteristics of at least one speaker of the speaker arrangement, similar to determining the type of speaker, i.e., subwoofer, woofer, mid-woofer, tweeter, etc. The determination may include determining speaker-specific coefficients / specifications, such as directivity modes, type of membrane, resonant frequencies, etc., as mentioned below. The determination may include determining the position of the speaker relative to other speakers, relative to a virtual sound source, relative to a listener, etc. The determination may include angular information about the speaker, such as the orientation of an individual speaker, the emission angle of an individual speaker, etc.

[0030] The speaker-dependent expansion factor may be used to modulate a sound signal or wave emitted by a speaker that generates or contributes to the generation of a virtual sound source. Thus, the parameters of the signal may be changed depending on the location of the sound signal or wave propagating through a room or space. For example, the gain of the sound signal or wave may be increased / decreased depending on the distance to the virtual sound source, or the gain may be adjusted based on obstacles or other objects that may affect the propagation characteristics of the sound signal or wave. By modulating the sound signal or wave, a uniform distribution of the sound field may be achieved.

[0031] The speaker-dependent expansion factor may include determined properties of individual speakers of the speaker arrangement, in particular, the relative position of the individual speakers relative to the user, the gain of the individual speakers, wherein the gain may also include the directionality information of the speakers. The speaker-dependent expansion factor may include a delay of the individual speakers, wherein the delay may be a time point relative to another time point when the individual speakers emit a sound (e.g., a time point when a signal is received, or another speaker emits a sound). The delay may be based on position information of the individual speakers relative to each other, relative to a virtual sound source, relative to a listener, etc.

[0032] The speaker arrangement may be a plurality of at least two individual speakers, wherein the individual speakers may be distributed arbitrarily (e.g. also randomly or in a predetermined manner) in a room, in several rooms, outside a room, outside a house, inside a vehicle, in headphones, in a sound bar, in a television set, in a radio, in an audio system (such as a stereo system, a surround system, an ambient sound system, a 3D audio rendering system, a sound field generation system, etc.).

[0033] The specification of at least one individual loudspeaker of the loudspeaker arrangement may be a frequency range and / or a directivity pattern, such as an angular dependence of the intensity of the emitted sound waves. The angular dependence may be a dependence of a spherical angle, a solid angle, a spatial angle, etc. The directivity pattern may include: an omnidirectional pattern, a directional pattern, a super-directional pattern, a bidirectional pattern, a figure-of-eight pattern, a subcardioid pattern, a cardioid pattern, a unidirectional pattern, a supercardioid pattern, a hypercardioid directivity pattern, etc.

[0034] Generally speaking, the specifications of the at least one individual speaker of the speaker arrangement may be based on simulations, manufacturer's implementation choices, user input, taken from a table, manual, or the like.

[0035] Controlling the output (i.e., the sound emitted) of each speaker of the speaker device may include generating a control signal, which can be output for transmission to the speaker device, and the control may be based on wired technology, such as fiber optic technology, electronic technology, etc. It may be based on wireless technology, such as Bluetooth, Wi-Fi, wireless LAN (local area network), infrared, etc. In addition, the control may be performed by a speaker (or multiple speakers), wherein the speaker (each) may include a device as described herein (or a subset of multiple speakers may include the device). The signal may cause at least one individual speaker of the speaker device to emit a sound. The sound may be emitted immediately after the speaker receives the signal, at a predetermined time point, or after a certain delay. In this context, the predetermined time point may be part of the signal or part of the inherent programming of at least one individual speaker. Moreover, an indication of the time point may be included in the signal.

[0036] The generation of at least one virtual sound source may be based on sound field synthesis techniques. The virtual sound source may be, for example, a sound field that gives the impression that the sound source is located in a predetermined space and / or at a predetermined position. For example, the use of virtual sound sources may allow the generation of spatially confined audio signals. In particular, the generation of virtual sound sources may be considered as a form of generating virtual speakers in a three-dimensional space, including behind, above or below the listener.

[0037] For example, in order to generate the effect to the listener that the sound is located behind (right / left) the listener, the virtual sound source may be placed behind (right / left) the listener or at any other suitable position.

[0038] In some embodiments, the speaker-dependent expansion factor depends on the distance of the virtual sound source to at least one individual speaker of the speaker arrangement, as described above.Thereby, the expansion factor can be adjusted according to the distance of the virtual sound source.

[0039] For example, if the distance of the virtual sound source to the at least one individual speaker generating the virtual sound source is too high / too low, it may be desirable to have a high / low directivity in order not to lose / have too much sound signals or waves contributing to the virtual sound source.

[0040] In some embodiments, the circuit is further configured to determine the point in time at which at least one individual speaker generates sound to generate a virtual sound source based on the distance (from the virtual sound source to at least one individual speaker of the speaker device). This may refer to a delay, as described above. Therefore, the emitted sound waves of the individual speakers contributing to the virtual sound source are generated so that they arrive at the desired position of the virtual sound source at the same point in time.

[0041] For example, if a virtual sound source is generated by two or more speakers, it is desirable that the signals emitted by the two or more speakers overlap at a predetermined position where the virtual sound source is placed. Therefore, by introducing, for example, a delay in the emission of a sound signal or wave, the sound signals of the speakers can be synchronized, and interference (such as beat frequency, comb filter effects, etc.) can be avoided or suppressed.

[0042] In some embodiments, the loudspeaker-dependent expansion factors are determined according to a linear or non-linear function. In some embodiments, the non-linear function may depend one-dimensionally on the distance, or multi-dimensionally on vectors determined for individual loudspeakers.

[0043] The vector may comprise coordinates, which indicate the positions of the individual loudspeakers. The non-linear function may also depend on time, a multi-dimensional vector comprising at least one position information and time, etc. The non-linear function may allow a simple and / or fast calculation of the expansion factor.

[0044] Using nonlinear functions can generate better sound fields than using linear functions. For example, in the case where the sound field is generated in a room where, for example, a non-uniform distribution of furniture may shield or prevent good sound propagation, a nonlinear function can be included in the speaker-related expansion factor to address such issues.

[0045] The nonlinear function may be a cardioid function, a command function, a sigmoid function, etc. In some embodiments, the nonlinear function may be related to a directivity pattern (e.g., the directivity pattern described above). Thus, the nonlinear function may be selected based on the (frequency emitting) type of the speaker, such as a tweeter, a woofer, a midrange speaker, a subwoofer, etc. In some embodiments, the nonlinear function may be converted into a directivity pattern by a coordinate transformation in order to simulate and visualize the resulting sound of a single speaker.

[0046] In some embodiments, the virtual sound source is generated by a contribution from each loudspeaker which is an amplified and delayed version of the input audio signal.

[0047] The contribution can be a sound wave, a sound pulse, etc. emitted by a separate speaker.

[0048] The input audio signal may be a signal transmitted to a separate speaker, or in some embodiments a desired audio signal at a predetermined position, or the like.

[0049] In some embodiments, the circuit is further configured to adjust the gain of the individual speakers of the speaker arrangement. Depending on the adjusted gain, the individual speakers can contribute more or less to the generation of the virtual sound source, so the adjustment of the gain can, for example, lead to an improved sound impression of the listener.

[0050] The gain may be of the nature described above. The gain may also be a factor that modulates the amplitude of the sound field, modulating the amplitude or intensity of only certain frequencies of the sound emitted by an individual speaker, such as treble frequencies, bass frequencies, mid-range frequencies, etc.

[0051] In some embodiments, the gain is modified by the spreading factor, ie may depend on the spreading factor or may be (dynamically) adaptive when the spreading factor changes.

[0052] In some embodiments, the gain is adjusted depending on the distance between the listener and the virtual sound source.

[0053] For example, in some embodiments, if the listener is farther (closer) from the virtual sound source, the gain may be higher (lower). On the other hand, in some embodiments, if the listener is closer (farther) from the virtual sound, the gain may be higher (lower).

[0054] In the latter case, if two sound sources (e.g., a separate speaker or a virtual sound source or a separate speaker and a virtual sound source), particularly for two different types of speakers (e.g., a subwoofer and a tweeter), one of which is closer to the listener than the other, the gain of the sound source closer to the listener can be increased in order to generate a pleasant sound impression for the listener.

[0055] In some embodiments, determining includes determining a position of at least one individual speaker of the speaker arrangement relative to a listener position, as described above. In some embodiments, the listener position may be a relative distance from the at least one individual speaker or a three-dimensional position based on a vector. The position may include an angle relative to other speakers of the speaker arrangement and / or relative to the listener. Based on the determined position, parameters such as gain, delay, etc. may be adjusted to generate a virtual sound source.

[0056] In some embodiments, the speaker-dependent expansion factor is based on the formula:

[0057]

[0058] in,

[0059] γ n，l is a speaker-dependent expansion factor of at least one individual speaker of the speaker arrangement;

[0060] r n，l is the distance between at least one loudspeaker of the loudspeaker arrangement and the generated virtual sound source;

[0061] r n，min is the distance between the speaker of the speaker device closest to the virtual sound source and the virtual sound source;

[0062] r n，max is the distance between the speaker of the speaker device farthest from the virtual sound source and the virtual sound source;

[0063] σ n，l is the loudspeaker related expansion factor.

[0064] This formula can be referred to Figure 1 , Figure 2 ,as well as Figure 3 The explanation is as follows.

[0065] Figure 1 A system 100 is shown, comprising: a virtual sound source 2 , a user 3 , and a speaker arrangement, the arrangement comprising speakers 4 , 5 , 6 , 7 .

[0066] Arrows 32, 34, 35, 36, 37, 42, 52, 62, 72 represent vectors, wherein the reference signs of the arrows indicate the start and end of the corresponding vectors, such that an exemplary vector XY (X and Y are selected from reference sign pools 2, 3, 4, 5, 6, 7) starts from an element with reference sign X and ends at an element with reference sign Y. For example, arrow 32 shows a vector starting from user 3 and ending at virtual sound source 2; arrow 35 shows a vector starting from user 3 and ending at speaker 5; arrow 62 shows a vector starting from speaker 6 and ending at virtual sound source 2, and so on.

[0067] The virtual sound source 2 is depicted as an extended object. However, this is for illustration purposes only, and in this embodiment, it is assumed that the virtual sound source is a point source. Therefore, vectors 32, 42, 52, 62, 72 are considered to end at the same point, although these vectors are depicted as ending at different points.

[0068] Furthermore, for illustration purposes, a two-dimensional arrangement of elements 2 to 7 is shown. However, the embodiment is not limited to a two-dimensional arrangement. In general, a three-dimensional arrangement should be considered.

[0069] Furthermore, for illustration purposes, only one virtual sound source is depicted. However, the present disclosure is not limited to one virtual sound source. Other embodiments may refer to any number of virtual sound sources greater than one.

[0070] Therefore, in this embodiment, a plurality of virtual sound sources are assumed.

[0071] The number of speakers is not limited to four. It can also be 2, 3, or any number greater than 4.

[0072] To explain the formula, the variables and examples of how to retrieve them are described below.

[0073] First, the distance (in Cartesian coordinates) between each virtual sound source n and each corresponding loudspeaker 1 is determined, for example, by determining the norm of the associated vector X2, where X in this case is the element of {4,5,6,7}, resulting in the distance r:

[0074]

[0075] Among them, index n refers to a virtual sound source (2) among multiple virtual sound sources; l refers to a speaker (4 to 7) of the speaker device; m refers to a vector of virtual sound source 2; X refers to a vector of speakers 4 to 7 of the speaker device; and indices x, y and z refer to the x-coordinate, y-coordinate and z-coordinate of the vector in three-dimensional space, respectively.

[0076] For example, 2,5 It can refer to the distance between the virtual sound source 2 and the speaker 5; m 2,x Can refer to the x coordinate of virtual sound source 2; X 5,y It could refer to the y coordinate of speaker 5, and so on.

[0077] Second, determine the gain G of each speaker relative to the virtual sound source according to the equation:

[0078]

[0079] However, the present disclosure is not limited to determining the gain in this manner, and any other manner of determining the gain is possible. For example, the value of the gain may be a dimensionless character or have other dimensions. For example, depending on the speaker type of the speakers 4 to 7, another manner of determining the gain compared to other speakers 4 to 7 in the same system may also be used.

[0080] Third, the delay D of each loudspeaker 4 to 7 relative to the virtual sound source 2 is determined according to the equation:

[0081]

[0082] Among them, c 0 is the speed of sound, and T s Refers to the sampling period. However, the present disclosure is not limited to determining the delay in this manner, and any other manner of determining the delay is possible. For example, the delay may not be a rounded value, and the delay may be a dimension of time, space, etc. For example, depending on the speaker type of the speakers 4 to 7, another manner of determining the delay compared to other speakers 4 to 7 in the same system may also be used.

[0083] These first three steps may be performed iteratively for each loudspeaker 4 to 7, and for each sound source 2. However, they may be performed for only one loudspeaker, such as the loudspeaker 4, and one virtual sound source, such as the virtual sound source 2, or for a subset of the loudspeakers 4 to 7, and a subset of the sound sources 2. These first three steps may also be performed in another order, such as exchanging the second and third steps, without limiting the present disclosure in this regard.

[0084] The fourth step may comprise determining, for each virtual sound source 2, a minimum distance r between the virtual sound source 2 (eg n=2) and the loudspeakers 4 to 7 of the loudspeaker arrangement. n，min and the maximum distance r n,max .

[0085] The fifth step may be to calculate an expansion factor similar to the expansion factor described in the above formula

[0086]

[0087] Among them, σ n is the expansion coefficient of the virtual sound source n. In some embodiments, the expansion coefficient may have a positive value.

[0088] The sixth step can be applied to γ n，l : The conditions include:

[0089] If γ n，l >0, then G n，l =γ n，l *G n，l Otherwise G n，l =0(6).

[0090] The fifth and sixth steps may be performed iteratively for each loudspeaker 4 to 7 , or an individual loudspeaker 4 , or a subset of loudspeakers 4 to 7 .

[0091] For the expansion factor σ n = 1, the result is that the nearest speaker r n,min and the farthest r n,max The expansion factor betweenn，l From γ n，min = 1 linearly decreases to γ n，max = 0. In σ n →∞, the larger expansion coefficient converges to the identity (γ n，l =1), and in σ n = 0, a smaller expansion factor increases the directivity (γ n，l →-∞). In the latter case, only the speaker closest to the source produces the sound (γ n，min =1,γ n，l≠min →-∞,G n，l≠min =0).

[0092] Figure 2 is a factor of expansion including different types of n，l (ordinate) as a function of normalized distance (abscissa), where r min corresponds to a distance of zero, and r max corresponds to a distance of 1.

[0093] The function includes: an identification function 201, a linear decreasing function 202, an indication function 203 when the expansion factor is 0.5, and a cardioid function 204. The function is not limited to the function shown in this context. Any other function for the expansion factor, such as an omnidirectional function, a directional function, a super directional function, a bidirectional function, a figure-of-eight function, a subcardioid function, a cardioid function, a unidirectional function, a supercardioid function, a supercardioid directivity function, etc., can also be derived and implemented.

[0094] The function can be converted into Figure 3 Polar coordinates shown.

[0095] Figure 3 A diagram of a polar coordinate system 200' is shown including different types of expansion factors (radii) as a function of normalized angle, where r min corresponds to an angle of zero degrees, and r max Corresponds to an angle of 180 degrees.

[0096] Figure 3 Also includes the distance r (corresponding to Figure 2 The first scale converts the distance) into polar angles from zero to 180 degrees and shows the radius from zero dB (decibel) to 30 dB gain levels.

[0097] The polar coordinate system 200' is derived by transforming the ordinate coordinates of the coordinate system 200 by linearly assigning the values ​​[0; 1] of the coordinate system 200 to the values ​​[0°; 180°]. Figure 3 The function can be interpreted as Figure 2 Another explanation of the function, Figure 3 With the label 201′, linearly decreasing (σ n =1) 202', indicator function 203' (σ n =0.5), and a cardioid function 204' assigned to the second scale 202. Any other function that can be converted from a linear system to a polar system can also be used in this context, such as an omnidirectional function, a directional function, a super directional function, a bidirectional function, a figure-of-eight function, a subcardioid function, a cardioid function, a unidirectional function, a supercardioid function, a hypercardioid directivity function, etc.

[0098] Without limiting the present disclosure in this regard, for purposes of parameterization, the expansion coefficients may be constrained to be in the range of [0; 1] (in other embodiments, any other interval may be used).

[0099] Furthermore, a parameter directional gain or DirGain may be introduced which may be multiplied with the expansion factor in order to obtain an arbitrary number in the real domain.

[0100] In addition, a parameter angle1 may be introduced. angle1 may depend on the type, position, posture, etc. of the speakers 4 to 7. angle1 may be determined by the device according to an embodiment of the present disclosure by measuring properties related to the speaker 4, or may be obtained from a database (such as a database stored in a circuit within the speaker 4), or from the Internet, etc.

[0101] Therefore, a loudspeaker-dependent expansion factor can be introduced based on the formula:

[0102] σ n，l =σ n *DirGain*angle l (7).

[0103] The loudspeaker-related expansion factor can be substituted for the expansion factor σ in formula (5) n , leading to formula (1):

[0104]

[0105] Some embodiments relate to a method comprising determining a speaker-dependent expansion factor for at least one individual speaker of a speaker arrangement, wherein the speaker-dependent expansion factor depends on specifications of the at least one individual speaker; and controlling outputs of the speakers of the speaker arrangement based on the speaker-dependent expansion factor of the at least one individual speaker to generate at least one virtual sound source, as described above.

[0106] The method may be performed on the apparatus as described above, or by any other apparatus, device, processor, circuit, etc.

[0107] The speaker-dependent expansion factor may depend on the distance of the virtual sound source to at least one individual speaker of the speaker arrangement, as discussed herein, wherein a point in time at which the at least one individual speaker generates a sound to generate the virtual sound source is determined based on the determined distance of the virtual sound source to the at least one individual speaker of the speaker arrangement, as discussed herein.

[0108] The speaker-dependent expansion factor may also be determined according to a non-linear function, as discussed herein, which may depend on the distances of the individual speakers of the speaker arrangement to the virtual sound source, as discussed herein.

[0109] The method may further comprise generating a virtual sound source from contributions from respective speakers, the contributions being amplified and delayed versions of the input audio signal, as described herein.

[0110] As described herein, the method may further comprise: adjusting the gain of individual loudspeakers of the loudspeaker arrangement, wherein the gain may be modified by an expansion factor, wherein the adjustment of the gain may also depend on the distance between the listener and the virtual sound source, as described herein. In particular, wherein the gain of the loudspeaker closest to the listener in the loudspeaker arrangement may be higher than the gain of the individual loudspeakers, as described herein.

[0111] As discussed herein, the method may also include determining a position of at least one individual speaker of the speaker arrangement relative to a listener position.

[0112] The method may also include determining a speaker-dependent expansion factor based on equation (1) discussed herein.

[0113] The introduction of the expansion factor discussed in this article can solve the following situations, which will refer to Figure 4 Have a discussion. Figure 4 A system 310 is shown that includes two speakers 311 and 312. For this example, speakers 311 and 312 are assumed to be located in a car. Speakers 311 and 312 may have different frequency ranges, i.e., in this example, speaker 311 is a tweeter and speaker 312 is a woofer.

[0114] The speakers 311 and 312 generate three virtual sound sources 313 , 314 , and 315 .

[0115] The frequency range of the speaker 311 (312) is shown in the graph 316 (317). The abscissa of the graph 316 (317) represents the frequency of the speaker 311 (312), and the ordinate represents the gain of the speaker 311 (312).

[0116] The frequency range of the virtual sound source 313 (314, 315) is depicted in a graph 318 (319, 320). The abscissa of the graph 318 (319, 320) represents the frequency of the virtual sound source 313 (314, 315), and the ordinate represents the gain of the virtual sound source 313 (314, 315).

[0117] As can be seen from the thickness of the arrows shown between the speakers 311 and 312 and the virtual sound sources 313 to 315, the influence of the speaker 311 (312) is dominant compared to the speaker 312 (311) in generating the virtual sound source 313 (315), while the speakers 311 and 312 contribute equally to generating the virtual sound 314.

[0118] Typically, this can lead to the result that for the virtual sound source 313 the frequencies of the loudspeaker 311 can be perceived in advance primarily, as can be obtained from diagram 318. This can also apply to the primary perception of the timbre of the loudspeaker 312 of the virtual sound source 315, as can be obtained from diagram 320. Diagram 319 shows that for the virtual sound source 314 the frequencies of both loudspeakers 311 and 312 can be perceived equally.

[0119] However, as described herein, applying the expansion factor according to the present disclosure may result in that, for each of a plurality of virtual sound sources generated by the plurality of speakers, the perception of the timbre emitted by the plurality of speakers may be (almost) equal.

[0120] In some embodiments, the method described herein is also implemented as a computer program, which, when executed on a computer and / or a processor, causes the computer and / or the processor to perform the method. In some embodiments, a non-transitory computer-readable recording medium is also provided, which has a computer program product stored therein, which, when executed by a processor such as the above-mentioned processor, causes the method described herein to be performed.

[0121] Below, reference Figure 5 Discussing embodiments of the device, Figure 5 A block diagram of an apparatus implemented as an audio system 400 (or alternatively as an electronic device 401) is depicted.

[0122] The audio system 400 includes an electronic device 401 connected to a microphone device 410, a speaker device 411, a user interface 412, and a sensor 413. The electronic device 401 is a 3D sound rendering system in this embodiment.

[0123] The electronic device 401 has a CPU 402 as a processor, a data memory 403, and a data memory 404 (here, RAM).

[0124] The data memory 404 is arranged to temporarily store or cache data and / or computer instructions for processing by the processor 402 .

[0125] The data storage 403 is used to store recorded sensor data obtained from, for example, the microphone device 410 .

[0126] The electronic device 401 is configured to execute software for 3D audio rendering operations that virtually place sound sources anywhere in the room, including the listener (such as Figure 1 3) behind, above, or below the audience.

[0127] The electronic device 401 has a WLAN interface 405, a Bluetooth interface 406, and an Ethernet interface 407. These interfaces 405, 406, 407 serve as I / O interfaces for data communication with external devices.

[0128] For example, a smartphone may be connected to the 3D sound rendering system by means of a Bluetooth interface 406 and / or a WLAN interface 405. Additional speakers, microphones and cameras with Ethernet, WLAN or Bluetooth connections may be coupled to the electronic device 401 via these wireless / wired interfaces 405, 406 and 407.

[0129] For example, microphone arrangement 410 may be comprised of one or more microphones distributed around a listener.

[0130] The user interface 412 is connected to the processor 402. The user interface 412 acts as a human-machine interface and allows a dialog between the administrator and the audio system 400.

[0131] The sensor 413 is connected to the processor 402. The sensor 413 includes a temperature sensor and a camera. The sensor 413 is configured to obtain the presence and position of one or more listeners and the position and direction of the listeners' heads. The cameras may be distributed over a predetermined space, or a single camera may be used to obtain images.

[0132] The audio system 400 uses a microphone array 410 to output sound from a speaker of a speaker device 411 and at least one virtual sound source (eg, Figure 1 The virtual sound source 2 of the present invention receives audio data in order to monitor the generated virtual sound source (for example, Figure 1 virtual sound source 2), and if necessary, adjust the speaker device 411 to affect the generated virtual sound source.

[0133] Figure 6 A flowchart of an embodiment of a method 500 for generating a virtual sound source according to an embodiment of the present disclosure is depicted, wherein the method 500 is composed of Figure 5 The audio system 400 is executed.

[0134] First, the position of the speaker is determined in 501. This may be performed by object recognition techniques with an image generation system, using mapping techniques such as SLAM (Simultaneous Localization and Mapping), measuring the position of the speaker by a sensor (e.g., by a radar-based approach), by obtaining user input indicating the position of the speaker via a user interface, without limiting the present disclosure in this regard.

[0135] Then, in 502, the type of the speaker is determined, for example, by reading an internal database of the speaker, by obtaining user input indicating the type of the speaker via a user interface, and the like.

[0136] In 503, the angle parameter as described above is determined, such as angle l The information about the angle parameters is implicitly provided in the type of loudspeaker, or is obtained from a database similar to the database in 501, or is obtained via a user interface (such as in 502 or 503).

[0137] In 504, expansion factors are determined, which factors depend on the type of loudspeaker in this embodiment and are therefore implicitly defined by the type of loudspeaker. Alternatively, they are obtained from a database by user input or the like, as described above.

[0138] In 505, the position of the listener is determined by using one of the techniques for determining the positions of the speakers as described in 501, or the listener may input via a user interface where he is.

[0139] In 506, the position of the virtual sound source is determined. It should be noted that the virtual sound source may not be generated at this point in time. Therefore, this step can be understood as determining the position of the virtual sound source at a future point in time. However, the present disclosure is not limited to any of these situations, and the position of the virtual sound source can be determined based on the position of the listener (e.g., two meters in front of the listener's face), the position of the speaker (e.g., the balance point of the speaker geometry), parameters including these two positions, or by input via a user interface.

[0140] At 507 , a speaker-dependent expansion factor is determined according to equation (5) as described herein without limiting the present disclosure in this regard.

[0141] In 508 , a virtual sound source is generated by applying all determined parameters to a computer program, as the program may be executed in the electronic device 401 , for example.

[0142] In some embodiments, 3D audio rendering is implemented based on a digital monopole synthesis algorithm, which will be referred to below. Figure 7 Have a discussion.

[0143] The target sound field is modeled as at least one target monopole placed at a defined target position. In one embodiment, the target sound field is modeled as a single target monopole. In other embodiments, the target sound field is modeled as multiple target monopoles placed at respective defined target positions. For example, each target monopole can represent a noise cancellation source that is included in a set of multiple noise cancellation sources located at a specific position in space. The position of the target monopole may be moving. For example, the target monopole can adapt to the movement of the noise source to be attenuated. If multiple target monopoles are used to represent the target sound field, the method of synthesizing the sound of the target monopole based on a set of defined synthetic monopoles as described below can be applied independently to each target monopole, and the contribution of the synthetic monopole obtained for each target monopole can be added to reconstruct the target sound field.

[0144] The source signal x(n) is fed to The delay unit and the amplifier unit are marked as a p , where p=1, ..., N, p is the index of each synthetic monopole for synthesizing the target monopole signal. The delay and amplification unit according to this embodiment can apply equation (117) of reference US2016 / 0037282 A1 to calculate the resulting signal y for synthesizing the target monopole signal p (n) = s p (n). The resulting signal s p (n) is power amplified and fed to the speaker S p .

[0145] In the present embodiment, the synthesis is therefore performed in the form of delayed and amplified components of the source signal x.

[0146] According to this embodiment, the delay n of the synthetic monopole for index p p Corresponding to the target monopole r 0 and generator r P The Euclidean distance between p0 =|r p -r 0 |Sound travel time.

[0147] In addition, according to this embodiment, the amplification factor and distance r = Rp 0 Inversely proportional.

[0148] In an alternative embodiment of the system, a modified amplification factor according to equation (118) of reference US 2016 / 0037282 A1 may be used.

[0149] It should be appreciated that the embodiments describe method 500 with an exemplary ordering of method steps. However, the specific order of the method steps is for illustrative purposes only and should not be construed as binding. For example, Figure 6 The order of 502 to 508 in the embodiment can be arbitrarily exchanged.

[0150] Please note that the electronic device 401 is divided into units 401 to 407 for illustration purposes only, and the present disclosure is not limited to any specific functional division in a specific unit. For example, the electronic device 401 can be implemented by a corresponding programmed processor, a field programmable gate array (FPGA), etc.

[0151] All units and entities described in this specification and claimed in the appended claims may, if not stated otherwise, be implemented as integrated circuit logic, for example on a chip, and the functions provided by these units and entities may be implemented by software, if not stated otherwise.

[0152] With respect to the above-disclosed embodiments, which are at least partially implemented using a software-controlled data processing device, it should be understood that a computer program controlled and transmitted by such software is provided, and as an aspect of the present disclosure, a memory or other medium for providing such a computer program is contemplated.

[0153] Note that the present technology can also be configured as described below.

[0154] (1) An apparatus comprising a circuit, the circuit being configured to:

[0155] determining a speaker-dependent expansion factor for at least one individual speaker of the speaker arrangement, wherein the speaker-dependent expansion factor depends on specifications of the at least one individual speaker; and

[0156] The output of the loudspeakers of the loudspeaker arrangement is controlled based on a loudspeaker-dependent expansion factor of at least one individual loudspeaker to generate at least one virtual sound source.

[0157] (2) The apparatus according to (1), wherein the speaker-dependent expansion factor depends on a distance of the virtual sound source to at least one individual speaker of the speaker arrangement.

[0158] (3) The device according to any one of (1) or (2), wherein the circuit is further configured to: determine a time point at which at least one separate speaker generates sound to generate a virtual sound source according to the distance.

[0159] (4) The apparatus according to any one of (1) to (3), wherein the speaker-dependent extension factor is determined according to a nonlinear function.

[0160] (5) The device according to any one of (1) to (4), wherein the nonlinear function depends on the distance of the individual speakers of the speaker device to the virtual sound source.

[0161] (6) The apparatus of any one of (1) to (5), wherein the virtual sound source is generated by contributions from each speaker, the contributions being amplified and delayed versions of the input audio signal.

[0162] (7) The device of any of (1) to (6), wherein the circuit is further configured to adjust a gain of an individual speaker of the speaker device, wherein the gain is modified by the expansion factor.

[0163] (8) The apparatus according to any one of (1) to (7), wherein the gain is adjusted depending on a distance between a listener and a virtual sound source.

[0164] (9) The device according to any one of (1) to (8), wherein a gain of a speaker closest to a listener in the speaker device is higher than gains of other speakers.

[0165] (10) The apparatus of any one of (1) to (9), wherein the speaker-dependent expansion factor is based on the formula:

[0166] in,

[0167] γ n，l is a speaker-dependent expansion factor of at least one individual speaker of the speaker arrangement;

[0168] r n，l is the distance between at least one loudspeaker of the loudspeaker arrangement and the generated virtual sound source;

[0169] r n，min is the distance between the speaker of the speaker device closest to the virtual sound source and the virtual sound source;

[0170] r n，max is the distance between the speaker of the speaker device farthest from the virtual sound source and the virtual sound source;

[0171] σ n，l is the loudspeaker related expansion factor.

[0172] (11) A method comprising:

[0173] determining a speaker-dependent expansion factor for at least one individual speaker of the speaker arrangement, wherein the speaker-dependent expansion factor depends on specifications of the at least one individual speaker; and

[0174] The output of the loudspeakers of the loudspeaker arrangement is controlled based on a loudspeaker-dependent expansion factor of at least one individual loudspeaker to generate at least one virtual sound source.

[0175] (12) The method according to (11), wherein the speaker-dependent expansion factor depends on the distance of the virtual sound source to at least one individual speaker of the speaker arrangement.

[0176] (13) According to the method of any one of (11) or (12), the method further includes: determining a time point at which at least one separate speaker generates sound to generate a virtual sound source according to the distance.

[0177] (14) The method according to any one of (11) to (13), wherein the speaker-dependent expansion factor comprises a nonlinear function.

[0178] (15) A method according to any one of (11) to (14), wherein the nonlinear function depends on the distance of the individual loudspeakers of the loudspeaker arrangement from the virtual sound source.

[0179] (16) A method according to any one of (11) to (15), wherein the virtual sound source is generated by contributions from each speaker, which are amplified and delayed versions of the input audio signal.

[0180] (17) According to the method of any one of (11) to (16), the method also includes adjusting the gain of an individual speaker of the speaker device, wherein the gain is modified by the expansion factor.

[0181] (18) The method according to any one of (11) to (17), wherein the adjustment of the gain depends on the distance between the listener and the virtual sound source.

[0182] (19) The method according to any one of (11) to (18), wherein a gain of a speaker closest to a listener in the speaker device is higher than gains of other speakers.

[0183] (20) A method according to any one of (11) to (19), wherein the speaker-dependent expansion factor is based on the formula:

[0184] in

[0185] γ n，l is a speaker-dependent expansion factor of at least one individual speaker of the speaker arrangement;

[0186] r n，l is the distance of at least one loudspeaker of the loudspeaker arrangement to the generated virtual sound source;

[0187] r n，minis the distance between the speaker of the speaker device closest to the virtual sound source and the virtual sound source;

[0188] r n，max is the distance between the speaker of the speaker arrangement farthest from the virtual sound source and the virtual sound source;

[0189] σ n，l is the loudspeaker related expansion factor.

[0190] (21) A computer program comprising a program code which, when executed on a computer, causes the computer to execute the method according to any one of (11) to (20).

[0191] (22) A non-transitory computer-readable recording medium storing a computer program product which, when executed by a processor, causes execution of the method according to any one of (11) to (20).

Claims

1. An apparatus for operating spatial audio, comprising a circuit, the circuit being configured to: A speaker-dependent expansion factor of at least one individual speaker of the speaker arrangement is determined, wherein said loudspeaker-dependent expansion factor being dependent on specifications of said at least one individual loudspeaker, said specifications of said at least one individual loudspeaker comprising a directivity pattern and a type of membrane of said at least one individual loudspeaker; and The output of each loudspeaker of the loudspeaker arrangement is controlled based on the loudspeaker-dependent expansion factor of the at least one individual loudspeaker to generate at least one virtual sound source.

2. The device according to claim 1, wherein: The loudspeaker-dependent expansion factor further depends on the distance of the virtual sound source to the at least one individual loudspeaker of the loudspeaker arrangement.

3. The device according to claim 2, wherein: The circuit is further configured to determine a point in time at which the at least one individual speaker generates sound to generate the virtual sound source according to the distance.

4. The device according to claim 1, wherein: The loudspeaker-dependent expansion factor is determined according to a non-linear function.

5. The device according to claim 4, wherein: The non-linear function depends on the distance of an individual loudspeaker of the loudspeaker arrangement from the virtual sound source.

6. The device according to claim 5, wherein: The virtual sound source is generated by contributions from each loudspeaker, which are amplified and delayed versions of the input audio signal.

7. The device according to claim 1, wherein: The circuit is further configured to adjust a gain of an individual speaker of the speaker arrangement, wherein the gain is modified by the expansion factor.

8. The device according to claim 7, wherein: The gain is adjusted depending on the distance between the listener and the virtual sound source.

9. The device according to claim 8, wherein: The gain of the speaker closest to the listener in the speaker arrangement is higher than the gains of the other speakers.

10. The device according to claim 1, wherein: The speaker-dependent expansion factor is based on the formula: in, γn,l is the speaker-dependent expansion factor of the at least one individual loudspeaker of the loudspeaker arrangement; r n，l is the distance between at least one speaker of the speaker device and the generated virtual sound source; r n，min is the distance between the speaker of the speaker device closest to the virtual sound source and the virtual sound source; r n，max is the distance between the speaker of the speaker device farthest from the virtual sound source and the virtual sound source; σ n，l is the loudspeaker related expansion factor.

11. A method for operating spatial audio, comprising: determining a speaker-dependent expansion factor for at least one individual speaker of the speaker arrangement, wherein the speaker-dependent expansion factor depends on specifications of the at least one individual speaker, the specifications of the at least one individual speaker comprising a directivity pattern and a type of membrane of the at least one individual speaker; and The output of each loudspeaker of the loudspeaker arrangement is controlled based on the loudspeaker-dependent expansion factor of the at least one individual loudspeaker to generate at least one virtual sound source.

12. The method according to claim 11, wherein: The loudspeaker-dependent expansion factor further depends on the distance of the virtual sound source to the at least one individual loudspeaker of the loudspeaker arrangement.

13. The method according to claim 12, further comprising: A point in time at which the at least one individual speaker generates sound to generate the virtual sound source is determined according to the distance.

14. The method according to claim 11, wherein: The loudspeaker-dependent expansion factor is determined according to a non-linear function.

15. The method according to claim 14, wherein: The non-linear function depends on the distance of an individual loudspeaker of the loudspeaker arrangement from the virtual sound source.

16. The method according to claim 15, wherein: The virtual sound source is generated by contributions from each loudspeaker, which are amplified and delayed versions of the input audio signal.

17. The method according to claim 11, further comprising: The gain of an individual loudspeaker of the loudspeaker arrangement is adjusted, wherein the gain is modified by the expansion factor.

18. The method according to claim 17, wherein: The gain is adjusted depending on the distance between the listener and the virtual sound source.

19. The method according to claim 18, wherein: The gain of the speaker closest to the listener in the speaker arrangement is higher than the gains of the other speakers.

20. The method according to claim 11, wherein: The speaker-dependent expansion factor is based on the formula: in, γ n，l is the speaker-dependent expansion factor of the at least one individual speaker of the speaker arrangement; r n，l is the distance between at least one speaker of the speaker device and the generated virtual sound source; r n，min is the distance between the speaker of the speaker device closest to the virtual sound source and the virtual sound source; r n，max is the distance between the speaker of the speaker device farthest from the virtual sound source and the virtual sound source; σ n，l is the loudspeaker related expansion factor.

Images