Generating device and room related impulse responses

Abstract

The invention regards a computer-implemented method for generating a device specific room impulse response (DSRIR) describing an acoustic characteristic of a device and a room as received by the device, wherein the method comprises generate an at least first device related transfer function (DRTF), wherein the at least first device related transfer function describes the acoustic characteristic of the device as received by the at least first microphone, generate a spatial room impulse response (SRIR), wherein the spatial room impulse response describes the acoustic characteristic of the room from at least one room sound source in the room and received at an at least one listening point in the room from at least one direction and generating the device specific room impulse response (DSRIR) by combining the device related transfer function and the spatial room impulse response.

Description

[0001] Generating Device And Room Related Impulse Responses

[0002] Background

[0003] Acoustic simulations refer to the computer-based modeling and simulation of sound propagation and interactions in various environments. These simulations are valuable tools in fields such as engineering, architecture and audio design, allowing researchers and professionals to predict, analyze, and optimize acoustic conditions in real-world scenarios. Acoustic simulations are often used to generate impulse responses and transfer functions of simulated object, whether to simulate an actual condition and determine where acoustic issues arise or when designing new object, e.g. buildings and room, in order to avoid acoustic problems in the architecture.

[0004] In acoustic simulation tools today the most dominant solvers used are ray based or image based, often referred to as geometrical solvers. These are fast and processor efficient but as they are ray based they do not take into account wave-based issues which for example causes them to be very bad at detecting wave-based phenomena such as standing waves, wave cancellation and similar issues. This is in particular an issue at lower or mid acoustic frequencies.

[0005] However, with increasing computational processing power being available e.g. through cloud computing, using so called wave-based solver becomes feasible. However, still due to computational constraints care has to be taken when implementing such method in order for them to provide a high-fidelity result as fast as possible.

[0006] Audio device modelling and optimization is a critical aspect of audio engineering. Current methods require time-consuming simulation methods or measurements in real- life conditions, in specific rooms equipped with several loudspeakers in order to be able to characterize how the sound is received by the audio device from several sound sources. Moreover, when considering how the device performs in a certain room or space, such simulation will have to take into account both the device geometry associated with its acoustic characteristics and the space or room geometry associated with its acoustic characteristics creating an even more complex and time consuming simulation.

[0007] Wave-based simulations such as finite element method (FEM), finite-difference timedomain method (FDTD), boundary element method (BEM) or finite volume method (FVM) are used in order to simulate acoustic waves with a high fidelity. One of the challenges of the previously mentioned wave-based simulations is the complexity of the calculations and the inability to parallelize computational resources, which makes it time-consuming. Some recent improvements has been performed in order to reduce the simulation time, especially by using the discontinuous Galerkin method (DG), which allows an efficient parallelization of the simulation, thereby drastically reducing the simulation time [1], Wave-based simulations remain dependent on a meshing, which consists of meshing the inner volume of the 3D model and the geometry of the elements comprised in the 3D model, where the acoustic waves needs to be simulated by a wave-based simulation. The size of the mesh elements or geometrical features in the mesh dictates the time stepping of the wave-based simulation. Thereby, if complex geometries are comprised in the inner volume of the 3D model, the simulation time can become extremely long, sometimes up to days, if somehow the simulation can converge.

[0008] Audio device geometries are sometimes quite complex and relatively small compared to a size of a room on which they are arranged, and they may comprise several microphones. Simulating how the acoustic waves, thereby the sound may be captured by an audio device from one or more audio sources in a 3D model of a room can be challenging. This becomes drastically complex if the audio device comprises several microphones.

[0009] Summary

[0010] Thus, there exists a need for making wave-based simulations and generation of impulse response and transfer function more effective, in particular when considering an object, such as an audio device in relation to its performance in a space or room, e.g. a meeting room. As disclosed here this can be achieved by performing separate simulations and generate impulse responses and transfer functions for the device in one step and the room in another step, such that the results can subsequently be combined to give a full evaluation of the device and room together, while allowing a rotation of the audio device without a cost of running additional simulations of impulse responses.

[0011] In one aspect, the present disclosure relates to a computer-implemented method for generating a device specific room impulse response (DSRIR) describing an acoustic characteristic of a device and a room as received by the device, wherein the device comprises at least a first microphone. The computer implemented method further comprises steps to

[0012] - generate an at least first device related transfer function (DRTF), wherein the at least first device related transfer function describes the acoustic characteristic of the device as received by the at least first microphone

[0013] - generate a spatial room impulse response (SRIR), wherein the spatial room impulse response describes the acoustic characteristic of the room from at least one room sound source in the room and received at an at least one listening point in the room from at least one direction, and

[0014] - generating the device specific room impulse response (DSRIR) by combining the device related transfer function and the spatial room impulse response.

[0015] Generating the DRTF and the SRIR in two separate steps, e.g. by two separate simulations, can have the advantage that one large simulation is avoided which especially can be cumbersome in case wave-based solvers are used.

[0016] Wave-based solvers use meshes to solve and are in such cases limited by the smallest mesh in the model. As the device will typically be represented by much smaller mesh elements than the room, running a simulation with a meshed model in a room will become very slow. Even further, it is often desired to run several room simulations, either of different rooms or different listening points in order to evaluate how a device performs in such a room. Therefore, providing a method where the DRTF and the SRIR are generated separately but being able to be combined to a DSRIR provides a lot of options in generating new room, re-orient the device, or use a different device, at a higher speed than if everything was simulated in the same step while maintaining a high fidelity.

[0017] In one embodiment of the above aspect, such advantages become apparent in a computer-implemented method for generating a device specific room impulse response (DSRIR) describing an acoustic characteristic of a device and a room as received by the device, wherein the device comprises at least a first microphone, the method comprises

[0018] - generate an at least first device related transfer function (DRTF), wherein the at least first device related transfer function describes the acoustic characteristic of the device as received by the at least first microphone, wherein generating the first device related transfer function further comprises o obtain a device mesh model representing the geometry of the device and the position of the at least first microphone on the device mesh model, o arrange a digital representation of a device receiver array comprising a plurality of digital representations of device receivers around the device mesh model, such that the distance between any of the digital representations of the device receivers and the device mesh model is not below a predetermined distance, o determine on the device mesh model a first closest mesh element, which is closest to the at least first microphone, o arrange a digital representation of a first source correction microphone located at a first source distance from the first closest mesh element, wherein the first source distance is smaller than the predetermined distance, o digitally emit a first impulse signal using the first closest mesh element as a sound source, o determine a first source correction signal using a wave-based solver, where the first source correction signal describes the first impulse signal as received at the first source correction microphone, o determine a plurality of first device impulse responses using a wavebased solver, and where each first device impulse response describes the impulse response of the first impulse signal received at the respective device receiver, o determine a plurality of first source corrected device impulse responses by source correcting each of the plurality of first device impulse responses using the first source correction signal, o generate the first device related transfer function of the device for the first microphone by combining the plurality of first source corrected device impulse responses, o determine an energy content for at least one frequency of the first device related transfer function,

[0019] - generate a spatial room impulse response (SRIR), wherein the spatial room impulse response describes the acoustic characteristic of the room from at least one room sound source in the room and received at an at least one listening point in the room from at least one direction, wherein generating the spatial room impulse response further comprises o obtain a 3D room model representing the geometry of the room and at least one acoustic characteristic, o arrange at least one digital representation of at least one room sound source in the 3D room model, o arrange a digital representation of a room receiver array comprising a number of digital representations of room receivers, wherein the room receiver array is centred on an at least one listening point in the 3D room model and where the number of digital representations of room receivers are determined based on the energy content for the at least one frequency of the first device related transfer function, o digitally emit a room impulse signal from the at least one audio sound source, o determine a number of room impulse responses using at least a wavebased solver for at least one wave-based frequency, where each room impulse response describes the emitted room impulse as received at a corresponding one of the number of digital representations of room receivers, o generate a spatial room impulse response based on the number of room impulse responses,

[0020] - generating the device specific room impulse response (DSRIR) by combining the device related transfer function and the spatial room impulse response.

[0021] This embodiment is for example suitable to be used for ambisonics as the device receiver array used for generating the first DRTF and the room receiver array SRIR can be dimensioned to use spherical harmonics to encode and decode to and from ambisonics. For example, the number of receivers in the device array determines the highest ambisonics order N based on (N+1)2. Similarly the spatial impulse response for the room can be encoded to ambisonics using spherical harmonics, where the number of receivers in the room array determines the highest ambisonics order N also based on (N+1)2.

[0022] The number of receivers in the device receiver array or the room receiver array can determine the maximum truncation order N that can be considered for the spherical harmonics decomposition. The sound field may be sampled with at least as many receivers as the number of terms used for the expansion, which leads to a number of receivers higher or equal to (N+1)2. This condition can be necessary to avoid undersampling but cannot guarantee accuracy. The optimal number of receivers can depend on the chosen quadrature rule; an exact integration can for instance be performed with a Gauss quadrature, where the number of receivers may be higher or equal to 2(N+1)2. Other approaches may require less samples. Another constraint may pertain to spatial aliasing. Aliasing can be mitigated by choosing a sufficiently large order N, or by using anti-aliasing spatial filters.

[0023] Using spherical harmonics to encode and decode ambisonics is generally known but as mentioned, it can be a compute-intensive process which lately has become more feasible within increased computing power available. However, controlling and using the optimal ambisonics order N will allow the system to be more effective and efficient.

[0024] Since the wave-based simulation used to derive the device related transfer function DRTF is done or executed in a smaller space, e.g. within the device receiver array, the ambisonics order N(device) for the device can generally be higher. However, for the room based simulation used for determining the spatial room impulse response SRIR, the wave-based simulation is generally done for a larger space. Therefore, determining the optimal ambisonics order N(room) for the room, e.g. by using the energy content from the device simulation, will allow for faster but still high fidelity result.

[0025] The wave-based simulation as discussed above can also partly be a geometrical acoustic simulation. By combining or merging a wave-based simulation and a geometrical acoustic simulation, this may allow for an optimal simulation when considering speed of simulation vs accuracy.

[0026] Merging and hybridizing the wave-based simulation and the geometrical acoustic simulation can be done by merging and hybridizing a wave based impulse response obtained with the wave-based simulation and a geometrical impulse response obtained with the geometrical acoustic simulation. This requires careful consideration in order to avoid non-physical artefacts to be introduced into the combined broadband solution forming the simulated impulse response. The simulated wave based propagation and the simulated geometrical acoustic based propagation can be simulated in a first frequency range and a second frequency range, respectively. The simulated wave based propagation may be performed with the wavebased simulation and the simulated geometrical acoustic based propagation can be performed with the geometrical acoustic simulation. The first frequency range and the second frequency range can partly overlap or completely overlap. A transition frequency is defined as being the frequency where the first frequency range and the second frequency range overlaps. The transition frequency may be defined as the centre frequency of the overlapping between the first frequency range and the second frequency range.

[0027] The transition frequency may be a default transition frequency. The default transition frequency may be proportional to the square root or the cubic root of the volume of the 3D room model. The default transition frequency may be based on a simplified calculation of the Schroeder frequency assuming a reverberation time of 1 second and multiplying it by a factor. The reverberation time may be assumed to be 1 second. The factor can be comprised between 0.5 and 2, preferably 0.1 and 5, more preferably 0.1 and 10. The factor may be chosen depending on the volume of the 3D room model. The reverberation time can be chosen as a compromise between simulation time and accuracy of the simulation. A low reverberation time may yield inaccurate results, thereby generating an inaccurate simulated impulse response, while a high reverberation time may drastically increase simulation time without increasing accuracy of the simulated impulse response.

[0028] The first frequency range and the second frequency range can also be named the first acoustic frequency range and the second acoustic frequency range, respectively.

[0029] Thus, the step of merging the wave based impulse response and the geometrical based impulse response may further comprise the step of calibrating the sound power of the at least one room sound source in the 3D model so that they match in both the wave based solver and in the geometrical acoustic solver.

[0030] The calibration can for example be done by estimating the sound pressure level radiated from the at least one room sound source at 1 meter in free field conditions. For example, in a frequency range near the transition frequency between the first acoustic frequency range and the second acoustic frequency range. The pressure level may be estimated at other distances as well, such as 0,5 meters, 2 meters and or 3 meters. The distance may be between 0,5 meters and the extend of the 3D room model in which the simulation is done.

[0031] The merging step may even further comprise that the boundary conditions are coherent for the wave based solver and the geometrical acoustic solver. If the boundary inputs are not aligned there is a risk that artefacts, such as a phase mismatch, and / or undesirable results may occur in the transition frequency region.

[0032] Preferably, the sound pressure level radiated from the at least one room sound source at 1 meter may be 94 dB sound pressure level (SPL). The step of calibrating the sound power of the at least one room sound source in the 3D room model so that the at least one room sound source match in both the wave-based solver and in the geometrical acoustic solver can be independently performed in a source correction step or calibration step comprised in the wave-based solver and / or in the geometrical acoustic solver. More preferably, the calibration step can be independently performed in the image source simulation and / or the ray tracing simulation, wherein the image source simulation and / or the ray tracing simulation are comprised in the geometrical acoustic solver.

[0033] Advantageously, the wave-based impulse response can be digitally filtered with a low- pass filter with a cutoff frequency substantially located at the transition frequency, thereby creating a filtered wave-based impulse response. This may be performed before performing the step of calibrating the sound power. Preferably, the low-pass filter is a fourth order Butterworth low-pass filter. The Butterworth filter is a type of signal processing filter designed to have a frequency response that is as flat as possible in the passband. The flatness of the frequency response in the passband may be appreciated in the context of signal processing, thus minimizing distortion and / or preserving impulse response shape. Preferably, for acoustic simulation as discussed herein, the flat passband response of a Butterworth filter can be desirable because it may help avoid coloration of the sound. Any uneven attenuation within the passband can alter the timber or tone of the audio signal, potentially leading to a less faithful reproduction of a sound. The order of the filter may be a compromise between having a sharp cut-off frequency while avoiding ripples in the time domain. The inventors found out that a fourth order may be a sufficient order to achieve desirable filtering.

[0034] Both the wave-based solver and the geometrical acoustic solver can output both monoaural and spatial ambisonics impulse responses for each of the listening points of the 3D room model, along with acoustic parameters derived from the impulse responses mentioned.

[0035] The merging step may digitally filter the geometrical acoustic impulse response with a high-pass filter with a cutoff frequency substantially located at the transition frequency, thereby creating a filtered geometrical acoustic impulse response. This may be performed after or before performing the step of calibrating the sound power. Preferably, the step of calibrating the sound power may be performed before digitally filtering the geometrical acoustic impulse response with a high-pass filter with a cutoff frequency substantially located at the transition frequency. Preferably, the high-pass filter is a fourth order Butterworth high-pass filter.

[0036] The merging step can sum both the filtered wave-based impulse response and the filtered geometrical acoustic impulse response, thereby creating a hybridized impulse response, wherein the hybridized impulse response combined the filtered wave-based impulse response and the filtered geometrical acoustic impulse response. The hybridized impulse response may be the impulse response as described herein, wherein the impulse response can combine the wave impulse response and the geometrical impulse response.

[0037] The merging step may be independently performed for all the channels comprised in the spatial ambisonics impulse responses. In other terms, the merging step may be independently performed for each of the number of room impulse responses, where each room impulse response describes the emitted room impulse as received at a corresponding one of the number of digital representations of room receivers.

[0038] In one embodiment, the device receiver array encompasses the device, which involves an interior problem.

[0039] In a further embodiment, the room receiver array processing involves additional mathematical steps to solve an exterior problem.

[0040] With the device related transfer function and the spatial room impulse response encoded into ambisonics, it is generally straightforward to combine them into the device specific room impulse response at an ambisonics order corresponding to the lowest of the N(device) or N(room). This also allows the device to be oriented / rotated freely relative to the room as the ambisonics encoding contains spatial positioning information.

[0041] As the computer implemented method as discussed herein is performed on a processor, computer or similar device for executing computer implemented methods, it should be understood that the steps are generally digitally executed unless described otherwise. Thus, reference to e.g. receivers, microphones, emitter, speakers, sound sources should be understood to be digital representation simulating or emulating the function of a corresponding physical part.

[0042] Similarly, the device model and the room model are also digital representation of physical or potentially physical elements and can be represented as meshes or other digital representations.

[0043] Although the signals emitted, received, simulated or generated also are used in a digital environment e.g. within a simulation, these may be processed through e.g. a digital-to-analog converter (DAC) in order to be played in a physical environment. For example the device specific room impulse response (DSRIR) may be convolved with an anechoic sound signal to generate physical audio experience of how a specific device receives sound in a specific room.

[0044] Detailed description

[0045] In one embodiment, generating the device related transfer function comprises obtaining a 3D device model representing the geometry of the device and the position of the at least first microphone on the 3D device model. The 3D device model may for example be a device mesh model representing the geometry of the device and the position of the at least first microphone on the device mesh model. Meshing can be understood as a common way of generating models in a digital environment such as computers. A mesh can be preferably a discretization of a geometry into small, simple shapes. Shapes can be triangles or quadrilaterals in 2D, and / or tetrahedra or hexahedra in 3D. A mesh density control can determine the appropriate mesh density, as too coarse a mesh may lead to inaccurate results, while an excessively fine mesh can increase computational cost and simulation time, or sometimes lead to a non-convergence of the simulation using the mesh. A mesh can be understood as a polygon mesh, which is a collection of vertices, edges and faces that defines the shape of a polyhedral object. Throughout this patent application, the term mesh elements can be faces, such as triangles, quadrilaterals, other simple convex polygons, or any other combinations thereof.

[0046] In one embodiment, at least one direction is at least two directions, at least three directions, at least four directions, at least five directions.

[0047] Obtaining the 3D device model can be done in different ways. A physical device can for example be scanned in order to obtain the 3D device model. A physical device can be scanned with different 3D scanning methods, such as handheld 3D scanners, desktop 3D scanners, photogrammetry and / or LiDAR scanning. Handheld 3D scanners can be devices such as the Artec Eva or Structure Sensor that can capture physical objects in 3D. LiDAR scanning are laser-based scanning for capturing large environments or detailed objects. The 3D device model may have been digitally obtained by modelling the 3D device model, e.g. using a CAD (Computer Aided Design) software. The 3D device model may also have been obtained by loading a file onto the computer, e.g. an STL file which is a common format for storing digital models.

[0048] In another embodiment, generating the device related transfer function comprises arrange a digital representation of a device receiver array comprising a plurality of digital representations of device receivers around the 3D device model, such as the device mesh model. The device receiver array is arranged such that the distance between any of the digital representations of the device receivers and the 3D device model is not below at predetermined distance.

[0049] In one embodiment the shape of the device receiver array comprising the plurality of digital representations of device receivers is a sphere. Alternatively, the shape may be an off-set shape where the digital representations of the device receiver are placed / arranged at a predetermined off-set distance from the device mesh model. The predetermined off-set distance may be the same as the predetermined distance discussed or described herein.

[0050] The predetermined distance may for example comprise the radius of the device receiver array and an additional distance in order to properly surround the 3D device model. This can for example be determined such that the distance from any device receiver on the array to any point on the 3D model of the device is preferably not below the predetermined distance.

[0051] In one embodiment, the predetermined distance is between 0,5 - 1,5 meters, preferably 0,8 - 1,2 meters or most preferred 1 meter. Such predetermined distance has shown to be a good choice for general room simulations, or preferably for generating and / or simulating a transfer function of a device. Choosing a distance of 1 meter is advantageous because, at this predetermined distance, the acoustic waves can be considered as plane waves. This approximation simplifies various aspects, such as maintaining consistent amplitude and phase and applying easier boundary conditions. As a result, the reduced complexity enables more efficient simulation of plane waves due to their simpler representation and propagation.

[0052] Determining the minimal order N can then be used to select the number of device receivers in the device receiver array which is determined by (N+1)2. This can for example be multiplied by a factor, e.g. 1 ,5 or 2,0 in order to obtain higher fidelity but also at the cost of increased simulation time.

[0053] When applying wave-based solvers to generate an at least first device related transfer function for a device comprising at least one microphone, it is generally the number of sound sources that determines the time and resources required to determine acoustic simulation. Thus, if the at least one microphone was set to function as a microphone, all the receivers in the device array would have to function as sound sources and emit an impulse signal. However, each of these would have to be solved individually and the time and resources used for solving would increase significantly based on the number of sound sources in the receiver array. Instead, for determining the device related transfer function as discussed here, a first impulse signal may advantageously be emitted using the first microphone as a sound source, using the law of reciprocity. This would drastically save the acoustic simulation time, since only one simulation per microphone would characterize all the microphones of the device, instead of having one simulation per sound sources in the receiver array.

[0054] In one embodiment where the 3D device model is a mesh model, a first closest mesh element, which is closest to the at least first microphone on the 3D device model can be determined. The first closest mesh element may in one embodiment be used a sound source for emitting the first impulse signal. By having and / or identifying a first closest mesh element closest to the at least first microphone on the 3D device model, a sound source for emitting the first impulse signal can be set up.

[0055] In a preferred embodiment, the first closest mesh element can be automatically determined by the computer-implemented method. The computer-implemented method can identify the position of all mesh elements of the mesh model. The position of the at least first microphone can be programmed by the user when designing the 3D device model. Thereby, the computer-implemented method can calculate a mesh distance between the position of the at least first microphone and a plurality of mesh elements of the mesh model. The mesh element being located at the shortest distance from the position of the at least first microphone can be identified by the computer-implemented method as the first closest mesh element.

[0056] Based on the emitted first impulse response, a plurality of first device impulse responses may thus be determined using a wave-based solver, where each first device impulse response may describe the impulse response of the first impulse signal received at the respective device receiver. The respective device receiver is at least one of the device receivers from the device receiver array as described herein.

[0057] The first device related transfer function of the device for the first microphone may thus be generated by combining the plurality of first device impulse responses.

[0058] The first impulse signal should preferably be a perfect impulse and have a flat spectrum. However, this may not be possible and in order to increase the fidelity of the generated device related transfer function, the first device impulse responses should preferably be source-corrected by a reference signal.

[0059] This can in one embodiment be done by arranging a digital representation of a first source correction microphone located at a first source distance from the at least first microphone or the first closest mesh element, wherein the first source distance is smaller than the predetermined distance.

[0060] A first source correction signal may thus be determined using a wave-based solver, where the first source correction signal describes the first impulse signal as received at the first source correction microphone. A plurality of first source corrected device impulse responses can then be determined by source correcting each of the plurality of first device impulse responses using the first source correction signal, and the first device related transfer function of the device for the first microphone can then be determined by combining the plurality of first source corrected device impulse responses.

[0061] In one embodiment, generating the device related transfer function comprises determining an energy content for at least one frequency of the first device related transfer function. As will be discussed, this can for example be used to determine the ambisonics order N which can be used when generating the spatial room impulse response SRIR.

[0062] In one embodiment, determining the energy content for the at least one frequency of the first device related transfer function comprises determining different ambisonics orders, thereby identifying the different levels of energy content.

[0063] In one embodiment, determining the energy content for at least one frequency of the first device related transfer function comprises determining the ambisonics order N for the energy content of the at least one frequency.

[0064] In a further embodiment, the ambisonics order N is determined for multiple frequencies, where the energy content for each frequency is determined.

[0065] In yet a further embodiment, determining the ambisonics order N for the energy content is based on determining the energy content as a sum of the ambisonics coefficients for each order N and then normalized to unity for each frequency.

[0066] In one embodiment, the energy content is determined for a range of frequencies, such as from 0 to 20 kHz, such as 0 to 10 kHz, such as 10 to 20 kHz, such as 0 to 9 kHz, such as 0 to 8 kHz, such as 0 to 7 kHz, such as 0 to 6 kHz, such as 0 to 5 kHz, such as 0 to 4 kHz, such as 0 to 3 kHz, such as 0 to 2 kHz, such as 0 to 1 kHz. Preferably, the range of frequencies may be comprised in the audible spectrum. The maximum frequency of the range may dictate the ambisonics order N to be chosen. Generally, a higher maximum frequency requires a larger ambisonics order N. In one embodiment, the device comprises a plurality of microphones, such as a second, third, fourth, fifth microphone. In such a case the computer implemented method as discussed herein is repeated for each microphone. In other words, each of the plurality of microphones can be treated as the first microphone such that a plurality of device related transfer functions, such as a second, third, fourth, fifth device related transfer function of the device is generated for each of the microphones.

[0067] In one embodiment, generating an at least first device related transfer function (DRTF) comprises

[0068] - obtain a 3D box model comprising high acoustic absorption surfaces, or a 3D box model with a predefined size such that the first impulse signal received from the sound source to each of the plurality of digital representations of device receivers do not comprise reflections caused by the surfaces of the 3D box model,

[0069] - arrange the device receiver array and the device mesh model in the 3D box model.

[0070] By obtaining a 3D box model comprising high acoustic absorption surfaces, the first impulse signal emitted from the first closest mesh element is not reflected by the surfaces of the 3D box model, thereby not being received by the digital representation of the device receiver array comprising a plurality of digital representations of device receivers. Preferably, the plurality of digital representations of device receivers may receive the incoming impulse signal and preferably not the reflections caused by any surfaces, obstacles or geometries external to the digital representation of the device receiver array. In another embodiment, the 3D box model has a predefined size such that the first impulse signal received from the sound source to each of the plurality of digital representations of device receivers do not comprise reflections caused by the surfaces of the 3D box model. The predefined size may be estimated such as the surfaces are far away from the plurality of digital representations of device receivers. Having a predefined size too high may be costly in terms of computation cost and time, thereby the predefined size should be estimated and / or calculated such as the simulation of the impulse signal propagation may get stopped before the impulse signal reaches the surfaces of the 3D box model or substantially reaches the surfaces of the 3D box model. As an alternative, the 3D box model can be a 3D spherical model. Preferably, the 3D spherical model can consist of a sphere.

[0071] In one embodiment, generating the spatial room impulse response comprises obtain a 3D room model representing the geometry of the room and at least one acoustic characteristic or at least one acoustic property. As discussed above, obtaining a 3D model, such as the 3D room model, can be done in different ways.

[0072] Furthermore, the 3D room model may comprise at least one acoustic characteristic which for example can be acoustic absorption values for different materials included in the 3D room model and can be different for different elements in the model such as elements representing windows, carpet, furniture etc. The at least one acoustic characteristic can be a complex surface impedance. The 3D room model may comprise at least one boundary. The at least one boundary may comprise at least one acoustic characteristic or at least one acoustic property.

[0073] The at least one boundary of the 3D room model may refer to the surfaces or edges that define the limits or extents of the 3D room model. The at least one boundary can encompass the physical barriers that can enclose the space or volume of the 3D room model. The at least one boundary can be a wall, a floor, a ceiling, a door, a window, a corner, an edge, or any combinations thereof.

[0074] In one embodiment, at least one digital representation of at least one room sound source is arranged in the 3D room model. The at least one room sound source may for example be an omnidirectional sound source or a directional sound source. A directional sound source can be an audio emitter that emits sound in a specific direction rather than uniformly in all directions. A directional sound source may be characterized by its ability to focus acoustic or sound waves along a particular direction or path, which can result in higher sound intensity in the particular direction, and lower intensity outside this particular direction. The directional pattern of a directional sound source can be narrow or wide. The directional pattern can comprise cardioid, hypercardioid, supercardioid, bidirectional, any arbitrary directivity pattern, or any combinations thereof. Any arbitrary directivity pattern can be other directional patterns that may not be a combinations of the directional patterns as described herein. The directional sound source may use beamforming, which can manipulate the phase and amplitude of the sound or acoustic waves emitted from at least one speaker to create a focused beam of sound. The directional sound source can be modelled using spherical harmonics functions.

[0075] In a further embodiment, generating the spatial room impulse response may further comprise arrange a digital representation of a room receiver array comprising a number of digital representations of room receivers, wherein the room receiver array is centred on an at least one listening point in the 3D room model.

[0076] The number of digital representations of room receivers may in one embodiment be determined based on the energy content for the at least one frequency of the first device related transfer function. Preferably, the energy content may generate the information on the ambisonics order to be used, thereby generating the information of the number of receivers to be used in the digital representation of room receivers.

[0077] In one embodiment, where the number of digital representations of room receivers are determined based on the energy content for at least one frequency of the device related transfer function (DRTF) further comprises determining the number of digital representation of room receivers based on the ambisonics order N, where the number of digital representations of room receivers can be (N+1)2, 1,5*(N+1)2or 2*(N+1)2. Having more digital representations of room receivers may give a better accuracy, at the cost of a larger amount of data generated.

[0078] In a further embodiment, generating the spatial room impulse response may comprise digitally emit a room impulse signal from the at least one audio sound source. A number of room impulse responses using at least a wave-based solver for at least one wave-based frequency may be determined, where each room impulse response describes the emitted room impulse signal as received at a corresponding one of the number of digital representations of room receivers. A spatial room impulse response may be based on the number of room impulse responses.

[0079] In a further embodiment, generating the spatial room impulse response may comprise determine a second number of room impulse responses using at least a geometrical acoustic solver for at least one geometrical acoustic frequency.

[0080] The number of impulse responses generated using the wave-based solver and the second number of room impulse responses generated using the geometrical acoustic solver may in one embodiment be merged to generate a number of merged room impulse responses.

[0081] For example, in a further embodiment, the number of room impulse responses generated using the wave-based solver may be generated in low frequencies of an acoustic spectrum and the second number of impulse responses may be generated using the geometrical acoustic solver in high frequencies of the acoustic spectrum.

[0082] The acoustic spectrum can for example be between 0 and 20 kHz, such as between 0 and 15 kHz, such as between 0 and 12 kHz, such as between 0 and 10 kHz, such as between 0 and 8 kHz, such as between 0 and 6 kHz, such as between 20 Hz and 20 kHz, such as between 20 Hz and 15 kHz, such as between 20 Hz and 12 kHz, such as between 20 Hz and 10 kHz, such as between 20 Hz and 8 kHz, such as between 20 Hz and 6 kHz. Preferably, the acoustic spectrum as defined herein may preferably be the acoustic spectrum as being heard by humans. Some frequency ranges of the acoustic spectrum may preferably be more useful for such acoustic applications such as human speech, where most of the human speech frequencies can generally be comprised between 100 to 17 kHz, where fundamentals and harmonics of human voice can be comprised. Male voice may cover a frequency range of 100 Hz to 8 kHz, while female voice can cover a frequency range of 350 Hz up to 17 kHz.

[0083] In one embodiment, the low frequencies of the acoustic spectrum are comprised between 0 and 20 kHz, such as between 0 and 15 kHz, such as between 0 and 12 kHz, such as between 0 and 10 kHz, such as between 0 and 8 kHz, such as between 0 and 6 kHz, such as between 20 Hz and 20 kHz, such as between 20 Hz and 15 kHz, such as between 20 Hz and 12 kHz, such as between 20 Hz and 10 kHz, such as between 20 Hz and 8 kHz, such as between 20 Hz and 6 kHz, such as between 20 Hz and 5 kHz, such as between 20 Hz and 4 kHz, such as between 20 Hz and 3 kHz, such as between 20 Hz and 2 kHz, such as between 20 Hz and 1,5 kHz, such as between 20 Hz and 1 kHz.

[0084] In one embodiment, the high frequencies of the acoustic spectrum are comprised between 1 kHz and 20 kHz, such as between 1,5 kHz and 20 kHz, such as between 2 kHz and 20 kHz, such as between 3 kHz and 20 kHz, such as between 4 kHz and 20 kHz, such as between 5 kHz and 20 kHz, such as between 6 kHz and 20 kHz, such as between 8 kHz and 20 kHz, such as between 10 kHz and 20 kHz, such as between 12 kHz and 20 kHz, such as between 1 kHz and 15 kHz, such as between 1,5 kHz and 15 kHz, such as between 2 kHz and 15 kHz, such as between 3 kHz and 15 kHz, such as between 4 kHz and 15 kHz, such as between 5 kHz and 15 kHz, such as between 6 kHz and 15 kHz, such as between 8 kHz and 15 kHz, such as between 10 kHz and 15 kHz, such as between 12 kHz and 15 kHz, such as between 1 kHz and 12 kHz, such as between 1,5 kHz and 12 kHz, such as between 2 kHz and 12 kHz, such as between 3 kHz and 12 kHz, such as between 4 kHz and 12 kHz, such as between 5 kHz and 12 kHz, such as between 6 kHz and 12 kHz, such as between 8 kHz and 12 kHz, such as between 10 kHz and 12 kHz.

[0085] As discussed herein, using ambisonics to encode and decode acoustic signals may have many advantages and in particular when treating high fidelity acoustic data and signals it can provide a flexible way to communicate with many different applications and uses. For example, as the first (and possible further) device related transfer functions and spatial impulse response are generated separately but linked by the energy content, which for example can be used to determine the ambisonics order N, it is possible to e.g. rotate the device relative to the room. A new device related transfer function can be generated for a new device and combined with an already generated spatial room impulse response having corresponding ambisonics order N (or higher), or vice versa, a new spatial room impulse response can be generated for a new room.

[0086] In one embodiment, the generated device specific room impulse response may thus be encoded and decoded using ambisonics. However, generating the device specific room impulse response (DSRIR) by combining the device related transfer function and the spatial room impulse response may be avoided and the at least first device related transfer function (DRTF) and the spatial impulse response (SRIR) may be processed individually or further processed in a matrix. For instance, the DSRIR can be used to analyze the sound field in the room or around the device, or to generate spatial sound field visualizations.

[0087] In one embodiment of the above aspect, such advantages become apparent in a computer-implemented method for generating a device specific room impulse response (DSRIR) describing an acoustic characteristic of a device and a room as received by the device, wherein the device comprises at least a first microphone, the method comprises

[0088] - generate an at least first device related transfer function (DRTF), wherein the at least first device related transfer function describes the acoustic characteristic of the device as received by the at least first microphone, wherein generating the first device related transfer function further comprises o obtain a device mesh model representing the geometry of the device and the position of the at least first microphone on the device mesh model, o arrange a digital representation of a device receiver array comprising a plurality of digital representations of device receivers around the device mesh model, such that the distance between any of the digital representations of the device receivers and the device mesh model is not below at predetermined distance, o determine on the device mesh model a first closest mesh element, which is closest to the at least first microphone, o arrange a digital representation of a first source correction microphone located at a first source distance from the first closest mesh element, wherein the first source distance is smaller than the predetermined distance, o digitally emit a first impulse signal using the first closest mesh element as a sound source, o determine a first source correction signal using a wave-based solver, where the first source correction signal describes the first impulse signal as received at the first source correction microphone, o determine a plurality of first device impulse responses using a wavebased solver, and where each first device impulse response describes the impulse response of the first impulse signal received at the respective device receiver, o determine a plurality of first source corrected device impulse responses by source correcting each of the plurality of first device impulse responses using the first source correction signal, o generate the first device related transfer function of the device for the first microphone by combining the plurality of first source corrected device impulse responses, o determine an energy content for at least one frequency of the first device related transfer function,

[0089] - generate a spatial room impulse response (SRIR), wherein the spatial room impulse response describes the acoustic characteristic of the room from at least one room sound source in the room and received at an at least one listening point in the room from at least one direction, wherein generating the spatial room impulse response further comprises o obtain a 3D room model representing the geometry of the room and at least one acoustic characteristic, o arrange at least one digital representation of at least one room sound source in the 3D room model, o arrange a digital representation of a room receiver array comprising a number of digital representations of room receivers, wherein the room receiver array is centred on an at least one listening point in the 3D room model and where the number of digital representations of room receivers are determined based on the energy content for the at least one frequency of the first device related transfer function, o digitally emit a room impulse signal from the at least one audio sound source, o determine a number of room impulse responses using at least a wavebased solver for at least one wave-based frequency, where each room impulse response describes the emitted room impulse as received at a corresponding one of the number of digital representations of room receivers, o generate a spatial room impulse response based on the number of room impulse responses,

[0090] - generating the device specific room impulse response (DSRIR) by combining the device related transfer function and the spatial room impulse response.

[0091] The device specific room impulse response can be convolved with an anechoic sound to generate a device specific room sound. This can provide an audio rendering of how the sound may be captured by the device in the room. Using high-order ambisonics can allow for a proper ambisonics rendering, generating high-fidelity, device specific room sound. The device specific room impulse response can be evaluated in order to account for many different scenarios of device positions and orientations in a plurality of 3D room model. This enables the user to evaluate any devices performance in any rooms and in any orientations, based on one room simulation and one device simulation, which can be defined as the SRIR generation and the DRTF generation, respectively.

[0092] In another aspect, a computer-implemented method for recovering a device-related transfer function (DRTF) for a primary device is disclosed. The computer-implemented method for recovering a DRTF for a primary device, wherein the DRTF for the primary device comprises at least one missing amplitude level, may comprise the steps of: obtaining a plurality of amplitude levels of the DRTF for the primary device, as a function of a plurality of elevation angles and a plurality of azimuth angles, wherein the DRTF for the primary device comprises at least one missing amplitude level; obtaining a matrix comprising the spherical harmonics basis functions; applying a truncated singular value decomposition method to the matrix comprising the spherical harmonics basis functions, wherein the truncated singular value decomposition method comprises the steps of: generating singular values based on the matrix comprising the spherical harmonics basis functions according to a singular value decomposition (SVD); selecting a top selection of singular values and a corresponding set of singular vectors, wherein the top selection of singular values and the corresponding set of singular vectors are selected based on a predetermined threshold; generating a secondary matrix comprising the spherical harmonics basis functions based on the top selection of singular values and the corresponding set of singular vectors, according to the singular value decomposition (SVD); multiplying a matrix corresponding to the plurality of amplitude levels of the DRTF for the primary device with the inverse of the secondary matrix, thereby generating a plurality of ambisonics coefficients; generating a plurality of recovered amplitude levels of the DRTF for the primary device based on the plurality of ambisonics coefficients, wherein the plurality of recovered amplitude levels of the DRTF for the primary device comprises an image of the amplitude levels of the DRTF for the primary device and a reconstruction of the at least one missing amplitude level.

[0093] In one embodiment, the amplitude levels of the DRTF for the primary device are represented as a function of elevation and azimuth angles.

[0094] In another embodiment, the amplitude levels of the DRTF for the primary device are for a DRTF frequency. The DRTF frequency may be comprised between 20 Hz and 20 kHz. Preferably, the DRTF frequency may be comprised between 100 Hz and 10 kHz.

[0095] In a preferred embodiment, the amplitude levels are complex-valued, or the amplitude levels are represented in the form of complex numbers.

[0096] The DRTF for the primary device as disclosed herein can be generated according to computer-implemented method for generating a device related transfer function as described herein.

[0097] Description of the drawings

[0098] In the following embodiments and examples will be described in greater detail with reference to the accompanying drawings:

[0099] Fig. 1 illustrates schematically an embodiment of a method of generating a device specific room impulse response as disclosed here.

[0100] Fig. 2A-B shows amplitude levels of a DRTF for a primary device, as a function of elevation and azimuth angles at 1000 Hz with missing data, and amplitude levels of the same DRTF reconstructed from the ambisonics representation of the primary device, as a function of elevation and azimuth angles at 1000 Hz, where the missing data is recovered using amplitude levels of a DRTF reconstruction or recovering method as described herein.

[0101] Detailed description of the drawings A computer implemented method for generating a device specific room impulse response 100 as discussed herein is shown schematically in Fig. 1.

[0102] The method can be considered to be formed of two sub-methods, one shows a method for generating a device related transfer function (DRTF) 101, and the other shows a method for generating a spatial room impulse response (SRIR) 102.

[0103] For generating a device related transfer function a 3D model of the device 110 is provided as an input to the method. The 3D model of the device 110 comprises three microphones, a first microphone 111, a second microphone 112 and a third microphone 113.

[0104] When entering the method the 3D model of the device 110 is meshed if not already meshed as a model and placed in a simulation tool which applies a wave-based solver. As should be understood herein a wave-based solver applies wave-based methods that may apply numerical techniques to directly solve governing partial differential equations that describe wave motion in a virtual domain, such as representing an air volume. This can for example be a wave equation in the time domain or the Helmholtz equation in the frequency domain. The concept of the different wave-based methods is thus to divide the virtual domain of interest into small subdomains (discretization) and solve algebraic equations on each subdomain. Accordingly, wave-based methods used in wave-based solvers as disclosed herein may be understood to be methods that solve the partial differential equations using discretization techniques. The Treble simulation tool can for example be used for performing such wave-based simulation using a wave-based solver.

[0105] A device array of receivers 114 are arranged around the 3D model of the device 110 comprising a plurality of device receivers 114’ (for simplicity in the drawing not all are indicated with a reference number). In the current case, the array of device receivers 114 forms a spherical pattern around the 3D model of the device 110. A spherical pattern is often preferred as this facilitates using spherical harmonics to encode the signals into ambisonics. Preferably, the spherical pattern may be a Lebedev grid.

[0106] The radius of the array of receivers is determined such that the distance from any receiver on the array to any point on the 3D model of the device is not below a predetermined distance, which in the current example is set to 1 meter, which has shown to be a good choice for general room simulations to ensure far-field sound radiation conditions. The ambisonics order N can be determined using the formula N > 2*pi*f*R(device) / c, where ‘R(device)’ is the largest distance from the center of the array of receivers to any point on the 3D model of the device, ‘f’ is the considered frequency and ‘c’ is the sound speed (typically 344 m / s). Determining the minimal order N can then be used to select the number of receivers necessary in the array which is determined by (N+1)2. This can for example be multiplied by a factor, e.g. 1 ,5 or 2,0 in order to obtain higher fidelity but also at the cost of increased data generation and simulation time.

[0107] However, other patterns could be used where an additional step of e.g. transposing the shape onto a sphere could be used for spherical harmonics. Such a pattern could for example be an offset from the surface of the 3D model of the device with a predetermined distance, for example 1 meter as discussed.

[0108] The microphones 111 , 112 and 113 on the 3D model of the device 110 are then set to function as sound sources. Although not shown, this can for example be done by determining the closest mesh element on the meshed model for each microphone and use the closest mesh elements for virtually emitting an impulse signal as will be described. A remeshing step could be performed to re-mesh the closest mesh element to a specific size or shape so as to resemble the dimensions of the microphone as close as possible. Preferably, a preliminary meshing step is performed such as the location of the microphone are specified as an input to a meshing tool being used, thereby generating appropriate mesh elements located at the microphone locations.

[0109] Switching the function of the microphone on the device to function as a sound source such as a speaker or other sound emitter increases the speed of the subsequent wavebased solver considerably as a number of sound sources affect the processing speed considerably. Thus, in case the microphones on the device are set to function as microphones, each receiver 114’ in the array of receivers 114 would have to function as sound sources which will increase the processing time greatly since the number of receivers in the array generally greatly exceeds the number of microphones on the device. Thus, being able to switch the functions around for the wave-based simulation reduces the time significantly and is one large advantage of using a computer implemented simulation tool such as the Treble software.

[0110] The method will then perform, one at a time, a wave-based simulation for each microphone 111 , 112, 113 where the microphone, or in this case the corresponding closest mesh element, emits an impulse and the signal received at each receiver 114’ in the array is recorded.

[0111] Ideally, the impulse will have a flat spectrum, however, this is typically not possible. Thus, a source correction of the signal received at each receiver is performed [2], The reference signal used for source correction is recorded using a source correction receiver 121, 122, 123 placed very close to the microphone functioning as a speaker, for example 1 mm in front of the microphone or the mesh used as an emitter. Thus, in the current embodiment three source correction receivers 121 , 122 and 123 are placed 1 mm in front of the microphones 111, 112 and 113 or the respective closest meshes.

[0112] The source-corrected signals received at each array receiver for the impulse from one of the microphones on the device forms a transfer function describing the specific microphone. Thus, in the current case, as there are three microphones on the device, three distinct transfer functions, also described as the first 145, second 146 and third 147 device related transfer function herein, are generated and together they form the general device related transfer function, which for example can be stored as a three dimensional matrix.

[0113] In addition to the first, second and third device related transfer functions, the method for generating a device related transfer function also generates an energy map after encoding to ambisonics. The energy map 140 is generated where the energy at different frequencies (Hz) is used to determine the ambisonics order (n) and can be used in the room simulation (when generating the spatial room impulse response as will be discussed) to provide an efficient simulation and an ambisonics encoding and decoding that allows for the device to be freely rotated as will be discussed in the following. For example, in the current case, a frequency to ambisonics curve 141 is determined indicating the level where the energy content is 95% at the respective frequencies along the x-axis and is used to determine the ambisonics order (n) on the y-axis. The method for generating a spatial room impulse response 102 uses a 3D room model 150 which represents the geometry of a room as input. In the room model, a sound source 152 and a listening point 151 are arranged. The 3D room model may also include geometries of furniture, such as tables and chairs, door openings and / or monitors. It may also include acoustic characteristics of the different geometries and materials, such as windows, carpets, different wall materials, etc.

[0114] The spatial room impulse response 170 is subsequently determined based on a wavebased spatial room impulse response 171 for the low to mid frequency range and a geometrical acoustic spatial room impulse response 172 for the mid to high frequency range of the audible spectrum.

[0115] The spatial impulse responses 171 and 172 embeds spatio-temporal information regarding direction of arrival of incoming acoustic waves at the receiver position. Typically, a spatial impulse response comprises a plurality of single channel impulse responses, where each impulse response records the sound from a specific direction or angle at the same listening point.

[0116] The wave-based spatial impulse response 171 can be constructed in a simulation by emitting an impulse signal from the sound source 152 and record a number of room impulse response at multiple room receivers 160’ around the listening point 151 in a room receiver array 160. The room receiver array comprising the room receivers 160’ (for simplicity in the drawing not all are indicated with a reference number) arranged in a spherical array shape around the listening position 151. The receivers may either be omnidirectional or with cardioid directivity pattern to optimize the operating frequency range of the array.

[0117] The number of receiver(s) and the size used in the room receiver array is initially determined by the ambisonics order N derived from energy map 140 where the ambisonics curve shows the order N for a desired frequency range for which the wavebased spatial impulse response 171 is determined. Knowing the order N, the number of receivers 160’ can be decided as discussed above by (N+1)2, which for example can be multiplied by a factor, e.g. 1 ,5 or 2,0 for higher fidelity. Further, the radius of the array R(array) can be determined using the formula discussed above N > 2*pi*f*R(array) / c. For a given ambisonics order N and radius R(array), this imply that the maximum frequency is constrained due to spatial aliasing as f<N*c / (2*pi*R(array)). R(array) must therefore be chosen in accordance with the maximum frequency of interest. Once the impulse response has been recorded for all the receivers in the array, the spatial impulse response can be encoded into ambisonics.

[0118] The geometrical acoustic spatial impulse response 172 can be determined by analyzing the incoming directions of all image sources and rays at the listening point 151 using commonly-applied image source and ray tracing techniques. A geometrical acoustics spatial impulse response can then be straightforwardly generated and encoded into ambisonics.

[0119] The wave-based spatial room impulse response 171 and the geometrical acoustic spatial room impulse response 172 can subsequently be combined or hybridized to the combined spatial room impulse response 170. In some cases, either the wave-based spatial room impulse response or the geometrical acoustic spatial room impulse response can be further used independently.

[0120] Thus, the device specific room impulse response 180 is provided by the first 145, second 146 and third 147 device related transfer function herein are generated and together they form the general device related transfer function, which for example can be stored as a matrix, and the combined spatial room impulse response 170.

[0121] Fig. 2A-B shows amplitude levels of a DRTF for a primary device, as a function of elevation and azimuth angles at 1000 Hz with missing data, and amplitude levels of the same DRTF reconstructed from the ambisonics representation of the primary device, as a function of elevation and azimuth angles at 1000 Hz, where the missing data is recovered using amplitude levels of a DRTF reconstruction or recovering method as described herein.

[0122] Fig. 2A shows amplitude levels of a DRTF for a primary device, as a function of elevation and azimuth angles at 1000 Hz with missing amplitude levels or missing data. The missing amplitude levels are substantially located at elevation angles below -45 degrees, for all azimuth angles. This can correspond to a DRTF measurement or DRTF simulation case, where the device is arranged on a table, or a support, preferably horizontal, which would compromise or challenge the measurement or simulation of DRTF for elevation angles which can correspond to positions located below the table or support, where the primary device is arranged on. Other cases could be similar, for instance where the primary device is closed to a wall, wherein other angles such as the azimuth angles would be missing or potentially corrupted for positions located behind the wall. The amplitude levels of a DRTF for a primary device as shown in Fig. 2A may represent a challenge if a spherical harmonics / ambisonics decomposition is performed. The missing amplitude levels would render the spherical harmonics / ambisonics decomposition non-optimal. This challenge can be for instance solved with a DRTF reconstruction or recovering method where the missing amplitude levels are filled with random amplitude levels, preferably low amplitude levels, more preferably lower amplitude levels than the amplitude levels as measured or simulated in the DRTF for the primary device. This method can be cumbersome, case-dependent and may not demonstrate sufficiently accurate results.

[0123] In one embodiment, the missing amplitude levels of the DRTF for the primary device are reconstructed or recovered using a truncated singular value decomposition approach or method during or prior to the ambisonics decomposition procedure.

[0124] The truncated singular value decomposition approach is based on singular value decomposition. Singular Value Decomposition (SVD) can be used for dimensionality reduction, data compression, noise reduction or solving linear systems. In the case of the amplitude levels of a DRTF, the graph as shown in Fig. 2A can be assimilated to a matrix of amplitude levels associated with each direction, where each amplitude level is an element of the matrix, and where the element of the matrix can be complex-valued. A complex-valued element can comprise phase and / or amplitude information. The ambisonics decomposition of the said matrix, up to order N, can involve finding the (N+1)2coefficients associated with each spherical harmonics contribution. The coefficients may be directly obtained as a multiplication between the matrix of the energy levels and the inverse of a matrix comprising the value of the spherical harmonics basis functions, evaluated for each direction of the receivers in the device array. The inverse of the matrix comprising the value of the spherical harmonics basis functions can be processed by SVD prior to the multiplication stage for regularization purposes. By choosing the appropriate number of singular values based on their relative amplitude, a proper conditioning of the basis function matrix is ensured, allowing an accurate estimation of the ambisonics coefficients, and thus keeping a good approximation of the amplitude levels of the DRTF, after reconstruction or recovering.

[0125] This method as described in the previous paragraph can be described as a truncated singular value decomposition (TSVD), where a truncated singular value decomposition can be a method where a limited number of singular values are used to reconstruct or recover a matrix, preferably in the context of dealing with missing data, such as missing amplitude levels of a DRTF of a primary device. As described herein, the truncated singular value decomposition method involves using a top selection of singular values and their corresponding singular vectors, instead of using all the singular values and vectors. This truncation may help in capturing the most significant features of the matrix, thereby the most significant features of the amplitude levels of a DRTF with missing amplitude levels as shown in Fig. 2A. By applying TSVD, the missing amplitude levels can be reconstructed, emulated or recovered without impacting the non-missing amplitude levels, since the TSVD method may fill the missing amplitude levels with plausible amplitude levels derived from the amplitude levels measured or simulated, and reported in the original DRTF as shown in Fig. 2A.

[0126] The TSVD method is used in order to achieve the amplitude levels of a DRTF for a primary device, as a function of elevation and azimuth angles, as shown in Fig. 2B for a frequency of 1 kHz. The amplitude levels of the DRTF for the primary device as shown in Fig. 2B can be efficiently used to perform a spherical harmonics / ambisonics decomposition, with smooth and efficient results.

[0127] Reference list:

[0128] [1] F. Pind, “Wave-based Virtual Acoustics”, 2020, Technical University of Denmark - https: / / orbit.dtu.dk / en / publications / wave-based-virtual-acoustics.

[0129] [2] S. Sakamoto et al., “Calculation of impulse responses and acoustic parameters in a hall by the finite-difference time-domain method”, Acoust. Sci. & Tech. 29, 4 (2008). Embodiment List

[0130] Disclosed herein are the following embodiments

[0131] 1. A computer-implemented method for generating a device specific room impulse response (DSRIR) describing an acoustic characteristic of a device and a room as received by the device, wherein the device comprises at least a first microphone, the method comprises

[0132] - generate an at least first device related transfer function (DRTF), wherein the at least first device related transfer function describes the acoustic characteristic of the device as received by the at least first microphone

[0133] - generate a spatial room impulse response (SRIR), wherein the spatial room impulse response describes the acoustic characteristic of the room from at least one room sound source in the room and received at an at least one listening point in the room from at least one direction.

[0134] 2. The computer-implemented method according to item 1, wherein the at least one direction is at least two directions, at least three directions, at least four directions, at least five directions.

[0135] 3. The computer-implemented method according to item 1, wherein the computer- implemented method further comprises

[0136] - generating the device specific room impulse response (DSRIR) by combining the device related transfer function and the spatial room impulse response.

[0137] 4. The computer implemented method according to item 1, wherein generating the device related transfer function comprises

[0138] - obtain a device mesh model representing the geometry of the device and the position of the at least first microphone on the device mesh model,

[0139] - arrange a digital representation of a device receiver array comprising a plurality of digital representations of receivers around the device mesh model, such that the distance between any of the digital representations of the receivers and the device mesh model is not below at predetermined distance,

[0140] - determine on the device mesh model a first closest mesh element, which is closest to the at least first microphone, - arrange a digital representation of a first source correction microphone located at a first source distance from the first closest mesh element, wherein the first source distance is smaller than the predetermined distance,

[0141] - digitally emit a first impulse signal using the first closest mesh element as a sound source,

[0142] - determine a first source correction signal using a wave-based solver, where the first source correction signal describes the first impulse signal as received at the first source correction microphone,

[0143] - determine a plurality of first device impulse responses using a wave-based solver, and where each first device impulse response describes the impulse response of the first impulse signal received at the respective receiver,

[0144] - determine a plurality of first source corrected device impulse responses by source correcting each of the plurality of first device impulse responses using the first source correction signal,

[0145] - generate the first device related transfer function of the device for the first microphone by combining the plurality of first source corrected device impulse responses. The computer implemented method according to any one of the preceding items, wherein generating the device related transfer function comprises determine an energy content for at least one frequency of the first device related transfer function. The computer implemented method according to any one of the preceding items, wherein generating the spatial room impulse response comprises,

[0146] - obtain a 3D room model representing the geometry of the room and at least one acoustic characteristic,

[0147] - arrange at least one digital representation of at least one room sound source in the 3D room model,

[0148] - arrange a digital representation of a room receiver array comprising a number of digital representations of receivers, wherein the room receiver array is centred on an at least one listening point in the 3D room model and where the number of digital representations of receivers are determined based on the energy content for the at least one frequency of the first device related transfer function, - digitally emit a room impulse signal from the at least one audio sound source,

[0149] - determine a number of room impulse responses using at least a wave-based solver for at least one frequency, where each room impulse response describes the emitted room impulse as received at a corresponding one of the number of digital representations of receivers,

[0150] - generate a spatial room impulse response based on the number of room impulse responses. The computer-implemented method according to any of the preceding items, wherein the predetermined distance is between 0,5 - 1,5 meters, preferably 0,8 - 1 ,2 meters or most preferred 1 meter. The computer-implemented method according to any one of the preceding items, wherein the device comprises a plurality of microphones, such as a second, third, fourth, fifth microphone, wherein each of the plurality of microphones are treated as the first microphone such that a plurality of device related transfer functions, such as a second, third, fourth, fifth device related transfer function of the device is generated for each of the microphones. The computer-implemented method according to any one of the preceding items, wherein determining the energy content for the at least one frequency of the first device related transfer function comprises determining different ambisonics orders, thereby identifying the different levels of energy content. The computer-implemented method according to any one of the preceding items, wherein the energy content is determined for a range of frequencies, such as from 0 to 20 kHz, such as 0 to 10 kHz, such as 10 to 20 kHz, such as 0 to 9 kHz, such as 0 to 8 kHz, such as 0 to 7 kHz, such as 0 to 6 kHz, such as 0 to 5 kHz, such as 0 to 4 kHz, such as 0 to 3 kHz, such as 0 to 2 kHz, such as 0 to 1 kHz. The computer-implemented method according to any one of the preceding items, wherein generating an at least first device related transfer function (DRTF) comprises - obtain a 3D box model comprising high acoustic absorption surfaces, or a 3D box model with a predefined size such that the first impulse signal received from the sound source to each of the plurality of digital representations of receivers do not comprise reflections caused by the surfaces of the 3D box model,

[0151] - arrange the device receiver array and the device mesh model in the 3D box model. The computer-implemented method according to item 11 , wherein the 3D box model is a 3D spherical model. The computer-implemented method according to any one of the preceding items, wherein generating an at least first device related transfer function (DRTF) comprises arranging the device receiver array comprising the plurality of digital representations of receiver as a sphere or as an off-set shape where the digital representations of the receiver are placed / arranged at a predetermined off-set distance from the device mesh model. The computer-implemented method according to any one of the preceding items, wherein the method further comprises determine a second number of room impulse responses using at least a geometrical acoustic solver for at least one geometrical acoustic frequency. The computer-implemented method according to any one of the preceding items, wherein the number of impulse responses generated using the wavebased solver and the second number of room impulse responses generated using the geometrical acoustic solver are merged to generate a number of merged room impulse responses. The computer-implemented method according to any one of the preceding items, wherein the number of room impulse responses generated using the wave-based solver are generated in low frequencies of an acoustic spectrum and the second number of impulse responses generated using the geometrical acoustic solver are generated in high frequencies of the acoustic spectrum. 17. The computer-implemented method according to item 16, wherein the acoustic spectrum is comprised between 0 and 20 kHz, such as between 0 and 15 kHz, such as between 0 and 12 kHz, such as between 0 and 10 kHz, such as between 0 and 8 kHz, such as between 0 and 6 kHz, such as between 20 and 20 kHz, such as between 20 and 15 kHz, such as between 20 and 12 kHz, such as between 20 and 10 kHz, such as between 20 and 8 kHz, such as between 20 and 6 kHz.

[0152] 18. The computer-implemented method according to item 16, wherein the low frequencies of the acoustic spectrum are comprised between 0 and 20 kHz, such as between 0 and 15 kHz, such as between 0 and 12 kHz, such as between 0 and 10 kHz, such as between 0 and 8 kHz, such as between 0 and 6 kHz, such as between 20 and 20 kHz, such as between 20 and 15 kHz, such as between 20 and 12 kHz, such as between 20 and 10 kHz, such as between 20 and 8 kHz, such as between 20 and 6 kHz, such as between 20 and 5 kHz, such as between 20 and 4 kHz, such as between 20 and 3 kHz, such as between 20 and 2 kHz, such as between 20 and 1,5 kHz, such as between 20 and 1 kHz.

[0153] 19. The computer-implemented method according to item 16, wherein the high frequencies of the acoustic spectrum are comprised between 1 kHz and 20 kHz, such as between 1,5 kHz and 20 kHz, such as between 2 kHz and 20 kHz, such as between 3 kHz and 20 kHz, such as between 4 kHz and 20 kHz, such as between 5 kHz and 20 kHz, such as between 6 kHz and 20 kHz, such as between 8 kHz and 20 kHz, such as between 10 kHz and 20 kHz, such as between 12 kHz and 20 kHz, such as between 1 kHz and 15 kHz, such as between 1,5 kHz and 15 kHz, such as between 2 kHz and 15 kHz, such as between 3 kHz and 15 kHz, such as between 4 kHz and 15 kHz, such as between 5 kHz and 15 kHz, such as between 6 kHz and 15 kHz, such as between 8 kHz and 15 kHz, such as between 10 kHz and 15 kHz, such as between 12 kHz and 15 kHz, such as between 1 kHz and 12 kHz, such as between 1,5 kHz and 12 kHz, such as between 2 kHz and 12 kHz, such as between 3 kHz and 12 kHz, such as between 4 kHz and 12 kHz, such as between 5 kHz and 12 kHz, such as between 6 kHz and 12 kHz, such as between 8 kHz and 12 kHz, such as between 10 kHz and

[0154] 12 kHz. The computer-implemented method according to any one of the preceding items, where the generated device specific room impulse response is encoded and decoded using ambisonics. The computer-implemented method according to any one of the preceding items, where determining the energy content for at least one frequency of the first device related transfer function comprises determining the ambisonics order N for the energy content of the at least one frequency. The computer-implemented method according to item 21 , wherein the ambisonics order N is determined for multiple frequencies, where the energy content for each frequency is determined. The computer-implemented method according to item 21 or 22, comprises determining the ambisonics order N for the energy content is based on determining the energy content as a sum of the ambisonics coefficients for each order N and then normalized to one for each frequency. The computer-implemented method according to any one of the items 21 - 23, where the number of digital representations of receivers are determined based on the energy content for at least one frequency of the device related transfer function further comprises determining the number of digital representation of receivers based on the ambisonics order N, where the number of digital representations of receivers are (N+1)2, 1,5*(N+1)2or 2*(N+1)2. The computer implemented method according to any one of the preceding items, wherein generating the device related transfer function comprises

[0155] - obtain a device mesh model representing the geometry of the device and the position of the at least first microphone on the device mesh model,

[0156] - determine on the device mesh model a first closest mesh element, which is closest to the at least first microphone. The computer implemented method according to any one of the preceding items, wherein generating the device related transfer function comprises

[0157] - arrange a digital representation of a device receiver array comprising a plurality of digital representations of receivers around the device mesh model, such that the distance between any of the digital representations of the receivers and the device mesh model is not below at predetermined distance. The computer implemented method according to any one of the preceding items, wherein generating the device related transfer function comprises

[0158] - digitally emit a first impulse signal using the first closest mesh element as a sound source. The computer implemented method according to any one of the preceding items, wherein generating the device related transfer function comprises

[0159] - arrange a digital representation of a first source correction microphone located at a first source distance from the first closest mesh element, wherein the first source distance is smaller than the predetermined distance,

[0160] - determine a first source correction signal using a wave-based solver, where the first source correction signal describes the first impulse signal as received at the first source correction microphone,

[0161] - determine a plurality of first source corrected device impulse responses by source correcting each of the plurality of first device impulse responses using the first source correction signal,

[0162] - generate the first device related transfer function of the device for the first microphone by combining the plurality of first source corrected device impulse responses. The computer implemented method according to any one of the preceding items, wherein generating the device related transfer function comprises

[0163] - determine a plurality of first device impulse responses using a wave-based solver, and where each first device impulse response describes the impulse response of the first impulse signal received at the respective receiver. 30. The computer implemented method according to any one of the preceding items, wherein generating the spatial room impulse response comprises,

[0164] - obtain a 3D room model representing the geometry of the room and at least one acoustic characteristic.

[0165] 31. The computer implemented method according to any one of the preceding items, wherein generating the spatial room impulse response comprises,

[0166] - arrange at least one digital representation of at least one room sound source in the 3D room model.

[0167] 32. The computer implemented method according to any one of the preceding items, wherein generating the spatial room impulse response comprises,

[0168] - arrange a digital representation of a room receiver array comprising a number of digital representations of receivers, wherein the room receiver array is centred on an at least one listening point in the 3D room model and where the number of digital representations of receivers are determined based on the energy content for the at least one frequency of the first device related transfer function.

[0169] 33. The computer implemented method according to any one of the preceding items, wherein generating the spatial room impulse response comprises,

[0170] - digitally emit a room impulse signal from the at least one audio sound source.

[0171] 34. The computer implemented method according to any one of the preceding items, wherein generating the spatial room impulse response comprises,

[0172] - determine a number of room impulse responses using at least a wave-based solver for at least one wave-based frequency, where each room impulse response describes the emitted room impulse as received at a corresponding one of the number of digital representations of receivers.

[0173] 35. The computer implemented method according to any one of the preceding items, wherein generating the spatial room impulse response comprises,

[0174] - generate a spatial room impulse response based on the number of room impulse responses. A computer-implemented method for recovering a device-related transfer function (DRTF) for a primary device, wherein the DRTF for the primary device comprises at least one missing amplitude level, the method comprising the steps of:

[0175] - obtaining a plurality of amplitude levels of the DRTF for the primary device, as a function of a plurality of elevation angles and a plurality of azimuth angles, wherein the DRTF for the primary device comprises at least one missing amplitude level;

[0176] - obtaining a matrix comprising the spherical harmonics basis functions;

[0177] - applying a truncated singular value decomposition method to the matrix comprising the spherical harmonics basis functions, wherein the truncated singular value decomposition method comprises the steps of:

[0178] - generating singular values based on the matrix comprising the spherical harmonics basis functions according to a singular value decomposition (SVD);

[0179] - selecting a top selection of singular values and a corresponding set of singular vectors, wherein the top selection of singular values and the corresponding set of singular vectors are selected based on a predetermined threshold;

[0180] - generating a secondary matrix comprising the spherical harmonics basis functions based on the top selection of singular values and the corresponding set of singular vectors, according to the singular value decomposition (SVD);

[0181] - multiplying a matrix corresponding to the plurality of amplitude levels of the DRTF for the primary device with the inverse of the secondary matrix, thereby generating a plurality of ambisonics coefficients;

[0182] - generating a plurality of recovered amplitude levels of the DRTF for the primary device based on the plurality of ambisonics coefficients, wherein the plurality of recovered amplitude levels of the DRTF for the primary device comprises an image of the amplitude levels of the DRTF for the primary device and a reconstruction of the at least one missing amplitude level. 37. The computer-implemented method according to item 36, wherein the amplitude levels of the DRTF for the primary device are represented as a function of elevation and azimuth angles. 38. The computer-implemented method according items 36-37, wherein the amplitude levels of the DRTF for the primary device are for a DRTF frequency.

[0183] 39. The computer-implemented method according to item 38, wherein the DRTF frequency is comprised between 20 Hz and 20 kHz.

[0184] 40. The computer-implemented method according to items 36-39, wherein the amplitude levels are complex-valued, or wherein the amplitude levels are represented in the form of complex numbers. 41. The computer-implemented method according to items 36-40, wherein the

[0185] DRTF for the primary device is generated according to any one of items 1-35.

Claims

Claims1. A computer-implemented method for generating a device specific room impulse response (DSRIR) describing an acoustic characteristic of a device and a room as received by the device, wherein the device comprises at least a first microphone, the method comprises- generate an at least first device related transfer function (DRTF), wherein the at least first device related transfer function describes the acoustic characteristic of the device as received by the at least first microphone, wherein generating the first device related transfer function further comprises o obtain a device mesh model representing the geometry of the device and the position of the at least first microphone on the device mesh model, o arrange a digital representation of a device receiver array comprising a plurality of digital representations of device receivers around the device mesh model, such that the distance between any of the digital representations of the device receivers and the device mesh model is not below at predetermined distance, o determine on the device mesh model a first closest mesh element, which is closest to the at least first microphone, o arrange a digital representation of a first source correction microphone located at a first source distance from the first closest mesh element, wherein the first source distance is smaller than the predetermined distance, o digitally emit a first impulse signal using the first closest mesh element as a sound source, o determine a first source correction signal using a wave-based solver, where the first source correction signal describes the first impulse signal as received at the first source correction microphone, o determine a plurality of first device impulse responses using a wavebased solver, and where each first device impulse response describes the impulse response of the first impulse signal received at the respective device receiver, o determine a plurality of first source corrected device impulse responses by source correcting each of the plurality of first device impulse responses using the first source correction signal,o generate the first device related transfer function of the device for the first microphone by combining the plurality of first source corrected device impulse responses, o determine an energy content for at least one frequency of the first device related transfer function,- generate a spatial room impulse response (SRIR), wherein the spatial room impulse response describes the acoustic characteristic of the room from at least one room sound source in the room and received at an at least one listening point in the room from at least one direction, wherein generating the spatial room impulse response further comprises o obtain a 3D room model representing the geometry of the room and at least one acoustic characteristic, o arrange at least one digital representation of at least one room sound source in the 3D room model, o arrange a digital representation of a room receiver array comprising a number of digital representations of room receivers, wherein the room receiver array is centred on an at least one listening point in the 3D room model and where the number of digital representations of room receivers are determined based on the energy content for the at least one frequency of the first device related transfer function, o digitally emit a room impulse signal from the at least one audio sound source, o determine a number of room impulse responses using at least a wavebased solver for at least one wave-based frequency, where each room impulse response describes the emitted room impulse as received at a corresponding one of the number of digital representations of room receivers, o generate a spatial room impulse response based on the number of room impulse responses,- generating the device specific room impulse response (DSRIR) by combining the device related transfer function and the spatial room impulse response.

2. The computer-implemented method according to claim 1 , wherein the predetermined distance is between 0,5 - 1,5 meters, preferably 0,8 - 1 ,2 meters or most preferred 1 meter.

3. The computer-implemented method according to claim 1 or 2, wherein the device comprises a plurality of microphones, such as a second, third, fourth, fifth microphone, wherein each of the plurality of microphones are treated as the first microphone such that a plurality of device related transfer functions, such as a second, third, fourth, fifth device related transfer function of the device is generated for each of the microphones.

4. The computer-implemented method according to claim 1, 2 or 3, wherein determining the energy content for the at least one frequency of the first device related transfer function comprises determining different ambisonics orders, thereby identifying the different levels of energy content.

5. The computer-implemented method according to claim 3 or 4, wherein the energy content is determined for a range of frequencies, such as from 0 to 20 kHz, such as 0 to 10 kHz, such as 10 to 20 kHz, such as 0 to 9 kHz, such as 0 to 8 kHz, such as 0 to 7 kHz, such as 0 to 6 kHz, such as 0 to 5 kHz, such as 0 to 4 kHz, such as 0 to 3 kHz, such as 0 to 2 kHz, such as 0 to 1 kHz.

6. The computer-implemented method according to any one of the preceding claims, wherein generating an at least first device related transfer function (DRTF) comprises- obtain a 3D box model comprising high acoustic absorption surfaces, or a 3D box model with a predefined size such that the first impulse signal is received once by the device receiver array.- arrange the device receiver array and the device mesh model in the 3D box model.

7. The computer-implemented method according to any one of the preceding claims, wherein generating an at least first device related transfer function (DRTF) comprises arranging the device receiver array comprising the plurality of digital representations of device receivers as a sphere or as an off-set shape where the digital representations of the device receivers are placed / arranged at a predetermined off-set distance from the device mesh model.

8. The computer-implemented method according to any one of the preceding claims, wherein the method further comprises determine a second number of room impulse responses using at least a geometrical acoustic solver for at least one geometrical acoustic frequency.

9. The computer-implemented method according to claim 8, wherein the number of impulse responses generated using the wave-based solver and the second number of room impulse responses generated using the geometrical acoustic solver are merged to generate a number of merged room impulse responses.

10. The computer-implemented method according to any one of the claims 8 or 9, wherein the number of room impulse responses generated using the wavebased solver are generated in low frequencies of an acoustic spectrum and the second number of impulse responses generated using the geometrical acoustic solver are generated in high frequencies of the acoustic spectrum.

11. The computer-implemented method according to any one of the preceding claims, where the generated device specific room impulse response is encoded and decoded using ambisonics.

12. The computer-implemented method according to any one of the preceding claims, where determining the energy content for at least one frequency of the first device related transfer function comprises determining the ambisonics order N for the energy content of the at least one frequency.

13. The computer-implemented method according to claim 12, wherein the ambisonics order N is determined for multiple frequencies, where the energy content for each frequency is determined.

14. The computer-implemented method according to claim 12 or 13, comprises determining the ambisonics order N for the energy content is based on determining the energy content as a sum of the ambisonics coefficients for each order N and then normalized to one for each frequency.

15. The computer-implemented method according to any one of the claims 12 - 14, where the number of digital representations of room receivers are determined based on the energy content for at least one frequency of the device related transfer function further comprises determining the number of digital representation of room receivers based on the ambisonics order N, where the number of digital representations of room receivers are (N+1)2, 1,5*(N+1)2or 2*(N+1)2.

Images