Apparatus and method for determining a virtual sound source

Abstract

An acoustic mirror-image source model of the early reflections in the room is generated by iteratively mirroring the room (305) around a room boundary (e.g., a wall) from the previous iteration. The boundary along which the mirroring occurs in each iteration is determined by specific selection criteria (303), including the requirements that the mirror direction must not be reversed, must not be an excluded direction, and must not be repeated unless in a consecutive series of mirrors.

Description

Technical Field

[0001] The present invention relates to an apparatus and method for determining a virtual sound source representing the reflection of a sound source in a room, and more particularly, but not limited to, a virtual sound source for rendering sound in an extended / virtual reality application.

Background Art

[0002] In recent years, the diversity and scope of experiences based on audiovisual content have increased significantly, and new services and methods for utilizing and consuming such content have been continuously developed and introduced. In particular, many spatial interactive services, applications, and experiences have been developed to provide users with more engaging and immersive experiences.

[0003] Examples of such applications include virtual reality (VR) applications, augmented reality (AR) applications, and mixed reality (MR) applications that are rapidly becoming mainstream, and many solutions are targeting the consumer market. Also, various standards have been developed by various standardization bodies. Such standardization activities are actively developing standards for various aspects of VR / AR / MR systems, such as streaming, broadcasting, and rendering.

[0004] While VR applications tend to provide a user experience corresponding to a user in a different world / environment / scene, AR (including mixed reality MR) applications tend to provide a user experience corresponding to a user in the current environment with additional information or virtual objects or information added. Thus, VR applications tend to provide a completely immersive artificial world / scene, while AR applications tend to provide a partially artificial world / scene superimposed on the real scene where the user physically exists. However, these terms are often used synonymously and overlap significantly. Hereinafter, the term virtual reality / VR is used to represent both virtual reality and extended / mixed reality.

[0005] As an example, what is becoming increasingly popular is a service that provides images and sound so that users can interact with the system actively and dynamically in order to change the rendering parameters to adapt to changes in the movement, position, and orientation of the user. In many applications, functions that change the viewer's virtual perspective and line of sight, for example, functions that allow the viewer to move within the presented scene and "look around", are very attractive.

[0006] Such functions can, in particular, provide users with a virtual reality experience. This can enable the user to move around (relatively) freely within the virtual environment and dynamically change their position and line of sight. Usually, such virtual reality applications are based on a three-dimensional model of the scene, and the model is dynamically evaluated to provide a specific required view. This approach is well known in game applications such as the first-person shooting game genre for computers and consoles.

[0007] Also, especially in the case of virtual reality applications, it is desirable for the presented image to be a three-dimensional image. In fact, in order to optimize the viewer's sense of immersion, it is usually preferable for the user to experience the presented scene as a three-dimensional scene. In fact, a virtual reality experience preferably enables the user to select their position, the camera's perspective, and the point in time with respect to the virtual world.

[0008] In addition to visual rendering, most VR / AR applications further provide a corresponding audio experience. In many applications, it is preferable for the audio to provide a spatial audio experience where the sound source is perceived to reach from a position corresponding to the position of the corresponding object in the visual scene. Therefore, the audio scene and the video scene are preferably consistent and are perceived as both providing a complete spatial experience.

[0009] For example, many immersive experiences are provided by virtual audio scenes generated by headphone playback using binaural audio rendering technology. In many cases, such headphone playback may be based on head tracking so that it can be rendered in response to the movement of the user's head, which greatly improves the sense of immersion.

[0010] However, in order to provide the user with a highly immersive and personalized natural experience, it is important that the rendering of the audio scene be as realistic as possible. Also, for audiovisual combined experiences, such as many VR experiences, it is important that the audio experience exactly matches that of the visual experience, that is, the rendered audio scene and the video scene exactly match.

[0011] In order to provide a high-quality experience, it is important that the acoustic environment be characterized by an accurate and realistic model, especially for the audio to be perceived as real. This is required whether the presented audio scene is a purely virtual scene or the scene is desired to correspond to a specific real-world scene.

[0012] When simulating the acoustics of a room, or more generally the acoustics of an environment, due to the reflection of sound waves on the walls, floor, and ceiling of the environment (if they exist), delayed and attenuated (usually frequency-dependent) versions of the source signal reach the listener from various directions. This results in an impulse response called the room impulse response (RIR).

[0013] As shown in Figure 1, the room impulse response consists of a direct sound / anechoic part that depends on the distance from the sound source to the listener, followed by a reverberant part that characterizes the acoustic properties of the room. The size and shape of the room, the positions of the sound source and the listener in the room, and the reflection characteristics of the room's surfaces all affect the characteristics of this reverberant part.

[0014] The reverberant part can usually be divided into two overlapping time domains. The first domain includes the so-called early reflections. These are the isolated reflections of the sound source on the walls and obstacles in the room before reaching the listener. As the time lag increases, the number of reflections present within a certain time interval increases, and secondary and higher-order reflections are also included.

[0015] The second domain within the reverberant part is where the density of these reflections has increased to such an extent that they cannot be isolated and separated in the human brain. This domain is called diffuse reverberation, late reverberation, or reverberation tail.

[0016] The reverberant part contains clues that provide the auditory system with information about the distance of the sound source, the size of the room, and the acoustic characteristics of the room. The energy of the reverberant part relative to the energy of the anechoic part greatly influences the perceived distance of the sound source. The level and delay of the earliest reflections can provide clues about how close the sound source is to the walls, and by filtering this with anthropometry, the evaluation of which wall, floor, or ceiling can be improved.

[0017] (Initial) reflection density contributes to the perceived size of the room. The time it takes for the energy level of the reflections to drop by 60 dB, i.e., the reverberation time, is denoted as T 60 and is a measure of how quickly the reflections dissipate within the room. The reverberation time provides information about the acoustic characteristics of the room, whether the walls are highly reflective (e.g., a bathroom), or absorb sound easily (e.g., a bedroom with furniture, carpets, and curtains).

[0018] To provide an immersive experience, multiple RIRs are required for the reflections to represent the directions in which they reach the listener. These are associated with the speaker configuration, with each RIR associated with one of a plurality of speakers at known positions. Panning algorithms such as VBAP can be used to generate RIRs from known reflection directions.

[0019] Furthermore, when it is part of the binaural room impulse response (BRIR), the underwater RIR is filtered by the head, ears, and shoulders, i.e., by the head-related impulse response (HRIR), so the underwater RIR may depend on the anthropometric characteristics of the user.

[0020] Since the reflections of the late reverberation can no longer be separated, they can be parametrically simulated using a parametric reverberator such as a feedback-delay network like the Jot reverb unit. Regarding the early reflections, the incidence direction and distance-dependent delays are important cues for humans to extract information about the room and the relative position of the sound source. Therefore, to achieve a realistic immersive experience, it is necessary to make the simulation of the early reflections clearer than the late reverberation.

[0021] One approach for modeling early reflections is to mirror the sound source at each boundary of the room to generate virtual sound sources that represent the reflections. Such models are known as image-source models and are described in Allen JB, Berkley DA. “Image method for efficiently simulating small-room acoustics”, The Journal of the Acoustical Society of America 1979;65(4):943-50. However, while such models can provide an efficient and high-quality modeling of early reflections compared to methods with fewer restrictions due to room shape such as ray tracing or finite element modeling, they tend to have several drawbacks. Specifically, they are still relatively complex and tend to require unduly high computational resources, especially for finding virtual reflection sources. For example, this process can generate many duplicates of the source, which then need to be further considered, processed, and culled. The drawbacks tend to increase with the number of reflections considered, and in many practical applications, the number of reflections is correspondingly limited, reducing the accuracy and quality of the model.

[0022] Accordingly, an improved model and method for generating virtual sources representing reflections would be advantageous. In particular, methods / models that enable improvements in operation, increased flexibility, reduced complexity, ease of implementation, improved audio experience, reduced complexity, reduced computational load, improved audio quality, improved accuracy and quality of the model, and / or improved performance and / or operation would be advantageous. SUMMARY OF THE INVENTION

[0023] Accordingly, the present invention aims to suitably mitigate, reduce, or eliminate one or more of the above drawbacks, either alone or in any combination.

[0024] According to one aspect of the present invention, there is provided a method for determining a virtual sound source representing the reflection of a first sound source in a first room. The method includes the steps of: a computer receiving data representing the boundary of the first room and the sound source position of the first sound source in the first room; and repeatedly determining a virtual sound source as a mirrored sound source by performing sound source mirroring of the sound source determined in the previous iteration. Each iteration includes, for each source room in a set of source rooms including the mirror room determined in the immediately previous iteration, determining a set of mirror boundaries of the source room; for each mirror boundary in the set of mirror boundaries, determining a mirror room by mirroring the source room about the mirror boundary; and determining a mirror sound source by mirroring the source sound source about the mirror boundary. The source sound source is the mirror sound source of the source room, the mirroring has a mirroring direction from the source room to the mirror room, and the step of determining the set of mirror boundaries includes selecting a boundary of the source room according to a selection criterion. The selection criterion includes the requirement that the first mirroring direction of the candidate boundary must not be in the opposite direction to the direction of any of the previous mirrorings that brought about the source room for the candidate boundary to be included in the set of mirror boundaries; the requirement that the first direction must not be an exclusion direction for the candidate boundary to be included in the set of mirror boundaries, where the exclusion direction depends on the boundary of the first room that was the axis of the mirroring that brought about the source room; and the requirement that the first direction must not be in the same direction as the mirroring direction of any of the previous mirrorings that brought about the source room, excluding the mirroring direction of the mirroring that generated the source room in the immediately previous iteration, for the candidate boundary to be included in the set of mirror boundaries.

[0025] The present invention can improve and / or facilitate the determination of virtual sound sources representing reflections in a room. This approach can facilitate and / or make more efficient the generation of models for early reflections in a room. This approach can, in many embodiments, prevent the generation of overlapping virtual sound sources. This approach can, in many embodiments, enable the generation of an accurate model representing reflections in a room with lower computational requirements and / or lower complexity.

[0026] The first room may be represented as a two-dimensional rectangle and / or a three-dimensional rectangle. The first room may be a two-dimensional or three-dimensional hyper-rectangular prism (also called a cuboid).

[0027] The boundaries of the room can be planar elements that define the boundaries of the room, such as walls, floors, or ceilings. The boundaries may be acoustically reflective elements, or alternatively, for example, in some cases, a virtual or theoretical (arbitrary) depiction of a boundary where (significantly) no acoustically reflective element exists.

[0028] The room can be any acoustic environment that is substantially planar and is usually bounded by acoustically reflective elements. The planar elements can be parallel in pairs. A two-dimensional room can have two such parallel pairs, and a three-dimensional room can have three such parallel pairs (corresponding to four walls, a floor, and a ceiling).

[0029] The mirroring of a sound source across a boundary can correspond to determining the mirrored sound source position by mirroring the sound source position of the source sound source about the boundary as an axis.

[0030] Specifically, the method can include determining the mirror sound source position of the reflected sound source in the mirror room by mirroring the reflection sound source position of the sound source in the source room about the boundary as an axis.

[0031] According to an optional feature of the present invention, the selection criterion includes the requirement that for the candidate boundary to be included in the set of mirror boundaries, the first direction must be the same as the second direction if the second mirroring direction of any previous mirroring that led to the source room is perpendicular to the exclusion direction and the mirroring direction of the first room that led to the source room.

[0032] This approach can provide, in many embodiments, an efficient generation of a model for representing three-dimensional reflections in a three-dimensional room. This approach can typically prevent the generation of overlapping sound sources (in combination with other requirements), and can typically enable all virtual sound sources corresponding to a given reflection order to be represented by the generated virtual sound sources.

[0033] According to an optional feature of the present invention, the first room has a pair of reference mirroring directions that are opposite to each other, and the selection criterion includes the requirement that for the candidate boundary to be included in the set of mirror boundaries, the first direction must be the same as the second direction if the second mirroring direction of any previous mirroring that led to the source room is a direction belonging to the associated pair of reference mirroring directions.

[0034] This approach can provide, in many embodiments, an efficient generation of a model for representing three-dimensional reflections in a three-dimensional room. This approach can typically prevent the generation of overlapping sound sources (in combination with other requirements), and can typically enable all virtual sound sources corresponding to a given reflection order to be represented by the generated virtual sound sources.

[0035] The pair of reference mirroring directions may be the directions of two parallel boundaries of the first room. The pair of reference mirroring directions may be a predetermined pair of reference mirroring directions.

[0036] In some embodiments, two opposing boundaries of the first room are each designated as reference boundaries having a reference mirror direction, and the selection criterion may include the requirement that for the candidate boundary to be included in the set of mirror boundaries, the first direction must not be the reference mirror direction if the previous mirroring that resulted in the source room had the reference mirror direction.

[0037] According to an optional feature of the present invention, for the first iteration, the first room is designated as the source room of the set of source rooms for the first iteration.

[0038] This may provide an advantageous approach for initializing / starting the iterative generation of virtual sound sources.

[0039] According to an optional feature of the present invention, all boundaries of the first room are included in the set of mirror boundaries for the first iteration.

[0040] This may improve performance and / or reduce complexity / resource usage in many embodiments. This may typically enable the generation of an improved model with lower complexity and computational resource usage. Additionally, an advantageous approach for initializing / starting the iterative generation of virtual sound sources may be provided.

[0041] According to an optional feature of the present invention, each boundary of the first room is associated with an attenuation coefficient, and the method includes determining a combined attenuation coefficient for each mirror sound source by combining the attenuation coefficients of all boundaries included in the mirroring that resulted in the mirror room containing the mirror sound source.

[0042] This may provide an improved model capable of providing a more realistic voice rendering.

[0043] According to an optional feature of the present invention, the selection criterion includes the requirement that for the candidate boundary to be included in the set of mirror boundaries, the result of combining the combined attenuation coefficient of the source sound source with the attenuation coefficient of the candidate boundary must not exhibit attenuation below a threshold.

[0044] This can lead to an improvement of the model in many embodiments.

[0045] According to an optional feature of the present invention, the coupling attenuation coefficient is frequency-dependent.

[0046] This can lead to an improvement of the model in many embodiments.

[0047] According to an optional feature of the present invention, the attenuation coefficient of an acoustically non-reflective boundary exhibits complete attenuation.

[0048] This can lead to an improvement of the model in many embodiments.

[0049] According to an optional feature of the present invention, the method further comprises a step (309) of rendering an audio signal for a listening position in a first room, the audio signal comprising at least one audio component representing audio from at least one mirror sound source reaching the listening position.

[0050] This approach may provide improved audio rendering, particularly more accurate rendering of early reflections.

[0051] According to an optional feature of the present invention, the set of mirror boundaries includes all boundaries that meet the selection criteria.

[0052] This can improve performance in many embodiments and / or reduce complexity / resource usage. This may typically enable the generation of an improved model with lower complexity and computational resource usage.

[0053] According to an optional feature of the present invention, the iteration is performed a predetermined number of times.

[0054] This can improve performance and / or reduce complexity / resource usage in many embodiments. This can typically enable the generation of an improved model with lower complexity and computational resource usage.

[0055] According to an optional feature of the present invention, the first room is a hyper-rectangular parallelepiped.

[0056] According to one aspect of the present invention, there is provided an apparatus for determining a virtual sound source representing a reflection of a first sound source in a first room, the apparatus comprising: a receiver that receives data representing a boundary of the first room and a sound source position of the first sound source in the first room; and a processing circuit that repeatedly determines a virtual sound source as a mirrored sound source by performing sound source mirroring of the sound source determined in the previous iteration. Each iteration includes: for each source room in a set of source rooms including the mirror room determined in the immediately preceding iteration, determining a set of mirror boundaries of the source room; for each mirror boundary in the set of mirror boundaries, determining a mirror room by mirroring the source room about the mirror boundary; and determining a mirror sound source by mirroring the source sound source about the mirror boundary, where the source sound source is the mirror sound source of the source room, and the mirroring has a mirroring direction from the source room to the mirror room. The step of determining the set of mirror boundaries includes the step of selecting the boundaries of the source room according to selection criteria, where the selection criteria are that for a candidate boundary of the source room to be included in the set of mirror boundaries, the first mirroring direction of the candidate boundary must not be the opposite direction to the direction of any of the previous mirrorings that brought about the source room, and that for the candidate boundary to be included in the set of mirror boundaries, the first direction must not be the exclusion direction, where the exclusion direction depends on the boundary of the first room that became the axis of the mirroring that brought about the source room, and that for the candidate boundary to be included in the set of mirror boundaries, the first direction must not be the same direction as the mirroring direction of any of the previous mirrorings that brought about the source room, except for the mirroring direction of the mirroring that generated the source room in the immediately previous iteration.

[0057] The above and other aspects, features, and advantages of the present invention will be described and become apparent with reference to the embodiments described below.

Brief Description of the Drawings

[0058] Hereinafter, embodiments that are merely examples of the present invention will be described with reference to the following drawings.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

[0059] Voice rendering aimed at providing a natural and realistic effect to a listener usually includes rendering of an acoustic environment. The rendering is based on a model of the acoustic environment and usually includes modeling of a direct path, (initial) reflections, and reverberation. In the following description, an efficient approach for generating a model suitable for (initial) reflections in a real or virtual room is focused on.

[0060] This approach will be described with reference to a voice rendering apparatus as disclosed in FIG. 2. The voice rendering apparatus comprises a receiver 201 configured to receive room data characterizing a room representing an acoustic environment to be emulated by the rendering. The room data specifically represents the boundaries of the first room and at least one sound source position for a sound source within the room. In the following, to distinguish between the generated virtual (mirrored) room and the virtual (mirrored) sound sources generated for the described reflection model, the room is also referred to as the original room (or first room), and the sound sources within the original room are also referred to as original sound sources.

[0061] The receiver 201 can be implemented in any suitable manner, including, for example, using an individual or dedicated electronic device. The processing circuit 203 can be implemented as an integrated circuit, such as an application specific integrated circuit (ASIC) for example. In some embodiments, the circuit can be implemented as a programmed processing device executed on a suitable processor, such as a central processing unit, a digital signal processing device, or a microcontroller for example, such as firmware or software for example. In such embodiments, the processing device can include an on-board or external memory, a clock drive circuit configuration, an interface circuit configuration, a user interface circuit configuration, and the like. Such circuit configurations can also be implemented as part of the processing device, as an integrated circuit, and / or as a discrete electronic circuit configuration.

[0062] The receiver 201 can receive room data from any suitable source in any suitable format, such as, for example, as part of an audio signal. The room data can be received from an internal or external source. The receiver 201 can be configured to receive the room data via, for example, a network connection, a wireless connection, or any other suitable connection to an internal source. In many embodiments, the receiver can receive data from a local source, such as a local memory. In many embodiments, the receiver 201 can be configured to retrieve the room data from a local memory, such as a local RAM or ROM memory for example.

[0063] The boundaries define the outline of the room and typically represent the walls, ceiling, and floor (or, in the case of a 2D application, typically only the walls). The room is a 2D or 3D hyper-rectangular prism, such as a 2D rectangle or a 3D rectangle for example. The boundaries are parallel in pairs and are substantially planar. Further, the boundaries of one pair of parallel boundaries are perpendicular to the boundaries of one or more other pairs of parallel boundaries. The boundaries specifically define a hyper-rectangular prism (2D or 3D). The boundaries can reflect any physical property, such as a material for example. The boundaries can also represent acoustic properties.

[0064] The room represented by the room data corresponds to the acoustic environment intended for rendering and can thus represent a real room / environment or a virtual room / environment. The room can be any area / environment that can be delimited / bounded by four (in 2D) or six (in 3D) substantially planar boundaries that are parallel in pairs and substantially perpendicular between pairs. In some embodiments, the room data can represent a suitable approximation of the intended room that is not parallel in pairs and / or does not form right angles between the joined boundaries.

[0065] In most embodiments, the room data can further include acoustic data for one or more of the boundaries, or typically all of the boundaries. The acoustic characteristic data can specifically include a reflection attenuation scale for each wall, indicating the attenuation caused by the wall when sound is reflected by the wall. Alternatively, the reflection coefficient can indicate the portion of the signal energy reflected specularly from the boundary surface. In many embodiments, the attenuation scale can be frequency-dependent to model the possibility that reflections can vary for each frequency. Further, the acoustic characteristics can depend on the position on the boundary surface.

[0066] Receiver 201 is coupled to processing circuit 203. Processing circuit 203 is configured to generate a reflection model for the room / acoustic environment that represents (initial) reflections within the room and enables emulation of these during rendering execution. Specifically, processing circuit 203 is configured to determine virtual sound sources that represent the reflections of the original sound sources within the original room.

[0067] The processing circuit 203 can be implemented in any suitable form, including, for example, using discrete or dedicated electronic devices. The processing circuit 203 can be implemented as an integrated circuit, such as an application specific integrated circuit (ASIC) for example. In some embodiments, the circuit can be implemented as a programmed processing device executed on a suitable processor, such as a central processing unit, a digital signal processing device, or a microcontroller for example, such as firmware or software. In such embodiments, the processing device can include on-board or external memory, a clock drive circuit configuration, an interface circuit configuration, a user interface circuit configuration, etc. Such circuit configurations can also be implemented as part of the processing device, as an integrated circuit, and / or as a discrete electronic circuit configuration.

[0068] The processing circuit 203 is coupled to a rendering circuit 205 configured to render an audio signal representing a sound source. To provide the rendering of an audio scene, the audio signal typically further represents a plurality of other sound sources. The rendering circuit 205 specifically receives audio data characterizing the audio from the original sound source and can render this according to any suitable rendering method and technique. The rendering of the original sound source can include the generation of reflected audio based on a reflection model generated by the processing circuit 203. Further, typically, the signal components of the original sound source corresponding to the direct path and reverberation are also rendered. Those skilled in the art are aware of various methods for rendering audio (including, for example, those using binaural processing, spatial speaker configurations, and headphones), and these are not described in further detail for the sake of brevity.

[0069] The rendering circuit 205 can be implemented in any suitable form, including, for example, using discrete or dedicated electronic devices. The rendering circuit 205 can be implemented as an integrated circuit, such as an application specific integrated circuit (ASIC). In some embodiments, the circuit can be implemented as a programmed processing device executed on a suitable processor, such as a central processing unit, a digital signal processing device, or a microcontroller, for example, as firmware or software. In such embodiments, the processing device can include on-board or external memory, a clock drive circuit configuration, an interface circuit configuration, a user interface circuit configuration, and the like. Such circuit configurations can also be implemented as part of the processing device, as an integrated circuit, and / or as a discrete electronic circuit configuration.

[0070] Specifically, the processing circuit 203 is configured to generate a mirror source model for reflections. In the mirror source model, reflections are modeled by a plurality of separate virtual sound sources. Each virtual sound source is a replica of the original sound source and has a (virtual) position outside the original room. This virtual position is such that the direct path from the virtual position to the listening position exhibits the same characteristics as the reflected path from the original sound source to the listening position. Specifically, the path length of the virtual sound source representing the reflection is equal to the path length of the reflected path from the original sound source to the listening position. Further, the direction of arrival at the listening position in the virtual sound source path is equal to the direction of arrival of the reflected path. Further, for each reflection by a boundary (e.g., a wall) in the reflected path, the direct path passes through a boundary corresponding to the reflection boundary. Thus, the reflection effect can be directly modeled using the transmittance when passing through the model boundary. For example, an attenuation corresponding to the reflection attenuation of the boundary can be assigned to the transmittance when passing through the corresponding model boundary.

[0071] A particularly important characteristic of the mirror source model is that it can be independent of the listening position. The determined position and the room structure provide correct results for all positions within the original room. Specifically, virtual mirror sound sources and a virtual mirror room are generated, and using these, the reflection performance at any position within the original room can be modeled. That is, it can be used to determine the path length, reflection, and arrival direction at any position within the original room. Therefore, during the initial setup process, the mirror source model is generated, and the generated model can be continuously and dynamically used and evaluated, for example, as if the user is moving around (translating and / or rotating) within the original room. Thus, the generation of the mirror source model is executed without considering the actual listening position, and a more general model is generated.

[0072] The process of generating the mirror source model is an iterative process, and an example of a method for generating the model by generating virtual sound sources representing reflections is shown in FIG. 3.

[0073] The method starts at step 301 where the initial setup of the process is performed. This includes, for example, initializing the method to use specific characteristics of the room, i.e., initializing the method to be based on the characteristics extracted from the room data.

[0074] The process is based on the iterative mirroring of the room about the room boundary and the corresponding mirroring of the sound source about the room boundary. In each iteration, new rooms and sound sources are generated by mirroring the room and sound source (especially the position of the sound source) generated in the previous iteration about the boundary (part of the boundary) of the room generated in the previous iteration. When the process is initialized, the original room is initialized / considered as the room of the previous iteration, and the original sound source is initialized / considered as the sound source of the previous iteration. Therefore, the first iteration is based on considering a single original room and sound source as the result of the previous iteration.

[0075] The first iteration starts at step 303, and a set of mirror boundaries is determined for the rooms generated in the previous iteration. Specifically, a set of source rooms is determined as the rooms generated in the previous iteration. In the first iteration, the set of source rooms includes the original room (only the original room). Next, one of these source rooms is processed in step 303.

[0076] All the boundaries of the source rooms are initially candidate boundaries for the set of mirror boundaries, and none, one, several, or all of them can be selected from among these to be included in the set of mirror boundaries. This selection will be detailed later.

[0077] After step 303, step 305 follows, and mirroring is performed about each boundary in the set of mirror boundaries (hereinafter referred to as mirror boundaries). Each mirror includes what is obtained by mirroring the source room about the mirror boundary. Further, it includes what is obtained by mirroring the sound source of the source room about the mirror boundary. Therefore, mirroring about the mirror boundary generates new (virtual) mirror rooms and new (virtual) mirror sound sources. Therefore, mirroring converts the source room and the source sound source into new mirror rooms and mirror sound sources (which are respectively mirrored copies of the source room and the source sound source).

[0078] The mirroring of the source sound source can be performed by determining a line passing through the boundary and the source sound source such that the line is perpendicular to the surface of the boundary, and placing the mirror sound source at the same distance from the boundary (but on the opposite side, i.e., inside the mirror room).

[0079] The mirroring essentially defines a direction from one side of the mirror boundary to the other side, i.e., from the source room to the mirror room. The direction can be regarded as perpendicular to the mirror surface, or equivalently, the relative position of the mirror boundary of the room may be considered to indicate the direction. The direction can be associated with, for example, the original room. For example, the position of each boundary of the original room can be regarded as representing the direction. That is, for a 3D room, six individual directions can be defined, and for a 2D room, four individual directions can be defined. Since the mirror room is generated by mirroring, the alignment of the boundaries does not change, and thus, the boundaries of the mirror room also align with the four or six directions of the original room (however, of course, the mirroring reverses the relative position of the boundaries, for example, mirroring about the left or right boundary will reverse the positions of the left and right boundaries).

[0080] In the first iteration, since the original room is regarded as the mirror room of the previous iteration and the original sound source is regarded as the mirror sound source of the previous iteration, a set of mirror boundaries including the boundaries of the original room can be generated. Usually, the set of mirror boundaries in the first iteration includes all the boundaries of the original room.

[0081] Next, by performing mirroring with respect to the boundaries of the set of mirror boundaries, a plurality (usually up to four or six) of new mirror rooms each containing a new mirror sound source are generated.

[0082] The method then proceeds to step 307, where it is determined whether all the source rooms in the set of source rooms have been processed, i.e., whether all the mirror rooms generated in the previous iteration have been processed. If not all have been processed, the method proceeds to step 309 where the next source room is selected and the method returns to step 303.

[0083] If all have been processed, the method proceeds to step 311, where it is determined whether further iterations should be performed. If further iterations should be performed, the method proceeds to step 313, where the next iteration is set, for example, by determining a new set of source rooms that includes all the mirror rooms generated in the current iteration. Next, the method returns to step 303, where this new set of source rooms is processed and possibly mirrored. Thus, in each iteration, the number of mirror rooms / mirror sound sources increases based on the mirroring of the results of the previous iteration.

[0084] The iteration can continue, for example, until a predetermined number of iterations have been performed. If this is detected in step 311, the method proceeds to step 315, where the method, for example, stops or rendering based on the generated model can be performed.

[0085] This approach can generate a mirror source model in which reflections in the original room can be emulated by direct paths from virtual mirror sound sources.

[0086] As shown in FIG. 4, the reflected sound component is rendered as a direct path of the mirrored sound source, which represents the correct incidence distance and incidence direction for the listener. This holds for all positions within the original room, and there is no need to determine the positions of new mirror sound sources for different listening positions. The virtual mirror source is valid for all user positions within the original room.

[0087] When generating this virtual mirror source, the reflection effect can be taken into account as described above. This can typically be done by assigning an attenuation or frequency-dependent filtering that represents the portion of the energy of the sound source that is specularly reflected by the surface of the boundary crossed for each transition between rooms.

[0088] Since sound can reach the user through multiple boundary reflections, the technique may be repeated as shown in Figure 5. The iterative technique described enables modeling multiple reflections by generating multiple "layers" of the mirror room and mirror sources. Each iteration increases the number of reflections in the path; that is, the first iteration represents the sound components that reach the listening position through one reflection, and the second iteration represents the sound components that reach the listening position through two reflections.

[0089] This technique typically generates a diamond-shaped representation of the original room and the mirrored rooms when sequentially mirroring the room up to a certain order (a fixed number of iterations). In Figure 6, this is shown in 2D up to the second order, i.e., two iterations. In 3D, there is a similar structure when looking at a cross-section through the original room (i.e., the same pattern is seen in a vertical plane passing through a row of five rooms).

[0090] However, although the principle of the described technique may seem relatively simple, the practical implementation is not, and in fact, practical considerations are important for the performance of the technique.

[0091] For example, in many applications, the coordinate system used to represent the room and sound source may not be aligned with the direction of the boundaries. Since this affects two or more dimensions simultaneously, the mirroring calculations become more complex. In such cases, it is necessary to rotate either the room boundary or the sound source to align with the coordinate system and then reverse-rotate all the virtual mirror sources determined later, or perform the mirroring itself in two or more dimensions (e.g., using the normal vector of the boundary). In many cases, the latter technique is more efficient.

[0092] A significant problem with this approach is that it tends to require high resources, specifically having high computational resource requirements. The inventors recognize that the important problem is that a large number of duplicate mirror rooms are generated, and the high resource usage is due not only to the resource usage in performing many mirror operations, but also to the requirements for post-processing the obtained mirror rooms and mirror sound sources to identify and eliminate duplicates.

[0093] As an example, FIG. 7 shows a 2D example of how secondary and quartic mirroring sequences result in duplicate mirror rooms. The relative number of duplicates increases progressively with the order of reflection, and in the case of 3D rooms and fifth-order reflections, when the mirror image source method is applied directly, up to 7776 virtual sources are recognized, of which only 230 are unique.

[0094] In the method of FIG. 3, a specific technique for selecting boundaries for a set of mirror boundaries is used so that in most applications, the generation of duplicate mirror rooms and mirror sound sources can be reduced and in some cases completely prevented.

[0095] Thus, the method is configured to select a subset of the room boundaries of the original room and the subsequent mirrors of each mirrored room. The subset of each room is selected so that duplicate rooms are not obtained, thus avoiding duplicate virtual mirror sources.

[0096] This is achieved by selecting boundaries for a set of mirror boundaries according to selection criteria that include a plurality of rules / constraints for selection. The selection criteria are used to control the progression of mirroring of rooms / sound sources through the room array within some amount of steps from the original room. This approach can specifically be regarded as selecting one path to each (potential) mirror room and excluding all other paths to that room. Since all of the possible different paths cross the corresponding boundaries, the possible different paths all include the same boundaries, although in a different order. However, usually, since reflections can be regarded as a linear operation, the order in which the sound reaches the listener from the sound source is not important, and thus the order of crossing the boundaries is irrelevant.

[0097] First, the selection criteria can be considered for 2D applications. In the following, it is considered to take into account four directions, namely, up and down corresponding to two parallel boundaries forming a pair, and back and forth corresponding to another pair of two parallel boundaries (perpendicular to the first pair).

[0098] Specifically, the selection criteria include the constraint / requirement that for a candidate boundary of the source room to be included in the set of mirror boundaries, the first mirroring direction of the candidate boundary must not be the opposite direction of the direction of any of the previous mirrorings that brought about the source room.

[0099] Thus, in step 303, the method can sequentially consider, for example, all the mirror rooms generated in the previous iteration as source rooms for potential further mirroring. And for the currently considered source room, all boundaries are evaluated as to whether to include them or not. For example, in the 2D example, all the walls of the source room are considered, and in the 3D example, the ceiling and floor are further considered.

[0100] Furthermore, (except for the first iteration), the current source room is a mirror room generated by one or more sequences of mirroring, and thus the current source room is associated with a sequence of one or more mirror directions that reflects how the source room was brought about as a result of any mirror operations.

[0101] Due to the above requirements, all boundaries of the source room corresponding to a mirror direction that is the opposite of a direction already included in the sequence of past mirror directions are then excluded from further consideration.

[0102] For example, if the source room was generated by a sequence that includes an upward mirroring, the boundaries corresponding to the downward mirror direction are excluded from the set of options for the mirror boundaries. Similarly, if the preceding direction sequence includes a forward mirroring, the boundaries corresponding to the backward mirror direction are excluded.

[0103] Thus, considering the sequential generation of mirror rooms via the mirroring path / sequence, after a mirroring is performed in a given direction, mirroring in the opposite direction is not permitted.

[0104] The selection criteria further include requirements related to the direction of the mirroring that was performed on the original room and led to the current source room.

[0105] Specifically, each boundary of the first room is associated with an excluded direction. The excluded direction for a given boundary is specifically the direction perpendicular to the mirroring direction of that boundary. Thus, the boundaries belonging to the first pair of parallel boundaries are excluded mirror directions for the boundaries belonging to a different pair of parallel boundaries. Two pairs of boundaries correspond to two of the dimensions in a 2D application, or two of the three dimensions in the case of 3D.

[0106] Specifically, the four boundaries belonging to the two pairs of parallel boundaries of the source room each have an associated exclusion direction, and the exclusion direction of each boundary is the mirror direction of the boundary belonging to the other pair of parallel boundaries. Further, the four associated exclusion directions of the four boundaries are all different, and thus the four exclusion directions correspond to four mirror directions.

[0107] As a specific example, the associated directions can be as follows.

Table 1

[0108] The selection criterion includes the constraint / requirement that the mirroring direction of a candidate boundary must not be an exclusion direction for the candidate boundary to be included in the set of mirror boundaries. The exclusion direction depends on the boundary of the first room that became the axis of the mirroring that brought about the source room.

[0109] Thus, when the method considers all the boundaries of a given source room to select boundaries for the set of mirror boundaries in step 303, the method specifically considers the first mirroring performed, i.e., the mirroring of the original room that ultimately led to the current source room. And the exclusion direction can be identified. For example, if the first mirroring was in the forward direction, the associated exclusion direction is determined to be right. And the boundaries having a mirror direction corresponding to the exclusion direction are excluded.

[0110] Due to this requirement, the boundaries of the source room corresponding to the mirror direction of the exclusion direction are excluded from subsequent consideration and are not included in the set of mirror boundaries. Thus, no mirroring is performed in the exclusion direction.

[0111] For example, if the source room is generated by a sequence initiated by a leftward mirroring, the boundaries corresponding to the forward direction are excluded from the options for the set of mirror boundaries. Thus, the progression of the generation of the mirror rooms is always in one direction for the dimension / boundary pair corresponding to the exclusion direction.

[0112] The selection criteria further include the constraint / requirement that, for a candidate boundary to be included in the set of mirror boundaries, the mirroring direction of the candidate boundary must not be the same as any of the previous mirroring directions that resulted in a source room in any previous iteration, except for the mirroring direction that resulted in the generation of the source room in the immediately preceding iteration.

[0113] Therefore, the mirroring direction of a mirror boundary must not be the same as the direction in which previous mirroring was performed, unless that direction is not the same as the mirror direction applied in the previous iteration, i.e., not the same as the one used to generate the source room itself.

[0114] Therefore, the selection requirement is that the direction of mirroring is not repeated, unless it was also used in the previous iteration, i.e., unless it is a continuation of mirroring in that given direction. Therefore, the selection criteria include the requirement that a given mirroring sequence never returns to a previously applied mirroring direction from which it has since deviated. Therefore, if mirroring starts in a certain initial direction, this can continue as long as desired, but if mirroring occurs in a different direction, the mirroring cannot return to the initial direction. Since mirroring in one direction excludes mirroring in the opposite direction, only mirroring in one direction is permitted for each dimension, and when the mirroring sequence switches from mirroring in one dimension to mirroring in a different dimension, it cannot return to the first dimension. That is, mirroring in one dimension is only possible in one direction and in a continuous mirroring sequence.

[0115] Therefore, in step 303, when the method considers all the boundaries of a given source room to select boundaries for the set of mirror boundaries, specifically, it considers all the previous mirror directions leading to the source room and excludes all boundaries having the same mirror direction as the previous mirror directions, except for the boundaries having the same mirror direction as the mirroring direction that generated the source room.

[0116] For example, the source room may be generated by a sequence of two mirrorings in the forward direction followed by two mirrorings in the left direction (corresponding to the sequence (forward, forward, left, left)). In that case, due to the requirement that the previous mirror direction is not returned except for the latest mirror direction, the boundary of the previous mirror direction is excluded, but the boundary of the left mirror direction is not excluded.

[0117] The above constraints and requirements can be closely coordinated (when considering a 2D application) to ensure that the mirroring executed in step 305 does not generate overlapping rooms. Further, to allow for the generation of all possible mirror rooms, all potential reflections (e.g., reflections up to a given number of times) can be automatically modeled.

[0118] Specifically, in many embodiments, the set of mirror boundaries is selected to include all boundaries that meet the selection criteria. Thus, for a mirror room considered as the source room, the set of mirror boundaries is generated to include all boundaries not excluded by the requirements. Typically, in 2D, one or two boundaries are included.

[0119] As described above, in the first iteration, the original room is considered as the only source room / mirror. The set of mirror boundaries is generated to include all boundaries of the original room. Also, note that in the case of the first iteration, since there is no previous direction or excluded direction, all four boundaries inherently meet the above criteria. Further, the first iteration determines the excluded direction for each new mirror room.

[0120] The selection criteria can interact closely and synergistically to enable the determination of mirror rooms and virtual sources that represent all reflections up to a given order without duplication in a 2D plane. This can be shown by FIG. 8 showing the room of the model generated after 4 iterations by the above process, i.e., the room of the model representing reflections up to the fourth order. The requirement to introduce exclusion directions essentially divides the space into 4 quadrants, and the other requirements ensure that each mirror room can only be reached through one specific mirror ring sequence / path. Further, the requirements result in the generation of all possible mirror rooms.

[0121] This approach is used to generate 3D models in many embodiments and can also include, for example, modeling of reflections including the ceiling and floor of the original room.

[0122] In this case, the above-described criteria described with reference to the 2D model continue to be used, but in addition, the selection criteria can include specific requirements for corresponding to the third dimension.

[0123] Specifically, the processing circuit 203 can include a requirement that for a candidate boundary to be included in the set of mirror boundaries, the mirror ring direction of the candidate boundary must be the same as a second direction if the second mirror ring direction of any previous mirror ring that resulted in the source room is perpendicular to the exclusion direction and the mirror direction of the mirror ring of the first room connected to the source room.

[0124] The previously described requirements mainly considered the 2D case where each room has two pairs of parallel boundaries. However, in the more typical 3D case, each room further has a third pair of parallel boundaries about which mirroring can be performed. Thus, each mirror boundary set can further include additional boundaries in two directions of the third dimension, for example, specifically in the up and down directions corresponding to the ceiling and floor of the room. The previously described requirements do not prevent such mirroring from being performed from the rooms in the original 2D plane. Thus, for each new mirror room in the original 2D plane generated in the previous iteration, in the current iteration, new mirror rooms can be generated above and below respectively.

[0125] The previously described requirements operate in two dimensions and provide a method for expanding the 2D model by mirroring the previously generated rooms. Specifically, the requirements enable mirroring to be performed in two dimensions as long as the requirements are met, resulting in a diamond-shaped area in the 2D plane. The 2D plane is determined by the direction of the first mirroring performed, i.e., the direction of mirroring of the original room connected to the current source room under consideration, and the associated exclusion direction.

[0126] In this specific example, the first mirroring direction is any one of (front, left, back, right), and similarly the exclusion direction is any one of (front, left, back, right). Thus, the requirements consider whether other mirroring directions have been previously performed, i.e., whether there has been previous mirroring in these other directions, i.e., whether there has been up or down mirroring. If there has been no such mirroring, the requirements impose no restrictions, thus not restricting mirroring within the 2D plane and also not restricting the first mirroring outside the 2D plane, i.e., the first up or down mirroring.

[0127] However, if previous mirroring was performed out of the 2D plane, i.e., if the first top or bottom mirroring was performed, the requirement imposes a strict constraint that only mirroring in the same direction is feasible. Thus, once top (or bottom) mirroring is performed, all subsequent mirroring must be in the top (or bottom) direction. Therefore, once the direction is changed out of the 2D plane, this direction must be maintained and a change of direction is not permitted. Thus, the direction of the first step within the mirror sequence in the third dimension prohibits other directions following this first step.

[0128] In some embodiments, the third dimension may not be specified by the exclusion direction and the first mirroring direction. Instead, the third dimension may simply be a specified reference dimension, specifically, a predefined reference dimension. Since one dimension represents mirroring about an axis with opposite boundaries, a dimension represents mirroring directions in two directions, i.e., opposite directions. Thus, the original room may be associated with a pair of reference mirroring directions in opposite directions. In this specific example, the reference direction pair may specifically be the top and bottom directions.

[0129] In such embodiments, the selection criterion may include the requirement that if the second mirroring direction of any previous mirroring that resulted in the source room is one of the two reference mirroring directions, then for the candidate boundary to be included in the set of mirror boundaries, the mirroring direction of the candidate boundary must be the same as the second direction.

[0130] Therefore, in such embodiments, once mirroring is performed in a reference direction, specifically, up or down in this example, all subsequent mirroring must be in the same direction. Thus, once top mirroring is performed, subsequent mirroring is only possible in the top direction and mirroring in other directions (right, left, back, front, or down) is no longer feasible.

[0131] This method ensures that the method can generate a set of (typically symmetric) mirror rooms for three - dimensionally modeling reflections. This interacts closely and synergistically with the requirements described above to enable efficient generation of a 3D model with low complexity while including accurate modeling of reflections. In particular, a 3D model of mirror rooms can be generated without duplication. Further, the requirements result in the generation of all possible mirror rooms.

[0132] The criteria used to determine the number of iterations to be executed can vary depending on the priorities and requirements of individual applications. In many embodiments, the iterations can be executed a predetermined number of times corresponding to a predetermined maximum number of reflections.

[0133] In other embodiments, a more adaptable criterion may be used, for example, the iterations continue until the combined attenuation coefficient (combined reflection coefficient) of all generated mirror sound sources falls below a threshold. Thus, in such implementations, the iterations can be repeated until the reflected signal is considered weak enough to be ignored.

[0134] It will be understood that any stopping criterion can be used to generate a model with the desired characteristics and / or to ensure that the process has the desired characteristics. For example, the iterations may continue until all attenuation coefficients fall below a threshold or until a predetermined number of iterations have been executed.

[0135] In many embodiments, the generated model can be used to render the audio signal of the original sound source at a given listening position within the original room. Step 315 may specifically include performing the rendering by the renderer 205 based on the model generated in the previous step.

[0136] Rendering may specifically include determining the audio components of each sound source corresponding to the direct (non-reflected) path from each sound source to the listening position. Further, for each path, the signal may be attenuated by an attenuation coefficient determined to directly correspond to the path length and a combined attenuation coefficient corresponding to the combined attenuation by all boundaries crossed by the path. Further, in many embodiments, the signal may be delayed by only a delay directly corresponding to the path length that simulates the flight time from the (virtual) sound source to the listener. The speed of sound is used to determine the delay from the path length. Thus, each audio component can generate a combined audio reaching the listening position by emulating one initial reflection and combining all audio components (including those directly from the original sound source) and optionally late reverberation components (which can be generated using any suitable means such as a Jot reverb unit).

[0137] Rendering the audio components as direct non-reflected propagation from the virtual sound source enables generating a rendering that is perceived as a natural and realistic sound by providing an efficient emulation of reflections in the room / acoustic environment.

[0138] Many rendering algorithms are known (such as spatial rendering algorithms using a spatial speaker configuration or binaural processing for headphone playback), and it will be understood that any suitable technique can be used.

[0139] As described above, each boundary of the original room is associated with acoustic characteristics. Specifically, the room data may represent the attenuation or reflection coefficient of each boundary. The attenuation / reflection coefficient specifically indicates the attenuation of the acoustic signal reflected by the wall, i.e., it may indicate the level difference / ratio between the incident audio signal and the reflected audio signal. The attenuation / reflection coefficient is frequency-dependent and may, for example, directly correspond to frequency-dependent filtering of the incident audio signal.

[0140] The attenuation coefficient of a boundary depends on the acoustic properties of the boundary and specifically on the material of the elements that make up the boundary. Some materials result in strong reflections (e.g., tiles), while other materials have high sound absorption (e.g., shaggy carpet), attenuating the sound and only a much smaller signal being reflected. This can be indicated by the attenuation coefficient.

[0141] For each virtual sound source, the path to the original room crosses a plurality of boundaries, the number of which is equal to the iteration in which the room was generated. Further, each boundary crossed models / corresponds to the reflections in the actual room. For example, a virtual sound source that crosses two boundaries to reach the original room models the path within the original room formed by two reflections. Further, the two reflections have attenuation coefficients, and when these attenuation coefficients are assigned to the mirror boundaries, the attenuation coefficient when crossing the mirror boundary directly reflects the influence of the reflection it models.

[0142] In many embodiments, a combined reflection / attenuation coefficient can be determined for each virtual sound source by combining the attenuation coefficients of the plurality of boundaries that are the axes of the mirroring that has been performed to generate the virtual mirror sound sources and the corresponding mirror rooms. Thus, by combining the attenuation coefficients of all the boundaries included in the mirroring that leads to the mirror room including the mirror sound source, a combined attenuation coefficient for that mirror sound source can be generated.

[0143] Thus, this combined attenuation coefficient reflects the combined reflection attenuation of all the reflections of the initial reflection modeled by the virtual sound source. Thus, the combined attenuation coefficient can be used by rendering, for example, to determine the signal level and / or frequency distribution of the audio components reaching the listening position. Further, this may not depend on the specific position of the listening position within the original room, and thus, only the distance-dependent path loss attenuation needs to be determined for the specific current listening position.

[0144] In some embodiments, the selection criterion may include the requirement that the result of combining the coupling attenuation coefficient of the source sound source with the attenuation coefficient of the mirror boundary must not exhibit attenuation below a threshold value. Thus, for a boundary to be recognized as a mirror boundary that generates a new mirror room and a mirror sound source, it is required that the attenuation of this mirror sound source does not exceed a predetermined attenuation amount. Thus, when a reflection path that attenuates the original sound source to an extent that can be considered not to contribute to the perception of the original sound is generated, the progress of the mirror ring is stopped. This can reduce complexity and resource requirements in many embodiments.

[0145] Using the attenuation coefficient may also make it possible to model very special scenarios. Specifically, it may enable efficient modeling of a room where one (or more) boundaries have high sound absorption or are acoustically transmissive and no reflections are generated at all.

[0146] Specifically, an acoustically non-reflective boundary can be represented by an attenuation coefficient that exhibits complete attenuation, i.e., indicates that no reflected signal is generated. Thus, when a non-reflective element forms a boundary, an attenuation coefficient of 100% can be assigned (corresponding to a reflection coefficient of zero). Thus, since the virtual mirror sound source generated by a mirror ring sequence that includes this boundary as a mirror boundary has a coupling attenuation coefficient of 100%, it does not generate any voice components at all. This corresponds to the fact that the reflection path including this boundary does not reach the listening position. In fact, the technique of setting the attenuation coefficient to 100% attenuation can also be applied to boundaries that do not include physical elements such as the absence of walls or ceilings.

[0147] In many embodiments, this may be combined with the selection of boundaries for a set of mirror boundaries that do not include boundaries that result in an attenuation coefficient below a given threshold value in order to cause the mirror sequence to stop when it reaches a non-reflective wall.

[0148] In some embodiments, the threshold may be adaptive. For example, it may depend on the order of reflection or on the relative (current, time-limited) level of the original sound source signal.

[0149] Thus, by using the above method, an acoustic mirror sound source model of the early reflections in a room can be generated by repeatedly mirroring the room about the boundaries (e.g., walls) of the previous iteration of the room. The boundaries that serve as the axes of mirroring in each iteration are determined by specific selection criteria that include the requirement that the mirror direction must not reverse, must not be an exclusion direction, and must not be repeated unless it is part of a continuous series of mirrorings.

[0150] This method can sequentially expand the model so that it includes modeling of progressively higher-order (i.e., more) reflections. An example of a tree representing the model is shown in FIG. 9. The tree represents the progression of an algorithm for finding all the mirror rooms of the third-order reflections (i.e., the depth of the tree is three mirroring iterations). In FIG. 9, up, down, left, right, front, and back are represented by U, D, L, R, F, and B, respectively. In this example, the dimensions and directions are represented as front-back, left-right, and up-down. Each node in the graph represents a room. The first node represents the original room, and the remaining nodes represent 62 mirror-imaged versions of the original room.

[0151] This method can use a very efficient algorithm to generate a model that can be very accurate and can be used to render sound so that it sounds real and natural.

[0152] This approach can, in particular, reduce the computational complexity and / or the required computational resource requirements / usage. It can be implemented using less computational power than typical applications. Specifically, compared to other approaches for generating a mirrored sound source model, it is typically possible to significantly reduce the computational resource requirements by reducing the number of mirroring operations necessary to determine the mirrored virtual sound sources. Also, post-processing typically associated with the need to resolve overlapping virtual sound sources can be avoided. Usually, a much more efficient process can be achieved.

[0153] Typically, the above approach for generating a mirrored sound source model can be part of an initial setup component / routine that derives a set of virtual mirror sound sources representing the reflections of the original sound source that must be executed at least once for a given room. If the sound source is moving, the mirrored sound source model can be recalculated or partially recalculated for one or more moving sound sources.

[0154] Thus, in many embodiments, the approach can be based on an iterative process of each process that includes two steps for each mirrored room determined in the previous iteration. For a given room (the sound source room), these steps can include the following. 1. Determine a set of mirror boundaries where mirroring of the sound source room and at least one point within the room is performed to obtain higher order reflections. 2. Mirror the source room and at least one (sound source) position / sound source across each boundary included in the set of mirror boundaries and update the combined reflection coefficient (attenuation coefficient) to include the reflection coefficient corresponding to the boundary across which the mirroring was performed.

[0155] Many embodiments can perform the iteration using a recursive process where the mirrored room is used as the source room for the next iteration. An example of the process in pseudocode is shown below. function [reflectionList, reflectionAttList] = optimalImageSource(roomDef, srcPos, reflectionAtt, state) [mirrorBoundarySet, state] = getSetOfMirrorBoundaries(roomDef, state); idx = 0; for b = mirrorBoundarySet idx = idx + 1; mirroredRoomDef(idx) = mirrorRoom(roomDef, roomDef.boundary(mirrorBoundarySet(b))); mirroredSrcPos(idx) = mirrorSrc(srcPos, roomDef.boundary(mirrorBoundarySet(b))); mirroredReflAtt(idx) = reflectionAtt * roomDef.reflectionCoeff(b); end reflectionList = mirroredSrcPos; reflectionAttList = mirroredReflAtt; state.order = state.order + 1; if (state.order < state.maxOrder) for idx = 1:length(mirrorBoundarySet) [reflectionListPart, reflectionAttListPart] = optimalImageSource( mirroredRoomDef(idx), mirroredSrcPos(idx), mirroredReflAtt(idx), state); reflectionList = concat(reflectionList, reflectionListPart); reflectionAttList = concat(reflectionAttList, reflectionAttListPart); end end This can be initialized and started using the following. maxOrder = 5; state = initOptimalImageSource(maxOrder); [reflectionList, reflectionAttList] = optimalImageSource(roomDef, srcPos, 1, state);

[0156] Often, the room is rectangular (also called the shoebox model) or may be approximated by a rectangular equivalent. As shown in FIG. 10, the boundaries of such a rectangular room model are not necessarily aligned with the coordinate system in which the boundaries of the room model are defined.

[0157] This gives rise to two problems that complicate the mirror source method. · Mirroring of points across the boundary affects both the measurement and adjustment in two, or in some cases three, dimensions simultaneously, rather than simple subtraction and addition in a single dimension. · The three mirroring dimensions (front-back, left-right, and up-down) of the source room are not directly mapped to the coordinate system.

[0158] One approach is to align the room in a coordinate system by rotating all room definition coordinates and sound source position coordinates, then calculate the virtual sound source positions, and rotate all of these back to their original positions by inverse rotation.

[0159] Another approach is to arbitrarily define mirroring dimensions for three pairs of parallel boundaries that define the room and perform mirroring using geometric calculations.

[0160] An exemplary approach based on the latter option is described below.

[0161] As a first step, it is necessary to find the three pairs of parallel boundaries that define the original room and assign them to three mirroring dimensions. This mapping can be arbitrarily selected. No particular order is required.

[0162] As an exception, for example, in order to further reduce the complexity of the calculation, there are cases where reflections are calculated only within the horizontal plane. In that case, it is necessary to detect the pair of the floor and the ceiling, which can be achieved by identifying the boundary having the normal vector closest to the upward axis of the coordinate system. For example, it is the boundary where the absolute dot product of each normalized normal vector is maximum.

Number

[0163] To identify the pairs, the normal vectors of each boundary are calculated (described in more detail in the later description regarding point mirroring). The pairs can be identified by the correlation matrix of all normal vector pairs, and it can also be confirmed that the room model is rectangular.

[0164] Each element c of the correlation matrix C ij = c jihas the dot product of the normal vectors at the boundaries of indices i and j. In the case of a rectangular room definition, all values are very close to either 0, 1, or -1. Pairs with values close to 1 or -1 are parallel pairs and can be considered as one of the three dimensions in which mirroring occurs.

[0165] As part of the process, the source room definition needs to be mirrored across a plane defined by one of the boundaries of the source room. Also, other positions within the source room, such as the (reflected) sound source position, can be mirrored across the same plane.

[0166] Mirroring a point across a plane is a well-known mathematical operation and can be performed using the normal vector of the plane. This vector, which is a direction vector, is perpendicular to all vectors within the plane and can be used to determine the point within the plane that is closest to the point being mirrored. This is the mirror point. By finding this point, the point can be mirrored to the opposite side of the plane by reversing the sign of the direction vector connecting the mirror point to the point being mirrored or doubling the length of the direction vector connecting the point being mirrored to the mirror point.

[0167] The normal vector of a plane can be derived from two vectors or three points within the plane. A particularly advantageous way to define the boundaries of a rectangular room is to define the coordinates of the four corners of the (rectangular) boundary. Therefore, to calculate the normal vector, it is sufficient to select three of these four coordinates.

[0168] In other representations, the room and the boundaries of the room may be defined as a mesh of three-point polygons. Similarly, the normal vector can be calculated using the three vertices of one polygon that defines the boundary.

[0169] The three selected points within the plane are

Number

Number

Number

Number

[0170] By taking the cross of two vectors, this orthogonal normal vector can be obtained.

Number

[0171] As a means of reducing the complexity in further calculations, the normal vector may be normalized.

Number

[0172] As a result, a normalized direction vector starting from the origin and perpendicular to the (infinite) plane passing through the boundary of the room is obtained. The normal vector alone is not sufficient to define a plane. Any point in the plane

Number

Number

Number

Number

[0173] Next, the vector connecting the point to be mirrored ( [Number] ) to the mirror point ( [Number] ) is calculated, where α is used to scale the direction vector to the correct length and sign. [Number]

[0174] Mirror point [Number] needs to be in the plane, so [Number]

[0175] transforming this gives the following. [Number]

[0176] According to this method, the mirrored point (s’) is obtained by the following calculation. [Number]

[0177] In most embodiments, the attenuation resulting from reflections is also calculated for each mirror sound source. Thus, in many embodiments, for each mirror ring operation, the reflection attenuation of the sound sources in the mirrored room is calculated and expressed by a coupling attenuation coefficient. Hereinafter, the attenuation coefficient can be directly represented by the reflection coefficient, but it will be understood that in many embodiments, the attenuation can depend, for example, on the frequency.

[0178] The reflection attenuation of the source room is combined with the reflection coefficient of the boundary that is the axis of the mirroring of that source room. The reflection coefficient of the boundary can be broadband or frequency-dependent. For example, the attenuation coefficient may be represented by FIR / IIR filter coefficients or attenuation coefficients within a plurality of frequency bands / bins. For example,

Number

[0179] The reflection coefficient is not necessarily uniform across the entire boundary. In such cases, the reflection coefficient used can be made uniform across the entire boundary by calculating an average reflection coefficient over the surface of the room boundary. Similarly, an average may be calculated for all the multiple boundaries of the room.

[0180] In a more accurate embodiment, for each position of the mirrored sound source, a mirror point (

Number

[0181] According to the rules outlined above, for each source room in the iterative process, a subset of 6 boundaries (typically) is selected as the set of mirror boundaries. In each iteration, each mirror room generated by the previous (immediately preceding) iteration is considered a source room, i.e., each newly created mirror room can be evaluated for further mirroring possibilities in the next iteration.

[0182] Each boundary of the original room specifies / represents a direction (e.g., the direction of the normal vector of the boundary from the room to the outside). Since the boundaries are parallel in pairs, each boundary pair defines one dimension in two directions (corresponding to the two boundaries that define the mirroring dimension).

[0183] For example, a parallel boundary pair can be represented as follows (where b i represents boundary i).

Number

[0184] In the first iteration (i.e., the generation of primary reflections), all directions are permitted, so the set of mirror boundaries of the source room corresponding to the original room includes all the boundaries of the source room. As a result, six (or four in the case of 2D modeling) branches are generated, and from these branches, higher-order reflections are calculated by further iterations (e.g., B = [b1, b5, b2, b4, b3, b6]). Therefore, six new mirror rooms and six new mirror sound sources are generated in the first iteration.

[0185] In any of these branches (e.g., the branch after mirroring across boundary b4), the next (i.e., the second) iteration can continue in the same direction with respect to the original room. This corresponds to the mirror that crosses the other boundary of the boundary pair in the corresponding dimension (since it is the mirroring across the boundary of the room generated by mirroring the original room). In this example, it is b2. This boundary alternation in each dimension is also shown in FIGS. 6 and 7.

[0186] The direction may be further restricted depending on the progression dimension of the previous iteration within the branch. For example, if the previous step was along the first dimension, only a single direction in the second dimension is permitted due to the exclusion direction within the second dimension. However, since the direction was only within the two dimensions, both directions in the third dimension are permitted. For example, when proceeding in the first direction of the first dimension, only the first direction in the second dimension is permitted due to the associated exclusion direction. When proceeding in the second direction of the first dimension, only the second direction in the second dimension is permitted.

[0187] In the same example, if the first step was proceeding in the first direction of the second dimension, only the second direction in the first dimension is permitted, and if the first step was proceeding in the second direction of the second dimension, the first direction in the first dimension is permitted. This inverse relationship between the permitted directions in the second step, depending on whether the first (first) step was in the first dimension (1-1, 2-2) or in the second dimension (1-2, 2-1), prevents the duplication of mirrored rooms without omitting the mirrored rooms.

[0188] Still in the same example, if any of the mirroring steps is along the third dimension, all subsequent mirroring can only be done in that direction and cannot be done along other directions or dimensions.

[0189] The above notations of the first, second, and third dimensions are not necessarily related to the defined order of the dimension pairs. It will be clear that "first", "second", and "third" are interchangeable when referring to dimensions. Similarly, "first" and "second" are also interchangeable when referring to the directions within a dimension. To repeat, in the above example, the directions within a dimension should be considered with reference to the original room, and the boundaries associated with a specific direction within a dimension alternate for each mirroring step in that direction.

[0190] Any branches with changed dimensions cannot return to mirroring in the previous dimension. For example, a branch that has been mirrored in the second dimension in the first step and then mirrored along the first dimension in the second step can only continue mirroring in that direction of the first dimension and any direction of the third dimension.

[0191] In advanced embodiments, when determining the set of permitted directions, attenuation coefficients such as reflection coefficients or total reflection attenuation may be considered. This can further reduce the complexity of the calculation. For example, if a direction is permitted according to the above rules but the reflection coefficient of the corresponding boundary is below a threshold (i.e., less than 0.05 or -20 dB or less), that direction may be excluded from the set of mirror boundaries.

[0192] Additionally, or alternatively, a rule may be included that if the combined reflection attenuation is below a threshold (e.g., less than 0.02), the boundary is excluded.

[0193] In the case of frequency-dependent coefficients, the threshold is frequency-dependent and may be related to a weighted average coefficient across all frequency bands, the maximum coefficient among all frequency bands, or a coefficient associated with a specific frequency (e.g., 1000 Hz).

[0194] Similarly, when the reflection coefficients are different between multiple regions within a boundary, the threshold may be applied to the positions of individual sound sources or the reflection coefficient averaged across that entire boundary of the room may be used.

[0195] For clarity, the above description has described embodiments of the present invention in relation to different functional circuits, units, and processors. However, it will be understood that the functions can be appropriately distributed among different functional circuits, units, or processors without detracting from the present invention. For example, functions described as being performed by a plurality of separate processors or controllers may be performed by the same processor or controller. Thus, references to specific functional units or circuits are not intended to denote a strict logical or physical structure or configuration, but rather should be considered as references to suitable means for providing the described functions.

[0196] The present invention may be implemented in any suitable form including hardware, software, firmware, or any combination thereof. The present invention may be at least partially implemented as computer software operating on one or more data processors and / or digital signal processors. The elements and components of embodiments of the present invention may be physically, functionally, and logically implemented in any suitable manner. In practice, the functions may be implemented as a single unit, multiple units, or part of other functional units. Thus, the present invention may be implemented within a single unit or physically and functionally distributed among different multiple units, circuits, and processors.

[0197] Generally, examples of apparatus and methods for determining virtual sound sources are shown by the following embodiments.

[0198] Embodiments 1. A method for determining a virtual sound source representing a reflection of a first sound source in a first room, the method comprising the computer receiving data representing a boundary of the first room and a sound source position of the first sound source in the first room; repeatedly determining the virtual sound source as a mirrored sound source by performing sound source mirroring of the sound source determined in a previous iteration. For each source room in the set of source rooms including the mirror room determined in the immediately preceding iteration, determining a set of mirror boundaries of the source room (step 303); for each mirror boundary in the set of mirror boundaries, determining a mirror room by mirroring the source room about the mirror boundary (step 305), and determining a mirror sound source by mirroring a source sound source about the mirror boundary, wherein the source sound source is the mirror sound source of the source room, and the mirroring has a mirroring direction from the source room to the mirror room, the step of determining the set of mirror boundaries (step 303) includes selecting a boundary of the source room according to a selection criterion, and the selection criterion is for a candidate boundary of the source room to be included in the set of mirror boundaries, the first mirroring direction of the candidate boundary must not be opposite to the direction of any previous mirroring that brought about the source room, for a candidate boundary to be included in the set of mirror boundaries, the first direction must not be an exclusion direction, and the exclusion direction depends on the boundary of the first room that served as the axis of the mirroring that brought about the source room, for a candidate boundary to be included in the set of mirror boundaries, the first direction must not be the same as the mirroring direction of any previous mirroring that brought about the source room, except for the mirroring direction of the mirroring that generated the source room in the immediately preceding iteration, a method.

[0199] 2. The selection criterion includes the requirement that for the candidate boundary to be included in the set of mirror boundaries when the second mirroring direction of any previous mirroring that resulted in the source room is perpendicular to the exclusion direction and the mirroring direction of the first room that resulted in the source room, the first direction must be the same as the second direction, according to the method described in Embodiment 1.

[0200] 3. The first room has a pair of reference mirroring directions that are opposite to each other, and the selection criterion includes the requirement that for the candidate boundary to be included in the set of mirror boundaries when the second mirroring direction of any previous mirroring that resulted in the source room is a direction belonging to the associated pair of reference mirroring directions, the first direction must be the same as the second direction, according to the method described in Embodiment 1.

[0201] 4. For the first iteration, the first room is designated as the source room of the set of source rooms for the first iteration, according to the method described in any one of Embodiments 1 to 3.

[0202] 5. All boundaries of the first room are included in the set of mirror boundaries for the first iteration, according to the method described in any one of Embodiments 1 to 4.

[0203] 6. Each boundary of the first room is associated with an attenuation coefficient, and the method includes the step of obtaining the combined attenuation coefficient of each mirror sound source by combining the attenuation coefficients of all boundaries included in the mirroring that resulted in the mirror room including the mirror sound source, according to the method described in any one of Embodiments 1 to 5.

[0204] 7. The selection criterion includes the requirement that for the candidate boundary to be included in the set of mirror boundaries, the result of combining the combined attenuation coefficient of the source sound source with the attenuation coefficient of the candidate boundary must not indicate attenuation less than a threshold value, according to the method described in Embodiment 6.

[0205] 8. The method according to embodiment 6 or 7, wherein the coupling attenuation coefficient is frequency-dependent.

[0206] 9. The method according to any one of embodiments 6 to 8, wherein the attenuation coefficient of the acoustically non-reflective boundary exhibits complete attenuation.

[0207] 10. The method according to any one of embodiments 1 to 9, further comprising the step (309) of rendering an audio signal for the listening position in the first room, the audio signal including at least one audio component representing audio from at least one mirror sound source reaching the listening position.

[0208] 11. The method according to any one of embodiments 1 to 10, wherein the set of mirror boundaries includes all boundaries that meet the selection criteria.

[0209] 12. The method according to any one of embodiments 1 to 11, wherein the iteration is performed a predetermined number of times.

[0210] 13. The method according to any one of embodiments 1 to 12, wherein the first room is a hyper-rectangular parallelepiped.

[0211] 14. An apparatus for determining a virtual sound source representing a reflection of a first sound source in a first room, the apparatus comprising: a receiver (201) for receiving data representing the boundaries of the first room and the sound source position of the first sound source in the first room; a processing circuit (203) for repeatedly determining the virtual sound source as a mirrored sound source by performing sound source mirroring of the sound source determined in the previous iteration; Each iteration is for each source room in a set of source rooms including the mirror room determined in the immediately preceding iteration, a step (303) of determining a set of mirror boundaries of the source room; For each mirror boundary in the set of mirror boundaries, determining a mirror room by mirroring the source room about the mirror boundary (step 305), and determining a mirror sound source by mirroring a source sound source about the mirror boundary, wherein the source sound source is a mirror sound source of the source room, and the mirroring has a mirroring direction from the source room to the mirror room, The step of determining the set of mirror boundaries includes the step of selecting a boundary of the source room according to a selection criterion, and the selection criterion is For a candidate boundary of the source room to be included in the set of mirror boundaries, a requirement that a first mirroring direction of the candidate boundary must not be opposite to the direction of any previous mirroring that resulted in the source room, For a candidate boundary to be included in the set of mirror boundaries, a requirement that the first direction must not be an exclusion direction, where the exclusion direction depends on a boundary of the first room that was the axis of the mirroring that resulted in the source room, For a candidate boundary to be included in the set of mirror boundaries, a requirement that the first direction must not be the same as the mirroring direction of any previous mirroring that resulted in the source room, excluding the mirroring direction of the mirroring that generated the source room in the immediately previous iteration, apparatus.

[0212] 15. A computer program product including computer program code means for performing all the steps of Embodiments 1 to 13 when the program is executed on a computer.

[0213] 1. A method for determining a virtual sound source representing a reflection of a first sound source in a first room, the method comprising a computer receiving data representing a boundary of the first room and a sound source position of the first sound source in the first room, executing a step of repeatedly determining the virtual sound source as the mirrored sound source by performing sound source mirroring of the sound source determined in the previous iteration; each iteration includes, for each source room in a set of source rooms including the mirror room determined in the immediately preceding iteration, a step (303) of determining a set of mirror boundaries of the source room of the current step; for each mirror boundary in the set of mirror boundaries, determining a mirror room by mirroring the source room about the mirror boundary (step 305), and determining a mirror sound source by mirroring a source sound source about the mirror boundary, the source sound source being the mirror sound source of the source room, the mirroring having a mirroring direction from the source room to the mirror room; the step of determining the set of mirror boundaries includes selecting a boundary of the source room according to a selection criterion, the selection criterion being a requirement that, for a candidate boundary of the source room to be included in the set of mirror boundaries, a first mirroring direction of the candidate boundary must not be opposite to the direction of any previous mirroring that brought about the source room; a requirement that, for the candidate boundary to be included in the set of mirror boundaries, the first direction must not be an exclusion direction, the exclusion direction depending on a boundary of the first room that was the axis of the mirroring that brought about the source room; a requirement that, for the candidate boundary to be included in the set of mirror boundaries, the first direction must not be the same as the mirroring direction of any previous mirroring that brought about the source room, except for the mirroring direction of the mirroring that generated the source room in the immediately preceding iteration. An apparatus.

[0214] 1. A method for determining a virtual sound source representing a reflection of a first sound source in a first room, the method comprising: Receiving data representing the boundary of the first room and the sound source position of the first sound source in the first room; repeatedly determining the virtual sound source as the mirrored sound source by performing sound source mirroring of the sound source determined in the previous iteration, including: For each source room in a set of source rooms including the mirror room determined in the immediately preceding iteration, determining a set of mirror boundaries of each source room (303); for each mirror boundary in the set of mirror boundaries, determining a mirror room by mirroring each source room about the mirror boundary (305), and determining a mirror sound source by mirroring a source sound source about the mirror boundary, wherein the source sound source is the mirror sound source of each source room, and the mirroring has a mirroring direction from each source room to the mirror room; The step (303) of determining the set of mirror boundaries includes selecting the boundary of each source room according to a selection criterion, and the selection criterion is: for a candidate boundary of each source room to be included in the set of mirror boundaries, a requirement that a first mirroring direction of the candidate boundary must not be in the opposite direction to the direction of any previous mirroring that brought about each source room; for the candidate boundary to be included in the set of mirror boundaries, a requirement that the first direction must not be an exclusion direction, wherein the exclusion direction depends on the boundary of the first room that was the axis of the mirroring that brought about each source room; for the candidate boundary to be included in the set of mirror boundaries, a requirement that the first direction must not be the same as the mirroring direction of any previous mirroring that brought about each source room, except for the mirroring direction of the mirroring that generated each source room in the immediately preceding iteration.

[0215] More specifically, the present invention is defined by the appended claims.

[0216] Although the present invention has been described in connection with several embodiments, the present invention is not limited to the specific forms described in the specification. The scope of the present invention is limited only by the appended claims. Further, even if a certain feature appears to be described in connection with a particular embodiment, those skilled in the art will recognize that the various features of the above embodiments can be combined in accordance with the present invention. In the claims, terms such as "comprising," "including," etc. do not exclude the presence of other elements or steps.

[0217] Furthermore, even if a plurality of means, elements, circuits, or method steps are individually listed, they may be implemented, for example, by a single circuit, unit, or processor. Further, even if individual features are included in different claims, these may be suitably combined, and the fact that they are included in different claims does not mean that the combination of features is impossible and / or not advantageous. Also, just because a feature is included within one claim category does not mean that the feature is limited to that category, and the feature may equally be applicable to other claim categories as appropriate. Further, the order of features in the claims does not refer to a specific order in which the features should act, and in particular, the order of individual steps in a method claim does not mean that the steps must be performed in that order. The steps may be performed in any suitable order. Also, singular expressions do not exclude plurals. Thus, expressions such as "first," "second," etc. do not exclude a plurality. Reference signs within the claims are merely examples for clarity and do not limit the scope of the claims in any way.

Claims

Claim 1 A method for determining a virtual sound source representing a reflection of a first sound source in a first room, the method comprising the computer receiving: data representing a boundary of the first room and a sound source position of the first sound source in the first room; iteratively determining the virtual sound source as a mirrored sound source by performing sound source mirroring of the sound source determined in the previous iteration, each iteration comprising, for each source room in a set of source rooms including the mirror room determined in the immediately preceding iteration: determining a set of mirror boundaries of the source room; determining a mirror room by mirroring the source room about each mirror boundary in the set of mirror boundaries; determining a mirror sound source by mirroring a source sound source about the mirror boundary, wherein the source sound source is a mirror sound source of the source room, the mirroring having a mirroring direction from the source room to the mirror room, the step of determining the set of mirror boundaries including selecting a boundary of the source room according to a selection criterion, the selection criterion including: a requirement that a first mirroring direction of a candidate boundary of the source room must not be in an opposite direction to any of the directions of any previous mirroring that resulted in the source room for the candidate boundary to be included in the set of mirror boundaries; a requirement that the first mirroring direction must not be an exclusion direction, the exclusion direction depending on a boundary of the first room that was the axis of the mirroring that resulted in the source room; a requirement that the first mirroring direction must not be in the same direction as any of the mirroring directions of any previous mirroring that resulted in the source room, except for the mirroring direction of the mirroring that resulted in the source room in the immediately preceding iteration, for the candidate boundary to be included in the set of mirror boundaries. Claim 2 The method according to claim 1, wherein the selection criterion includes the requirement that, for the candidate boundary to be included in the set of mirror boundaries when the second mirroring direction of any previous mirroring that brought about the source room is perpendicular to the exclusion direction and the mirroring direction of the first room that brought about the source room, the first mirroring direction must be the same as the second mirroring direction.

3. The first room has a pair of reference mirroring directions that are opposite to each other, and the selection criterion includes the requirement that, for the candidate boundary to be included in the set of mirror boundaries when the second mirroring direction of any previous mirroring that brought about the source room is a direction belonging to the associated pair of reference mirroring directions, the first mirroring direction must be the same as the second mirroring direction. The method according to claim 1.

4. The method according to any one of claims 1 to 3, wherein, for the first iteration, the first room is designated as the source room of the set of source rooms for the first iteration.

5. The method according to any one of claims 1 to 4, wherein all boundaries of the first room are included in the set of mirror boundaries for the first iteration.

6. Each boundary of the first room is associated with an attenuation coefficient, and the method includes the step of obtaining a combined attenuation coefficient for each mirror sound source by combining the attenuation coefficients of all boundaries included in the mirroring that brought about the mirror room including the mirror sound source. The method according to any one of claims 1 to 5.

7. The method according to claim 6, wherein the selection criterion includes the requirement that, for the candidate boundary to be included in the set of mirror boundaries, the result of combining the combined attenuation coefficient of the source sound source with the attenuation coefficient of the candidate boundary must show attenuation less than a threshold value.

8. The method according to claim 6 or 7, wherein the combined attenuation coefficient is frequency-dependent.

9. The method according to any one of claims 6 to 8, wherein the attenuation coefficient of an acoustically non-reflective boundary indicates complete attenuation.

10. The method further includes the step of rendering an audio signal for a listening position in the first room, the audio signal including at least one audio component representing audio from at least one mirror sound source reaching the listening position, the method according to any one of claims 1 to 9.

11. The set of mirror boundaries includes all boundaries that meet the selection criteria, the method according to any one of claims 1 to 10.

12. The iteration is performed a predetermined number of times, the method according to any one of claims 1 to 11.

13. The first room is a hyper-rectangular parallelepiped, the method according to any one of claims 1 to 12.

14. An apparatus for determining a virtual sound source representing a reflection of a first sound source in a first room, the apparatus comprising: a receiver that receives data representing a boundary of the first room and a sound source position of the first sound source in the first room; a processing circuit that repeatedly determines the virtual sound source as a mirrored sound source by performing sound source mirroring of the sound source determined in a previous iteration, each iteration including, for each source room in a set of source rooms including a mirror room determined in the immediately preceding iteration, a step of determining a set of mirror boundaries of the source room; a step of determining a mirror room by mirroring the source room about each mirror boundary in the set of mirror boundaries, the mirroring having a mirroring direction from the source room to the mirror room; a step of determining a mirror sound source by mirroring a source sound source about the mirror boundary, the source sound source being a mirror sound source of the source room, and the mirroring having a mirroring direction from the source room to the mirror room; the step of determining the set of mirror boundaries includes a step of selecting a boundary of the source room according to a selection criterion, the selection criterion including: a requirement that for a candidate boundary of the source room to be included in the set of mirror boundaries, a first mirroring direction of the candidate boundary must not be opposite to the direction of any previous mirroring that brought about the source room; a requirement that for the candidate boundary to be included in the set of mirror boundaries, the first mirroring direction must not be an exclusion direction, the exclusion direction depending on a boundary of the first room that was the axis of the mirroring that brought about the source room. An apparatus, including a requirement that, for the candidate boundary to be included in the set of mirror boundaries, the first mirroring direction must not be the same as the mirroring direction of any of the previous mirrorings that resulted in the source room, except for the mirroring direction of the mirroring that generated the source room in the immediately previous iteration. **Claim 15**: A computer program including computer program code means for performing all the steps of the method according to claims 1 to 13 when the computer program is executed on a computer.

Images