Method and system for handling local transitions between listening positions in a virtual reality environment - Patents.com

Abstract

To provide a method for rendering an audio signal in a virtual reality environment.SOLUTION: A method comprises: rendering an origin audio signal of an audio source (311, 312, 313) from an origin source position on an origin sphere around an origin listening position (301) of a listener; determining that the listener moves from the origin listening position (301) to a destination listening position (302); determining a destination source position of the audio source (311, 312, 313) on a destination sphere around the destination listening position (302) based on the origin source position, and determining a destination audio signal of the audio source (311, 312, 313) based on the origin audio signal; and rendering the destination audio signal of the audio source (311, 312, 313) from the destination source position on the destination sphere around the destination listening position (302).SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application No. 62 / 599,848 (Docket No. D17086USP1), filed December 18, 2017, and European Application No. 17208087.1 (Docket No. D17086EP), filed December 18, 2017, the contents of which are incorporated herein by reference.

[0002] Technical Field This paper is concerned with handling transitions between auditory viewports and / or listening positions in a virtual reality (VR) rendered environment in an efficient and consistent manner. [Background technology]

[0003] Virtual reality (VR), augmented reality (AR) and mixed reality (MR) applications are rapidly evolving to include increasingly sophisticated acoustic models of sound sources and scenes that can be enjoyed from different viewpoints / perspectives or listening positions. Two different classes of flexible audio representations may be used for e.g. VR applications: sound field representations and object-based representations. Sound field representations are physically based methods that encode the wavefront incident on the listening position. For example, methods such as B-format or Higher Order Ambisonics (HOA) represent the spatial wavefront using spherical harmonic decomposition. Object-based methods represent complex auditory scenes as a collection of single elements that contain audio waveforms or signals and possibly time-varying associated parameters or metadata.

[0004] Enjoying VR, AR and MR applications may include users experiencing different auditory perspectives or viewpoints. For example, room-based virtual reality may be provided based on mechanisms that use six degrees of freedom (DoF). Figure 1 shows an example of 6 DoF interaction showing translational movements (forward / backward, up / down and left / right) and rotational movements (pitch, yaw, roll). Unlike 3 DoF spherical video experiences that are restricted to head rotation, content created for 6 DoF interaction allows navigation within the virtual environment (e.g., physically walking around a room) in addition to head rotation. This can be achieved based on position trackers (e.g., camera-based) and orientation trackers (e.g., gyroscopes and / or accelerometers). 6 DoF tracking technology may be available on high-end mobile VR platforms (e.g., Google Tango) as well as on high-end mobile VR platforms (e.g., PlayStation®VR, Oculus Rift, HTC Vive). The user's experience of the directionality and spatial extent of a sound or audio source is crucial to the realism of the 6 DoF experience, especially the experience of navigation within a scene and around virtual audio sources.

[0005] Available audio rendering systems (e.g. MPEG-H 3D renderers) are typically limited to rendering 3 DoF (i.e., rotational movements of the audio scene caused by the listener's head movements). Translational changes of the listener's listening position and the associated DoF typically cannot be handled by such renderers. Summary of the Invention [Problem to be solved by the invention]

[0006] This paper is addressed to the technical problem of providing a resource-efficient method and system for handling translation in the context of audio rendering. [Means for solving the problem]

[0007] According to an aspect, a method for rendering an audio signal in a virtual reality rendering environment is described. The method includes rendering an origin audio signal of an audio source from an origin source position on an origin sphere around an origin listening position of a listener. Furthermore, the method includes determining that a listener moves from the origin listening position to an end listening position. Furthermore, the method includes determining an end source position of the audio source on an end sphere around the end listening position based on the origin source position. The end source position of the audio source on the end sphere may be determined by a projection of the origin source position on the origin sphere onto the end sphere. The projection may be, for example, a perspective projection with respect to the end listening position. The origin sphere and the end sphere may have the same radius. For example, both spheres may correspond to a unit sphere in the context of the rendering, for example a sphere of radius 1 meter. Furthermore, the method includes determining an end audio signal of an audio source based on the origin audio signal. The method further includes rendering an end audio signal of the audio source from an end source position on an end sphere around the end listening position.

[0008] According to a further aspect, a virtual reality audio renderer for rendering an audio signal in a virtual reality rendering environment is described. The audio renderer is configured to render an origin audio signal of an audio source from an origin source position on an origin sphere around an origin listening position of a listener. The virtual reality audio renderer is further configured to determine a listener moving from the origin listening position to an end listening position. The virtual reality audio renderer is further configured to determine an end source position of the audio source on an end sphere around the end listening position based on the origin source position. The virtual reality audio renderer is further configured to determine an end audio signal of the audio source based on the origin audio signal. The virtual reality audio renderer is further configured to render an end audio signal of the audio source from an end source position on an end sphere around the end listening position.

[0009] According to another aspect, a method for generating a bitstream is described, the method including: determining an audio signal of at least one audio source; determining position data relating to a position of the at least one audio source within a rendering environment; determining environment data indicative of audio propagation characteristics of audio within the rendering environment; and inserting the audio signal, the position data and the environment data into the bitstream.

[0010] According to a further aspect, an audio encoder is described that is configured to generate a bitstream indicative of: an audio signal of at least one audio source; a position of the at least one audio source within a rendering environment; and environment data indicative of audio propagation characteristics of the audio within the rendering environment.

[0011] According to another aspect, a bitstream is described that is indicative of: an audio signal of at least one audio source; a position of the at least one audio source within a rendering environment; and environment data indicative of audio propagation characteristics of the audio within the rendering environment.

[0012] According to a further aspect, a virtual reality audio renderer is described for rendering an audio signal in a virtual reality rendering environment. The audio renderer includes rendering an audio signal of an audio source from a source position on a sphere around a listening position of a listener in the virtual reality rendering environment. The virtual reality audio renderer further includes a pre-processing unit configured to determine a new listening position of the listener in the virtual reality rendering environment. The pre-processing unit is further configured to update the audio signal and the source position of the audio source with respect to the sphere around the new listening position. The 3D audio renderer is configured to render an updated audio signal of the audio signal from the updated source position on the sphere around the new listening position.

[0013] According to a further aspect, a software program is described, which may be adapted for execution on a processor and adapted to perform the method steps outlined herein when executed on the processor.

[0014] According to another aspect, a storage medium is described, the storage medium may have a software program adapted for execution on a processor and adapted to perform the method steps outlined herein when executed on the processor.

[0015] According to a further aspect, a computer program product is described that may include executable instructions for performing the method steps outlined herein when executed on a computer.

[0016] The methods and systems, including the preferred embodiments, outlined in this patent application may be used alone or in combination with other methods and systems disclosed herein. Furthermore, all aspects of the methods and systems outlined in this patent application may be combined in any manner. In particular, the features of the claims may be combined with each other in any manner. [Brief description of the drawings]

[0017] The invention is described below in an exemplary manner with reference to the accompanying drawings, in which: [Figure 1a] 1 illustrates an exemplary audio processing system that provides 6DoF audio. [Figure 1b] 1 illustrates an example situation within a 6DoF audio and / or rendering environment. [Figure 1c] 1 illustrates an exemplary transition from a source audio scene to a destination audio scene. [Diagram 2] 1 illustrates an exemplary scheme for determining a spatial audio signal during a transition between different audio scenes. [Diagram 3] An exemplary audio scene is shown. [Figure 4a] Demonstrates remapping of audio sources in response to changes in listening position within the audio scene. [Figure 4b] 2 illustrates an exemplary distance function. [Figure 5a] Illustrates an audio source with a non-uniform directional profile. [Figure 5b] 3 illustrates an exemplary directivity function for an audio source. [Figure 6] An example audio scene with acoustically significant obstructions is shown. [Figure 7] Indicates the listener's field of vision and focus of interest. [Figure 8] Demonstrates the treatment of ambient audio when the listening position changes within the audio scene. [Figure 9a] 3 shows a flowchart of an exemplary method for rendering a 3D audio signal during a transition between different audio scenes. [Figure 9b] 4 shows a flowchart of an exemplary method for generating a bitstream for transitions between different audio scenes. [Figure 9c] 1 shows a flowchart of an exemplary method for rendering a 3D audio signal during a transition in an audio scene. [Figure 9d] 1 shows a flowchart of an example method for generating a bitstream for a local transition. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] As outlined above, this paper is concerned with the efficient provision of 6DoF in a 3D (three-dimensional) audio environment. FIG. 1a shows a block diagram of an exemplary audio processing system 100. An acoustic environment 110, such as a stadium, includes a variety of different audio sources 113. Exemplary audio sources 113 in a stadium are individual spectators, stadium speakers, players on the field, etc. The acoustic environment 110 may be subdivided into different audio scenes 111, 112. As an example, a first audio scene 111 may correspond to a home team cheering block and a second audio scene 111 may correspond to a guest team cheering block. Depending on where the listener is located in the audio environment, the listener perceives an audio source 113 from the first audio scene 111 or an audio source from the second audio scene 112.

[0019] The different audio sources 113 of the audio environment 110 may be captured by means of an audio sensor 120, in particular by means of a microphone array. In particular, said one or more audio scenes 111, 112 of the audio environment 110 may be described by means of a multi-channel audio signal, one or more audio objects and / or a Higher Order Ambisonics (HOA) signal. In the following, it is assumed that the audio source 113 is associated with audio data captured by the audio sensor 120, where the audio data indicates the audio signal and the position of the audio source 113 as a function of time (at a certain sampling rate, for example 20 ms).

[0020] A 3D audio renderer, such as an MPEG-H 3D audio renderer, typically assumes that a listener is located at a particular listening position within the audio scenes 111, 112. Audio data for various audio sources 113 of the audio scenes 111, 112 is typically provided under the assumption that the listener is located at this particular listening position. The audio encoder 130 may comprise a 3D audio encoder 131 configured to encode audio data for the audio sources 113 of one or more of the audio scenes 111, 112.

[0021] Furthermore, virtual reality (VR) metadata may be provided, which allows a listener to change the listening position within the audio scene 111, 112 and / or move between different audio scenes 111, 112. The encoder 130 may comprise a metadata encoder 132 configured to encode the VR metadata. The encoded VR metadata and the encoded audio data of the audio source 113 may be combined in a combination unit 133 to provide a bitstream 140 indicative of the audio data and the VR metadata. The VR metadata may for example include environmental data describing acoustic characteristics of the audio environment 110.

[0022] The bitstream 140 may be decoded using a decoder 150 to provide (decoded) audio data and (decoded) VR metadata. An audio renderer 160 for rendering audio in a rendering environment 180 allowing 6DoF may comprise a preprocessing unit 161 and a (conventional) 3D audio renderer 162 (such as MPEG-H 3D audio). The preprocessing unit 161 may be configured to determine a listening position 182 of a listener 181 in the listening environment 180. The listening position 182 may indicate an audio scene 111 in which the listener 181 is located. Furthermore, the listening position 182 may indicate an exact position within the audio scene 111. The preprocessing unit 161 may further be configured to determine a 3D audio signal for the current listening position 182 based on the (decoded) audio data and possibly based on the (decoded) VR metadata. The 3D audio signal may then be rendered using a 3D audio renderer 162 .

[0023] It should be noted that the concepts and techniques described herein may be specified in a frequency-varying manner, may be defined globally or in an object / media-dependent manner, may be applied directly in the spectral or time domain, and / or may be hard-coded into the VR renderer 160 or specified via a corresponding input interface.

[0024] FIG. 1b shows an exemplary rendering environment 180. A listener 181 may be located in an origin audio scene 111. For rendering purposes, the audio sources 113, 194 may be assumed to be located at different rendering positions on a (unit) sphere 114 around the listener 181. The rendering positions of the different audio sources 113, 194 may change over time (according to a given sampling rate). Different situations may arise in the VR rendering environment 180: the listener 181 may perform a global transition 191 from the origin audio scene 111 to the destination audio scene 112. Alternatively or additionally, the listener 181 may perform a local transition 192 to a different listening position 182 in the same audio scene 111. Alternatively or additionally, the audio scene 111 may exhibit acoustically significant environmental features (e.g. walls), which may be described using environmental data 193 and should be taken into account when a change in the listening position 182 occurs. Alternatively or additionally, the audio scene 111 may contain one or more ambient sound audio sources 194 (e.g. for background noise), which should be taken into account when a change in listening position 182 occurs.

[0025] FIG. 1c shows audio sources 113A1 to A n From the origin audio scene 111 having audio sources 113B1 to B m 1c shows an exemplary global transition 191 to a destination audio scene 112 having a local transition 192 between listening positions "B" and "C". The audio sources 113 may be characterized by corresponding inter-location object properties (coordinates, directivity, distance sound attenuation function, etc.). The global transition 191 may be performed within a transition time interval (e.g., within 5 seconds, 1 second or less). The listening position 182 in the origin scene 111 at the beginning of the global transition 191 is marked with "A". Furthermore, the listening position 182 in the destination scene 112 at the end of the global transition 191 is marked with "B". Furthermore, FIG. 1c shows a local transition 192 in the destination scene 112 between listening positions "B" and "C".

[0026] 2 illustrates a global transition 191 from an origin scene 111 (or origin viewport) to an end scene 112 (or end viewport) during a transition time interval t. Such a transition 191 may occur when a listener 181 switches between different scenes or viewports 111, 112, for example in a stadium. At an intermediate point in time 213, the listener 181 may be located at an intermediate position between the origin scene 111 and the end scene 112. The 3D audio signal 203 rendered at the intermediate point and / or at the intermediate point in time 213 may be generated by extracting the audio sources 113A1 to 113A2 of the origin scene 111, taking into account the sound propagation of each audio source 113. n and the audio sources 113B1 to B m However, this may lead to a relatively high amount of computation (especially in the case of a relatively large number of audio sources 113).

[0027] At the beginning of the global transition 191, the listener 181 may be positioned at the origin listening position 201. During the entire transition 191, the 3D origin audio signal A G may be generated, where the origin audio signal only depends on the audio sources 113 of the origin scene 111 (and not on the audio sources 113 of the destination scene 112). Furthermore, at the beginning of the global transition 191, it may be fixed that the listener 181 will arrive at the destination listening position 202 in the destination scene 112 at the end of the global transition 191. During the entire transition 191, the 3D destination audio signal B G may be generated for the destination listening position 202, where the destination audio signal only depends on the audio sources 113 of the destination scene 112 (and not on the audio sources 113 of the source scene 111).

[0028] To determine an intermediate audio signal 203 at an intermediate position and / or intermediate time point 213 during the global transition 191, the origin audio signal at the intermediate time point 213 may be combined with the end audio signal at the intermediate time point 213. In particular, a fade-out factor or gain derived from a fade-out function 211 may be applied to the origin audio signal. The fade-out function 211 may be such that a fade-out factor or gain "a" decreases within increasing distance of the intermediate position from the origin scene 111. Furthermore, a fade-in factor or gain derived from a fade-in function 212 may be applied to the end audio signal. The fade-in function 212 may be such that a fade-in factor or gain "b" increases with decreasing distance of the intermediate position from the end scene 112. An exemplary fade-out function 211 and an exemplary fade-in function 212 are shown in FIG. 2. The intermediate audio signal may then be given by a weighted sum of the source and destination audio signals, where the weights correspond to the fade-out and fade-in gains, respectively.

[0029] Thus, a fade-in function or curve 212 and a fade-out function or curve 211 may be defined for the global transition 191 between different 3DoF viewports 201, 202. The functions 211, 212 may be applied to pre-rendered virtual objects or 3D audio signals representing the origin audio scene 111 and the destination audio scene 112. In this way, a consistent audio experience may be provided during the global transition 191 between the different audio scenes 111, 112 with degraded VR audio rendering calculations.

[0030] intermediate position x i The intermediate audio signal 203 at may be determined using linear interpolation of the source and destination audio signals. The strength F of the audio signal is F(x i )=a*F(A G )+(1-a)*F(B GThe factors "a" and "b=1-a" may be given by a norm function a=a() that depends on the start listening position 201, the end listening position 202 and the intermediate positions.

[0031] As an alternative to a function, a look-up table a=[1,...,0] may be provided for the various intermediate positions.

[0032] Additional effects may be taken into account during the global transition 191, for example the Doppler effect and / or reverberation. The functions 211, 212 may be adapted by the content provider, for example to reflect artistic intent. Information about the functions 211, 212 may be included as metadata in the bitstream 140. Thus, the encoder 130 may be configured to provide information about the fade-in function 212 and / or the fade-out function 211 as metadata in the bitstream 140. Alternatively or additionally, the audio renderer 160 may apply the functions 211, 212 stored in the audio renderer 160.

[0033] A flag may be communicated from the listener to the renderer 160, in particular to the VR pre-processing unit 161, to indicate to the renderer 160 that a global transition 191 is being performed from the origin scene 111 to the destination scene 112. The flag may trigger the audio processing described herein to generate intermediate audio signals during the transition phase. The flag may be signaled explicitly or implicitly through related information (e.g. via the coordinates of a new viewport or listening position 202). The flag may be sent from any data interface side (e.g. server / content, user / scene, auxiliary). Together with the flag, the origin audio signal A G and destination audio signal B GFor example, the IDs of one or more audio objects or audio sources may be provided. Alternatively, a request may be provided to the renderer 160 to compute the source audio signal and / or the destination audio signal.

[0034] Thus, a VR renderer 160 with a pre-processing unit 161 for a 3DoF renderer 162 is described that enables 6DoF functionality in a resource-efficient manner. The pre-processing unit 161 allows the use of a standard 3DoF renderer 162, such as an MPEG-H 3D audio renderer. The VR pre-processing unit 161 processes pre-rendered virtual audio objects A, B, C, D, D, E ... G and B. G The present invention may be configured to efficiently perform the computations for the global transition 191 using a pre-rendered virtual object A. By utilizing only two pre-rendered virtual objects during the global transition 191, the amount of computation is reduced. Each virtual object may contain multiple audio signals for multiple audio sources. Furthermore, during the transition 191, the pre-rendered virtual audio object A G and B. G The bitrate requirements may be reduced since only the first bit may be provided in the bitstream 140. Furthermore, processing delays may be reduced.

[0035] 3DoF functionality may be provided for all intermediate positions along the global transition trajectory, which may be achieved by overlapping the origin and destination audio objects with fade-out / fade-in functions 211, 212. Furthermore, additional audio objects may be rendered and / or additional audio effects may be included.

[0036] 3 illustrates an exemplary local transition 192 from an origin listening position B 301 to an end listening position C 302 within the same audio scene 111. The audio scene 111 includes different audio sources or objects 311, 312, 313. The different audio sources or objects 311, 312, 313 may have different directivity profiles 332. Furthermore, the audio scene 111 may have environmental characteristics, in particular one or more obstacles, that have an effect on the propagation of audio within the audio scene 111. The environmental characteristics may be described using the environmental data 193. Furthermore, the relative distances 321, 322 of the audio objects 311 to the listening positions 301, 302 may be known.

[0037] 4a and 4b show a scheme for handling the effect of a local transition 192 on the intensity of different audio sources or objects 311, 312, 313. As outlined above, the audio sources 311, 312, 313 of an audio scene 111 are typically assumed by the 3D audio renderer 162 to be located on a sphere 114 around the listening position 301. Thus, at the beginning of the local transition 192, the audio sources 311, 312, 313 may be located on an origin sphere 114 around the origin listening position 301, and at the end of the local transition 192, the audio sources 311, 312, 313 may be located on an end sphere 114 around the end listening position 302. The radius of the sphere 114 may be independent of the listening position. That is, the origin sphere 114 and the end sphere 114 may have the same radius. For example, the spheres may be unit spheres (e.g., in the context of rendering). In one example, the spheres may have a radius of 1 meter.

[0038] The audio sources 311, 312, 313 may be remapped (e.g. geometrically remapped) from the origin sphere 114 to the destination sphere 114. For this purpose, a ray may be considered that goes from the destination listening position 302 to the source positions of the audio sources 311, 312, 313 on the origin sphere 114. The audio sources 311, 312, 313 may be positioned at the intersection of the ray with the destination sphere 114.

[0039] The intensity F of the audio sources 311, 312, 313 on the end sphere 114 is typically different from their intensity on the origin sphere 114. The intensity F may be modified using an intensity gain function or distance function 415 that provides a distance gain 410 as a function of the distance 420 of the audio sources 311, 312, 313 from the listening positions 301, 302. The distance function 415 typically indicates a cutoff distance 421 beyond which the zero distance gain 410 is applied. The origin distance 321 of the audio source 311 to the origin listening position 301 provides the origin gain 411. For example, the origin distance 321 may correspond to the radius of the origin sphere 114. Furthermore, the end distance 322 of the audio source 311 to the end listening position 302 provides the end gain 412. For example, the end distance 322 may be the distance from the end listening position 302 to the source positions of the audio sources 311, 312, 313 on the origin sphere 114. The strength F of the audio source 311 may be rescaled using the origin gain 411 and the end gain 412 to give the strength F of the audio source 311 on the end sphere 114. In particular, the strength F of the origin audio signal of the audio source 311 on the origin sphere 114 may be divided by the origin gain 411 and multiplied by the end gain 412 to give the strength F of the end audio signal of the audio source 311 on the end sphere 114.

[0040] Thus, the position of the audio source 311 after the local transition 192 is given (e.g., using a geometric transformation) by C i =source_remap_function(B i , C). Furthermore, the intensity of the audio source 311 after the local transition 192 may be determined as F(C i )=F(B i)*distance_function(B i ,C i , C). Thus, the distance attenuation may be modeled by a corresponding intensity gain given by the distance function 415.

[0041] 5a and 5b show an audio source 312 with a non-uniform directivity profile 332. The directivity profile may be defined using a directivity gain 510 that indicates gain values ​​for various directions or directivity angles 520. In particular, the directivity profile of the audio source 312 may be defined using a directivity gain function 515 that indicates the directivity gain 510 as a function of the directivity angle 520 (where the angle 520 may range from 0° to 360°). For a 3D audio source 312, the directivity angle 520 is typically a two-dimensional angle including an azimuth angle and an elevation angle. Thus, the directivity gain function 515 is typically a two-dimensional function of the two-dimensional directivity angle 520.

[0042] The directional profile 332 of the audio source 312 may be taken into account in the context of the local transition 192 by determining an origin directivity angle 521 of an origin ray between the audio source 312 and the origin listening position 301 (the audio source is positioned on the origin sphere 114 around the origin listening position 301) and an end directivity angle 522 of an end ray between the audio source 312 and the end listening position 302 (the audio source is positioned on the end sphere 114 around the end listening position 302). Using a directional gain function 515 of the audio source 312, the origin directivity gain 511 and the end directivity gain 512 may be determined as function values ​​of the directional gain function 515 for the origin directivity angle 521 and the end directivity angle 522, respectively (see FIG. 5b). The strength F of the audio source 312 at the origin listening position 301 may then be divided by the origin directivity gain 511 and multiplied by the end directivity gain 512 to determine the strength F of the audio source 312 at the end listening position 302 .

[0043] Thus, the sound source directivity may be parameterized by a directional factor or gain 510, which is indicated by a directional gain function 515. The directional gain function 515 may indicate the strength of an audio source 312 at some distance as a function of the angle 520 relative to the listening position 301, 302. The directional gain 510 may be defined as a ratio to the gain of an audio source 312 at the same distance and with the same total power, which radiates uniformly in all directions. The directional profile 332 may be parameterized by a set of gains 510 corresponding to vectors emanating from the center of the audio source 312 and terminating at points distributed on a unit sphere around the center of the audio source 312. The directional profile 332 of the audio source 312 may depend on the use case scenario and available data (e.g., a uniform distribution for 3D flight cases, a flattened distribution for 2D+ use cases, etc.).

[0044] The resulting audio intensity of the audio source 312 at the end listening position 302 is F(C i )=F(B i )*Distance_function()*Directivity_gain_function(C i ,C,Directivity_parametrization), where Directivity_gain_function depends on the directional profile 332 of the audio source 312. Distance_function() takes into account the modified intensity caused by the change in distance 321, 322 of the audio source 312 due to the transition of the audio source 312.

[0045] 6 illustrates an example obstacle 603 that may need to be taken into account in the context of a local transition 192 between different listening positions 301, 302. In particular, an audio source 313 may be hidden behind an obstacle 603 at an end listening position 302. The obstacle 603 may be described by environment data 193 that includes a set of parameters, such as the spatial dimensions of the obstacle 603 and an obstacle attenuation function that indicates the attenuation of sound caused by the obstacle 603.

[0046] The audio source 313 may indicate an obstacle-free distance 602 (OSD) to the end listening position 302. The OFD 602 ​​may indicate the length of the shortest path between the audio source 313 and the end listening position 302 that does not pass through obstacles 603. Additionally, the audio source 313 may indicate a going-through distance 601 (GHD) to the end listening position 302. The GHD 601 may indicate the length of the shortest path between the audio source 313 and the end listening position 302 that typically passes through obstacles 603. The obstacle attenuation function may be a function of the OFD 602 ​​and the GHD 601. Additionally, the obstacle attenuation function may be a function of the strength F(B i ) may be a function of

[0047] Audio source C at end listening position 302 i The intensity of may be a combination of sounds from the audio source 313 passing around the obstacle 603 and sounds from the audio source 313 passing through the obstacle 603.

[0048] Thus, the VR Renderer 160 may be provided with parameters to control the effect of the environment geometry and media. The obstacle geometry / media data 193 or parameters may be provided by the content provider and / or the encoder 130. The audio intensity of the audio source 313 is: F(C i )=F(B i)*Distance_function(OFD)*Directivity_gain_function(OFD)+Obstacle_attenuation_function(F(Bi),OFD,GHD). The first term corresponds to the contribution of sound going around the obstacle 603. The second term corresponds to the contribution of sound going through the obstacle 603.

[0049] The minimum Obstacle Free Distance (OFD) 602 may be determined using an A* Dijkstra's pathfinding algorithm and may be used to control direct sound attenuation. The Ground Path Distance (GHD) 601 may be used to control reverberation and distortion. Alternatively or additionally, a ray casting technique may be used to describe the effect of obstacles 603 on the intensity of the audio source 313.

[0050] FIG. 7 illustrates an exemplary field of view 701 of a listener 181 located at an end listening position 302. Additionally, FIG. 7 illustrates an exemplary focus of attention 702 of a listener located at an end listening position 302. The field of view 701 and / or focus of interest 702 may be used to enhance (e.g., amplify) audio coming from audio sources within the field of view 701 and / or focus of interest 702. The field of view 701 may be considered a user-driven effect and may be used to enable sound enhancers for audio sources 311 associated with the user's field of view 701. In particular, a "cocktail party effect" simulation may be performed by removing frequency tiles from background audio sources to improve the intelligibility of speech signals associated with audio sources 311 within the listener's field of view 701. The attention focus 702 may be viewed as a content-driven effect and may be used to enable sound enhancers for audio sources 311 associated with a content region of interest (e.g., to draw the user's attention to look and / or move in the direction of the audio sources 311).

[0051] The audio intensity of audio source 311 is: F(Bi )=Field_of_view_function(C,F(B i ), Field_of_view_data), where Field_of_view_function describes the modification applied to the audio signal of the audio sources 311 that are within the field of view 701 of the listener 181. Furthermore, the audio intensity of the audio sources that are within the listener's focus of interest 702 may be modified as: F(B i )=Attention_focus_function(F(B i ), Attention_focus_data), where attention_focus_function describes the modification applied to the audio signal of the audio source 311 that is within the focus of interest 702.

[0052] The functions described herein for handling the transition of the listener 181 from an origin listening position 301 to an end listening position 302 may be applied in a similar manner to position changes of the audio sources 311, 312, 313.

[0053] Thus, this paper describes efficient means for computing the coordinates and / or audio intensity of virtual audio objects or audio sources 311, 312, 313 representing the local VR audio scene 111 at any listening position 301, 302. The coordinates and / or intensity may be determined taking into account sound source distance attenuation curves, sound source orientation and directivity, environmental geometry / medium effects and / or "field of view" and "focus of interest" data for additional audio signal enhancement. The described methods may significantly reduce the amount of computation by only performing the calculations when the listening positions 301, 302 and / or audio objects / sources 311, 312, 313 change position.

[0054] Additionally, this document describes concepts for specifying distance, directivity, geometric functions, processing and / or signaling mechanisms for the VR Renderer 160. Additionally, concepts for minimum "obstruction-free distance" for controlling direct sound attenuation and "pass distance" for controlling reverberation and distortion are described. Additionally, concepts for source directivity parameterization are described.

[0055] Fig. 8 illustrates the handling of ambient sound sources 801, 802, 803 in the context of local transitions 192. Specifically, Fig. 8 illustrates three different ambient sound sources 801, 802, 803, where the ambient sounds may be attributed to point audio sources. An ambient sound flag may be provided to the pre-processing unit 161 to indicate that the point audio source 311 is an ambient sound audio source 801. The processing during local and / or global transitions of the listening positions 301, 302 may depend on the value of the ambient sound flag.

[0056] In the context of the global transition 191, the ambient sound source 801 may be treated like a normal audio source 311. Figure 8 illustrates the local transition 192. The positions of the ambient sound sources 801, 802, 803 may be copied from the origin sphere 114 to the destination sphere 114, thereby giving the positions of the ambient sound sources 811, 812, 813 at the destination listening position 302. Furthermore, if the environmental conditions remain unchanged, the intensity of the ambient sound source 801 may be kept unchanged. That is, F(C Ai )=F(B Ai On the other hand, in the case of an obstacle 603, the intensity of the surrounding sound sources 803, 813 is calculated using the obstacle attenuation function, e.g., F(C Ai )=F(B Ai )*Distance_function Ai (OFD)+Obstacle_attenuation_function(F(B Ai ), OFD,GHD).

[0057] FIG. 9a shows a flowchart of an exemplary method 900 for rendering audio in a virtual reality rendering environment 180. The method 900 may be performed by a VR audio renderer 160. The method 900 includes rendering 901 an origin audio signal of an audio source 113 of an origin audio scene 111 from an origin source position on a sphere 114 around a listening position 201 of a listener 181. The rendering 901 may be performed using a 3D audio renderer 162 that may be limited to handling only 3DoF, and in particular limited to handling only rotational movements of the head of the listener 181. In particular, the 3D audio renderer 162 is not configured to handle translational movements of the listener's head. The 3D audio renderer 162 may include or be an MPEG-H audio renderer.

[0058] It should be noted that the phrase "rendering an audio signal of the Audio Source 113 from a particular source position" indicates that a listener perceives the audio signal as coming from that particular source position. This phrase should not be understood as a limitation on how the audio signal is actually rendered. A variety of different rendering techniques may be used to "render an audio signal from a particular source position", i.e., to provide the listener 181 with the perception that the audio signal is coming from a particular source position.

[0059] Further, the method 900 includes determining 902 that the listener 181 moves from a listening position 201 in the origin audio scene 111 to a listening position 202 in a different destination audio scene 112. Thus, a global transition 191 from the origin audio scene 111 to the destination audio scene 112 may be detected. In this context, the method 900 may include receiving an indication that the listener 181 moves from the origin audio scene 111 to the destination audio scene 112. The indication may include or be a flag. The indication may be communicated from the listener 181 to the VR audio renderer 160, for example, via a user interface of the VR audio renderer 160.

[0060] Typically, the origin audio scene 111 and the destination audio scene 112 each include one or more distinct audio sources 113. In particular, the origin audio signals of the one or more origin audio sources 113 may not be audible in the destination audio scene 112 and / or the destination audio signals of the one or more destination audio sources 113 may not be audible in the origin audio scene 111.

[0061] The method 900 may include applying 903 a fade-out gain to the origin audio signal (in response to determining that the global transition 191 to the new destination audio scene 112 is to be performed) to determine a modified origin audio signal. Additionally, the method 900 may include rendering 904 the modified origin audio signal of the origin audio source 113 from an origin source position on a sphere 114 around the listener positions 201, 202 (in response to determining that the global transition 191 to the new destination audio scene 112 is to be performed).

[0062] Thus, a global transition 191 between different audio scenes 111, 112 may be performed by gradually fading out the origin audio signal of the one or more origin audio sources 113 of an origin audio scene 111. As a result, a computationally efficient and acoustically consistent global transition 191 between different audio scenes 111, 112 is provided.

[0063] It may be determined that the listener 181 moves from the origin audio scene 111 to the destination audio scene 112 during a transition time interval, where the transition time interval typically has a duration (e.g., 2s, 1s, 500ms or less). The global transition 191 may be performed progressively within the transition time interval. In particular, during the global transition 191, an intermediate time point 213 within the transition time interval may be determined (e.g., according to a sampling rate of 100ms, 50ms, 20ms or less). The fade-out gain may then be determined based on the relative position of the intermediate time point 213 within the transition time interval.

[0064] In particular, the transition time interval for the global transition 191 may be subdivided into a sequence of intermediate time points 213. For each intermediate time point 213 of the sequence of intermediate time points 213, a fade-out gain may be determined for modifying the origin audio signal of the one or more origin audio sources. Furthermore, at each intermediate time point 213 of the sequence of intermediate time points 213, the modified origin audio signal of the one or more origin audio sources 113 may be rendered from an origin source position on a sphere 114 around the listening positions 201, 202. In this way, an acoustically consistent global transition 191 may be performed in a computationally efficient manner.

[0065] The method 900 may include providing a fade-out function 211 indicative of a fade-out gain at various intermediate points 213 within the transition time interval, where the fade-out function 211 is typically such that the fade-out gain decreases with the progressing intermediate points 213, thereby providing a smooth global transition 191 to the end audio scene 112. In particular, the fade-out function 211 may be such that the origin audio signal remains unmodified at the beginning of the transition time interval, the origin audio signal becomes increasingly attenuated at the progressing intermediate points 213, and / or the origin audio signal is completely attenuated at the end of the transition time interval.

[0066] The origin source position of the origin audio source 113 on the sphere 114 around the listening positions 201, 202 may be maintained (in particular during said entire transition time interval) as the listener 181 moves from the origin audio scene 111 to the destination audio scene 112. Alternatively or additionally, it may be assumed that the listener 181 stays at the same listening position 201, 202 (during said entire transition time interval). In this way, the amount of computation for the global transition 191 between the audio scenes 111, 112 may be further reduced.

[0067] The method 900 may further include determining an end audio signal of the end audio source 113 of the end audio scene 112. Additionally, the method 900 may include determining an end source position on a sphere 114 around the listening positions 201, 202. Additionally, the method 900 may include applying a fade-in gain to the end audio signal to determine a modified end audio signal. The modified end audio signal of the end audio source 113 may then be rendered from the end source position on the sphere 114 around the listening positions 201, 202.

[0068] Thus, similar to fading out the origin audio signal of the one or more origin audio sources 113 of the origin scene 111, the end audio signal of one or more end audio sources 113 of the end scene 112 may be faded in, thereby providing a smooth global transition 191 between the audio scenes 111, 112.

[0069] As described above, the listener 181 may move from the originating audio scene 111 to the destination audio scene 112 during the transition time interval. A fade-in gain may be determined based on the relative position of the intermediate time points 213 within the transition time interval. In particular, a sequence of fade-in gains may be determined for a corresponding sequence of intermediate time points 213 during the global transition 191.

[0070] The fade-in gain may be determined using a fade-in function 212 indicating the fade-in gain at various intermediate points 213 within the transition time interval, where the fade-in function 212 is typically such that the fade-in gain increases with the progressing intermediate point 213. In particular, the fade-in function 212 may be such that the end audio signal is fully attenuated at the beginning of the transition time interval, becomes less attenuated at the progressing intermediate points 213 of the end audio signal, and / or the end audio signal remains unmodified at the end of the transition time interval, thereby providing a smooth global transition 191 between the audio scenes 111, 112 in a computationally efficient manner.

[0071] Similar to the origin source position of the origin audio source 113, the destination source position of the destination audio source 113 on the sphere 114 around the listening positions 201, 202 may be maintained, in particular during the entire transition time interval, when the listener 181 moves from the origin audio scene 111 to the destination audio scene 112. Alternatively or additionally, it may be assumed that the listener 181 stays at the same listening position 201, 202 (during the entire transition time interval). By doing so, the amount of computation for the global transition 191 between the audio scenes 111, 112 may be further reduced.

[0072] The fade-out function 211 and / or the fade-in function 212 may be derived from a bitstream representing the origin and / or destination audio signals. The bitstream 140 may be provided to the VR audio renderer 160 by the encoder 130. Thus, the global transition 191 may be controlled by a content provider. Alternatively or additionally, the fade-out function 211 and / or the fade-in function 212 may be derived from a storage unit of a virtual reality (VR) audio renderer 160 configured to render the origin and / or destination audio signals within a virtual reality rendering environment 180, thereby providing a reliable operation during the global transition 191 between the audio scenes 111, 112.

[0073] The method 900 may include sending an indication (e.g. a flag indicating so) to the encoder 130 that the listener 181 moves from the origin audio scene 111 to the destination audio scene 112. The encoder 130 may then be configured to generate a bitstream 140 indicating the origin audio signal and / or the destination audio signal. The indication enables the encoder 130 to selectively provide in the bitstream 140 the audio signals for the one or more audio sources 113 of the origin audio scene 111 and / or for the one or more audio sources 113 of the destination audio scene 112. Thus, providing an indication of an upcoming global transition 191 allows a reduction in the required bandwidth for the bitstream 140.

[0074] As already indicated above, the origin audio scene 111 may include a plurality of origin audio sources 113. Thus, the method 900 may include rendering a plurality of origin audio signals of a corresponding plurality of origin audio sources 113 from a plurality of different origin source positions on a sphere 114 around the listening positions 201, 202. Additionally, the method 900 may include applying a fade-out gain to the plurality of origin audio signals to determine a plurality of modified origin audio signals. Additionally, the method 900 may include rendering a plurality of modified origin audio signals of the origin audio source 113 from a corresponding plurality of different origin source positions on a sphere 114 around the listening positions 201, 202.

[0075] Similarly, the method 900 may include determining a plurality of destination audio signals of a corresponding plurality of destination audio sources 113 of the destination audio scene 112. Additionally, the method 900 may include determining a plurality of destination source positions on a sphere 114 around the listening positions 201, 202. Additionally, the method 900 may include applying a fade-in gain to the plurality of destination audio signals to determine a corresponding plurality of modified destination audio signals. Additionally, the method 900 includes rendering the plurality of modified destination audio signals of the plurality of destination audio sources 113 from a corresponding plurality of destination source positions on a sphere 114 around the listening positions 201, 202.

[0076] Alternatively or additionally, the origin audio signal rendered during the global transition 191 may be a superposition of audio signals of multiple origin audio sources 113. In particular, at the beginning of the transition time interval, the audio signals of (all) audio sources 113 of the origin audio scene 111 may be combined to give a combined origin audio signal. This origin audio signal may be modified with a fade-out gain. Furthermore, the origin audio signal may be updated with a certain sampling rate (e.g. 20 ms) during the transition time interval. Similarly, the destination audio signal may correspond to a combination of audio signals of multiple destination audio sources 113 (in particular, all destination audio sources 113). The combined destination audio source may then be modified during the transition time interval with a fade-in gain. By combining the audio signals of the origin audio scene 111 and the destination audio scene 112 respectively, the amount of computation may be further reduced.

[0077] Further, a virtual reality audio renderer 160 for rendering audio in the virtual reality rendering environment 180 is described. As outlined herein, the VR audio renderer 160 may have a pre-processing unit 161 and a 3D audio renderer 162. The virtual reality audio renderer 160 may be configured to render an origin audio signal of an origin audio source 113 of an origin audio scene 111 from an origin source position on a sphere 114 around a listening position 201 of a listener 181. Further, the VR audio renderer 160 is configured to determine that the listener 181 moves from the listening position 201 in the origin audio scene 111 to a listening position 202 in a different destination audio scene 112. Further, the VR audio renderer 160 is configured to apply a fade-out gain to the origin audio signal to determine a modified origin audio signal and to render the modified origin audio signal of the origin audio source 113 from an origin source position on a sphere 114 around the listening positions 201, 202.

[0078] Further described is an encoder 130 configured to generate a bitstream 140 indicative of an audio signal to be rendered within the virtual reality rendering environment 180. The renderer 130 may be configured to determine an origin audio signal of an origin audio source 113 of the origin audio scene 111. Further, the encoder 130 may be configured to determine origin position data relating to an origin position of the origin audio source 113. The encoder 130 may then generate the bitstream 140 including the origin audio signal and the origin position data.

[0079] The encoder 130 may receive an indication (via a feedback channel from the VR audio renderer 160 to the encoder 130) that the listener 181 is moving from the origin audio scene 111 to the destination audio scene 112 within the virtual reality rendering environment 180.

[0080] The encoder 130 may then determine (particularly only in response to receiving such an indication) an end audio signal of the end audio source 113 of the end audio scene 112 and end position data relating to the end source position of the end audio source 113. The encoder 130 may further generate a bitstream 140 including the end audio signal and the end position data. Thus, the encoder 130 may be configured to provide an end audio signal of one or more end audio sources 113 of the end audio source 112 only in response to receiving an indication for a global transition 191 to the end audio scene 112. By doing so, the required bandwidth for the bitstream 140 may be reduced.

[0081] 9b shows a flowchart of a corresponding method 930 for generating a bitstream 140 indicative of an audio signal to be rendered within the virtual reality rendering environment 180. The method 930 includes determining 931 an origin audio signal of an origin audio source 113 of the origin audio scene 111. Further, the method 930 includes determining 932 origin position data relating to an origin position of the origin audio source 113. Further, the method 930 includes generating 933 a bitstream 140 including the origin audio signal and the origin position data.

[0082] The method 930 includes receiving 934 an indication that the listener 181 moves from the origin audio scene 111 to the destination audio scene 112 within the virtual reality rendering environment 180. In response, the method 930 may include determining 935 an destination audio signal for the destination audio source 113 of the destination audio scene 112, and determining 936 destination position data relating to an destination source position of the destination audio source 113. Additionally, the method 930 includes generating 937 a bitstream 140 including the destination audio signal and the destination position data.

[0083] 9c illustrates a flowchart of an example method 910 for rendering an audio signal in a virtual reality rendering environment 180. The method 910 may be performed by the VR audio renderer 160.

[0084] The method 910 includes rendering 911 origin audio signals of audio sources 311, 312, 313 from origin source positions on a sphere of origin 114 around an origin listening position 301 of a listener 181. The rendering 911 may be performed using a 3D audio renderer 162. In particular, the rendering 911 may be performed under the assumption that the origin listening position 301 is fixed. Thus, the rendering 911 may be limited to three degrees of freedom (in particular, the rotational movement of the head of the listener 181).

[0085] To account for the additional three degrees of freedom (for translational movement of the listener 181), the method 910 may include determining 912 that the listener 181 moves from an origin listening position 301 to an end listening position 302, where the end listening position 302 is typically within the same audio scene 111. Thus, the listener 181 may be determined 912 to perform a local transition 192 within the same audio scene 111.

[0086] In response to determining that the listener 181 performs the local transition 192, the method 910 may include determining 913 end source positions of the audio sources 311, 312, 313 on the end sphere 114 around the end listening position 302 based on the origin source positions. In other words, the source positions of the audio sources 311, 312, 313 may be transferred from the origin sphere 114 around the origin listening position 301 to the end sphere 114 around the end listening position 302. This may be achieved by projecting the origin source positions from the origin sphere 114 onto the end sphere. For example, a perspective projection of the origin source positions on the origin sphere onto the end sphere for the end listening position 302 may be performed. In particular, the end source positions may be determined such that the end source positions correspond to an intersection of a ray between the end listening position 302 and the origin source positions with the end sphere 114. In the above, the origin sphere 114 and the destination sphere may have the same radius. This radius may be, for example, a predetermined radius. The predetermined radius may be a default value for the renderer performing the rendering.

[0087] Further, the method 910 may include determining 914 an end audio signal of the audio source 311, 312, 313 based on the origin audio signal (in response to determining that the listener 181 performs the local transition 192). In particular, an intensity of the end audio signal may be determined based on the intensity of the origin audio signal. Alternatively or additionally, a spectral composition of the end audio signal may be determined based on the spectral composition of the origin audio signal. Thus, how the audio signal of the audio source 311, 312, 313 is perceived from the end listening position 302 may be determined (in particular, the intensity and / or the spectral composition of the audio signal may be determined).

[0088] The above mentioned determining steps 913, 914 may be performed by a pre-processing unit 161 of the VR audio renderer 160. The pre-processing unit 161 may handle the translational movement of the listener 181 by relocating the audio signals of one or more audio sources 311, 312, 313 from an origin sphere 114 around the origin listening position 301 to an end sphere 114 around the end listening position 302. As a result of this, the relocated audio signals of said one or more audio sources 311, 312, 313 may also be rendered using the 3D audio renderer 162 (which may be limited to 3DoF). Thus, the method 910 allows an efficient provision of 6DoF within the VR audio rendering environment 180.

[0089] As a result, the method 910 may include rendering 915 (e.g. using a 3D audio renderer such as an MPEG-H audio renderer) the destination audio signals of the audio sources 311, 312, 313 from the destination source positions on the destination sphere 114 around the destination listening position 302.

[0090] Determining 914 the end audio signal may include determining an end distance 322 between the origin source position and the end listening position 302. The end audio signal (particularly the intensity of the end audio signal) may then be determined (particularly scaled) based on the end distance 322. In particular, determining 914 the end audio signal may include applying a distance gain 410 to the origin audio signal, where the distance gain 410 depends on the end distance 322.

[0091] A distance function 415 may be provided indicating a distance gain 410 as a function of the distance 321, 322 between the source positions of the audio signals 311, 312, 313 and the listening positions 301, 302 of the listener 181. The distance gain 410 to be applied to the origin audio signal (to determine the destination audio signal) may be determined based on the function value of the distance function 415 for the destination distance 322. In this way, the destination audio signal may be determined efficiently and precisely.

[0092] Further, determining 914 the end audio signal may include determining an origin distance 321 between the origin source position and the origin listening position 301. The end audio signal may then be determined based on the origin distance 321 (as well). In particular, the distance gain 410 applied to the origin audio signal may be determined based on a function value of the distance function 415 for the origin distance 321. In a preferred example, the function value of the distance function 415 for the origin distance 321 and the function value of the distance function 415 for the end distance 322 are used to rescale the intensity of the origin audio signal to determine the end audio signal. Thus, an efficient and precise local transition 191 in the audio scene 111 may be provided.

[0093] Determining 914 the destination audio signal may include determining a directional profile 332 of the audio sources 311, 312, 313. The directional profile 332 may indicate the strength of the origin audio signal in different directions. The destination audio signal may then be determined (also) based on the directional profile 332. By taking the directional profile 332 into account, the acoustic quality of the local transition 192 may be improved.

[0094] The directivity profile 332 may indicate a directivity gain 510 to be applied to the origin audio signal to determine the destination audio signal. In particular, the directivity profile 332 may indicate a directivity gain function 515, which may indicate the directivity gain 510 as a function of a (possibly two-dimensional) directivity angle 520 between the source positions of the audio sources 311, 312, 313 and the listening positions 301, 302 of the listener 181.

[0095] Thus, determining 914 the end audio signal may include determining an end angle 522 between the end source position and the end listening position 302. The end audio signal may then be determined based on the end angle 522. In particular, the end audio signal may be determined based on a function value of the directivity gain function 515 for the end angle 522.

[0096] Alternatively or additionally, determining 914 the end audio signal may include determining an origin angle 521 between the origin source position and the origin listening position 301. The end audio signal may then be determined based on the origin angle 521. In particular, the end audio signal may be determined based on a function value of the directivity gain function 515 for the origin angle 521. In a preferred example, the end audio signal may be determined by modifying an intensity of the origin audio signal using function values ​​of the directivity gain function 515 for the origin angle 521 and for the end angle 522 to determine an intensity of the end audio signal.

[0097] Additionally, the method 910 may include determining end environment data 193 indicative of audio propagation characteristics of a medium between the end source location and the end listening location 302. The end environment data 193 may be indicative of an obstacle 603 located on a direct path between the end source location and the end listening location 302; indicative of information regarding spatial dimensions of the obstacle 603; and / or indicative of an attenuation suffered by an audio signal on a direct path between the end source location and the end listening location 302. In particular, the end environment data 193 may be indicative of an obstacle attenuation function for the obstacle 603, which may be indicative of an attenuation suffered by an audio signal passing through the obstacle 603 on a direct path between the end source location and the end listening location 302.

[0098] The destination audio signal may be determined based on the destination environment data 193, thereby further enhancing the quality of the audio rendered within the VR rendering environment 180.

[0099] As indicated above, the end environment data 193 may indicate an obstacle 603 on a direct path between the end source location and the end listening location 302. The method 910 may include determining a through distance 601 between the end source location and the end listening location 302 on the direct path. The end audio signal may then be determined based on the through distance 601. Alternatively or additionally, an obstacle-free distance 602 between the end source location and the end listening location 302 on an indirect path that does not pass through the obstacle 603 may be determined. The end audio signal may then be determined based on the obstacle-free distance 602.

[0100] Specifically, an indirect component of the destination audio signal may be determined based on the origin audio signal propagating along the indirect path. Furthermore, a direct component of the destination audio signal may be determined based on the origin audio signal propagating along the direct path. The destination audio signal may then be determined by combining the indirect and direct components. In this way, the acoustic effect of the obstacle 603 may be taken into account in a precise and efficient manner.

[0101] Further, the method 910 may include determining focus information related to the field of view 701 and / or the focus of interest 702 of the listener 181. The end audio signal may then be determined based on the focus information. In particular, the spectral composition of the audio signal may be adapted depending on the focus information. In this way, the VR experience of the listener 181 may be further improved.

[0102] Further, the method 910 may include determining that the audio sources 311, 312, 313 are ambience audio sources. In this context, an indication (e.g., a flag) may be received in the bitstream 140 from the encoder 130. For example, the indication indicates that the audio sources 311, 312, 313 are ambience audio sources. The ambience audio sources typically provide background audio signals. An origin source position of the ambience audio source may be maintained as an end source position. Alternatively or additionally, an intensity of an origin audio signal of the ambience audio source may be maintained as an intensity of an end audio signal. In this way, the ambience audio sources can be efficiently and consistently handled in the context of the local transition 192.

[0103] The above-described aspects are applicable to an audio scene 111 including a plurality of audio sources 311, 312, 313. In particular, the method 910 may include rendering a plurality of origin audio signals of the corresponding plurality of audio sources 311, 312, 313 from a plurality of different origin source positions on the origin sphere 114. Further, the method 910 may include determining a plurality of destination source positions for the corresponding plurality of audio sources 311, 312, 313 on the destination sphere 114 based on the plurality of origin source positions, respectively. Further, the method 910 may include determining a plurality of destination audio signals of the corresponding plurality of audio sources 311, 312, 313 based on the plurality of origin audio signals, respectively. The plurality of destination audio signals of the corresponding plurality of audio sources 311, 312, 313 may then be rendered from a corresponding plurality of destination source positions on the destination sphere 114 around the destination listening position 302.

[0104] Further described is a virtual reality audio renderer 160 for rendering audio signals in a virtual reality rendering environment 180. The audio renderer 160 is configured to render origin audio signals of audio sources 311, 312, 313 (in particular using a 3D audio renderer 162 of the VR audio renderer 160) from origin source positions on an origin sphere 114 around an origin listening position 301 of a listener 181.

[0105] Additionally, the VR audio renderer 160 may be configured to determine that the listener 181 moves from the origin listening position 301 to the destination listening position 302. In response, the VR audio renderer 160 may be configured to determine (e.g., within a pre-processing unit 161 of the VR audio renderer 160) destination source positions of the audio sources 311, 312, 313 on the destination sphere 114 around the destination listening position 302 based on the origin source positions, and to determine destination audio signals of the audio sources 311, 312, 313 based on the origin audio signals.

[0106] Further, the VR audio renderer 160 (e.g., the 3D audio renderer 162 ) may be configured to render destination audio signals of the audio sources 311 , 312 , 313 from destination source positions on the destination sphere 114 around the destination listening position 302 .

[0107] Thus, the virtual reality audio renderer 160 may comprise a pre-processing unit 161 configured to determine the destination source positions and destination audio signals of the audio sources 311, 312, 313. Furthermore, the VR audio renderer 160 may comprise a 3D audio renderer 162 configured to render the destination audio signals of the audio sources 311, 312, 313. The 3D audio renderer 162 may be configured to adapt the rendering of the audio signals of the audio sources 311, 312, 313 on the (unit) sphere 114 around the listening positions 301, 302 of the listener 181 according to a rotational movement of the head of the listener 181 (to provide 3DoF within the rendering environment 180). On the other hand, the 3D audio renderer 162 may not be configured to adapt the rendering of the audio signals of the audio sources 311, 312, 313 according to a translational movement of the head of the listener 181. In this way, the 3D audio renderer 162 may be limited to 3DoF, and translational DoF can then be provided in an efficient manner using the pre-processing unit 161, thereby providing an overall VR audio renderer 160 with 6DoF.

[0108] Further described is an audio encoder 130 configured to generate a bitstream 140. The bitstream 140 is generated to indicate an audio signal of at least one audio source 311, 312, 313 and to indicate a position of said at least one audio source 311, 312, 313 within a rendering environment 180. Further, the bitstream 140 may indicate environment data 193 related to audio propagation characteristics of the audio within the rendering environment 180. By signaling the environment data 193 related to the audio propagation characteristics, local transitions 192 within the rendering environment 180 may be enabled in a precise manner.

[0109] Further described is a bitstream 140 indicative of an audio signal of at least one audio source 311, 312, 313; a position of said at least one audio source 311, 312, 313 within the rendering environment 180; and environment data 193 relating to audio propagation characteristics of the audio within the rendering environment 180. Alternatively or additionally, the bitstream 140 may indicate whether the audio source 311, 312, 313 is an ambient sound audio source 801 or not.

[0110] 9d shows a flow chart of an exemplary method 920 of generating a bitstream. The method 920 includes determining 921 an audio signal of at least one audio source 311, 312, 313. Furthermore, the method 920 includes determining 922 position data relating to a position of said at least one audio source 311, 312, 313 in the rendering environment 180. Furthermore, the method 920 may include determining 923 environment data 193 relating to audio propagation characteristics of the audio in the rendering environment 180. The method 920 may further include inserting 934 said audio signal, said position data and said environment data 193 into the bitstream 140. Alternatively or additionally, an indication of whether the audio source 311, 312, 313 is an ambient audio source 801 may be inserted into the bitstream 140.

[0111] Thus, in this document a virtual reality audio renderer 160 (corresponding method) audio sources 311, 312, 313 for rendering audio signals in a virtual reality rendering environment 180 is described. The audio renderer 160 comprises a 3D audio renderer 162 configured to render audio signals of the audio sources 113, 311, 312, 313 from source positions on a sphere 114 around a listening position 301, 302 of a listener 181 in the virtual reality rendering environment 180. Furthermore, the virtual reality audio renderer 160 comprises a pre-processing unit 161 configured to determine a new listening position 301, 302 of the listener 181 in the virtual reality rendering environment 180 (in the same or a different audio scene 111, 112). Further, the pre-processing unit 161 is configured to update the audio signals and source positions of the audio sources 113, 311, 312, 313 with respect to a sphere 114 around the new listening positions 301, 302. The 3D audio renderer 162 is configured to render the updated audio signals of the audio sources 311, 312, 313 from the updated source positions on the sphere 114 around the new listening positions 301, 302.

[0112] The methods and systems described herein may be implemented as software, firmware and / or hardware. Certain components may be implemented as software running on a digital signal processor or microprocessor. Other components may be implemented as hardware or as application specific integrated circuits. The signals encountered in the methods and systems described herein may be stored in a medium such as a random access memory or an optical storage medium. The signals may be transferred over a network such as a radio wave network, a satellite network, a wireless network or a wired network, e.g., the Internet. A typical device utilizing the methods and systems described herein is a portable electronic device or other consumer equipment used to store and / or render audio signals.

[0113] The enumerated example (EE) for this article is as follows: [EE1] A method (910) for rendering an audio signal in a virtual reality rendering environment (180), the method comprising: Rendering (911) origin audio signals of audio sources (311, 312, 313) from origin source positions on an origin sphere (114) around an origin listening position (301) of a listener (181); A step (912) of determining that a listener (181) moves from a starting listening position (301) to a finishing listening position (302); determining (913) end source positions of said audio sources (311, 312, 313) on an end sphere (114) around an end listening position (302) based on said origin source positions; determining (914) an end audio signal of the audio source (311, 312, 313) based on the start audio signal; rendering (915) the destination audio signals of the audio sources (311, 312, 313) from destination source positions on a destination sphere around the destination listening position (302), method. [EE2] The method of claim EE1, wherein the method (910) includes projecting the origin source position from the origin sphere onto the destination sphere to determine the destination source position. [EE3] The method of any one of claims 1 to 2, wherein the end source position is determined such that the end source position corresponds to an intersection of a ray between the end listening position (302) and the origin source position with the end sphere. [EE4] Determining (914) the destination audio signal determining an end distance (322) between the origin source location and the end listening location (302); determining (914) the end audio signal based on the end distance (322); A method described in any one of EE1 to 3. [EE5] determining (914) the destination audio signal includes applying a distance gain (410) to the source audio signal; the distance gain (410) depends on the end distance (322); Method described in EE4. [EE6] Determining (914) the destination audio signal providing a distance function (415) that indicates the distance gain (410) as a function of the distance (321, 322) between a source position of the audio signal (311, 312, 313) and a listening position (301, 302) of the listener (181); determining the distance gain (410) to be applied to the source audio signal based on a function value of the distance function (415) for the end distance (322), Method described in EE5. [EE7] Determining (914) the destination audio signal determining an origin distance (321) between the origin source location and the origin listening location (301); determining the end audio signal based on the origin distance (321), A method described in any one of EE4 to 6. [EE8] The method of claim EE7, where EE7 cites EE6, wherein the distance gain (410) to be applied to the origin audio signal is determined based on a function value of the distance function (415) for the origin distance (321). [EE9] The method of any one of claims EE1 to 8, wherein determining (914) the destination audio signal comprises determining a strength of the destination audio signal based on a strength of the origin audio signal. [EE10] Determining (914) the destination audio signal determining a directional profile (332) of the audio source (311, 312, 313), the directional profile (332) being indicative of the strength of the source audio signal in various directions; determining (914) the destination audio signal based on the directional profile (332); The method of any one of EE1 to 9. [EE11] The method of claim EE10, wherein the directional profile (332) indicates a directional gain (510) to be applied to the source audio signal to determine the destination audio signal. [EE12] the directional profile (332) exhibits a directional gain function (515); the directivity gain function (515) indicates the directivity gain (510) as a function of the directivity angle (520) between the source positions of the audio sources (311, 312, 313) and the listening positions (301, 302) of the listener (181); The method described in EE10 or 11. [EE13] Determining (914) the destination audio signal determining an end angle (522) between the end source position and the end listening position (302); determining (914) the end audio signal based on the end angle (522); The method of any one of EE10 to 12. [EE14] The method of claim EE13, where EE13 references EE12, wherein the end audio signal is determined based on a function value of the directivity gain function (515) for the end angle (522). [EE15] Determining (914) the destination audio signal determining an origin angle (521) between said origin source position and said origin listening position (301); determining (914) the end audio signal based on the start angle (521); The method of any one of EE10 to 14. [EE16] The method of claim EE15, wherein the end audio signal is determined based on a function value of the directivity gain function (515) for the start angle (521). [EE17] The method of claim EE16, wherein determining (914) the end audio signal includes modifying an intensity of the end audio signal using function values ​​of the directivity gain function (515) for the start angle (521) and for the end angle (522) to determine an intensity of the end audio signal. [EE18] Determining (914) the destination audio signal determining destination environment data (193) indicative of audio propagation characteristics of a medium between said destination source location and said destination listening location (302); determining the destination audio signal based on the destination environment data (193), The method of any one of EE1 to 17. [EE19] The end point environment data (193) is an obstacle (603) located on a direct path between the end source location and the end listening location (302); and / or information about the spatial dimensions of said obstacle (603); and / or indicating the attenuation suffered by an audio signal on a direct path between the end source position and the end listening position (302), The method described in EE18. [EE20] The end point environment data (193) indicates an obstacle damping function; the attenuation function describes the attenuation experienced by an audio signal passing through an obstacle (603) on the direct path between the end source position and the end listening position (302), The method according to claim EE18 or 19. [EE21] the end point environment data (193) indicating obstacles (603) on a direct path between the end point source location and the end point listening location (302); Determining (914) the end audio signal includes determining a path distance (601) between the end source position and the end listening position (302) on the direct path; The destination audio signal is determined based on the travel distance (601); A method described in any one of EE18 to 20. [EE22] the end point environment data (193) indicating obstacles (603) on a direct path between the end point source location and the end point listening location (302); Determining (914) the end audio signal includes determining an obstacle-free distance (602) between the end source location and the end listening location (302) on an indirect path that does not pass through the obstacle (603); the destination audio signal is determined based on the obstacle-free distance (602); The method of any one of EE18 to 21. [EE23] Determining (914) the destination audio signal determining an indirect component of the destination audio signal based on the source audio signal propagating along the indirect path; determining a direct component of the destination audio signal based on the source audio signal propagating along the direct path; determining the destination audio signal by combining the indirect component and the direct component, The method described in EE22 when EE22 cites EE21. [EE24] Determining (914) the destination audio signal determining focus information regarding the field of view (701) and / or focus of attention (702) of a listener (181); determining the end audio signal based on the focus information; The method of any one of EE1 to 23. [EE25] determining that the audio source (311, 312, 313) is an ambient audio source; maintaining the origin source positions of the ambient sound audio sources (311, 312, 313) as the destination source positions; maintaining the intensity of the source audio signal of the ambient sound audio source (311, 312, 313) as the intensity of the destination audio signal, The method of any one of EE1 to 24. [EE26] The method of any one of claims EE1 to 25, wherein determining (914) the end audio signal comprises determining a spectral composition of the end audio signal based on a spectral composition of the source audio signal. [EE27] The method of any one of claims EE1 to 26, wherein the source audio signal and the destination audio signal are rendered using a 3D audio renderer (162), in particular an MPEG-H audio renderer. [EE28] The method (910), rendering a plurality of origin audio signals of a corresponding plurality of audio sources (311, 312, 313) from a plurality of different origin source positions on the origin sphere (114); determining a plurality of end source positions for the corresponding plurality of audio sources (311, 312, 313) on the end sphere (114) based on the respective plurality of origin source positions; determining a plurality of destination audio signals of the corresponding plurality of audio sources (311, 312, 313) based on the plurality of source audio signals, respectively; rendering the plurality of destination audio signals of the corresponding plurality of audio sources (311, 312, 313) from the corresponding plurality of destination source positions on the destination sphere (114) around the destination listening position (302), The method of any one of EE1 to 27. [EE29] A virtual reality audio renderer (160) for rendering an audio signal in a virtual reality rendering environment (180), the audio renderer (160) comprising: rendering origin audio signals of audio sources (311, 312, 313) from origin source positions on an origin sphere (114) around an origin listening position (301) of a listener (181); determining that a listener (181) moves from the starting listening position (301) to the ending listening position (302); determining end source positions of said audio sources (311, 312, 313) on an end sphere (114) around said end listening position (302) based on said origin source positions; determining an end audio signal of the audio source (311, 312, 313) based on the start audio signal; rendering the destination audio signals of the audio sources (311, 312, 313) from the destination source positions on the destination sphere (114) around the destination listening position (302), Virtual reality audio renderer. [EE30] The virtual reality audio renderer (160) a pre-processing unit (161) configured to determine the end source positions of the audio sources (311, 312, 313) and the end audio signals; a 3D audio renderer (162) configured to render the destination audio signals of the audio sources (311, 312, 313), A virtual reality audio renderer as described in EE29. [EE31] The 3D audio renderer (162) configured to adapt the rendering of the audio signals of the audio sources (311, 312, 313) on the sphere (114) around the listening position (301, 302) of the listener (181) in response to a rotational movement of the head of the listener (181); and / or is not configured to adapt the rendering of the audio signals of the audio sources (311, 312, 313) in response to translational movements of the head of the listener (181), A virtual reality audio renderer described in EE30. [EE32] An audio encoder (130) configured to generate a bitstream (140), the bitstream (140) comprising: an audio signal of at least one audio source (311, 312, 313); the position of the at least one audio source (311, 312, 313) within the rendering environment (180); and indicating environment data (193) indicative of audio propagation characteristics of the audio within the rendering environment (180); Audio encoder. [EE33] an audio signal of at least one audio source (311, 312, 313); the position of the at least one audio source (311, 312, 313) within the rendering environment (180); and indicating environment data (193) indicative of audio propagation characteristics of the audio within the rendering environment (180); Bitstream(140). [EE34] A method (920) for generating a bitstream (140), comprising: determining (921) an audio signal of at least one audio source (311, 312, 313); determining (922) position data relating to a position of the at least one audio source (311, 312, 313) within the rendering environment (180); determining (923) environmental data (193) indicative of audio propagation characteristics of the audio within the rendering environment (180); inserting (934) said audio signal, said position data and said environmental data (193) into said bitstream (140), method. [EE35] A virtual reality audio renderer (160) for rendering an audio signal in a virtual reality rendering environment (180), the audio renderer comprising: a 3D audio renderer (162) configured to render audio signals of audio sources (311, 312, 313) from source positions on a sphere (114) around a listening position (301, 302) of a listener (181) within a virtual reality rendering environment (180); A pre-treatment unit (161), determining a new listening position (301, 302) for a listener (181) within a virtual reality rendering environment (180); a pre-processing unit (161) configured to update the audio signal and source positions of the audio sources (311, 312, 313) with respect to a sphere (114) around the new listening position (301, 302), the 3D audio renderer (162) is configured to render updated audio signals of the audio sources (311, 312, 313) from updated source positions on a sphere (114) around the new listening positions (301, 302); Virtual reality audio renderer.

Claims

1. 1. A method for rendering an audio signal in a virtual reality rendering environment, the method comprising: - determining an origin audio signal of an audio source from an origin source position on an origin unit sphere around an origin listening position of a listener; receiving an indication of a movement of a listener from an origin listening position to an end listening position; determining an end source position of the audio source on an end unit sphere around an end listening position based on the origin source position by projecting the origin source position from the origin unit sphere onto the end unit sphere; determining an end audio signal of the audio source based on the source audio signal; - rendering the destination audio signal of the audio source from the destination source position on a destination unit sphere around the destination listening position; the origin source position is projected from the origin unit sphere onto the destination unit sphere by a perspective projection with respect to the destination listening position; method.

2. The method of claim 1, wherein the end source position is determined so that it corresponds to the intersection of a ray between the end listening position and the origin source position with the end unit sphere.

3. The method of claim 2, further comprising: determining an end distance between the origin source location and the end listening location; determining the end audio signal based on the end distance; The method of claim 1.

4. - Determining the destination audio signal includes applying a distance gain to the source audio signal; The distance gain depends on the end distance; The method of claim 3.

5. The method of claim 1, further comprising: - providing a distance function indicating the distance gain as a function of the distance between a source position of the audio signal and a listening position of the listener; determining the distance gain to be applied to the source audio signal based on a function value of the distance function for the destination distance, The method of claim 4.

6. The method of claim 1, further comprising: determining an origin distance between the origin source location and the origin listening location; determining the end audio signal based on the origin distance; The method of claim 3.

7. The method of claim 5, wherein the distance gain applied to the origin audio signal is determined based on a function value of the distance function for the origin distance.

8. The method of claim 1, wherein determining the end audio signal includes determining an intensity of the end audio signal based on an intensity of the start audio signal.

9. The method of claim 8, further comprising: - determining a directional profile of the audio source, the directional profile indicating the strength of the source audio signal in various directions; determining the end audio signal based on the directional profile; The method of claim 1.

10. The method of claim 9, wherein the directional profile indicates a directional gain to be applied to the source audio signal to determine the destination audio signal.

11. The directional profile exhibits a directional gain function; The directivity gain function indicates the directivity gain as a function of the directivity angle between a source position of the audio source and a listening position of a listener; 10. The method of claim 9.

12. The method of claim 11, further comprising: - determining an end angle between the end source position and the end listening position; determining the end audio signal based on the end angle; 10. The method of claim 9.

13. The method of claim 12, wherein the end audio signal is determined based on a function value of a directional gain function for the end angle.

14. The method of claim 13, wherein determining the end audio signal comprises: - determining an origin angle between the origin source position and the origin listening position; determining the end audio signal based on the start angle; 10. The method of claim 9.

15. The method of claim 14, wherein the end audio signal is determined based on a function value of a directional gain function for the start angle.

16. The method of claim 15, wherein determining the end audio signal includes modifying the intensity of the start audio signal using the function value of the directional gain function for the start angle and the function value of the directional gain function for the end angle to determine the intensity of the end audio signal.

17. The method of claim 16, further comprising: - determining end point environment data indicative of audio propagation characteristics of a medium between the end point source location and the end point listening location; determining the destination audio signal based on the destination environment data; The method of claim 1.

18. The end point environmental data is an obstacle located on a direct path between the end source location and the end listening location; and / or - information regarding the spatial dimensions of said obstacle; and / or the attenuation experienced by an audio signal on a direct path between the end source position and the end listening position; The method of claim 17, wherein:

19. The method of claim 18, further comprising: - determining focus information regarding the listener's field of vision and focus of attention; determining the end audio signal based on the focus information; The method of claim 1.

20. - Determining that the audio source is an ambient audio source; - maintaining the origin source location of an ambient sound audio source as the destination source location; - further comprising maintaining the intensity of the source audio signal of the ambient sound audio source as the intensity of the destination audio signal. The method of claim 1.

21. The method of claim 1, wherein determining the end audio signal includes determining a spectral composition of the end audio signal based on a spectral composition of the start audio signal.

22. The method of claim 1, wherein the source audio signal and the destination audio signal are rendered using a 3D audio renderer, in particular an MPEG-H audio renderer.

23. A system for rendering an audio signal in a virtual reality rendering environment, the system comprising: a renderer configured to render an origin audio signal of an audio source from an origin source position on an origin unit sphere around an origin listening position of a listener; a receiver configured to receive an indication of a movement of a listener from said origin listening position to an end listening position; a processor; The processor: determining an end source position of the audio source on an end unit sphere around the end listening position based on the origin source position by projecting the origin source position from the origin unit sphere onto the end unit sphere; determining an end audio signal of the audio source based on the start audio signal, the renderer is further configured to render the end audio signal of the audio source from the end source position on the end unit sphere around the end listening position; the origin source position is projected from the origin unit sphere onto the destination unit sphere by a perspective projection with respect to the destination listening position; system.

24. The system comprises: a pre-processing unit configured to determine the end source position of the audio source and the end audio signal; the renderer is a 3D audio renderer; 24. The system of claim 23.

25. The processor, configured to adapt the rendering of the audio signal of the audio source on a unit sphere around the listening position of the listener in response to rotational movements of the listener's head but not in response to translational movements of the listener's head; 24. The system of claim 23.

26. A virtual reality audio renderer for rendering an audio signal in a virtual reality rendering environment, the audio renderer comprising: a 3D audio renderer configured to render an audio signal of an audio source from a source position on a unit sphere around a listening position of a listener within the virtual reality rendering environment; A pre-treatment unit comprising: determining a new listening position for the listener within the virtual reality rendered environment; a pre-processing unit configured to update the audio signal and a source position of the audio source relative to a unit sphere around the new listening position, the source position of the audio source relative to the unit sphere around the new listening position being determined by projecting a source position on the unit sphere around the listening position onto the unit sphere around the new listening position; the 3D audio renderer is configured to render an updated audio signal of the audio source from an updated source position on a unit sphere around the new listening position, the source position being projected from the unit sphere around the listening position onto a unit sphere around the new perspective position by a perspective projection with respect to the new listening position, the unit sphere around the listening position and the unit sphere around the new listening position having the same radius; Virtual reality audio renderer.

Images