Apparatus, method and computer program for repositioning a spatial audio stream

Abstract

Examples of the present disclosure relate to an apparatus, a method and a computer program for repositioning spatial audio streams. The apparatus is configured to receive a plurality of spatial audio streams, wherein a spatial audio stream comprises one or more audio signals and associated spatial metadata. The apparatus is further configured to obtain repositioning information related to at least one of the plurality of spatial audio streams, and to reposition the at least one of the plurality of spatial audio streams based on the repositioning information.

Description

Technical Field

[0001] Examples of this disclosure relate to apparatus, methods, and computer programs for repositioning spatial audio streams. Some examples relate to apparatus, methods, and computer programs for repositioning spatial audio streams so that a user can distinguish different audio scenes based on the repositioned spatial audio. Background Technology

[0002] Spatial audio enables users to recreate the spatial characteristics of a sound scene, allowing them to perceive these spatial features. Spatial audio can be used for teleconferencing, mediated reality content (such as augmented reality and virtual reality), or for any other suitable purpose. Summary of the Invention

[0003] According to various, but not necessarily all, examples of this disclosure, an apparatus is provided that includes components for performing the following:

[0004] Receive multiple spatial audio streams, wherein the spatial audio streams include one or more audio signals and associated spatial metadata;

[0005] Obtain relocation information associated with at least one of a plurality of spatial audio streams; and

[0006] Based on the relocation information, at least one spatial audio stream among multiple spatial audio streams is relocated.

[0007] Repositioning at least one of a plurality of spatial audio streams may include: modifying at least one audio signal and modifying the spatial metadata associated with at least one audio signal.

[0008] Repositioning at least one of a plurality of spatial audio streams may include:

[0009] Modify two or more audio signals from a set of multiple audio signals, and combine the modified audio signals.

[0010] Modify the spatial metadata associated with two or more of the modified audio signals from a plurality of modified audio signals, and combine the modified spatial metadata.

[0011] The modified audio signal can be combined with one or more unmodified audio signals, and the modified spatial metadata can be combined with the unmodified spatial metadata.

[0012] Multiple spatial audio streams may include a first audio stream captured from a first audio scene and a second audio stream captured from a second audio scene.

[0013] Repositioning at least one of a plurality of spatial audio streams can result in the first audio stream being repositioned while the second audio stream remains unmodified.

[0014] Repositioning multiple spatial audio streams allows the first audio stream to be positioned in a first direction and the second audio stream to be positioned in a second direction.

[0015] Repositioning allows users to distinguish between the first and second audio streams.

[0016] Relocation information can include a mapping between the directions of multiple spatial audio streams and the desired directions for multiple spatial video streams.

[0017] Relocating multiple spatial audio streams can include relocating the audio signal and spatial metadata.

[0018] Repositioning multiple spatial audio streams can include manipulating the direction parameters of the spatial audio streams.

[0019] For one or more frequency sub-bands, spatial metadata may include information indicating the following: sound direction, and sound directionality.

[0020] Relocating multiple spatial audio streams can include relocating the sound direction component and sound directionality component of spatial metadata.

[0021] Repositioning multiple spatial audio streams can include:

[0022] Determine modified directions for at least a first plurality of audio streams and a second plurality of audio streams, wherein the modified directions are determined based on relocation information, and the modified directions are used to relocate the sound direction components of spatial metadata;

[0023] Based on the original direct-to-total energy ratio and relocation information, the modified sound directionality parameters are determined, and the modified direct-to-total energy ratio is used to relocate the sound directionality components of the spatial metadata.

[0024] The component can be used to apply energy adjustments to account for repositioning.

[0025] The component can be used to combine multiple repositioned spatial audio streams.

[0026] According to various, but not necessarily all, examples of this disclosure, an apparatus is provided comprising: at least one processor; and at least one memory storing computer program code, the at least one memory and the computer program code being configured together with the at least one processor to cause the apparatus to perform at least the following:

[0027] Receive multiple spatial audio streams, wherein the spatial audio streams include one or more audio signals and associated spatial metadata;

[0028] Obtain relocation information associated with at least one of a plurality of spatial audio streams; and

[0029] Based on the relocation information, at least one spatial audio stream among multiple spatial audio streams is relocated.

[0030] Based on various, but not necessarily all, examples of this disclosure, a teleconferencing system is provided, which includes one or more devices as described herein.

[0031] Based on various, but not necessarily all, examples of this disclosure, a method is provided that includes:

[0032] Receive multiple spatial audio streams, wherein the spatial audio streams include one or more audio signals and associated spatial metadata;

[0033] Obtain relocation information associated with at least one of a plurality of spatial audio streams; and

[0034] Based on the relocation information, at least one spatial audio stream among multiple spatial audio streams is relocated.

[0035] Repositioning at least one of a plurality of spatial audio streams may include: modifying at least one audio signal and modifying the spatial metadata associated with at least one audio signal.

[0036] According to various, but not necessarily all, examples of this disclosure, a computer program comprising computer program instructions, which, when executed by processing circuitry, cause:

[0037] Receive multiple spatial audio streams, wherein the spatial audio streams include one or more audio signals and associated spatial metadata;

[0038] Obtain relocation information associated with at least one of a plurality of spatial audio streams; and

[0039] Based on the relocation information, at least one spatial audio stream among multiple spatial audio streams is relocated. Attached Figure Description

[0040] Some examples will now be described with reference to the accompanying drawings, in which:

[0041] Figure 1 Example audio scene is shown;

[0042] Figure 2 Example methods are shown;

[0043] Figure 3 Example audio scene is shown;

[0044] Figure 4 Example system shown;

[0045] Figure 5 Example server shown;

[0046] Figure 6 Example device is shown;

[0047] Figure 7 The module shown is an example device; and

[0048] Figure 8 The processing device is shown. Detailed Implementation

[0049] Examples of this disclosure relate to apparatus, methods, and computer programs for relocating spatial audio streams. The spatial audio stream used may include both an audio signal and associated spatial metadata. Spatial audio streams may be obtained from different audio scenarios and / or from different capture devices. Relocation of spatial audio streams may be used in teleconferencing systems (such as voice conferencing systems), mediated reality systems, or any other suitable spatial audio systems. Relocation of spatial audio streams allows a user to distinguish audio captured from different audio scenarios based on the new location.

[0050] Figure 1 Several example audio scenes 101 are illustrated schematically, and how these audio scenes are played to a user in a system without examples of this disclosure. Figure 1 Examples include a first audio scenario 101A and a second audio scenario 101B. The different audio scenarios 101A and 101B can be located in different locations. For example, in the case where the system is being used for teleconferencing, the first audio scenario 101A can be in a first geographical location, and the second audio scenario 101B can be in a different second geographical location. Different geographical locations can be different rooms, buildings, towns, countries, or any other suitable different locations. Figure 1 Two different positions are shown in the example, but any number of different positions may be used in other examples of this disclosure.

[0051] Different audio scenes 101A and 101B can be captured by different capture devices 103A and 103B, such that the first capture device 103A captures the first audio scene 101A, and the second capture device 103B captures the second audio scene 101B. The first capture device 103A and the audio capture device 103B can be independent of each other.

[0052] Capture devices 103A and 103B may include any device that can be configured to capture spatial audio. Capture devices 103A and 103B may include multiple microphones or one or more microphone arrays. Capture devices 103A and 103B may be configured to capture spatial information associated with audio scenes 101A and 101B. Capture devices 103A and 103B may be part of a teleconferencing system, used for capturing mediated reality, or used in any other suitable system.

[0053] It can process signals captured by capture devices 103A and 103B to provide spatial audio streams.

[0054] exist Figure 1 In the example, the first audio scene 101A includes three sound sources 105A and ambient sound 107A. The sound sources 105A are distributed throughout the sound scene 101A. The sound sources 105A are located at different positions within the sound scene 101A. The sound sources 105A can be fixed in their positions within the sound scene 101A or can be moved within the audio scene 101A. In some examples, the sound sources 105A included within the audio scene 101A can change as the sound sources 105A move into and out of the audio scene 101A.

[0055] The second sound scene 101B also includes three sound sources 105B and ambient sound 107B. The sound sources 105B in the second sound scene 101B can be fixed or movable, and can be located in any suitable position within the second sound scene 101B.

[0056] exist Figure 1 In the example, both the first sound scene 101A and the second sound scene 101B include three sound sources 105A and 105B. In other examples, sound scenes 101A and 101B may include different numbers of sound sources 105A and 105B. In some examples, the first sound scene 101A may include a different number of sound sources 105A than the second sound scene 105B.

[0057] Sound sources 105A and 105B may include any suitable sound source. In some examples, sound sources 105A and 105B may include people speaking. Other types of sound sources 105A and 105B may be used in other examples of this disclosure.

[0058] The spatial audio stream is sent to the playback device 109. The playback device 109 may be located in a third position 111, which is different from both the first and second positions.

[0059] Playback device 109 can be configured to process the received spatial audio stream to play spatial audio for a user. Figure 1The examples shown do not use examples of this disclosure. Spatial audio may include both an ambient component and a directional component. The directional component may be based on the location of the respective sound sources 105A, 105B.

[0060] The playback device 109 can also be configured to perform audio processing on the received spatial audio stream for real-time communication. Audio processing may include acoustic echo cancellation, noise suppression, automatic gain control, and / or any other suitable processing.

[0061] exist Figure 1 In the example, the played grouped spatial audio has the first sound source 105A and the second sound source 105B reproduced at positions corresponding to the positions of the first sound source 105A and the second sound source 105B within the respective audio scenes 101A, 101B. This may confuse and / or disorient the user of the playback device 109. Figure 1 In the example, this is shown as the first sound source 105A overlapping with the second sound source 105B. Ambient sounds 107A and 107B are provided so that they appear to surround the user and are therefore reproduced so that they overlap with each other. This can make it difficult for the user to distinguish between sound source 105A from the first audio scene 101A and sound source 101B from the second audio scene 101A. This can cause confusion and disorientation for the user.

[0062] Figure 2 Example methods are shown that can be used in the examples of this disclosure to enable the relocation of an audio stream. This method can be performed by any suitable device. In some examples, the method can be performed by a device within a teleconferencing system or a mediating reality system or any other suitable system. For example, the method can be performed by a server or any other suitable device. In some examples, a device can be provided within a user device (such as a playback device) that processes the spatial audio stream to enable the spatial audio to be played to the user.

[0063] The method includes, at box 201, receiving multiple spatial audio streams. Each spatial audio stream includes both audio signals and associated spatial metadata. Each spatial audio stream may include one or more audio signals. Audio signals and associated spatial metadata may be provided within the same spatial audio stream. Audio signals may include audio channels.

[0064] The audio signal may include transmitted audio signals. The audio signal may include microphone-based signals. The audio signal may include processed microphone signals or any other suitable type of signal.

[0065] Spatial metadata can be associated with audio signals because spatial metadata is related to the same audio scene that can be represented by the audio signal. Information for the audio signal and information for the spatial metadata can be captured simultaneously.

[0066] Spatial metadata may include any data obtained from audio scenes 101A and 101B that express the spatial characteristics of audio scenes 101A and 101B. For example, spatial metadata may include one or more of the following: direction parameters, direct-to-total ratio parameters, diffusion-to-total ratio parameters, spatial coherence parameters (indicating coherent sound in the surrounding directions), extended coherence parameters (indicating coherent sound in a spatial arc or region), direction vector values, and any other suitable parameters that express the spatial characteristics of the spatial sound distribution.

[0067] In some examples, spatial metadata may include information indicating the direction and directionality of sound. Sound directionality can indicate the directionality or non-directionality / environment of the sound. This spatial metadata may be a sound arrival direction parameter and a direct-to-total ratio parameter. Parameters that can provide spatial metadata within a frequency band may be used. Other parameters may be used in other examples of this disclosure.

[0068] Multiple spatial audio streams may include audio streams captured from different audio scenarios 101. For example, multiple spatial audio streams may include a first audio stream captured from a first audio scenario 101A and a second audio stream captured from a second audio scenario 101B. The first audio scenario 101A and the second audio scenario 101B may include different sound sources 105A and 105B, and are located in different geographical locations. An audio stream may include one or more audio signals.

[0069] Audio scenes 101A and 101B may include omnidirectional audio scenes. Audio scenes 101A and 101B may include sounds from a series of directions. Audio scenes 101A and 101B may also include ambient sounds 107A and 107B.

[0070] At box 203, the method includes obtaining relocation information associated with multiple spatial audio streams. The relocation information may include information that can be used to process the spatial audio streams so that they can be relocated. The relocation information may include information indicating a new position for each spatial audio stream. In some examples, the relocation information may include a mapping between the directions of the multiple spatial audio streams and desired directions for the multiple spatial audio streams.

[0071] In some examples, the repositioning information can be predetermined or determined without any user input. For example, the device can be configured such that each of a plurality of spatial audio streams is repositioned into an angular section. Each angular sector can have the same width, such that the width of the sector is determined by the number of received data audio streams. The position of the angular sector can be assigned to the spatial audio streams based on the order in which the spatial audio streams are received or any other suitable factor.

[0072] In some examples, the repositioning information may be determined at least in part by the user of playback device 109 or the user of any other part of the system. For example, the user of playback device 109 may indicate a preferred order for a particular spatial audio stream and may assign larger angular sections or preferred positions to these spatial audio streams.

[0073] At box 205, the method includes relocating multiple spatial audio streams based on relocation information. Relocating multiple spatial audio streams may include relocating both the audio signal and the associated spatial metadata. Relocating multiple spatial audio streams may include processing the audio signal and the associated spatial metadata such that the audio source and / or environment of the spatial audio streams are perceived as being at different locations. Relocating multiple spatial audio streams may include relocating both the audio signal and the associated spatial metadata in each of the multiple spatial audio streams. In some examples, relocation may include relocating only a subset of the multiple spatial audio streams.

[0074] Relocating at least one spatial audio stream from a plurality of spatial audio streams may include: modifying at least one audio signal, modifying spatial metadata associated with the at least one audio signal, and combining the modified at least one audio signal and the associated modified spatial metadata. In some examples, relocating at least one spatial audio stream from a plurality of spatial audio streams may include: modifying the plurality of audio signals and combining the modified plurality of audio data, modifying the spatial metadata associated with the plurality of modified audio signals and combining the modified spatial metadata, and combining the modified plurality of audio signals and the associated modified spatial metadata.

[0075] In some examples, only a subset of multiple spatial audio streams may be modified or relocated. In such examples, modified spatial audio signals may be combined with unmodified spatial audio signals, and modified spatial metadata may be combined with unmodified spatial metadata.

[0076] Repositioning may include manipulating the directional parameters of the spatial audio stream. Repositioning may include manipulating the spatial characteristics of both the audio signal and the associated spatial metadata. Manipulating spatial characteristics may include any suitable processing of the corresponding characteristics.

[0077] Repositioning multiple spatial audio streams can include repositioning individual components of spatial metadata. For example, in cases where the spatial metadata includes a sound direction component and a sound directionality component, these two components can be adjusted or repositioned.

[0078] In such an example, repositioning a spatial audio stream may include determining modified directions for at least a first set of multiple audio streams and a second set of multiple audio streams. The modified directions may be determined based on repositioning information. The modified directions can then be used to reposition the sound directionality components of the audio signal and spatial metadata. Repositioning a spatial audio stream may also include determining modified sound directivity parameters based on the original direct-to-total energy ratio and the repositioning information. The modified direct-to-total energy ratio can then be used to adjust the spatialization of the ambient components within the spatial audio stream.

[0079] Repositioning multiple spatial audio streams allows a first set of audio streams to be positioned in a first set of directions and a second set of audio streams to be positioned in a second set of directions. For example, sound source 105A in the first audio stream can be repositioned to an angular section in the first direction, and sound source 105B in the second audio stream can be repositioned to an angular section in the second direction. Repositioning sound sources 105A and 105B to different directions allows a user to distinguish between the first and second audio streams. For example, this can allow a user to distinguish between different speakers in different locations, or it can allow a user to distinguish between different mediating real-world sound objects.

[0080] In some examples, the method may include in Figure 2 The steps are not shown in the diagram. For example, in some examples, energy and / or spectral adjustments may be applied to the repositioned spatial audio stream. The energy and / or spectral adjustments may be configured to take repositioning into account.

[0081] In some examples, the method may include combining multiple repositioned spatial audio streams.

[0082] Figure 3 Example audio scenes 101A and 101B are shown. The first audio scene 101A and the second audio scene 101B can be as follows: Figure 1 As shown and as described above.

[0083] exist Figure 3 In the example, Figure 2The method shown has been implemented to reposition the audio stream. The audio stream has been repositioned such that the playback of sound source 105A and ambient sound 107A from the first audio scene 101A does not overlap with the playback of sound source 105B and ambient sound 107B from the second audio scene 101B. In this example, the capture of audio scenes 101A and 101B is... Figure 1 The examples shown are the same. However, the playback is different because the examples disclosed here are used to reposition spatial audio streams.

[0084] In this example, sound source 105A and ambient sound 107A from the first audio scene 101A have been repositioned so that they appear on the user's left side. Sound source 105B and ambient sound 107B from the second audio scene 101B have been repositioned so that they appear on the user's right side. In other examples, the corresponding sound sources 105A, 105B and ambient sounds 107A, 107B can be repositioned to different locations.

[0085] exist Figure 3 In the example, different sound sources 105A from the first audio scene 101A are each repositioned to different locations. These different locations are all within the user's angular range. For example, in... Figure 3 In the example, each sound source 105A from the first audio scene 101A is positioned on the user's left-hand side, but in a way that makes them appear non-overlapping. This allows the user to distinguish the sound sources 105A from the first audio scene 101A based on their repositioned locations. The ambient sound 107A from the first audio scene 107A has also been repositioned so that it appears only on the user's left-hand side.

[0086] Similarly, different sound sources 105B from the second audio scene 101B are each repositioned to different locations. These different locations for the second audio scene 101B are also within the user's angular range. The angular range for the second audio scene 101B differs from the angular range for the first audio scene 101A. The angular range for the second audio scene 101B does not overlap with the angular range for the first audio scene 101A. Figure 3 In the example, each sound source 105B from the second audio scene 101B is positioned on the user's right-hand side, but in a way that makes them appear non-overlapping. This allows the user to distinguish the sound sources 105B from the second audio scene 101B based on their repositioned locations. The ambient sound 107B from the second audio scene 101B has also been repositioned so that it appears only on the user's right-hand side.

[0087] exist Figure 3In the example, spatial audio from the first audio scene 101A is provided within an angular range of 10 degrees to 90 degrees, and a second audio scene 101B is provided within an angular range of -10 degrees to -90 degrees. Other angular ranges may be used in other examples of this disclosure. In this example, each angular range has the same width. In other examples, the sectors used for each audio scene 101 may have different widths.

[0088] exist Figure 3 In the example, sound sources 105A and 105B and ambient sounds 107A and 107B from two audio scenes 101A and 101B have been repositioned. In other examples, there may be more than two audio scenes 101 providing spatial audio streams to be repositioned. In such examples, each spatial audio stream from different audio scenes 101 can be repositioned so that sound sources 105 and ambient sounds 107 from each audio scene 101 are rendered at different locations. These different locations may include different angular ranges relative to the user. These different locations may include non-overlapping angular ranges.

[0089] Figure 4 An example system is shown that can be used to implement examples of this disclosure. In this example, system 401 is used to enable a voice conference session. System 401 includes multiple capture devices 103A, 103B, a playback device 109, and a server 403. Each capture device 103A, 103B can also be used as a playback device for other participants in the voice conference session.

[0090] System 401 may include any suitable number of capture devices 103. Each capture device may include multiple microphones. The multiple microphones within the capture device 103 may be configured to enable the capture device 103 to capture spatial audio.

[0091] exist Figure 4 In the example, system 401 includes a first capture device 103A and a second capture device 103B. The first capture device 103A is located in a first position and configured to capture spatial audio from a first audio scene 101A. The second capture device 103B is located in a second position and configured to capture spatial audio from a second audio scene 101B.

[0092] exist Figure 4 In the example, each of the first audio scene 101A and the second audio scene 101B includes multiple sound sources. In other examples of this disclosure, sound scenes 101A and 101B may include any number of sound sources, and do not need to include multiple sound sources.

[0093] exist Figure 4In the examples, each of the different audio scenarios 101A and 101B includes three people who can speak during a voice conference session. These different people can be positioned anywhere within audio scenarios 101A and 101B. In some examples, people can move around within audio scenarios 101A and 101B, or they can remain in a fixed position.

[0094] Capture devices 103A and 103B are configured to process the captured spatial audio to generate a spatial audio stream. The spatial audio stream includes an audio signal and associated spatial metadata. The audio signal may include a transmission audio signal, a processed microphone signal, or any other suitable type of signal. The audio signal may include the captured audio signal that has been processed based on the microphone signal to a format suitable for transmission by capture devices 103A and 103B. For example, generating an audio signal based on a microphone signal may involve beamforming, microphone noise reduction, ambient noise suppression, automatic gain control, limiters, speech enhancement, audio encoding, and / or any other suitable process. The audio signal may be transmitted from capture devices 103A and 103B to server 403.

[0095] Spatial metadata can be transmitted along with the audio signal. Spatial metadata can be associated with the audio signal. Spatial metadata can include any data that expresses the spatial characteristics of audio scenes 101A and 101B. Spatial metadata can be transmitted in quantized encoded form.

[0096] Server 403 may be located at a different location than the first capture device 103A and the second capture device 103B. Server 403 may be configured to receive multiple spatial audio streams from capture devices 103A and 103B using any suitable communication protocol. Server 403 may be configured to receive multiple spatial audio streams from capture devices 103A and 103B using wired and / or wireless communication links.

[0097] Server 403 may include processing unit 507. Processing unit 507 may, as... Figure 8 As shown or in any other suitable format. Processing unit 507 can be configured to receive multiple spatial audio streams from capture devices 103A, 103B and process the received spatial audio streams to reposition them. Server 403 can use, for example... Figure 2 The method shown or any other suitable method may be used to reposition the spatial audio stream.

[0098] Server 403 can also be configured to combine repositioned spatial audio streams. The repositioned spatial audio streams can be combined into a single signal or any other suitable format. The repositioned spatial audio streams can then be combined and sent to playback device 109.

[0099] Playback device 109 may include any device that enables spatial audio to be processed and played to a user. In some examples, playback device 109 may include stereo headphones that enable binaural playback. In other examples, playback device may include speaker devices or any other suitable device that enables spatial audio to be played to a user.

[0100] exist Figure 4 In one example, stereo headphones can be configured to deliver the played audio to a single user. In other examples, multiple headphones can be used so that repositioned spatial audio can be played to multiple users.

[0101] The spatial audio can be processed by playback device 109 before being played to the user. The spatial audio can be played such that a repositioned first audio scene 101A is provided at a first position and a repositioned second audio scene 101B is provided at a different second position. For example, the spatial audio can be repositioned such that sound source 105A from the first audio scene 101A is on the left-hand side for the user of playback device 109, while sound source 105 from the second audio scene 101B is on the right-hand side for the user of playback device 108.

[0102] Figure 5 Example server 403 is illustrated schematically in some examples that can be used in this disclosure.

[0103] Server 403 receives a first encoded spatial audio stream 501A and a second encoded spatial audio stream 501B. The first encoded spatial audio stream 501A may include audio signals from a first audio scene 101A and associated spatial metadata, and the second encoded spatial audio stream 501B may include audio signals from a second audio scene 101B and associated spatial metadata.

[0104] Encoded spatial audio streams 501A and 501B can be received from spatial audio capture devices 103A and 103B. Encoded spatial audio streams 501A and 501B can be received independently of each other.

[0105] A first coded spatial audio stream 501A is provided to a first decoder 503A. The first decoder 503A decodes the first coded spatial audio stream 501A to provide a first decoded spatial audio stream 505A. Similarly, a second coded spatial audio stream 501B is provided to a second decoder 503B. The second decoder 503B decodes the second coded spatial audio stream 501B to provide a second decoded spatial audio stream 505B. Any suitable process can be used to decode the corresponding coded spatial audio streams 501A and 501B. The process used to decode the coded spatial audio streams 501A and 501B can depend on the process used to encode the coded spatial video streams 501A and 501B.

[0106] Decoding spatial audio streams 505A and 505B are both provided to device 507. Device 507 can be a processing device or any other suitable type of device 507. Figure 8 Example device 507 is shown in the figure.

[0107] Device 507 also receives repositioning information 511 as input. This repositioning information can be received from control unit 509 or any other suitable entity. Control unit 509 can be configured to determine the desired positions of spatial audio scenes 101A and 101B. For example, control unit 509 can be configured to determine the positions where spatial audio scenes 101A and 101B should be repositioned so that the user can distinguish between the different audio scenes 101A and 101B. Control unit 509 can be configured to determine the angular positions and angular ranges where spatial audio scenes 101A and 101B should be repositioned.

[0108] In some examples, control unit 509 may define new positions for spatial audio scenes 101A, 101B based on the number of participants in the voice conference session, the number of received spatial audio streams, and / or any other suitable factors. In some examples, spatial audio scenes 101A, 101B may be positioned based on the order in which users join the voice conference session at the corresponding positions. For example, if a new spatial audio scene 101 joins the session, control unit 509 may define a new sector to be placed to the right of the user on playback device 109. The sector width of the current spatial audio scenes 101A, 101B may be compressed and / or the sector position may be changed to create space for the new sector. In this case, the repositioning of spatial audio scenes 101A, 101B can be performed automatically without any input from the user.

[0109] In some examples, the control unit 509 can reposition spatial audio to a location indicated by the user as preferred. For example, the user can input that they prefer the first audio scene 101A to always be on the right-hand side and the second audio scene to always be on the left-hand side, or the user can select the audio to be positioned to correspond to the position of the video image obtained from the respective capture devices 103A, 103B. For example, if the video image corresponding to the first audio scene 101A is displayed on the right-hand side of the user's monitor, they can select that the audio from the first audio scene 101A is also provided to their right-hand side.

[0110] Control unit 509 may use information related to the current positions of sound sources 105A and 105B within audio scenes 101A and 101B, and information related to the desired new positions, and use this information to determine relocation information. Relocation information 511 may include a mapping between the orientations of sound sources 105A and 105B in the spatial audio stream and the desired orientation of the spatial audio stream, and / or any other suitable information.

[0111] Device 507 uses relocation information 511 to relocate decoded spatial audio streams 505A and 505B. Device 507 can also be configured to combine the relocated decoded spatial audio streams 505A and 505B to provide a combined and relocated spatial audio stream 513 as output.

[0112] The combined and repositioned spatial audio stream 513 is provided to the encoder 515. The encoder 515 encodes the combined and repositioned spatial audio stream 513 to provide an encoded combined and repositioned spatial audio stream 517 as output. The encoded combined and repositioned spatial audio stream 517 can be sent to the playback device 109 so that the playback device can provide the repositioned spatial audio to the user.

[0113] It should be understood that Figure 5 The examples are illustrative, and different combinations and configurations of the corresponding modules may be used in the examples disclosed herein. For example, decoders 503A, 503B and / or encoder 515 may be provided as part of device 507.

[0114] Figure 6 The process that can be performed by example device 507 is shown.

[0115] Device 507 receives multiple spatial audio streams 505A-505N as input. The spatial audio streams 505A-505N can be decoded before being provided to device 507, such as... Figure 6 As shown.

[0116] Spatial audio streams 505A-505N may include any suitable type of spatial audio. In some examples, spatial audio streams 505A-505N may include parametric spatial audio streams. Spatial audio streams 505A-505N can be captured using any suitable spatial audio capture device 103A-103N. In some examples, mobile devices (such as mobile phones) or any other suitable type of device can be used to capture spatial audio streams 505A-505N. Different audio capture devices 103A-103N may be used to capture each of the spatial audio streams 505A-505N. Different audio capture devices 103A-103N may be located in different locations so that different audio scenes 101A-101N can be captured.

[0117] The captured audio can be processed using any suitable technology to provide spatial audio streams 505A-505N. Spatial audio streams 505A-505N can be encoded before being sent to server 403. Any suitable encoding process can be used to encode spatial audio streams 505A-505N. In some examples, IVAS (Immersive Speech and Audio Service) can be used to encode spatial audio streams 505A-505N.

[0118] Spatial audio streams 505A-505N can be sent from various audio capture devices 103A-103N to server 403. Spatial audio streams 505A-505N can be sent using MASA (Metadata-Auxiliary Spatial Audio) format or any other suitable format.

[0119] exist Figure 6 In the example shown, spatial audio streams 505A-505N may have been decoded by server 403 so that spatial audio streams 505A-505N are decoded spatial audio streams 505A-505N.

[0120] Each spatial audio stream 505A-505N includes one or more audio signals and associated spatial metadata. The spatial metadata may include any data expressing the spatial characteristics of the audio scene 101A-101N captured by the capture devices 103A-103N. The spatial metadata may include spatial information related to the audio within the audio signals.

[0121] Input spatial audio streams 505A-505N are provided to the repositioning module 601. The repositioning module 601 also receives repositioning information 511. The repositioning information 511 can be obtained from the control unit 509 or from any other suitable source. The control unit 509 can be provided within the server 403 or within any other suitable entity.

[0122] The repositioning information 511 may include information that can be used to process spatial audio streams so that they can be repositioned. In some examples, the repositioning information 511 may include a mapping between the directions of multiple spatial audio streams and desired directions for the multiple spatial audio streams. The repositioning information 511 may be provided in any suitable format. In some examples, the repositioning information 511 may include angular ranges for spatial audio streams 5051-505N. For example, the repositioning information 511 may indicate that a first audio stream 505A will be positioned between 30 and 90 degrees and a second audio stream 505B will be positioned between 90 and 120 degrees. In some examples, angular ranges may be defined for each received spatial audio stream 5051-505N. The defined angular ranges may be partially overlapping or non-overlapping.

[0123] The repositioning module 601 repositions the audio content of each spatial audio stream 505A-505N based on the repositioning information 511. The repositioning module 601 provides multiple repositioned spatial audio streams 603A-603N as outputs.

[0124] The repositioning module 601 can reposition both the direct and ambient components of the spatial audio streams 505A-505N. Therefore, the repositioned spatial audio streams 603A-603N can include both direct and ambient components.

[0125] exist Figure 6 In this example, the number of repositioned spatial audio streams 603A-603N is the same as the number of spatial audio streams 505A-505N received as input. In this example, each output repositioned spatial audio stream 603A-603N corresponds to an input spatial audio stream 505A-505N.

[0126] The repositioned spatial audio streams 603A-603N are provided as input to the combiner module 605. The combiner module 605 combines the multiple repositioned spatial audio streams 603A-603N into a single combined and repositioned spatial audio stream 513. Any suitable technique or process can be used to combine the multiple repositioned spatial audio streams 603A-603N. The combiner module 605 can be equipped with leveling and gain control functions or any other suitable processing.

[0127] A combined and repositioned spatial audio stream 513 is provided as the output of device 507. The combined and repositioned spatial audio stream 513 includes multiple repositioned spatial audio streams 603A-603N. The combined and repositioned spatial audio stream 513 includes both the direct component and the ambient component of the multiple repositioned spatial audio streams 603A-603N.

[0128] The combined and repositioned spatial audio stream 513 can be encoded and sent to the playback device 109 so that the repositioned spatial audio can be played to the user. When the repositioned spatial audio is played to the user, the user can more easily distinguish different audio scenes because they are all provided within different angular ranges. This also provides a stable output where the sound source does not fluctuate between different locations.

[0129] Figure 7 An example repositioning module 601 of example device 507 is schematically shown. Here, the process is shown for a single input spatial audio stream 505. The same or similar process will be applied to each of N received spatial audio streams 505A-505N.

[0130] The repositioning module 601 receives spatial audio streams 505A-505N as input. The spatial audio streams 505A-505N include spatial metadata 701 and audio signals 703. The spatial metadata 701 and audio signals 703 can be provided to the repositioning module 601 as separate inputs.

[0131] Spatial metadata 701 may include any information that expresses the spatial characteristics of the audio scene 101 captured by the spatial audio stream 505.

[0132] In this example, spatial metadata 701 includes MASA metadata. MASA metadata includes direction (azimuth θ(k,n) and elevation φ(k,n)) parameters in the time-frequency domain, direct to total energy ratio r(k,n) parameter, extended coherence ξ(k,n) parameter, and surround coherence γ(k,n) parameter (k is the frequency band index, n is the time frame index).

[0133] Audio signal 703 can include any number of audio signals. In this example, audio signal 703 includes two audio signals.

[0134] Spatial metadata 701 is provided as input to the spatial metadata modifier module 705. Audio signal 703 is provided as input to the audio signal modifier module 709.

[0135] The repositioning module 601 also receives repositioning information 511 as input. The repositioning information 511 can be provided in any suitable format. In this example, the repositioning information 511 is provided as an angular span [α]. left ,α right(In the azimuth direction), the input spatial audio stream 505 should be repositioned within that angular span. In this example, repositioning occurs on the azimuth axis; however, a similar principle can be applied to repositioning within the elevation axis. In such cases, the repositioning information may also include the elevation angular span and / or any other suitable information.

[0136] The relocation information 511 can be provided as input to both the spatial metadata modifier module 705 and the audio signal modifier module 709.

[0137] Spatial metadata 701 of the first spatial audio stream 505 is provided to the spatial metadata modifier module 705. The spatial metadata modifier module 705 repositions the spatial metadata 701 based on the repositioning information 511.

[0138] As an example, assume the angular span for the first spatial audio stream 505 is [90, 30] degrees. In order to reposition the spatial metadata 701, the orientation parameters and the sound directionality parameters are repositioned.

[0139] The orientation can be repositioned as follows. In this example, the elevation angle is not modified. The elevation angle is output as is (i.e., φ). mod (k,n)=φ(k,n)). Instead, the azimuth is modified based on the repositioning information 511.

[0140] In this example, it is assumed that the capture device 103 used to capture the spatial audio stream 505 is a mobile device (such as a mobile phone) or other similar type of device. Other types of capture devices 103 may be used in other examples.

[0141] In this example, the capture device 103 has been used to capture a teleconference so that the participant providing the sound source 105 is primarily located in front of the capture device 103.

[0142] Using these assumptions, the azimuth angle can be remapped as follows. First, the reverse direction is mirrored to the positive direction:

[0143] If θ(k,n) > 90°, then θ′(k,n) = 180° - θ(k,n)

[0144] Otherwise, if θ(k,n) < -90°, then θ′(k,n) = -180° - θ(k,n)

[0145] Otherwise, θ′(k,n)=θ(k,n).

[0146] Then, the center direction of the angular span is calculated as follows:

[0147]

[0148] And the width of the angular span is calculated as follows:

[0149]

[0150] Using these variables, the modified azimuth angle can be calculated as follows:

[0151]

[0152] This modification maps the input azimuth to the desired angular span, as indicated in the repositioning information 511. This enables the playback device 109 to render a spatial audio stream within that angular span.

[0153] The examples disclosed also enable the modification of the acoustic directivity parameters of the spatial metadata 701. In this example, the acoustic directivity parameters include those directly related to the total energy ratio.

[0154] Without modifying the direct ratio to the total energy, this would cause ambient sound 107 to be rendered in all directions. This could be confusing for users of playback devices.

[0155] The direct-to-total-energy ratio can be modified by increasing its value, which in turn reduces the angular span. For example, the following modification can be performed:

[0156]

[0157] This modification enables ambient sound 107 to be rendered within or substantially within the target angular span.

[0158] You can also modify the extended coherence to match variations in the width of the angular span. For example, you could perform the following modification:

[0159]

[0160] The circumferential coherence can be transmitted without modification (i.e., γ). mod (k,n)=γ(k,n)).

[0161] These modifications, performed by the spatial metadata modifier module 705, provide modified spatial metadata 707 as output. In this example, the modified spatial metadata includes the parameter θ. mod (k,n), φ mod (k,n), r mod (k,n), ξ mod (k,n) and γ mod (k,n).

[0162] The audio signal 703 of the first spatial audio stream 505 is provided to the audio signal modifier module 709. The audio signal modifier module 709 repositions the audio signal 703 based on the repositioning information 511 to provide the repositioned audio signal 711 as output.

[0163] The audio signal 703 can be repositioned using the following procedure or any other suitable procedure.

[0164] To begin this process, if the audio signals 703 are in the time domain, they are converted to a time-frequency representation. Any suitable filter bank can be used to convert the audio signals 703 from the time domain to a time-frequency representation. For example, the filter bank may include a complex modulation quadrature mirror filter (QMF) bank or a low-delay variant thereof. In the example using a filter bank, the repositioning module 601 can be configured as delay space metadata 701 to account for the delays of the forward and backward filter banks. In other examples, the audio signals 703 can be received in the time-frequency domain. In such examples, no conversion or filter bank will be used.

[0165] To reposition the audio signal 703, based on angle α left α right Determine the re-shift value p for audio signal 703:

[0166]

[0167] For a stereo audio signal, assuming the positive angle is on the left and the channel order is left-right, the retranslation matrix P is determined as follows:

[0168]

[0169] The time-frequency signal is represented as a column vector s(b,t), where b is the frequency bin index, t is the time index, and the rows represent the channels. The relocation is performed as follows:

[0170] s′(b,t)=Ps(b,t)

[0171] Then, the signal s′(b,t) is the repositioned audio signal in the time-frequency domain before equalization.

[0172] Energy adjustment can be applied to signals to account for repositioning. Signals can be equalized so that signal s′(b,t) has the same combined energy spectrum as signal s(b,t) across the channels, with suitable time averaging, and appropriately avoids excessive signal amplification (e.g., maximum amplification of 6dB).

[0173] If a forward filter bank is applied, the corresponding reverse filter bank is applied to convert the signal back to a time-domain signal.

[0174] The result of this process is a repositioned audio signal 711, which is provided as the output from the audio signal modifier module 709.

[0175] Variations of this process can be used in the examples disclosed herein. For instance, if the input audio signal consists of only one channel, the translation matrix can be replaced with a translation vector to generate a translated stereo audio signal:

[0176]

[0177] In this example, the signal is provided in the time-frequency domain or transformed to the time-frequency domain. In some examples, the signal can be processed as a time-domain signal. This can be used in situations requiring minimal delay, as it avoids the delay caused by using a filter bank.

[0178] In such cases, the equalization described above will not be applied; instead, no equalization, or static or time-varying FIR (Finite Impulse Response) or IIR (Infinite Impulse Response) equalization will be performed. In some examples, the audio signal 703 may take the form associated with the audio codec, such as MDCT (Modified Discrete Cosine Transform) or any other suitable form.

[0179] In some embodiments, the repositioning information 511 may also include distance information for the spatial audio stream. For example, one party may not only be located at a certain angular span, but may also be farther or closer than the other parties. The distance information may be, for example, a relative parameter, where a value of 1 represents no change in distance, a value of 2 represents twice the distance, a value of 0.5 represents half the distance, and so on. An example way to modify the distance of the spatial audio stream is to multiply the corresponding repositioning matrix P by the value 1 / d. r , where d r It is a relative distance parameter.

[0180] In a further example related to distance processing, the audio signal can be decorrelated based on a distance parameter. For example, when d r When the value is greater than 1, the audio signal can be decorrelated so that increasing the distance leads to an increase in decorrelation. The decorrelator used can be a simple decorrelator, such as different delays and / or phase modifications at different frequencies, or it can be a more complex decorrelator, such as an artificial reverberation. The amount of decorrelation can be controlled by fully decorrelating the audio signal and combining the undecorrelated and decorrelated versions of the audio signal with different gains to obtain the processed audio signal. In some examples, the ratio value can be modified based on the distance parameter so that the ratio value shifts towards zero as the distance increases.

[0181] In some examples, the locations of the direct component and the environment component can be different from each other. For example, the direct component can be translated at a first angular span, and the environment component can be translated at a second (overlapping or non-overlapping) angular span. In that case, a retranslation matrix P can be formulated such that the angular spans of the direct and environment components are combined into a larger span, and then the retranslation matrix can be formulated based on that larger angular span.

[0182] In some examples, the metadata can also be modified to orient the direct and ambient components differently. For instance, different modified azimuth angles θ can be determined for the direct and ambient components based on their respective angular spans. mod,dir (k,n) and θ mod,ambi (k,n). These can then be combined into a single modified azimuth angle θ. mod (k,n) is used so that when the energy ratio r(k,n) has a large value, the modified azimuth angle θ of the direct part is used more prominently. mod,dir (k,n), and when the energy is smaller than r(k,n), the modified azimuth angle θ of the environmental component is used more prominently. mod,ambi (k,n). Combination can be performed by converting azimuth values ​​into vectors, weighting them based on ratio parameters, summing the weighted vectors, and determining the direction of the resulting vector from the combined azimuth values, or by any other suitable process.

[0183] Figure 8 An apparatus 507 is schematically shown that can be used to implement examples of this disclosure. Apparatus 507 includes at least one processor 803 and at least one memory 805. It should be understood that apparatus 507 can be included in... Figure 1 Additional components not shown. These can be found in, for example... Figure 4 The device 507 is provided within the server 403 shown, or within any playback device 109, or within any suitable device.

[0184] exist Figure 8 In the example, device 507 includes a processing unit. Device 507 can be configured to process spatial audio data. Device 507 can be configured to process spatial audio data for use in teleconferencing or mediating reality or any other suitable purpose.

[0185] exist Figure 8 In some examples, device 507 can be implemented as processing circuitry. In some examples, device 507 can be implemented solely in hardware, have certain aspects of software (including separate firmware), or be a combination of hardware and software (including firmware).

[0186] like Figure 8As shown, device 507 can be implemented using instructions that enable hardware functions, for example, by using executable instructions of computer program 807 in a general-purpose or special-purpose processor 803, which can be stored on a computer-readable storage medium (disk, memory, etc.) for execution by such processor 803.

[0187] Processor 803 is configured to read from and write to memory 805. Processor 803 may also include an output interface and an input interface, through which processor 803 outputs data and / or commands, and through which data and / or commands are input to processor 803.

[0188] Memory 805 is configured to store computer program 807, which includes computer program instructions (computer program code 809) that control the operation of device 507 when loaded into processor 803. The computer program instructions of computer program 807 enable device 507 to execute. Figure 2 The logic and routines of the method are shown. Processor 803 is able to load and execute computer program 807 by reading memory 805.

[0189] Therefore, device 507 includes: at least one processor 803; and at least one memory 805 including computer program code 801, wherein the at least one memory 805 and the computer program code 809 are configured together with the at least one processor 803 to cause device 507 to perform at least the following:

[0190] Receive more than 201 spatial audio streams, wherein the spatial audio streams include one or more audio signals and associated spatial metadata;

[0191] Obtain relocation information associated with at least one of the multiple spatial audio streams; and

[0192] Based on the relocation information, at least one spatial audio stream among more than 205 spatial audio streams is relocated.

[0193] like Figure 8As shown, computer program 807 can reach device 507 via any suitable delivery mechanism 801. Delivery mechanism 801 can be, for example, a machine-readable medium, a computer-readable medium, a non-transitory computer-readable storage medium, a computer program product, a memory device, a recording medium (e.g., an optical disc read-only memory (CD-ROM) or a digital versatile optical disc (DVD) or solid-state storage), an article of art including or tangibly embodying computer program 807. The delivery mechanism can be a signal configured to reliably transmit computer program 807. Device 507 can propagate or transmit computer program 807 as a computer data signal. In some examples, computer program 807 can be transmitted to device 507 using wireless protocols such as Bluetooth, Bluetooth Low Energy, Bluetooth Smart, 6LoWPAN (IPv6 over Low Power Personal Area Networks), ZigBee, ANT+, Near Field Communication (NFC), Radio Frequency Identification, Wireless Local Area Network (Wireless LAN)) or any other suitable protocol.

[0194] Computer program 807 includes computer program instructions for causing device 507 to perform at least the following:

[0195] Receive more than 201 spatial audio streams, wherein the spatial audio streams include one or more audio signals and associated spatial metadata;

[0196] Obtain relocation information associated with at least one of the multiple spatial audio streams; and

[0197] Based on the relocation information, at least one spatial audio stream among more than 205 spatial audio streams is relocated.

[0198] Computer program instructions may be included in a computer program 807, a non-transitory computer-readable medium, a computer program product, or a machine-readable medium. In some, but not necessarily all, examples, computer program instructions may be distributed across more than one computer program 807.

[0199] Although the memory 805 is shown as a single component / circuit, it can be implemented as one or more separate components / circuits, some or all of which can be integrated / removable and / or can provide permanent / semi-permanent / dynamic / cache storage.

[0200] Although the processor 803 is shown as a single component / circuit, it can be implemented as one or more separate components / circuits, some or all of which can be integrated / removable. The processor 803 can be a single-core or multi-core processor.

[0201] References to “computer-readable storage medium,” “computer program product,” “tangible computer program,” or “controller,” “computer,” “processor,” etc., should be understood to encompass not only computers with different architectures (such as single / multiprocessor architectures and sequential (von Neumann) / parallel architectures) but also special-purpose circuits, such as field-programmable gate arrays (FPGAs), special-purpose circuits (ASICs), signal processing devices, and other processing circuits. References to computer programs, instructions, code, etc., should be understood to encompass software used in programmable processors or firmware, such as programmable content of hardware devices, whether instructions for processors or configuration settings for fixed-function devices, gate arrays, or programmable logic devices.

[0202] As used in this application, the term "circuit" may refer to one or more or all of the following:

[0203] (a) Hardware-only implementation (such as implementation only in analog and / or digital circuits) and

[0204] (b) A combination of hardware circuitry and software, such as (if applicable):

[0205] (i) A combination of analog and / or digital hardware circuitry and software / firmware, and

[0206] (ii) Any part of a hardware processor (including a digital signal processor), software, and memory (one or more), which work together to enable a device (such as a mobile phone or a server) to perform various functions, and

[0207] (c) (one or more) hardware circuits and / or (one or more) processors, such as (one or more) microprocessors or a portion thereof, which require software (e.g. firmware) for operation, but may not exist when not required for operation.

[0208] This definition of "circuit" applies to all uses of the term in this application (including in any claim). As another example, as used in this application, the term "circuit" also covers only the implementation of hardware circuitry or processors and their (or their) accompanying software and / or firmware. The term "circuit" also covers, for example (and if applicable to a particular claim element), baseband integrated circuits used in mobile devices or similar integrated circuits in servers, cellular network devices, or other computing or networking devices.

[0209] Figure 2The boxes shown may represent steps in the method and / or code segments in computer program 807. The description of a particular order of boxes does not necessarily imply a desired or preferred order, and the order and arrangement of boxes can vary. Furthermore, some boxes may be omitted.

[0210] exist Figure 8 In this example, device 507 is shown as a single entity. In other examples, device 507 may be provided with multiple different entities that can be distributed within the cloud or other suitable networks.

[0211] It should be understood that variations can be made to the examples described above in this disclosure. For example, in Figure 4 In this example, system 401 includes a centralized server 403. Other architectures for the system can be used in other examples. For example, system 401 could be a distributed conferencing system 401 that does not include a centralized server 403. In such an example, each participating device can receive spatial audio signals from other participating devices. In this case, participating devices (such as playback device 109) can be configured to obtain relocation information 511 and use it to relocate the received spatial audio stream.

[0212] The term “includes” as used in this document has an inclusive rather than exclusive meaning. That is, any reference to “X includes Y” indicates that X may include only one Y or may include more than one Y. If the intention is to use “includes” with an exclusive meaning, it will be made explicit in the context by referring to “includes only one…” or by using “comprises”.

[0213] Various examples are referenced in this specification. Descriptions of features or functionalities associated with an example indicate which features or functionalities are present in that example. The use of the terms "example," "for example," "may," or "possibly" in the text indicates that, whether explicitly stated or not, such a feature or functionality is present at least in the described example, and whether or not it is described as an example, and they may, but not necessarily, be present in some or all other examples. Thus, "example," "for example," "may," or "possibly" refers to a specific instance in the example class. An instance's characteristic can be a characteristic of only that instance, a characteristic of the class, or a characteristic of a class that includes some, but not all, instances of that class. Therefore, it is implicitly disclosed that a feature described with reference to one example rather than another may (if possible) be used as part of a working composition in that other example, but is not necessarily required to be used in that other example.

[0214] Although examples have been described with reference to various examples in the preceding paragraphs, it should be understood that modifications may be made to the given examples without departing from the scope of the claims.

[0215] The features described above can be used in combinations different from those explicitly described above.

[0216] Although some features have been described with reference to certain characteristics, those functions can be performed by other features, whether or not they are described.

[0217] Although features have been described with reference to some examples, those features may also exist in other examples, whether or not they are described.

[0218] The terms “a” or “the” as used in this document have an inclusive rather than exclusive meaning. That is, any reference to “X includes a / the Y” indicates that X may include only one Y or may include more than one Y, unless the context clearly indicates otherwise. If “a” or “the” is intended to have an exclusive meaning, it will be explicit in the context. In some cases, the use of “at least one” or “one or more” may be used to emphasize an inclusive meaning, but the omission of these terms should not be used to infer any exclusive meaning.

[0219] The presence of a feature (or combination of features) in a claim is a reference to the feature or (combination of features) itself, as well as a reference to a feature (equivalent feature) that achieves substantially the same technical effect. Equivalent features include, for example, features that are variations and achieve substantially the same result in substantially the same manner. Equivalent features include, for example, features that perform substantially the same function in substantially the same manner to achieve substantially the same result.

[0220] In this specification, various examples have been referenced that use adjectives or adjective phrases to describe the characteristics of the examples. Such descriptions of characteristics in relation to examples indicate that the characteristic exists exactly as described in some examples, and substantially as described in others.

[0221] While the foregoing specification has been intended to draw attention to those features that are considered important, it should be understood that the applicant may seek protection by means of any patentable feature or combination of features mentioned above and / or shown in the drawings, whether or not it is emphasized in this regard.

Claims

1. An apparatus for repositioning a spatial audio stream, comprising: At least one processor; as well as At least one memory storing computer program code, said at least one memory and said computer program code being configured together with said at least one processor such that the device at least: Receive multiple spatial audio streams, wherein each of the multiple spatial audio streams includes one or more audio signals and associated spatial metadata, wherein the multiple spatial audio streams include at least a first audio stream captured from a first audio scene and a second audio stream captured from a second audio scene, wherein the first audio scene is at least partially different from the second audio scene; Obtain relocation information associated with at least one of the plurality of spatial audio streams, wherein the obtained relocation information includes a mapping between the directions of the plurality of spatial audio streams and a desired direction for the plurality of spatial audio streams; and Based on the relocation information, at least one of the multiple spatial audio streams is relocated.

2. The apparatus according to claim 1, wherein, Repositioning at least one of the plurality of spatial audio streams causes the device to: modify at least one of the one or more audio signals included by the at least one of the plurality of spatial audio streams; and modify the spatial metadata associated with the at least one audio signal.

3. The apparatus according to claim 1, wherein, Repositioning at least one of the plurality of spatial audio streams causes the device to: Modify two or more audio signals from one or more audio signals included in at least one of the plurality of spatial audio streams, wherein the one or more audio signals include at least two audio signals; Combine two or more modified audio signals; Modify the spatial metadata associated with the two or more modified audio signals; and Combine the modified spatial metadata.

4. The apparatus according to claim 2, wherein, Repositioning at least one of the plurality of spatial audio streams further causes the device to at least: Combine at least one modified audio signal with one or more unmodified audio signals; as well as Combine the modified spatial metadata with the unmodified spatial metadata associated with the one or more unmodified audio signals.

5. The apparatus according to claim 1, wherein, The repositioning of at least one of the plurality of spatial audio streams results in the first audio stream being repositioned while the second audio stream remains unmodified.

6. The apparatus according to claim 1, wherein, The repositioning of at least one of the plurality of spatial audio streams causes the first audio stream to be positioned in a first direction and the second audio stream to be positioned in a second direction.

7. The apparatus according to claim 1, wherein, The repositioning of at least one of the plurality of spatial audio streams enables the user to distinguish between the first audio stream and the second audio stream.

8. The apparatus according to claim 1, wherein, Repositioning the plurality of spatial audio streams allows the device to manipulate the direction parameters of the plurality of spatial audio streams.

9. The apparatus according to claim 1, wherein, The spatial metadata includes information indicating the following for one or more frequency sub-bands: sound direction and sound directionality.

10. The apparatus according to claim 1, wherein, Repositioning the plurality of spatial audio streams causes the device to: reposition the sound direction component of the spatial metadata, and modify the sound directionality component of the spatial metadata.

11. The apparatus according to claim 10, wherein, The device is repositioned to reposition the plurality of spatial audio streams. For at least a first plurality of spatial audio streams and a second plurality of spatial audio streams, a modified direction is determined, wherein the modified direction is determined based on the obtained repositioning information; The modified direction is used to reposition the sound direction component of the spatial metadata; Based on the original direct and total energy ratio and the obtained repositioning information, the modified sound directionality parameters were determined; and The acoustic directional components of the spatial metadata are repositioned using a modified direct-to-total energy ratio.

12. The apparatus according to claim 1, wherein, Furthermore, the device applies energy adjustment based at least in part on the repositioning of at least one of the plurality of spatial audio streams.

13. The apparatus according to claim 1, wherein, Furthermore, the device combines at least one repositioned spatial audio stream.

14. The apparatus according to claim 1, wherein, The first audio scene includes one or more first sound sources reproducible at one or more first locations, wherein the second audio scene includes one or more second sound sources reproducible at one or more second locations, wherein when at least one of the one or more first sound sources is reproduced at at least one first location among the one or more first locations, and when at least one of the one or more second sound sources is reproduced at at least one second location among the one or more second locations, the at least one first sound source and the at least one second sound source at least partially overlap, wherein the repositioning of the at least one spatial audio stream among the plurality of spatial audio streams such that during playback, the at least one first location among the one or more first locations is repositioned to prevent the at least one of the one or more first sound sources from overlapping with the at least one of the one or more second sound sources.

15. A method for repositioning a spatial audio stream, comprising: Receive multiple spatial audio streams, wherein each of the multiple spatial audio streams includes one or more audio signals and associated spatial metadata, wherein the multiple spatial audio streams include at least a first audio stream captured from a first audio scene and a second audio stream captured from a second audio scene, wherein the first audio scene is at least partially different from the second audio scene; Obtain relocation information associated with at least one of the plurality of spatial audio streams, wherein the obtained relocation information includes a mapping between the directions of the plurality of spatial audio streams and a desired direction for the plurality of spatial audio streams; and Based on the relocation information, at least one of the multiple spatial audio streams is relocated.

16. The method according to claim 15, wherein, Repositioning at least one of the plurality of spatial audio streams includes: Modify at least one audio signal among the one or more audio signals included in at least one of the plurality of spatial audio streams; and Modify the spatial metadata associated with the at least one audio signal.

17. The method according to claim 15, wherein, Repositioning at least one of the plurality of spatial audio streams includes: Modify two or more audio signals from one or more audio signals included in at least one of the plurality of spatial audio streams, wherein the one or more audio signals include at least two audio signals; Combine two or more modified audio signals; Modify the spatial metadata associated with the two or more modified audio signals; and Combine the modified spatial metadata.

18. The method of claim 16, further comprising: Combine at least one modified audio signal with one or more unmodified audio signals; as well as Combine the modified spatial metadata with the unmodified spatial metadata associated with the one or more unmodified audio signals.

19. A computer program product comprising computer program instructions, characterized in that, When the computer program instructions are loaded into a processor for execution, they implement a method for repositioning a spatial audio stream, the method comprising: Receive multiple spatial audio streams, wherein each of the multiple spatial audio streams includes one or more audio signals and associated spatial metadata, wherein the multiple spatial audio streams include at least a first audio stream captured from a first audio scene and a second audio stream captured from a second audio scene, wherein the first audio scene is at least partially different from the second audio scene; Obtaining relocation information associated with at least one of the plurality of spatial audio streams, wherein the obtained relocation information includes a mapping between the directions of the plurality of spatial audio streams and a desired direction for the plurality of spatial audio streams; and Based on the relocation information, at least one of the multiple spatial audio streams is relocated.

Images