Reverberation decorrelation for ambient stereo audio compression

Abstract

A method is disclosed, comprising: receiving an audio signal comprising a plurality of audio channels; selecting a first portion of the plurality of audio channels; selecting a second portion of the plurality of audio channels; generating a first mixed audio channel by mixing a first portion of the plurality of audio channels with the first time-delayed audio channel; generating a second mixed audio channel by mixing a second portion of the plurality of audio channels with a second time-delayed audio channel; and generating an enhanced ambient stereo model based on the plurality of audio channels, the first mixed audio channel, and the second mixed audio channel.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 377,668, filed on September 29, 2022, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0002] Various embodiments relate to compressing ambisonic audio. Background Art

[0003] Ambient audio modeling can map audio signals having multiple directions into spherical harmonics having multiple channels. At least one benefit of the ambient audio modeling method is that each channel can have a fairly large number of N audio signals. In an analog processing unit, noise accumulates on average the same amount as the N audio signals, which can be problematic when processing the N audio signals. Summary of the invention

[0004] In a general aspect, an apparatus, a system, a non-transitory computer-readable medium having stored thereon a computer-executable program code that can be executed on a computer system, and / or a method can perform a process having a method comprising: receiving an audio signal comprising a plurality of audio channels; selecting a first portion of the plurality of audio channels; selecting a second portion of the plurality of audio channels; generating a first mixed audio channel by mixing the first portion of the plurality of audio channels with a first time-delayed audio channel; generating a second mixed audio channel by mixing the second portion of the plurality of audio channels with a second time-delayed audio channel; and generating an enhanced ambient stereo model based on the plurality of audio channels, the first mixed audio channel, and the second mixed audio channel.

[0005] Some implementations are directed to generating enhanced ambisonic models (e.g., sound sources) without using entropy for encoding reflections and reverberation on their respective channels. In an example implementation, reflections and reverberation between channels may be decorrelated by processing a subset of channels in an input or original ambisonic model (e.g., an ambisonic recording). BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Example embodiments will be more fully understood from the detailed description given herein below and the accompanying drawings, in which like elements are represented by like reference numerals, which are given by way of illustration only and thus do not limit example embodiments, and in which:

[0007] Figure 1 A block diagram illustrating data flow according to an example implementation is shown.

[0008] Figure 2 A schematic diagram of data flow according to an example implementation is shown.

[0009] Figure 3 A block diagram of a method of generating an enhanced ambisonic model according to an example implementation is shown.

[0010] Figure 4 A block diagram of a system according to an example implementation is shown.

[0011] Figure 5 A block diagram of a method of generating an enhanced ambisonic model according to an example implementation is shown.

[0012] It should be noted that these drawings are intended to illustrate the general characteristics of the methods and / or structures utilized in certain example embodiments and to supplement the written description provided below. However, these drawings are not drawn to scale and do not accurately reflect the precise structure or performance characteristics of any given embodiment and should not be interpreted as defining or limiting the range of values ​​or properties covered by the example embodiments. For example, the positioning of modules and / or structural elements may be reduced or enlarged for clarity. The use of similar or identical reference numerals in the various drawings is intended to indicate the presence of similar or identical elements or features. DETAILED DESCRIPTION

[0013] Ambient audio modeling may include multiple channels, each channel including multiple audio signals. The audio signals associated with each of the multiple channels may include reflections and reverberation between the channels. Therefore, as the number of channels increases, the reflections and reverberation may cause a significant increase in the number of audio signals that need to be processed. Decorrelating the reflections and reverberation between channels may significantly reduce the number of audio signals to be processed.

[0014] Decorrelation of reflections and reverberation can have several technical benefits, such as for user experience. As a specific example, decorrelation of reflections and reverberation between channels by processing a subset of the channels can reduce the number of audio signals to be processed while minimizing the impact of decorrelation on the user experience. In other words, audio signal processing can be reduced while a user listening to the reconstructed audio hears and / or perceives reflections and reverberation associated with ambient stereo audio.

[0015] The example implementations described herein can decorrelate audio reflections, which can include diffusion, reverberation, and sound echoes. Diffusion can be the scattering of audio energy. Reverberation can include reflected sound that causes a large number of reflections to gradually accumulate and then decay as the sound is absorbed by the surface of objects in the space. Echoes can include sound reflected from a speaker back to a microphone. Early reflections can be echoes of direct sound sources, rather than late reflections or diffuse mixing present in reverberation or sound sources. Late reverberation (sometimes referred to as reverberation tail) can be reflections that embed the original sound into the soundscape, and if the reverberation is strong enough, the soundscape can be blurred. Ambient stereo modeling using spherical harmonics may not be able to capture early reflections (e.g., 100 to 150 ms) and / or longer late reverberation (approximately 1.5 seconds).

[0016] The number of audio signals to be processed per channel can be based on the number of components used to represent the sound field. In ambient stereo, the sound field can be decomposed into spherical harmonic components (e.g., called W, X, Y, and Z). The spherical harmonic components can be collectively referred to as B format. High-order ambient stereo can include more channels (e.g., greater than 4) than the B format. B format and high-order ambient stereo recordings can be original ambient stereo recordings. Therefore, high-order ambient stereo with multiple audio signals associated with audio reflections may require a lot of resources to process. Therefore, using decorrelation in high-order ambient stereo can reduce processing resources.

[0017] Processing higher-order ambient stereo may include audio compression. When compressing ambient stereo audio, each channel may be sampled and windowed, and the signal of each channel may be individually transformed via, for example, a modified discrete cosine transform (MDCT) after windowing to obtain the transformed data of the frame being compressed. The compression decorrelation that may occur in each audio channel in audio compression (e.g., the source signal is transformed into multiple output signals) limits the compressed window (e.g., 20ms). Due to these two phenomena, reflections and reverberation (e.g., correlation between channels) may not be decorrelated in existing ambient stereo compression techniques. Therefore, the example implementation describes decorrelated audio reflections and reverberation for use with higher-order ambient stereo audio compression.

[0018] Figure 1 A block diagram of data flow according to an example implementation is shown. Figure 1 As shown in , the data flow includes an ambisonic source 105 block, an audio mixer 110 block, a filter bus 115 block, an audio mixer 120 block, and an ambisonic output 125 .

[0019] The ambient sound source 105 may be an ambient sound microphone. The ambient sound microphone may include multiple sensors pointing in different directions. The ambient sound microphone may capture audio signals in an arrangement based on an original ambient sound model (e.g., an ambient sound pre-reverberation model). The original ambient sound model may be defined as an audio source based on a polygon on a geodesic polyhedron such as an icosahedron, a geodesic polyhedron subdivision or other (geodesic) polyhedron, and / or a point source. Each point on the polygon may correspond to an audio channel of the ambient sound source 105. Therefore, the ambient sound source 105 may have N audio channels. If the ambient sound model is based on a geodesic polyhedron, the ambient sound source 105 may have, for example, 12 audio channels.

[0020] The audio mixer 110 may be configured to mix a portion of the N audio channels. The mixing may be a linear mix of ambient stereo channels. In an example implementation, the audio mixer 110 may be communicatively coupled to a portion of the N audio channels. Thus, the audio mixer 110 may be configured to mix audio signals associated with corresponding communicatively coupled audio channels.

[0021] The filter bus 115 may be a ring buffer filter bus. The filter bus 115 may be configured to reduce the amount of filtering required to produce reverberation. A tensor processing unit (TPU), a graphics processing unit (GPU), and / or a central processing unit (CPU) may be used to determine (e.g., calculate) reverberation. The ring buffer filter bus may contain t seconds (e.g., one second, two seconds, three seconds, five seconds, etc.) of audio in the form of a ring buffer. Each filter bus channel takes its input as a linear mix of the ambient stereo channels generated by the audio mixer 110. In addition, the audio mixer 110 may be communicatively coupled to the filter bus 115, thereby forming a feedback loop. Therefore, the audio mixer 110 may receive a time-delayed channel associated with the filter bus 115. The filter bus 115 signal may be a copy of the ambient stereo channel, but filtering is applied to the signal. The filter may include, for example, a low pass filter and a high pass filter, a notch filter, etc. The channels in the filter bus 115 may also mix time-delayed data (both intra-channel and inter-channel) from the ring buffer filter bus in the audio mixer 110, allowing for complex infinite impulse response (IIR) type reverberation. In an example implementation, the computational complexity of the filter bus 115 and the audio mixer 110 may be minimized by limiting the number of intra-channel interactions and channel-to-channel interactions with time delays.

[0022] The audio mixer 120 can be configured to mix the audio channels from the ambient stereo source 105 and / or the channels from the filter bus 115. The audio mixer 120 can be configured to generate an enhanced ambient stereo model as an ambient stereo output 125. Each channel in the enhanced ambient stereo model can be an original ambient stereo model channel mixed with a ring buffer filter bus channel (a time-delayed version). This allows an entropy source (e.g., a sound source) to be encoded and decoded on one of the channels, and in the absence of entropy for coding the early reflections and reverberations on the corresponding channels of the early reflections and reverberations (codify), the channel early reflections and reverberations can occur naturally in the enhanced model. The ambient stereo output 125 (e.g., an enhanced ambient stereo model) can be defined as being substantially similar to the ambient stereo sound source 105 (e.g., the original ambient stereo model). Thus, the ambisonic output 125 (e.g., an enhanced ambisonic model) may be defined as audio sources based on polygons on a geodesic polyhedron such as an icosahedron, a geodesic polyhedron tessellation, or other (geodesic) polyhedrons, and / or point sources.

[0023] Figure 2 A schematic diagram of data flow according to an example implementation is shown. Figure 2 As shown in , the data stream may include the original ambisonics model 205 , mixer 220 - 1 , mixer 220 - 2 , filter 225 - 1 , filter 225 - 2 , ring buffer filter bus 215 - 1 , ring buffer filter bus 215 - 2 , and enhanced ambisonics model 230 .

[0024] exist Figure 2 In an example implementation, the original ambient stereo model 205 includes N audio channels. The audio channels can be represented by the intersections of lines in the geodesic polyhedron representing the original ambient stereo model 205. Each channel has an arrow representing the audio direction relative to the user 210. In each direction, plane waves can propagate from evenly spaced directions. Mixers 220-1 and mixers 220-2 can be communicatively coupled to a portion of the N audio channels. For example, mixer 220-1 is shown as being communicatively coupled to three (3) of the N channels, and mixer 220-2 is shown as being communicatively coupled to two (2) of the N channels.

[0025] The ring buffer filter buses 215-1, 215-2 may include a time step (in Figure 2t in the figure), within which time step, the audio with a time step of t seconds (e.g., one second, two seconds, three seconds, five seconds, etc.) is in the form of a circular buffer (as represented by the dotted line from left to right). A bus buffer is a circuit whose I / O pins can be configured as inputs and outputs for receiving and sending data. A circular buffer (also called a circular buffer or circular queue) can be a buffer data structure that operates as if it has a circular shape. For example, the last element in the buffer can be connected to the first element. In audio processing, a filter can be configured to amplify or attenuate an audio signal within a frequency range. Therefore, a circular buffer filter bus can be a buffer with an I / O whose pins can be configured as inputs and outputs to receive and send data, and the circular buffer filter bus operates as an audio signal bus with filters associated with audio channels. The circular buffer filter buses 215-1, 215-2 can be considered to be channels of a circular buffer filter bus.

[0026] Arrow 10 can represent a write operation, wherein the direction of the arrow indicates the direction of the data flow (e.g., source to destination). In some implementations, the output of the filter 225-1, 225-2 can be written in time step t. Two or more time-delayed channels can be read from time step t. The two or more time-delayed channels can be communicatively coupled to one or more time-delayed channels forming a feedback loop. In addition, the two or more time-delayed channels can be communicatively coupled to the mixer 240. Arrow 5 can represent a read operation, wherein the direction of the arrow indicates the direction of the data flow (e.g., source to destination). In some implementations, the two or more time-delayed channels can be read and used for the generation of the enhanced ambient stereo model 230. A portion of the N audio channels can be communicatively coupled to the mixer 240 and / or the enhanced ambient stereo model 230. This portion of the N audio channels can be used for the generation of the enhanced ambient stereo model 230. In the enhanced ambisonics model 230, each direction may be enhanced with a signal from the ring buffer filter bus 215-1, 215-2.

[0027] The original ambisonic model 205 may be defined as an audio source based on polygons on a geodesic polyhedron, such as an icosahedron, a tessellation of a geodesic polyhedron, or other (geodesic) polyhedrons, and / or point sources. Each point on the polygon may correspond to an audio channel of an audio source (e.g., an ambisonic microphone). Thus, the original ambisonic model 205 may have N audio channels. If the ambisonic model is based on a geodesic polyhedron (e.g., Figure 2 ), the original ambient stereo model 205 may have, for example, 12 audio channels.

[0028] The ring buffer filter bus 215-1 and the second ring buffer filter bus 215-2 can be configured to reduce the amount of filtering required to produce reverberation. Each filter bus channel takes its input as a linear mix of the ambisonic channels generated by the mixers 220-1, 220-2 (e.g., a linear mix of the ambisonic channels). The mixing parameters can be communicated and inserted once every t seconds.

[0029] For example, additional commands may be used to manipulate the ring buffer filter buses 215-1, 215-2. For example, changing the read position index may gradually stop reading during a short time window (e.g., 50 ms) and gradually start a new read position during the same time (e.g., by cross-fading the read data). In some examples, parameter changes in the reverberation model for scene changes may be achieved by erasing content or reducing the absolute value of stored values. Parameters for filters of the ring buffer filter buses 215-1, 215-2 may be changed dynamically, and the ring buffer filter buses 215-1, 215-2 may interpolate the parameters during operation.

[0030] In addition, the mixers 220-1, 220-2 can be communicatively coupled to the ring buffer filter buses 215-1, 215-2, thereby forming a feedback loop. In other words, the mixers 220-1, 220-2 can be configured to receive output audio signals from the ring buffer filter buses 215-1, 215-2, mix the audio signals with the audio signals of the original ambient stereo model 205 to serve as inputs to the ring buffer filter buses 215-1, 215-2. The audio mixer can be configured to receive audio from multiple sources, combine the audio, and output the combined audio. In some implementations, mixing the audio signals can include processing the audio signals to adjust (e.g., volume balance) the audio signals.

[0031] Thus, the mixers 220-1, 220-2 can (and / or be configured to) receive time-delayed channels from the ring buffer filter buses 215-1, 215-2. The ring buffer filter bus 215-1, 215-2 signals can be copies of ambient stereo channels, but filtering is applied to the signals. The filters 225-1, 225-2 can include, for example, low-pass filters and high-pass filters, notch filters, etc. The channels in the ring buffer filter buses 215-1, 215-2 can also mix the time-delayed data (both intra-channel and inter-channel) from the ring buffer filter buses in the mixers 220-1, 220-2, thereby allowing complex infinite impulse response (IIR) type reverberation. In an example implementation, the computational complexity of the ring buffer filter buses 215-1, 215-2 and the mixers 220-1, 220-2 can be minimized by limiting the number of intra-channel interactions and channel-to-channel interactions with time delays. Although two ring buffer filter buses 215 - 1 , 215 - 2 and two mixers 220 - 1 , 220 - 2 are shown and described, more than two ring buffer filter buses 215 - 1 , 215 - 2 and two mixers 220 - 1 , 220 - 2 are also within the scope of the present disclosure.

[0032] The mixer 240 can be configured to mix the audio channels from the original ambient stereo model 205 and / or the channels from the ring buffer filter bus 215-1, 215-2. The mixer 240 can be configured to generate an enhanced ambient stereo model 230. Each channel in the enhanced ambient stereo model 230 can be an original ambient stereo model channel mixed with a ring buffer filter bus channel (a time-delayed version). This allows an entropy source (e.g., a sound source) to be encoded and decoded on one of the channels, and in the absence of entropy for coding the early reflections and reverberations on the corresponding channels of the early reflections and reverberations, the channel early reflections and reverberations can occur naturally in the enhanced model. The enhanced ambient stereo model 230 can be defined as being substantially similar to the original ambient stereo model 205. Therefore, the enhanced ambient stereo model 230 can be defined as an audio source based on a polygon on a geodesic polyhedron such as an icosahedron, a geodesic polyhedron subdivision or other (geodesic) polyhedron, and / or a point source.

[0033] Figure 3 A block diagram of a method according to an example implementation is shown. Figure 3As shown in, in step S305, a first portion of N audio channels is selected. In step S310, a second portion of N audio channels is selected. The number of N audio channels that form the first portion of N audio channels and the second portion of N audio channels can be a design choice. For example, the number of N audio channels that form the first portion of N audio channels and the second portion of N audio channels can be the number of ring buffer filter buses 215-1, 215-2 used. N can be based on the ambient stereo model 205. Each filter bus channel gets its input as follows: a linear mix of the ambient stereo channels generated by the mixer 220-1, 220-2 (e.g., a linear mix of the ambient stereo channels).

[0034] In step S315, a first portion of the N audio channels is mixed with the first time-delayed audio channel, and the result is then filtered. In step S320, a second portion of the N audio channels is mixed with the second time-delayed audio channel, and the result is then filtered.

[0035] In step S325, an enhanced ambient stereo model is generated based on the N audio channels, the first mixed and filtered audio channel, and the second mixed and filtered audio channel. Each channel in the enhanced ambient stereo model can be an original ambient stereo model channel mixed with a ring buffer filter bus channel (a time-delayed version). This allows an entropy source (e.g., a sound source) to be encoded and decoded on one of the channels, and in the absence of entropy for coding early reflections and reverberations on the corresponding channels of early reflections and reverberations, channel early reflections and reverberations can occur naturally in the enhanced model.

[0036] An advantage of the example implementations described herein can be that a more compact (eg, one-tenth the number of bits while achieving the same quality of experience) ambisonic format can be constructed because the signal can be decorrelated to a greater extent prior to entropy coding and decoding.

[0037] Figure 4 A block diagram of a system according to an example implementation is shown. Figure 4 In the examples of, a system (e.g., an audio compression system, an audio streaming system, an audio storage system, etc.) may include a computing system or at least one computing device, and should be understood to represent almost any computing device configured to perform the techniques described herein. Thus, a device may be understood to include various components that can be used to implement the techniques described herein or different or future versions thereof. By way of example, a system may include a processor 405 and a memory 410 (e.g., a non-transitory computer-readable memory). The processor 405 and the memory 410 may be coupled (e.g., communicatively coupled) via a bus 415.

[0038] The processor 405 may be used to execute instructions stored on the at least one memory 410. Thus, the processor 405 may implement various features and functions described herein or additional or alternative features and functions. The processor 405 and the at least one memory 410 may be used for various other purposes. For example, the at least one memory 410 may represent examples of various types of memory and related hardware and software that may be used to implement any of the modules described herein.

[0039] At least one memory 410 can be configured to store data and / or information associated with the device. At least one memory 410 can be a shared resource. Therefore, at least one memory 410 can be configured to store data and / or information (e.g., image / video processing or wired / wireless communication) associated with other elements in a larger system. Processor 405 and at least one memory 410 can be used together to implement the technology described herein. Therefore, the technology described herein can be implemented as a code segment (e.g., software) stored on the memory 410 and executed by the processor 405. Therefore, the memory 410 can include an audio mixer 110, a filter bus 115, and an audio mixer 120, each of which is described in more detail above.

[0040] Example 1. Figure 5 is a block diagram of a method for generating an enhanced ambient stereo model according to an example implementation. Figure 5 As shown in , in step S505, an audio signal including multiple audio channels is received. In step S510, a first portion of the multiple audio channels is selected. In step S515, a second portion of the multiple audio channels is selected. In step S520, a first mixed audio channel is generated by mixing the first portion of the multiple audio channels with the first time-delayed audio channel. For example, in some implementations, all channels of the corresponding portion can be mixed with the time-delayed version. For example, in some implementations, a subset of the channels can be mixed with the time-delayed version. For example, in some implementations, a portion of the channels can be mixed with the time-delayed version.

[0041] In step S525, a second mixed audio channel is generated by mixing a second portion of the plurality of audio channels with the second time-delayed audio channel. In step S530, an enhanced ambient stereo model is generated based on the plurality of audio channels, the first mixed audio channel, and the second mixed audio channel. In some implementations, the plurality of audio channels may be buffered, and i) the first time-delayed audio channel may be a version of one of the plurality of buffered audio channels delayed by approximately a predetermined time, and ii) the second time-delayed audio channel may be a version of another of the plurality of buffered audio channels delayed by approximately the same time.

[0042] Example 2. A method as described in Example 1, wherein generating a first mixed audio channel may further include: filtering a first portion of the multiple audio channels mixed with the first time-delayed audio channel into a filtered first mixed audio channel; and generating a second mixed audio channel may further include: filtering a second portion of the multiple audio channels mixed with the second time-delayed audio channel into a filtered second mixed audio channel.

[0043] Example 3. A method as described in Example 2, wherein the first time-delayed audio channel can be selected from a ring buffer filter bus having a time step of the filtered first mixed audio channel. In some implementations, the audio signal can be time-associated with a stamped audio signal corresponding to the audio channel (or portions of the audio channel). The timestamp can correspond to a time point at which the audio sensing device (e.g., a microphone) senses, captures, records, etc. the audio. In some implementations, these audio channels can be mixed audio channels part of a data stream. In some implementations, these audio channels can be stored in a ring buffer filter (in a memory). In some implementations, audio channels associated with the same timestamp (e.g., recorded at the same time) should be processed together. Therefore, in some implementations, when processing audio channels associated with mixed audio channels and a ring buffer filter bus, data with the same timestamp for each audio channel should be selected.

[0044] Example 4. A method as described in Example 2, wherein the second time-delayed audio channel can be selected from a ring buffer filter bus having a time step of the filtered second mixed audio channel. In some implementations, the audio signal can be time-associated with a stamped audio signal corresponding to the audio channel (or portions of the audio channel). The timestamp can correspond to a point in time when the audio sensing device (e.g., a microphone) senses, captures, records, etc. the audio. In some implementations, these audio channels can be mixed audio channels part of a data stream. In some implementations, these audio channels can be stored in a ring buffer filter (in a memory). In some implementations, audio channels associated with the same timestamp (e.g., recorded at the same time) should be processed together. Therefore, in some implementations, when processing audio channels associated with mixed audio channels and a ring buffer filter bus, data with the same timestamp for each audio channel should be selected.

[0045] Example 5. A method as described in Example 1, wherein a first loop filter can be used to filter a first portion of multiple audio channels and a second loop filter can be used to filter a second portion of the multiple audio channels, the method can further include at least one of the following: changing a read position index on the first loop filter and the second loop filter.

[0046] Example 6. A method as in Example 1, wherein the audio signals may be associated with sources (eg, microphone groups) arranged based on an original ambisonic model defined as audio sources based on polygons on a geodesic polyhedron.

[0047] Example 7. The method of Example 1, wherein the generating of the enhanced ambisonics model may include linear mixing of the plurality of audio channels, the first mixed audio channel, and the second mixed audio channel.

[0048] Example 8. A method may include any combination of one or more of Examples 1 to 7.

[0049] Example 9. A non-transitory computer-readable storage medium comprising instructions stored thereon, which when executed by at least one processor are configured to cause a computing system to perform the method of any one of Examples 1 to 8.

[0050] Example 10. An apparatus comprising means for performing the method of any one of Examples 1 to 8.

[0051] Example 11. A device comprising at least one processor and at least one memory comprising computer program code, the at least one memory and the computer program code being configured to cause the device to perform at least the method as described in any one of Examples 1 to 8 through the at least one processor.

[0052] An example implementation may include a non-transitory computer-readable storage medium including instructions stored thereon, which when executed by at least one processor are configured to cause a computing system to perform any of the methods described above. An example implementation may include a device including means for performing any of the methods described above. An example implementation may include a device including at least one processor and at least one memory including computer program code, the at least one memory and the computer program code being configured to cause the device, through at least one processor, to perform at least any of the methods described above.

[0053] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor that can be either special purpose or general purpose and can be coupled to receive data and instructions from and send data and instructions to a storage system, at least one input device, and at least one output device.

[0054] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages ​​and / or in assembly / machine languages. As used herein, the terms "machine-readable medium," "computer-readable medium," refer to any computer program product, apparatus, and / or device (e.g., a disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0055] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (LED (light emitting diode), or OLED (organic LED), or LCD (liquid crystal display) monitor / screen) for displaying information to the user and a keyboard and pointing device (e.g., a mouse or trackball) that the user can use to provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including sound, voice, or tactile input.

[0056] The systems and techniques described herein may be implemented in a computing system that includes a back-end component (e.g., as a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or any combination of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), and the Internet.

[0057] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of the client and the server is generated by computer programs that are executed on the respective computers and cause each other to have a client-server relationship. The components shown here, their connections and relationships, and their functions are intended to be examples only and are not intended to limit the implementation of the present invention described and / or claimed in this document.

[0058] Many embodiments have been described. However, it should be understood that various modifications may be made without departing from the spirit and scope of this specification. In addition, the logic flows depicted in the figures do not require the particular order or sequential order shown to achieve the desired results. In addition, other steps may be provided, or steps may be deleted from the described flows, and other components may be added to or removed from the described systems. Therefore, other embodiments are within the scope of the appended claims.

[0059] Although certain features of the described implementations have been shown as described herein, those skilled in the art will now expect many modifications, substitutions, variations and equivalents. Therefore, it should be understood that the appended claims are intended to cover all these modifications and variations that fall within the scope of the implementations. It should be understood that they are presented only in an exemplary and non-restrictive manner, and various changes in form and detail may be made. Except for mutually exclusive combinations, any part of the devices and / or methods described herein may be combined in any combination. The implementations described herein may include various combinations and / or sub-combinations of the functions, components and / or features of the different implementations described.

[0060] Although the example embodiments may include various modifications and alternative forms, embodiments thereof are shown by way of example in the drawings and will be described in detail herein. However, it should be understood that it is not intended to limit the example embodiments to the particular forms disclosed, but rather, the example embodiments will cover all modifications, equivalents, and alternatives falling within the scope of the claims. The same numbers refer to the same elements in the description of the drawings.

[0061] Some of the above example embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the operations as sequential processes, many operations may be performed in parallel, concurrently, or simultaneously. In addition, the order of the operations may be rearranged. Processes may terminate when their operations are completed, but may also have additional steps not included in the figure. A process may correspond to a method, function, procedure, subroutine, subprogram, etc.

[0062] The methods discussed above (some of which are illustrated by flow charts) may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented by software, firmware, middleware, or microcode, the program code or code segments for performing the necessary tasks may be stored in a machine or computer readable medium, such as a storage medium. One or more processors may perform the necessary tasks.

[0063] The specific structural and functional details disclosed herein are merely representative for describing example embodiments. Example embodiments may, however, be embodied in many alternate forms and should not be construed as limited to only the embodiments set forth herein.

[0064] It should be understood that although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the scope of the exemplary embodiments, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. As used herein, the terms and / or include any and all combinations of one or more of the associated listed items.

[0065] It should be understood that when an element is referred to as being connected or coupled to another element, the element may be directly connected or coupled to the other element, or there may be intervening elements. Conversely, when an element is referred to as being directly connected or coupled to another element, there are no intervening elements. Other words used to describe the relationship between elements should be interpreted in a similar manner (e.g., between versus directly between, adjacent versus directly adjacent, etc.).

[0066] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments. As used herein, the singular forms one, a, and described are also intended to include the plural forms unless the context clearly indicates otherwise. It should be further understood that the terms include, contain, contain, and / or have when used herein specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0067] It should also be noted that in some alternative implementations, the functions / actions noted may not occur in the order noted in the figures. For example, two figures shown in succession may actually be executed simultaneously or may sometimes be executed in the reverse order, depending on the functionality / behaviors involved.

[0068] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the example embodiments belong. It should be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and should not be interpreted in an idealized or overly formal sense unless explicitly defined as such herein.

[0069] Parts of the above exemplary embodiments and corresponding detailed descriptions are presented in terms of software or algorithms and symbolic representations of the operations of the data bits in the computer memory. These descriptions and representations are descriptions and representations that effectively convey the essence of their work to other persons of ordinary skill in the art. As a term used herein and as a commonly used term, an algorithm is considered to be a self-consistent sequence of steps leading to a desired result. A step is a step that requires physical manipulation of a physical quantity. Typically, although not necessarily, these quantities are in the form of optical, electrical or magnetic signals that can be stored, transmitted, combined, compared and otherwise manipulated. Mainly for general reasons, it has sometimes been proven to be convenient to refer to these signals as bits, values, elements, symbols, characters, items, numbers, etc.

[0070] In the above illustrative embodiments, references to symbolic representations of actions and operations that can be implemented as program modules or functional processes (e.g., in the form of flowcharts) include routines, programs, objects, components, data structures, etc., which perform specific tasks or implement specific abstract data types and can be described and / or implemented using existing hardware at existing structural elements. Such existing hardware may include one or more central processing units (CPUs), digital signal processors (DSPs), application specific integrated circuits, field programmable gate arrays (FPGAs), computers, etc.

[0071] It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise specifically noted or as is apparent from the discussion, terms such as processing or computing or calculating or determining display refer to the actions and processes of a computer system or similar electronic computing device that manipulates data represented as physical electronic quantities within the computer system's registers and memories and transforms that data into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage, transmission or display devices.

[0072] It is also noted that the software-implemented aspects of the example embodiments are typically encoded on some form of non-transitory program storage medium or implemented on some type of transmission medium. The program storage medium may be magnetic (e.g., a floppy disk or hard disk) or optical (e.g., a compact disk read-only memory or CD ROM), and may be read-only or random access. Similarly, the transmission medium may be twisted pair, coaxial cable, optical fiber, or some other suitable transmission medium known in the art. The example embodiments are not limited by these aspects of any given implementation.

[0073] Finally, it should also be noted that although the appended claims set forth specific combinations of features described herein, the scope of the present disclosure is not limited to the specific combinations claimed below, but extends to cover any combination of features or embodiments disclosed herein, regardless of whether the specific combination is specifically listed in the appended claims.

Claims

1. A method comprising: receiving an audio signal comprising a plurality of audio channels; selecting a first portion of the plurality of audio channels; selecting a second portion of the plurality of audio channels; generating a first mixed audio channel by mixing the first portion of the plurality of audio channels with a first time-delayed audio channel; generating a second mixed audio channel by mixing the second portion of the plurality of audio channels with a second time-delayed audio channel; as well as An enhanced ambisonics model is generated based on the plurality of audio channels, the first mixed audio channel, and the second mixed audio channel.

2. The method of claim 1, wherein: The generating of the first mixed audio channel further comprises: filtering the first portion of the plurality of audio channels mixed with the first time-delayed audio channel into a filtered first mixed audio channel; and The generating of the second mixed audio channel further comprises filtering the second portion of the plurality of audio channels mixed with the second time delayed audio channel into a filtered second mixed audio channel.

3. The method of claim 2, wherein the first time-delayed audio channel is selected from a ring buffer filter bus having a time step of the filtered first mixed audio channel.

4. The method of claim 2, wherein the second time-delayed audio channel is selected from a ring buffer filter bus having a time step of the filtered second mixed audio channel.

5. The method according to any one of claims 2 to 4, wherein A first loop filter is used to filter the first portion of the plurality of audio channels, and A second loop filter is used to filter the second portion of the plurality of audio channels, the method further comprising at least one of: changing a read position index on the first loop filter and the second loop filter.

6. The method of claim 1, wherein the audio signal is associated with a source arrangement based on an original ambisonic model, the original ambisonic model being defined as audio sources based on polygons on a geodesic polyhedron.

7. The method of claim 1, wherein the generating of the enhanced ambisonics model comprises linear mixing of the plurality of audio channels, the first mixed audio channel, and the second mixed audio channel.

8. A non-transitory computer-readable storage medium comprising instructions stored thereon, the instructions being configured, when executed by at least one processor, to cause a computing system to: receiving an audio signal comprising a plurality of audio channels; selecting a first portion of the plurality of audio channels; selecting a second portion of the plurality of audio channels; generating a first mixed audio channel by mixing the first portion of the plurality of audio channels with a first time-delayed audio channel; generating a second mixed audio channel by mixing the second portion of the plurality of audio channels with a second time-delayed audio channel; as well as An enhanced ambisonics model is generated based on the plurality of audio channels, the first mixed audio channel, and the second mixed audio channel.

9. The non-transitory computer-readable storage medium of claim 8, wherein: The generating of the first mixed audio channel further comprises: filtering the first portion of the plurality of audio channels mixed with the first time-delayed audio channel into a filtered first mixed audio channel; and The generating of the second mixed audio channel further comprises filtering the second portion of the plurality of audio channels mixed with the second time delayed audio channel into a filtered second mixed audio channel.

10. The non-transitory computer-readable storage medium of claim 9, wherein the first time-delayed audio channel is selected from a ring buffer filter bus having a time step of the filtered first mixed audio channel.

11. The non-transitory computer-readable storage medium of claim 9, wherein the second time-delayed audio channel is selected from a ring buffer filter bus having a time step of the filtered second mixed audio channel.

12. The non-transitory computer-readable storage medium of claim 8, wherein the audio signals are in an arrangement based on an original ambisonic model defined as audio sources based on polygons on a geodesic polyhedron.

13. The non-transitory computer-readable storage medium of claim 8, wherein the generating of the enhanced ambisonics model comprises linear mixing of the plurality of audio channels, the first mixed audio channel, and the second mixed audio channel.

14. A device comprising: at least one processor; as well as at least one memory, the at least one memory comprising computer program code; The at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to: receiving an audio signal comprising a plurality of audio channels; selecting a first portion of the plurality of audio channels; selecting a second portion of the plurality of audio channels; generating a first mixed audio channel by mixing the first portion of the plurality of audio channels with a first time-delayed audio channel; generating a second mixed audio channel by mixing the second portion of the plurality of audio channels with a second time-delayed audio channel; as well as An enhanced ambisonics model is generated based on the plurality of audio channels, the first mixed audio channel, and the second mixed audio channel.

15. The apparatus of claim 14, wherein: The generating of the first mixed audio channel further comprises: filtering the first portion of the plurality of audio channels mixed with the first time-delayed audio channel into a filtered first mixed audio channel; and The generating of the second mixed audio channel further comprises filtering the second portion of the plurality of audio channels mixed with the second time delayed audio channel into a filtered second mixed audio channel.

16. The apparatus of claim 15, wherein the first time-delayed audio channel is selected from a ring buffer filter bus having a time step of the filtered first mixed audio channel.

17. The apparatus of claim 15, wherein the second time-delayed audio channel is selected from a ring buffer filter bus having a time step of the filtered second mixed audio channel.

18. The apparatus of claim 15, wherein A first loop filter is used to filter the first portion of the plurality of audio channels, and A second loop filter is used to filter the second portion of the plurality of audio channels, and the computer program code is further configured to at least one of: change a read position index on the first loop filter and the second loop filter.

19. The apparatus of claim 14, wherein the audio signal is a raw ambisonic model defined as audio sources based on polygons on a geodesic polyhedron.

20. The apparatus of claim 14, wherein the generating of the enhanced ambisonics model comprises linear mixing of the plurality of audio channels, the first mixed audio channel, and the second mixed audio channel.

Images