System and method for providing interactive audio in a multi-channel audio environment

Abstract

DTS Interactive provides low cost fully interactive immersive digital surround sound environment suitable for 3D gaming and other high fidelity audio applications, which can be configured to maintain compatibility with the existing infrastructure of Digital Surround Sound decoders. The component audio is stored and mixed in a compressed and simplified format that reduces memory requirements and processor utilization and increases the number of components that can be mixed without degrading audio quality. Techniques are also provided for “looping” compressed audio, which is an important and standard feature in gaming applications that manipulate PCM audio. In addition, decoder sync is ensured by transmitting frames of “silence” whenever mixed auedio is not present either due to processing latency or the gaming application.

Description

[0001] This Application is a divisional of U.S. Ser. No. 09 / 432,917 filed on 2 Nov. 1999 and claims priority of that application.FIELD OF THE INVENTION [0002] This invention relates to fully interactive audio systems and more specifically to a system and method of rendering real-time multi-channel interactive digital audio to create a rich immersive surround sound environment suitable for 3D gaming, virtual reality and other interactive audio applications. BACKGROUND OF THE INVENTION [0003] Recent developments in audio technology have focused on creating real-time interactive positioning of sounds anywhere in the three-dimensional space surrounding a listener (the “sound field”). True interactive audio provides not only for the ability to create the sound on-demand but the ability to position that sound precisely in the sound field. Support for such technologies can be found in a variety of products but most frequently in software for video games to create a natural, immersive, and ...

AI Technical Summary

Benefits of technology

[0010] In view of the above problems, the present invention provides a low cost fully interactive immersive digital surround sound environment suitable for 3D gaming and other high fidelity audio applications, which can be configured to maintain compatibility with the existing infrastructure of Digital Surround Sound decoders.

[0012] More specifically, the components are preferably encoded into a subband representation, compressed and packed into a data frame in which only the scale factors and subband data change from frame-to-frame. This compressed format requires significantly less memory than standard PCM audio but more than that required by variable length code storage such as used in Dolby AC-3 or MPEG. More significantly this approach greatly simplifies the unpack / pack, mix and decompress / compress operations thereby reducing processor utilization. In addition, fixed length codes (FLCs) aid the random access navigation through an encoded bitstream. High levels of throughput can be achieved by using a single predefined bit allocation table to encode the source audio and the mixed output channels. In the currently preferred embodiment, the audio renderer is hardcoded for a fixed header and bit allocation table so that the audio renderer only need process the scale factors and subband data.

[0013] Mixing is achieved by partially decoding (decompressing) only the subband data from components that are considered audible and mixing them in the subband domain. The subband representation lends itself to a simplified psychoacoustic masking technique so that a large number of sources can be rendered without increasing processing complexity or reducing the quality of the mixed signal. In addition, since multi-channel signals are encoded into their compressed format prior to transmission, a rich high-fidelity unified surround sound signal can be delivered to the decoder over a single connection.

Problems solved by technology

However, this is often impractical and experimenters have searched for a set of general HRTFs that have good performance for a wide range of listeners.

This has been difficult to accomplish with a specific obstacle being front-back confusion, which describes the sensation that sounds either in front of or behind the head are coming from the same direction.

Although virtual surround technologies based on HRTFs provide significant benefits in situations where full home theater set-ups are not practical these current solutions do not provide any means for interactive positioning of specific sounds.

The surround sound tracks are pre-encoded in Dolby Surround format, and thus they are best suited for movies, and are not particularly useful in interactive applications such as video games.

Unfortunately, mixing PCM with Dolby Surround Sound is content dependant and overlaying PCM audio on the Dolby Surround sound audio tends to confuse the Dolby Prologic decoder, which can create undesirable surround artifacts and crosstalk.

Digital surround is a pre-recorded technology and thus best suited for movies and home A / V systems where the decoding latency can be nulled and in its present form it is not particularly useful for interactive applications such as video games.

Although a step forward, Desktop Theater is not compatible with the existing installed base of Dolby Digital decoders and requires sound cards supporting multiple channels of PCM output.

The richness of the DeskTop Theater's interactive audio is further limited by the computation requirements needed to process the PCM data.

Sideways localization, which is a critical component of a positional audio environment is computationally expensive to apply on time-domain data, as are the operations of filtering and equalization.

Images