Apparatus and method for providing seat-based natural landscape audio

Abstract

In at least one embodiment, a system including a mobile device is provided. The mobile device is located around a first seat in a vehicle. The mobile device includes at least one first controller programmed to communicate with an audio controller located in the vehicle and to generate, on a display, one or more options corresponding to a desired natural sound for selection. The at least one first controller is further programmed to transmit one or more options to select a desired natural sound on the display and to transmit a first signal indicative of the selected desired natural sound to the audio controller to play the selected desired natural sound in the vehicle.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 616,233, filed on December 29, 2023, the entire content of which is hereby incorporated by reference herein. Technical Field

[0003] Aspects disclosed herein generally relate to an apparatus and method for providing seat - based natural landscape audio. These and other aspects will be discussed in more detail herein. Background Art

[0004] The listening environment provided by a vehicle is very different from those provided by different venues such as, for example, stadiums, festivals, recording studios, concert halls (e.g., large, medium, or small concert halls), etc. Generally, audio recordings of audio captured in stadiums, festivals, recording studios, and concert halls may not be fully transferable to the listening environment of a vehicle. For example, the listening environment inside a vehicle is much smaller than those of the aforementioned venues. Additionally, the listening environment of a vehicle provides various body enclosures (e.g., front side, rear side, left / right side, and top) of different sizes and shapes when compared to the respective sides or walls of the above - mentioned venues. The different sizes and shapes that define the listening environment of a vehicle make it difficult to transfer sound recordings captured in other venues into the vehicle. As a result, the user experience in the vehicle is affected. Summary of the Invention

[0005] In at least one embodiment, a system including a mobile device is provided. The mobile device is located around a first seat in a vehicle. The mobile device includes at least one first controller programmed to communicate with an audio controller in the vehicle and generate one or more options corresponding to desired natural sounds for selection on a display. The at least one first controller is also programmed to transmit the one or more options to select a desired natural sound on the display and transmit a first signal indicating the selected desired natural sound to the audio controller to play the selected desired natural sound in the vehicle.

[0006] In at least one embodiment, a system for generating audio in a listening environment of a vehicle is provided. The system includes a memory device and at least one audio controller. The at least one audio controller includes the memory device and is programmed to receive a first signal indicating a first selected desired natural sound from a first mobile device located around a first seat of the vehicle, and to receive a second signal indicating a second selected desired natural sound from a second mobile device located around a second seat of the vehicle, the second selected desired natural sound being different from the first selected desired natural sound. The at least one audio controller is further programmed to generate a first audio output signal including the first selected desired natural sound and a second audio output signal including the second selected desired natural sound, and to transmit the first audio output signal to one of the first mobile device and a first speaker located around the first seat to play the first audio output signal. The at least one audio controller is further programmed to transmit the second audio output signal to one of the mobile device and a second speaker located around the second seat to play the second audio output signal.

[0007] In at least another embodiment, a method for generating audio in a listening environment of a vehicle is provided. The method includes receiving a first signal indicating a first selected desired natural sound from a first mobile device located around a first seat of the vehicle, and receiving a second signal indicating a second selected desired natural sound from a second mobile device located around a second seat of the vehicle, the second selected desired natural sound being different from the first selected desired natural sound. The method further includes generating a first audio output signal including the first selected desired natural sound and a second audio output signal including the second selected desired natural sound, and transmitting the first audio output signal to one of the first mobile device and a first speaker located around the first seat to play the first audio output signal. The method further includes transmitting the second audio output signal to one of the mobile device and a second speaker located around the second seat to play the second audio output signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The embodiments of the present disclosure are particularly pointed out in the appended claims. However, other features of the various embodiments will become more apparent and best understood by referring to the following detailed description in conjunction with the accompanying drawings, in which:

[0009] Figure 1 An example of a sound processing system for generating natural sounds in a vehicle is shown;

[0010] Figure 2 An audio profile for providing spatial mixing based on a scenic environment, environmental conditions, and a user's emotional state is shown;

[0011] Figure 3 Shows a schematic depiction of an example active control;

[0012] Figure 4 Shows a schematic depiction of an active control having an example control region;

[0013] Figure 5 Shows a schematic depiction of two active controls having an example control region;

[0014] Figure 6 Shows an audio attribute graph that can be modified according to the Figures 4 to 6 status of the user interface shown;

[0015] Figure 7 Shows an overview of a vehicle and its corresponding listening environment for a virtual venue;

[0016] Figure 8 Shows a method for generating a virtual venue within a vehicle's listening environment;

[0017] Figure 9 Shows a detailed view of a device for generating a virtual venue within a vehicle's listening environment according to one embodiment;

[0018] Figure 10 Shows an example of a user interface for controlling aspects related to a natural landscape and a virtual venue according to one embodiment;

[0019] Figure 11 Shows another example of a first selection control displayed on a user interface according to another embodiment;

[0020] Figures 12 to 21 Shows a second selection control displayed on a user interface according to one embodiment;

[0021] Figure 22 Shows an example of a sphere associated with the second selection control according to one embodiment;

[0022] Figure 23 Shows an example of a venue and corresponding nature-related features that can be presented on a user interface according to one embodiment, the corresponding features including a natural landscape;

[0023] Figure 24 Shows an example of a global selection area presented on a user interface on a display according to one embodiment;

[0024] Figure 25 Shows another example of a global selection area presented on a user interface on a display according to one embodiment;

[0025] Figure 26 Shows an example of an image of an actual natural scene presented on a user interface on a display according to one embodiment;

[0026] Figure 27 Shows an example of various control mechanisms for controlling aspects related to a natural landscape according to one embodiment;

[0027] Figure 28 Shows another example of various control mechanisms for controlling aspects related to a natural landscape according to one embodiment;

[0028] Figure 29 Shows another example of various control mechanisms for controlling aspects related to a natural landscape according to another embodiment;

[0029] Figure 30 Shows a schematic diagram for generating a natural landscape using an audio system according to one embodiment;

[0030] Figure 31 Shows a method for generating audio in an occupant compartment of a vehicle according to one embodiment;

[0031] Figure 32 Shows an example of a seat - based user interface for implementing playback of one or more natural landscapes for one or more occupants located in a vehicle;

[0032] Figure 33 Shows another example of a seat - based user interface for implementing one or more natural landscapes for one or more occupants located in a vehicle;

[0033] Figure 34 Shows another example of a seat - based user interface for implementing one or more natural landscapes for one or more occupants located in a vehicle;

[0034] Figure 35 Depicts a detailed example of touch points with respect to various natural landscapes according to one embodiment;

[0035] Figure 36 Depicts a detailed example of the way an occupant selects various natural landscapes and levels of immersion for touch points according to one embodiment;

[0036] Figure 37 Depicts a tutorial according to one embodiment that shows the way an occupant selects a natural landscape and level of immersion for a touch point; and

[0037] Figure 38 Depicts a schematic diagram for generating a seat-based natural landscape using an audio system according to an embodiment. Detailed Description

[0038] The present disclosure will be described in detail below with reference to exemplary embodiments of the present disclosure. However, the present disclosure is not limited to the embodiments described herein, but may be implemented in many different forms. The described embodiments are only intended to make the present disclosure thorough and complete, and to fully convey the concept of the present disclosure to those skilled in the art. The features of the various embodiments described may be combined with or substituted for each other unless explicitly excluded or excluded by context.

[0039] Aspects disclosed herein generally provide a system for providing audio-related features to one or more passengers in a vehicle. For example, the disclosed system may provide audio-related features that include enabling an occupant to play natural sounds while adjusting various virtual venues. An example of an audio system capable of playing natural sounds in a vehicle is set forth in U.S. Patent No. 11,485,231, entitled "SYSTEMS AND METHODS FOR PROVIDING NATURE SOUNDS," filed on December 4, 2020, the entire content of which is hereby incorporated by reference. Additionally, an example of an audio system capable of playing audio in a vehicle based on a virtual venue is set forth in U.S. Patent No. 10,728,691, entitled "APPARATUS AND METHOD FOR GENERATING VIRTUAL VENUES FOR A LISTENING ROOM," filed on August 29, 2019, the entire content of which is hereby incorporated by reference.

[0040] By broadcasting sound elements based on at least one vehicle state, natural sounds can be automatically played in a vehicle. For example, when the vehicle is traveling in a scenic area, sound elements corresponding to the natural sounds in the scenic area can be broadcast so as to incorporate elements of the external environment into the vehicle's sound environment. Additionally, when an environmental condition (such as weather) is detected, sound elements corresponding to the environmental condition can be played to alert the user of changes in the environmental condition. As another example, a user may be stressed, such as due to a tense driving environment (e.g., traffic), and sound elements corresponding to a more peaceful external environment can be played, thus reducing the user's stress level. Further, each sound element played in the vehicle can be spatially mixed to create an immersive surround sound experience. For example, each sound element can be indexed to a virtual location outside the vehicle, and the virtual locations outside the vehicle can be mapped to a virtual speaker space inside the vehicle.

[0041] In this way, the sound environment inside the vehicle can be adjusted based on at least one state of the vehicle, such as vehicle location, environmental condition, and the user's emotional state (e.g., mood). For example, by adjusting the sound environment based on at least one state of the vehicle, the user's engagement with the external environment can be increased, which can improve the user's satisfaction when driving the vehicle. Additionally, in some examples, adjusting the sound environment based on at least one state of the vehicle can reduce the user's stress level by reducing the impact of a tense external environment (such as when driving in traffic). In summary, the user's stress level can be reduced while the user's satisfaction is increased.

[0042] For the listening environment inside a vehicle, aspects disclosed herein also generally provide a vehicle audio device that changes audio parameters to make the audio or user experience inside the vehicle sound like a different venue (e.g., a stadium, a concert hall (i.e., a large, small, or medium-sized concert hall), a festival event, or a recording studio). For example, the vehicle audio device can change audio parameters based on a user selection to make the vehicle sound like it is in Carnegie Hall, Yankee Stadium, etc. Additionally, the vehicle audio device can be arranged to simply make the listening environment inside the vehicle sound larger than it actually is (e.g., if the vehicle is a compact vehicle, the vehicle audio device can provide an audio experience that sounds like the user is in a sport utility vehicle (“SUV”)). Thus, the disclosed embodiments can enable a vehicle to provide natural sounds and further enhance the playback of such natural sounds by enabling vehicle occupants to selectively increase the feeling that the driver or occupant is actually located in a natural scene that provides the natural sounds. These and other aspects will be discussed in more detail below.

[0043] Devices and methods for rendering natural audio and virtual venue-based audio

[0044] Figure 1 FIG. 1 is a block diagram of a vehicle 100 that includes an exemplary audio or sound processing system (AS) (or audio system) 102, which may include any one or combination of the sound processing systems and methods described below. The vehicle 100 includes vehicle doors 103, a driver's seat 109, a passenger seat 110, and a rear seat 112. Although a four-door vehicle is shown as including vehicle doors 103-1, 103-2, 103-3, and 103-4, the audio system (AS) 102 may also be used in vehicles having more or fewer doors. The vehicle 100 may be an automobile, a truck, a boat, etc. Although only one rear seat is shown, larger vehicles may have multiple rows of rear seats. Smaller vehicles may have only one or more seats. Although a specific exemplary configuration is shown, other configurations may be used, including those having fewer or additional components.

[0045] The audio system 102, which may include an amplifier and / or at least one other audio controller 175 for receiving, processing, and / or outputting audio to one or more speakers in the vehicle, may improve the spatial characteristics of a surround sound system. The audio system 102 may include various audio components such as a radio, CD, DVD, and their derivatives. The audio system 102 may include, for example, 2-channel source material (such as direct left and right, 5.1-channel, 6.2-channel, 7-channel, 12-channel, and / or any other source material from discrete source material digitally encoded / decoded by a matrix decoder, etc.). The audio system 102 may utilize multiple audio channels. For example, the audio system 102 may use a first channel for primary media such as one or more of warnings, media, navigation, and phone / telematics sound elements. Additionally, a second channel may be used for secondary media such as, for example, natural landscape features. Thus, both the primary media and the secondary media may be played substantially simultaneously via the audio system 102.

[0046] The reproduction of the amplitude and phase characteristics of the source material and the specific sound field characteristics in the listening environment may play a role in the successful reproduction of the surround sound field. Thus, in at least one example, the audio system 102 may increase the reproduction of the surround sound field by controlling the sound delay time, surround upmixer parameters (such as wrap, reverberation chamber size, etc.), amplitude, phase, and the mixing ratio between the discrete and passive decoder surround signals and / or the direct two-channel output signal. The amplitude, phase, and mixing ratio between the discrete and passive decoder output signals may be controlled. Especially in a vehicle environment, the spatial sound field reproduction at all seat positions may be increased by reorienting the direct, passive, and active mixing and manipulation parameters.

[0047] The mixing ratio, manipulation ratio, and spectral characteristics can be adaptively modified based on noise and other environmental factors. For example, the mixing ratio and manipulation ratio can be adjusted based on the sensed sound level outside the vehicle. As another example, the mixing ratio and manipulation ratio can be adjusted based on the estimated engine noise generated by the vehicle (such as increasing the volume in response to increased engine noise). In a vehicle, information from a data bus, microphones, and other sensing devices can be used to control the mixing and manipulation parameters.

[0048] Vehicle 100 includes a front center speaker (CTR speaker) 124, a left front speaker (FL speaker) 113, a right front speaker (FR speaker) 115, and at least a pair of surround speakers.

[0049] The surround speakers can be a left side speaker (LS speaker) 117 and a right side speaker (RS speaker) 119, a left rear speaker (LR speaker) 129 and a right rear speaker (RR speaker) 130, or a combination of speaker groups. Other speaker groups can be used. Although not shown, there may be one or more dedicated subwoofers or other drivers. Possible subwoofer mounting locations include the trunk 105, under the seats, or the rear parcel shelf 108. Vehicle 100 can also have one or more microphones 174 mounted inside. For example, at least one microphone 174 can be mounted near each seat of vehicle 100 to capture audio from the user, such as voice commands.

[0050] Each of the CTR speaker 124, FL speaker 113, FR speaker 115, LS speaker 117, RS speaker 119, LR speaker 129, and RR speaker 130 can include one or more transducers having a predetermined frequency response range, such as a tweeter, a midrange speaker, or a woofer. The tweeter, midrange speaker, or woofer can be mounted adjacent to each other at substantially the same location or at different locations. For example, the FL speaker 113 can be a tweeter located in the door 103-1 or at other locations at or above approximately the height of the side mirror. The FR speaker 115 can have an arrangement similar to that of the FL speaker 113 on the right side of vehicle 100 (e.g., in the door 103-2).

[0051] The LR speaker 129 and the RR speaker 130 can each be a woofer installed in the rear parcel shelf 108. The CTR speaker 124 can be installed in the front instrument panel 107, in the roof, on or near the rearview mirror, or elsewhere in the vehicle 100. In other examples, other configurations of speakers with other frequency response ranges are possible. In some embodiments, additional speakers can be added to the upper pillars of the vehicle to enhance the height of the sound image. For example, the upper pillars can include vertical or nearly vertical supports in the automotive window area. In some examples, additional speakers can be added to the upper region of the "A" pillar facing the front of the vehicle.

[0052] Further, in some examples, one or more speakers can be incorporated into one of the vehicle's seats (e.g., the driver's seat 109, the passenger seat 110, and the rear seat 112 (or the left rear passenger seat 112a and the right rear passenger seat 112b)) to improve sound quality. For example, the speakers can be integrated into the headrests of each of the driver's seat 109, the passenger seat 110, and the rear seat 112. For example, the left rear speaker 129 can be incorporated into the headrest of the rear seat 112, and the RR speaker 130 can be incorporated into the headrest of the rear seat 112. As another example, the FL speaker 113 can be incorporated into the headrest of the driver's seat 109, and the FR speaker 115 can be incorporated into the headrest of the passenger seat 110. For example, incorporating speakers into the headrests of the vehicle's seats can achieve more targeted mixing, such as increasing the media volume for a first user without significantly affecting the media volume of a second user. Additionally, incorporating speakers into the headrests of the vehicle's seats can increase user immersion in the media (such as in ambient sound elements reproduced via the audio system 102).

[0053] To enhance the media experience of the occupants of vehicle 100, the audio system 102 may include natural landscape features. When the natural landscape features are activated, the audio system 102 may reproduce a plurality of sound elements (e.g., reproduced as secondary media via a second channel), such as pre-recorded natural sound elements and the reproduction of artificially generated natural sound elements. The plurality of sound elements may be calibrated to provide a sonic representation of a natural environment or natural scene (such as a remote natural environment or the environment outside the vehicle). In some examples, the plurality of sound elements may be provided in response to a user input (e.g., in a manual mode of the natural landscape feature), while in other examples, the plurality of sound elements may be provided based on sensor data (e.g., in an automatic mode of the natural landscape feature). In the manual mode of the natural landscape feature, the user may select a natural scene having associated sound elements, and the user may adjust the volume and frequency of the associated sound elements. In the automatic mode of the natural landscape feature, at least one vehicle controller 180 (“vehicle controller 180”) may determine one or more states associated with vehicle 200 based on sensor data (such as, for example, vehicle location, environmental conditions, and the emotional state of the user), and may determine the sound elements to be played based on the one or more states.

[0054] Both primary media (e.g., such as music, voice calls, talk radio, etc.) and secondary media (e.g., provided by the natural landscape feature) may be played via the audio system 102. In some examples, the secondary media may be layered over the primary media without any adjustment to each of the primary media and the secondary media. As another example, the primary media may duck in response to the secondary media playback in order to reduce the overall volume in the vehicle and enhance the sound quality of the secondary media. In some examples, the duck amount may be adjusted based on the media type. For example, since the sound spectrum of music is wider than that of talk radio, the degree of ducking for talk radio may be lower than that for music. As another example, the secondary media may be muted during a voice call (e.g., via the primary media channel) in order to increase the clarity of the voice call. However, in some examples, the user may choose to keep the secondary media on during a voice call in order to share the sound experience via the voice call.

[0055] In addition, in each of the manual mode of the natural landscape feature and the automatic mode of the natural landscape feature, spatial mixing can be used to provide an immersive three-dimensional sound environment. For example, sound elements can be spatially mixed such that each sound element is perceived as originating from a certain spatial location. For example, the sounds of wild animals (e.g., bird calls) can be spatially mixed such that the sounds of wild animals are perceived as originating from different spatial locations. For example, each sound element can be continuously mapped to the virtual speaker area 162 or different speaker elements, and the virtual speaker area and / or each speaker can be adjusted to reproduce the perceived spatial location of the sound element. For example, each sound element can be indexed to a virtual location in the virtual sound space 164 (e.g., representing the space around the vehicle 100), and there can be a 1:1 mapping between the virtual sound space and the virtual speaker area 162. For example, the in-vehicle computing system can adjust the audio gain, panning settings, and other audio settings of each speaker of the audio system 102 based on the virtual location of the sound element. In addition, the desired sound location may be perceived as being inside or outside the vehicle cabin. As a first example, the virtual location in the virtual sound space 164 can be a location outside the vehicle. As a second example, the virtual location in the virtual sound space 164 can be inside the vehicle.

[0056] In addition, in some examples, the composition of the sound elements and / or the virtual spatial location of the sound elements can be adjusted based on vehicle characteristics such as movement, speed, location, or proximity. As an example, as the vehicle 100 travels through a scenic environment, various sound elements can be adjusted to impart a sense of movement. For example, as the vehicle moves closer to a waterfall, the sound elements corresponding to the waterfall can increase in volume. As another example, as the vehicle 100 moves laterally, the sounds of wild animals may pan from the first side of the vehicle to the second side of the vehicle 100. By spatially mixing the sound elements, a surround sound, immersive experience can be provided to the user, thereby increasing user satisfaction.

[0057] The sound elements can be considered synthetic (e.g., not corresponding to sounds in the external environment of vehicle 100) or real (e.g., corresponding to sounds in the external environment of vehicle 100), and can be spatially mixed to be perceived as spatially located. As a non-limiting example, a first sound element can represent the chirping of a bird and can be played via audio system 102 in response to the vehicle's position. For example, the chirping of a bird may be a synthetic sound and may not correspond to a specific chirping of a bird detected outside the vehicle. The chirping of a bird can be indexed to a virtual position 171 in a virtual sound space 164 around vehicle 100. The virtual positions in the virtual sound space can be mapped to positions in a virtual speaker region 162, such as position 170. To spatially mix the first sound element, the first sound element can be panned to the left speaker 117 and the left rear speaker 129. For example, the audio gain of the first sound element may be highest for the left speaker 117 and the left rear speaker 129, and may be lower for the other speakers of vehicle 100. For example, due to the known relationship between audio panning and the perceived spatial position (such as surround sound techniques known in the art (e.g., such as 5.1 surround sound, 7.1 surround sound, stereo reverberation surround sound, etc.)), each sound element may be perceived as originating from a certain spatial position. Additionally, some sound elements can be spatially mixed by upmixing one or two channels to multiple speakers.

[0058] As another non-limiting example, a second sound element can represent a real sound detected outside the vehicle. For example, a microphone 174 can be coupled to the exterior of the vehicle and can be configured to detect real sounds. In response to detecting a desired natural sound (such as lightning), the second sound element can be played inside the vehicle to represent the natural sound. In some examples, the sound element can be a recording of an external sound captured by microphone 174, while in other examples, the sound element can be a pre-recorded or computer-generated representation of an external sound, such as a pre-recorded sound file of lightning. Lightning can be indexed to a virtual position 173 in a virtual sound space 164 around the vehicle. The virtual positions in the virtual sound space can be mapped to positions in a virtual speaker region 162, such as position 172. To spatially mix the second sound element, the second sound element can be primarily panned to the right speaker 119 and the right front speaker 115. For example, the audio gain of the first sound element may be highest for the right speaker 119 and the right front speaker 115, and may be lower for the other speakers of vehicle 100.

[0059] Some of the sound elements may be spatially mixed to ambient sounds. For example, a third sound element may represent rain, and may be spatially mixed to be perceived as surrounding the vehicle. For example, according to surround sound techniques known in the art, the third sound element may be played through each speaker of the vehicle 100 at approximately the same volume.

[0060] In this way, due to the known relationship between audio panning and the perceived spatial location (such as surround sound techniques known in the art (e.g., such as 5.1 surround sound, 7.1 surround sound, stereo reverb surround sound, etc.)), each sound element may be perceived as originating from a certain spatial location. Some of the sound elements may be real sound elements and may correspond to (e.g., such as detected by the microphone 174) sounds detected outside the vehicle. Other sound elements may be synthetic sound elements and may not directly correspond to sounds detected outside the vehicle. Additionally, some of the sound elements may be spatially mixed by upmixing two channels to multiple speakers. For example, a sound element may have been recorded by a two-track recording device, but may be upmixed to multiple speakers in the vehicle 100 to be perceived as surround sound.

[0061] In some examples, the audio system 102 may be in an automatic mode for natural landscape features and may provide sound elements based on one or more states of the vehicle 100. Turning now to Figure 2 , three vehicle states are shown, each vehicle state having an associated audio profile. For example, the vehicle 100 may be operated by at least one user. One or more states associated with the vehicle 100 may include (by way of non-limiting example) the weather outside the vehicle 100, the speed of the vehicle, the environment in which the vehicle 100 is traveling, the occupancy state of the vehicle, the emotional state (e.g., mood) of one or more occupants of the vehicle 100, and the geographical location of the vehicle 100. Specifically, the vehicle controller 180 may monitor the presence of the vehicle 100 in the scenic area 400 (e.g., such as a park, beach, historical site, forest, etc.), the environmental conditions 425 (e.g., weather-related changes, such as one of an impending storm and a heatwave), and the emotional state 450 of one or more vehicle occupants (e.g., such as anxiety due to traffic).

[0062] If it is determined that the vehicle 100 is located in the scenic area 400, multiple sound elements may be provided. Specifically, based on the detected scenic area, the vehicle controller 180 may select and / or generate sound elements to bring the external environment into the vehicle 100 through an immersive sound experience. For example, ambient natural sounds 202, such as ambient sounds associated with the detected scenic area, such as rain, wind, ocean waves, etc., may be played. In addition, multiple spatial mixes (such as a first spatial mix 204 and a second spatial mix 206) may be perceived as originating from different locations. Further, information about the scenic area 208 may also be provided to enrich the user's experience of the scenic area. For example, the vehicle controller 180 may access information related to the detected scenic area and may provide this information to the user. While providing this secondary media due to the scenic area, the primary media 210 may optionally continue to be played via the audio system 102. For example, the primary media 210 may be adjusted based on the secondary media.

[0063] If environmental conditions 225 are detected, multiple sound elements may be provided. For example, the environmental conditions may be weather-related changes, and the natural landscape features may alert the user about the changing environmental conditions. For example, multiple spatial mix elements, such as a first spatial mix element 226 and a second spatial mix element 228, may be provided to alert the user about the type of environmental change and the location of the environmental change. In addition, information and / or warnings about the environmental change 230 may be provided. In some examples, other media 232 (such as lights) may be provided to further alert the user about the environmental change. For example, the lights may reproduce the effect of lightning, or warning lights may flash to alert the user about temperature changes.

[0064] If a user mood state 250 is detected (e.g., such as user stress or other adverse user moods), natural sounds may be provided to reduce user stress and increase user enjoyment. For example, although a user may be driving through traffic, sounds of a beach may be provided. For example, ambient natural sounds 252 (e.g., such as ocean waves) may create a baseline for a stress-relieving sound environment, and spatial mixing (such as a first spatial mix 254 and a second spatial mix 256) may increase the realism of the sound environment by providing spatially located sound elements. Additionally, the primary media 258 may continue to play to minimize disruption to the user experience. For example, the primary media 258 and the secondary media may be mixed such that the primary media 258 remains audible while the secondary media reduces user stress by reducing the effects of a tense external environment. Additionally, the audio controller 175 may adjust the secondary media based on a combination of vehicle conditions. For example, the audio controller 175 may provide the secondary media based on a scenic area but may further adjust the secondary media based on the user's mood state. As another example, the audio controller 175 may provide the secondary media based on environmental conditions but may adjust the secondary media based on the user's mood state.

[0065] Accordingly, based on the state associated with the vehicle 100 (determined by the vehicle controller 180 based on vehicle conditions and / or user input), the vehicle controller 180 may adjust the audio system 102 to provide an immersive sound experience. In other words, the vehicle controller 180 may change the sound environment in the passenger compartment by playing sound elements corresponding to a natural environment or natural scene. Accordingly, in some examples, the vehicle controller 180 may increase the enhancement of external environmental sounds, while in other examples, the vehicle controller 180 may reduce or even stop the enhancement of external environmental sounds. As an example, the sound elements may be calibrated to enhance the user's experience of the current vehicle location. For example, the vehicle 100 may be driving through a scenic area, such as one of a park, a forest, a beach, and a wildlife reserve. To enhance the user's experience of the scenic area, sound elements corresponding to the scenic area may be provided, such as ambient natural sound elements (e.g., such as wind and rain), sound elements of wildlife, etc. Additionally, information about the scenic environment may be provided. As another example, in response to an environmental change (such as an approaching storm), sound elements corresponding to the environmental change (e.g., such as pre-recorded lightning) and warning messages may be provided to alert the user of the environmental change. In this way, characteristics of the external environment of the vehicle 100 may be introduced via the audio system 102, thereby increasing the integration of the user with the external environment.

[0066] However, in some examples, the opposite effect may be desired, and the user may wish to reduce the impact of the external environment. For example, vehicle 100 may be in traffic or may be located in a stressful urban environment. Such environments may increase the user's stress level, which may in turn reduce user satisfaction. Therefore, a sound element corresponding to a remote location may be preferred. For example, while the user is in traffic in an urban environment, a sound element corresponding to a peaceful forest, such as one or more of a gentle breeze, a waterfall, and bird song, may be provided. In this way, user stress can be reduced. In some examples, the user stress may be estimated based on navigation data provided from a navigation system in vehicle 100. For example, when vehicle controller 180 (e.g., based on the navigation data) determines that vehicle 100 is in a traffic jam, vehicle controller 180 may determine that the user's emotional state includes stress. Additionally, in some examples, the user stress level may be determined based on biometric data such as, for example, from an infrared camera, a camera configured to detect an emotional state based on facial expressions, a wearable device that monitors heart rate and body temperature, and a sound element may be played in response to the user stress level exceeding a threshold user stress level.

[0067] Figures 3 to 5 An embodiment of a manual mode showing natural landscape features is presented, in which a sound element corresponding to a natural environment (e.g., secondary media) is provided in response to user input. For example, when operating in manual mode, activity controls may be provided to the user to control the sound element. For example, a sound element may be provided in response to user input obtained via the activity controls. Now turning to Figure 3 , an example of activity control 300 is shown. Activity control 300 may be displayed on touch screen 319 or mobile device 323. Alternatively, activity control 300 may be implemented as a three-dimensional device that is Figure 1 part of audio system 102. In this example, activity control 300 takes the form of a slider control that includes a slider bar 302 and a slider rail 322. However, in other examples, without departing from the scope or intent of this specification, activity control 300 may be implemented in the form of a knob or other known user input device. The position or state of activity control 300 may refer to the position or state of slider bar 302.

[0068] The human user 303 can longitudinally move the slider 302 along the length of the slider guide rail 322, as indicated by the arrow 304. The slider 302 can be moved to the left range 320 or the right range 324 to adjust the sound activity that can be associated with a particular scene that can be shown via a display. For example, a user may wish to become comfortable by listening to pre-recorded sounds of rain, waves crashing on the shore, or a stream flowing downstream, etc. The user can select which sound to play through the vehicle speakers and a visual representation of the sound can be displayed on the display panel. The attributes of the sound selected to be played through the vehicle speakers can be adjusted based on the position of the slider 302 or the position of the operating state of the activity control.

[0069] The slider 302 can be in a reference position when moved to the left range 320. The slider 302 can be fully advanced when moved to the right to the right range 324. In one example, when the slider 302 is located at the left range 320, the lowest level output of the controlled sound element or feature can be output. When the slider 302 is located at the right range 324, the maximum or highest level output of the controlled sound element or feature can be output.

[0070] Now referring to Figure 4 , an activity control 300 including a plurality of control regions is shown. In this example, the activity control 300 includes three control regions; however, in other examples, the actual total number of control regions may be greater than three or less than three. Additionally, in this example, each control region approximately occupies one-third of the length of the slider guide rail 322, but the control regions can be adjusted according to other dimensions if needed. The control range or authority range of the activity control 322 is as indicated by the arrow 325 and it spans three control regions.

[0071] The first control region 390 of the slider 302 begins at the left range 320 of the slider guide rail 322 and ends at the vertical line 330. The lead line 310 shows the range of the first control region 390. The slider 302 is shown in the first control region 390, so Figure 1 the audio system 102 shown in can play sound elements stored digitally in a non-volatile memory, which are included in a first set of sound elements associated with a particular theme, scene, or environment that the user has selected. The theme may be the conveyed mood or mental state (e.g., comfort, high emotional energy, happiness, etc.). The sound elements can be played or broadcast through the vehicle speakers. The first set of sound elements can be characterized as, for example, steady-state sound elements. Steady-state sound elements can include, but are not limited to, the sound of rain, the sound of waves crashing on the shore, the gentle chirping of crickets, the sound of a gentle breeze, and sound elements that aim to capture the essence of a pattern or selected theme, scene, or environment, etc. The steady-state sound elements may be perceived by the user as continuous.

[0072] The position of the slider 302 within the first control region 390 can define the volume or sound power output level of the speaker and the playback frequency or density of the steady-state sound elements included in the selected scene or environment. For example, if the slider 302 is located in the left range of the slider guide 322, the steady-state sound elements in the selected theme, scene, or environment can be played at a low repetition frequency (e.g., the rain sound sequence retrieved from the memory can be repeated at a rate of 0.03 Hz) and a very low volume or sound power output level. If the slider 302 is moved to the right and stops before the slider 302 enters the second control region 391, the same steady-state sound elements can be played at a higher frequency (e.g., 0.1 Hz) and a low volume or sound power level. Thus, as the slider 302 moves from left to right within the first control region, the sound power amount and playback frequency of the steady-state sound elements increase.

[0073] The second control region 391 of the slider 302 begins at the vertical line 330 and it ends at the vertical line 332. The lead 312 shows the extent of the second control region 391. In one example, when the slider 302 enters the second control region 391, the volume or sound power output of the steady-state sound elements is maintained at its most recent level, and the playback frequency of the steady-state sound elements is maintained at its most recent level. Figure 1 The audio system 102 shown in can start playing the sound elements stored in the non-volatile memory, which include a second set of sound elements associated with the selected specific theme, scene, or environment. The sound elements can be played or broadcast through the vehicle speakers. The second set of sound elements can be characterized as dynamic sound elements. The dynamic sound elements can include, but are not limited to, bird calls, distant lightning, owl calls, and similar sound elements that mimic the wild animals and fauna included in the selected theme, scene, or environment. The dynamic sound elements may have discontinuous perceptual characteristics.

[0074] The position of the slider 302 within the second control region 391 can define the volume or sound power output level of the speaker and the playback frequency or density of the dynamic sound elements included in the selected theme, scene, or environment. For example, if the slider 302 is located directly to the right of the vertical line 330, when the slider 302 reaches the position of the vertical line 330, the steady-state sound elements in the selected theme, scene, or environment can be played at its repetition frequency and volume or sound power output. When the slider 302 is located directly to the right of the vertical line 330, the dynamic sound elements in the selected theme, scene, or environment can be played at a low repetition frequency and a low volume or sound power output.

[0075] If the slider 302 is moved to the right and stops just before the slider 302 reaches the position of the vertical line 332, then when the slider 302 reaches the position of the vertical line 330, the steady-state sound elements in the selected theme, scene, or environment can continue to play at their repetition frequency and volume or sound power output. Compared to when the slider 302 is located directly to the left of the vertical line 330, the dynamic sound elements in the selected theme, scene, or environment can be played at a higher repetition frequency and a higher volume or sound power output.

[0076] The third control region 392 of the slider 302 begins at the vertical line 332 and it ends at the right range 324 of the slider guide 322. The lead 314 shows the extent of the third control region 392. In one example, when the slider 302 enters the third control region 392, the volume or sound power output of the steady-state sound elements and the dynamic sound elements can be maintained at their most recent levels, and the playback frequencies of the steady-state and dynamic sound elements can be maintained at their most recent levels. Figure 1 The audio system 102 shown in can start playing the sound elements stored in the non-volatile memory, and these sound elements include a third set of sound elements associated with a selected specific theme, scene, or environment. The sound elements can be played or broadcast through the vehicle speakers. The third set of sound elements can be characterized as surreal sound elements. The surreal sound elements can include, but are not limited to, coyote howls, elk bugles, lightning, and other non-natural sound elements designed to enhance the user's emotional response to the selected theme, scene, or environment.

[0077] The position of the slider 302 within the third control region 392 can define the volume or sound power output level of the speaker and the playback frequency or density of the surreal sound elements included in the selected theme, scene, or environment. For example, if the slider 302 is located directly to the right of the line 332, then when the slider 302 reaches the position of the line 332, the steady-state and dynamic sound elements in the selected theme, scene, or environment can be played at their repetition frequency and volume or sound power output. When the slider 302 is located directly to the right of the line 332, the surreal sound elements in the selected theme, scene, or environment can be played at a low repetition frequency and a low volume or sound power output.

[0078] If the slider 302 is moved to the right and stops just before the slider 302 reaches the right range 324 of the slider guide 322, then when the slider 302 reaches the position of the line 332, the steady-state and dynamic sound elements in the selected theme, scene, or environment can continue to play at their repetition frequency and volume or sound power output. Compared to when the slider 302 is located directly to the right of the line 332, the surreal sound elements in the selected theme, scene, or environment can be played at a higher repetition frequency and a higher volume or sound power output.

[0079] Thus, a single active control can be a basis for increasing the complexity of sound elements generated via Figure 1 the audio system 102 shown in FIG. Further, the volume and intensity of the sound generated and broadcast through the speakers can be adjusted via the same active control by defining the control area of the active control.

[0080] Now referring to Figure 5 , an example is shown in which Figure 1 the system includes two active controls. Figure 5 The active control 300 described herein is the same as the active controls shown in Figure 3 and Figure 4 . Further, the active control 300 includes the same control areas 390 to 392 mentioned previously. The active control 300 also includes the same slider 302 and slider guide 322. The active control 300 can operate and provide the functions described previously.

[0081] The second active control 350 includes a slider 352 and a slider guide 360. The slider 352 can be longitudinally moved left and right along the slider guide 360 by the user 303 and move between a left range 362 and a right range 364, as indicated by the arrow 354. In this example, the second active control 350 has a control range (or authority range) 350 that is subdivided into two control areas 370 and 372, but the authority range 375 can be subdivided into additional control areas if needed. In one example, the surround sound control parameters can be adjusted based on the position of the slider 352 and the control area in which the slider 352 is located. For example, when the slider 352 is located in the first control area 370, as the slider 352 moves from the range 362 towards the vertical line 399, the center spread may increase. Increasing the center spread may change the sound distribution from the center speaker to the left front speaker and the right front speaker. When the slider 352 reaches the position of the vertical line 399, the center spread can reach the maximum level. If the slider moves into the second control area 372, the level of the upmix channel can be adjusted. For example, as the slider 352 moves from the vertical line 399 to the right range 364, the level of the upmix channel may increase. The second active control 350 can also adjust other surround sound control parameters, such as room size simulation, delay time, and dynamic compression. Further, the second active control 350 can adjust the sound control parameters for vehicle-specific sound control parameters. For example, the second active control 350 can adjust the delivery of sound elements to the speakers to enhance the sound reception of a specific passenger (e.g., the front driver, the front passenger, etc.).

[0082] In an example, the second active control 350 can adjust the position of a specific sound according to different regions of the vehicle. For example, the second active control 350 can adjust the sound distribution and / or position (e.g., left front passenger, right front passenger, etc.) in each vehicle zone in different ways for different sound group regions (such as steady-state sound elements, dynamic sound elements, and / or surreal sound elements). Thus, the user input control can provide adjustments to the vehicle zones to provide different controls for each of the different group regions in each of the vehicle zones. In another example, the user input control can provide movement from one region to another within only one of the group regions, such as steady-state sound elements.

[0083] Accordingly, active controls can be assigned to adjust more than one sound control parameter. Additionally, two or more active controls can be provided to further enhance system flexibility and the user experience. In this way, one or more functions can be assigned to a single active control to reduce the amount of user input, thereby reducing the system complexity perceived by the user.

[0084] In addition, Figure 6 shows the relationship between the audio attributes of the sound elements and the position of other user interfaces of the active control when operating in the manual mode with natural landscape features. The graph shows Figure 1 the way the audio system 102 adjusts the sound elements as shown in

[0085] Starting from the Figure 6 top, the first graph is a graph of the sound output power amplitude or volume against the active control region (e.g., 390 to 392). The vertical axis represents the sound output power amplitude of the sound elements included in the first group of sound elements associated with the first control region of the active control (e.g., steady-state sound elements). The sound output power amplitude increases along the direction of the vertical axis arrow. The horizontal axis represents the active control region, and it is divided into three regions, as Figure 4 shown. The trace 402 represents the sound output power amplitude or volume of the sound elements included in the first group of sound elements.

[0086] Starting from the Figure 6 top, the second graph is a graph of the playback sound frequency of the sound elements included in the first group of sound elements (e.g., the frequency at which the sound repeats when broadcast via the speaker) against the active control region. The vertical axis represents the playback frequency of the sound elements included in the first group of sound elements associated with the first control region of the active control. The horizontal axis represents the active control region, and it is divided into three regions, as Figure 6 shown. The trace 404 represents the playback sound frequency of the sound elements included in the first group of sound elements.

[0087] Starting from the Figure 6The third graph starting from the top of Figure 6 shows the graph of the sound output power amplitude or volume against the active control area. The vertical axis represents the sound output power amplitude of the sound elements included in the second set of sound elements (e.g., dynamic sound elements) associated with the second control area of the active control. The sound output power amplitude increases along the direction of the vertical axis arrow. The horizontal axis represents the active control area, and it is divided into three areas, as

[0088] shown. Trace 406 represents the sound output power amplitude or volume of the sound elements included in the second set of sound elements. Figure 6 The fourth graph starting from the top of Figure 6 shows the graph of the playback sound frequency of the sound elements included in the second set of sound elements against the active control area. The vertical axis represents the playback frequency of the sound elements included in the second set of sound elements associated with the second control area of the active control. The horizontal axis represents the active control area, and it is divided into three areas, as

[0089] shown. Trace 408 represents the playback sound frequency of the sound elements included in the second set of sound elements. Figure 6 The fifth graph starting from the top of Figure 6 shows the graph of the sound output power amplitude or volume against the active control area. The vertical axis represents the sound output power amplitude of the sound elements included in the third set of sound elements (e.g., surreal sound elements) associated with the third control area of the active control. The sound output power amplitude increases along the direction of the vertical axis arrow. The horizontal axis represents the active control area, and it is divided into three areas, as

[0090] shown. Trace 410 represents the sound output power amplitude or volume of the sound elements included in the third set of sound elements. Figure 6 The sixth graph starting from the top of Figure 4 shows the graph of the playback sound frequency of the sound elements included in the third set of sound elements against the active control area. The vertical axis represents the playback frequency of the sound elements included in the third set of sound elements associated with the third control area of the active control. The horizontal axis represents the active control area, and it is divided into three areas, as

[0091] shown. Trace 412 represents the playback sound frequency of the sound elements included in the third set of sound elements. Figure 6 The seventh graph starting from the top of Figure 6 shows the graph of the active control state or position against the active control area. The vertical axis represents the state or position of the active control (e.g., the slider 302), and the active control moves from left to right along the direction of the vertical axis arrow. The horizontal axis represents the active control area, and it is divided into three areas, as

[0092] At the far left of the graph, the active control is located at the first range of the active control (e.g., Figure 3 and Figure 4 320). The active control is located in the first control area, so the sound output power amplitude or volume and the playback sound frequency of the sound elements included in the second and third groups of sound elements are zero. The sound output power amplitude of the sound elements included in the first group of sound elements increases as the position of the active control moves from the left side to the right side of the graph. Similarly, the playback or repetition frequency of the sound elements included in the first group of sound elements increases as the active control moves from the left side to the right side of the graph. When the active control reaches the position of the vertical line L1, the sound output power amplitude of the sound elements included in the first group of sound elements stops increasing. Similarly, when the active control reaches the position of the vertical line L1, the playback or repetition frequency of the sound elements included in the first group of sound elements stops increasing.

[0093] Continuing to move from left to right in the graph, the sound output power amplitude and the playback frequency of the sound elements included in the first group remain unchanged. The sound output power amplitude of the sound elements included in the second group of sound elements increases as the position of the active control moves from the position of the vertical line L1 to the position of the vertical line L2. Similarly, the playback or repetition frequency of the sound elements included in the second group of sound elements increases as the active control moves from the position of the vertical line L1 to the position of the vertical line L2. The sound output power amplitude and the playback frequency of the sound elements included in the third group remain zero. When the active control reaches the position of the vertical line L2, the sound output power amplitude of the sound elements included in the second group of sound elements stops increasing. Similarly, when the active control reaches the position of the vertical line L2, the playback or repetition frequency of the sound elements included in the second group of sound elements stops increasing.

[0094] After the sound active control reaches the position of the line L2, the sound output power amplitude and the playback frequency of the sound elements included in the first and second groups remain unchanged. The sound output power amplitude of the sound elements included in the third group of sound elements increases as the position of the active control moves from the position of the vertical line L2 to the active control range (e.g., Figure 3 324) or the end of the travel. Similarly, the playback or repetition frequency of the sound elements included in the third group of sound elements increases as the active control moves from the position of the vertical line L2 to the active control range (e.g., Figure 3 324) or the end of the travel.

[0095] In this way, natural sounds can be provided to the user via natural landscape features in either manual mode or automatic mode. As described above, the automatic mode can automatically monitor vehicle conditions such as vehicle position, environmental changes, and the user's emotional state, and can provide sound elements to the user. Based on the state of vehicle 100, audio system 102 can provide sound elements. Some sound elements can be based on actual sounds detected outside the vehicle (e.g., real sound elements), while other sound elements may not be detectable outside the vehicle (e.g., synthetic sound elements). Additionally, in manual mode, the user can select a desired natural scene and can adjust the mix of the corresponding sound elements according to user preferences. By providing natural sounds to vehicle users, a more immersive sound experience can be created, thereby reducing user stress and increasing customer satisfaction.

[0096] For the listening environment within vehicle 100, the aspects disclosed herein provide that audio system 102 (e.g., via vehicle controller 180) can also change audio parameters so that the audio or user experience within vehicle 100 sounds like a different venue (e.g., a stadium, a concert hall (i.e., a large, small, or medium-sized concert hall), a festival event, or a recording studio). For example, the vehicle audio device can change audio parameters based on user selection so that the vehicle sounds like it is in Carnegie Hall, Yankee Stadium, etc. Additionally, the vehicle audio device can be arranged to simply make the listening environment within the vehicle sound larger than it actually is (e.g., if the vehicle is a compact vehicle, the vehicle audio device can provide an audio experience that sounds like the user is in a sport utility vehicle (“SUV”)). These aspects and others will be discussed in more detail.

[0097] Figure 7 An overview of vehicle 100 and its corresponding listening environment 512 (or listening chamber) is generally depicted. In connection with the above Figure 1 similar to vehicle 100 described Figure 7 vehicle 100 includes an audio system 102 that is arranged to play audio within the listening environment 512 (or listening chamber) of vehicle 100. The listening environment 512 of vehicle 100 generally defines a front wall 516 (e.g., the area of vehicle 100 including the dashboard, the front windshield, and the floorboard including the interior and plastic trim (if applicable) located below the dashboard), a rear wall 518 (e.g., the area of vehicle 100 including the passenger row seats, the rear cargo area of an SUV, and / or the rear windshield), side walls 520 (e.g., the area of vehicle 100 including the A-pillars, B-pillars, C-pillars, and / or D-pillars, the front door and / or rear door, and the windows), and a top wall 522 (or ceiling) (e.g., the area of vehicle 100 including the top metal panel and the headliner).

[0098] The audio system 102 may also include a plurality of microphones 524a to 524d located around the interior of the vehicle 100. For example, each of the microphones 524a to 524d may be equidistantly located within the headliner (not shown) of the top wall 522 of the vehicle 100 to capture sounds or captured audio signals within the listening environment 512 of the vehicle 100 (e.g., music; noise captured from vehicle occupants, which corresponds to entertainment data regarding incoming audio signals, the entertainment data including entertainment data from an electronic audio source (e.g., a mobile device); speech (or conversations from vehicle occupants); ambient noise from outside the vehicle 100 entering the interior of the vehicle 100 and / or ambient noise within the passenger compartment, etc.). It should be appreciated that any number of microphones 524 may be located within the listening environment 512 of the vehicle 10. However, if only one or two microphones 524 are located within the vehicle 100, the audible sounds close to the microphones 524 may have a greater reverberation effect on the microphones 524, resulting in relatively unnatural sounds.

[0099] As described above, the audio system 102 includes an audio controller 175 that is arranged to employ one or more noise cancellation techniques (e.g., active noise cancellation (ANC), room noise cancellation (RNC), and engine order cancellation (EOC)) to minimize the impact of road and engine noise (or any unwanted audible noise in the environment that is inconsistent with the incoming audio data being played in the vehicle 100) picked up by the microphones 524a to 524d. Similarly, in Figure 1 the embodiment shown in, the vehicle 100 includes speakers 104 located therein to play the audio processed by the audio controller 175. The location of the speakers 104 may be similar and / or different from the locations described in connection with Figure 1 For example, one or more speakers 104 may be located in one or more of the walls 516, 518, 520, and 522 to transmit audio into the passenger compartment. For example, the audio controller 175 may transmit signals to the speakers 104 located around or within the walls (e.g., the front wall 516, the rear wall 518, the side walls 520, and the top wall 522) of the vehicle 100 to provide the desired reverberation (or echo) based on the selected venue. The speakers 104 located within the headliner (or top wall 522) of the vehicle 100 may replicate the reverberation of the ceiling of the selected venue, the speakers 104 located within the headrests of the rear seats (or rear wall 518) may replicate the reverberation of the rear wall of the selected venue, the speakers 104 located within the doors of the vehicle 100 (or side walls 520) may replicate the reverberation of the side walls of the selected venue. The speakers 104 located on or within the dashboard (or front wall 516) may replicate the reverberation of the front wall of the selected venue.

[0100] Generally, a user (e.g., a driver or passenger in vehicle 100) may select a corresponding venue (e.g., a stadium, a concert hall (a large concert hall, a small concert hall, or a medium-sized concert hall), a recording studio) so that audio controller 175 plays audio parameters to conform to the audio in listening environment 512. It should be recognized that the venue may refer to a specific venue, such as Yankee Stadium, Carnegie Hall, etc. User interface 530 may be electrically coupled to audio controller 175 to enable the user to select the corresponding venue so that audio controller 175 plays audio. Audio controller 175 is configured to adjust the amount of reverberation added to the incoming audio signal from audio source 560 (i.e., FM channel, AM channel, high-definition (HD) radio, satellite radio, mobile phone, tablet computer, etc.) and / or the audio signal captured before audio playback to provide the user with the feeling of actually listening to the audio when located in the selected venue environment (e.g., stadium, concert hall, recording studio, etc.), rather than hearing the audio based on the actual walls 516, 518, 520, and 522 of vehicle 100 (or listening environment 512). For example, audio controller 175 may simulate any one or more of the various walls (e.g., front wall, side walls, rear wall, and ceiling) of the corresponding virtual venue by adjusting the reverberation effect of the incoming audio and / or the desired ambient sound before playback, such that the user perceives to hear the audio in the selected venue.

[0101] The audio controller 175 can add a reverb effect directly to the audio, regardless of any sound captured by the microphones 524a to 524d, to implement or generate a selected venue. On the other hand, the audio controller 175 can capture any audio data of the vehicle occupants in the vehicle, such that the audio controller 175 plays the audio in the vehicle 100 based on the selected virtual venue. Additionally, the audio controller 175 can add a reverb effect to the incoming audio signal from an electronic audio source (not shown) as well as the audio signal captured by the microphones 524a to 524d. These aspects can create a more realistic effect of feeling actually being at one of the corresponding venues. For example, assuming the user selects a large concert hall venue via the user interface 530, the audio controller 175 can play the audio at a much louder volume than normal, because the volume in a large concert hall may be much louder than the volume typically played in the vehicle 100. Additionally, the user may be able to select a venue corresponding to the listening environment 512 of the larger vehicle 100. The audio controller 175 can adjust the reverb effect accordingly and play the audio at the position where the audio is perceived to be played in the larger vehicle 100. Generally, the reverb aspects related to each venue can be recorded or measured at the venue and then stored in the memory (not shown) of the audio controller 175 for the user to select. It should also be recognized that in one embodiment, the vehicle 100 can also allow the user to be able to (via the user interface 530) select a specific position within the selected venue, such that the user can not only experience listening to the audio from the selected venue, but also experience listening to the audio from a specific position within the venue. The positions within the venue can correspond to the main floor, upper level (i.e., rafters), lower level, intermediate levels (i.e., the various levels between the upper and lower levels), on stage, right or left of the stage, etc.

[0102] It should be further recognized that when (i) any window in the vehicle 100 is open, (ii) the sunroof in the vehicle 100 is open, or (iii) the convertible roof in the vehicle 10 is down, any incoming audio captured by the microphones 524a to 524d may be muted by the audio controller 175. For example, various electronic controllers (not shown) in the vehicle 10 can monitor the vehicle state, such as the window state (e.g., open or closed of any window), sunroof state (e.g., open or closed), and convertible roof state (e.g., up or down), and transmit the information on the data communication bus to the audio controller 175 via the controller area network (CAN) or other multiplexed data bus. The mixer 576 can mute the audio captured at the microphones 524 and apply the reverb effect to the incoming audio signal to generate the audio for playback in the venue selected by the user via the user interface 530. For example, if it is detected that the window is rolled down or the convertible roof is detected to be down, the mixer 576 can mute the captured audio because the captured audio may be too loud or too noisy.

[0103] Figure 8 Generally depicts a method 600 for generating a virtual venue within the listening environment 512 of a vehicle 100 according to one embodiment.

[0104] In operation 602, the audio controller 175 receives captured audio from any one or more of the microphones 524 located within the vehicle 100.

[0105] In operation 604, the audio controller 175 processes incoming audio from the audio source 560 to recreate the acoustic characteristics of the desired or selected venue. For example, the audio controller 175 adds a corresponding reverb effect to generate the desired virtual venue selected by the user.

[0106] In operation 607, the audio controller 175 plays the processed audio through the respective speakers 104 with the corresponding reverb effect so that the user can listen to the audio based on the acoustic effects of the selected venue.

[0107] Figure 9 Depicts a detailed implementation of the audio controller 175 for generating a virtual venue within the listening environment 512 of a vehicle 100 according to one embodiment. It should be appreciated that the audio controller 175 may include any number of processors and memories that cooperate with each other to perform any of the operations described herein. The audio source 560 is configured to provide an incoming audio signal to the vehicle 100 (or the audio controller 175). It should be appreciated that the audio source 560 can be any one of an FM radio station, an AM radio station, a high-definition (HD) audio radio station, a satellite radio provider, an input from a mobile phone, an input from a tablet computer, etc. The user can select the corresponding audio source 560 via the source selector 562 located on the audio controller 175 or via a user interface 530 that may be located at other locations on the vehicle 100 or on a mobile device (not shown).

[0108] The reverb extraction block 564 (or extraction block 564) removes reverb from the incoming audio signal to provide a dry audio signal. This operation is performed to prepare the incoming audio signal to receive the corresponding reverb effect of the selected venue. It should be appreciated that the reverb extraction block 564 may not be able to completely remove the reverb from the incoming audio signal, and there may still be some reverb remnants on the dry audio signal. The stereo equalization block 566 receives the dry audio signal from the reverb extraction block 564. Generally, the stereo equalization block 566 functions as a conventional stereo equalizer in the vehicle 100 and is configured to equalize the incoming audio signal for user playback. The adder 568 receives the output from the stereo equalization block 66. The relevance of the adder 568 will be discussed in more detail below.

[0109] The audio controller 175 is configured to receive inputs from each corresponding microphone 524 in the vehicle 100. As described above, the audio captured by the microphone 524 can correspond to music, voice, and ambient noise within the vehicle 100. The microphone equalization block 570 receives the captured audio from the microphone 524 and equalizes the captured audio (i.e., enhances or attenuates the energy of individual frequency bands). The feedback equalization block 572 receives the output from the microphone equalization block 570. The audio controller 175 includes a delay block 574, an audio mixer 576, and a spider reverb block 578. The delay block 574 receives the dry audio from the reverb extraction block 564 to time-align the dry audio with the audio captured from the microphone 524. This condition accounts for the delay in the audio controller 175 processing the incoming audio signal. It is desired to ensure that the playback of entertainment data regarding the incoming audio signal is time-aligned with the audio signal captured from the microphone 524. Consider an example where a vehicle occupant claps or sings along with the entertainment data of the incoming audio signal. In this case, it is desired to time-align the playback of the entertainment data regarding the incoming audio signal with the clapping or voice input of the vehicle occupant (captured by the microphone 24) for playback. By capturing the playback of the entertainment data of the incoming audio signal and the clapping or voice input (or other actions performed by the vehicle occupant consistent with the entertainment data) through the microphone 524, this aspect further provides the vehicle occupant with the experience of being in the desired venue, just as one would expect to hear noise consistent with the audio playback of a venue that includes an audience to some extent. Thus, by capturing the ambient noise in the vehicle 100 with the microphone 524 and combining this data with the entertainment data of the incoming audio signal and then adjusting the reverb of the mix, this aspect enhances the vehicle occupant's experience and provides the feeling that the vehicle occupant is in the desired venue.

[0110] It should be recognized that the delay block 574 may or may not apply a delay, and this condition is based on the processing speed of the audio controller 175. The mixer 576 is configured to mix the reverb in the audio captured by the microphone 524 with any remaining reverb remnants in the incoming audio signal. The mixer 576 receives the signal WINDOW / CONVERTIBLE_STATUS, which indicates whether the windows, convertible top, or sunroof is open or closed. Similarly, if the windows, convertible top, or sunroof is open and there is too much noise in the signal, the mixer 576 may mute the signal captured from the microphone 524. Similarly, the mixer 576 controls how much noise or voice data (i.e., audio data captured from multiple microphones 24) in the vehicle 100 is fed back to the spider reverb block 578 and how much audio is input to the spider reverb block 578. Generally, the mixer 576 determines the blend of the audio captured at the microphone 524 with the direct audio (or dry audio) to achieve the desired blend.

[0111] The user interface 530 provides a control signal to the audio controller 175 that indicates a selected location (or virtual location) for the audio controller 175 to play audio. As described above, the selected location can correspond to any one of a stadium, a concert hall (e.g., a large concert hall, a small concert hall, or a medium-sized concert hall), a recording studio, and the listening environment of the vehicle 100, which is different from the listening environment 512 of the vehicle 100 where the user is located. The spider reverb block 578 receives an output from the mixer 576 that corresponds to the mixed dry audio and the captured audio. The spider reverb block 578 generally includes a plurality of spider reverb blocks 580a to 580n (or "580") and a plurality of location equalization blocks 582a to 582n (or "582"). Generally speaking, each spider reverb block 580 and its corresponding location equalization block 582 add or adjust the amount of reverb on the output of the mixer 576 in order to provide the user with the selected or desired location. Specifically, the spider reverb block 580 replicates the different reverb characteristics of different walls for the selected location. The spider reverb block 578 adjusts the reverb to correspond to the designated or selected location, and the location equalization block 582 controls the brightness characteristics of the walls 516, 518, 520, and 522 of the vehicle 100 in order to provide the desired brightness characteristics for the selected location. The selected location can correspond to a stadium location, a large concert hall, a medium-sized concert hall, etc. For example, if the user selects the audio controller 175 to play audio as if the user were located in Carnegie Hall, the spider reverb block 580 is configured to provide the reverb effect of the walls 516, 518, 520, and 522 of the vehicle 100 to make it sound like the walls of Carnegie Hall. Thus, this makes the user feel that he / she is actually sitting in the vehicle 10 listening to the audio of Carnegie Hall. The audio controller 175 includes a memory (not shown) that can store any number of desired locations and also take into account the various front walls, side walls, rear walls, and top walls of the selected location and the way audio reflects or reverberates from such surfaces of the walls. For example, the memory can store various preset frequency values corresponding to the characteristics of the walls of a specific location, and the location equalization block 582 can enhance or reduce the frequency levels of the audio output of the mixer 576 and the spider reverb block 580 to further enhance the feeling that the user is actually located in the corresponding or selected location.

[0112] For example, consider a selected venue that typically features a low ceiling made of metal and a far wall covered with carpet. The ceiling may have very loud and fast reflection characteristics compared to other walls where the sound would be very dull and have a slower reflection time. The spider reverb block 578 adjusts the reverb of the incoming audio signal and the captured audio signal to provide the desired venue, and the corresponding venue equalization block 582 controls the equalization of the incoming audio signal and the captured audio signal to simulate playback in the desired venue and to simulate the brightness characteristics of the walls of the desired venue. Generally, the speakers 104 in the vehicle 100 collectively provide an output corresponding to the desired venue, and the corresponding speakers in a given wall can each receive discrete inputs to simulate the desired brightness characteristics of that given wall of the desired venue. For example, the speakers 104 in the ceiling of the vehicle 100 can receive an equalized output to provide the appearance that the sound bouncing off the ceiling has a fast reflection time, consistent with the low ceiling of the selected venue as described above. Similarly, the equalization can be adjusted in different ways for each audio output provided to the corresponding speakers 104 in the specific walls 516, 518, 520, and 522 to be consistent with the respective walls in the selected venue.

[0113] The speaker equalization block 584 receives the output from the spider reverb block 578 to provide a more uniform audio response in the vehicle 100. The speaker equalization block 584 compensates for problems with the speakers 104 in the vehicle 100. If the user selects to listen to the incoming audio in the normal mode, the mute block 586 is provided to simply remove the amount of reverb added by the spider reverb block 578. The user interface 530 can transmit a signal indicating the request to the audio controller 175 to disable the reverb effect added to obtain the selected venue. In response to this request, the audio controller 175 can activate the mute block 86 to simply disable the playback of the audio in the selected venue. The adder 568 receives the output from the spider reverb block 578 (or from the mute block 586) and also receives the output from the stereo equalization block 566 and adds these two audio inputs together to provide a digital version of the input. The digital-to-analog converter (DAC) 588 receives the digital output from the adder 568 and converts this digital output to an analog output. This analog output corresponds to the audio played for the user in the selected venue. The various power stages (or power amplifiers) 590 enhance the analog output for playback on the speakers 104.

[0114] In view of the foregoing disclosure, a user (or vehicle occupant) in vehicle 100 can be enabled to select a corresponding natural landscape audio format (e.g., audio typically heard in a forest, stream, canyon, etc.), such that the audio system 102 plays sounds from that natural environment and a location corresponding to the natural landscape environment (such as, for example, a forest, stream, canyon, beach, etc.). Thus, in this regard, the audio (e.g., music and / or voice played in vehicle 100) generally sounds as if it were actually in the natural environment. For example, a user can select, via the user interface 530, to listen to sounds from a beach, where the audio controller 175 plays sounds typically heard on a beach (e.g., the sound of waves crashing on the shore, seagull calls, other ambient noise typically heard on a beach, etc.), and adjusts the location via the virtual location feature such that the corresponding audio being played sounds like the sound of waves crashing on the shore, seagull calls in the beach environment, and other ambient noise typically heard on a beach. These and other aspects will be discussed in more detail below. Figure 10 An example of a user interface 530 for controlling aspects related to natural landscapes and virtual location features according to one embodiment is shown. In one example, the user interface 530 can provide a slider control 602 for controlling the natural landscape audio and corresponding virtual location provided by the audio system 102. For example, the slider control 602 generally includes a gain selection 604 and corresponding mix levels 606a through 606e (or “606”). The gain selection 604 generally enables a user to adjust the overall gain of the selected mix levels 606a through 606e. Each mix level 606 generally defines a setting for a corresponding natural landscape. For example, the various mix levels 606 correspond to mixes / scenes composed of different natural landscape audio tracks with reference to a “mood reference” to ensure alignment. In other words, the selected mix levels 606a through 606e can provide dynamic and loud sounds, e.g., that may include bird calls, or higher pitch and higher beats per minute (bpm), and provide warm or comforting sounds, e.g., that may be associated with a fireplace or other similar calming things or aspects (e.g., lower bpm).

[0115] For example, mix level 606a generally corresponds to an ambient scene, mix level 606b generally corresponds to a dynamic mix, mix level 606c generally corresponds to a loud mix, mix level 606d generally corresponds to a warm mix, and 606e generally corresponds to a comforting mix. The slider control 604 is generally similar to the slider control 602; however, the slider control 604 only shows the different levels that can be selected for each mix level. Figure 10Generally, it is shown that one or more of the various mixing levels 606a to 606e can be activated at any time. For example, both mixing levels 606c (e.g., loud) and 606e (e.g., comfortable) can be activated, which may involve, for example, mixing the sound of ocean waves with the sound corresponding to seagull calls. Both mixing levels 606d (e.g., warm) and 606e (e.g., comfortable) can be activated, which may involve, for example, sounds related to rain and a fireplace.

[0116] Figure 10Also shown is a first selection control 620, and the first selection control can be used as an alternative to the slider control 602 to select the gain selection 604 and the mixing levels 606a to 606e. As shown, the first selection control 620 can be circular, with the mixing levels 606a to 606e formed as corresponding circular portions of the control 620. The first selection control 602 can also include the corresponding mixing level 606a placed at its center point. A selection device 622 (which can also be circular, for example) can be moved over the corresponding mixing levels 606a to 606e so that an occupant can select one or more of the mixing levels 606a to 606e. With the first selection control 620, the occupant can simply move the selection device 622 over one or more of the mixing levels 606a to 606e to select the corresponding mixing level. As shown, the selection device 622 is placed on the boundary formed by, for example, the mixing levels 606a and 606e to select the mixing levels 606a and 606e. In one example, the center point of the first selection control 620 can correspond to a real virtual location, and placing the first selection device 622 outward from the center point increases the sound of the corresponding mixing levels 606a to 606e. For example, the placement of the selection device 622 relative to the center point or the outer perimeter of the first selection control 620 determines the gain level applied to the corresponding mixing levels 606a to 606e. Additionally, an intensity slider 630 is provided so that an occupant can select the amount of intensity applied to the selected mixing levels 606a to 606e. For example, the intensity slider 630 includes a first end point 632a and a second end point 632b, which form the extremes or limits of the amount of intensity applied to the selected mixing level. Thus, the occupant can move the selector 634 to any position between the first end point 632a and the second end point 632b or to the first and second end points. Positioning the selector 634 closer to the first end point 632a generally indicates that the user wishes to use a lower intensity on the selected mix 606a to 606e. Positioning the selector 634 closer to the second end point 632b generally indicates that the user wishes to use a higher intensity on the selected mix 606a to 606e. Gain generally corresponds to the ratio between the audio being played in the vehicle (e.g., audio from a recording artist or other non-natural landscape audio, etc.) and the natural landscape sounds, while mixing corresponds to the ratio between the audio tracks that make up the natural landscape audio.

[0117] As generally shown at 640, this condition indicates that the occupant desires to use the minimum intensity level (or mix level 606a to 606e) of the selected natural landscape. For example, at 604, this condition corresponds to the minimum intensity level applied to the natural landscape and / or other audio played in the vehicle (e.g., a recording of an artist's audio or audio played by a broadcast (i.e., non-natural landscape audio)). For example, this condition may involve the intensity of the selected natural landscape being at a minimum, and the virtual venue applied by the audio system 102 and the audio played by the audio system 102 also being at a minimum. As generally shown at 642, this condition indicates that the occupant desires to use the maximum intensity level (or mix level 606a to 606e) of the selected natural landscape. For example, this condition may involve the intensity of the selected natural landscape being at a maximum, and the virtual venue applied by the audio system 102 and the audio played by the audio system 102 (e.g., a recording of an artist's audio or audio played by a broadcast (i.e., non-natural landscape audio)) also being at a maximum.

[0118] Figure 11 Another example of a first selection control 620 displayed on the user interface 530 according to another embodiment is shown. As Figure 11 shown, the first selection control 620 includes a different number of mix levels 606a to 606d. It should be recognized that the number of mix levels 606 for the first selection control 620 may vary based on the requirements of a particular implementation.

[0119] Figure 12 A second selection control 700 displayed on the user interface 530 according to one embodiment is shown. Generally, the user interface 530 may display the selection control 700 that typically corresponds to the virtual venue of a forest. In this case, the occupant may select the virtual venue of the forest via another selection control mechanism before seeing the second selection control 700. Additionally, the virtual venue of the forest and its associated selection control 700 show various natural landscapes associated with the forest (such as birds, flowing water, etc.) that may be played by the audio system 102. The second selection control 700 may increase / decrease the level or volume of the audio playback of the natural landscape in response to the occupant's selection of the control 700.

[0120] For example, the occupant can simply move their corresponding finger along the image of the forest (e.g., virtual venue) as shown in the second selection control 700 to control the audio playback of the secondary image (or natural landscape image / audio (e.g., birds, water flow, etc.)) shown in the forest. For example, the sphere is generally shown as 702, and a touch point 703 is also generally shown, which corresponds to the point in the second selection control 700, that is, the position of the occupant's finger on the image shown in the second selection control 700. The sphere 702 generally shows the area where the occupant can move the touch point 703 so that the occupant can select various aspects related to the natural landscape (or natural image in the forest). It should be recognized that the actual touch point 703 is not visible to the occupant for selection and subsequent movement control. In one example, the touch point 703 can be superimposed on the image shown in the second selection control 700. Although the touch point 703 is generally shown as circular, it should be recognized that the touch point 703 can be arranged in any shape or configuration.

[0121] Generally speaking, the occupant can move the touch point 703 vertically downward along the axis 704 towards the water flow to control the audio controller 175 to increase the level / volume of the water flow playback. If the occupant wants to decrease the volume or level of the water flow playback, the occupant can move the touch point 703 vertically upward along the axis 704 away from the image of the water flow, as shown in the second selection control 700. Additionally, the occupant can move their corresponding finger vertically upward along the axis 706 towards the bird to control the audio controller 175 to increase the level / volume of the bird natural landscape playback (e.g., bird chirping). If the occupant wants to decrease the volume or level of the bird chirping playback, the occupant can move the touch point 703 vertically downward along the axis 706 away from the image of the bird, as shown in the second selection control 700. Similarly, via the second selection control 700, the user interface 530 enables the user to move the touch point 703 horizontally across the axis 708 to move the touch point around the visual effect depicting the forest (or virtual venue). In this regard, the occupant can move through the forest and move closer to other items shown in the forest scene or image so that the occupant can select them as natural landscapes for the audio system 102 to play for the user. In one example, the occupant can touch the image of the forest and move the touch point 703 to the right along the axis 708 to move closer to the water flow, as shown on the left side of the second selection control 700. It should be recognized that the touch point 703 is generally visible in the image of the forest shown on the second selection control 700. For example, the touch point 703 can be superimposed on the image presented in conjunction with the second selection control 700. Except for the moment when the system enters the timeout screen with the selected natural landscape in the large image, the touch point 703 can always be visible to the user. When the user moves the touch point 703, the touch point 703 will remain in the position where the user left it. The audio will change based on the position where the touch point 703 moves to.

[0122] In another embodiment, the touch point 703 may not have any natural sound (or natural landscape). However, in such a case, the corresponding visual effect can show the location, and dragging your finger towards the area where the bird sound (or natural landscape) is located will cause a bird to appear. Alternatively, after the occupant moves their finger away from the second selection control 700, the touch point 703 can be recentered.

[0123] Figure 13 Another aspect of the second selection control 700 displayed on the user interface 530 according to one embodiment is shown. As described above, the occupant can pass the touch point 703 along axis 708 through the forest image shown in the second selection control 700. As Figure 14 The second selection control 700 shown depicts an example where the occupant touches the image of the forest and moves the touch point 703 to the left along axis 708 to move closer to the water flow (as shown on the left side of the second selection control 700). By this selection, the audio system 102 can play the audio related to the natural landscape of the water flow in the vehicle 100 at a higher level or volume in response to detecting that the occupant has moved the touch point 703 closer to the image of the water flow in the forest.

[0124] Figure 14 Another aspect of the second selection control 700 displayed on the user interface 530 according to another embodiment is shown. For example, the occupant can move the touch point 703 closer to the bird in the image shown in the second selection control 700 (e.g., compared to the image referenced in the second selection control 700 shown in Figure 13 ), to increase the level or volume at which the audio system 102 plays the audio of the bird as a natural landscape. It should be recognized that through the example shown in Figure 15 , the audio system 102 can play the audio of the water flow and the bird equally in the vehicle 100.

[0125] Figure 15 Another aspect of the second selection control 700 displayed on the user interface 530 according to another embodiment is shown. For example, the occupant can move the touch point 703 closer to the bird in the image shown in the second selection control 700 (e.g., compared to the image referenced in the second selection control 700 shown in Figure 14 ), to increase the level or volume at which the audio system 102 plays the audio of the bird as a natural landscape. It should be recognized that through the example shown in Figure 14In the example shown, the audio system 102 can play the audio of a bird louder than the sound of water flow in the vehicle 100. It should also be recognized that the size of the image of the bird is larger than that shown in the previous image on the second selection control 700, indicating that the audio system 102 has increased the gain or volume of the natural landscape associated with the bird. Therefore, the bird audio playback may be louder (or more prominent) compared to the bird audio playback in Figure 14 The bird audio playback in

[0126] Figure 16 Another aspect of the second selection control 700 displayed on the user interface 530 according to another embodiment is shown. For example, the occupant can move the touch point 703 closer to the forest while keeping it close to the bird in the image shown in the second selection control 700 (e.g., compared to the image referenced in the second selection control 700 shown in Figure 15 To increase the level or volume of the audio of the trees in the forest where the audio system 102 plays the audio of the trees as part of the natural landscape. It should be recognized that through the example shown in Figure 14 The audio system 102 can play the audio of the bird and the trees in the forest equally. In this case, since there is no water flow in the image of the second selection control 700, the audio system 102 prohibits playing any audio related to the water flow because the occupant does not want to hear the audio related to the water flow.

[0127] Figure 17 Another aspect of the second selection control 700 displayed on the user interface 530 according to another embodiment is shown. For example, the occupant can place the touch point 703 closer to the trees in the forest while keeping it on the bird and the water flow in the image shown in the second selection control 700 (e.g., compared to the image referenced in the second selection control 700 shown in Figure 16 To increase the level or volume of the audio where the audio system 102 plays the audio of the trees and also plays the audio of the bird and the water flow simultaneously. It should be recognized that through the example shown in Figure 14 The audio system 102 can play the audio of the trees, the water flow, and the bird equally in the vehicle 100. It should also be recognized that the sizes of the images of the bird, the water flow, and the trees are similar to each other compared to the previous image on the second selection control 700, indicating that the audio system 102 plays the audio of the bird, the trees, and the water flow equally based on the position where the occupant places the touch point in the sphere 702. Therefore, the bird audio playback may be louder (or more prominent) compared to the bird audio playback in Figure 14 The bird audio playback in

[0128] Figure 18Shows another aspect of the second selection control 700 displayed on the user interface 530 according to another embodiment. For example, the occupant can place the touch point 703 closer to the trees in the forest while being positioned further away from the bird and slightly closer to the water flow in the image shown in the second selection control 700 (e.g., when compared with Figure 17 the image referred to in the second selection control 700 shown in Figure 18 to continue playing the bird and the water flow at the same level as in

[0129] Figure 19 Shows another aspect of the second selection control 700 displayed on the user interface 530 according to another embodiment. For example, the occupant can place the touch point 703 closer to the water flow while remaining close to the trees in the forest and further away from the bird in the image shown in the second selection control 700 (e.g., when compared with Figure 18 the image referred to in the second selection control 700 shown in Figure 18 to continue playing the water flow and the trees at the same level as in

[0130] Figure 20 Shows another aspect of the second selection control 700 displayed on the user interface 530 according to another embodiment. For example, the occupant can place the touch point 703 closer to the water flow while remaining close to the trees in the forest and further away from the bird in the image shown in the second selection control 700 (e.g., when compared with Figure 19 the image referred to in the second selection control 700 shown in Figure 19 to continue playing the water flow and the trees at the same level as in

[0131] Figure 21 Shows another aspect of the second selection control 700 displayed on the user interface 530 according to another embodiment. For example, the occupant can place the touch point 703 closer to the water flow while remaining close to the trees in the forest and further away from the bird in the image shown in the second selection control 700 (e.g., when compared with Figure 19 the image referred to in the second selection control 700 shown in Figure 18 to continue playing the water flow and the trees at the same level as in Figure 20 shown in Figure 21 shown in, the difference being that the touch point 703 is positioned closer to the water flow. Thus, the audio system 102 can play at a level lower than that inFigure 20 a higher-level graded-play water flow played in [it]. In this regard, the user will hear more water flow in the audio output.

[0132] Figure 22 An example of a sphere 702 associated with a second selection control 700 according to an embodiment is shown. The sphere 702 generally includes a plurality of detection quadrants 750a through 750w located therein. The detection quadrants 750a through 750w (“750”) generally correspond to portions of the sphere where a touch point 703 can be positioned by an occupant to select various natural landscape items in a corresponding location. Generally, the second selection control 700 and its corresponding image are superimposed on top of the detection quadrants 750a through 750w. Thus, when the touch point 703 moves within the corresponding detection quadrants 750a through 750w, the user interface 530 transmits a signal indicating the position of the point 703 on the quadrant corresponding to a particular natural landscape to the audio system 102. The audio system 102 then plays the corresponding natural landscape (e.g., water flow, birds, trees) and corresponding volume or gain level based on the position of the touch point 703 on the detection quadrants 750.

[0133] Figure 23 An example of a virtual location 800 and corresponding nature-related features that can be presented on the user interface 530 according to an embodiment is shown, the corresponding features including natural landscapes. Figure 24 The example of the virtual location 800 shown in [it] generally corresponds to the natural location described above, which has various nature-related features that can be played via the audio system 102, such as water flow, birds, trees. Based on the description provided above, the occupant can traverse the graphical representation of the nature-related features shown in the virtual location 800 on the user interface 530. It should be appreciated that there are any number of ways to create different scenes for the virtual location 800. In one example, the virtual location 800 can also include specific elements, such as birds as an additional layer. It should be appreciated that any of the controllers disclosed herein can utilize artificial intelligence (AI) to generate one or more of the scenes.

[0134] Figure 24 An example of a global selection area 850 presented on the user interface 530 according to an embodiment is shown. As shown, the global selection area 850 is generally formed in the shape of the Earth, depicting various geographical locations on the Earth. The user can select a geographical location on the global selection area 850 such that the audio system 102 plays the audio unique to the selected location. As shown, the user can adjust the volume of the audio being played via a first volume control 852.

[0135] Figure 25Shows another example of the global selection area 870 presented on the user interface 530 according to an embodiment. The global selection area 870 is shown as being presented in a planar layout on the user interface 530, rather than on the sphere as shown in Figure 24 The global selection area 870 includes a control mechanism 900 that adjusts the levels of the natural landscape and the virtual venue. The control mechanism 900 will be discussed in more detail below. The global selection area 870 shows a library area 872 that the user can select. Once selected, in addition to the reverberation in the library, the user can also hear the natural sounds expected to be heard in the library, such as, for example, footsteps, the sound of turning pages, etc. The user can select a geographical location on the global selection area 870 such that the audio system 102 plays the audio unique to the selected location. As shown, the user can adjust the volume of the playing audio via the first volume control 852. A second volume control 854 is also located on the user interface 530 to control the overall volume control of the audio system.

[0136] Figure 26 Shows an example of an image 880 of an actual natural scene presented on the user interface 530 according to an embodiment. The user can manipulate a touch point 703 within the image 880 to select, for example, a specific natural landscape (or sound), such as wind / trees, the sound made by a train when it appears in a specific scene, and the venue for the audio system 102 to play. In one example, the image 880 shows a forest, and the venue elements can only be heard when the touch point 703 is located within the specific scene shown in the image 880. The user can slide the touch point 703 to one side of the scene (e.g., the forest where the train is located) to hear the sound made by the train, and slide it to the other side of the scene to hear the sound of wind / trees. The user interface 530 can also indicate the location corresponding to the image of the natural scene. In this example, the natural scene of the image corresponds to the Black Forest located in Germany.

[0137] Figure 27An example of a control mechanism 900 for controlling aspects related to a natural landscape (e.g., mix level 606) and a selected virtual venue 905 according to an embodiment is shown. The control mechanism 900 enables a user to control the level of the natural landscape 606 within the selected virtual venue 905 (e.g., mix level 606 (or natural landscape, natural sounds, etc.)) to different extents via an audio system 102. It should be recognized that the audio system 102 enables the user to select a virtual venue (e.g., a church, a hall / amphitheater (e.g., Carnegie Hall), etc. as described above), and also enables the user to control the volume or level of the natural sounds 606 played in the selected virtual venue 905 via the control mechanism 900. In this case, the control mechanism 900 is not used within the audio system 102 to adjust or change the playback of the main audio that the user is playing through the audio system 102 (e.g., a song or audio (non-music audio) from a mobile device or a song or other audio (non-music audio) being played by the audio system 102 in a vehicle).

[0138] The control mechanism 900 can generally be defined by a bounded region 901 (e.g., a funnel-type shape as Figure 27 shown) and includes a first axis (or horizontal axis) 902 and a second axis (or vertical axis) 904. The user can move a touch point 703 along the first axis 902 and the second axis 904 of the bounded region 901 and change the playback between various mix levels 606 of the natural landscape within the selected virtual venue 905 (e.g., loud sounds and warm sounds). Loud sounds may correspond to birds, ocean waves, etc. Warm sounds may correspond to wind, fire, etc. In Figure 27 the example shown, the touch point 703 is located at the rightmost point of the selected virtual venue 905, which indicates that the gain of the natural landscape 606 is fully turned down so that only the main audio is played in the selected virtual venue 905.

[0139] As generally shown at 909, the main music level (or main audio) 910 is not affected by the position of the touch point 703 within the bounded region 901. However, the natural sounds 912 are turned down completely, as reflected by the position of the touch point 703 within the bounded region 901 (e.g., the touch point 703 is near or adjacent to the icon representing the selected virtual venue 905). Similarly, as shown at 916, the microphones located throughout the vehicle are selected to be turned off, and thus, the voices of the occupants, along with the main audio and the natural landscape 606 within the virtual venue 905, are not captured and played by the audio system 102. It should be appreciated that the user may select the microphone playback feature, where the user selects, via the user display 530, the option to activate one or more microphones located throughout the vehicle, such that the microphones capture audio from within the passenger compartment and the audio system 102 plays the audio captured from within the passenger compartment along with the main audio and the natural sounds within the selected virtual venue. Consider an example where the user selects a church as the selected virtual venue. In this case, the user (or occupant) may clap along with the gospel music played as the main audio, and the captured clapping audio may be played within the virtual venue of the church. Additionally, the user may activate the natural sounds (e.g., rain, ocean waves, etc.) for playback, such that the natural sounds are played along with the main audio (e.g., gospel music) and the audio signal captured from within the passenger compartment (e.g., clapping) within the selected virtual venue corresponding to the church.

[0140] Figure 28 An example of a control mechanism 900 for controlling aspects related to the natural landscape (e.g., mix level 606) and the selected virtual venue 905 according to one embodiment is shown. In this example, it can be seen that the user (or occupant) has significantly moved the touch point 703 towards the bounded region 901 at the level of the mix 606 of the natural landscape. In this case, the audio system 102 can play the loud sounds (or mix 606) at a higher level within the selected virtual venue 905 than the warm sound. Similarly, with respect to the main audio, the audio system 102 mainly plays the natural landscape of the loud sounds. This aspect is generally shown at 909, where the level of the main audio 910 is shown to be lower than the level of the natural sounds corresponding to the loud sounds including ocean waves and birds. Assuming that the touch point 703 is moved along the bounded region 901 to increase the level of the natural sounds played by the audio system 102, this necessarily results in a reduced sound image of the selected virtual venue 905 within the selected venue. This can be seen at 914, as the level of the selected virtual venue 905 is low.

[0141] Figure 29An example of a control mechanism 900 for controlling aspects related to a natural landscape (e.g., mix level 606) and a selected virtual venue 905 according to one embodiment is shown. In this example, it can be seen that the user (or occupant) has moved the touch point 703 along the bounded area 901 towards the midpoint of the bounded area 901 such that the level of the natural sound (see 912), the effect of the selected virtual venue 905 (or the immersion level of the selected virtual venue (see 914)) are similar to each other. It can also be seen that the level of the main audio (see 910) is also similar to the level of the number of virtual venues applied and the level of the natural landscape.

[0142] Figure 30 A schematic diagram 950 for controlling the levels of a natural landscape (or mix levels 606a to 606d) and virtual venues using an audio system 102 according to one embodiment is shown. Specifically, the block diagram 950 shows that the audio system 102 generally includes an audio controller 175 and a user interface 530. As described in connection with Figures 27 to 29 the user interface 530 generally enables the user to select or adjust various mix levels 606a to 606d (or natural landscape) and their corresponding levels via the control mechanism 900 through user selection. The first axis input 952 corresponds to the input provided by the control mechanism 900 relative to the first axis 902, as selected and shown in Figures 27 to 29 Then, the immersion control (or virtual venue input) 914 provides an output to the mixer 960 of the audio controller 175.

[0143] The second axis input 954 corresponds to the selected level between the natural landscape selected relative to the second axis 904 and the selected virtual venue 905 (see Figures 27 to 29 ). As shown in Figures 27 to 29 the user can select the levels of the natural landscape and the selected virtual venue 905 based on the position of the touch point 703 on the second axis 904 within the bounded area 901. The user interface 530 provides an output to the audio controller 175 indicating the desired levels of the selected mix levels 606a to 606d (or natural landscape) and / or the selected virtual venue 905. The user interface 530 adjusts the level of the selected virtual venue 905. The audio controller 175 is generally configured to receive the main audio (or music) 970 from another audio source (e.g., a mobile device, a radio, a CD player, etc.) to play together with the selected mix levels (or natural landscape) 606a to 606d of the selected virtual venue 905.

[0144] Generally, the audio controller 175 includes a mixer 960 to mix the levels of the natural landscape 606, the main audio 970, and the selected virtual venue 905. Thus, in combination with the natural landscape 606, the mixer 960 adjusts the overall level (or volume) of the natural landscape 606 based on the volume information provided by the level circuits 956a to 956c. The level circuits 956a to 956c typically provide information corresponding to the selected level (or volume magnitude) requested to be applied via the control mechanism 900. Additionally, the mixer 960 may also receive a signal from the immersion control circuit 914, which typically provides information corresponding to the level amount to be applied to the selected virtual venue 905 based on the position of the touch point 703 on the first axis 902, as Figures 28 to 30 elaborated in. The mixer 906 also receives the main audio 970 to combine the main audio 970 with the selected level (or volume) of the natural landscape 606, the selected level (or volume) of the selected virtual venue 905, and the main audio. The mixer 906 combines the selected levels of the natural landscape 606 and the virtual venue 905 along with the main audio and provides a first audio output to the amplifier 962. The amplifier 962 amplifies the audio output to generate a final audio output for transmission to the speakers in the vehicle.

[0145] The user interface 530 also provides a control signal to the audio controller 175 to receive the input of any one or more of the microphones 150 located in the vehicle. In this case, the audio controller 175 may receive the audio captured from inside the passenger compartment of the vehicle and play the captured audio together with the natural landscape 606 and the main audio 970 within the selected virtual venue 905. An example of this aspect was elaborated above in connection with the passenger clapping, where such audio was captured and played together with the main audio 970 in the selected virtual venue 905 of the church. Further mentioned, one of the natural landscapes 606 may also be played together with the main audio 970 and the audio captured inside the vehicle.

[0146] Figure 31 A method 1000 for generating audio in the listening room of the vehicle 100 according to one embodiment is shown. It should be recognized that the method 1000 can be used for the user interface 530 located on the vehicle and / or the user interface 530 located on the mobile device 1102, as disclosed in connection with the seat-based natural landscape described below.

[0147] In operation 1002, the user interface 530 receives various inputs from a user (e.g., a driver or a vehicle occupant). In operation 1004, the audio controller 175 receives an incoming audio signal including primary audio data from an audio source 560 for playing audio in the vehicle 100. In operation 1006, the audio controller 175 receives a first control signal from the user interface 530 in any of the manners disclosed above. The first control signal may indicate a desired location (i.e., the selected virtual location 905).

[0148] In operation 1008, the audio controller 175 adjusts at least the reverberation of the incoming audio signal to play the primary audio data in a desired location (e.g., the selected virtual location) in the vehicle 100. In operation 1010, the audio controller 175 receives a second control signal indicating desired natural sounds (e.g., natural scenery) to be played in the listening room. The audio controller 175 receives the second control signal from the user interface 530. In operation 1012, the audio controller 175 plays the desired natural sounds together with the primary audio data in the desired location in the vehicle 100. Additionally or alternatively, the audio controller 175 may receive audio captured inside the vehicle's cabin and play the captured audio together with the natural scenery 606 and the primary audio 970 within the selected virtual location 905. The audio controller 175 may transmit an audio output signal including the primary audio data, the desired natural sounds, and / or the captured audio to one or more speakers in the vehicle. Alternatively, the audio controller 175 may wirelessly transmit an audio output signal including the primary audio data, the desired natural sounds, and / or the captured audio to one or more mobile devices 1120 located around one or more seats 109, 110, 112a, and 112b in the vehicle.

[0149] Systems and methods for providing seat-based natural scenery audio

[0150] Based on one or more of the above aspects, the audio system 102 may be extended to include seat-based components, where each seat occupant (e.g., a passenger) located in the vehicle 100 may independently adjust their own scene-related natural scenery mix. For example, any occupant may be located on the driver's seat 109, the passenger seat 110, the left rear passenger seat 112a, and the right rear passenger seat 112b and control their own natural scenery mix (see the reference to seats 110 and 112a to 112b in Figure 1 ). As Figure 1As further shown, the occupant may include a mobile device 1102 that is located around any one or more of the driver seat 109, the passenger seat 110, the left rear passenger seat 112a, and the right rear passenger seat 112b. In this regard, the mobile device 1102 may communicate bi-directionally wirelessly with the audio controller 175. The mobile device 1102 may transmit (but is not limited to): (i) a command indicating the desired (or primary) audio content to be played by the audio controller 175, (ii) a command indicating the desired natural landscape to be played together with the primary audio, and / or (iii) a command indicating the desired immersion control to focus the audio in an independent sound zone (ISZ) around each of the driver seat 109, the passenger seat 110, the left rear passenger seat 112a, and the right rear passenger seat 112b.

[0151] As Figure 1 shown, the interior cabin generally includes an acoustic environment 177 that defines the space in which the driver and passengers experience the audio played by the audio system 102. For example, the acoustic environment 177 generally defines various ISZs that cover the areas around the seats 109, 110, 112a, and 112b. With sound immersion and its control, the audio heard by the driver and / or passengers on each of the seats 109, 110, 112a, and 112b can be made to extend beyond the stereo left and right channels, thus providing all-round musical height, depth, and presence in a 3-D sound field. Thus, in this regard, the driver and / or passengers can control the focus of the audio output on a fixed point (referred to as the listening position) around each of the seats 109, 110, 112a, and 112b. Various formats of 3D-based audio formats may include but are not limited to: Dolby Sony and Auro 3D.

[0152] It should be appreciated that the audio system 102 may utilize cross cancellation and / or active noise cancellation to cancel the noise leaking from adjacent areas in the vehicle 100. It should also be appreciated that the disclosed concepts can be extended to include not only mobile devices used as audio sources, but also any audio source having a user interface 530 that is directly coupled to the vehicle 100, such as an audio head unit that may be attached to various seat backs or other areas near or within the ISZ covering a particular seat 109, 110, 112a, and 112b. Thus, in this regard, any reference to the mobile device 1102 and any operations performed herein may also apply to an audio source having a user interface 530 located within (or attached to) various aspects of the vehicle 100 (e.g., seat backs, headrests, roof liners, etc.).

[0153] In terms of returning to a reference natural landscape, the audio system 102 can generally provide a "stereo bed" of sound files that cover steady-state and general sounds (e.g., nature-related sounds). Each occupant can adjust the blend of dynamic sound elements via the user interface 530 on the mobile device 1102 or by using the user interface that controls the audio controller 175. Thus, these signals can be routed to respective near-field speakers (e.g., CTR speaker 124, FL speaker 113, FR speaker 115, and at least a pair of surround speakers such as LS speaker 117, RS speaker 119, LR speaker 129, and RR speaker 130 or a combination of speaker groups) located near each ISZ or seat 109, 110, 112a, and 112b.

[0154] Generally speaking, for example, the driver can select a natural landscape corresponding to "forest" as the desired natural scene. The audio system 102 can play the stereo bed, which typically includes steady signals such as wind noise, the sound of gurgling brooks, or similar signals. On the user interface of each mobile device 1102, each passenger can move their virtual avatar between, for example, 3 or more specific dynamic sound elements. For example, the chirping of birds, the chirping of insects, the bugling of elk, or similar dynamic sounds. The user can independently adjust the blend of these dynamic elements, for example, by adding more birds and fewer insects, more elk and fewer birds, etc. A matrix mixer is disclosed that is located downstream of the shared natural sound file player and the near-field mixing for each driver or occupant is adjusted accordingly. Additionally, each occupant can adjust their personal immersion level based on the present disclosure as set forth herein.

[0155] Figure 32FIG. 0 shows an example of a seat - based user interface 530 for implementing playback of one or more natural landscapes for one or more occupants located in a vehicle 100. The user interface 530 can be part of a mobile device 1102 or directly coupled to an audio controller 175. If the user interface 530 is part of the mobile device 102, it should be recognized that the mobile device 102 can transmit wireless commands to the audio controller 175. The user interface 530 depicts a legend based on a three - axis image that specifies: (i) a first natural landscape corresponding to the audio playback of lightning selected on a first axis 1200, (ii) a second natural landscape corresponding to the audio playback of a bird selected on a second axis 1202, and (iii) a third natural landscape corresponding to the audio playback of ocean waves selected on a third axis 1204. In one embodiment, the user interface 530 provides a touch point 1103 that can be moved along any one or more of the first axis 1200, the second axis 1202, and the third axis 1204. For example, the touch point 1103 is generally shown as a series of concentric spheres 1105 (or shapes 1105) around a central sphere 1107. The outermost sphere 1109 generally encloses the innermost sphere and the central sphere 1107. It should be recognized that the shapes can be different and the general shape of the spheres can take any number of different forms.

[0156] Generally, the first axis 1200 and the second axis 1202 typically correspond to natural landscapes based on lightning - based audio output. Thus, as the touch point 1103 moves onto or around the first axis 1200 and the second axis 1202, the audio controller 175 transmits lightning - based audio output. The second axis 1202 and the third axis 1204 typically correspond to natural landscapes of bird calls (chirping). Thus, as the touch point 1103 moves onto or around the second axis 1202 and the third axis 1204, the audio controller 175 transmits bird - chirping - based audio output. The first axis 1200 and the third axis 1204 typically correspond to natural landscapes based on ocean - wave - based audio output. Thus, as the touch point 1103 moves onto or around the first axis 1200 and the third axis 1204, the audio controller 175 transmits ocean - wave - based audio output. The user can move the touch point 1103 such that the series of concentric spheres 1105 can overlap onto one or more of the first axis 1200, the second axis 1202, and the third axis 1204, such that any one or more of waterfall - based audio, bird - chirping audio, and / or ocean - wave - based audio can be played by the audio controller 175. The greater the extent to which the concentric spheres 1105 are positioned on one or more of the first axis 1200, the second axis 1202, and the third axis 1204, the greater (or higher) the level or volume played by the audio controller 175.

[0157] AsFigure 32 The touch point 1103 shown in Figure 32 indicates that the occupant desires that most of the audio playback includes natural landscape audio corresponding to bird audio, where portions of lightning and ocean waves are also included in the overall audio output. In this regard, the volume (or audio level) of the birds may be greater than that of the lightning and ocean waves. The occupant can move the center sphere 1107 to any position around the first axis 1200, the second axis 1202, and the third axis 1204 to select the corresponding natural landscape.

[0158] In another embodiment, the way the occupant selects the corresponding natural landscape may involve the occupant selecting various points on the outermost sphere 1109 while the touch point 1103 remains substantially stationary. In this case, the user can select a corresponding point on the outermost sphere 1109 and move it deeper into the corresponding natural landscape area to increase the level or volume of the audio playback. Thus, in this regard, it can be seen that one or more points on the outermost sphere 1109 are mainly located in the natural landscape area corresponding to the bird audio.

[0159] It should be recognized that the main audio transmitted by the audio controller 175 can also be played while the natural landscape is being played. The occupant can perform the immersion control selection 1210 (see Figure 36 ) to control the amount of immersion provided by the audio system 102. The immersion control selection 1210 particularly involves selecting the outer sphere 1109 with two fingers and moving the two fingers closer to each other to reduce the total width of the touch point 1103, such that all the concentric spheres 1105 also decrease. In one example, the narrower the width of the concentric spheres 1105, the stronger the immersion and applicability of the selected virtual venue. It should be recognized that the opposite situation can be applied. This aspect causes the audio system 102 to increase the sound immersion level of the bird audio output generated for a particular occupant who wishes to increase the sound immersion level. Audio immersion generally particularly involves the audio system 102 playing the audio in such a way that the audio is sensed as coming from all directions around the occupant. Stereo-based audio generally involves a scenario where the audio is played from the center of the occupant as well as from the left and right sides of the occupant. Thus, compared to stereo-based sound implementations, audio immersion can improve the overall natural landscape experience.

[0160] In another embodiment, the sound immersion control can be performed as follows: The occupant can select the center sphere 1107 and move the sphere 1107 inward away from the outermost sphere 1109 or outward towards the outermost sphere 1109 while the touch point 1103 (and the outermost sphere 1109) remains substantially stationary. This movement of the center sphere 1107 causes a change in the visual topographical generation of the touch point 1103, thereby causing a change in the immersion control. This aspect will be discussed in more detail below.

[0161] It should be recognized that each occupant on each of the seats 109, 110, 112a, and 112b can independently select the various natural landscapes he / she desires, regardless of what other occupants may select. It should also be recognized that each occupant can continue to play the main audio being played by the audio system in the vehicle 100 while controlling the specific type of natural landscape, the level of sound immersion of the natural landscape, and the volume of the natural landscape. Conversely, if the user interface 530 is part of the mobile device 1102, the occupant can choose to utilize the natural landscape of the vehicle provided by the audio system 102 while using the main audio signal being played by the occupant's mobile device 1102. Although not typically shown, Figure 32 the image may also include a reference to the seat (e.g., 109, 110, 112a, and 112b) in which the occupant is located, so that the occupant knows that his / her mobile device 1102 has been properly paired and is communicating with the audio controller 175 from the location where the occupant is within the vehicle 100.

[0162] Figure 32 It also includes a main volume level selector 1502 that enables the occupant to select the overall volume of the entire natural landscape mix through his / her mobile device 1102. The main volume level selector 1502 can implement the volume selection of the main audio being played by the audio system 102. In other embodiments, the main volume level selector 1113 can be used to control only the volume level of any and all natural landscapes being played by the audio system 102. Additionally, the mobile device 1102 may also include occupant visual indicators 1115a to 1115d that correspond to a visual representation of the occupant located in their respective seats 109, 110, 112a, and 112b.

[0163] Generally, the visual indicator 1115a is shown highlighted and corresponds to the position of the occupant who is currently controlling the mobile device 1102 to play the desired natural landscape. In this case, it can be seen that the driver sitting in seat 109 has paired his / her mobile device 1102 with the audio system 102 and selected the current bird natural landscape for playback. The visual indicator 1115b corresponds to the passenger in the right front seat 110, and the display on the mobile device 1102 provides an indication to the driver sitting in seat 109 of what natural landscape is being played for the passenger in the right front seat 110. In this case, the right front passenger is listening to the natural landscape with a bird audio output. It can also be seen that the left rear passenger is listening to a wave-based natural landscape via indicator 115c, and the right rear passenger is listening to at least a wave-based audio.

[0164] Figure 33Shows another example of a seat - based extended user interface 530 for implementing the playback of one or more natural landscapes for one or more occupants located in vehicle 100. Figure 33 Similar to Figure 32 the display depicted in, except for the natural landscapes provided for playback. For example, the first axis 1200 and the second axis 1202 generally correspond to a natural landscape of waterfall - based audio output. Thus, as the touch point 1103 moves onto or around the first axis 1200 and the second axis 1202, the audio controller 175 transmits waterfall - based audio output. The second axis 1202 and the third axis 1204 generally correspond to a natural landscape of animal sounds. Thus, as the touch point 1103 moves onto or around the second axis 1202 and the third axis 1204, the audio controller 175 transmits animal - based sounds. The first axis 1200 and the third axis 1204 generally correspond to a natural landscape of wind - based audio output. Thus, as the touch point 1103 moves onto or around the first axis 1200 and the third axis 1204, the audio controller 175 transmits wind - based audio output. The user can move the touch point 1103 such that the series of concentric spheres 1105 can overlap onto one or more of the first axis 1200, the second axis 1202, and the third axis 1204, such that any one or more of the waterfall - based audio, animal - based sounds, and / or wind - based audio can be played by the audio controller 175. The greater the extent to which the concentric spheres 1105 are positioned on one or more of the first axis 1200, the second axis 1202, and the third axis 1204, the greater (or higher) the level or volume played by the audio controller 175 for the corresponding natural landscape.

[0165] As Figure 33 shown, the touch point 1103 shows that the occupant desires that most of the audio playback includes natural landscape audio corresponding to animal sounds, with portions of waterfall and wind - based audio also included in the total audio output. In this regard, the volume (or audio level) of the animal sounds may be greater than that of the waterfall and the wind. It should also be recognized that the occupant can also select various natural landscapes by selecting points on the outermost sphere 1109, as described above in connection with Figure 32 described. Similarly, the immersion control can be selected based on the manner described above in connection with Figure 32 described.

[0166] Figure 34 Shows another example of a seat - based extended user interface 530 for implementing the playback of one or more natural landscapes for one or more occupants located in vehicle 100. Figure 34 Similar to Figures 32 to 33The displays depicted are generally similar, except for the natural landscapes provided for playback. For example, the first axis 1200 and the second axis 1202 generally correspond to natural landscapes of traffic-based audio output. Thus, as the touch point 1103 moves onto or around the first axis 1200 and the second axis 1202, the audio controller 175 transmits traffic-based audio output. The second axis 1202 and the third axis 1204 generally correspond to natural landscapes of city-based sounds. Thus, as the touch point 1103 moves onto or around the second axis 1202 and the third axis 1204, the audio controller 175 transmits city-based sounds. The first axis 1200 and the third axis 1204 generally correspond to natural landscapes of human-based audio output. In this regard, the human-based audio output (or natural landscape) corresponds to sounds made by people in a crowd, children playing with each other, etc. Thus, as the touch point 1103 moves onto or around the first axis 1200 and the third axis 1204, the audio controller 175 transmits city-based audio output. The user can move the touch point 1103 such that the series of concentric spheres 1105 can overlap onto one or more of the first axis 1200, the second axis 1202, and the third axis 1204, and any one or more of traffic-based audio, city-based sounds, and / or human-based audio can be played by the audio controller 175. The greater the extent to which the concentric spheres 1105 are positioned on one or more of the first axis 1200, the second axis 1202, and the third axis 1204, the greater (or higher) the level or volume played by the audio controller 175 for the corresponding natural landscape.

[0167] As Figure 33 shown, the touch point 1103 indicates that the occupant desires that most of the audio playback includes natural landscape audio corresponding to city sounds, where portions of traffic and human-based audio are also included in the total audio output. In this regard, the volume (or audio level) of the city sounds may be greater than the volume of the traffic and human-based audio outputs. It should also be recognized that the occupant can also select various natural landscapes by selecting points on the outermost sphere 1109, as described above in connection with Figure 32 described. Similarly, the immersion control can be selected based on the manner described above in connection with Figure 32 described.

[0168] Figure 35 depicts various examples of ways in which various natural landscapes can be selected according to one embodiment. Figure 35Shows various natural landscapes, such as water flow, wind, and bird audio outputs. In one example, and as described above, the touch point 1103 includes the outermost sphere 1109. In this case, the outermost sphere 1109 defines a plurality of points 1111 that the user can select to move the touch point 1103 away from the center point 1120 where the first axis 1200, the second axis 1202, and the third axis 1204 intersect each other. For example, Figure 35 Shows a first point 1180, a second point 1182, and a third point 1184 formed on the outermost sphere 1109, which the user can select to move or manipulate away from the center point 1120 to correspondingly select the corresponding natural landscape. Although Figure 35 Shows three points (e.g., 1180, 1182, 1184), it should be recognized that the touch point 1103 can include any number of points that the user can select / grab and move towards and away from the center point 1120 to correspondingly move the touch point 1103 to select one or more of the natural landscapes. In one example, a Bezier curve can be used to define the overall shape of the touch point 1103, as generally shown at 1170.

[0169] In another embodiment that can be used to select the corresponding natural landscape, a first additional axis 1190, a second additional axis 1192, and a third additional axis 1194 are shown. In this regard, the first additional axis 1190 can correspond to the natural landscape of water flow, the second additional axis 1192 can correspond to the natural landscape of wind, and the third additional axis 1194 can correspond to the natural landscape of birds. As described above, the touch point 1103 in this embodiment is generally stationary, but the occupant can move the first point 1180 on the first additional axis 1190 towards or away from the center point 1120 to control the amount of the water flow-based natural landscape played by the audio system 102. Additionally, the occupant can move the second point 1182 on the second additional axis 1192 towards or away from the center point 1120 to control the amount of the wind-based natural landscape played by the audio system 102. Additionally, the occupant can move the third point 1184 on the third additional axis 1194 towards or away from the center point 1120 to control the amount of the bird-based natural landscape played by the audio system 102. It should be recognized that the mobile device 1102 can provide any number of additional axes consistent with any number of natural landscapes. For example, the mobile device 1102 can display five natural landscape sounds for selection, where each natural landscape has a corresponding axis that allows the occupant to be able to select points on the outermost sphere 1105 relative to the center point 1120 to select the desired natural landscape.

[0170] Figure 36Illustrates various examples of how a user can select immersion control for touch point 1103 according to one embodiment. In the first embodiment, as generally shown at 1300, touch point 1103 can be moved around a first axis 1200, a second axis 1202, and a third axis 1204 to select one or more natural landscapes, such as wind, marine life, and birds. For example, an occupant can drag touch point 1103 around the first axis 1200, the second axis 1202, and the third axis 1204 to change the amount of the natural landscape sound that the occupant hears. As generally shown at 1302, the occupant can pinch or spread the outermost sphere 1109 to adjust the immersion level. Additionally, as described in the second embodiment in connection with Figure 35 the occupant can simply select a point on the outermost sphere 1109 and move that point toward or away from the center point 1120 to respectively decrease or increase the desired amount of the natural landscape played by the audio system 102.

[0171] As generally shown at 1302, in the first embodiment, the occupant can pinch or spread the outermost sphere 1109 to adjust the immersion level. In another embodiment, and as described above, the sound immersion control can be performed as follows: The occupant can select the center sphere 1107 and move the sphere 1107 inward away from the outermost sphere 1109 or outward toward the outermost sphere 1109 while the touch point 1103 (and the outermost sphere 1109) remains substantially stationary. This movement of the center sphere 1107 causes a change in the visual terrain generation of the touch point 1103, thereby causing a change in the immersion control.

[0172] Figure 37 Illustrates a tutorial according to one embodiment that shows how an occupant selects a natural landscape and an immersion level on the mobile device 1102. Figure 37 。 Figure 37 The visual effects on the left illuminate the respective points 1180, 1182, and 1184 on the outermost sphere 1109 that the occupant can select to select the corresponding natural landscapes. Figure 37 The visual effects on the right show the pinch and spread operations to adjust the immersion level. For example, the user can spread the outermost sphere 1109 relative to the center sphere 1107 to increase the immersion level. The user can pinch the outermost sphere 1109 and move two fingers closer to each other relative to the center sphere 1107 to decrease the immersion level.

[0173] Figure 38 Illustrates a schematic diagram 1500 of playing seat-based natural landscapes 1606a to 1606n using the audio system 102 according to one embodiment. Specifically, block diagram 1500 shows that the audio system 102 generally includes an audio controller 175 and a user interface 530. As described in connection with Figures 32 to 25As described, the user interface 530 generally enables a user to select or adjust various natural landscapes 1606a to 1606n, as depicted on a mobile device 1102 within the vehicle or on a user interface located on the seat back of the vehicle 100 and electrically coupled to the audio controller 175. Generally, it should be recognized that multiple mobile devices 1102a to 1102n with corresponding user interfaces 530 can be located within the vehicle. For example, the mobile device 1102a can be located around the seat 109, the mobile device 1102b can be located around the seat 110, the mobile device 102c can be located around the seat 112a, and the mobile device 102d can be located around the seat 112b. Except Figure 37 as shown in detail below, the following description also applies to each of the mobile devices 1102a to 1102n.

[0174] The natural landscapes 1606a to 1606n can correspond to lightning-based audio, bird-based audio, wave-based audio, traffic-based audio, city-based audio, human-based audio, waterfall-based audio, animal-based audio, and wind-based audio, etc., as combined Figures 32 to 35 as shown. It should be recognized that the natural landscapes 1606a to 1606n can correspond to other types of audio and should not be limited to those mentioned in this disclosure. Generally, the audio controller 175 can include a memory that stores a set of natural landscapes 1606a to 1606n (or audio files) depicted on the user interface 530 for selection.

[0175] The user interface 530 also provides the overall position of the touch point 1103 located thereon for the selected natural landscape and provides an indication of the position of the touch point 1103 on any one or more of the first axis 1200, the second axis 1202, and the third axis 1204. The user interface 530 includes level circuits 956a to 956c, and each level circuit provides an output (or digital information) indicating the required natural landscapes 1606a to 1606n and the corresponding volume (or level) based on the position of the touch point 1103 on the respective axes 1200, 1202, and 1204. Generally, the level circuit 956a receives a signal indicating the position of the first point 1180 on the first additional axis 1190 (see Figure 36 ), and this signal indicates the magnitude of the volume applied to the natural landscape 1606a. The level circuit 956b receives a signal indicating the position of the second point 1182 on the second additional axis 1192, and this signal indicates the magnitude of the volume applied to the natural landscape 1606b. The level circuit 956c receives a signal indicating the position of the third point 1184 on the third additional axis 1194, and this signal indicates the magnitude of the volume applied to the natural landscape 1606c.

[0176] The user interface 530 includes the main volume level selector 1502 as described above in connection with Figure 32 The main volume level selector 1502 enables an occupant to select the overall volume of the entire nature scene mix via his / her mobile device 1102. The level circuit 956d provides an output to the audio controller 175 indicative of the overall volume applied to the nature scene mix. The immersion control and virtual venue circuit 958 receives information indicative of the desired level of immersion selected via the immersion selection 1210 as discussed above. In one embodiment, each of the mobile devices 1102a to 1102d may also enable an occupant to provide a selected virtual venue such that the occupants on the corresponding seats 109, 110 and 112a to 112b can experience the nature scene audio and / or the main audio in the desired virtual venue. It should be appreciated that this may be provided via a separate input to the audio controller 175. Each of the mobile devices 1102a to 1102d transmits information indicative of the selected virtual venue and / or immersion control to be applied to the desired nature scene to the audio controller 175. In response, the audio controller 175 applies the selected virtual venue and / or immersion control to the selected nature scenes 160a to 160n. It should be appreciated that one or more of the mobile devices 1102a to 11022n may also provide the main audio 970. In other examples, the mobile device 1102 may continue to provide signals indicative of the nature scenes 1606a to 1060n and / or immersion and / or virtual venue, however, a different audio source may provide the main audio 970. Assuming the vehicle includes one or more microphones 150, the captured audio received by the one or more microphones 150 may be provided to the audio controller 175.

[0177] For a given mobile device 1102, the audio controller 175 includes a plurality of mixers 1560 (e.g., one mixer 1560 for each seat or mobile device near a seat) to mix the respective levels of the natural landscapes 1606a to 160n, the main audio 970, the captured audio, and the selected immersion and / or virtual venue (from the immersion control and virtual venue circuitry 958). Thus, for a single mobile device 1102 (or seat) incorporating the natural landscapes 1606 to 1606n, the corresponding mixer 1560 adjusts the overall level (or volume) of the natural landscapes 1606a to 1606n based on the volume information provided by the level circuitry 956a to 956d. Each corresponding mixer 1560 also receives a signal from the immersion control and virtual venue circuitry 958 that typically provides information corresponding to the amount of level to be applied to the selected virtual venue based on the position of the touch point 703 of the corresponding mobile device 1102. The mixer 1560 also receives the main audio 970 to combine the main audio 970 with the selected levels (or volumes) of the natural landscapes 1606a to 1606n, the selected levels (or volumes) of the selected immersion and / or virtual venue, and the main audio 970.

[0178] The mixer 1560 combines the selected levels of the natural landscapes 1606a to 1606n, the immersion level and / or virtual venue along with the main audio and provides a first audio output to the amplifier 1562. The amplifier 1562 amplifies the audio output to generate a final audio output for transmission to the corresponding mobile device 1102 and / or the speakers in the vehicle. It should be appreciated that the audio controller 175 may be equipped with a plurality of amplifiers 1562, each amplifier providing the final audio output (or audio output signal) including the selected natural landscapes 1606a to 1606n, the main audio 970, and the captured audio to the corresponding mobile device 1102 located around the corresponding seats 109, 110, 112a, 112b or to the respective near-field speakers located on each of the seats 109, 110, 112a, 112b. It should be appreciated that each occupant located on the seats 109, 110, 112a, 112b in the vehicle may adjust the blend of the dynamic sound elements (e.g., the natural landscapes 1606a to 1606n, the main audio 970, and / or the captured audio) via their respective user interfaces on their mobile devices 1102, and the audio controller 175 may transmit the desired dynamic sound elements of each occupant directly to their corresponding mobile devices 1102 and / or to the speakers located around each of the seats in which the user is located.

[0179] The user interface 530 also provides control signals to the audio controller 175 to receive inputs from any one or more of the microphones 150 located in the vehicle. In this case, the audio controller 175 can receive the audio captured from within the passenger compartment of the vehicle and play the captured audio together with the natural landscape 606 and the main audio 970 within the selected virtual venue 905. An example of this aspect was described above in connection with the occupant clapping, where such audio was captured and played together with the main audio 970 within the selected virtual venue 905 of a church. Further mentioned, one of the natural landscapes 606 can also be played together with the main audio 970 and the audio captured within the vehicle. The audio controller 175 is generally configured to receive the main audio (or music) 970 from another audio source (e.g., a mobile device, a radio, a CD player, etc.) to play together with the selected immersion and / or the selected mixing level (or natural landscape) 1606a to 1606d of the virtual venue.

[0180] It should be appreciated that the controllers as disclosed herein can include various microprocessors, integrated circuits, memory devices (e.g., flash memory, random access memory (RAM), read only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), or other suitable variations thereof), and software that cooperate with each other to perform one or more operations disclosed herein. Additionally, such controllers as disclosed herein can utilize one or more microprocessors to execute a computer program embodied in a non-transitory computer-readable medium that is programmed to perform any number of the disclosed functions. Furthermore, the controllers provided herein include a housing and various numbers of microprocessors, integrated circuits, and memory devices (e.g., flash memory, random access memory (RAM), read only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM)) located within the housing. The one or more controllers as disclosed also include hardware-based inputs and outputs to receive data from and transmit data to other hardware-based devices as discussed herein.

[0181] Although the exemplary embodiments have been described above, these embodiments are not intended to describe all possible forms of the invention. Rather, the words used in the specification are words of description rather than limitation, and it should be understood that various changes can be made without departing from the spirit and scope of the invention. Additionally, the features of the various embodiments can be combined to form other embodiments of the invention.

Claims

1. A system for generating audio in a listening environment of a vehicle, the system comprising: A moving device located around a first seat in a vehicle, the moving device comprising: at least one first controller, the at least one first controller being programmed to: communicating with an audio controller located in the vehicle; generating one or more options corresponding to the desired natural sounds on a display for selection; transmitting one or more options to select a desired natural sound on a display; and A first signal indicative of the selected desired nature sound is transmitted to the audio controller to play the selected desired nature sound in the vehicle.

2. The system of claim 1, wherein the at least one first controller is further programmed to adjust the volume of the selected desired nature sound to indicate an overall volume to be applied by the audio controller when playing the selected desired nature sound.

3. The system of claim 1, wherein the audio controller is programmed to receive an incoming audio signal including primary audio data from the mobile device for playback.

4. The system of claim 3, wherein the audio controller is further programmed to receive a first control signal from the mobile device indicating a desired location to play the primary audio data.

5. The system of claim 4, wherein the audio controller is further programmed to adjust a reverberation of at least the incoming audio signal to play the primary audio data in a desired location at the first seat in the vehicle.

6. The system of claim 5, wherein the audio controller is further programmed to play the selected desired natural sounds together with the primary audio data in the desired location at the first seat in the vehicle in the listening environment.

7. The system of claim 6, wherein the audio controller is further programmed to receive a captured audio signal indicative of captured audio from within the listening environment of the vehicle.

8. The system of claim 7, wherein the audio controller is further programmed to play the captured audio along with the selected desired natural sounds and the primary audio data at the first seat in the vehicle in the listening environment.

9. A system for generating audio in a listening environment of a vehicle, the system comprising: Memory device; as well as at least one audio controller, the at least one audio controller comprising the memory device and being programmed to: receiving a first signal indicative of a first selected desired natural sound from a first mobile device located about a first seat of the vehicle; receiving a second signal indicative of a second selected desired natural sound from a second mobile device located about a second seat of the vehicle, the second selected desired natural sound being different from the first selected desired natural sound; generating a first audio output signal comprising the first selected desired natural sound and a second audio output signal comprising the second selected desired natural sound; transmitting the first audio output signal to one of the first mobile device and a first speaker located around the first seat to play the first audio output signal; as well as The second audio output signal is transmitted to one of the mobile device and a second speaker located around the second seat to play the second audio output signal.

10. The system of claim 9, wherein the audio controller is further programmed to receive a first incoming audio signal including first master data from the first mobile device for playback.

11. The system of claim 10, wherein the audio controller is further programmed to receive a second incoming audio signal including second primary data from the second mobile device for playback.

12. The system of claim 11, wherein the audio controller is further programmed to receive a first control signal from the first mobile device indicating a first desired location to play the first primary audio data.

13. The system of claim 12, wherein the audio controller is further programmed to adjust a reverberation of the first incoming audio signal based on the first control signal to play the first primary audio data in a desired first location at the first seat in the vehicle.

14. The system of claim 12, wherein the audio controller is further programmed to receive a second control signal from the second mobile device indicating a second desired location to play the second primary audio data.

15. The system of claim 14, wherein the audio controller is further programmed to adjust a reverberation of the second incoming audio signal based on the second control signal to play the second primary audio data in a desired second location at the second seat in the vehicle.

16. The system of claim 14, wherein the audio controller is further programmed to play the first selected desired natural sound along with the first primary audio data in the first desired location at the first seat in the vehicle in the listening environment.

17. The system of claim 16, wherein the audio controller is further programmed to play the second selected desired natural sound together with the second primary audio data in the second desired location at the second seat in the vehicle in the listening environment.

18. The system of claim 9, wherein the audio controller is further programmed to receive a captured audio signal indicative of captured audio from within the listening environment of the vehicle.

19. The system of claim 18, wherein the audio controller is further programmed to: generating the first audio output signal comprising the captured audio and the second audio output signal comprising the captured audio; transmitting the first audio output signal having the first selected desired natural sound together with the captured audio to one of the first mobile device and the first speaker located around the first seat to play the first audio output signal; and The second audio output signal having the second selected desired natural sound is transmitted along with the captured audio to one of the second mobile device and the second speaker located around the second seat to play the second audio output signal.

20. A method for generating audio in a listening environment of a vehicle, the method comprising: receiving a first signal indicative of a first selected desired natural sound from a first mobile device located about a first seat of the vehicle; receiving a second signal indicative of a second selected desired natural sound from a second mobile device located about a second seat of the vehicle, the second selected desired natural sound being different from the first selected desired natural sound; generating a first audio output signal including the first selected desired natural sound and a second audio output signal including the second selected desired natural sound; transmitting the first audio output signal to one of the first mobile device and a first speaker located around the first seat to play the first audio output signal; as well as The second audio output signal is transmitted to one of the mobile device and a second speaker located around the second seat to play the second audio output signal.

Images