Method and system for pickup of external sounds and reproduction in the cabin to generate

Abstract

A method performed at a processing entity of a vehicle cabin is provided. The method comprises the steps of: receiving, from one or more external microphones of the acoustic system located outside the vehicle cabin, an external acoustic signal associated with an external acoustic sound originating from the outside of the vehicle cabin; generating a processed acoustic signal based on the external acoustic signal; and causing the processed acoustic signal to be reproduced inside the vehicle cabin by the acoustic system. The method allows an immersive audio effect to be generated in the vehicle cabin, immersing an occupant of the vehicle into a natural sound-producing environment, while allowing an atmosphere inside the vehicle cabin that is isolated from the atmosphere of the natural sound-producing environment. Furthermore, an acoustic system, a processing entity and a computer program product for carrying out the steps of the method and uses thereof are provided.

Description

Technical Field

[0001] This application relates to a method and system for an audio acoustic system for a vehicle. Background Technology

[0002] Typically, car manufacturers tend to provide good acoustic isolation between the vehicle's interior and exterior sounds. This makes the atmosphere inside the vehicle cabin (e.g., the car interior) more comfortable, especially when making phone calls, listening to music, focusing on navigation instructions, or simply relaxing in a quiet environment. Therefore, modern cars typically have acoustic systems that isolate them from external sounds. However, in some situations, hearing external sounds can be advantageous, such as being able to hear approaching cars on nearby roads hidden by obstacles; or hearing the noise of the sea or waterfall; or hearing city noise. In such cases, vehicle occupants tend to open the vehicle windows to hear external sounds, which is sometimes nearly impossible due to inclement weather conditions (e.g., wind, rain, excessive heat, or excessive cold), and can therefore negatively impact climate control. Summary of the Invention

[0003] In view of the above, there is a need for a method and system for picking up sound originating from outside the vehicle cabin and reproducing the sound inside the vehicle cabin while keeping the vehicle windows closed.

[0004] These requirements must be met by the features defined in the independent claims. Dependent claims define additional embodiments.

[0005] A method is provided. The method is performed at a processing entity within a vehicle compartment. The method includes receiving, from one or more external microphones of an acoustic system located outside the vehicle compartment, an external acoustic signal associated with external acoustic sounds originating outside the vehicle compartment. The method further includes generating a processed acoustic signal based on the external acoustic signal. The method further includes causing the processed acoustic signal to be reproduced by the acoustic system inside the vehicle compartment.

[0006] A processing entity is provided. The processing entity includes at least one computer processor configured to perform the steps of a method. The method includes receiving, from one or more external microphones located outside a vehicle compartment, an external acoustic signal associated with external acoustic sounds originating outside the vehicle compartment. The method further includes generating a processed acoustic signal based on the external acoustic signal. The method further includes causing the acoustic system to reproduce the processed acoustic signal inside the vehicle compartment.

[0007] A computer program product is provided. The computer program product includes computer-readable instructions stored on an electronic storage medium, which, when executed on a processing entity, cause the processing entity to perform the steps of a method. The method includes receiving, from one or more external microphones located outside a vehicle compartment, an external acoustic signal associated with external acoustic sounds originating outside the vehicle compartment. The method further includes generating a processed acoustic signal based on the external acoustic signal. The method further includes causing the acoustic system to reproduce the processed acoustic signal inside the vehicle compartment.

[0008] An acoustic system for a vehicle is provided. The acoustic system includes one or more external microphones and one or more speakers controlled by a processing entity. The processing entity includes at least one computer processor configured to perform the steps of a method. The method includes receiving external acoustic signals associated with external acoustic sounds originating outside the vehicle compartment from one or more external microphones located outside the vehicle compartment. The method further includes generating a processed acoustic signal based on the external acoustic signals. The method further includes causing the acoustic system to reproduce the processed acoustic signal inside the vehicle compartment.

[0009] A vehicle including an acoustic system is provided. The acoustic system includes one or more external microphones and one or more speakers controlled by a processing entity. The processing entity includes at least one computer processor configured to perform the steps of the method. The method includes receiving external acoustic signals associated with external acoustic sounds originating from outside the vehicle compartment from one or more external microphones located outside the vehicle compartment. The method further includes generating a processed acoustic signal based on the external acoustic signals. The method further includes causing the acoustic system to reproduce the processed acoustic signals inside the vehicle compartment.

[0010] The uses of the method, processing entity, computer program product, and acoustic system are provided. These uses allow for the creation of immersive audio effects within a vehicle cabin, immersing vehicle occupants in a natural acoustic environment while simultaneously allowing for an atmosphere within the vehicle cabin that is isolated from the natural acoustic environment. This and further advantages will become more apparent from the detailed description and accompanying drawings.

[0011] Without departing from the scope of this disclosure, the features set forth above and those described below may be used not only in the explicitly stated corresponding combinations, but also in other combinations or separately. Attached Figure Description

[0012] The present disclosure will be described more fully below with reference to the accompanying drawings, which illustrate embodiments of the present disclosure. However, the present disclosure should not be construed as limiting itself to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art. Similar reference numerals always refer to similar elements.

[0013] Figure 1 An acoustic system 100 according to one of several embodiments is schematically shown.

[0014] Figure 2 An acoustic system 200 according to one of several embodiments is schematically shown.

[0015] Figure 3 A method 300 according to one of the multiple embodiments is illustrated schematically.

[0016] Figure 4 The processing entity 1000 according to one of the multiple implementations is illustrated schematically. Detailed Implementation

[0017] The properties, features, and advantages described above, as well as the ways in which said properties, features, and advantages are realized, will become clearer and more readily understood in conjunction with the following description of exemplary embodiments, explained in more detail with reference to the accompanying drawings. For simplicity and illustrative purposes, this disclosure is described primarily by reference to exemplary embodiments thereof. In the following description, numerous specific details are set forth to provide a thorough understanding of this disclosure. However, it will be apparent to those skilled in the art that this disclosure may be practiced without being limited to these specific details. Well-known methods and structures are not described in detail in this specification so as not to unnecessarily obscure this disclosure.

[0018] Some examples of this disclosure generally provide multiple circuits or other electrical devices. All references to the circuits and other electrical devices and the functionality provided for each are not intended to be limited to what is shown and described herein. While specific labels may be assigned to the various circuits or other electrical devices disclosed, such labels are not intended to limit the scope of operation of the circuits and other electrical devices. Such circuits and other electrical or electronic devices may be combined and / or separated from each other in any way based on a particular type of desired electrical implementation. It should be appreciated that any circuit or other electrical or electronic device disclosed herein (e.g., an acoustic system) may include any number of microcontrollers, central processing units (CPUs), integrated circuits, memory devices, and software that cooperate with each other to perform the operations disclosed herein. Additionally, any one or more of the electrical or electronic devices may be configured to execute program code (for infotainment systems) implemented on a non-transitory computer-readable medium, which is programmed to perform any number of functions as disclosed.

[0019] In the following, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. It should be understood that the following description of the embodiments should not be interpreted in a limiting sense. The scope of this disclosure is not intended to be limited by the embodiments described below or by the drawings, which are considered to be illustrative only.

[0020] The diagrams should be considered schematic representations, and the elements shown are not necessarily to scale. Rather, the elements are represented such that their function and general purpose are obvious to those skilled in the art. Any connection or coupling between functional blocks, devices, components, or other physical or functional units shown in the diagrams or described herein may also be achieved through indirect connections or couplings. Coupling between components may also be established through wired connections. Functional blocks may be implemented in hardware, firmware, software, or a combination thereof.

[0021] In the context of this disclosure, the term "vehicle" can be understood as any vehicle with a vehicle compartment, such as a human-driven and controlled automobile, including trucks and trailers, warehousing machinery, construction machinery, etc. Generally, automobile manufacturers tend to provide good acoustic isolation between the interior and exterior sounds of a vehicle. This makes the car cabin atmosphere more comfortable, especially when making phone calls, listening to music, focusing on navigation instructions, or simply relaxing in a quiet environment. However, in some situations, the user, or in other words, the occupants of the vehicle, desires to eliminate obstructions to external sounds. For example, when a car is driving on a busy street, where visual observation of the street is blocked by obstacles such as parked cars, trees, billboards, etc. These conditions can create a risk of collision with an approaching vehicle. To overcome insufficient visibility, drivers frequently open their car windows before driving onto the street to be able to hear the noise of approaching vehicles. Sometimes, this approach can be uncomfortable for the driver or passengers due to external conditions such as precipitation, heat or cold, strong winds, insects, or unpleasant odors.

[0022] Another reason for opening a car window might be that the user (car occupant, driver) wants to hear the sounds around them. For example, it could be the sounds of the city, the sound of ocean waves, the sound of a river, the sound of a jungle, the sound of a nearby festival, and other sounds that the user might want to hear.

[0023] As mentioned above, opening car windows is not always pleasant or efficient in terms of vehicle climate control. Therefore, there is a need to address this issue and provide the aforementioned immersion in the surrounding sound environment. This need is met by receiving external acoustic signals from one or more external microphones. Furthermore, external microphones used in a microphone matrix can provide the opportunity to hear sounds from specific directions. For example, when driving in a canyon with a river on the right and rocks on the left, a user can hear the sound of the river on the right while masking unpleasant, noisy reflections from the rocks on the left, which include echoes of the river and the car. In some examples, the external sounds received by the external microphones and sent to a processing entity can be mixed by the processing entity with an audio signal (e.g., with a music signal) at a certain weight (e.g., according to a mix ratio), which can then be played in the car at that moment, thus creating a pleasant atmosphere in the cabin, or in other words, producing additional audio effects from the mixed audio / acoustic signals. These and further advantages will become more apparent from the following detailed description and accompanying figures.

[0024] Figure 1 An acoustic system 100 according to one of several embodiments is schematically shown.

[0025] The acoustic system 100 for vehicle 110 includes one or more external microphones 102. The acoustic system 100 may further include one or more speakers 104. The one or more external microphones 102 and one or more speakers 104 of the acoustic system 100 are controlled by a processing entity 106. In some examples, the processing entity 106 may be part of an automotive multimedia unit 108.

[0026] In some examples, one or more external microphones 102 may include only one external microphone. For example, an omnidirectional external microphone may be located, for instance, on the vehicle's roof to cover all incoming directions of external sound. This would be the simplest configuration to allow the detection and processing of external sound and the reproduction of the processed signal (or a safety notification based on that processed signal) inside the vehicle. However, for more precise determination of the incoming direction of external sound, it may be advantageous to include one or more external microphones 102, particularly a microphone matrix or array, and even more preferably, to have said matrix or array located on different sides of the vehicle.

[0027] One aspect of this disclosure relates to a method implemented at a processing entity 106 in a vehicle compartment 218. The vehicle compartment 218 is... Figure 2 The method is further described in the context of [the above description]. It receives external acoustic signals associated with external acoustic sounds originating outside the vehicle compartment from one or more external microphones 102 of an acoustic system 100 located outside the vehicle compartment 218. The method generates a processed acoustic signal based on the external acoustic signal and causes the processed acoustic signal to be reproduced inside the vehicle compartment by the acoustic system (i.e., by at least one speaker 104 of the acoustic system).

[0028] In some examples, the processed acoustic signal may be referred to as a digital audio signal, which is obtained from the analog electrical input audio signal at least by means of an analog-to-digital converter (ADC).

[0029] In some examples, causing the acoustic system to reproduce a processed acoustic signal inside the vehicle cabin can be achieved by converting the processed acoustic signal into an output analog electrical audio signal using at least one digital-to-analog converter (DAC), and then converting the output analog electrical audio signal into sound waves using at least one speaker located in the vehicle cabin. Thus, the processed digital audio signal can be reproduced inside the vehicle cabin. Therefore, in one implementation, the method includes receiving an external acoustic signal associated with external acoustic sounds originating outside the vehicle cabin from at least one external microphone of the acoustic system (located outside the vehicle cabin), and converting the received external acoustic signal into an analog (continuous) electrical input audio signal. The method may further include generating a processed digital audio signal, which is obtained from the analog electrical input audio signal at least by means of an analog-to-digital converter (ADC). The method may further include: converting the processed digital audio signal into an output analog electrical audio signal by means of at least one digital-to-analog converter (DAC), and then converting the output analog electrical audio signal into sound waves by means of at least one speaker located in the vehicle cabin, so as to cause the processed digital audio signal to be reproduced by an acoustic system inside the vehicle cabin.

[0030] The processing entity 106 can be understood as a computing device with processing units, such as in Figure 4 The processing unit 1200 is further described in the context of the above. This processing unit is capable of executing software instructions to manipulate audio signals. The acoustic system 100 can be understood as a collection of microphones (i.e., one or more external microphones 102) and speakers (i.e., one or more speakers 104) designed to capture and reproduce sound waves. The external microphones 102 may be mounted on the exterior of the vehicle to capture sound from the surrounding environment.

[0031] The effect can be improved driving safety when it's necessary to hear sounds from potentially approaching cars that might be obscured by obstacles such as billboards, trees, or parked vehicles. This eliminates the need to open windows, a need that can be problematic in inclement weather conditions, with unpleasant odors from outside, dust, etc. Additionally, it creates new immersive audio effects, making car occupants feel part of an environment with sounds, such as urban or natural sounds, even in adverse weather conditions.

[0032] In some examples, the sound source 112, the human-computer interaction (HMI) unit 114, and the sound control unit 116 may be communicatively coupled to the processing entity 106.

[0033] In some examples, processing entity 106 may include an audio processing unit. The audio processing unit may include an upmixing unit (or simply a "mixing unit") configured to mix various audio / acoustic signals (e.g., an external acoustic signal with another acoustic signal, such as an audio signal generated from an audio source).

[0034] According to this aspect, processing entity 106 generates a processed acoustic signal based on an external acoustic signal. In some examples, the processed acoustic signal may include a mixture of the external acoustic signal and an audio signal generated from an audio source and played in the vehicle cabin by the acoustic system (i.e., the speakers of the acoustic system). For example, the audio source of the audio signal may be a radio, a connected portable device (e.g., a telephone, smartphone, tablet PC, etc.), a CD player or MP3 player, a Bluetooth audio device, a USB stick and SD card reader, and other external and internal (i.e., integrated into the vehicle acoustic system) devices and sources. Therefore, in some examples, generating a processed acoustic signal based on an external acoustic signal can be understood as mixing the external acoustic signal with an audio signal generated from an audio source, which may be a radio, telephone, CD, MP3, Bluetooth audio, USB stick, and other sources. Additionally or alternatively, generating a processed acoustic signal based on an external acoustic signal may involve other types of processing of the external acoustic signal, such as spatial or immersive audio processing, automatic adjustment of the level or intensity of external sounds, etc.

[0035] In some examples, generating a processed acoustic signal based on an external acoustic signal can be understood as receiving the original external acoustic signal from one or more external microphones and reproducing the original external acoustic signal as received (i.e., without any manipulation of it).

[0036] In some examples, the received external acoustic signal can be converted into an analog (continuous) electrical input audio signal.

[0037] Therefore, in some examples, Figure 1 The sound source 112 can be a radio, telephone, CD, MP3, Bluetooth audio, USB stick and similar sources, which can be mixed by the processing entity 106 with external acoustic signals from the external microphone 102.

[0038] In some examples, generating a processed signal may include changing the mixing ratio of the audio signal mixed in the processed acoustic signal with the external acoustic signal. The mixing ratio can be in the range of 0%-100%. When the mixing ratio is 100%, generating a processed signal can be understood as stopping the output of the audio signal (which may be referred to as the master signal, such as music, personal assistant, or mobile phone) and replacing the audio signal with the processed acoustic signal. When the mixing ratio is 0%, generating a processed signal can be understood as stopping the mixing and (temporarily) replacing the external acoustic signal with the audio signal in the processed acoustic signal. When the mixing ratio is higher than 0% and less than 100%, generating a processed signal can be understood as jointly reproducing the external acoustic signal and the audio signal according to the mixing ratio with weights for each signal.

[0039] For example, the mixing ratio could be 25%, meaning the external acoustic signal has a weight of 0.25, while the audio signal in the mixed (processed) acoustic signal has a weight of 0.75. In another example, the mixing ratio could be 50%, meaning the external acoustic signal has a weight of 0.50, while the audio signal in the mixed (processed) acoustic signal has a weight of 0.50. In yet another example, the mixing ratio could be 75%, meaning the external acoustic signal has a weight of 0.75, while the audio signal in the mixed (processed) acoustic signal has a weight of 0.25. In an advantageous implementation, the weight of the external acoustic signal in the mixed (processed) acoustic signal can be between 0.25 and 0.75, allowing vehicle occupants to hear both the external acoustic sound reproduced from the external acoustic signal and the audio signal, without the external acoustic sound overpowering the audio signal. The favorable weighting range of 0.25 to 0.75 for the external acoustic signal in the mixed (processed) acoustic signal can also be combined with an option to stop mixing (i.e., a weight equal to zero, combined with the 0.25-0.75 range). In this case, vehicle occupants can be provided with the option to mix the signal in the 0.25-0.75 range and to stop mixing. For acoustic systems with multiple speakers, the weighting in the above sense can be applied individually to each pair of "speaker (output) channel - external microphone (input) channel," that is, each pair of "speaker channel - external microphone channel" can have its own mixing weight. For example, all woofer channels can have a mixing weight of 0.25, while all midrange speaker channels can have a weight of 0.4, and for all tweeter channels, the weight can be 0.3. The weighting can also vary over time depending on various factors (such as internal or external conditions, events), for example, depending on the instantaneous power of the audio signal to be played and the instantaneous power of the received external signal received by the external microphone; depending on the number of vehicle occupants and the position of their seats; depending on the level of cabin noise (e.g., the intensity of passenger sounds inside the vehicle cabin), etc.

[0040] In some examples, a mixing ratio ranging from 0% to 100% (or a weight of 0 to 1.0) or from 25% to 75% (or a weight of 0.25 to 0.75) can be applied to a first plurality of audio channels that generate one or more first audio signals, while a second plurality of audio channels that generate second audio signals can be excluded from the mixing. For example, when a telephone rings, the radio channel can switch to the telephone channel, and the external acoustic signal can be stopped to answer the call; and subsequently, the audio channel can switch back to the radio, which can continue playing in the background of the radio along with the external acoustic signal. For example, the external acoustic signal can have a volume level of 25% to 75% of the radio signal volume, which would correspond to a weight of 0.25 to 0.75 for the external acoustic signal in the mixed (processed) acoustic signal (and would also correspond to a mixing ratio of 25% to 75%).

[0041] In some examples, generating the processed signal may include user input that identifies the user of the acoustic system, determines how external acoustic sounds should be processed, and generates the processed acoustic signal based on the user input. For example, the user input may be via a human-machine interface (HMI) unit 114 or something similar. Figure 4 The input / output interface 1100, which is a similar computer interface receiver, is further described in the context of the above.

[0042] In some examples, determining user input may include any of the following: determining one or more playback directions; determining one or more mixing conditions to mix the audio signal with an external acoustic signal or to stop mixing; and / or determining deactivation criteria for deactivating the directional reproduction of the processed acoustic signal, thereby resulting in omnidirectional reproduction.

[0043] "Playback direction" can be understood as the direction from which the processed acoustic signal is reproduced in the vehicle cabin. For example, some of the speakers 104 may be placed at the rear of the vehicle cabin, while others may be placed at the front. A front playback direction can be understood as sound being played from the front of the vehicle cabin, i.e., by the speakers 104 placed at the front. The same can be understood for rear, left, and right playback directions, and for the rear, left, and right speakers 104. An omnidirectional playback direction can be understood as all speakers 104 (placed at the front and rear) reproducing (playing) the processed acoustic signal (from all directions).

[0044] In some examples, generating a processed acoustic signal may simply refer to determining output conditions for the reproduction of the processed acoustic signal inside the vehicle cabin. Therefore, the processed acoustic signal can be reproduced based on these output conditions. There can be multiple output conditions. In some examples, the output conditions may simply be the reproduction of the original external acoustic signal.

[0045] In additional or alternative examples, the output condition may refer to the reproduction of an external acoustic signal from one or more playback directions. As mentioned above, "playback direction" can be understood as the direction from which the processed acoustic signal is reproduced in the vehicle cabin. Therefore, one or more playback directions can determine from which the processed signal is reproduced in the vehicle cabin.

[0046] In some further examples, generating the processed acoustic signal may include determining the direction of entry, which indicates the direction from which the external acoustic signal originates relative to the vehicle compartment. "Relative to the vehicle compartment" can be interpreted as relative to the longitudinal axis 214 of the vehicle itself, which may correspond to the longitudinal axis 214 of the vehicle compartment, such as... Figure 2 As shown in the diagram. "Relative to the vehicle compartment" can also be interpreted as relative to the lateral axis 212 of the vehicle (vehicle compartment), as... Figure 2 As shown in the diagram. For example, the left-right direction can be determined relative to the longitudinal axis 214; and the front-back direction can be determined relative to the transverse axis 212.

[0047] "Incoming direction" can be understood as the direction from which an external acoustic signal originates relative to the vehicle cabin, or more specifically, the direction from which the vehicle's longitudinal and / or lateral axes originate. (As will be...) Figure 2 The incoming directions “from the front” and “from the rear or rearward” as described in the context can be determined relative to the lateral axis 212 that divides the vehicle in half from the front 204 to the rear 206. The incoming directions “from the right” and “from the left” can be determined relative to the longitudinal axis 214 that divides the vehicle in half from the left 210 to the right 208. In the case of using an array of external microphones (i.e., a number of closely positioned microphones on the same line (e.g., spaced 0.5 cm to 3 cm apart, thus constructing an “array”) or using a microphone matrix (i.e., a number of closely positioned microphones on a planar surface, thus forming a “matrix”), specialized signal processing methods based on manipulating the phase of the signal received from each microphone in the microphone array can be applied to create the directivity of the microphone array, thereby filtering out signals from undesired directions. These methods (which are known in signal processing theory and described in detail in the literature) allow for the selection of arbitrary (i.e., continuous) spatial directions of the received signal (e.g., from a direction +72 degrees from the longitudinal axis).

[0048] In some further examples, generating the processed acoustic signal may include generating a directional acoustic signal from the same first playback direction as the determined (incoming) direction, which, when output by the acoustic system, gives the impression of coming from the incoming direction. For example, an external acoustic signal originating from the front (having an "incoming" direction from the front) may be played from the forward playback direction (through a speaker placed at the front of the vehicle cabin).

[0049] In some further examples, generating the processed acoustic signal may include generating a directional acoustic signal from the same first playback direction as the determined direction, which, when output by the acoustic system, gives the impression of originating from the same direction, while maintaining that the direction of the external sound reproduced by the vehicle speakers depends on the vehicle's geographical orientation on the ground. This produces an effect where, regardless of the vehicle's orientation on the ground at any given moment, the external sound is reproduced in the vehicle's cabin from the same geographical direction of origin. For example, geographical orientation may be relative to north, south, east, or west. For instance, geographical orientation can be defined by the angle formed with the axes of north, south, east, or west.

[0050] In some further examples, generating the processed acoustic signal may include generating a directional acoustic signal from a second playback direction different from the determined direction, wherein the second playback direction may be any of the following: a direction opposite to the incoming direction to give the impression of mirrored external acoustic sound; a forward playback direction to give the impression of a stage effect; or a rear playback direction relative to the vehicle cabin.

[0051] In some further examples, multiple external acoustic signals may be available. For example, some of the external microphones 102 may be positioned on the left side 210 of the vehicle, while others may be positioned on the right side 208. Thus, one external acoustic signal may come from the left, and another from the right. In this case, generating the processed acoustic signal may include determining multiple incoming directions (e.g., from the left and from the right), each incoming direction indicating which direction one of the multiple external acoustic signals originates relative to the vehicle cabin. In this example, the output conditions may include one or more playback directions (e.g., left, right, front, and rear playback directions). For example, external acoustic signals from the left and from the right may be played from the front playback direction and / or from the rear playback direction. In another example, external acoustic signals from the left and from the right may be played from the front, rear, left, and right, resulting in omnidirectional reproduction from all directions by all speakers. In yet another example, an external acoustic signal from the left may be played from the same direction as the incoming direction (i.e., the left playback direction), and an external acoustic signal from the right may be played from the same direction as the incoming direction (i.e., the right playback direction), thus maintaining the incoming direction as the playback direction. In another example, an external acoustic signal from the left can be played from the opposite playback direction (right playback direction), and an external acoustic signal from the right can be played from the opposite playback direction (left playback direction), thus mirroring the incoming and playback directions. Therefore, the processing entity can cause each of a plurality of external acoustic signals to be reproduced from the front or rear playback direction relative to the vehicle compartment (or more specifically, relative to the vehicle's longitudinal axis). Alternatively, the processing entity can cause multiple external acoustic signals to be reproduced omnidirectionally by multiple speakers of the acoustic system. In yet another alternative example, the processing entity can cause multiple external acoustic signals to be reproduced from multiple playback directions, including a front or rear playback direction that allows maintaining or mirroring the front and rear incoming directions, and a side playback direction that allows mirroring the incoming direction.

[0052] Each of the above arrangements can create a different immersive effect for vehicle occupants, allowing them to feel part of their environment even in adverse weather conditions. Reproducing external acoustic signals from the playback direction (mirroring, omnidirectional reproduction, and maintaining that direction) can add various audio effects to the overall immersive effect.

[0053] In some examples, processing entity 106 may cause external acoustic signals to be reproduced by multiple loudspeakers 104 of acoustic system 100, either alone or together with audio signals.

[0054] In some additional or alternative examples, generating a processed acoustic signal based on an external acoustic signal includes adding one or more surround effects, such as reverberation derived from the received external sound, wherein the reverberation may include naturally generated (e.g., based on the recorded impulse response of a real room or concert hall) or artificially generated reverberation.

[0055] In some additional or alternative examples, generating a processed acoustic signal based on an external acoustic signal may include providing a gain value to the external acoustic signal, thereby controlling the volume of the reproduced acoustic signal.

[0056] In some additional or alternative examples, generating a processed acoustic signal based on an external acoustic signal may include controlling the bass function of the external acoustic signal by setting the amplification or attenuation of the low-frequency components of the external acoustic signal.

[0057] In some additional or alternative examples, generating a processed acoustic signal based on an external acoustic signal may include controlling the midrange function of the external acoustic signal by setting the amplification or attenuation of the mid-frequency component of the external acoustic signal.

[0058] In some additional or alternative examples, generating a processed acoustic signal based on an external acoustic signal may include controlling the treble function of the external acoustic signal by setting the amplification or attenuation of the low-frequency components of the external acoustic signal.

[0059] In some additional or alternative examples, generating a processed acoustic signal based on an external acoustic signal may include, for example, automatically adjusting the sound pressure level of the external acoustic signal to be reproduced based on the distance to the source of the external acoustic sound or based on the level of internal noise inside the vehicle cabin.

[0060] In some additional or alternative examples, generating processed acoustic signals based on external acoustic signals may include determining and tracking the direction of entry of a selected external acoustic signal, for example, from a plurality of received external acoustic signals, wherein the direction of entry may be relative to the longitudinal axis of the vehicle, and wherein this tracking may allow filtering out the remaining external acoustic signals from the plurality of external acoustic signals, wherein the remaining external acoustic signals may originate from a direction of entry different from the direction of entry of the selected external acoustic signal. The selected signal may be a desired signal, a preferred signal, or a signal of interest (e.g., characteristic signals of an approaching motor-based vehicle). Tracking the direction of the signal of interest (desired or preferred signal) relative to the longitudinal axis of the vehicle (which may be moving and thus changing its orientation relative to the ground) may allow filtering out sounds received by the microphone matrix from other (undesired) directions. Undesired signals can be understood as signals unrelated to the tracking; for example, signals that introduce noise and are unrelated to the safety risk of, for example, an approaching vehicle; or echoes from the main signal, such as the sound of a river, the noise of its own motor and wheels, and the corresponding echoes, etc. The desired signal, preferred signal, or signal of interest can be understood as a signal relevant to tracking, such as a signal associated with the safety risk of an approaching car, or a signal associated with the natural acoustic environment of a river. The relevance of a signal to tracking can be determined based on, for example, the characteristics of its spectral components.

[0061] The sound of an approaching car may be mixed with the sounds of natural environments such as a river, as well as other unwanted sounds such as wind noise, echoes, etc. Each sound can be distinguished by one or more characteristics of the external acoustic signal received from an external microphone (e.g., characteristic spectral components; analyzing the signal strength at discrete time steps, applying machine learning).

[0062] Immersive audio effects (creating a natural sound environment for vehicle occupants) can be combined with receiving safety notifications that inform the driver of the risks associated with an approaching vehicle. The immersive effect can be interrupted while the safety notification is being played by the acoustic system. Alternatively or additionally, the safety notification can be an electronic notification (e.g., a message on a display or control data for controlling the vehicle).

[0063] In some examples, external acoustic signals can be received from one or more external microphones in discrete sequential time periods to determine at least one of the following: distance to an approaching vehicle and speed of the approaching vehicle. In this example, it may be possible to determine the approaching vehicle from one or more sound components of the processed acoustic signal, the sound intensity of the processed acoustic signal in discrete sequential time periods, distance, and / or speed. Therefore, it may be possible to determine the safety risk of an approaching vehicle that at least partially generates the external acoustic signal and generate a notification of the safety risk (or in other words, a safety notification). For example, combustion- or electric motor-based vehicles may produce characteristic audible sounds of the motor that can be distinguished from other components of the external acoustic signal. Other vehicles may also produce characteristic audible sounds that allow them to be distinguished from other components of the external acoustic signal. Thus, the characteristic audible sound may refer to a relevant or desired signal to be tracked, while other undesired signals can be filtered out.

[0064] An approaching vehicle may be a false approach, i.e., moving away. In this case, alternative terms can be used; for example, "approaching vehicle" simply refers to the vehicle generating noise that forms part of the external acoustic signal. Therefore, in an alternative formulation, the method would include receiving external acoustic signals from one or more external microphones over discrete sequential time periods to determine at least one of the following: the distance to the noise-generating vehicle (approaching or moving away), and the speed of the noise-generating vehicle, including the sign of the speed value, i.e., the fact that the vehicle is approaching or moving away. Thus, an approaching vehicle can be distinguished from a moving away vehicle by its speed sign. The external acoustic signal may also include a portion of the signal generated by the approaching vehicle and a portion generated by the moving away vehicle. Those vehicles can be distinguished from the moving away vehicle by their speed sign, which can be determined from the time series (i.e., the external acoustic signals received over discrete sequential time periods).

[0065] In some examples, the notification may include a safety warning signal to be reproduced by the acoustic system 100. The safety warning signal may be another audio sound reproduced (played) by the acoustic system 100. In further examples, the notification may take other forms, such as a warning message displayed on a vehicle's display. The vehicle's display may be part of the vehicle's infotainment system and also assist the driver in the driving environment. Therefore, the acoustic system 100 may be integrated with other systems in the vehicle, such as the infotainment system.

[0066] Acoustic system 100 can be used, for example, in Figure 2 Implemented in the vehicle described in the context of [the previous sentence]. Therefore, Figure 1 The characteristics of the acoustic system 100 in the middle can be compared with those in the middle.Figure 2 The features of the acoustic system 200 described further in the context of which can be interchanged and / or combined.

[0067] Acoustic system 100; 200 can also be referred to as audio acoustic system. Audio system typically refers to the hardware that generates and plays sound (e.g., loudspeakers), while acoustic system can encompass both audio system and vehicle characteristics that affect the sound quality and behavior within the vehicle cabin.

[0068] Figure 2 An acoustic system 200 according to one of several embodiments is schematically shown.

[0069] although Figure 2 As not shown, the acoustic system 200 includes one or more external microphones, such as... Figure 1 Those shown: one or more external microphones 102. Although... Figure 2 As not shown, the acoustic system 200 further includes one or more loudspeakers, such as Figure 1 Those shown: Speaker 104. The microphone and speaker are handled by a processing entity (which...) Figure 2 Not shown in the image, but... Figure 1 and Figure 4 The diagram shows: Processing entity 106; Processing entity 1000) control.

[0070] In some examples, one or more external microphones may be located on one or more outer sides of the vehicle. One or more outer sides of the vehicle may include any of the following: the front side 204, the rear side 206, the left side 210, the right side 208, and / or the top side of the vehicle roof.

[0071] In some examples, one or more external microphones may be located at one or more vehicle components, including any one of the following: a front trim panel 204.2 and / or a rear trim panel 206.4, a front bumper 204.2 and / or a rear bumper 206.4, one or more vehicle mirrors, one or more vehicle fenders 204.4, 206.2 (including any one of the left front panel, right front panel, left rear panel and / or right rear panel) and / or a vehicle roof.

[0072] As can be understood from the above, the possible locations 202 of the external microphone may include one or more locations on the front side 204, rear side 206, right side 208, and left side 210 of the vehicle 216. The front trim / bumper 204.2 of the vehicle 216 may be located at the front side 204. The front fender 204.4 of the vehicle 216 may also be located at the front side 204. Therefore, the external microphone may be located at the front bumper 204.2 and / or the front side panel 204.4. Similarly, the rear bumper 206.2 and the rear side panel 206.4 may be located at the rear side 206.

[0073] As can be understood from the above, one or more external microphones mounted at possible location 202 can cover different directions: front 204; rear 206; both sides (right side 208 and left side 210). It is also possible to mount the external microphones on the car's roof. However, receiving sound from directions other than the top (from the roof) may be more preferable.

[0074] The direction of incoming side and external audio sounds / signals of the vehicle can be understood relative to the lateral axis 212 and longitudinal axis 214 of the vehicle 216 or vehicle compartment 218. Specifically, the incoming directions "from the front" and "from the rear" can be determined relative to the lateral axis 212 that divides the vehicle in half from the front 204 to the rear 206. The incoming directions "from the right" and "from the left" can be determined relative to the longitudinal axis 214 that divides the vehicle in half from the left side 210 to the right side 208.

[0075] Therefore, one or more external microphones can be mounted on different vehicle components. Each component can be configured to receive one or more microphones or microphone matrices located at one or more locations (i.e., possible location 202). Thus, each microphone or microphone matrix will be located outside the vehicle body.

[0076] One or more external microphones can convert external acoustic sounds into electrical signals, such as voltage. Therefore, this voltage can be referred to as an "external acoustic signal." The external acoustic signal can be routed to the audio processing unit of the automotive multimedia unit 108, which has a processing entity 106. At the processing entity 106, the external acoustic signal can be converted into a digital format. This digitally formatted external acoustic signal, to which signal processing methods (such as applying gain, filtering, applying reverberation, etc.) can be applied, can be referred to as a "processed acoustic signal." In other words, converting the analog, raw external acoustic signal into a digital format provides a processed acoustic signal. The processed acoustic signal can then be reproduced by the speakers of the acoustic system, or further processed, for example, mixed into the currently playing audio signal (e.g., music) in an upmixing unit. Therefore, the played music will have additional external acoustic sounds in the background.

[0077] In some examples, the mixing can be based on user preferences, which can be set in the car's multimedia system within the HMI unit. This unit can also be used to define the direction from which external sounds are received. This direction can be referred to as the "incoming direction." For example, the user can be prompted to select one of the following options: "left" and / or "right" and / or "front" and / or "rear." By selecting all these options, the user can hear sounds from all incoming directions (i.e., omnidirectional hearing). The user can also specify how external sounds should be played: (a) what level the external acoustic signal should be compared to the played (main) audio signal (e.g., 25% of the main audio signal level); or (b) the direction in which the external sounds are played within the cabin (playback direction). For example, the following options can be provided to the user via the computer interface / HMI unit: "Keep Direction" (i.e., external sounds will be played from the directions from which they originate), "Mirror All Directions" (sounds from the front will be played from the rear, and vice versa; sounds from the left will be played from the right, and vice versa), "Mirror Side Directions" (front and rear signals will be played from the same direction from which they originate, but left and right sounds will be mirrored), "From Front" (all external sounds will be played from the front, i.e., "Stage"), "From Rear" (all external sounds will be played from the rear), etc. After storing all settings, the HMI unit can interact with the sound control unit, which can translate the settings into corresponding control signals for the digital signal processing (DSP) modules in the audio processing unit. A portion of these modules can be used to implement the upmixing unit. As mentioned above, the (main) audio signal can come from various signal sources available in the car, such as radio, telephone, CD, MP3, Bluetooth audio, USB stick, and other sources. Using the HMI, the user can turn the upmixing on and off, thereby stopping the mixing of sound from external microphones with the (main) audio signal (e.g., by setting the weight value of the external acoustic signal to zero). Furthermore, users can have the possibility of configuring the mixing: which sources should be exempt from mixing features. For example, during a telephone call, some users may not want to keep the signal from an external microphone active. Therefore, users can be offered the option to select one or more audio channels in the car's acoustic system to be exempt from mixing with external acoustic signals. Thus, the user configuration for mixing can include one or more audio channels selected by the user from a channel list, wherein the selected one or more channels can be exempt from mixing with external acoustic signals, and wherein the selected one or more channels can provide audio signals (telephone, radio, music) different from the external acoustic signals. In a preferred implementation, mobile phone, personal assistant, and / or navigation channels can be exempt from mixing, while entertainment, music, and / or radio channels can be mixable.

[0078] External microphone 102 can be a single microphone or a microphone array (microphone matrix, microphone array) consisting of multiple microphones. A microphone array can significantly improve directional selection (determining the direction of incoming sound). This is particularly advantageous for tracking sources of external acoustic sounds relative to the vehicle's longitudinal axis. For example, if vehicle occupants only want to play specific external sounds within the vehicle cabin while filtering out other external sounds, this can be achieved by using a microphone array in conjunction with directional sound processing algorithms for use by processing entity 106 at the audio processing unit. These algorithms are well known in the art. The playback direction of the external sound to be played can be selected via the HMI module.

[0079] Because roads can be curved, the direction of the source of a selected external sound to be played in a vehicle can vary (relative to the vehicle's longitudinal axis). This difficulty can be overcome by using a source tracking algorithm that automatically detects the direction of the selected sound at each moment (e.g., using frequency analysis or machine learning-based algorithms) and provides the updated direction to a directional sound processing algorithm, allowing it to track the desired sound.

[0080] In the case of machine learning algorithms, one implementation could include automatically detecting approaching cars or other audible vehicles that may be visually obscured by obstacles when a car / vehicle is about to turn from a side road or a yard onto a main road, and notifying the driver of the approaching car / vehicle. Specifically, machine learning can improve the differentiation between the sound generated by the approaching car / vehicle and other components of external acoustics. However, machine learning is not essential for the aforementioned automatic detection of approaching cars or other audible vehicles. Detection without machine learning can be achieved simply by tracking external acoustic signals originating from the approaching car or other audible vehicle. Furthermore, signals unrelated to tracking (noise not originating from the approaching car or other audible vehicle) can be filtered out using spatial filtering.

[0081] In some further examples, speed information can be used to notify the driver of an approaching vehicle, for example, at a very low speed (e.g., below 20 km / h-30 km / h). This speed limit can be used to activate the aforementioned algorithm for detecting approaching vehicles. For example, the user can preset this limit using a settings menu in the HMI unit. Therefore, the method will track any audible vehicles approaching at or above the speed limit.

[0082] Algorithms used to detect the speed of an approaching car or other audible vehicle can include spectral analysis of received external acoustic signals, such as noise from a vehicle engine or tires. The higher the speed of the approaching car or other audible vehicle, the higher the dominant tone of the received noise will be. These noises are generally distinguishable on a spectrogram, which can be based on a series of short-time discrete Fourier transforms. To distinguish the acoustic sounds generated by the audible vehicle, the change in noise intensity over time can be used. If it decreases, it can be interpreted that the approaching vehicle emitting or generating the noise is moving away from or towards the vehicle equipped with the acoustic system. If the noise intensity increases over time (in discrete time steps), then the "approaching vehicle" is indeed approaching, not a false approach or departure. Therefore, the "noise" from the audible vehicle engine or tires can be reliably identified and associated with the approaching vehicle.

[0083] To determine the distance to an approaching car or other audible vehicle, two options can be used: direct sound from the approaching car or other vehicle and / or reflected sound (i.e., sound reflected from, for example, a building on the side of the street where the vehicle with the acoustic system intends to turn). The angle at which the intensity of the external acoustic sound is greatest can give the angular position of the approaching vehicle relative to the building reflecting the sound. Distance estimation to the approaching vehicle can be achieved using both direct and reflected sound. The accuracy of determining the direction of the reflected sound can depend on the number of external microphones. The higher the number, the more accurate the direction of the reflected sound is determined, and therefore the more accurate the estimated distance to the approaching vehicle. When estimating and tracking the direction of external acoustic sound, various filtering methods (e.g., Kalman filtering) or pre-trained machine learning algorithms for determining the distance to the approaching vehicle can be used for various applications such as narrow streets, busy streets, streets with or without buildings, wet or dry surfaces, etc.

[0084] Safety-related sounds (such as safety risk notifications from approaching vehicles) may not be of interest to all vehicle occupants and are only relevant to the driver of vehicles equipped with acoustic systems. Therefore, driver-specific (personalized) speakers can be used, delivering safety-related sounds only to the driver without disturbing other occupants. These personalized speakers can be left and right headrest speakers or directional speakers. For example, by using headrest speakers (which can be located near the driver's ears), the distance to other occupants will be considerably greater than the distance from the speaker to the driver's ears, resulting in a significantly lower intensity of the safety-related sounds played through the speakers at the passenger's seating position compared to the driver's listening position (i.e., the position designated for the driver's left and right ears). The passenger's designated seating position can define the passenger's designated listening position (i.e., the center of the passenger seat can define the passenger's listening position).

[0085] In some examples, one or more speakers may include a left headrest speaker and a right headrest speaker. Figure 2 (Not shown in the image). The left headrest speaker can be positioned up to 15 cm from the designated location for the driver's left ear, and the right headrest speaker can be positioned up to 15 cm from the designated location for the driver's right ear, thereby allowing the safety warning signal to be reproduced for the driver, with the intensity of the safety warning sound gradually decreasing, for one or more passengers (…). Figure 2 (Not shown) The sound is below a threshold at one or more listening positions specified. One or more listening positions may be determined by the central position of one or more passenger seats. In this example, the sound reproduced from the headrest speaker can be advantageously blocked by the headrest speaker in the direction of the listening position specified for the passenger, thereby providing a gradually decreasing intensity.

[0086] In some examples, one or more speakers may include directional speakers designated for the driver, such that, due to the directionality of the directional speakers, the intensity of the safety warning sound gradually decreases, falling below a threshold at one or more listening positions designated for one or more passengers. Therefore, in this example, due to the directionality of the speakers, the intensity of the sound reproduced from the directional speakers can gradually decrease towards the listening positions designated for the passengers.

[0087] Using a microphone externally on the exterior of a vehicle, such as that shown in possible location 202, can present several problems. For example, as the car moves, incoming airflow can generate unpleasant noise at the external microphone, which may eventually be played back by the speakers. This effect often occurs because the interaction between the wind and the diaphragm of the external microphone causes it to overload, thus converting this overload into electrical signals similar to noise. To overcome this problem, the external microphone can be covered by a wind shield (covering component). The covering component can be made of a foam material that is transparent to sound waves within the audible range of 20 Hz to 20,000 Hz. While being transparent to the audible range of 20 Hz to 20,000 Hz, the covering component prevents direct interaction between the wind and the diaphragm of the external microphone, thereby (virtually) eliminating noise caused by airflow.

[0088] Therefore, in some examples, one or more external microphones may include a cover comprising foam rubber or a rubber-based material, forming a porous cover to protect against wind-induced interference that causes parasitic audible components in external acoustic signals received from the one or more external microphones. The cover is transparent to external acoustic sounds with frequencies above 20 Hz. The one or more external microphones may be arranged in a microphone matrix for integration into vehicle components.

[0089] In some examples, the use of an external microphone can be activated electronically (e.g., upon request from a user, or automatically during the installation of an acoustic system or computer program product).

[0090] One effect provided by the method, processing entity, acoustic system, computer program product, and their application can involve improved road / driving safety, especially when it may be necessary to hear the sounds of approaching vehicles that could be obscured by obstacles (e.g., billboards, large vehicles, etc.). Another effect involves the fact that when those sounds are reproduced inside the vehicle cabin using an external microphone, it is not necessary to open the windows to hear them. As mentioned above, opening windows can be problematic in adverse weather conditions, with unpleasant odors from outside, dust, etc. Furthermore, opening windows while driving typically increases the noise intensity in the cabin due to gusts of wind caused by vehicle movement. These noise levels are particularly high at high vehicle speeds. Attempts to compensate for noise by increasing the volume of the played sound may not be sufficient and may also result in an unpleasant cabin experience due to the high sound pressure levels at the ears of vehicle occupants. When driving on roads with natural (rocks, trees, etc.) or man-made (buildings, road boundaries, tunnels, etc.) walls alongside, opening both windows allows reflected sound to enter. This reflected sound can be much stronger than the desired sound, thus masking the expected sound and simply creating more noise in the cabin. At high speeds (70 km / h or higher), opening only one window on the side where the desired sound is present does not completely solve the problem. Without opening the second window on the other side of the vehicle, very unpleasant low-frequency noise is generated. This is due to the input and output airflows competing with each other through the same opening (the window), causing airflow interruptions as the air pressure in the vehicle cabin changes at a frequency of 5 Hz–50 Hz. This results in very unpleasant audible noise and sometimes even ear pain for vehicle occupants.

[0091] When visibility of the street is limited, but noise from approaching vehicles can still be heard, this method, processing entity, computer program product, and acoustic system can improve driving safety in various situations. For example, if a driver is turning from a yard or side road onto a main street, and visual visibility of the main street is limited by other parked vehicles (e.g., trucks), or by billboards or trees, only external sounds can signal an approaching vehicle that the driver's car may collide with, unless the driver hears its noise.

[0092] Further effects can be related to the creation of new immersive audio effects. These new immersive effects allow car / vehicle occupants to become part of their environment (city sounds, natural sounds, etc.). These immersive effects can be experienced by vehicle occupants even in adverse weather conditions.

[0093] Using a microphone board and directional signal processing algorithms for external microphones enables directional filtering of external sounds played within the car cabin. This directional filtering can improve immersion and / or tracking of selected sources of external acoustic signals (e.g., an approaching vehicle). Direction tracking algorithms can be applied to selected external sounds to automatically adapt to the direction from which the external sound is being captured.

[0094] Therefore, the method, acoustic system, processing entity, computer program product, and their application enable the automatic detection of approaching vehicles (both motor-based vehicles that produce audible sounds, such as from the motor, and non-motor-based vehicles that produce audible sounds, such as from the tires) by using acoustic signals from an external microphone. This can improve road safety, especially when entering a main road from a side road from an auxiliary road, where optical visibility of traffic is obscured by obstacles. Furthermore, it improves driving safety associated with approaching vehicles, even in adverse weather conditions that would otherwise prevent the driver from opening the window to hear approaching vehicles, and is particularly relevant when vehicles (cars) are turning from a side road or yard onto a main street.

[0095] External sounds related to safety can be played via driver-specific speakers (e.g., via headrest speakers or driver-specific directional speakers). These dedicated speakers can provide safety-related signals to the driver while avoiding disturbing the vehicle's passengers.

[0096] As can be understood from the above, the method, acoustic system, processing entity, computer program product, and their uses can provide an acoustic immersion effect for vehicle occupants by playing natural or urban sounds. These sounds may be direction-specific or direction-independent. Direction-specific sounds can be tracked to filter out and prevent the playback of other external sounds (e.g., unpleasant noises or irrelevant sounds).

[0097] In some examples, additional immersive effects can be achieved by applying spatial sound processing algorithms to external acoustic signals (e.g., adding reverberation) or by passing external sounds through a finite impulse response filter, the coefficients of which can be taken from the impulse response of a reference acoustic hall (e.g., a world-class concert hall) or derived from head-related impulse responses corresponding to a specific direction relative to the human head, and so on.

[0098] Figure 3 A method 300 according to one of the multiple embodiments is illustrated schematically.

[0099] Method 300 is performed at a processing entity within a vehicle compartment. Method 300 includes steps S302, S304, and S306. Step S302 involves receiving an external acoustic signal associated with external acoustic sounds originating outside the vehicle compartment from one or more external microphones located outside the vehicle compartment of the acoustic system. Step S304 involves generating a processed acoustic signal based on the external acoustic signal. Step S306 involves causing the processed acoustic signal to be reproduced by the acoustic system inside the vehicle compartment.

[0100] The processing entity configured to perform method steps S302, S304, and S306 can be as follows: Figure 1 and Figure 4 The implementation described in the context is as follows: processing entity 106; processing entity 1000. An acoustic system including processing entities can be implemented as follows: Figure 1 and Figure 2 The implementation described in the context of: acoustic system 100; acoustic system 200. Therefore, in Figures 1 to 4 The features described in the context are composable / interchangeable.

[0101] Figure 4 The processing entity 1000 according to one of the multiple implementations is illustrated schematically.

[0102] More specifically, Figure 4 The steps of the above method can be shown (i.e., Figure 3 A schematic architecture view of entity 1000 (steps S302, S304, S306) in the diagram. Entity 1000 can be incorporated into any module (e.g., Figure 2The entity 1000 includes an interface 1100 provided for transmitting data to other entities via a transmitter or controlling the analysis of data by other entities, and for receiving data from other entities using a receiver. The interface may be referred to as an input / output interface (I / O). Interface 1100 is particularly qualified to receive external acoustic signals (e.g., voltage signals from an external microphone). Interface 1100 is further qualified to transmit data (e.g., processed acoustic signals; notifications) to another entity (e.g., memory; acoustic systems 100, 200; infotainment system). Entity 1000 further includes a processing unit 1200 responsible for the operation of entity 1000. Processing unit 1200 includes at least one computer processor and can execute instructions stored in memory 1300, wherein the memory may include read-only memory, random access memory, mass storage device, hard disk, etc. The memory may also include suitable program code (computer-readable instructions) to be executed by processing unit 1200 to readily implement the aforementioned functionality of the entity. Entity 1000 can be implemented on a single node or distributed across several nodes or entities in a cloud implementation. Each node or entity may include a computer processor, computer memory, computer interface, or can be implemented on a cloud platform.

[0103] For example, a processing unit 1200 may be part of an upmixing unit, such as Figure 2 As shown. Another processing unit 1200 may be part of an HMI unit. Another processing unit 1200 may be part of an audio processing unit, and so on. All processing units 1200 may be communicatively coupled to form a (distributed) processing entity 1000. Each processing unit 1200 may receive data from the cloud or another entity and / or send data to the cloud or another entity. For example, a processing unit 1200 may be part of an automotive multimedia unit 108, which may be communicatively coupled to a mobile phone processing unit 1200. The computer program product may be stored in a memory 1300, which may be an electronic storage medium or the cloud.

[0104] Entity 1000 can receive data via interface 1100. The received data can be user input.

[0105] In one aspect, processing entity 1000 is used in acoustic system 100. Processing entity 1000 is adapted to perform method steps S302, S304, and S306. Processing entity 1000 includes at least one computer processor.

[0106] In summary, a method is provided for implementation at a processing entity within a vehicle cabin. The method includes the steps of: receiving external acoustic signals associated with external acoustic sounds originating outside the vehicle cabin from one or more external microphones located outside the vehicle cabin of an acoustic system; generating a processed acoustic signal based on the external acoustic signals; and causing the acoustic system to reproduce the processed acoustic signal inside the vehicle cabin. This method allows for the creation of an immersive audio effect within the vehicle cabin, immersing vehicle occupants in a natural acoustic environment while simultaneously allowing for an atmosphere within the vehicle cabin that is isolated from the atmosphere of the natural acoustic environment. Additionally, an acoustic system, processing entity, and computer program product for implementing the steps of this method are provided, along with their uses.

[0107] Based on the above, a general conclusion can be drawn, which can be summarized by the following example.

[0108] In some examples, generating a processed acoustic signal may include determining output conditions for the reproduction of the processed acoustic signal inside a vehicle cabin, wherein the processed acoustic signal is to be reproduced based on the output conditions.

[0109] In some examples, output conditions may include one or more playback directions that determine from which the processed signal is reproduced in the vehicle cabin. In some examples, generating the processed acoustic signal may include: determining an incoming direction indicating the direction from which the external acoustic signal originates relative to the vehicle cabin; and generating a directional acoustic signal from a first playback direction identical to the determined direction, which, when output by the acoustic system, gives the impression of originating from the incoming direction; or alternatively, generating a directional acoustic signal from a second playback direction different from the determined direction, wherein the second playback direction is any of the following: a direction opposite to the incoming direction to give the impression of mirroring the external acoustic sound; a forward playback direction to give the impression of a stage effect; or a rear playback direction relative to the vehicle cabin.

[0110] In some examples, when multiple external acoustic signals are available, generating the processed acoustic signal may include determining multiple incoming directions, each indicating which direction one of the multiple external acoustic signals originates relative to the vehicle compartment. Output conditions may include one or more playback directions. The processing entity may cause each of the multiple external acoustic signals to be reproduced from a front playback direction or a rear playback direction relative to the vehicle compartment. Alternatively or additionally, the processing entity may cause the multiple external acoustic signals to be reproduced omnidirectionally by multiple speakers of the acoustic system. Alternatively or additionally, the processing entity may cause the multiple external acoustic signals to be reproduced from multiple playback directions, including a front playback direction or a rear playback direction that allows maintaining or mirroring the front and rear incoming directions, and a side playback direction (left or right) that allows mirroring the incoming direction (left or right).

[0111] In some examples, generating the processed signal may include mixing an external acoustic signal with an audio signal generated from an audio source and played by an acoustic system in the vehicle cabin.

[0112] In some examples, generating a processed signal may include changing the mixing ratio of the audio signal mixed in the processed acoustic signal with the external acoustic signal. When the mixing ratio is 100%, generating a processed signal includes stopping the output of the audio signal and replacing the audio signal with the processed acoustic signal. When the mixing ratio is 0%, generating a processed signal includes stopping the mixing and (temporarily) replacing the external acoustic signal with the audio signal in the processed acoustic signal. When the mixing ratio is higher than 0% and less than 100%, generating a processed signal includes jointly reproducing the external acoustic signal and the audio signal according to the mixing ratio with weights for each signal. In some examples, different mixing ratios may be applied to different speaker channels, and / or depending on one or more acoustic conditions inside the vehicle cabin (e.g., noise level inside the vehicle cabin, cabin noise level), the mixing ratio may vary over time.

[0113] In some examples, generating the processed signal may include determining user input from a user of the acoustic system, determining how external acoustic sounds should be processed, and generating the processed acoustic signal based on the user input. Determining the user input may include any of the following: determining one or more playback directions; determining one or more mixing conditions to mix the audio signal with the external acoustic signal or to stop mixing; and / or determining deactivation criteria for deactivating the directional reproduction of the processed acoustic signal, thereby resulting in omnidirectional reproduction.

[0114] In some examples, generating the processed signal may include: selecting a first geographic direction, where a first component of the (desired) external acoustic signal originating from that first geographic direction will be received by one or more external microphones to filter out a second component of the (undesired) external acoustic signal originating from a geographic direction other than the first geographic direction; and tracking the first direction of the first component, regardless of the vehicle's geographic orientation on the road surface. For example, the first component may be a desired sound, such as an approaching vehicle, a river, etc., while the second component may be any other component irrelevant to tracking, for listening to pleasant sounds, and for safety (e.g., undesired noise, wind from the north, etc.). The geographic direction may be north, south, west, or east. The vehicle's geographic orientation may be determined relative to these geographic directions (i.e., north, south, west, and east) (e.g., by one or more angles between the vehicle's longitudinal / lateral axis and the axes defining the geographic direction (i.e., north, south, west, and east axes)).

[0115] In some examples, generating a processed acoustic signal based on an external acoustic signal can include any of the following:

[0116] (1) Add one or more surround effects, which include reverberation derived from received external sounds, wherein the reverberation includes natural or artificially generated reverberation;

[0117] (2) Provide a gain value for the external acoustic signal, thereby controlling the volume of the reproduced acoustic signal;

[0118] (3) Control the bass function of the external acoustic signal by setting the amplification or attenuation of the low-frequency component of the external acoustic signal.

[0119] (4) The mid-range function of the external acoustic signal is controlled by setting the amplification or attenuation of the mid-frequency component of the external acoustic signal;

[0120] (5) Control the high-frequency function of the external acoustic signal by setting the amplification or attenuation of the low-frequency component of the external acoustic signal.

[0121] (6) The sound pressure level of the external acoustic signal to be reproduced is automatically adjusted based on the distance from the source of the external acoustic sound, or based on the level of the internal noise inside the vehicle cabin, or based on the level of the audio signal to be played and mixed with the external acoustic sound.

[0122] and / or

[0123] (7) A filtering algorithm used to reduce or increase the sound level at certain frequencies to make external acoustic sounds more pleasing to the listener; and / or

[0124] (8) Determine and track the incoming direction of a selected external acoustic signal chosen from a plurality of received external acoustic signals, wherein the incoming direction is relative to the longitudinal axis of the vehicle, and wherein tracking allows filtering out the remaining external acoustic signals from the plurality of external acoustic signals, wherein the remaining external acoustic signals originate from an incoming direction different from the incoming direction of the selected external acoustic signal. The level of the played audio signal may refer to the amplifier gain, thereby determining the volume of the audio signal.

[0125] In some examples, external acoustic signals may be received from one or more external microphones in discrete sequential time periods to determine at least one of the following: the distance to an approaching vehicle and the speed of the approaching vehicle. The approaching vehicle may be determined by a processing entity from one or more sound components of the processed acoustic signal, the sound intensity of the processed acoustic signal in discrete sequential time periods, the distance, and / or the speed. Further, the safety risk of the approaching vehicle, which at least partially generates the external acoustic signal, may be determined by the processing entity. The processing entity may then generate a notification of the safety risk.

[0126] In some examples, the notification may include a safety warning signal to be reproduced by the acoustic system.

[0127] In some examples, one or more external microphones of the acoustic system may be located on one or more outer sides of the vehicle, including: the front side of the vehicle, the rear side of the vehicle, the left side of the vehicle, the right side of the vehicle, and / or the top side of the vehicle roof. In some examples, one or more external microphones of the acoustic system may be located at one or more vehicle components, including: the front and / or rear trim panels, the front and / or rear bumpers, one or more vehicle mirrors, one or more vehicle fenders (including any one of the left front, right front, left rear, and / or right rear panels) and / or the vehicle roof.

[0128] In some examples, one or more speakers in the acoustic system may include a left headrest speaker and a right headrest speaker, wherein the left headrest speaker may be located at a distance of up to 15 cm from a position designated for the driver's left ear, and the right headrest speaker may be located at a distance of up to 15 cm from a position designated for the driver's right ear, thereby allowing the safety warning signal to be reproduced for the driver, such that the intensity of the safety warning sound gradually decreases and falls below a threshold at one or more listening positions designated for one or more passengers, wherein the one or more listening positions may be determined by the central position of one or more passenger seats. Alternatively, one or more speakers may include directional speakers designated for the driver, such that due to the directionality of the directional speakers, the intensity of the safety warning sound gradually decreases and falls below a threshold at one or more listening positions designated for one or more passengers.

[0129] In some examples, one or more external microphones of the acoustic system may include a cover comprising foam rubber or a rubber-based material, forming a porous cover to protect against wind-induced interference that causes parasitic audible components in the external acoustic signals received from the one or more external microphones. The cover may be made transparent to external acoustic sounds with frequencies above 20 Hz (e.g., due to the acoustic properties of the material). The one or more external microphones may be arranged in a microphone matrix to be integrated into vehicle components.

[0130] The embodiments have been described for illustrative and descriptive purposes. Suitable modifications and variations of the embodiments can be performed based on the foregoing description or can be obtained based on practical methods. For example, unless otherwise indicated, one or more of the described methods can be performed by suitable means and / or combinations of means. The methods can be performed by executing stored instructions using one or more logical means (e.g., a processor) in conjunction with one or more additional hardware elements (such as storage devices, memories, hardware network interfaces / antennas, switches, actuators, clock circuits, etc.). In addition to the order described herein, the described methods and associated actions can be performed in parallel and / or simultaneously in various sequences. The described systems are exemplary in nature and may include additional elements and / or omissions. The subject matter of this disclosure includes all novel and non-obvious combinations and sub-combinations of the various systems and configurations disclosed, as well as other features, functions, and / or properties.

[0131] As used herein, elements or steps listed in the singular and preceded by the word "a" or "an" should be understood as not excluding multiple said elements or steps, unless such exclusion is specified (e.g., a processor does not exclude multiple processors). Furthermore, references to "an embodiment" or "an example" in this disclosure are not intended to exclude the existence of additional embodiments that also include the listed features. The terms "first," "second," and "third," etc., are used merely as labels and are not intended to impose numerical requirements or a specific order of position on their objects. The appended claims specifically point to subject matter from the foregoing disclosure that is considered novel and not obvious.

[0132] Figure Labels

[0133] 100: Acoustic systems used in vehicles;

[0134] 102: External microphone;

[0135] 104: Speaker;

[0136] 106: Processing entities;

[0137] 108: Car multimedia unit;

[0138] 110: Vehicles;

[0139] 112: Sound source;

[0140] 114: Human-Computer Interaction (HMI) Unit;

[0141] 116: Voice control unit;

[0142] 200: Acoustic systems used in vehicles;

[0143] 202: Possible location (positioning) of the external microphone;

[0144] 204: Front side;

[0145] 204.2: Front trim panel / bumper;

[0146] 204.4: Front fender;

[0147] 206: Rear side;

[0148] 206.2: Rear trim panel / bumper;

[0149] 206.4: Rear fender;

[0150] 208: Right side;

[0151] 210: Left side;

[0152] 212: Horizontal axis;

[0153] 214: Longitudinal axis;

[0154] 216: Vehicles;

[0155] 218: Vehicle compartment;

[0156] 300: Method;

[0157] S302: Receive external acoustic signals associated with external acoustic sounds originating from outside the vehicle compartment from one or more external microphones located outside the vehicle compartment of the acoustic system;

[0158] S304: Generates processed acoustic signals based on external acoustic signals;

[0159] S306: Causes the acoustic system to reproduce the processed acoustic signal inside the vehicle cabin;

[0160] 1000: Processing entities;

[0161] 1100: I / O interface (computer interface; human-computer interaction interface);

[0162] 1200: Processing unit (computer processor);

[0163] 1300: Memory (electronic storage medium).

Claims

1. A method (300) implemented at a processing entity (1000) in a vehicle compartment, the method comprising the following steps: (S302) Receive (from one or more external microphones located outside the vehicle cabin of the acoustic system) external acoustic signals associated with external acoustic sounds originating from outside the vehicle cabin; A processed acoustic signal is generated based on the external acoustic signal (S304); as well as This causes (S306) the processed acoustic signal to be reproduced by the acoustic system inside the vehicle cabin.

2. The method as described in claim 1, wherein, Generating the processed acoustic signal includes determining output conditions for the reproduction of the processed acoustic signal inside the vehicle cabin, wherein the processed acoustic signal is to be reproduced based on the output conditions.

3. The method as described in claim 2, wherein, The output conditions include one or more playback directions, which determine from which direction the processed signal is reproduced in the vehicle cabin, and wherein generating the processed acoustic signal includes: determining an incoming direction, the incoming direction indicating from which direction the external acoustic signal originates relative to the vehicle cabin; and - Generate a directional acoustic signal from a first playback direction that is the same as the determined direction, the directional acoustic signal giving the impression as if it were coming from the incoming direction when output by the acoustic system, or - Generate a directional acoustic signal from a second playback direction different from the determined direction, wherein the second playback direction is any of the following: a direction opposite to the incoming direction to give the impression of mirroring the external acoustic sound; a forward playback direction to give the impression of a stage effect; or a rear playback direction relative to the vehicle cabin.

4. The method of claim 2, wherein, When multiple external acoustic signals are available, generating the processed acoustic signal includes determining multiple incoming directions, each incoming direction indicating the direction from which one of the multiple external acoustic signals originates relative to the vehicle cabin, and wherein the output conditions include one or more playback directions; and - This causes each of the plurality of external acoustic signals to be reproduced from either the front or rear playback direction relative to the vehicle compartment. - This causes the multiple external acoustic signals to be reproduced omnidirectionally by the multiple loudspeakers of the acoustic system. - Or cause multiple external acoustic signals to be reproduced from multiple playback directions, including a front playback direction or a rear playback direction that allows maintaining or mirroring the front and rear input directions, and a side playback direction that allows mirroring the input directions.

5. The method according to any one of claims 2 to 4, wherein, Generating the processed signal involves mixing the external acoustic signal with an audio signal generated from an audio source and played by the acoustic system in the vehicle cabin.

6. The method as described in claim 5, in, Generating the processed signal includes changing the mixing ratio of the audio signal mixed in the processed acoustic signal with the external acoustic signal. When the mixing ratio is 100%, generating the processed signal includes stopping the output of the audio signal and replacing the audio signal with the processed acoustic signal. When the mixing ratio is 0%, generating the processed signal includes stopping the mixing and temporarily replacing the external acoustic signal with the audio signal in the processed acoustic signal. When the mixing ratio is greater than 0% and less than 100%, generating the processed signal involves jointly reproducing the external acoustic signal and the audio signal according to the mixing ratio and with weights for each signal. Different mixing ratios can be applied to different speaker channels, and / or The mixing ratio can vary over time, depending on one or more acoustic conditions inside the vehicle cabin.

7. The method according to any one of claims 3 to 6, wherein, Generating the processed signal includes determining user input from a user of the acoustic system, determining how the external acoustic sound should be processed based on the user input, and generating the processed acoustic signal based on the user input. Optionally, the determination of the user input includes any one of the following: Determine one or more playback directions; Determine one or more mixing conditions to mix the audio signal with the external acoustic signal or to stop mixing; Determine the deactivation criteria for deactivating the directional reproduction of the processed acoustic signal, thereby resulting in omnidirectional reproduction; and / or Selecting a first geographic direction, a first component of the external acoustic signal from the first geographic direction is received by the one or more external microphones to filter out a second component of the external acoustic signal, wherein the second component comes from a geographic direction other than the first geographic direction; and tracking the first direction of the first component, regardless of the geographic orientation of the vehicle on the road surface.

8. The method as described in any of the preceding claims, wherein, The processed acoustic signal generated based on the external acoustic signal (S304) includes any one of the following: Add one or more surround effects, the one or more surround effects including reverb derived from received external sounds, wherein the reverb includes natural or artificially generated reverb; Provide a gain value for the external acoustic signal, thereby controlling the volume of the reproduced acoustic signal; The bass function of the external acoustic signal is controlled by setting the amplification or attenuation of the low-frequency component of the external acoustic signal. The mid-range function of the external acoustic signal is controlled by setting the amplification or attenuation of the mid-frequency component of the external acoustic signal. The treble function of the external acoustic signal is controlled by setting the amplification or attenuation of the low-frequency component; and / or The sound pressure level of the external acoustic signal to be reproduced is automatically adjusted based on the distance to the source of the external acoustic sound, or based on the level of the internal noise level inside the vehicle cabin, or based on the level of the audio signal to be mixed with the external acoustic sound; and / or The method determines and tracks the incoming direction of a selected external acoustic signal chosen from a plurality of received external acoustic signals, wherein the incoming direction is relative to the longitudinal axis of the vehicle, and wherein the tracking allows filtering out the remaining external acoustic signals from the plurality of external acoustic signals, wherein the remaining external acoustic signals originate from an incoming direction different from the incoming direction of the selected external acoustic signal.

9. The method of any one of claims 2 to 8, wherein the method further comprises: The external acoustic signals are received from the one or more external microphones during discrete sequential time periods to determine at least one of the following: the distance to an approaching vehicle and the speed of the approaching vehicle. The approaching vehicle is determined from one or more sound components of the processed acoustic signal, the sound intensity from the processed acoustic signal during the discrete sequential time periods, the distance, and / or the speed. To determine the safety risk posed by an approaching vehicle that at least partially generates the external acoustic signal. And generate a notification of the security risk, wherein, optionally, the notification includes a security warning signal to be reproduced by the acoustic system.

10. A processing entity (1000) comprising at least one computer processor, said at least one computer processor being configured to perform the steps of the method as described in any of the preceding claims.

11. A computer program product comprising computer-readable instructions stored on an electronic storage medium (1300), the computer-readable instructions causing the processing entity (1000) to perform the steps of the method according to any one of claims 1 to 9 when executed on a processing entity (1000).

12. An acoustic system (100) for a vehicle, the acoustic system comprising one or more external microphones (102) and one or more speakers (104) controlled by a processing entity (1000) as claimed in claim 10.

13. The acoustic system as described in claim 12, in, The one or more external microphones are located on one or more sides of the vehicle, the one or more sides including: Front side of the vehicle, rear side of the vehicle, left side of the vehicle, right side of the vehicle and / or top side of the vehicle roof; and / or Wherein, the one or more external microphones are located at one or more vehicle components, and the one or more vehicle components include: The front trim panel and / or rear trim panel of the vehicle, The vehicle's front bumper and / or rear bumper, One or more vehicle rearview mirrors, One or more vehicle fenders, the one or more vehicle fenders including any one of a front left fender, a front right fender, a rear left fender and / or a rear right fender, and / or Vehicle roof.

14. The acoustic system as described in any one of claims 12 to 13, in, The one or more speakers include a left headrest speaker and a right headrest speaker, wherein the left headrest speaker is located at a distance of up to 15 cm from a position designated for the driver's left ear, and the right headrest speaker is located at a distance of up to 15 cm from a position designated for the driver's right ear, thereby allowing the safety warning signal to be reproduced for the driver, such that the intensity of the safety warning sound gradually decreases, falling below a threshold at one or more listening positions designated for one or more passengers, wherein the one or more listening positions are determined by the center position of one or more passenger seats; or The one or more speakers include directional speakers designated for the driver, such that due to the directionality of the directional speakers, the intensity of the safety warning sound gradually decreases and falls below the threshold at the one or more listening positions designated for the one or more passengers.

15. The acoustic system as described in any one of claims 12 to 14, in, The one or more external microphones include a cover component comprising foam rubber or a rubber-based material, forming a porous cover component for protection against wind-induced interference that causes parasitic audible components in the external acoustic signals received from the one or more external microphones. The cover component is transparent to external acoustic sounds having frequencies higher than 20 Hz, and the one or more external microphones are arranged in a microphone matrix for integration into vehicle components.

16. An application of the method as described in any one of claims 1 to 9, the processing entity as described in claim 10, the computer program product as described in claim 11, and / or the acoustic system as described in any one of claims 12 to 15, for producing an immersive audio effect in a vehicle cabin, immersing the occupants of the vehicle in a natural acoustic environment while allowing an atmosphere inside the vehicle cabin to be isolated from the atmosphere of the natural acoustic environment.

17. A vehicle comprising an acoustic system as described in any one of claims 12 to 15.

Images