Method for outputting an adaptive sound pattern in a vehicle interior, and vehicle

Abstract

The invention relates to a method for outputting an adaptive sound pattern (3) in a vehicle interior depending on the driving situation, wherein a vehicle-internal computing unit impresses an effect, which is dependent on the driving situation, on the sound tracks (4) of a song (5) divided into sound tracks (4) and actuates a sound system of the vehicle in order to output the correspondingly manipulated sound tracks (4) via vehicle-internal loudspeakers. The method according to the invention is characterised in that - the computing unit extracts from each sound track (4) a sound track section that is characteristic of the sound track, wherein using granular synthesis, the computing unit generates a basic sound track section (7.1) having a first sampling rate and a first duration and at least one layer sound track section (7.2) having a second sampling rate and a second duration, wherein the first sampling rate is lower than the second sampling rate and the first duration is greater than the second duration; and - when the vehicle is started, the computing unit causes the basic sound track section (7-1) to be output and causes a layer soundtrack section (7.2) to be output for each driving event (1) on the basis of the driving event when driving events (1) occur.

Description

[0001] Method for outputting an adaptive soundscape in a vehicle interior and vehicle

[0002] The invention relates to a method for outputting an adaptive soundscape in a vehicle interior according to the type defined in more detail in the preamble of claim 1, and to a vehicle with a computing unit and a sound system for carrying out the method.

[0003] The noise level inside a vehicle affects the well-being and attention of its occupants. For example, conversations or noise can be distracting for the driver, while music played in the vehicle can be pleasant or calming. Some people enjoy the sounds produced by the vehicle itself, such as squealing tires or a crisp engine sound. In particular, the generation of such sounds during dynamic driving can enhance or even increase the driver's enjoyment of the vehicle. This can influence how the driver interacts with the vehicle.

[0004] Besides purely objective facts, such as engine power, fuel consumption, trunk volume, and the like, emotions can also influence people to buy a particular vehicle. Therefore, it is in a car manufacturer's interest to offer potential buyers the most immersive driving experience possible, one that is accompanied by a high level of driving pleasure. The goal is to make the driving dynamics accessible to the driver through all sensory channels.

[0005] WO 2023 / 011911 A1 discloses a sound experience generator usable in a vehicle. The sound experience generator comprises an application running in the vehicle for reading vehicle data, such as sensor readings or information transmitted via the vehicle's fieldbus. The application running in the vehicle can process the vehicle data and forward it to an application running on an external user device. Here, the processed vehicle data is normalized and fed into a sound synthesizer. The sound synthesizer contains several different orchestrated acoustic scenes, each comprising different sets of instruments, noises, effects, and loops. These orchestrated acoustic scenes can be selected as input signals for playback in the vehicle.Individual channels of the instruments, sounds, effects, and loops contained in the respective orchestrated acoustic scenes are then manipulated by the sound synthesizer based on the vehicle data. This allows new soundscapes to be generated adaptively depending on the driving situation. The corresponding soundscapes are then output via the vehicle's audio system.

[0006] The generation and output of adaptive soundscapes depending on the driving situation is also known, for example, from US 2023 / 0410774 A1 or US 2022 / 0019402 A1. Various vehicle parameters can be used to adjust the soundscape to be output in the vehicle interior, such as steering angle, speed, acceleration, the vehicle's location (e.g., in the form of a GPS position), and the like.

[0007] Furthermore, DE 10 2013 011 144 A1 discloses a control unit for electric vehicles that records driving parameters and generates synthetic sounds from them to simulate a driving noise.

[0008] Furthermore, US patent 2023 / 0139893 A1 discloses a system for synthesizing engine sounds using granular synthesis. The pitch is varied depending on the engine speed.

[0009] The solutions described above have several drawbacks. For example, the song or underlying audio file used to create the soundscape must possess certain characteristics to be suitable for adaptive soundscapes. This can necessitate extensive manual preprocessing, resulting in significant user effort and limiting the number of songs that can be used for adaptive soundscapes. Often, the sound tracks within an audio file or song are simply faded in and out depending on vehicle parameters. This prevents the soundscape from adapting to the driving dynamics, which is essential for the driver to experience true immersion in the driving experience. Furthermore, such generated soundscapes are often perceived as artificial or unnatural.Therefore, there is a need to further improve corresponding methods for outputting adaptive soundscapes in vehicle interiors.

[0010] Furthermore, the application of effects to sounds is known from the general state of the art. For example, entirely new noises and sounds can be created using synthesizers, and sound effects can be applied to existing noises, such as distortion, frequency shifting, delay, echo, reverberation, and the like.

[0011] The use of existing sound or music recordings in a new context is also known as "sampling" in the fields of acoustics and music. With the help of so-called granular synthesis, it is also possible to create, or simulate, a continuous sound from relatively short sound elements. This continuous sound actually consists of many individual pieces or fragments. These pieces are also called "grains." These grains are digital sound fragments with a length or duration on the order of a few milliseconds. Granular synthesis is often used to resynthesize previously sampled audio material. Here, the recorded sound is analyzed and broken down into these grains, which are then reassembled. The individual sound fragments or...The artificially simulated continuous sound can be modulated, for example by so-called time-stretching or pitch-shifting.

[0012] The present invention is based on the objective of providing a method for outputting an adaptive soundscape in a vehicle interior that is further improved compared to the prior art. According to the invention, this objective is achieved by a method for outputting an adaptive soundscape in a vehicle interior depending on the driving situation, with the features of claim 1. Advantageous embodiments and further developments, as well as a vehicle for carrying out the method, are described in the dependent claims.

[0013] A generic method for outputting an adaptive soundscape in a vehicle interior depending on the driving situation, wherein a vehicle-internal computing unit imprints a driving-dependent effect on the sound tracks of a song broken down into sound tracks and controls a sound system of the vehicle to output the correspondingly manipulated sound tracks via vehicle-internal loudspeakers, is further developed according to the invention in that

[0014] - the processing unit extracts a sound-track-characteristic sound track segment from each sound track, wherein the processing unit, using granular synthesis, generates a basic sound track segment with a first sampling rate and a first duration and at least one layered sound track segment with a second sampling rate and a second duration, wherein the first sampling rate is smaller than the second sampling rate and the first duration is larger than the second duration; and

[0015] - the processing unit, when the vehicle is started, causes the output of the basic sound track segment and, depending on the driving situation, causes the output of a layered sound track segment for each driving event when driving events occur.

[0016] The process involves digitally processing a song, for example, in the form of a computer-readable audio file, to extract its constituent sound tracks. A song thus represents a composition of these various sound tracks. For instance, each sound track can be assigned its own instrument, noise, or vocals. A song already broken down into sound tracks can be provided by a music provider, also known as a content provider. It is also possible to process a song that has not yet been broken down into individual sound tracks using an algorithm that extracts the respective sound tracks from the song. Such an algorithm can be executed within the vehicle itself, for example, on the vehicle's own processing unit, or on a central computing device connected to the vehicle via a communication link, such as a cloud server.The algorithm can be based on artificial intelligence; in particular, machine learning methods can be applied.

[0017] For each audio track, the processing unit extracts at least one audio track segment. This means that for all audio tracks contained in the song, both a basic audio track segment and a layered audio track segment are generated, or alternatively, the basic audio track segment is generated from one audio track and at least one layered audio track segment is generated from another audio track. Further layered audio track segments can be extracted from other audio tracks. The basic audio track segment and the at least one layered audio track segment differ in their sampling rate and duration. Thus, the basic audio track segment and the at least one layered audio track segment sound different, even if the same audio track is used to generate the respective audio track segments.However, it is preferred that different sound tracks are used to form the basic sound track section and the layered sound track section(s) in order to create an even more diverse sound experience.

[0018] Granular synthesis makes it possible to extract samples—that is, the aforementioned sound track segments—from the respective sound tracks and seamlessly loop them together, thus providing a continuous sound experience. In the following, we will only refer to sound track segments, specifically basic sound track segments and layered sound track segments, with the understanding that these segments are played back repeatedly, i.e., in the manner of the loop described previously.

[0019] Due to its longer duration and lower sampling rate, the base sound track segment sounds calmer than the respective layered sound track segments. Therefore, by combining the base sound track segment and the layered sound track segments, a soundscape can be built up layer by layer in the vehicle interior, depending on the driving situation. Thus, when the vehicle is started, only the base sound track segment, or the seamlessly strung repetitions of the base sound track segment, are output. "Starting the vehicle" refers to the activation of the vehicle's electronics or the start of the vehicle's engine. Using the base sound track segment, a soundscape emitted by the engine or the vehicle's electronics can therefore be simulated.During operation, driving events occur, such as acceleration, braking, and steering maneuvers. These will be discussed in more detail below. Such driving events cause changes in the soundscape perceived by vehicle occupants. For example, when accelerating in a vehicle with an internal combustion engine, the noise emitted by the vehicle's drivetrain increases. With increasing engine speed, the perceived frequency also rises. The same applies to vehicles with electric motors, including battery-electric vehicles. As the speed of travel increases, the perceived noisescape also becomes more complex. In particular, wind noise and / or tire noise may be noticeable.By building up the adaptive soundscape layer by layer, through the additional output of sound track segments depending on the driving events taking place, corresponding effects that contribute to an increased complexity of the noise perceived in the vehicle interior can be simulated. Adding appropriate sound layers in correlation with the driving events ensures that a psychoacoustic link is established between the adaptive soundscape and the driving experience. This results in a particularly deep immersion of the driver in both the soundscape and the driving experience. The driving dynamics can thus be conveyed to the driver via an acoustic transmission channel.

[0020] The applicant recognized that deep immersion is made possible by the layered construction and dismantling of the adaptive soundscape. Accordingly, the soundscape can be formed by at least two sound layers: a first sound layer based on the repeated output of the base sound track segment, and a second sound layer based on the repeated output of the layered sound track segment. According to the invention, this layered construction can be repeated any number of times, allowing more and more sound layers to be incorporated into the adaptive soundscape. These additional sound layers are based on the repeated output of further layered sound track segments. These further layered sound track segments can be based on or have the same sampling rate and duration as the first layered sound track segment.Several or all layered sound track segments can thus be generated through granular synthesis, based on the second sampling rate and the second duration. It is also possible to further adjust the sampling rate and duration to generate additional layered sound track segments. As the soundscape builds up, it becomes increasingly extensive and turbulent. Accordingly, the sampling rate is increased and the duration is decreased.

[0021] Since the respective sound tracks are already altered in the sound through the application of granular synthesis, this can be understood as imprinting effects onto the respective sound tracks.

[0022] With the aid of the method according to the invention, it is no longer necessary to manually preprocess the corresponding songs or the underlying audio files. It is only necessary to extract the sound tracks from a corresponding song, if this has not already been done. Thus, it is possible to use all available songs to create corresponding adaptive soundscapes. The corresponding songs or audio files can be carried in a computer-readable storage medium to which the vehicle's internal processing unit has at least read access. The computer-readable storage medium can be integrated into the processing unit or be external, for example, in the form of a USB stick, a flash memory card, a mobile device connected to the vehicle via a wired or wireless connection, and the like.The corresponding songs can also be streamed into the vehicle via an internet service, allowing access to relatively large music libraries. For this purpose, the vehicle can be equipped with a telecommunications unit, enabling the computer to connect to the internet via mobile network or Wi-Fi.

[0023] The songs used to create the adaptive soundscape can be selected by the driver according to their personal preferences. Alternatively, a curated selection of songs can be provided. This curation can be done manually or with the help of an algorithm, for example, based on the driver's musical preferences or historical listening habits. The adaptive soundscape emitted in the vehicle interior using the inventive method sounds natural and therefore not artificial. This reduces stress, increases concentration, and enhances driving pleasure. This, in turn, improves road safety. Furthermore, each journey in the vehicle is experienced as unique. The content of the soundscapes is modified from the original depending on the driving events, creating an exciting driving experience.Thus, the use of the vehicle remains particularly memorable for the person driving the vehicle.

[0024] An advantageous further development of the method according to the invention provides that the processing unit stops the output of the basic sound track segment as soon as the output of at least one layered sound track segment begins. As already described, the layered sound track segments can be superimposed on the basic sound track segment to create the adaptive soundscape. Depending on the driving situation, it is then advantageously possible to also mute at least the basic sound track segment. Since the basic sound track segment is intended to simulate the activated on-board electronics or the switched-on drive unit, the basic sound track segment is characterized by corresponding harmony or tranquility. By switching off the basic sound track segment, the soundscape perceived in the vehicle interior can thus be made even more exciting.

[0025] Advantageously, further segments of the layered sound track are gradually integrated into the adaptive soundscape, with the subsequently generated segments sounding more "exciting" to the driver. These segments can thus increase in tempo, volume, dissonance, and other characteristics. The earlier, and therefore calmer or more harmonious, sound track segments can also be gradually faded out. If the driving situation becomes "calmer"—for example, if acceleration ends or the vehicle transitions from a curve to a straight road—the adaptive soundscape can be gradually reduced layer by layer, so that the driving situation also appears acoustically calmer.This means that the output of subsequently added layered sound track segments is terminated in reverse order until only a few layered sound track segments, or even just the basic sound track segment, are reproduced in the vehicle interior. According to a further advantageous embodiment of the method according to the invention, the first sampling rate is 20 grains per second with a grain size of 1000 ms and a grain overlap of 50, and the first duration is 10 seconds, wherein the sampled grains are distributed randomly over the first duration to generate the basic sound track segment. The use of the comparatively large grain size ensures that acoustic artifacts are minimally perceptible. Transitions between the individual grains can be eliminated by the high overlap.

[0026] The processing unit preferably distributes the grains within the base audio track segment such that each grain is played back within a time window of plus or minus 5 seconds around its corresponding sample in the base audio track segment. The individual grains are then smeared over a period of 10 seconds through temporal diffusion, i.e., playback 5 seconds before or after the corresponding sample in the song or audio track. This ensures smooth transitions between chords. Dynamic variations are eliminated due to the hard compression.

[0027] The corresponding basic sound track segment is then seamlessly concatenated to form a loop, as previously described. Ideally, this loop has the same length or duration as the underlying song. For example, if the driver uses a favorite song to create the adaptive soundscape, the adaptive soundscape will be output based on this song for the same duration. The underlying song can then be changed.

[0028] According to a further advantageous embodiment of the method according to the invention, the second sampling rate is 719 grains per second with a grain size of 121 ms and a grain overlap of 87, and the second duration is 8 seconds, wherein the sampled grains are distributed randomly over the second duration to generate the layered sound track segment. Based on these parameters for granular synthesis, harsh transitions in the sound track used can be smoothed. Even higher parameters can be used, which result in the corresponding layered sound track segment sounding more like noise and acquiring a metallic character.A further advantageous embodiment of the method according to the invention provides that the processing unit imposes an additional effect on the basic sound track segment, on at least one layered sound track segment, and / or on a section of a sound track, depending on a driving event, and causes the output of a basic sound track segment, layered sound track segment, and / or sound track section manipulated by the additional effect. In this way, further additional effects can be imprinted on the sound tracks in the song, in addition to the effects created by applying granular synthesis. Such additional effects can be imprinted only on the basic sound track segment, the layered sound track segment, or a section of a sound track, or also on several sound track segments and / or the section of a sound track.If additional effects are applied to a section of a soundtrack, meaning the underlying soundtrack itself is enhanced with additional effects, then the section used should be carefully selected. Its psychoacoustic effect on the driver should align with the character of the song and the current driving situation. Such sections in soundtracks can be identified by a separate algorithm, particularly one based on artificial intelligence. Manually reading out the relevant soundtrack sections is also a viable option.

[0029] Preferably, at least one of the following additional effects is applied: an echo, volume adjustment, frequency shift, frequency cancellation, frequency amplification, chorus, reverb, delay, saturation, compression, a change in the position of a virtual sound source within the vehicle interior, and / or distortion. This allows for a wide variety of ways to enhance individual soundtrack segments or even sections of a soundtrack itself with these additional effects. Consequently, the adaptive soundscape can be tailored to the driving experience in numerous ways, contributing to an even deeper immersion for the driver.

[0030] A further advantageous embodiment of the method according to the invention provides that the computing unit manipulates the soundscape to be emitted in the vehicle in at least one of the following driving situations: exceeding a defined accelerator and / or brake pedal actuation speed, exceeding a defined accelerator and / or brake pedal position, exceeding a defined power output and / or power change from the vehicle's drive unit, exceeding a defined speed and / or speed change from the vehicle's drive unit, exceeding a defined torque output and / or torque change from the vehicle's drive unit, exceeding a defined longitudinal and / or lateral acceleration acting on the vehicle, and / or the use of kickdown.

[0031] This provides a wide range of different driving events that can be used to manipulate the soundscape. "Manipulation of the soundscape" means that individual sound layers can be added to or removed from the adaptive soundscape. Furthermore, additional effects can be applied to a given sound layer, or the application of these effects can be adjusted.

[0032] For example, the position of the accelerator or brake pedal can be monitored. Once a certain position is reached, the adaptive soundscape is modified. The soundscape can also be changed if a change in the corresponding pedal position is detected. A distinction can be made between positive and negative changes in pedal position. For example, if a positive pedal travel is detected, the frequency, i.e., the pitch, of the correspondingly manipulated soundscape layer or the base sound track segment, the layered sound track segment, and / or the section of a respective sound track can be increased.

[0033] The power output of the drive unit changes with the torque output and the engine speed. The soundscape can be altered when one of these parameters changes. A distinction can also be made between positive and negative power output. This allows the vehicle to recover energy through recuperation. Such a recuperation process can be accompanied by the integration of characteristic sound layers into the overall soundscape.

[0034] The adaptive soundscape can also be modified when using kickdown. In particular, the duration for which kickdown is engaged can be used as a parameter to adjust the soundscape. For example, with continued use of kickdown, the frequency or pitch of the underlying acoustic signal can be continuously increased.

[0035] A further advantageous embodiment of the method according to the invention provides that the processing unit categorizes the song, as well as the basic sound track segment and the at least one layered sound track segment, based on psychoacoustic characteristics and / or the genre, and outputs only those basic sound track segments and layered sound track segments that correspond to the song's category. This further enhances the deep immersion in the driving experience for the driver. Music can be categorized based on a variety of characteristic features. For example, music differs in timbre, rhythm, the type of instruments used, the semantics contained in the lyrics, and the like. Assigning a piece of music to a genre already constitutes a categorization that can provide orientation. Such an assignment to a genre can be applied to the song or...The underlying audio file is associated with metadata. The processing unit can read and process this metadata. Corresponding audio files can also be processed and analyzed by the processing unit. This allows the relevant psychoacoustic characteristics to be automatically identified and the song to be assigned to a category accordingly. Similarly, the basic sound track segments and layered sound track segments are assigned to their respective categories.

[0036] A further advantageous embodiment of the method according to the invention provides that the processing unit plays the song without outputting it via the loudspeakers and causes the basic sound track segment and / or the at least one layered sound track segment to be output via the loudspeakers to supplement the soundscape and substitute for the song, if the sound track underlying the basic sound track segment or layered sound track segment contains content temporally within the played song. This is also referred to as a "passive effect" and is explained in detail in the figure description.

[0037] A vehicle of this type, comprising a computing unit and a sound system, is further developed according to the invention in that the computing unit and the sound system are configured to execute a method described above. The computing unit thus has read access to a computer-readable storage medium on which machine-interpretable instructions are stored. When executed by a processor of the computing unit, these instructions cause the processing unit to provide the method according to the invention. The computing unit controls the sound system and outputs the corresponding adaptive soundscape via the loudspeakers of the sound system. For example, this can be a stereo sound system. The sound system can also be capable of generating so-called 3D sound or providing a three-dimensional acoustic soundscape. The sound system can, for example, be capable of processing the Dolby Atmos sound format.This allows individual sound layers of the adaptive soundscape, or the corresponding virtual sound sources of the respective sound layers, to be virtually placed at different locations within the vehicle interior. One or more virtual sound sources can be used to output a specific sound layer. The position of such a virtual sound source can, for example, be changed depending on lateral accelerations acting on the vehicle in order to adapt the soundscape to the driving situation.

[0038] The vehicle in question can be any road vehicle such as a car, truck, van, bus, or similar. Generally, it could also be a rail vehicle, watercraft, or aircraft.

[0039] Further advantageous embodiments of the inventive method for outputting adaptive soundscapes in a vehicle interior also result from the exemplary embodiments, which are described in more detail below with reference to the figures.

[0040] This shows:

[0041] Fig. 1 shows a schematic flowchart of the method according to the invention;

[0042] Fig. 2 shows a schematic layer structure of the adaptive soundscape emitted in the vehicle interior;

[0043] Fig. 3 is a diagram showing the categorization of music based on psychoacoustic criteria;

[0044] Fig. 4 shows a schematic signal flow representation for different layers of the soundscape (Fig. 11) to which a passive effect is applied; and Fig. 12 shows a schematic signal flow representation for different

[0045] - Fig. 17 Layers of the soundscape to which an active effect is imprinted.

[0046] Using a method according to the invention for outputting adaptive soundscapes in vehicle interiors depending on the driving situation, it is possible to achieve deep immersion of the driver in the driving experience. The driving dynamics can be conveyed to the driver via an acoustic transmission path.

[0047] In step 101, as shown in Figure 1, a song 5 or an underlying audio file is fed to an in-vehicle processing unit. The song 5 consists of several sound tracks 4. The sound tracks 4 may already be extracted or extracted from the song 5 by the processing unit itself, preferably using machine learning methods. In particular, each sound track 4 comprises the sounds, vocals, or the like emitted by a single instrument. For example, there may be one sound track 4 each for the individual drums and cymbals of a drum kit, an electric guitar, a piano, and vocals. Such a sound track 4 is often simply referred to as a "track."

[0048] In step 102, the processing unit analyzes the respective sound tracks 4 and the song 5, assigning them to different categories based on the underlying genre and / or psychoacoustic parameters. This analysis can include, for example, dynamics, intensity, interval matching, noise matching, and similar parameters. Possible categories could be, for example, "Moderate Dance," "Tonal Synthetic," "Disharmonie Electro," "Guitar Overdrive," "Natural Soft," and so on.

[0049] In step 103, the respective sound tracks 4 are subjected to a so-called granular synthesis. In this process, a basic sound track segment 7.1 and at least one layered sound track segment 7.2 are generated by the processing unit. An algorithm 6.1 for generating the basic sound track segment 7.1 can also be referred to as the "Idleizer Algorithm," and an algorithm 6.2 for generating the layered sound track segment(s) 7.2 as the "Summarizer Algorithm." The "Idleizer Algorithm" uses a first sampling rate and generates the basic sound track segment 7.1 with a first duration. The "Summarizer Algorithm" uses a second sampling rate and generates the respective layered sound track segments 7.2 with a second duration. The first sampling rate is lower than the second sampling rate, and the first duration is higher than the second duration. Further layered sound track segments 7.2 can also be generated.2 are generated, which have an even shorter duration than the second duration and a higher sampling rate than the second sampling rate.

[0050] In the embodiment shown in Figure 1, both the basic sound track segment 7.1 and a layered sound track segment 7.2 are generated from the piano's sound track 4. Further layered sound track segments 7.2 can then be generated from other sound tracks 4 by the processing unit.

[0051] Preferably, the first sampling rate is 20 grains per second with a grain size of 1000 ms and a grain overlap of 50. The first duration is advantageously 10 seconds. The sampled grains are preferably distributed randomly over the first duration. In particular, for the basic audio track segment 7.1, the grains are distributed such that they are played back within a time window of plus / minus 5 seconds around their respective recording.

[0052] The second sampling rate is preferably 719 grains per second with a grain size of 121 ms and a grain overlap of 87. Preferably, the second duration is 8 seconds, wherein, to generate the respective layer sound track segments 7.2, the sampled grains are also distributed in random order over the respective second duration.

[0053] In an optional step 104, further additional effects can be applied to the respective basic sound track segments 7.1, layered sound track segments 7.2, and sections of a respective sound track 4 itself. This will be discussed in more detail below.

[0054] Preferably, the processing unit outputs only those basic sound track excerpts 7.1, layered sound track excerpts 7.2, and / or sections of sound tracks 4 that are assigned to the same category as the song 5 itself. If this is irrelevant, step 102 can be omitted.

[0055] In step 105, the respective channels or sound tracks 4 or sound layers are mixed together and output via the vehicle's sound system in the vehicle interior. When the vehicle is started, the basic sound track segment 7.1 is output by the processing unit, whereby, depending on the driving situation, when driving events 1 shown in Figure 2 occur, layered sound track segments 7.2 in the form of sound layers are additionally integrated into the adaptive soundscape 3 also shown in Figure 2 for each driving event 1.

[0056] Figure 2 shows such a schematic layer structure of the adaptive soundscape 3 in the form of a diagram. Time is represented on the abscissa and the soundscape spectrum or pitch on the ordinate.

[0057] First, the vehicle starts, meaning the starter motor is engaged and the on-board electronics are activated. The processing unit then begins outputting the basic sound track segment 7.1. This segment is repeated seamlessly to produce an uninterrupted sound loop. The output of basic sound track segment 7.1 can also be referred to as the "idle layer."

[0058] Over time, various driving events 1 occur, such as acceleration, braking, steering, kickdown, and the like. As shown in Figure 2, with each driving event 1, further layers 2 are added to the adaptive soundscape 3. A first layer 2 could, for example, include drums, a second layer 2 bass, a third layer 2 another instrument, and one of the uppermost layers 2 vocals. The uppermost layers 2 have a more intense effect on the driver. As time progresses, driving events 1 end, and the respective layers 2 are gradually reduced. The output of the corresponding sound layer is therefore discontinued. Before the vehicle comes to a complete stop, only the "idle layer" is output again.Figure 3 shows a possible visualization of the different categories to which corresponding songs 5 ​​or song components can be assigned in step 102. Figure 3 shows two two-dimensional diagrams, each of which is divided into four sections. Each song 5 or song component is assigned a category from each diagram, resulting in a total of 16 different categories. The abscissa of the first diagram represents interval consonance, and the ordinate represents pitch consonance. The abscissa of the second diagram represents intensity, and the ordinate represents dynamics of the respective piece of music or section of music. Each of the 16 different categories has specific properties that can influence the design of the effects or additional effects subsequently applied to the respective layers 2 of the soundscape 3.The basic principle is that an effect should not be more dissonant than the piece or song 5 to which it is assigned. However, an effect should also not be too subtle, otherwise it would not be sufficiently noticeable. An effect in the category "interval dissonant, audible dissonance / intense and dynamic" should therefore possess the same characteristics as the respective song 5 in order to be perceived as belonging to it.

[0059] The term "sound consonance" depends on the parameters of noise and sharpness. It describes the average overall sound of a piece. Mildness and mellowness result in sound consonance, while sharpness and noise result in sound dissonance.

[0060] The concept of interval consonance depends on the parameters of roughness and interval friction. It describes the average harmonic tension of a piece. Interval friction and roughness lead to interval dissonance, while interval fusion and smoothness lead to interval consonance.

[0061] Intensity describes the potential of a sound to attract attention. It results from a combination of the five consonance parameters and the change in selection parameters (defined as a change within the time window of the subjective present).

[0062] Dynamics describes changes in intensity within a piece. It can be defined using the selection parameter of novelty (defined as a change occurring outside the timeframe of the subjective present). High dynamics signify strong differences in instantaneous intensity.

[0063] Before output via the loudspeakers of the sound system in the vehicle interior, a said additional effect can be applied to a respective basic sound track segment 7.1, layered sound track segment 7.2, and / or even to a section of a sound track 4 itself. These additional effects can be divided into passive effects and active effects, with Figures 4 to 11 showing various embodiments of passive effects and Figures 12 to 17 presenting various active effects.

[0064] The main characteristic of passive effects is the high degree of similarity between the output signal and the sonic character of the piece, i.e., the underlying song 5. This is why passive effects blend particularly naturally into the soundscape 3. Depending on their specific sonic character, the effects suit certain sonic qualities and are appropriate for a corresponding musical category. Since their musicality creates only a weak associative technical connection to the vehicle, they are primarily used selectively and briefly, serving to generate interactive details. The principles of cross-modal associations are observed and utilized to create a direct connection to the music.

[0065] Passive effects therefore occur depending on the timing of the appearance of the content of the underlying sound track 4 in song 5 and are varied in their manifestation depending on the driving events 1.

[0066] Figure 4 shows a first possible embodiment of such a passive effect. The embodiment shown in Figure 4 is called "Pedal Bubbles." The effect occurs when the accelerator or brake pedal is pressed quickly. Figure 4 shows a signal flow diagram. The input signal is provided by a box 401. This is referred to here as a channel and corresponds to the basic sound track segment 7.1, a layered sound track segment 7.2, or even a section of a sound track 4 itself. The input signal first undergoes a so-called grain delay, indicated by a box 402. The grain delay is an echo with granular synthesis, the operating principle of which is similar to a pitch shifter. Subsequently, the signal is shifted in time by means of a panorama control indicated by a box 403, in order to cause the resulting layer 2 to sound from a different position in the vehicle interior.The panorama control 403 is coupled to a low-frequency oscillator (LFO), indicated by box 404. Finally, the signal passes through a volume control, indicated by box 405.

[0067] Both the frequency of the low-frequency oscillator and the volume are coupled to a derivative-based envelope. The effect generates a fluttering cloud of sound fragments. The specific sonic characteristics of the input signal are preserved but decorrelated in time. Transients and dynamic differences disappear due to the granulation. The effect is triggered when the pedal travel exceeds a defined difference within 50 ms. The envelope reaches its maximum within 100 ms and decays again within 1500 ms. To control the low-frequency oscillator 404 and the volume control 405, the processing unit accesses vehicle data 406. The respective pedal position or pedal travel is detected, indicated by a corresponding box 407. The vehicle data 406 can be read by the processing unit, for example, via a fieldbus system of the vehicle.For example, this fieldbus could be a CAN bus, an Ethernet data line, or something similar.

[0068] If the vehicle's sound system is capable of distributing audio three-dimensionally within the vehicle interior, the distribution of a corresponding sound source can be randomized. This makes the sound appear even more blurred to the vehicle occupants.

[0069] Figure 5 shows another passive effect, referred to here as "Pedal Waves." This effect occurs when the accelerator or brake pedal is pressed quickly and forcefully. It behaves similarly to the Pedal Bubbles effect. Its sound is created by the use of a frequency shifter 501, which, when summed with the signal, produces periodic cancellations. The signal first passes through the frequency shifter 501, whose differential constant is, for example, 5 Hz. The signal then passes through a volume control 405, with the volume being proportional to the pedal position. The effect produces a wave-like modulation. It is more noticeable at the beginning and eventually blends smoothly back into the music. Overall, the effect is subtle, as the entire tonal character of the input signal is preserved, except for the isolated cancellations.

[0070] Figure 6 shows another passive effect, referred to here as "kick boost." This effect occurs under load and selectively amplifies low-frequency peaks. The input signal is fed through a bandpass filter 601 into the sidechain 602 of a gate 603. The filter allows a specific frequency spectrum to be selected to prevent unintentional gate opening. The gate 603 determines the waveform of a low-pass filtered input signal. A corresponding low-pass filter 604 is provided for this purpose. Peaks with sufficient energy that pass through the bandpass filter 601 and open the gate 603 thus selectively amplify existing impulses. This effect is most commonly used to amplify rhythmic signals with low-frequency components, such as drums or bass.

[0071] Accordingly, for channel 401, the drum or bass track (or basic track section 7.1 or layered track section 7.2) is used. This intensifies the music. Therefore, it is noticeable but subtle. Its intensity is proportional to the load 605. In this context, load can be understood, for example, as the power, torque, and / or speed delivered by the vehicle's drive unit.

[0072] Figure 7 shows another passive effect, referred to here as "Load Widening." This effect also occurs under load and decorrelates multichannel audio signals to widen the stereo image. The input signal passes through several parallel bandpass filters 601, followed by various decorrelation effects such as chorus 701, reverb 702, and delay 703. The effect results in an intensification at the moment of acceleration. The change in the stereo acoustics simulates a greater distance to the sound sources in the music. It has a strong effect yet remains unobtrusive.

[0073] Figure 8 shows another embodiment of a passive additive effect, referred to here as "load phattening." This effect also occurs under load and primarily amplifies low and high frequencies. In addition, the input signal is saturated and compressed. Overall, this makes the sound more massive and present. This leads to an intensification of the musical experience during acceleration. Associatively, this sensation corresponds to the expectation, previously associated with internal combustion engines, of receiving acoustic information from the engine and exhaust system during periods of high load. In this way, the effect integrates seamlessly into the overall experience, both associatively and musically. Here, the input signal undergoes saturation, indicated by a box 801, followed by adjustment of an equalizer 802, and is then fed to a corresponding compressor 803 for compression.

[0074] Figure 9 shows another embodiment of a passive additional effect, referred to here as "Lateral Shift." This effect occurs during lateral acceleration. It is primarily used for the vocal track. It influences the stereo distribution of the audio signal in cornering situations. If the vehicle steers to the left, the stereo panorama is also shifted more strongly to the left. This makes the sound source appear to change its position in the direction of travel. Thus, attention is drawn to the curve not only visually but also acoustically. Due to the strong correlation between visual, kinesthetic, and acoustic perception, the effect enhances the overall impression while remaining unobtrusive itself. To control the stereo panorama 403, corresponding lateral accelerations—that is, the lateral acceleration acting on the vehicle—are tapped from the vehicle data 406, indicated by a box 901.

[0075] Figure 10 illustrates another embodiment of a passive effect, referred to here as "Load Overdrive." This effect also occurs under load. Its purpose is to increase the instantaneous intensity. The input signal passes through a distortion unit 1001 and then a volume control 405. The volume control 405 is linked to the vehicle's load parameters and controls the ratio between the distorted and clean signals. In this way, noisy characteristics are added to the overall sound. The effect accompanies acceleration curves. It blends noticeably, yet naturally, into the overall sound.

[0076] Figure 11 shows another embodiment of a passive effect. This effect is referred to here as "Pedal Glitch." Pedal Glitch occurs during rapid pedal interaction. The effect reacts to both positive and negative pedal travel. The effect is track-specific, referring here to individual sound tracks 4 or layers 2 of the soundscape 3, and not to the vehicle's driving lanes. The effect is not suitable for consonant, harmonic, or percussive signals. Unlike most passive effects, this is not a parallel effect, but a serial one that affects the input signal as a whole. It should therefore be noted that not only one effect / additional effect, but also several such effects can be applied to a given basic sound track segment 7.1 and / or layered sound track segment 7.2.With a positive pedal stroke, the signal is shifted positively by, for example, 200 Hz; with a negative pedal stroke, it is shifted negatively by, for example, 100 Hz. This process is directly coupled to a derivative curve. This curve is derived from the position difference of the pedal within 350 ms. Since the frequency shifter 501 linearly transforms all partials of the signal, the resulting partials are no longer in integer ratios to each other. The sound becomes noisier and therefore more dissonant.

[0077] Figures 12 to 17 illustrate signal flow diagrams for active effects. Active effects are additional sound generators that are not directly related to the musical content. They form the link between vehicle sounds and music and are intended to provide the driver with information about the current driving situation without conflicting with the music. Either a section of a sound track 4 is played back directly, or the effect is applied to a corresponding layered sound track segment 7.2. In this way, the fundamental sonic characteristics of the source material are preserved. This ensures that the effect fits well with the respective piece of music, i.e., song 5. Adaptation to different musical categories is achieved via specific effect chains and parameter dependencies.

[0078] Active effects are only triggered when their specific trigger signal occurs as a result of a driving event 1. Active effects are also varied depending on the driving events 1.

[0079] Figure 12 shows an embodiment of an active effect, referred to here as the "Acceleration Riser." The effect occurs during acceleration. Its intensity depends on the load (605), and its pitch (1204) on the rotational speed (1201). It is synthesized granularly from portions of the source material. Furthermore, a chain of comb filters (1205) generates a resonance pattern that imparts mechanical, metallic qualities to the sound. The result is an effect that blends harmonically and sonically with the music, while simultaneously sounding technical enough to be perceived as belonging to the vehicle. A box (1202) represents a trigger, abbreviated here as MIDI. A box (1203) represents a sound generator, also referred to as a synthesizer.

[0080] Depending on musical characteristics, the effect is rendered more consonant or dissonant. Tonality is relativized by the pitch contour and, in this respect, does not lead to prolonged interval friction. Accordingly, its consonance is primarily defined by the distribution of sharp and noisy components.

[0081] Figure 13 shows another exemplary embodiment of an active effect, referred to here as the "Deceleration Chopper". The effect occurs during braking. Its intensity depends on the braking intensity. A corresponding recuperation power is indicated by a box 1301.

[0082] The defining characteristic of this effect is its rhythmic pattern, the frequency of which depends on the rotational speed (1201). Depending on musical characteristics, this rhythm can be more or less pronounced, synchronous, or asynchronous, as can the pitch curve, which is also dependent on the rotational speed (1201). The associative link to the vehicle and rotationally dependent, periodically occurring mechanical sound events arises from these characteristics. The effect can be compared to the sound of a helicopter rotor. The tonality of the effect can remain static if its pitch (1204) does not cause any disturbing interval dissonances with the music. If it remains static, it can also assume a harmonic function (tonic, dominant) and, at the moment of braking, lead either to tension or resolution. If the effect behaves variably and its pitch (1204) follows the rotational speed (1201), the same criteria apply as for the Acceleration Riser effect.This pushes interval dissonances into the background and brings the characteristics of sharpness and noise to the forefront. A filter, indicated by a Box 1302, can also be controlled via a low-frequency oscillator 404.

[0083] Figure 14 illustrates another embodiment of an active effect, referred to here as "Lateral Friction." The effect occurs during lateral acceleration, and its intensity is proportional to this acceleration. The effect has a low frequency spectrum, allowing it to blend well with driving noise and remain in the background. Its sound is reminiscent of tire noise, such as rolling noise, which is why it integrates naturally and intuitively into the driving situation. The spectrum contains no sharp components. The effect can therefore range between tonality and noise, with the latter not manifesting as dissonance due to the low frequency discrimination.

[0084] Figure 15 illustrates another embodiment of an active effect, referred to here as "Acceleration Morph." The effect occurs under load and at low speeds 1503. The vehicle's current speed is also part of the vehicle data 406. Its purpose is to interactively convey information about the driving situation at low speeds 1503. The effect is controlled by two key parameters: the load and the load change. The cutoff frequency 1501 is directly dependent on the load 605 and is proportional to it. Furthermore, the resonance 1502 of the filter 1302 is controlled by a derivative-based envelope that amplifies the resonance 1502 at the moment of a load change. The resonance 1502 responds to a specific increase in the load 605 within 50 ms.The resulting envelope rises over 100 ms and then falls again over 1500 ms. The noise generated by the Acceleration Morph resembles the sound of a combustion engine at low speeds.

[0085] An active effect can also be applied to the basic sound track segment 7.1, or the "Idle Layer," as illustrated by Figure 16. This additional effect depends on the driving events or vehicle parameters Load 605 and the driving speed 1503. The ignition status 1601 serves as the trigger. The sound character encompasses all the core features of the underlying song 5. The Idle Layer serves as a summary and introduction. Depending on the harmonic structure of the underlying piece of music, the Idle Layer must also meet certain harmonic requirements to ensure a seamless transition into the music. Granular synthesis parameters can be controlled via Load 605, as indicated by Box 1602.

[0086] Figure 17 illustrates another embodiment of an active auxiliary effect. The auxiliary effect shown in Figure 17 is also referred to as the "Kickdown Riser." The Kickdown Riser occurs while the vehicle's kickdown function is active. The effect emphasizes the acceleration through a rising, tonal progression. A corresponding kickdown signal is indicated by a box 1701, which taps from the vehicle data 406. The Kickdown Riser is similar in design to the Acceleration Riser; however, its pitch progression correlates not with the engine speed, but with the duration of the kickdown activation.

[0087] The effects shown in Figures 4 to 17 can be integrated as layers 2 within the adaptive soundscape 3 shown in Figure 2. Using the method according to the invention, this allows the driver to experience the driving dynamics or situation more intuitively, leading to deeper immersion in the driving experience. The soundscape 3 sounds natural, which enhances driving pleasure and the driver's concentration, and reduces stress. By appropriately decomposing songs 5 ​​or underlying audio files into individual sound tracks 4, optionally with AI support, the wide use of all kinds of music is possible.

Claims

Patent claims 1. Method for outputting an adaptive soundscape (3) in a vehicle interior depending on the driving situation, wherein an in-vehicle computing unit imposes a driving-related effect on the sound tracks (4) of a song (5) decomposed into sound tracks (4) and controls a sound system of the vehicle to output the correspondingly manipulated sound tracks (4) via in-vehicle loudspeakers, characterized in that - the processing unit extracts a sound-track characteristic segment from each sound track (4), wherein the processing unit, using granular synthesis, generates a basic sound track segment (7.1) with a first sampling rate and a first duration and at least one layered sound track segment (7.2) with a second sampling rate and a second duration, wherein the first sampling rate is smaller than the second sampling rate and the first duration is larger than the second duration; and - the processing unit, when the vehicle is started, causes the output of the basic sound track segment (7.1) and, depending on the driving situation, when driving events (1) occur, causes the output of a layered sound track segment (7.2) for each driving event (1).

2. Method according to claim 1, characterized in that the computing unit stops the output of the basic sound track segment (7.1) as soon as the output of at least one layered sound track segment (7.2) begins.

3. Method according to claim 1 or 2, characterized in that the first sampling rate is 20 grains per second at a grain size of 1000 ms and a grain overlap of 50, and the first duration is 10 seconds, whereby to generate the basic sound track segment (7.1) the sampled grains are distributed in random order over the first duration.

4. Method according to claim 3, characterized in that the computing unit distributes the grains in the basic sound track section (7.1) such that a respective grain is reproduced in a time window of plus / minus 5 seconds around its respective recording in the basic sound track section (7.1).

5. Method according to one of claims 1 to 4, characterized in that the second sampling rate is 719 grains per second at a grain size of 121 ms and a grain overlap of 87, and the second duration is 8 seconds, wherein to generate the layered sound track section (7.2) the sampled grains are distributed in random order over the second duration.

6. Method according to one of claims 1 to 5, characterized in that the computing unit imposes an additional effect on the basic sound track segment (7.1), the at least one layered sound track segment (7.2) and / or a section of a sound track (4) depending on a driving event (1) and causes the output of a basic sound track segment (7.1), layered sound track segment (7.2) and / or sound track section manipulated by the additional effect.

7. Method according to claim 6, characterized in that at least one of the following additional effects is applied: an echo, a volume adjustment, a frequency shift, a frequency cancellation, a frequency amplification, a chorus (701), a reverb (702), a delay (703), a saturation (801), a compression, a change in the position of a virtual sound source in the vehicle interior and / or a distortion.

8. Method according to any one of claims 1 to 7, characterized in that the computing unit manipulates the soundscape (3) to be output in the vehicle in at least one of the following driving situations: exceeding a specified accelerator pedal and / or brake pedal actuation speed, exceeding a specified accelerator pedal and / or brake pedal position, exceeding a specified power output and / or power change from the vehicle's drive unit, exceeding a specified speed and / or speed change from the vehicle's drive unit, exceeding a specified torque output and / or torque change from the vehicle's drive unit, exceeding a specified longitudinal and / or lateral acceleration acting on the vehicle and / or the use of kickdown.

9. Method according to one of claims 1 to 8, characterized in that the computing unit categorizes the song (5) as well as the basic sound track segment (7.1) and the at least one layered sound track segment (7.2) based on psychoacoustic features and / or the genre and causes the output to only such basic sound track segments (7.1) and layered sound track segments (7.2) that correspond to the category of the song (5).

10. Method according to one of claims 1 to 9, characterized in that the computing unit plays the song (5) without outputting it via the loudspeakers and causes the basic sound track segment (7.1) and / or the at least one layered sound track segment (7.2) to be output via the loudspeakers to supplement the soundscape (3) and to substitute the song (5), if the sound track (4) underlying the basic sound track segment (7.1) or layered sound track segment (7.2) has content temporally in the played song (5).

11. Vehicle comprising a computing unit and a sound system, characterized in that the computing unit and the sound system are configured to perform a method according to one of claims 1 to 10.

Images