Method and processor circuit for simulating the acoustic interior situation of a motor vehicle, as well as a computer-readable storage medium

Abstract

Method for replicating an acoustic interior situation (31) of a motor vehicle (25) by means of a processor circuit (10) reproducing an audio signal (12) via an acoustic model (11) of a passenger cabin (29) of the motor vehicle (25) and by means of a respective sound output system (19) to at least one user (20), wherein a data set (50) of the acoustic model (11) is provided which contains at least one respective transfer function (13) for sound transmission from the sound source location (52) to several different seats (28) of the motor vehicle (25) for at least one in-vehicle sound source location (52) and / or for at least one external sound source location (52).and by means of the acoustic model (11) the respective user (20) is assigned to one of the seats (28) and, for the purpose of reproducing the audio signal (12) to the respective user (20) from a sound source location (52) described in the data set (50), at least one transfer function (13) which links the sound source location (52) with the seat (28) to which the user (20) is assigned is selected and the audio signal (12) is processed by means of the transfer function (13) and the respective processed audio signal (15) is acoustically output to the respective user (20), characterized in that , a) the respective user (20) is given a self-perception by the user (20) virtually listening from a seat (28) in the cabin (29) to how his voice sounds when it is played back from another of the seats (28) or from a sound source location (52) from the data set (50), wherein the voice of the respective user (20) is captured by means of a microphone (32) and the voice is played back as the audio signal (12) in the acoustic model (11) from the respective sound source location (52) with a time delay, and / or b) one or some or each of the transfer functions (13) are calculated on the basis of a digital geometric spatial model of the passenger cabin (29) by means of a simulation and an input gesture of the respective user (20) is converted into an associated geometric change of the passenger cabin (29) in the spatial model and the transfer functions (13) of the data set (50) are adjusted according to the geometric change.

Description

[0001] The invention relates to a method for acoustically simulating the interior environment of a motor vehicle. For this purpose, an audio signal is output to a user via a sound output system using an acoustic model of the vehicle's cabin, i.e., the passenger compartment or occupant compartment. The acoustic model processes the audio signal in such a way that the user experiences a change in the audio signal that would occur if the audio signal were played back via a sound source in or in front of the cabin and the user were located in a specific seat within the cabin. Thanks to the acoustic model, the user does not actually need to be in the cabin to experience the acoustic environment.The invention also relates to a processor circuit for carrying out the method and a computer-readable storage medium in order to be able to implement the method in a processor circuit.

[0002] When developing and / or evaluating a motor vehicle, the acoustic impression or the acoustic environment in the passenger compartment is a possible evaluation criterion for developers and / or potential buyers. Therefore, as part of the so-called sound design of a motor vehicle, it is common practice to make an audio recording at the various seating positions in the passenger compartment, especially at the possible head positions, using a microphone while an audio signal is played back in the passenger compartment or even outside the vehicle. Thus, the resulting auditory impression at the seats is described by the microphone signal and can be listened to again later by a user using headphones. However, this form of acoustic simulation of the interior environment in the passenger compartment is inflexible.If one wishes to recreate the acoustic indoor situation using a different audio signal, the described experimental setup is necessary again.

[0003] Conveying the acoustic environment of a vehicle's interior using such an acoustic model is also relevant for virtual reality simulations, particularly when presenting a user with a virtual drive in a virtual vehicle. Here, too, the goal is to enhance immersion or realism by presenting an audio signal that corresponds to the acoustic environment of the simulated vehicle cabin. This can be achieved, for example, by recording an audio signal during a real-world drive using a microphone, which can then be played back as background noise or an acoustic backdrop in the virtual reality environment. However, this approach is limited to the audio recordings made once.

[0004] Such virtual world formats also represent a new class of marketable and experiential content. The design of a vehicle's interior acoustics can also be considered a manufacturer's intellectual property (IP). The owner of such IP can make it reproducible using virtual acoustic models. However, this requires effective protection of the IP rights.

[0005] From US patent 2012 / 0113224 A1, it is known to measure the transfer functions for a room that result between several loudspeakers in the room on the one hand and a predefined listening position on the other, and to use these to manipulate the loudspeaker signal in such a way that the influence of the room, e.g. reverberation, is compensated and loudspeaker signals are perceived without the influence of the interior situation.

[0006] From WO 2013 / 101073 A1, it is known to determine the transfer function between a speaker's position and the respective microphone using microphones. The microphone signal is to be processed for a speech recognition system. To compensate for the influence of the acoustic transmission path between speaker and microphone, the microphone signal is processed using an inverse transfer function, which eliminates reverberation and other room influences so that the processed microphone signal can be recognized more robustly by the speech recognition system.

[0007] According to the state of the art, the acoustic evaluation of a vehicle's passenger compartment is also possible without a prototype of the vehicle being available, by conveying the acoustic impression of driving noises in the laboratory via headphones to a developer, as is known from the publications WO 2019 / 123 345 A1, WO 02 / 052 895 A1, DE 10 2017 113 275 A1, DE 10 2017 203 970 A1 and US 2007 / 0 156 540 A1.

[0008] The invention is based on the objective of being able to flexibly design the acoustic interior situation of a motor vehicle cabin.

[0009] The problem is solved by the subject matter of the independent patent claims. Advantageous further developments of the invention result from the dependent patent claims, the following description, and the figure.

[0010] As one solution, the invention comprises a method for simulating the acoustic interior situation of a motor vehicle. "Interior situation" here means that the simulation or replication of sound propagation does not assume free-field conditions, but rather takes into account reflections and / or diffraction and / or attenuation in order to account for reverberation and / or echo, or more generally, interactions with objects in the passenger compartment or the acoustic propagation path within the passenger compartment, as they are present in the passenger compartment of the motor vehicle. For this purpose, an audio signal is reproduced by a processor circuit via a respective sound output system to at least one user, but this does not occur directly; instead, the audio signal is reproduced via or routed through an acoustic model of the passenger compartment of the motor vehicle.The audio signal is processed using the acoustic model, and only the processed audio signal is output to the respective user via the respective sound output system. The sound output system can be headphones (stereo headphones) or a loudspeaker array with multiple loudspeakers in combination with a sound field synthesis control system for the loudspeaker array. For the sake of clarity, the invention is described below using the headphone embodiment as a representative example of the sound output system, without this being construed as limiting the description to this specific example.

[0011] Based on the processed audio signal, the acoustic assessment or evaluation of the vehicle cabin described above can be carried out by the respective user. To make this flexible and not dependent on a fixed, predetermined audio recording, the method according to the invention provides a data set which contains at least one transfer function for sound transmission from at least one internal sound source location and / or at least one external sound source location (in front of the vehicle cabin) to several different possible head positions on seats in the vehicle. In other words, the acoustic model comprises a data set with the transfer functions. These head positions can be defined, in particular, in front of a respective headrest of the seat.An internal sound source location could, for example, be the installation position of a loudspeaker in the passenger compartment. Further advantageous examples will be mentioned later. An external sound source location could, for example, beside, in front of, or behind the vehicle, in which case the transfer function also models how the penetration of window panes and / or a vehicle body panel affects the audio signal of the sound from the external sound source.

[0012] In this process, each user, to whom the processed audio signal is to be played back via headphones, is assigned to one of the seats. To output the audio signal to the respective user, at least one transfer function is selected from a sound source location considered in the data set (for example, the aforementioned loudspeaker position) that links this sound source location to the seat to which the user is assigned. Thus, although the user is not physically present in the vehicle, this assignment determines or defines which seat in the passenger compartment the user should "sit" in for the purpose of simulating the acoustic interior situation—in other words, where they are virtually located.The audio signal is processed using the transfer function, meaning it is treated or modified as it would be transmitted from the sound source to the seating position, specifically to the head position. The processed audio signal is then output acoustically to the user via the headphones.

[0013] The data set thus provides a system or description of how a sound or the audio signal of a sound is modified or influenced on its path or during propagation from a given sound source location (either inside or outside the vehicle) to the respective seat in the passenger compartment. The transfer function can replicate or model the acoustic propagation path between the sound source location and the user's head position, as would be the case in the passenger compartment, as is typical for a transfer function. Another term for transfer function is impulse response when a time-domain representation is present. The acoustic model of the vehicle can operate in both the time and frequency domains. In the time domain, spatial information is provided in the form of impulse responses, which are used to convolve the signals.In the frequency domain, spatial information is present in the form of frequency-dependent transfer functions, with which section-wise Fourier-transformed parts of the signal are multiplied and then inversely transformed. Both variants are real-time capable, but differ in the points mentioned. Both variants can be used for the invention.

[0014] By including the transfer function for each of the cabin's multiple seats in the dataset, users can select their preferred seat. Instead of using pre-recorded audio, a dataset containing transfer functions allows the acoustic environment to be simulated for a user-defined or predefined audio signal. This enables users to assess the acoustic environment by specifying an audio signal or selecting from several predefined signals, and then dynamically or in real time simulating the acoustic environment for their chosen seat.

[0015] According to one aspect, the invention provides the user with a so-called self-perception; that is, the user can virtually hear from a seat in the vehicle cabin how their voice sounds when played back from another seat or, more generally, from a sound source location within the data set. A further development provides that the respective user's voice is captured by the microphone and the voice is played back as the audio signal in the acoustic model from the respective sound source location. This can be done with a time delay (e.g., via a timeout or after being triggered by the user) so that the user only hears the processed audio signal after the voice input has ended. For example, a user can select a seat in the rear of the vehicle cabin and have their voice played back from a sound source location at the driver's or passenger's seat in the vehicle.For example, the user can estimate how a passenger in the back seat, such as a child, perceives and / or understands a voice speaking from the driver's or front passenger's seat. This is done without actually using the vehicle, solely through the acoustic model.

[0016] According to one aspect of the invention, one, some, or all of the transfer functions of the data set can be calculated by simulation based on a digital geometric spatial model of the passenger cabin. The geometric spatial model can describe the geometry of objects within the passenger cabin (e.g., walls and / or ceiling and / or window panes and / or seatbacks) as well as their acoustic properties (e.g., damping and / or reflection). An input gesture from the respective user can be captured, for example, a hand movement or an input at a control unit, such as a keyboard. This input gesture can then be converted into a corresponding geometric change of the passenger cabin within the spatial model. For example, the user can specify that a passenger seat should be adjusted, for example, moved forward or backward.This changes the modeling of the passenger cabin within the spatial model. The transfer functions of the dataset are then adjusted via simulation according to this geometric change. This allows the user to modify the cabin geometry that influences or defines the transfer functions. This can be used, for example, for the sound design described earlier. A state-of-the-art acoustic simulation program can be used for this purpose.

[0017] The invention also includes further developments that result in additional advantages.

[0018] According to a further development, a separate audio recording of driving noise, assigned to each seat, is superimposed onto the processed audio signal. This superimposition can be achieved, for example, by adding the signal values ​​of the audio recording to the processed audio signal. The audio recording can be generated, for instance, during an actual drive in the actual vehicle by operating a microphone at the respective head position or seat while the vehicle is in motion, thus obtaining a separate audio recording for that seat. Using an audio recording offers the advantage that even a diffuse sound source, which lacks a point source location, can be used to simulate the acoustic environment of the vehicle interior. This has proven particularly realistic when simulating driving noises.

[0019] To enhance immersion in the simulated acoustic environment, it can be taken into account that when the user moves or turns their head, the sound source location is perceived from a different direction relative to the changed head position. This can be simulated particularly well with stereo playback via headphones. A further development of this approach involves providing a directional transfer function for each sound source location and / or seat in the dataset for each seat, representing several different directions of incidence of the audio signal or directions of incidence at the intended head position.The transfer function not only describes the transmission of sound from the sound source to the user's head position while seated, but also specifies the head pose (head position at the seat and / or head orientation, for example, the direction vector of facial orientation) for which the transfer function applies. When the audio signal is played back, head tracking detects the user's current head pose and selects the corresponding direction-specific transfer function for processing the audio signal. Therefore, if the user turns and / or tilts their head during playback, this is detected by the head tracking system, which then switches between direction-specific transfer functions depending on the currently detected head pose.This conveys to the user a change in the auditory impression and / or a relative movement of the sound source location with respect to the moving head. Head tracking can be achieved using state-of-the-art technology, such as that used in smart glasses (VR glasses and / or AR glasses). Alternatively, two transfer functions can be provided for each individual head position, one for each ear. Thus, there are effectively two transfer functions or impulse responses per head position, one for each ear. If an Ambisonics microphone (TM) is used in the modeling, there are N transfer functions or impulse responses per transducer in the Ambisonics microphone. From these N, the two for the ears (headphones) or the M for the loudspeaker array can then be calculated.

[0020] A further development involves using smart glasses to visually display a virtual environment of the passenger cabin from the perspective of the user's assigned seat. Thus, when a user is assigned a seat or selects one, the passenger cabin is simulated or displayed as a virtual environment from that seat or perspective. Since the audio signal is also reproduced using an acoustic model tailored to the seat, i.e., with corresponding acoustic influences, the result is a more immersive passenger cabin experience.

[0021] This further development can also be used to provide multiple users, who may be located in different real-world spaces, with a shared virtual ride via the virtual environment and acoustic model. In other words, users can meet or join forces for this virtual ride by each putting on smart glasses and headphones, and the same virtual space or the same virtual environment of the ride cabin being presented to this group of users. Each user can be represented in the virtual environment by an avatar or displayed to the other users. For this virtual ride, the data set contains transfer functions for sound source locations, each corresponding to one of the seats. A voice signal from each user is captured by a dedicated microphone.The microphone can, for example, be located in the respective smart glasses. The communication between users is processed via a transmission function that links the virtual seat of the speaking user with the virtual seat of the other user, and then outputs to the other user. The other user thus hears the speaking user, i.e., their speech signal, with a modification or impairment (such as reverberation and / or attenuation), just as would occur in the actual passenger compartment of the vehicle. In this way, the respective speech signal is used during a conversation between users to convey the acoustic environment of the vehicle's interior.

[0022] To generate the data set containing the transfer functions, a further development involves emitting a test signal from the respective sound source location (for which a transfer function is required) to the seat and capturing it at the actual seat in the vehicle's cabin using multiple microphones. Thus, several microphones are provided for each seat, which can be arranged, for example, in a plane or on an imaginary surface of a 3D object, particularly a spherical surface. The sound is then converted into a separate microphone signal by each microphone, and a 3D sound field for each seat is determined from these multiple microphone signals. This 3D sound field describes the time delay and / or phase shift with which the test signal arrives at the respective microphones via the sound.Since the spatial position or relative geometric arrangement of the microphones is known or described, the direction of sound incidence at the seating position can be determined. Using the 3D sound field and the test signal, several transfer functions are then calculated for each seating position, taking the direction of incidence into account. This means that the described directional transfer functions can be calculated. For example, the microphone signals can be combined using beamforming, and the required spatial orientation for the directional transfer function can be defined using this beamforming.

[0023] The acoustic spatial model can also be used to verify whether, and with which acoustic characteristics (e.g., volume and / or spectrum), a warning signal, such as a warning tone indicating engine damage and / or an artificial activity sound from a turn signal, can be perceived by a user during a driving situation, and / or whether it possesses a prescribed acoustic property. For this purpose, according to a further development in the dataset, a loudspeaker position of a vehicle loudspeaker is defined as a sound source location.The user is presented with an acoustic representation of a driving situation (for example, by playing or reproducing driving sounds as described above), and simultaneously, the playback of at least one warning signal, such as the described warning tone, is simulated as a processed audio signal (with the speaker position as the sound source) emanating from the speaker position. As an alternative or additional warning signal, a voice prompt, such as one generated by a navigation system, can be played back using a text-to-speech engine. Another possible warning signal could be, for example, the simulation of a hands-free telephone system.By also acoustically reproducing the driving situation, for example driving noises and / or noises from other road users, or conveying the respective seating position or seat, it is shown how the warning signal is perceptible in the driving situation and / or how it overlaps with the noises.

[0024] A further development involves securing the dataset using a non-fungible token (NFT). This enables the protection of intellectual property described at the beginning.

[0025] For use cases or application situations that may arise during the procedure and are not explicitly described here, it may be provided that, according to the procedure, an error message and / or a request for user feedback is issued and / or a default setting and / or a predetermined initial state is set.

[0026] To carry out the method according to the invention, the invention further comprises a processor circuit configured to execute one embodiment of the method. The processor circuit includes a connection for at least one headphone, enabling a user to listen to a processed audio signal calculated by the processor circuit via the connection. The processor circuit can include at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). Furthermore, the processor circuit can include program code configured to execute the embodiment of the method according to the invention. The program code can be stored in a data memory of the processor circuit.The processor circuit can, for example, include at least a circuit board and / or at least a SoC (System on Chip).

[0027] The underlying vehicle is preferably a motor vehicle, in particular a passenger car or truck, or a passenger bus. However, it can also be a passenger cabin in an aircraft or a passenger train. The term 'motor vehicle' here is not limited to cars, but can include all possible vehicles (buses, trains, aircraft, etc.). Accordingly, the number of transfer functions or impulse responses included in the model is also unlimited.

[0028] As a further solution, the invention also includes a computer-readable storage medium comprising program code which, when executed by a processor circuit of a computer or a computer network, causes it to execute an embodiment of the method according to the invention. The storage medium can, for example, be provided at least partially as a non-volatile data storage medium (e.g., as flash memory and / or as an SSD - solid state drive) and / or at least partially as a volatile data storage medium (e.g., as RAM - random access memory). The storage medium can be located within the processor circuit's data storage. Alternatively, the storage medium can also be operated, for example, as an app store server on the internet. The computer or computer network can provide a processor circuit with at least one microprocessor.The program code can be provided as binary code or assembler and / or as source code of a programming language (e.g. C) and / or as a program script (e.g. Python).

[0029] The invention also includes combinations of the features of the described embodiments. The invention therefore also includes realizations that each exhibit a combination of the features of several of the described embodiments, provided that the embodiments have not been described as mutually exclusive.

[0030] The following describes exemplary embodiments of the invention. The single figure illustrates this: Fig. a schematic representation of an embodiment of the processor circuit according to the invention, which can carry out an embodiment of the method according to the invention.

[0031] The exemplary embodiments described below are preferred embodiments of the invention. In these exemplary embodiments, the described components each represent individual features of the invention, which can be considered independently of one another and each further develops the invention independently. Therefore, the disclosure is intended to include combinations of features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.

[0032] In the figure, identical reference symbols denote functionally equivalent elements.

[0033] The figure shows a processor circuit 10, which can be implemented, for example, by a computer or a network of several computers. The processor circuit 10 can operate an acoustic model 11, by means of which at least one audio signal 12 with at least one transfer function 13 can be processed, for example, by means of a mathematical convolution 14, in particular a fast convolution, in order to obtain a respective processed audio signal 15.

[0034] It may also be provided to superimpose an audio signal onto another audio recording 17 by means of a signal superposition 16, for example a signal addition.

[0035] The processor circuit 10 can have a connection device 18 through which headphones 19 of different users 20 can be connected to the processor circuit 10 in order to listen to the overall processed audio signal 15, as it can be individually provided or generated for each user 20. The connection device can be based, for example, on a sound card and / or a digital-to-analog converter for a direct headphone connection and / or a network interface controller (NIC) for network transmission of audio data and / or a radio module, for example, for Bluetooth audio transmission.

[0036] Each user 20 can be located in a different room 21 or have their own self-chosen position. However, it can be provided that the users 20 jointly experience a virtual ride 22 acoustically in a virtual environment 23, in which, for example, virtual representations (avatars 20') of the users 20 sit together in a virtual, digital, or graphical replica 24 of a motor vehicle 25 in a different position than the actual positions of the users 20. To visualize the virtual ride 22, the users 20 can wear data glasses 26 to display the virtual environment 23. Each user 20 can be assigned a seat 28 in a passenger compartment 29 of the motor vehicle 25 within the virtual environment 23.Depending on the respective seat 28, the resulting acoustic interior situation 31 in the passenger cabin 29 can then be presented or displayed to the respective user 20 during the virtual journey 22.

[0037] In order for the users 20 to be able to converse with each other, a voice signal 33 of each user can be recorded by means of a microphone 32.

[0038] The figure also shows how a head tracker 34 can capture a given head pose 35 as a pose signal and provide it to the processor circuit 10. The microphone signals from the microphones 32, i.e., the speech signals 33, can also be provided to the processor circuit 10. The connection between the headphones 19, microphones 32, and a given head tracker 34 and the processor circuit 10 can be established via a suitable data connection 40, which may include, for example, an internet connection, a wireless connection, and / or a wired connection.

[0039] In order to simulate the acoustic interior situation 31 for each user 20 according to their seat 28 or to present it via the headphones 19, a data set 50 can be provided in a data container (e.g. a file or a so-called data BLOB; BLOB - Binary Large Object) in which the transfer functions 13 for the acoustic model 11 can be provided.

[0040] The figure illustrates, using the example of the motor vehicle 25, how, for a seat 51, the transfer functions 13 can be stored starting from different sound source locations 52 or from the respective sound source location 52 as the origin of a respective sound to seat 28 (shown only at seat 51). This can be provided for all of the seats 28. A format for the respective transfer function can be selected according to the prior art.

[0041] The figure shows how a transfer function 13 can be provided to a sound source location 52 of a loudspeaker 53 in the passenger cabin 29. Further transfer functions 13 can describe the transmission of the speech signals 33 between the vehicle seats or seating positions 28, so that the speech signals 33 can be processed by means of the respective transfer function 13 and presented to the users 20 each other, i.e., to the other user 20 who is to hear the speech signal 33, taking into account the room acoustics or the acoustic interior situation 31 in the passenger cabin 29. A driving noise 56, for example, can be played or presented as a further audio recording 17, for example, by means of the described acoustic signal superposition 16, making the simulated journey 22 acoustically perceptible.

[0042] Users 20 thus receive the acoustic and visual impression that they are sitting in the respective seat 28 of their avatars 20'. Furthermore, by means of the respective head tracker 34, a change in the acoustic environment 31 can be conveyed to each user 20 depending on their current head pose 35. This is achieved by selecting a suitable direction-related transfer function 13 from the data set 50 based on or depending on the respective head pose 35 and using it for processing the audio signals 12. For the sake of clarity, this is shown in the figure for only one user.

[0043] The idea is therefore to create a virtual acoustic vehicle interior model based on spatial transfer functions or impulse responses of audio sources (radio, music streaming, etc., but also speakers in the vehicle) to all seats in a vehicle.

[0044] The data glasses 26 can, for example, be operated with a state-of-the-art VR software module.

[0045] The complete set of all transfer functions or impulse responses is provided to the virtual system in the form of a data container, which contains the data set of all transfer functions. This contains, in a specific assignment, the spatial impulse responses of the infotainment system, e.g., in Ambisonics format of any order, specifying the sample rate, bit depth, position label, and length of the impulse response. Using this information, the corresponding data set for the recipient's seat can then be assigned to their position.

[0046] The acoustic vehicle model can be used in virtual worlds of any kind, making it a digital object. NFT technology allows for the digital protection of this object. This enables the exclusive use of the object within virtual worlds, opening up a wide range of commercial opportunities for the vehicle manufacturer, such as strengthening brand identification through product placement, or selling or licensing the rights. In combination with NFT (Non-Fungible Token) technology, a manufacturer can thus exploit their rights (in particular, licensing, selling, and renting).

[0047] The combination of a virtual acoustic model and NFT technology represents a system unlike any other. On the one hand, it allows the user a highly realistic acoustic representation of models or virtual environments; on the other hand, it offers vehicle manufacturers new revenue streams and marketing methods that cannot be protected or controlled by current technologies.

[0048] Current systems describe a model for a single seat. A method for combining these models into a dataset describing multiple seats is not known.

[0049] This container of seating information can hold any number of seats, so buses, trains, airplanes, etc., could also be represented using such a system. Seat numbering in this model runs clockwise and is done by row and seat. The front left seat is designated 1.1, the front right seat 1.2, the second row left 2.1, the second row right 2.2, and so on.

[0050] By using transfer functions or impulse responses, it is possible, with slightly increased effort, to map the self-perception of a person speaking in the car, as well as the perception of this person in all other seats, which significantly increases the level of immersion of the virtual acoustic world.

[0051] The output of text-to-speech engines can also be displayed very realistically. These impulse responses can be determined using devices for measuring communication equipment. These are established dummy head microphones with an additional artificial oral cavity and a speech output. This makes it possible to position a virtual person as a sound source in a vehicle.

[0052] The first step involves creating a three-dimensional acoustic model of the interior and the infotainment playback system for each seat. This is achieved by recording driving noises, impulse responses from the infotainment system, and, if necessary, dummy heads (speakers) using established spatial audio techniques. The required impulse responses can also be calculated from an acoustic simulation. This process is repeated for all sources and all seats, ensuring that different seats in a vehicle will sound acoustically distinct to the user. The entirety of these measurements forms the virtual model, a container that is then verified using NFT (Non-Field Testing).

[0053] This model is used in the virtual world to play content such as music, navigation system announcements, assistance systems and driving sounds, or even the voices of users of the virtual world.

[0054] The recipient can immersively experience the simulated interior space via binaural auralization of the content in the playback of the virtual model. The technology of convolution of audio material with impulse responses is state-of-the-art. As an alternative to headphones, playback via sound field synthesis through a loudspeaker array is possible.

[0055] A loudspeaker array is a loudspeaker system (2-channel stereo to N channels, arranged as a 2- or 3-dimensional array). Instead of, for example, a binauralization tool including head tracking for stereo headphone playback, a suitable decoder can be used. This decoder allows you to specify the number of loudspeakers and their positioning relative to a reference point (the listener's / user's head position), and calculate the individual loudspeaker signals from a multi-channel audio data stream. These decoders represent the state of the art (e.g., IEM Plug in Suite™, AIIRA Decoder™ https: / / plugins.iem.at / #tab-AllRADecoder). When playing back audio through loudspeaker arrays, head tracking is advantageously no longer necessary. This makes a loudspeaker array highly suitable for use in driving simulators.

[0056] The container can only contain sound sources that are described by an impulse response, supplemented by driving noise, or spatially realistically contain the voice of the users of the virtual space.

[0057] Only the vehicle manufacturer knows the details of the infotainment system's signal processing. Accordingly, they would be able to create a highly detailed virtual model. One implementation example could therefore include the complete audio UI or just specific parts of the infotainment UI (balance, equalizer, etc.).

[0058] One workaround is to use the acoustics of any other vehicle instead of the original acoustics of a vehicle.

[0059] Another workaround is to simplify the vehicle acoustics. This simplification could involve, for example, omitting room acoustics altogether, using any space that isn't actually a vehicle (e.g., a rectangular room), or using simpler effects (reverberation, comb filters, etc.), as well as neglecting the seating arrangement or the differentiation between seats.

[0060] Another approach is to not measure the transfer functions or impulse responses of the acoustic model, but to calculate them computationally using simulations (e.g., finite element method, finite difference method, ray tracing, etc.).

[0061] The more comprehensive the virtual model, the more natural the immersive experience becomes.

[0062] A graphical representation of the UI (User Interface) might result in an excessive degree of complexity. However, this would allow the vehicle manufacturer to more clearly emphasize its intellectual property rights.

[0063] Combining such a model with an NFT enables effective protection against data theft and product piracy. This technology also opens up new revenue streams for the rights holder.

[0064] Future technologies for generating and experiencing virtual content benefit from the most immersive possible perception of the experienced space, that is, the interplay of visual and auditory sensory impressions. Extending this to all seating positions and incorporating all possible audio and sound sources significantly increases immersion. Furthermore, the tendency towards simulator sickness (analogous to seasickness, motion sickness) is reduced the more the recipient's senses align in their perception of the experienced reality.

[0065] Overall, the examples show how a virtual cabin model can be provided using NFT.

Claims

[1] Method for replicating an acoustic interior situation (31) of a motor vehicle (25) by means of a processor circuit (10) reproducing an audio signal (12) via an acoustic model (11) of a passenger cabin (29) of the motor vehicle (25) and by means of a respective sound output system (19) to at least one user (20), wherein a data set (50) of the acoustic model (11) is provided which contains at least one respective transfer function (13) for sound transmission from the sound source location (52) to several different seats (28) of the motor vehicle (25) for at least one in-vehicle sound source location (52) and / or for at least one external sound source location (52),and through the acoustic model (11) the respective user (20) is assigned to one of the seats (28) and, for the purpose of reproducing the audio signal (12) to the respective user (20) from a sound source location (52) described in the data set (50), at least one transfer function (13) that links the sound source location (52) with the seat (28) to which the user (20) is assigned is selected and the audio signal (12) is processed by means of the transfer function (13) and the respective processed audio signal (15) is acoustically output to the respective user (20), , characterized by , that a) the respective user (20) is given a self-perception by the user (20) virtually listening from a seat (28) in the cabin (29) to how his voice sounds when it is played back from another of the seats (28) or from a sound source location (52) from the data set (50), wherein the voice of the respective user (20) is captured by means of a microphone (32) and the voice is played back as the audio signal (12) in the acoustic model (11) from the respective sound source location (52) with a time delay, and / or b) one or some or each of the transfer functions (13) are calculated on the basis of a digital geometric spatial model of the passenger cabin (29) by means of a simulation and an input gesture of the respective user (20) is converted into an associated geometric change of the passenger cabin (29) in the spatial model and the transfer functions (13) of the data set (50) are adjusted according to the geometric change. [2] Method according to claim 1, wherein for each of the seats (28) a respective audio recording (17) of a driving noise (56) assigned to the seat (28) is superimposed on the processed audio signal (15). [3] Method according to one of the preceding claims, wherein in the data set (50) a respective direction-related transfer function (13) is provided for several different directions of incidence per sound source location (52) and / or seat (28) and when playing back the audio signal (12) by means of head tracking a head pose (35) of the user (20) is detected and the corresponding direction-related transfer function (13) is selected for the detected head pose (35) and used to process the audio signal (12). [4] Method according to one of the preceding claims, wherein a virtual environment (23) of the cabin (29) is displayed to the respective user (20) by means of data glasses from the respective perspective of the seat (28) assigned to the user (20). [5] Method according to claim 4, wherein a common virtual ride (22) is output to several users (20) who are located in different real spaces (21) via the virtual environment (23) and the acoustic model (11), and the data set (50) provides transfer functions (13) for sound source locations (52), each of which corresponds to one of the seats (28), and a speech signal (33) of the respective user (20) is captured by means of a respective microphone (32) and is processed between the users (20) by means of the respective transfer function (13) that couples the virtual seat (28) of the speaking user (20) with the virtual seat (28) of a respective other user (20), and is output to the respective other user (20). [6] Method according to one of the preceding claims, wherein, to generate the data set (50) at the respective seat (28) of the motor vehicle (25), a sound of a test signal emitted from the respective sound source location (52) is captured by means of several microphones and converted into a respective microphone signal, and a 3D sound field for the seat (28) is determined from the several microphone signals, and several of the transfer functions (13) are calculated using the 3D sound field and the test signal for each seat (28). [7] Method according to one of the preceding claims, wherein in the data set (50) a loudspeaker position of a loudspeaker (53) of the motor vehicle (25) is provided as a sound source location (52) and a driving situation is acoustically reproduced to the user (20) and during this time the reproduction of at least one warning signal emanating from the loudspeaker position is simulated as a processed audio signal (15). [8] Method according to any of the preceding claims, wherein the data set (50) is secured by means of a non-fungible token, NFT. [9] Processor circuit (10) comprising a connection device (18) for at least one sound output system (19) and configured to perform a method according to one of the preceding claims. [10] Computer-readable storage medium comprising program code which, when executed by a processor circuit (10), causes it to execute a method according to any one of claims 1 to 8.

Images