Method, device, mobile user device, computer program for controlling an audio system of a vehicle

Abstract

A method for controlling an audio system of a vehicle, comprising: Determining, in particular, predicting a parameter of an auditory communication in the interior of a vehicle, and - Changing at least one parameter of an audio signal of the vehicle's audio system depending on the determined, in particular predicted, parameter of the auditory communication, where - the audio signal is a multimedia content played via a multimedia system of the vehicle, a part of a sound design of the vehicle and / or an artificially generated engine noise, and the change in the parameter of the audio signal includes - a modification of the audio signal in the frequency domain such that signal components of the audio signal relating to the frequency range of the speaker's voice are modified such that the signal energy is distributed over one or more frequency ranges which do not relate to the frequency range of the speaker's voice or relate to it to a lesser extent.

Description

[0001] The invention relates to a method, a device, a mobile user device and a corresponding computer program for controlling an audio system of a vehicle.

[0002] Document DE 10 2015 118 553 A1 describes a method for filtering noise in a vehicle. The methods include generating a microphone signal corresponding to noise in the passenger compartment. The methods also include receiving an audio-based service used by at least one occupant. A noise separation mode is selected from the plurality of noise separation modes. The input received by the microphone is filtered according to the selected noise separation mode to generate at least one filtered signal.

[0003] The document US 2015 / 0 310 865 A1 describes a speech recognition system comprising a microphone for receiving speech from a user and processing electronics.

[0004] The publication DE 10 2015 206 021 A1 describes a computer system for a vehicle, comprising a computing unit and an output unit. The computing unit comprises a virtualization layer, a central virtual management unit, and at least one virtual signal generator unit. The virtualization layer is designed to operate an operating system of the central virtual management unit on the computing unit in parallel with a respective operating system of the at least one virtual signal generator unit. The at least one virtual signal generator unit is designed to provide a signal representative of an audio signal to be output. The central virtual management unit is designed to control the output of a respective signal of the at least one virtual signal generator unit via the output unit.

[0005] The document US 2013 / 0 028 443 A1 describes a system for enhancing audio outputs using a computer and an output device. The computer includes a sensor configured to determine a user location relative to the computer. The sensor is also configured to collect environmental data corresponding to an environment of the computer. The computer also includes a processor in communication with the sensor and configured to process the user location and the environmental data and to adjust at least one of an audio output and a video output. The output device is in communication with the processor and configured to output at least one of the two outputs, namely the audio output or the video output.

[0006] Modern vehicles can access a variety of different audio sources, which can be played via the vehicle's audio system in the interior of the vehicle. These audio sources include, for example, a radio receiver, music or audiobooks from a permanently installed storage device (e.g., a hard drive) or from a device carried in the vehicle. This also includes access to or playback of streaming content, such as an entertainment program from the internet.

[0007] Likewise, sound can be output from a video, a television program, a video game, or a video simulation, for example, via a so-called rear-seat entertainment system. Furthermore, audio systems for vehicles are known that adjust the volume of a multimedia program as the vehicle speed increases while driving.

[0008] Artificially generated engine noise is also increasingly being emitted via the vehicle's audio system. Such engine noise can be part of a specific sound design. In electric vehicles, for example, the artificially generated engine noise serves to create the familiar feeling of a combustion engine for the driver.

[0009] In addition, there is typically a need or necessity for the vehicle's occupants to communicate with each other. Such communication, especially verbal communication, can often be of great relevance to the vehicle's user. The ability of the occupants to communicate with each other as undisturbed as possible can at times be much more important to the vehicle's users than the vehicle's technical parameters, such as engine power, manufacturing quality, vehicle design, or vehicle brand. However, communication between the occupants is very often disrupted by a multitude of audio signals emitted in the vehicle. Therefore, there is a contradiction between the occupants' desire to hear certain audio signals and the desire to speak undisturbed while driving. The occupants of the vehicle, e.g. members of a family, can have very different, sometimes contradictory needs, e.g.regarding active verbal communication or listening to other occupants or listening to certain audio signals.

[0010] The object underlying the invention is to provide a method, a device, a mobile user device, and a corresponding computer program for improved control of a vehicle audio system. In particular, at least one audio signal to be output is adapted in such a way that a functional improvement or functional enhancement results.

[0011] The problem is solved by the features of the independent patent claims. Advantageous embodiments are characterized in the subclaims.

[0012] According to a first aspect, the invention is characterized by a method for controlling an audio system of a vehicle, comprising determining, in particular predicting, a parameter of an auditory communication in the interior of a vehicle, and changing at least one parameter of an audio signal of the audio system of the vehicle depending on the determined, in particular predicted, parameter of the auditory communication.

[0013] In a simplified example, if the method detects that, for example, the passenger is starting to speak or intends to speak, the audio signal to be output, in particular the one currently being output, is modified accordingly. Preferably, the modification occurs only or primarily with regard to the appropriate time interval.

[0014] Thus, the method modifies the audio signal in such a way that the (modified) audio signal and / or the occupant, or at least the occupant's verbal message, can be heard or understood better.

[0015] The procedure may be designed to achieve specific objectives. Such objectives, or the resulting benefits, may include, in particular: - A speaker (driver, front passenger, person in the back seat) does not have to drown out a running audio program in order to be heard or understood sufficiently. - A listener has to make less effort. - The speaker and listener do not have to abandon the audio program to conduct a conversation. The audio program can continue to be heard. - Even urgent messages can be communicated reliably, even when the music is very loud.

[0016] The audio signal to be output can, for example, already be present as a stereo signal, preferably a multi-channel signal, or as a surround sound signal, e.g., encoded, from an audio source, e.g., as a data stream or corresponding data packets. The audio signal to be output can, for example, be present in one of the surround sound formats known from home theater applications. Preferably, a surround sound signal corresponding to the audio signal, e.g., from the data or data packets of an audio source, can be determined, amplified, and output in the interior of the vehicle according to the features of the invention described below.

[0017] In a simplified example, changing the audio signal parameter can be changing the volume or the volume of one of several specific speakers in the vehicle's audio system. Further variations will be discussed in the further description.

[0018] The modification, in particular the adaptation, of the audio signal to be output can occur during its generation, e.g., when determining the data segments or the data stream representing the audio signal. The modification, in particular the adaptation, of the audio signal to be output in the vehicle interior can occur by directly or indirectly controlling the vehicle's audio system or a computing unit carried in the vehicle that generates, processes, or outputs the audio signal.

[0019] For example, changing, in particular adapting, at least one parameter of the audio signal to be output in the interior can be carried out by controlling an audio system carried in the interior of the vehicle depending on the determined, in particular predicted, parameter of the auditory communication, wherein the control signals for controlling the audio system are carried out, in particular at least partially, by means of an on-board computer of the vehicle or a mobile user device carried in the vehicle.

[0020] The determination, in particular prediction, of the parameter of an auditory communication can be carried out, for example, using means of the vehicle, e.g. microphones of a hands-free system, e.g. the hands-free system integrated in the vehicle or another sensor in the interior of the vehicle, e.g. an interior camera.

[0021] In particular, a parameter characterizing the beginning and / or end of the voice message, volume level, pitch, etc., can be determined.

[0022] Auditory communication can be a conversation inside the vehicle (a monologue, a dialogue or a multidirectional conversation, a “multilogue”, between several people in the vehicle).

[0023] In particular, at least one portion of said auditory communication comprises a verbal message, or the sounds intended for communication by an occupant in the vehicle's interior. This may also be a monologue, with at least one potential listener being present in the vehicle.

[0024] The method can preferably comprise: detecting that there are at least two passengers in the vehicle, in particular determining the number of passengers in the vehicle's interior, and incorporating the detected information into the method. For example, when human speech is detected and there are at least two passengers in the vehicle's interior, a different change in the parameters of an audio signal to be output in the interior can occur compared to the case where it is detected that there is only one passenger (e.g., only the driver) in the interior.

[0025] The audio signal in this invention is as follows: - Multimedia content played via a vehicle's multimedia system, e.g. music, video, radio broadcast, video game, simulation, etc., this also includes a so-called rear-seat entertainment system. - Part of a vehicle's sound design and / or an artificially generated engine noise or driving noise, such as may be provided for in electric vehicles.

[0026] In particular, an audio signal can be an artificially generated, e.g., synthesized engine noise or an artificially generated driving noise. Such an engine noise or an artificially generated driving noise can be generated for the vehicle's interior in addition to or independently of the actual, physical engine noise and (likewise) output via the vehicle's audio system. This can be combined, in particular mixed, with one or more other audio signals in the vehicle's audio system, in particular placed as one or more virtual sound sources (perceivable by an occupant in a specific position, distance, or direction).

[0027] An artificially generated engine noise or driving noise can be generated in the method depending on the actual power generated or output by the vehicle's drive and / or depending on the power requested by the driver from the vehicle's drive (for example, before the corresponding power is actually available). In this case, the method can convey to the occupants, in particular to the driver, a desired driving experience with subjectively improved driving dynamics. According to the invention, a corresponding audio signal to be output can be generated or controlled in such a way that the transmission of verbal communication is not impaired or is only slightly impaired. This results in considerable creative freedom for the implementation of such an engine noise or driving noise.

[0028] The determined, in particular predicted, parameter of auditory communication is in particular an acoustic parameter, preferably: - a spatial sound parameter, e.g. spatial distribution of the acoustic signal of speech, - a frequency parameter, e.g. an energy density spectrum - a parameter characterising the amplitude curve, or amplitude change, or amplitude dynamics, or a so-called dynamic compression, - a phase parameter, e.g. phase shift parameter.

[0029] Several parameters of auditory communication can also be determined and / or predicted.

[0030] According to an optional embodiment, the change in the parameter of the audio signal relates to at least one surround sound parameter, wherein, preferably, at least one direction from which the audio signal of the audio system of the vehicle comes or is heard, in particular heard by the driver of the vehicle, is adapted.

[0031] Preferably, the method determines the occupancy of one or more seats (e.g., by adults, children, so-called child seats, and / or infant seats). Particularly preferably, the method determines the number and / or distribution of the vehicle's occupants across specific seats or seating positions within the vehicle interior and takes this into account in the method.

[0032] In a simplified case, the number and / or distribution of persons present in the vehicle can be determined by evaluating the actuation of corresponding door contacts and / or seat belt fastening detection and / or by a sensory, e.g. capacitive, seat occupancy detection known to those skilled in the art.

[0033] Furthermore, the number and / or distribution of people present in the vehicle can also be determined via the number of mobile user devices detected, in particular localized, in the vehicle interior, e.g. mobile radio devices and / or wearables (e.g. smart watches). Furthermore, the number and / or distribution of people present in the vehicle can be determined using so-called smart closes or near-field communication using NFC (= Near Field Communication) or, for example, RFID chips (= Radio Frequency Identification). Identification of the personal identity of the occupants is not necessary in the process. Nevertheless, such identification of specific people can also be carried out, in particular to assign a specific user profile relevant for the process.For example, it is also possible to determine which of the people present is sitting at which of the vehicle's seats, for example in order to then assign certain optimization criteria to the corresponding people or seats.

[0034] In particular, it is also possible to determine the assignment of persons or groups of persons, e.g., groups of persons: an adult, a child, a baby without a baby seat, a baby seat with or without a baby, to specific seating positions or seats within the vehicle. The method can also be applied differently to different groups of persons (each as "speakers" and at least potential "listeners"). For example, methods can take into account predefined audio parameters, e.g., limit parameters that should and / or should not be exceeded in a specific seating area, e.g., the seating area of ​​a child or baby.

[0035] Subsequently, a change can be made to at least one audio signal to be output, resulting in a shift in the acoustic power generated or to be generated in specific parts of the vehicle interior and / or a change in the phase of the audio signal output from one or more specific loudspeakers. In particular, a frequency-dependent change in a phase shift between the audio signal output by at least two loudspeakers of the vehicle can be performed.

[0036] Optionally, the source of the audio signal can be shifted objectively, e.g. with regard to a power generated by a loudspeaker (in a specific spatial area), depending on the, in particular, predicted parameter of the auditory communication, for example in the case of a recognized communication intention and / or a voice of at least one occupant of the vehicle.

[0037] Alternatively or additionally, a subjectively perceived direction from which the audio signal originates, generated using the surround sound method, can preferably be shifted depending on the, in particular, predicted parameter of the auditory communication, for example, depending on a recognized communication intention and / or a voice of at least one occupant of the vehicle. This can particularly preferably be done in combination with varying the amplitude and / or frequency parameters of the audio signal.

[0038] Particularly preferably, the at least one (to be adjusted) surround sound parameter represents one or more parameters for generating one or more (also virtual, at least audible by human perception) sound sources. For example, a surround sound parameter represents a direction, distance, position, and acoustic parameters of one or more sound sources generated by the vehicle's audio system. The generation of the sound sources in the method can each relate to a specific interior area from which they are made perceptible (particularly well or to a suitable extent).Preferably, a surround sound parameter can also determine which (perceptible) direction, distance, position, and acoustic parameters are to be made perceptible to at least one sound source in a spatial area of ​​the interior, and / or in connection with a seat and / or by a person, a person belonging to a group of people, or a specific person. For example, based on the audio signal to be adapted, depending on the said parameters of auditory communication, auditorily perceptible sound sources are generated for at least two different spatial areas of the interior of the vehicle or for two specific occupants, or occupants in specific seats, which have different, each appropriately adapted, objectively measurable, or subjectively perceptible acoustic parameters by the actually present or assumed occupants.

[0039] A surround sound parameter in the method can also be adjusted depending on the determined distribution of the people across the vehicle seats and / or their positions in such a way that a cumulative value for at least two people, representing the quality of the perceptibility and / or intelligibility of at least one audio signal and / or at least one verbal communication, is increased or controlled towards a target value.

[0040] For example, only or predominantly a certain part of the audio signal that represents speech (e.g. the voice of a protagonist) in a surround sound system, e.g. in a cinema format, can be changed, in particular spatially shifted for the perception of at least one occupant.

[0041] At the same time, another part or channel of the audio signal, e.g. representing background music to be output in the surround sound system, can be adjusted less or in a different way depending on the determined, in particular predicted, parameter of the auditory communication.

[0042] The process can be used to optimize the audibility of certain parts, in particular channels of a surround sound or an audio signal comprising surround sound.

[0043] In this context, the method can also generate a distribution, in particular of content, of the auditory communication and the audio signal to “two of the ears of one or a specific listener”, which in turn reduces mutual masking of the auditory communication and the audio signal to be output and / or enables improved acoustic perceptibility by the listener.

[0044] Furthermore, the human brain can process the content much better. The corresponding content (which matches the audio stimuli) can thus be "sorted" and processed much more effectively when perceived by a human.

[0045] Preferably, a distinction can also be made as to how many people, in particular which of the people communicate (speak) with which person aurally and, depending on this, one or more parameters of the audio signal are appropriately adjusted with regard to the spatial sound or spatial distribution of the audio signal.

[0046] The change in the parameter of the audio signal comprises a modification of the audio signal in the frequency domain such that signal components of the audio signal which relate to the frequency range within the speaker's voice are modified such that the signal energy is distributed over one or more frequency ranges which do not relate to the frequency range of the speaker's voice or relate to it to a lesser extent.

[0047] In other words, a transformation of the frequencies of the audio signal can be carried out in such a way that the frequencies of the signal components of the audio signal, which have a substantial part of their signal energy within the frequency range of human speech, in particular within the voice of the speaker, are transformed into one or more frequency ranges which do not or less affect the frequency range of human speech.

[0048] For example, depending on at least one parameter of the communication, the method can: A change, in particular an adaptation of the audio signal to be output, may comprise a modification of the frequency components of the audio signal which have an overlap with the frequency range of human speech, in particular the voice of the speaker, into frequency components which have a lesser overlap with the frequency range of the human voice, in particular the voice of the speaker.

[0049] For example, depending on at least one parameter of the communication, the method can: A change, in particular an adaptation, is applied to the audio signal encoded in the frequency domain or transformed into the frequency domain, wherein the spectral properties, in particular a frequency density spectrum of at least part of the audio signal, are adapted.

[0050] The modification, in particular transformation of several signal components of the audio signal can be carried out in such a way that a mathematical relationship between the signal components, in particular one or more proportions, e.g. relating to the frequency density spectrum, is essentially retained.

[0051] In particular, a frequency band transformation can also be applied, whereby the frequency components of the audio signal which lie within the frequency band of human speech, in particular within the speaker's voice, are transformed to the frequency ranges which lie essentially outside the aforementioned frequency band which is particularly important for human communication.

[0052] According to a further optional embodiment, the change in the parameter of the audio signal comprises an at least partial shift of the frequencies of the audio signal, which at least statistically correspond to the frequencies of the speech signal of the speaker, in particular with respect to a position and / or head orientation of at least one listener from a first spatial direction in which the speaker is located to a second spatial direction, which is arranged differently than the first spatial direction, in particular substantially opposite to the first direction.

[0053] An at least partial shift is designed, for example, in such a way that the audio signal from the direction of the speaker is reduced at least in a frequency range, e.g. from 120 - 6000 Hz, in particular 350 - 3000 Hz, and frequencies of the audio signal from the audio system of the vehicle, which essentially comes from a different direction than the direction of the speaker, and in a frequency range of 200 - 6000 Hz, in particular 350 - 3000 Hz, are amplified.

[0054] Particularly preferably, a spatial shift of the audio signal with certain frequencies takes place in such a way that an overall frequency weighting of the audio signal arriving at the listener remains essentially unchanged or at least is significantly lower than the reduction of the frequencies from the direction of the speaker.

[0055] Particularly preferably, different audio signals, in particular different sections or channels of a multi-channel audio signal, can be modified, in particular adapted, in different ways in the method. These can be adapted in a specific manner, e.g., to achieve a specific goal pursued by an optimization criterion.

[0056] Preferably, in the method, the audio signals representing (predominantly) speech, or sections or channels of an audio signal, for example radio broadcasts, can be adapted differently, in particular more strongly, than the audio signals representing (predominantly) music, or sections or channels of an audio signal.

[0057] The method can comprise a step for evaluating an audio signal, e.g. coming from an audio source, whereby an indicator for the composition of the content of the audio signal, e.g. portions of speech, singing, background music, scenic background noises, is determined and taken into account in the method.

[0058] According to a further optional embodiment, the method additionally comprises learning a specific voice and / or a specific physical movement of a speaker (speaking occupant) at one or more (earlier) time intervals and then, upon recognition of the specific voice and / or the specific physical movement of a speaker at one or more later time intervals, applying an optimized or pre-stored adaptation of the audio signal for the speaker's voice.

[0059] Preferably, the following dependency arises: - If a physical movement, e.g. a certain lip movement pattern, is detected in a person who has always spoken quietly and / or unclearly in the vehicle, then the audio signal is adjusted comparatively strongly and to match the speech frequency of the speaker. - If a physical movement, e.g. a certain lip movement pattern, is detected in a person who has always spoken loudly and / or clearly in the vehicle, then the audio signal is not detected or is comparatively weak, and is adjusted to match the speech frequency of the speaker.

[0060] According to a further optional embodiment, the method comprises learning a specific physical movement which precedes speaking, in particular in the case of a specific occupant, in a verbal communication at one or more time intervals and then, upon recognition of the physical movement, in particular also upon recognition of the specific speaker and / or the physical movement typical for the speaker from several time intervals, applying an optimized or pre-stored adaptation of the audio signal for the speaker's voice.

[0061] Particularly preferred is the ability to detect a physical movement preceding a particular pitch or tone of voice, and then to adjust the audio signal (even more precisely).

[0062] According to a further optional embodiment, the method additionally comprises detecting a communication intention and / or a voice of at least one, in particular a specific occupant of the vehicle and / or an occupant at a specific seat of the vehicle, and changing the audio signal depending on the detected communication intention and / or the voice of the occupant.

[0063] The intended communication and / or the voice of at least one, in particular a specific, occupant of the vehicle can be recorded, for example, using vehicle resources, e.g., microphones of a hands-free system and / or other interior sensors. Particularly preferably, a communication intention can be carried out using an interior camera of the vehicle and the correspondingly configured image processing system.

[0064] In one example, the communication intention represents that linguistic communication, in particular from a specific passenger or from a passenger at a specific seat, is to be expected, in particular with a certain degree of probability (exceeding a threshold). The communication intention can also be used to capture and consider a predicted value of a parameter, e.g., an acoustic parameter, of the expected linguistic communication. Similarly, the "communication intention" parameter can also indicate the conclusion or termination of a section of linguistic communication, e.g., the end of a sentence, phrase, or a specific linguistic structure.

[0065] In particular, a parameter characterizing the beginning and / or end of the voice message, volume level, pitch, etc., can be determined.

[0066] According to a further optional embodiment, it is also recognized that the recorded voice is a personal voice, in particular a voice of a specific person or group of people, and the audio signal is changed depending on the recognition of the voice as a personal voice, in particular depending on the recognition of the voice of a specific person or group of people.

[0067] In other words, the system reacts specifically to personal voices, and not to other noise sources, such as the barking of a dog or the honking of other road users, or to background noise, such as wind noise or specific background noise. Alternatively, the system reacts to other sound sources in a fundamentally different way and / or with a different change to the audio signal, e.g., by adjusting audio signal parameters.

[0068] For example, an audio signal to be output, particularly one comprising speech or personal voices, is adapted predominantly or more strongly to the parameter(s) of the auditory communication than, for example, to the acoustic parameters of another noise source, such as the barking of a dog or the honking of other road users or to a background noise, such as wind noise or specific background noise.

[0069] For example, a parameter of the audio signal is adapted differently, in particular more strongly, to detect that the audio signal contains speech, compared to an audio signal that does not contain speech or that contains predominantly music content.

[0070] This is particularly advantageous because the intelligibility of a spoken message is significantly more affected by other people's voices than by other noises.

[0071] The voice of a specific person and / or the voice belonging to a specific group of people can be a learned voice. A learned voice is a voice, particularly one characterized by specific, learned parameters or parameter combinations. Preferably, the method can also learn the voices of specific people, and an appropriate adjustment of the audio signal can be performed specifically to recognize the people's voices.

[0072] The learning of the voice (of a specific person) or the group of people can be done in the vehicle or outside the vehicle or using another device, e.g. a mobile user device.

[0073] Particularly preferably, a different adaptation (characteristic curve, mathematical relationship) of at least one section of the audio signal can be carried out for at least two voices of two different persons.

[0074] In a further example, at least one parameter of the audio signal can be adapted differently, in particular much more strongly, to certain persons highlighted by a recognition or presetting, or to occupants in certain seats.

[0075] For example, if a person authorized to lead, e.g. a manager or an operations manager, speaks to or at his employees or team during a journey, the procedure can be used to ensure sufficient silence or audibility of the verbal message.

[0076] The “groups of people” include, for example, family members, familiar voices, especially those learned in the vehicle, “women”, “men”, “children”.

[0077] In particular, this refers to a person's voice spoken (live) in the interior of the vehicle (not a voice reproduced there, e.g. from a radio broadcast).

[0078] According to a further optional embodiment, the voice of a speaker in the interior of the vehicle is differentiated from a voice from a multimedia player, in particular from the sound system of the vehicle or a multimedia player connected to the vehicle.

[0079] Preferably, the voice of a speaker in the interior of the vehicle can be distinguished from the voice from a multimedia system by comparing the signals of a microphone system of the vehicle and a signal from the multimedia system of the vehicle.

[0080] For example, the voice of a speaker in the vehicle interior can be differentiated from a voice from a multimedia player, in particular from the vehicle's sound system or a multimedia player connected to the vehicle, by determining whether the voice of a speaker in the vehicle is only contained or recognizable in the vehicle's microphone system, or whether the voice can (also) be recognized in an evaluated signal from a multimedia system. Such a comparison can be performed, for example, by evaluating a similarity measure or correlation measure between a signal captured with a microphone and an audio signal read in, for example, from the multimedia system.

[0081] According to a further optional embodiment, the communication intention of at least one occupant of the vehicle is detected by detecting his lip movement and / or a physical movement, in particular by means of an interior camera.

[0082] Preferably, a parameter of the audio signal can also be adjusted in advance (before or shortly after the start of speaking). This has the advantage of preemptively preempting the person who wishes to communicate or intends to communicate, or who is in the process of communicating verbally.

[0083] This can be achieved, for example, by means of an interior camera of the vehicle, which records at least the driver of the vehicle (and possibly the passenger) and can detect corresponding physical movements.

[0084] One or more physical movements (from the analysis of body language and / or emotion evaluation) that precede speech can also be recognized as a communication intention. This can be done based on corresponding scientifically based principles or observations, and / or based on user preferences and / or, preferably, based on a machine learning process.

[0085] It is particularly preferred that a smile and speaking and / or a face that is clearly frightened by a strong facial gesture and loud speaking or screaming are recognized as a condition for adapting the audio signal, and then an adaptation, in particular an increased adaptation, of a parameter of the audio signal takes place.

[0086] Particularly preferably, the audio signal can then be switched off, paused, interrupted or adjusted more strongly when a frightened face (facial expression) and / or when an argument or screaming is detected, than with an average level of willingness to communicate and / or voice recognition.

[0087] According to a further optional embodiment, the audio signal is adapted depending on the position of the speaker and / or a relative direction from which the spoken content comes or will come according to the communication intention.

[0088] For example, the (current or planned) audio signal can be adjusted differently, in particular more strongly, to the driver speaking or willingness to communicate and / or the speaking of a passenger in the front passenger seat than to the speaking and / or willingness to communicate of a passenger in the back seat.

[0089] For example, passengers in the back seat cannot change a traffic message or can only do so in certain cases, while the passenger in the front passenger seat has a greater influence.

[0090] According to the invention, the position of the speaker can be determined by comparing the signals of a microphone system in the vehicle (e.g. at least two microphones of a hands-free system) and / or seat occupancy detection and / or interior camera.

[0091] For example, the position of the speaker can be determined from a phase shift of an acoustic signal that is recorded, for example, by two spatially offset microphones in the vehicle.

[0092] According to a further optional embodiment, an emotion recognition and / or intention recognition of an occupant of the vehicle is recognized and taken into account in the method, whereby the audio signal is then changed, preferably in a different way.

[0093] In this case, emotion recognition and / or intent recognition can be performed, in particular, using vehicle resources, e.g., using a vehicle's interior camera. For example, emotion recognition and / or intent recognition can involve evaluating the occupant's facial expressions and gestures over time, with or without other physical movements.

[0094] For example, if a very high level of excitement of the occupant (speaker) is detected, the audio signal can be adjusted earlier and / or more strongly.

[0095] Furthermore, the method can alternatively or additionally switch from a first audio signal to a second audio signal, and possibly vice versa, depending on the detected parameter of the auditory communication. In this case, at least one section of the audio signal can be replaced by another section of the audio signal. This can be varied in the method by accessing the corresponding signal source or data sections.

[0096] Furthermore, the determination, in particular prediction, of a parameter of auditory communication can be followed by a modification of the spatial sound parameters of the audio system such that modified acoustic spatial properties, in particular adapted to the parameter of auditory communication, in particular virtual acoustics, are simulated for the output of a section of the audio signal. For example, this can involve a particularly smooth switching or conversion between at least two parameter data sets representing different spatial properties, e.g., a (specific) "concert hall," "jazz club," "science fiction," "cinema," "gaming," etc., "news."

[0097] Depending on the recorded parameter of the auditory communication (in the result), a quick switch or a gradual transition can be carried out between an optimized music perception and an optimized audibility of the auditory communication or to an overall optimized perceptibility.

[0098] Furthermore, the method can optionally provide for a repetition of a section of an audio signal that is or was significantly influenced by the detected auditory communication. For example, a music track that has already started playing and was significantly (significantly or for a relatively long time interval) drowned out by the auditory communication during playback can be played repeatedly.

[0099] For example, a portion of a speech-based program that has been significantly (significantly or for a significant period of time) drowned out by the auditory communication during playback can be repeatedly played. The repetition can be performed in such a way that the repeated section of the audio signal represents a valid and sufficiently complete, in particular a perceptible, meaningful musical section, e.g., a vocal verse and / or a comprehensible piece of information and / or a linguistic structure (phrase, sentence, subordinate clause, etc.).

[0100] A repetition is also to be understood as meaning that a new section of the audio signal or a section of the audio signal is generated again, at least an insufficiently transmitted content is compensated, supplemented or replaced.

[0101] It may also be the case that determining a measure for the applicability and / or probability of success with regard to the output of at least one section of the audio signal to be output, taking into account the determined parameter of the auditory communication and / or at least one further acoustic content audible in the interior of the vehicle, e.g. a driving noise or engine noise, e.g. also with regard to predefined optimization criteria, is or was not sufficient, and then: selecting a measure with regard to the audio signal to be output.

[0102] For example, a measure of the appropriate applicability of the adaptation of the audio signal and / or a measure of the probability of success can be determined at least partially in advance, i.e., before or partially during the output of the audio signal. A measure of the probability of success can also be determined with respect to a section of the audio signal that is already being output or has already been output. For example, a measure of the applicability or probability of success can represent a, in particular anticipated, achievement of an optimization criterion, e.g., with respect to the perceptibility of the relevant section of the auditory and / or audio signal, e.g., from one or more specific seats of the vehicle.

[0103] Depending on the determined measure of applicability and / or probability of success, various parameters of the audio signal described in this document, e.g., the logic for adapting the audio signal, can be adjusted. For example, such a measure can include varying the timing of the output of at least a portion of the audio signal and / or a repeated output of a portion of the audio signal.

[0104] Alternatively or additionally, at least one time parameter can be determined for outputting at least a portion of the audio signal, which time parameter characterizes an (absolute or relative) point in time, in particular a beginning and / or end or the duration of a time interval, which is (explicitly) unsuitable for outputting the audio signal, in particular a specific portion of the audio signal or an audio signal with a specific content, for example, taking into account at least one optimization criterion. The output of at least a portion of an audio signal can then take place taking into account such a time parameter.

[0105] Preferably, said adaptation of the audio signal of the vehicle takes place as a control of one or more parameters of said audio signal, wherein a target value of the control is selected, in particular continuously adjusted, depending on a predetermined audibility and / or acoustic perceptibility of a content represented by the audio signal and / or at least a section of the auditory communication.

[0106] According to a further optional embodiment, the method comprises determining an applicability, in particular a probability of success, that the parameters of the audio signal were or are not sufficiently modifiable for certain parameters and / or content of the spoken language, in particular taking background noise into account. Subsequently, a measure or combination of measures is selected with respect to the vehicle's audio signal, in particular for modifying the vehicle's audio signal, and / or a renewed playback of at least a portion of the audio signal. The parameters for further or renewed adaptation of the audio signal can also be determined on this basis.

[0107] According to a second aspect, the invention is characterized by a device for controlling an audio system of a vehicle, wherein the device is designed to carry out the method. The device according to the invention has the same advantages as those described above in connection with the method according to the invention.

[0108] In an expedient embodiment, the device has further means for carrying out the method.

[0109] According to a third aspect, the invention is characterized by a mobile user device which comprises an interface with the vehicle and is configured to carry out at least part of the method.

[0110] According to a fourth aspect, the invention is characterized by a computer program, in particular a computer program product comprising the computer program, wherein the computer program is designed, when executed on a computing unit, to carry out an essential part of the method according to the invention or an advantageous embodiment of the method according to one or more further features of the method. In particular, the computer program is a computer program that can be run, for example, as an app (= "application") on a device installed in the vehicle or carried along, comprising at least one computing unit.

[0111] The computer program or computer program product comprises an executable program code which, when executed by a device comprising a computing unit, executes at least part of the method according to an aspect described herein or an advantageous embodiment of the method described herein. The computer program product can be embodied as an update of a previous computer program, which, for example, comprises the parts of the computer program or the corresponding program code for a device in the vehicle as part of a functional expansion, for example, as part of a so-called "remote software update."

[0112] For the purposes of this description, the vehicle is preferably a motor vehicle, in particular a passenger car (= passenger car). This is advantageous because the interior of such a vehicle forms a comparatively small, acoustically largely insulated or enclosed space in which multiple auditory communications and one or more audio signals are to be generated and heard within a confined space. In principle, however, the method can also be applied to other types of vehicles, e.g., a bus, train, watercraft, or aircraft, possibly in an appropriately adapted form.

[0113] Exemplary embodiments of the invention are explained in more detail below, without limiting the scope, using the schematic drawing. It shows: Fig. 1 A flowchart for controlling a vehicle audio system.

[0114] The Fig.1 shows a flowchart for controlling a vehicle audio system.

[0115] The program can be executed by a single device or distributed among multiple devices. For this purpose, the device comprises, in particular, a computing unit, a program and data memory, and, for example, one or more communication interfaces.

[0116] The program is started in a step S1, in which variables can be initialized if necessary.

[0117] In a step S3, a parameter of an auditory communication in the interior of a vehicle is determined, in particular predicted.

[0118] In this example of the method, the voice of a person (a speaker) in the vehicle interior is distinguished from content comprising a human voice output in the vehicle, e.g., from the vehicle's audio system or from another audio output device, in particular another mobile user device. Such a distinction can be made in the method depending on a comparison of a signal from the audio system with the signal from a microphone.

[0119] The said parameter of auditory communication can comprise a temporally ordered sequence of multiple parameter values, e.g., with respect to the recent past, and / or the current time, and / or a prediction for the near future. For example, a parameter of auditory communication can also represent a measure of the change in the values ​​of a specific parameter value over time and / or with respect to a spatially related quantity (spatial dimension, distance unit, position value, etc.), e.g., direction or distance.

[0120] At least one parameter of auditory communication can be determined in the method based on sensor data, a history, e.g., in connection with one or more communications in the vehicle interior or communications involving specific persons, and / or taking into account a probability calculation. For example, at least one parameter of auditory communication can be predicted, in particular based on a comparison or calculation of a plurality of parameters from the history.

[0121] The recognition of keywords or specific communication sounds (exclamation, interjection, tongue clicking, screaming, laughter, sobbing, crying, etc.) can also be taken into account. In particular, at least one parameter of the audio signal to be output in the interior can be adjusted, particularly proactively, depending on the recognized gesture and a recognized keyword. Preferably, the method can determine and consider a degree of correspondence between the at least one recognized gesture and at least one keyword or communication sound.

[0122] In a step S5, at least one parameter of an audio signal of the audio system of the vehicle is changed depending on the determined, in particular predicted, parameter of the auditory communication.

[0123] In this example, the change includes an adjustment of the audio signal, in particular individual channels of a surround sound signal, proportions between individual channels, instructions for processing (e.g. mixing) individual channels of the audio signal, also depending on certain optimization criteria.

[0124] The optimization criteria (with regard to the process) can include, for example, the lowest possible overall noise level in the interior, especially at certain seating positions, improved subjective perceptibility and / or intelligibility of communication, and / or the content of the audio signal. These can be changed, for example, via user settings.

[0125] For example, depending on the at least one determined parameter of the auditory communication in the interior, an adjustment of one or more parameters which are characteristic of the spatial distribution of one or more audio signals to be output, in particular audio signals with different contents and / or from different audio sources and / or several channels of a surround sound signal, within the interior of the vehicle takes place.

[0126] The sound irradiation with specific audio signals, of which there may be not just one but several, can also be adjusted by selecting the loudspeakers or optimizing the balance between specific loudspeakers ("left," "right," "center," "front," "rear") in the vehicle interior. Furthermore, alternatively or additionally, the phase parameters of at least one signal component of an audio signal can be adjusted in such a way that a passenger has the subjective impression that the sound source corresponding to a (specific) part or section of audio content is coming from a specific direction and / or is located and / or moving at a specific distance from the passenger.

[0127] The parameters of at least one or two of the audio signals to be output can be adjusted in such a way that an (optimized) distribution of the two different audio contents represented therein within the interior of the vehicle results. In particular, a distribution or mapping of the audio content components to specific spatial areas and / or directions (possibly provided for in a surround sound signal of the audio source) can be deviated from.

[0128] Furthermore, the method can also consider the propagation, attenuation, and reflection properties of acoustic waves, particularly from multiple frequency ranges in the vehicle interior. The corresponding parameters, e.g., a data set representing an acoustic model of the vehicle interior, can be imported into the method and taken into account.

[0129] Furthermore, the method can take into account an association between the (preset, determined, or estimated) interests of the occupants, e.g., of certain persons, occupants belonging to certain groups of persons on the one hand, and the types and / or contents of the audio signal on the other hand.

[0130] A measure of the relevance of certain sections of the audio signal for specific occupants, e.g., those belonging to specific groups of people or occupants in specific seats of the vehicle, can also be determined and taken into account. For example, at least one section of the audio signal can be adapted to take into account the age (baby, child, teenager, adult, senior citizen, etc.) and / or gender of an occupant. In this process, the specific properties of auditory perception for a specific occupant, e.g., occupants belonging to a specific group of people, such as phenomena or differences known from psychoacoustics, can also be determined and taken into account.

[0131] In a simplified case, the interests and / or specific characteristics of perception can be determined, in particular estimated, from a stored or determined user profile. This can be done, for example, in the form of deviations or differences, e.g., compared to standardized values.

[0132] In an alternative or additional variant of the invention, it is provided that, depending on the recognized topic, intonation, and direction of communication of the specific occupants involved in the communication, a decision is made to output one of two or more different audio signals, e.g., from different audio signal sources and / or with different content. For example, switching between at least two different audio signals, in particular with significantly different content, topics, timbres, or from different audio sources, can be performed.

[0133] For example, if spouses or partners are talking about a relationship topic, a topic concerning children, a typical argument topic, and the corresponding (typical, objectively measurable) characteristics or parameters of this topic are recognized, a section of the audio signal with a suitable content, e.g. a quieter music track or program (or a section of the audio signal that can produce a desired psychological effect) is played out, in particular instead of a previously played or planned content, e.g. a more hectic content.

[0134] Optionally, a physical movement (more or less conscious for the person themselves, or carried out intuitively or reflexively) that indicates the intention to listen in a specific direction, e.g., comprising a turn of the head or a hand on the right or left ear, can be detected in the method and processed in the method. In one example, a parameter, in particular representing the signal volume of the audio signal to be output from the respective direction or the content from a specific direction in question, can be changed, in this example, increased, and / or a section of the audio signal, e.g., the last sentence or phrase, can be repeated.

[0135] For example, the output audio content can be redistributed between at least two spatial areas of the vehicle, for example by the front left area being exposed to a specific audio content more loudly and / or with a different frequency characteristic than the rear right area. This can therefore be distributed between the front and rear seats, between the driver's seat and the other seats, or between the right and left halves of the interior. This can, for example, improve the auditory communication of an occupant occupying the front passenger seat and addressing occupants in the back seat, for example by an engine noise or driving noise that is or is to be output via the rear loudspeakers being significantly attenuated in the area of ​​the back seat and / or being output with different parameters. The engine noise or driving noise output via the loudspeakers facing the driver's seatDriving noise, however, may persist.

[0136] In this case, for example, so-called sound sources for at least two audio signals corresponding to different audio contents and / or signal sources can be controlled, in particular regulated, depending on the at least one determined position of a speaker (e.g. of the occupant currently communicating aurally).

[0137] For example, an assignment of one or more channels of a surround sound signal to specific physical loudspeakers and / or virtual sound sources can be adjusted depending on the head position and / or head direction and / or viewing direction of the occupant.

[0138] In particular, a temporal variation of the spatial distribution of the audio content (a time-changing, adaptive spatial distribution) can be carried out.

[0139] Varying the spatial distribution of the audio content can occur depending on a detected change, in particular a prediction of a change in at least one parameter of the auditory communication.

[0140] In particular, the method adapts a subjectively perceived direction and / or distance, and / or a subjectively perceived location at which a certain content represented by the audio signal is perceived (heard) by the human (binoural) acoustic perception.

[0141] For example, the sound sources corresponding to the specific contents of the audio signal can be varied with regard to their perceived position and / or direction and / or distance.

[0142] In the method, one or more phase parameters, in particular a phase offset of at least one signal component of the parts of the audio signal or of two audio signals that are or are to be output from at least two different loudspeakers in the interior, can be appropriately adapted.

[0143] In particular, a control, preferably a regulation, of a parameter of one or more audio signals to be output or channels of a surround sound signal to be output can be carried out depending on the at least one parameter of a current or predicted auditory communication. The parameters of the audio signal can also be adjusted in such a way that a so-called envelope and / or an integrated value and / or a low-pass filtered value of at least one parameter results within predetermined limits.

[0144] For example, the audio signal to be output in the interior of the vehicle can be adapted differently, in particular more strongly, to the speech and / or a detected or predicted parameter of auditory communication of the at least one first occupant, depending on which seat he is sitting in the vehicle and / or in which direction the first occupant is communicating, than to the speech and / or an expected parameter of the auditory communication of a further, second occupant in the interior of the vehicle who is located in a different seat within the interior.

[0145] The program is terminated in step S7 and can be started again in step S1 if necessary.

[0146] The procedure may optionally include one or more of the following steps: A step in which the change in the parameter of the audio signal relates to at least one surround sound parameter, wherein, preferably, at least one direction from which the audio signal of the audio system of the vehicle comes or is heard, in particular heard by the driver of the vehicle, is adjusted.

[0147] A step in which the parameter of the audio signal changes: a modification of the audio signal in the frequency domain such that signal components of the audio signal which relate to the frequency range of human speech, in particular within the speaker's voice, are modified such that the signal energy is distributed over one or more frequency ranges which do not relate to the frequency range of the human voice, in particular the speaker's voice, or relate to it to a lesser extent.

[0148] A step in which the change in the parameter of the audio signal comprises an at least partial shift of the frequencies of the audio signal, which at least statistically correspond to the frequencies of the speech signal of the speaker, from a first spatial direction in which the speaker is located to a second spatial direction which is arranged differently than the first spatial direction, in particular substantially opposite to the first direction.

[0149] One step comprehensive: - learning a specific voice and / or a specific physical movement of a speaker at one or more time intervals, and then: - upon recognition of a specific voice and / or physical movement of a speaker at one or more later time intervals: - Applying an optimized or pre-stored audio signal adjustment for the speaker's voice.

[0150] Learned voices can be represented as one or more parameter values ​​typical for the voice. Based on such parameters, recognition data for one or more specific voices can be determined or processed in the process. For example, the recognition data for one or more voices can also be obtained based on data from a voice control system, a vehicle's hands-free system, or a mobile user device (e.g., during a phone call).

[0151] One step comprehensive: - learning a specific physical movement that precedes speaking, especially in the case of a specific occupant, a verbal communication at one or more time intervals, and then: - when recognizing the physical movement, in particular when recognizing the specific speaker and / or the physical movement typical for the speaker from several time intervals: - Applying an optimized or pre-stored audio signal adjustment for the speaker's voice.

[0152] In particular, the learning of voices and / or physical movements relates to certain characteristics that are characteristic of certain inmates, in particular inmates belonging to certain groups of people, e.g. corresponding qualitative and / or quantitative values ​​or relationships between such.

[0153] Preferably, learning refers to the determination of a (scientific, mathematical) relationship between a parameter of one or more physical movements that depend on certain patterns of gestures and / or facial expressions, and an accompanying or subsequent auditory communication, in particular the speech of a person (occupant). This can be learned in the method with reference to a specific person or with reference to a person with certain, e.g., sensory-detectable characteristics (body size, clothing, movement parameters, group of people, etc.) or with reference to a person at a specific spatial position and / or at a seat in the vehicle.

[0154] According to the invention, said learning may further comprise: detecting a feedback indicator which is characteristic of an at least one, expedient, suitable or satisfactory adaptation of the audio signal, and then taking the feedback indicator into account when the method is repeatedly executed.

[0155] One step comprehensive: - detecting a communication intention and / or a voice of at least one, in particular a specific occupant of the vehicle and / or an occupant at a specific seat of the vehicle, and - Changing the audio signal depending on the detected communication intention and / or the voice of the occupant.

[0156] In this example, the communication intention can represent a probability measure that a verbal communication can be expected, in particular from a specific occupant or from a specific area of ​​the vehicle's interior, in particular with a certain acoustic parameter. Alternatively, at least a probability threshold can be specified, whereby only a communication intention above the threshold is considered.

[0157] A step in which it is also recognized that the recorded voice is a personal voice, in particular a voice of a specific person or group of people, and the modification of the audio signal occurs depending on the recognition of the voice as a personal voice, in particular depending on the recognition of the voice of a specific person or group of people.

[0158] A step in which the voice of a speaker in the interior of the vehicle is distinguished from a voice from a multimedia player, in particular the vehicle's sound system or a multimedia player connected to the vehicle.

[0159] A step in which the communication intention of at least one occupant of the vehicle is detected by detecting his lip movement and / or a physical movement, in particular by means of an interior camera.

[0160] A step in which the audio signal is adapted depending on the position of the speaker and / or a relative direction from which the spoken content comes or will come according to the communication intention.

[0161] A step in which an emotion recognition and / or intention recognition of an occupant of the vehicle is recognized and taken into account in the method, whereby the audio signal is then changed, preferably in a different way.

[0162] One step comprehensive: - Determining the applicability, in particular the probability of success, that the parameters or contents of the spoken language and the parameters of the audio signal are not sufficiently modifiable, in particular with regard to background noise, and then: Selecting an action or combination of actions relating to the vehicle's audio signal, in particular to modify the vehicle's audio signal.

Claims

[1] Method for controlling an audio system of a vehicle, comprising: In particular, they determine prediction, a parameter of auditory communication in the interior of a vehicle, and - Changing at least one parameter of an audio signal of the vehicle's audio system depending on the determined, in particular predicted, parameter of auditory communication, where - the audio signal is multimedia content played via a vehicle's multimedia system, part of the vehicle's sound design and / or an artificially generated engine noise, and includes the modification of the audio signal's parameter - a modification of the audio signal in the frequency domain such that signal components of the audio signal which affect the frequency range of the speaker's voice are modified in such a way that the signal energy is distributed over one or more frequency ranges which do not affect the frequency range of the speaker's voice or affect it to a lesser extent. [2] Method according to claim 1, wherein the change of the parameter of the audio signal relates to at least one spatial sound parameter, wherein, preferably, at least one direction from which the audio signal of the audio system of the vehicle comes or is heard, in particular heard by the driver of the vehicle, is adjusted. [3] Method according to one of the preceding claims, wherein the change of the parameter of the audio signal comprises at least a partial shift of the frequencies of the audio signal, which correspond at least statistically to the frequencies of the speech signal of the speaker, from a first spatial direction in which the speaker is located to a second spatial direction which is arranged differently from the first spatial direction, in particular substantially opposite to the first direction. [4] Method according to any of the preceding claims, including comprehensive methods: - the learning of a specific voice and / or a specific physical movement of a speaker at one or more time intervals, and subsequently: - in the case of recognition of a speaker's specific voice and / or physical movement at one or more later time intervals: - Application of an optimized or pre-stored adjustment of the audio signal for the speaker's voice. [5] Method according to any of the preceding claims, including comprehensive methods: - the learning of a specific physical movement that precedes speech, particularly in a specific inmate, or linguistic communication at one or more time intervals, and subsequently: - in the case of recognition of the physical reaction, especially also in the case of recognition of the specific speaker and / or the physical reaction typical for the speaker from several time intervals: - Application of an optimized or pre-stored adjustment of the audio signal for the speaker's voice. [6] Methods according to any of the preceding claims also comprehensively: - Recording a communication attempt and / or a voice of at least one, in particular a specific, occupant of the vehicle and / or an occupant in a specific seat of the vehicle, and - Changing the audio signal depending on the detected communication attempt and / or the voice of the occupant. [7] Method according to claim 6, in which it is also recognized that the recorded voice is a personal voice, in particular a voice of a specific person or a specific group of persons, and the modification of the audio signal is dependent on the recognition of the voice as a personal voice, in particular dependent on the recognition of the voice of a specific person or a person belonging to a group of persons. [8] Method according to one of claims 6 or 7, wherein the voice of a speaker inside the vehicle is distinguished from a voice from a multimedia playback device, in particular the vehicle's sound system or a multimedia playback device connected to the vehicle and carried along with the vehicle. [9] Method according to any one of claims 6 to 8, wherein the detection of the communication intention of at least one occupant of the vehicle is carried out by means of recognizing his lip movement and / or a physical movement, in particular by means of an interior camera. [10] Method according to any one of claims 6 to 9, wherein the audio signal is adapted depending on the position of the speaker and / or a relative direction from which the spoken content comes or will come according to the communication plan. [11] Method according to one of claims 6 to 10, wherein an emotion recognition and / or intention recognition of an occupant of the vehicle is recognized and taken into account in the method, wherein the audio signal is then changed, preferably in different ways. [12] Method according to any one of claims 6 to 11, also comprising, - Determining the applicability, in particular the probability of success, that the parameters of the audio signal were or are not sufficiently modifiable for certain parameters and / or content of the spoken language, especially taking into account background noise, and subsequently: Selection of a measure or combination of measures relating to the vehicle's audio signal, in particular to modify the vehicle's audio signal and / or to replay at least a section of the audio signal. [13] Device for controlling an audio system of a vehicle, wherein the device is configured to perform a method according to any one of claims 1 to 12. [14] Mobile user device for controlling an audio system of a vehicle, comprising an interface with the vehicle and configured to perform at least part of the method according to any one of claims 1 to 12. [15] Computer program, in particular a computer program product comprising a computer program, wherein the computer program is configured to perform the method according to one of claims 1-12 when executed on a computing unit.

Images