Method, device, mobile user device, computer program for controlling an audio system of a vehicle

Abstract

Method for controlling an audio system of a vehicle, comprising the following steps: - Determining, in particular predicting, an acoustic content in the interior of a vehicle, and - Adapting a time parameter for outputting at least part of an auditory message depending on the determined, in particular predicted, acoustic content in the interior of the vehicle, wherein - the time parameter for outputting at least part of the auditory message indicates a time for output.

Description

[0001] The invention relates to a method, a device, a mobile user device and a corresponding computer program for controlling an audio system of a vehicle.

[0002] Document DE 10 2015 118 553 A1 describes a method for filtering noise in a vehicle. The methods include generating a microphone signal corresponding to noise in the passenger compartment. The methods also include receiving an audio-based service used by at least one occupant. A noise separation mode is selected from the plurality of noise separation modes. The input received by the microphone is filtered according to the selected noise separation mode to generate at least one filtered signal.

[0003] The document US 2015 / 0 310 865 A1 describes a speech recognition system comprising a microphone for receiving speech from a user and processing electronics.

[0004] The document DE 10 2015 206 021 A1 discloses a computer system for a vehicle, comprising a computing unit and an output unit. The computing unit comprises a virtualization layer, a central virtual management unit, and at least one virtual signal generator unit. The virtualization layer is configured to operate an operating system of the central virtual management unit on the computing unit in parallel with a respective operating system of the at least one virtual signal generator unit. The at least one virtual signal generator unit is configured to provide a signal representative of an audio signal to be output. The central virtual management unit is configured to control the output of a respective signal of the at least one virtual signal generator unit via the output unit.

[0005] A vehicle's audio system usually includes a volume control. Audio systems are known to increase in volume as background noise increases.

[0006] A vehicle interior is a zone with numerous, more or less desirable, mutually overpowering auditory contents. On the one hand, there is a need to provide the vehicle's occupants with as much audible information as possible in order to meet their information needs. On the other hand, there is typically a need or necessity for the vehicle's occupants to communicate with each other. Such communication, especially verbal communication, can often be of great relevance to the vehicle's occupants.

[0007] The object underlying the invention is to provide a method, a device, a mobile user device and a corresponding computer program for generating auditory messages.

[0008] The problem is solved by the features of the independent patent claims. Advantageous embodiments are characterized in the subclaims.

[0009] According to a first aspect, the invention is characterized by a method for controlling an audio system of a vehicle. The method involves determining, in particular predicting, an acoustic content in the interior of a vehicle. A time parameter for outputting at least part of an auditory message is determined, in particular adjusted, depending on the determined, in particular predicted, acoustic content in the interior of the vehicle.

[0010] Preferably, one or more parameters of the acoustic content are determined, in particular predicted. Two or more different acoustic contents, in particular acoustic contents of different types, can be taken into account.

[0011] A parameter of the acoustic content can comprise a sequence, e.g. ordered in time, of several parameter values, e.g. in relation to the recent past, for example to the current point in time, and / or a prediction for the near future. For example, a parameter of the acoustic content can also represent a measure of the change in the values ​​of a certain parameter value over time and / or in relation to a spatially related quantity, e.g. direction or distance. At least one parameter of the acoustic content can be determined in the method on the basis of sensor data, a history, e.g. in connection with one or more communications in the vehicle interior or communications involving specific people and / or taking into account a probability calculation. Preferably, at least one parameter of the acoustic content can be determined, in particular on the basis of a comparison orCalculation of a large number of parameters from history can be predicted.

[0012] The time parameter for outputting at least part of the auditory message indicates an (absolute or relative) time for output, in particular a beginning and / or end of the output of the part of the auditory message, a duration, preferably a specific time interval, for outputting the part of the auditory message. Particularly preferably, said time parameter represents at least the time for outputting at least part of the auditory message. However, several more or less suitable times for outputting two or more parts of the auditory messages can also be determined and considered.

[0013] Alternatively or additionally, the time parameter can refer to at least one event, in particular an event related to a given driving context, a given vehicle route, e.g., reaching a route section, or a predetermined event. The time parameter indicates a time frame in relation to a specific event or an event of a specific type. For example, said time parameter can represent that a part of the auditory message should not be output earlier than a certain time value and / or later than a certain time value in relation to an event.

[0014] In particular, depending on a determined (in particular predicted) acoustic content in the interior of the vehicle, one or more time intervals, e.g. represented by a time and / or duration, can be determined which are suitable and / or unsuitable for outputting at least one (specific) part of the auditory message and / or one or more parts of an auditory message with (each) a specific time duration.

[0015] The time parameter for outputting one or more parts of the auditory message can then be determined depending on the determined time intervals and whether these are (more or less, particularly in a quantitative measure) suitable and / or unsuitable for outputting a (certain) part of the auditory message. Further predetermined criteria, e.g., optimization criteria, can be taken into account.

[0016] In a simplified case, the “suitable times” or intervals can be determined from the times or intervals that are not suitable.

[0017] In this case, at least one time parameter can be determined for outputting at least part of the auditory message, which time parameter characterizes an (absolute or relative) point in time, in particular the beginning and / or end of the output of the part of the auditory message. Particularly preferably, the time parameter can comprise the point in time for the beginning and / or end of the output of a part of the auditory message and a value characterizing the duration of a corresponding (suitable) time interval.

[0018] The time parameter for outputting at least part of the auditory message can, in the method, relate in particular to the output of a specific part of the auditory message and / or to a part of the auditory message with a specific content and / or with a specific type of content.

[0019] In particular, it is characteristic of said auditory message that it includes content to be interpreted by at least one occupant of the vehicle, for example, with regard to a current journey, a state of the vehicle, or a driving context. In particular, the auditory message is generated within the vehicle based on information from an on-board vehicle network.

[0020] In the process, the auditory message can be distinguished, in particular explicitly distinguished, from other content to be output in the vehicle, for example, content transmitted or played from a data carrier or data stream.

[0021] For example, the auditory message can comprise a predefined section of an audio signal or a combination of sections of audio signals that represent, for example, driver information, a warning, confirmation, or incorrect operation of the vehicle. Preferably, an auditory message can be a (particularly automatically, situation-dependent) generated voice message within the vehicle.

[0022] By means of the method, an auditory message is adapted to a current or predicted acoustic content in the interior of the vehicle, in particular with respect to a time interval in the near future, so that the auditory message and / or the acoustic content can be better perceived.

[0023] The procedure can be geared toward achieving specific goals. The procedure thus results in the following advantages and achievable goals: - Both a speaker and actual or potential listeners have to make less effort. - Artificially generated voice messages and content spoken by inmates are coordinated in time. - the sequence of at least two auditory messages or individual contents of the auditory message is optimized so that no crosstalk occurs and the pauses in the speech flow of the occupants are optimally utilized. - The procedure also helps if a passenger in the back seat wants to communicate something to the driver or other passengers without, for example, the auditory message of a voice computer being “lost”.

[0024] For example, announcements from a navigation system, announcements from the on-board computer about the condition of the vehicle or current information about certain hazards can be perceived as an auditory message.

[0025] In particular, the auditory message relates to acoustically emitted sounds or sections of linguistically represented messages which are intended not only for mere perception (hearing, such as music), but for the interpretation of the corresponding message, at least for understanding and / or differentiation and / or classification by at least one occupant, e.g. by the driver.

[0026] The auditory messages can be generated or acoustically output depending on a predetermined condition. For example, auditory messages can include auditory, particularly verbal, information that is generated depending on the vehicle's driving context (e.g., traffic signs, traffic jam reports, other road users) and / or depending on the driver's state (drowsiness detection, distraction) and / or driver actions (indications of possible consequences of an action).

[0027] Furthermore, an automated reading out of texts, for example incoming SMS, e-mails, or entries in social networks (posts, tweets, etc.) and the like in the interior of the vehicle, using the computer (on-board computer) can also be regarded as auditory messages.

[0028] An auditory message, or the information represented therein, can also be intended, at least primarily, for a specific occupant, e.g., the driver of the vehicle or a specific other occupant (passenger). An auditory message can be driver information that, for example, concerns recommended actions, instructions, status messages in connection with at least partially automated driving and / or a maneuver that can be performed automatically or by the driver, e.g., a parking maneuver of the vehicle. Auditory messages can, for example, be a human-machine interface of a digital, artificial intelligence-based vehicle assistant and / or a "co-pilot." Such a "co-pilot" can also be designed to co-steer the vehicle.

[0029] In addition, the term auditory messages also includes other, particularly interactively generated voice messages, which, for example, serve for interactive conversation between occupants using technical means. For example, an auditory message issued in the vehicle can be an answer, e.g. information or a reply to a question posed by an occupant, in particular by the driver, or a gesture or facial expression executed through body language. The auditory message preferably comprises at least one verbal message generated by means of speech synthesis or from a database with prefabricated sounds, words, sentence segments or sentences. Auditory messages can comprise linguistic structures generated, e.g., by means of artificial intelligence, and preferably also non-prefabricated or predetermined ones. In a simplified case, the auditory message comprises information, e.g.a section of text is output acoustically in a language preset (in the computer or vehicle).

[0030] A further expedient embodiment provides that the type and / or content of a further acoustic content that is audible in the interior of the vehicle and that differs from information represented linguistically is determined and processed as a function of an analysis result to determine, in particular to adapt, a time parameter for outputting at least part of an auditory message to be output.

[0031] This design provides for the auditory message to be adapted depending on the type of one or more other acoustic contents audible in the vehicle.

[0032] The type of audio content can be divided into the following or similar categories: music track; voice program; telecommunications content (phone call, read-aloud SMS, messenger, etc.). The audio message can be adapted depending on the content of this additional audio content in the vehicle or the vehicle.

[0033] The acoustic content that is audible in the interior of the vehicle can be recorded in the process, e.g. acoustically and / or as an audio signal to be output or data to be output, and divided into the following or comparable classes: beginning / end of a sentence and / or phrases of a multimedia program (radio broadcast) comprising a language; news reports, breaking news, entertainment programs; voice messages, telecommunications content, e.g. relating to a telephone call conducted by an occupant with or without a hands-free device, a telephone or video conference or the like.

[0034] According to an optional embodiment, the acoustic content comprises an auditory communication and / or a voice of at least one, in particular a specific, occupant of the vehicle and / or an occupant at a specific seat.

[0035] According to a further optional embodiment, the auditory message is paused depending on an existing or predicted communication intention of the auditory communication and / or the voice of at least one, in particular a specific, occupant of the vehicle and / or an occupant at a specific seat.

[0036] For example, the auditory message, such as a long announcement from an on-board computer, including the navigation system, or a voice control system, is paused or (if necessary in advance, ie before it is output) split into two parts.

[0037] Particularly preferably, the auditory message is continued, in particular the auditory message is continued in a corresponding manner, if it is determined that there is no existing or predicted communication intention of the auditory communication and / or the voice of at least one particular occupant of the vehicle and / or of an occupant at a particular seat.

[0038] In other words, the acoustic content in the interior of the vehicle in this case comprises an auditory communication and / or a voice of at least one, in particular a specific, occupant of the vehicle and / or an occupant at a specific seat, wherein the auditory message is paused depending on the existing or predicted communication intention of the auditory communication and / or the voice of at least one, in particular a specific, occupant of the vehicle and / or an occupant at a specific seat.

[0039] The time parameter of the output of at least part of the auditory message can be adapted differently to the determination of an existing, predicted communication intention of the auditory communication and / or the voice of at least one occupant, depending on the specific occupant of the vehicle and / or the specific seat in which the occupant is sitting.

[0040] According to a further optional embodiment, the content of the auditory message is divided into at least two parts depending on a determined existing or predicted communication intention of the auditory communication and / or the voice of at least one particular occupant of the vehicle and / or an occupant at a particular seat.

[0041] For example, the content of the auditory message is divided into two or more parts, each of which corresponds to the predicted pauses in the occupants' auditory communication.

[0042] In particular, the respective content to be output in the form of the auditory message can be broken down into two parts (possibly in advance, i.e., before it is output). These two parts can be complete sentences or phrases that make sense on their own, or do not significantly change the content of the communication message to be conveyed.

[0043] For example, the vehicle's on-board computer can calculate in advance which content must be output as an auditory message and distribute this across the existing or predicted pauses in auditory communication in the vehicle (sensible, appropriate in terms of content).

[0044] According to a further optional embodiment, the order of at least two auditory messages and / or individual contents or at least one section of the auditory message is changed depending on the existing or predicted communication intention of the auditory communication and / or the voice of at least one, in particular a specific, occupant of the vehicle and / or an occupant at a specific seat.

[0045] In particular, the method can predict time intervals suitable for outputting, in particular, certain auditory messages, depending on the prediction of the acoustic content in the vehicle interior, in particular taking into account at least two different, preferably all essential acoustic contents. Put simply, the said time intervals can, in particular, characterize pauses in which the auditory message can be output at a suitable time, preferably without temporal overlap and / or without a significant increase in volume.

[0046] In this case, shorter (predicted) pauses in the acoustic content in the interior of the vehicle, in particular in the speech flow of at least one occupant, can be used to output a shorter part of the auditory message and / or longer (predicted) pauses in the acoustic content in the interior of the vehicle, in particular in the speech flow of at least one occupant, can be used to output a longer part of the auditory message.

[0047] A further expedient embodiment provides that the order of at least two auditory messages and / or individual sections or contents of a respective auditory message is changed depending on a current communication or a predicted communication intention and / or a voice of at least one occupant and / or an occupant at a specific seat. For example, the auditory messages to be output can be adapted such that the data sections belonging to different auditory messages and / or individual parts of a (split) auditory message are output in a changed order.

[0048] It is advisable to choose an order in which the content to be output in the form of the auditory message has to be broken down into as few parts as possible and / or a certain length of pauses is maintained between the verbal communication of the occupants in the vehicle and at least part of the auditory message.

[0049] A further expedient embodiment provides for a content-related feature of the spoken communication between at least two vehicle occupants to be determined and / or the type and / or priority of one or more auditory messages to be output. Subsequently, an order of the at least two auditory messages and / or individual contents of the auditory message, for example, the two sections (parts) of a split auditory message, is determined, depending on the determined content-related feature of the spoken communication and / or the determined type and / or priority of the at least two auditory messages to be output.

[0050] The adaptation of at least one auditory message can be carried out in the method as an optimization according to one or more predetermined criteria, e.g. the user settings.

[0051] Preferably, at least one optimization criterion for adapting at least one auditory message represents a minimization of a temporal overlap between the speaking of at least one occupant and the output of the auditory message.

[0052] A further optimization criterion can be at least one vehicle setting, a user setting, which can be assigned, for example, to a seat or a passenger's user profile. This can represent, for example, the extent to which auditory communication, particularly from a specific direction, and / or the extent to which a particular auditory message should be audible. Certain perceptual characteristics of at least one specific person, e.g., parameters representing an audiogram of a specific passenger, can also be taken into account.

[0053] This means that the timeline can be used optimally and as smoothly as possible for the output of the auditory message.

[0054] According to a further optional embodiment, a prediction of a temporal parameter, in particular a temporal progression of a speaker's voice, is also carried out, at least with regard to: - a beginning, and / or - an end, and / or - a temporal progression of the amplitudes and / or frequency parameters, particularly with regard to sentences or phrases, and is taken into account when generating the auditory message.

[0055] Particularly preferred is then a predictive adjustment of at least one auditory message.

[0056] In this case, predicting the temporal progression of the speaker's voice, especially acoustic parameters of the voice, can also be done by analyzing physical movements, facial expressions, lip movement patterns, and body language.

[0057] For example, if it is recognized that the speaker has finished his sentence and / or phrase and / or message, the adaptation of the auditory message can be reversed, especially predictively.

[0058] According to a further optional embodiment, the method comprises determining the type and / or priority of the acoustic content in the interior of the vehicle, and / or determining the type and / or priority of the auditory message to be output and then determining, in particular adapting, the time parameter of the output of the auditory message depending on the determined type and / or priority of the acoustic content in the interior of the vehicle and / or the auditory message to be output.

[0059] In other words, different acoustic contents and / or different auditory messages result in different adaptations, in particular different pausing and / or different division of the auditory message.

[0060] For example, depending on the type of detected or predicted acoustic content in the vehicle interior, the auditory message can be: - an immediate pause and / or - pausing after the end of a set and / or - a division of the content of the auditory message.

[0061] For example, if it is determined that the acoustic content is a voice message from an occupant and / or a noise in the interior and / or multimedia content in the interior and / or a sound from a user device carried on board, e.g., a cell phone ringtone, the auditory message is subsequently generated differently, in particular varied and adapted. This can, for example, be paused accordingly, and / or be shorter, in particular contain less detailed information, and / or be generated as if spoken more quickly, and / or be split into at least two parts.

[0062] Preferably, a first characteristic value of the priority of the auditory message and / or a second characteristic value of the priority of an acoustic content in the interior of the vehicle, in particular indicative of their respective meaning or importance and / or indicative of their respective urgency or urgency, can be taken into account in the method.

[0063] For example, determining a high priority for current or predicted acoustic content in the vehicle interior, e.g., "important" and / or "urgent," can lead to a significantly greater adaptation of the auditory message than for acoustic content with a medium or lower priority. For example, if a temporal overlap is detected between acoustic content of a certain type and a specific auditory message, the auditory message can be output at an earlier or later time and / or in an abbreviated form and / or with a modified acoustic parameter.

[0064] For example, playback of multimedia content, such as a piece of music, can be paused or not started at all. However, if the audible message is an urgent audio message being played in the vehicle, the audible message will not be varied, or a different variation, such as an amplification, will be made to the audible message.

[0065] According to a further optional embodiment, the acoustic content is a multimedia content and a prediction of the acoustic content is carried out by means of an analysis of a data stream streamed, in particular from a database and / or a cloud, or of a content read from a data carrier before its playback.

[0066] In other words, the prediction of the acoustic content can, for example, consist of or be carried out by analyzing the content of a streamed data stream or content read from a data storage device before its playback. For example, such a data stream (read from the internet, a cloud, or a CD / DVD) is present in a corresponding device, computer, storage, or RAM area at least a few seconds before its playback. In the invention, this data stream is also analyzed (at least roughly, e.g., based on the amplitude curve). The system can then look at least into the near future and plan the auditory messages accordingly.

[0067] According to a further optional embodiment, it is also recognized that a recorded voice is a personal voice, in particular a voice of a specific person or a voice belonging to a specific group of people within the vehicle, and the generation, in particular variation of the auditory message takes place depending on the recognition of the voice of a specific person or a voice belonging to a specific group of people.

[0068] In other words, the system reacts specifically to personal voices, and not to other noise sources, such as the barking of a dog or the honking of other road users, or to background noise, such as wind noise or specific background noises. Alternatively, the system reacts to other sound sources in a fundamentally different way and / or with a different variation of the auditory message, e.g., by adjusting their parameters.

[0069] The voice of a specific person and / or the voice belonging to a specific group of people can be a learned voice. Preferably, the process can also learn the voices of specific people, and the auditory message can be adapted specifically to recognize these people's voices.

[0070] The learning of the voice (of a specific person) or the group of people can be done in the vehicle or outside the vehicle or using another device, e.g. a mobile user device.

[0071] Particularly preferably, a different adaptation (characteristic curve, mathematical relationship) of the auditory message can be carried out for at least two voices of two different persons.

[0072] In a further example, the at least one auditory message can be adapted differently, in particular much more strongly, to certain persons highlighted by a recognition or presetting, or to certain occupants in certain seats, e.g. with regard to the time of output of at least a section (part) of the auditory message.

[0073] For example, if a person authorized to lead, e.g. a manager or an operations manager, speaks to or at his employees or team during a journey, the procedure can be used to ensure sufficient silence or audibility of the verbal message.

[0074] The “groups of people” include, for example, family members, familiar voices, especially those learned in the vehicle, “women”, “men”, “children”.

[0075] According to a further optional embodiment, certain keywords, word sequences, and / or facial expression patterns of at least one occupant are learned. This is followed by recognition of a willingness to communicate and / or recognition of the relevance, in particular the urgency and / or importance, of a verbal message and / or determination of the adjustment of one or more parameters of the auditory message.

[0076] According to a further optional embodiment, the method comprises learning a specific voice and / or a specific physical movement of a speaker at one or more time intervals. Upon recognition of the specific voice and / or the specific physical movement of a speaker at one or more later time intervals, an optimized or pre-stored adaptation of the auditory message is applied to the speaker's voice.

[0077] Preferably, the following dependency arises: - If a physical movement, e.g. a certain lip movement pattern, is detected in a person who has always spoken quietly and / or unclearly in the vehicle, then the auditory message is adjusted comparatively strongly and to match the speech frequency or style of the speaker. - If a physical movement, e.g. a certain lip movement pattern, is detected in a person who has always spoken loudly and / or clearly in the vehicle, then the auditory message will not be heard or will be comparatively weak, and will be adjusted to match the speech frequency of the speaker.

[0078] Learned voices can be represented as one or more parameter values ​​typical for the voice. Based on such parameters, recognition data for one or more specific voices can be determined or processed in the process. For example, the recognition data for one or more voices can also be obtained based on data from a voice control system, a vehicle's hands-free system, or a mobile user device (e.g., during a phone call).

[0079] According to a further optional embodiment, the prediction of the acoustic content is carried out by means of a recognition of a communication intention of at least one occupant of the vehicle, preferably by means of the recognition of his lip movement and / or a physical movement, for example by means of an interior camera.

[0080] The method preferably comprises a comparison between a physical movement recorded in the past, e.g., certain facial expressions, gestures, or lip movements, and a subsequent communication. In particular, one or more parameters from the past can be taken into account that determine a causal relationship and / or a time interval between a specific physical movement and a subsequent verbal communication. Such a comparison can, in particular, apply to specific inmates or inmates belonging to specific groups of people.

[0081] Particularly preferably, corresponding qualitative and / or quantitative values ​​or relationships between the parameters of a physical stimulus, in particular a physical stimulus of a specific type, and at least one parameter of a subsequent verbal communication can be recorded and taken into account. A time for outputting at least part of an auditory message to be output can then be determined.

[0082] In particular, the process can also include the learning of voices and / or physical movements and take them into account. Such learning can refer to certain characteristics that are characteristic of certain inmates, especially inmates belonging to certain groups of people.

[0083] Preferably, learning refers to the determination of a (scientific, mathematical) relationship between a parameter of one or more physical movements that depend on certain patterns of gestures and / or facial expressions, and an accompanying or subsequent auditory communication, in particular the speech of a person (occupant). This can be learned in the method with reference to a specific person or with reference to a person with certain, e.g., sensory-detectable characteristics (body size, clothing, movement parameters, group of people, etc.) or with reference to a person at a specific spatial position and / or at a seat in the vehicle.

[0084] According to the invention, said learning may further comprise: detecting a feedback indicator which is characteristic of an at least one, expedient, appropriate or satisfactory adaptation of at least part of the auditory message, and then taking the feedback indicator into account when the method is repeatedly executed.

[0085] Preferably, the auditory message can also be adjusted in advance (before or shortly after the start of speaking). This has the advantage of preempting the person who is about to speak.

[0086] This can be done, for example, with the vehicle's interior camera, which records at least the driver of the vehicle (and possibly the passenger) and can detect corresponding movements.

[0087] One or more physical movements (from the analysis of body language and / or emotion evaluation) that precede speaking can also be recognized as a communication intention of auditory communication.

[0088] In this example, the communication intention can represent a probability measure that a verbal communication can be expected, in particular from a specific occupant or from a specific area of ​​the vehicle's interior, in particular with a certain acoustic parameter. Alternatively, at least a probability threshold can be specified, whereby only a communication intention above the threshold is considered.

[0089] It is particularly preferred that a smile and speaking and / or a frightened face and loud speaking or screaming are recognized as a condition for adapting the auditory message and that an adaptation, in particular an increased adaptation, of the auditory message then takes place.

[0090] Particularly preferentially, the auditory message can be switched off, paused, interrupted or adjusted more strongly when a frightened face and / or an argument or shouting is detected than when there is an average level of willingness to communicate and / or voice recognition.

[0091] According to a further optional embodiment, a voice of a speaker in the vehicle interior is differentiated from a voice from a multimedia playback device, in particular the said audio system, another sound system of the vehicle, or a multimedia playback device connected to the vehicle. For example, one or more (live) spoken voices in the vehicle interior can tend to lead to a stronger adaptation of the time parameter of the auditory message than music content of the same volume or a multimedia program including a human voice. For example, the adaptation to a voice spoken in the interior is significantly stronger than the voice of a presenter in a program.

[0092] Preferably, the voice of a speaker in the interior of the vehicle can be distinguished from the voice from a multimedia system by comparing the signals of a microphone system of the vehicle and a signal from the multimedia system of the vehicle.

[0093] For example, the voice of a speaker in the vehicle interior can be differentiated from a voice from a multimedia playback device, in particular the vehicle's audio system, another vehicle sound system, or an on-board audio playback device connected to the vehicle, by determining whether the voice of a speaker in the vehicle is only contained or recognizable in the vehicle's microphone system or whether the voice can (also) be recognized in an evaluated signal from a multimedia system. Such a comparison can, for example, include determining a similarity measure or correlation measure.

[0094] According to a further optional embodiment, the auditory message is adapted depending on a position of a speaker and / or a relative direction from which the acoustic content comes or will come according to a communication intention of the auditory communication.

[0095] For example, the vehicle's (ongoing or planned) auditory message can be adapted differently, in particular more strongly, to the driver's speech or willingness to communicate and / or the speech of a passenger in the front passenger seat than to the speech and / or willingness to communicate of a passenger in the back seat.

[0096] For example, people in the back seat cannot change a traffic announcement or can only do so in certain cases, while the passenger in the front seat has a greater influence.

[0097] According to the invention, the position of the speaker can be determined by comparing the signals of a microphone system in the vehicle (e.g. both microphones of a hands-free system) and / or seat occupancy detection and / or interior camera.

[0098] For example, the position of the speaker can be determined depending on a phase shift of an acoustic signal that is recorded, for example, by two spatially offset microphones in the vehicle.

[0099] According to a further optional embodiment, an emotion recognition and / or intention recognition of an occupant of the vehicle is recognized and taken into account in the method, whereby the auditory message is then adapted, preferably in different ways.

[0100] In this case, emotion recognition and / or intent recognition can be carried out, in particular, using vehicle resources, e.g., using an interior camera. For example, emotion recognition and / or intent recognition can involve evaluating the occupant's facial expressions and gestures over time, with or without other physical movements.

[0101] For example, if the occupant (speaker) is detected to be very excited, the auditory message can be adjusted earlier and / or more strongly.

[0102] The system can also be configured so that the output of auditory messages (also in a spoken form) creates the impression of another "speaking party" in the vehicle, behaving politely and courteously. Like a good conversation partner who perceives their fellow people and chooses an appropriate (starting) and / or (ending) time in a discussion to deliver their verbal message.

[0103] Furthermore, communication conducted by at least one inmate by telephone can also be considered a form of verbal communication in the process. This can lead to a similar or different adjustment of the audible message to be output, or the timing of at least a section (part) of the audible message.

[0104] The method can be applied, in particular simultaneously, to multiple verbal communications in the vehicle (e.g. between two or more occupants). For example, it can be applied to different communications taking place simultaneously or at different times. For example, communications conducted between different occupants and / or directed at different occupants can be regarded as different communications. Although this description refers to one communication, this can refer to one, in particular a specific, communication of multiple communications taking place or possible in the vehicle, e.g. a communication that directly concerns the driver of the vehicle (as "speaker" or "listener").

[0105] Furthermore, the audible message can be adapted depending on the type of one or more other audio signals emitted in the vehicle. The type of audio signal can be divided into the following or comparable types: music track; voice program; telecommunications content (phone call, read-aloud SMS, messenger, etc.). The audible message can be adapted depending on the content, in particular on one or more specific content features of another audio signal.

[0106] The content of the audio content, in particular audio content that is audible in the interior, in particular audio content that is currently present or output, can be recorded in the method, e.g. acoustically, as an audio signal to be output or data to be output, and divided into the following or comparable classes: beginning / end of a sentence and / or phrases of a language comprising multimedia program (radio broadcast); news reports, breaking news, entertainment program; voice messages, telecommunications content, e.g. relating to a telephone call conducted by an occupant with or without a hands-free device, a telephone or video conference and the like.

[0107] Furthermore, the method may also comprise determining a measure of the applicability and / or probability of success that the output of an auditory message, in particular taking into account a parameter of linguistic communication, is or was not sufficient, and then selecting a measure with respect to at least part of the auditory message.

[0108] Depending on the determined measure of applicability and / or probability of success, various features of the auditory message described in this document, e.g., one or more time parameters, acoustic parameters of at least a portion of the auditory message, etc., may be adjusted. For example, such a measure may include varying the timing and / or duration of the output of at least a portion of the auditory message and / or a repeated output of a portion of the auditory message.

[0109] Repetition also means that a new section of the auditory message is created again, or at least an insufficiently transmitted content is compensated, supplemented or replaced.

[0110] For example, a measure of the appropriate applicability of the adaptation of the audio signal and / or a measure of the probability of success can be determined at least partially in advance, i.e., before or partially during the output of said auditory message. A measure of the probability of success can also be determined with respect to an auditory message or a section of the auditory message that is already being output or has already been output. For example, a measure of the applicability or probability of success can represent a, in particular anticipated, achievement of an optimization criterion, e.g., with respect to the perceptibility of the relevant section of the auditory and / or audio signal, e.g., from one or more specific seats of the vehicle.

[0111] Furthermore, the method can also consider the propagation, attenuation, and reflection properties of acoustic waves, particularly from multiple frequency ranges in the vehicle interior. The corresponding parameters, e.g., a data set representing an acoustic model of the vehicle interior, can be imported into the method and taken into account.

[0112] According to a second aspect, the invention is characterized by a device for controlling an audio system of a vehicle, wherein the device is designed to carry out the method.

[0113] According to a third aspect, the invention is characterized by a mobile user device which comprises an interface with the vehicle and is configured to carry out at least part of the method.

[0114] According to a fourth aspect, the invention is characterized by a computer program, in particular a computer program product comprising the computer program, wherein the computer program is designed, when executed on a computing unit, to carry out an essential part of the method according to the invention or an advantageous embodiment of the method according to one or more further features of the method. In particular, the computer program is a computer program that can be run, for example, as an app (= "application") on a device installed in the vehicle or carried along, comprising at least one computing unit.

[0115] The computer program or computer program product comprises an executable program code which, when executed by a device comprising a computing unit, executes at least part of the method according to an aspect described herein or an advantageous embodiment of the method described herein. The computer program product can be embodied as an update of a previous computer program, which, for example, comprises the parts of the computer program or the corresponding program code for a device in the vehicle as part of a functional expansion, for example, as part of a so-called "remote software update."

[0116] For the purposes of this description, the vehicle is preferably a motor vehicle, in particular a passenger car (= passenger car). This is advantageous because the interior of such a vehicle forms a comparatively small, acoustically largely insulated or enclosed space in which multiple auditory communications and the auditory message are to be generated and heard within a confined space. In principle, the method can also be applied to other types of vehicles, e.g., a bus, train, watercraft, or aircraft, if appropriately adapted. This also reduces the driver's visual perception and eliminates the need to (usually frequently) adjust their gaze between the vehicle's surroundings and a display instrument.

[0117] Furthermore, the application of the invention to an at least partially automated vehicle is particularly advantageous because, due to the reduction in driving tasks, the driver of such a vehicle can close their eyes for extended periods or turn away from the driving process. Auditory communication in general, and in particular between the driver and another passenger and / or the computer, becomes correspondingly more important. The invention allows this to be made significantly more convenient and, if necessary, expanded. Thus, for example, important information for operating the vehicle can also be output as generated speech.

[0118] Exemplary embodiments of the invention are explained in more detail below, without limiting the scope, with reference to the schematic drawings. It shows: Fig. 1 A flowchart for controlling a vehicle audio system.

[0119] The Fig.1 shows a flowchart for controlling a vehicle audio system.

[0120] The program can be executed by a single device or distributed among multiple devices. For this purpose, the device comprises, in particular, a computing unit designed for operation in a vehicle, a program and data memory, and, for example, one or more communication interfaces.

[0121] The program is started in a step S1, in which variables can be initialized if necessary.

[0122] In a step S3, an acoustic content in the interior of a vehicle is determined, in particular predicted.

[0123] Furthermore, a distinction can also be made between different types of acoustic content audible in the interior (e.g., music, audio program, verbal communication). Different occupants or occupants in different seats can be assigned a different influence on determining the time for outputting the auditory message and / or on the content-related influence on the auditory message. For example, when outputting the auditory message, the computer can take into account the driver's communication with another occupant with regard to the time parameter for outputting at least part of the auditory message, while communication between a first occupant in the front passenger seat and a second occupant in the rear right seat is considered irrelevant (for adaptation) or less relevant.This can be the case, for example, when issuing navigation announcements, which are usually only directed at the driver (e.g. fourth passenger) of the vehicle.

[0124] Optionally, the method can determine and take into account the seat occupancy of the vehicle, the number of occupants in the vehicle, and the distribution of certain occupants across seats. In this case, at least one potential listener of the spoken communication and / or the auditory message in the interior of the vehicle can be determined. Furthermore, the adaptation of the auditory message can be carried out taking into account the at least one potential listener, in particular a seating position of the potential listener. If, on the other hand, it is determined that there is no one in a certain part of the vehicle interior, then an adaptation or optimization for the relevant part of the interior can be dispensed with and / or an optimized adaptation of the at least one auditory message can be carried out for at least one other part of the interior, e.g. seats.

[0125] Preferably, the method can comprise determining a type and / or a priority of the (in particular predicted) acoustic content in the interior of the vehicle, and / or determining a type and / or a priority of an auditory message to be generated, and generating the auditory message depending on the determined type and / or the determined priority of the acoustic content in the interior of the vehicle and / or the determined type and / or the determined priority of the auditory message to be generated. This allows the generation of an auditory message to be carried out depending on a type and a priority. In this case, the generation of the auditory message can be controlled depending on its respective importance and / or urgency in comparison to the acoustic content that was determined or predicted in at least one time interval with the output of the auditory message.

[0126] Particularly preferably, the method involves comparing one or more parameters of the acoustic content, representing its type and / or a predefined priority measure of the acoustic content or a priority measure determined from the type of acoustic content, with one or more parameters with an auditory message to be output, representing its type and / or a predefined priority measure of the auditory message to be output or a priority measure determined from the type of auditory message to be output. Such a comparison of the at least two priority measures can be carried out, for example, according to a predefined rule, logic, assignment table, etc. A priority measure determined or processed in the method can indicate at least a respective "importance" and / or at least an "urgency".For example, in the process, for example, in the case of an acoustic content with approximately average parameters, auditory messages may have the following priority identifiers:. - “Important” and “Urgent”; - “Important” but not “Urgent”; - “Urgent” but not “Important”; - not “urgent” and not “important”; are treated differently.

[0127] Depending on the comparison of the read, determined, or estimated priority measures, the auditory message to be output can be adjusted.

[0128] In this case, one or more time parameters for outputting at least part of the auditory message and / or at least one, in particular acoustic, parameter for outputting said part of the auditory message can be adjusted. For example, if only a short time interval between verbal communication is detected or predicted, an auditory message can be output at the appropriate time interval. In this case, at least the time or a time interval for outputting the auditory message is adjusted. Furthermore, the method can also adjust the time at which the output of the auditory message is to end and / or the time interval for outputting the auditory message. In particular, the auditory message can be placed or scheduled to match an available time interval within a communication between the at least two occupants of the vehicle.For example, it can be output between two phrases from a speaker or a phrase from a speaker, whereby a time interval until an expected response to the phrase by another occupant is determined, predicted, or estimated as sufficient. Furthermore, depending on the comparison of the aforementioned parameters, information intended for output as an auditory message can alternatively or additionally be output as a visual message, e.g., an image, graphic, text, or symbol, or a haptic message, e.g., vibration.

[0129] For example, if it is determined from the comparison of the parameters that the acoustic content should be given a higher priority overall than the acoustic message, the auditory message can be suppressed or output at a later time and / or with changed parameters. For example, if it is determined from the comparison of the priority measures that a certain auditory message has a higher priority overall than a certain acoustic content, the auditory message can be output with a changed parameter, e.g. louder or predominantly from a different loudspeaker, and a visual or graphic and / or haptic message can be output in addition. One or more auditory messages can be generated on this basis in the method, with the times for their output being chosen such that, with regard to the output time interval, they essentially lie between two further acoustic contents orAmplitude maxima of two acoustic contents fit.

[0130] Optionally, an adaptation of the at least one parameter of the auditory message can further comprise an assignment of certain contents and / or signal components of the at least one auditory message to be output to at least one loudspeaker and / or by adjusting the balance between at least two, preferably between at least four loudspeakers arranged in the vehicle interior.

[0131] Preferably, at least two parameters of the two different types of parameters are determined, in particular predicted and, preferably in one of the manifold expedient combinations, processed in the method and thereby expediently taken into account.

[0132] In a step S5, a time for outputting at least part of an auditory message is determined, in particular adapted, depending on the determined, in particular predicted acoustic content in the interior of the vehicle.

[0133] The program is terminated in step S7 and can be started again in step S1 if necessary.

[0134] The acoustic content includes, for example, an auditory communication and / or a voice of at least one, in particular a specific, occupant of the vehicle and / or an occupant at a specific seat.

[0135] The procedure does not require analysis of the spoken content. However, this can be optionally applied and taken into account in the procedure.

[0136] The method may optionally include one or more of the following steps: A step in which the auditory message is paused depending on an existing or predicted communication intention of the auditory communication and / or the voice of at least one, in particular a specific, occupant of the vehicle and / or an occupant at a specific seat. A step in which the content of the auditory message is divided into at least two parts depending on a determined existing or predicted communication intention of the auditory communication and / or the voice of at least one particularly specific occupant of the vehicle and / or an occupant at a specific seat. A step in which the order of at least two auditory messages and / or individual contents of the auditory message is changed depending on the existing or predicted communication intention of the auditory communication and / or the voice of at least one, in particular a specific, occupant of the vehicle and / or an occupant at a specific seat.

[0137] A step which also includes a prediction of a temporal parameter, in particular a temporal progression of a speaker’s voice, at least with regard to: - a beginning, and / or - an end, and / or - a temporal progression of the amplitudes and / or frequency parameters, particularly with regard to sentences or phrases, is carried out and is taken into account when generating the auditory message.

[0138] One step comprehensive - Determine the type and / or priority of the acoustic content in the vehicle interior, and / or - Determine the type and / or priority of the auditory message to be issued and then: - Determining, in particular adjusting, the timing of the output of the audible message depending on the determined type and / or priority of the acoustic content in the vehicle interior and / or the audible message to be output. For example, an audible message that depends on a specific event or criterion, e.g., depending on navigation information, e.g., 800 meters before a turn, can then be output before or after the process, depending on the acoustic content in the interior. Inappropriate temporal overlap with a spoken comment from the driver can be avoided or minimized.

[0139] A step in which the acoustic content is multimedia content and a prediction of the acoustic content is carried out by means of an analysis of a data stream streamed, in particular from a database and / or a cloud, or of content read from a data carrier before its playback.

[0140] A step in which it is also recognized that a recorded voice is a personal voice, in particular a voice of a specific person or a voice belonging to a specific group of people within the vehicle, and the generation, in particular variation of the auditory message takes place depending on the recognition of the voice of a specific person or a voice belonging to a specific group of people.

[0141] A step in the - learning of certain keywords, word sequences and / or facial patterns of at least one occupant takes place and then - a recognition of a willingness to communicate and / or - a recognition of the relevance, in particular the urgency and / or importance, of a linguistic message and / or - determining the adjustment of one or more parameters of the auditory message.

[0142] A step that also includes: - learning a specific voice and / or a specific physical movement of a speaker at one or more time intervals, and then: - upon recognition of a specific voice and / or physical movement of a speaker at one or more later time intervals: - Applying an optimized or pre-stored adaptation of the auditory message to the speaker's voice.

[0143] A step in which the prediction of the acoustic content takes place by means of a recognition of a communication intention of the auditory communication of at least one occupant of the vehicle, preferably by means of the recognition of his lip movement and / or a physical movement, for example by means of an interior camera.

[0144] A step in which a voice of a speaker in the interior of the vehicle is distinguished from a voice from a multimedia player, in particular the audio system of the vehicle, another sound system of the vehicle or a multimedia player connected to the vehicle.

[0145] A step in which the auditory message is adapted depending on a speaker's position and / or a relative direction from which the acoustic content comes or will come according to a communication intention of the auditory communication.

[0146] A step in which an emotion recognition and / or intention recognition of an occupant of the vehicle is recognized and taken into account in the method, whereby the auditory message is then adapted, preferably in different ways.

[0147] Preferably, the timing of the audible message is determined based on predicted acoustic content within the vehicle's interior. A distinction is made between acoustic content generated within the vehicle and acoustic content coming from outside (and audible within the vehicle). A distinction is also made between acoustic content from an audio output system that is onboard the vehicle, connected to the vehicle, or carried within the vehicle.

[0148] Furthermore, a distinction is preferably made between a type of acoustic content (music, audio program, spoken communication, and spoken communication of specific individuals), each of which has a different influence on determining the time parameter for outputting at least part of the auditory message. The priority of the respective acoustic content can also be automatically assessed.

[0149] For example, depending on the type of acoustic content detected or predicted in the vehicle interior, the following can occur: - an immediate pause - pausing after the end of a sentence - a division of the content of the auditory message.

[0150] The auditory message is preferably paused depending on the predicted acoustic content, in particular depending on the type of acoustic content, and / or the content of the auditory message is split into at least two parts. Such splitting can be initiated depending on the predicted acoustic content, e.g., before or during its output.

[0151] For example, the auditory message, such as a long announcement from the on-board computer, including the navigation system, voice control, is paused or (if necessary in advance, ie before it is output) split into two parts.

[0152] Alternatively or additionally, an audio signal corresponding to the auditory message can be adapted, wherein the signal energy of at least a portion of the auditory message is distributed across one or more frequency ranges that exhibit a lower, in particular the lowest possible, overlap or correlation with the frequency ranges of the human voice, in particular with the voice of a specific occupant (the "speaker"). This can increase the perceptibility and / or intelligibility of the verbal communication between the vehicle occupants and / or the auditory message, or reduce mutual interference, in particular masking (known, for example, from psychoacoustics).

[0153] In a further example of the method, the computer, in particular an application corresponding to a "vehicle assistant" or a "co-pilot," can be configured for a specific (speech) behavior, for example as a "co-speaking" party in the interior of the vehicle, which does not adversely affect communication between the occupants in the vehicle. The computer or the corresponding application can be provided with its own predefined speech priority, which perceives the communication of the occupants in the vehicle and selects a suitable time in the communication to output the audible message. If it can be assumed that a certain audible message was not sufficiently heard, the computer can repeat it without any action from the occupants being necessary.

[0154] When applying the method to at least two simultaneous or temporally overlapping communications, the resulting adaptation of the content and / or a parameter of the audio signal of an audible message to be output can be carried out taking into account the at least two communications in the vehicle interior. The method can consider the at least two different communications, for example, taking into account different predetermined criteria and / or with different influences on the at least one audible message.

[0155] Preferably, a suitably selected combination of the said parameters of the communication between at least two persons in the interior of the vehicle is taken into account and / or a suitably selected combination of the adjustments of the parameters of the audio signal to be output is then carried out.

[0156] Furthermore, the procedure may also include: - Determine that at least a (relevant) part of the auditory message has not been sufficiently perceptibly drowned out (at least with a certain high probability), and then, - Repetition of the part of the auditory message.

[0157] The repetition can be designed in such a way that the repeated part of the auditory message results in a valid (sufficiently complete) sentence. This means that, for example, an entire interrupted sentence or a part of a sentence, such as an otherwise understandable subordinate clause, can be repeated.

[0158] If, during the output of the auditory message, it is determined that at least one, particularly relevant, part of the auditory message was not sufficiently perceived by the occupants with a given probability, at least one relevant part (the part and / or a part related to the part) of the auditory message is repeated. The repetition is preferably performed in such a way that the repeated part of the auditory message results in a valid, i.e., sufficiently complete, sentence.

[0159] All aspects of the invention that have been applied here to the auditory message can also be applied in an analogous or appropriately adapted manner to a part or section of a multimedia program to be output in the vehicle, e.g. a radio program, audio book and the like.

Claims

[1] A method for controlling an audio system of a vehicle, comprising the following steps: - Determining, in particular predicting, an acoustic content in the interior of a vehicle, and - Adapting a time parameter for outputting at least part of an auditory message depending on the determined, in particular predicted, acoustic content in the interior of the vehicle, wherein - the time parameter for outputting at least part of the auditory message indicates a time for output. [2] Method according to claim 1, wherein the acoustic content comprises an auditory communication and / or a voice of at least one, in particular a specific, occupant of the vehicle and / or an occupant at a specific seat. [3] Method according to claim 2, in which the auditory message is paused depending on an existing or predicted communication intention of the auditory communication and / or the voice of at least one, in particular a specific, occupant of the vehicle and / or an occupant at a specific seat. [4] Method according to claim 2 or 3, in which the content of the auditory message is divided into at least two parts depending on a determined existing or predicted communication intention of the auditory communication and / or the voice of at least one particular occupant of the vehicle and / or an occupant at a particular seat. [5] Method according to one of claims 2 to 4, in which the order of at least two auditory messages and / or individual contents of the auditory message is changed depending on the existing or predicted communication intention of the auditory communication and / or the voice of at least one, in particular a specific, occupant of the vehicle and / or an occupant at a specific seat. [6] Method according to one of the preceding claims, in which a prediction of a temporal parameter, in particular a temporal course of a speaker’s voice, at least with regard to: - a beginning, - an end, - a temporal progression of the amplitudes, and / or - Frequency parameters, particularly with regard to sentences or phrases, are taken into account when generating the auditory message. [7] Method according to one of the preceding claims, comprising - Determine the type and / or priority of the acoustic content in the vehicle interior, and / or - Determine the type and / or priority of the auditory message to be issued and then: - Determining, in particular adapting, the time parameter of the output of the auditory message depending on the determined type and / or priority of the acoustic content in the interior of the vehicle and / or the auditory message to be output. [8] Method according to one of the preceding claims, wherein the acoustic content is a multimedia content and a prediction of the acoustic content is carried out by means of an analysis of a data stream streamed, in particular from a database and / or a cloud, or of a content read from a data carrier before its reproduction. [9] Method according to one of the preceding claims, in which it is also recognized that a recorded voice is a personal voice, in particular a voice of a specific person or a voice belonging to a specific group of people within the vehicle, and the generation, in particular variation of the auditory message takes place depending on the recognition of the voice of a specific person or a voice belonging to a specific group of people. [10] Method according to one of the preceding claims, in which - learning of certain keywords, word sequences and / or facial expression patterns of at least one occupant takes place and then - a recognition of a willingness to communicate and / or - a recognition of the relevance, in particular the urgency and / or importance, of a linguistic message and / or - determining the adjustment of one or more parameters of the auditory message. [11] Method according to one of the preceding claims, also comprising: - learning a specific voice and / or a specific physical movement of a speaker at one or more time intervals, and then: - upon recognition of a specific voice and / or physical movement of a speaker at one or more later time intervals: - Applying an optimized or pre-stored adaptation of the auditory message to the speaker's voice. [12] Method according to one of the preceding claims, in which the prediction of the acoustic content is carried out by means of a recognition of a communication intention of the auditory communication of at least one occupant of the vehicle, preferably by means of the recognition of his lip movement and / or a physical movement, for example by means of an interior camera. [13] Method according to one of the preceding claims, in which a voice of a speaker in the interior of the vehicle is distinguished from a voice from a multimedia player, in particular the audio system of the vehicle, another sound system of the vehicle, or a multimedia player connected to the vehicle. [14] Method according to one of the preceding claims, in which the auditory message is adapted depending on a position of a speaker and / or a relative direction from which the acoustic content comes or will come according to a communication intention of the auditory communication. [15] Method according to one of the preceding claims, in which an emotion recognition and / or intention recognition of an occupant of the vehicle is recognized and taken into account in the method, wherein the adaptation of the auditory message then takes place, preferably in a different way. [16] Device for controlling an audio system of a vehicle, wherein the device is designed to carry out a method according to one of claims 1 to 15. [17] Mobile user device for controlling an audio system of a vehicle, comprising an interface with the vehicle and configured to carry out at least part of the method according to one of claims 1 to 15. [18] Computer program, in particular a computer program product comprising a computer program, wherein the computer program is designed to carry out the method according to one of claims 1-15 when executed on a computing unit.

Images