Vehicle comprising an audio system, audio system, method to operate an audio system, computer program product and non-transitory computer-readable storage

Abstract

A vehicle 10 comprising an audio system 12 (as might be called an in car communication or ICC system) comprises a control unit 18, configured to identify language signals 26 in audio input signals 16 received from microphone devices 14; identify a conversation of a respective group 28 of passengers 24 in the cabin 32 of the vehicle 10 and a respective position of the respective passengers 24, based on the language signals 26; isolate the language signals 26 of a current speaker 30 of the group 28 of passengers 24 in the audio input signals 16; provide the output signals 22, comprising the isolated language signals 26 of the current speaker 30 to the speaker devices 20 associated with the respective position of the other passengers 24 of the group 28. The control unit might use speakers to produce audio seemingly directional from the actual speakers. The speakers might be identified spatially or using voice characteristics. Also claimed is the system in isolation, a method of use and a computer program to perform the above.

Description

[0001] The present invention relates to the field of automobiles. More specifically, the present invention relates to a vehicle comprising an audio system. Furthermore, the present invention relates to an audio system and a method to operate an audio system. Furthermore, the present invention relates to a corresponding computer program product, as well as to a corresponding non-transitory computer-readable storage medium. BACKGROUND INFORMATION

[0002] The noise level in vehicles can disturb conversations among passengers. This especially applies to spacious cars with more than two seat rows. For example, sitting in a large vehicle in a third row, talking to the driver is usually difficult. SUMMARY OF THE INVENTION

[0003] It is an object of the present invention to provide a vehicle comprising an audio system, a corresponding audio system, a corresponding method to operate an audio system, a corresponding computer program product as well as a corresponding non-transitory computer-readable storage medium, by which a conversation among passengers in a large vehicle can be enhanced.

[0004] This object is solved by a vehicle comprising an audio system, a corresponding audio system, a method to operate an audio system, a corresponding computer program product as well as a corresponding non-transitory computer-readable storage medium according to the independent claims. Advantageous embodiments are presented in the dependent claims.

[0005] A first aspect of the present invention relates to a vehicle comprising an audio system. The audio system comprises multiple microphone devices, wherein a respective one of the microphone devices is arranged at a respective microphone position in a cabin of the vehicle. The respective one of the microphone devices is configured to provide respective audio input signals recorded by the respective microphone device to a control unit of the audio system.

[0006] The audio system comprises multiple speaker devices, wherein a respective one of the speaker devices is arranged at a respective speaker position in the cabin of the vehicle. The respective one of the microphone devices is configured to output respective output signals for the respective speaker device, received from the control unit.

[0007] The control unit of the audio system is configured to identify language signals in the audio input signals received from the microphone devices.

[0008] The control unit of the audio system is configured to identify a conversation of a respective group of passengers in the cabin of the vehicle. Further, the control unit of the audio system is configured to identify a respective position of the respective passengers. The identification of the conversation of the respective group of passengers in the cabin of the vehicle and of the respective position of the respective passengers is based on the language signals identified in the audio input signals received from the microphone devices.

[0009] The control unit of the audio system is configured to isolate language signals of a current speaker of the group of passengers in the audio input signals.

[0010] The control unit of the audio system is configured to provide the output signals comprising the isolated language signals of the current speaker of the group to the speaker devices associated with the respective of the other passengers of the group.

[0011] Therefore, the invention provides a solution to enhance a conversation among passengers in a large vehicle.

[0012] According to an embodiment, the control unit is configured to adjust a level of amplification of the isolated language signal in the respective output signal as a function of in-cabin noise and a distance between the position of the respective member of the group to the position of the current speaker of the group.

[0013] According to an embodiment, the control unit is configured to identify two or more conversations of a respective group of passengers in the cabin of the vehicle occurring in parallel.

[0014] According to an embodiment, the control unit is configured to provide the respective output signals comprising the isolated language signals to a predefined set of the speaker devices associated with the location of the passenger to emulate a source of the isolated language signals in a direction of the speaker.

[0015] According to an embodiment, the control unit is configured to identify the current speaker by analyzing voice characteristics of the language signals.

[0016] According to an embodiment, the control unit is configured to identify the current speaker by locating a source position of the language signals in the input signals.

[0017] A second aspect of the invention relates to an audio system for a vehicle. The audio system comprises multiple microphone devices, wherein a respective one of the microphone devices is arranged at a respective microphone position in a cabin of the vehicle. The respective one of the microphone devices is configured to provide respective audio input signals recorded by the respective microphone device to a control unit of the audio system.

[0018] The audio system comprises multiple speaker devices, wherein a respective one of the speaker devices is arranged at a respective speaker position in the cabin of the vehicle. The respective one of the microphone devices is configured to output respective output signals for the respective speaker device, received from the control unit.

[0019] The control unit of the audio system is configured to identify language signals in the audio input signals received from the microphone devices.

[0020] The control unit of the audio system is configured to identify a conversation of a respective group of passengers in the cabin of the vehicle. Further, the control unit of the audio system is configured to identify a respective position of the respective passengers. The identification of the conversation of the respective group of passengers in the cabin of the vehicle and of the respective position of the respective passengers is based on the language signals identified in the audio input signals received from the microphone devices.

[0021] The control unit of the audio system is configured to isolate language signals of a current speaker of the group of passengers in the audio input signals.

[0022] The control unit of the audio system is configured to provide the output signals comprising the isolated language signals of the current speaker of the group to the speaker devices associated with the respective of the other passengers of the group.

[0023] A third aspect of the present invention relates to a method to an audio system.

[0024] The method comprises a provision of respective audio input signals recorded by respective microphone devices, arranged at respective microphone positions in a cabin of a vehicle to a control unit of the audio system.

[0025] The method comprises an identification of language signals in the audio input signals received from the microphone devices by the control unit.

[0026] The method comprises an identification of a conversation of a respective group of passengers in the cabin of the vehicle and of a respective position of the respective passengers in the cabin of the vehicle by the control unit, based on the language signals by the control unit.

[0027] The method comprises an isolation of isolated language signals of the current speaker of the group of passengers in the audio input signals by the control unit.

[0028] The method comprises a provision of the output signals, comprising the isolated language signals of the current speaker to the speaker devices associated with the respective position of the other passengers of the group.

[0029] Another aspect of the invention relates to a computer program product comprising program code means for performing a method according to the preceding aspect.

[0030] A still further aspect of the invention relates to a non-transitory computer-readable storage medium comprising at least the computer program product according to the preceding aspect.

[0031] Advantageous embodiments of the method are to be regarded as advantageous embodiments of the computer program product, the non-transitory computer-readable storage medium, as well as the system. Therefore, the surface detection device of the vehicle comprises means for performing the method.

[0032] The control unit may comprise an computing unit / electronic computing device. A computing unit / electronic computing device may in particular be understood as a data processing device, which comprises processing circuitry. The computing unit can therefore in particular process data to perform computing operations. This may also include operations to perform indexed accesses to a data structure, for example a look-up table, LUT.

[0033] In particular, the computing unit may include one or more computers, one or more microcontrollers, and / or one or more integrated circuits, for example, one or more application-specific integrated circuits, ASIC, one or more field-programmable gate arrays, FPGA, and / or one or more systems on a chip, SoC. The computing unit may also include one or more processors, for example one or more microprocessors, one or more central processing units, CPU, one or more graphics processing units, GPU, and / or one or more signal processors, in particular one or more digital signal processors, DSP. The computing unit may also include a physical or a virtual cluster of computers or other of said units.

[0034] In various embodiments, the computing unit includes one or more hardware and / or software interfaces and / or one or more memory units.

[0035] A memory unit may be implemented as a volatile data memory, for example a dynamic random access memory, DRAM, or a static random access memory, SRAM, or as a non-volatile data memory, for example a read-only memory, ROM, a programmable read-only memory, PROM, an erasable programmable read-only memory, EPROM, an electrically erasable programmable read-only memory, EEPROM, a flash memory or flash EEPROM, a ferroelectric random access memory, FRAM, a magnetoresistive random access memory, MRAM, or a phase-change random access memory, PCRAM.

[0036] Further advantages, features, and details of the invention derive from the following description of preferred embodiments as well as from the drawings. The features and feature combinations previously mentioned in the description as well as the features and feature combinations mentioned in the following description of the figures and / or shown in the figures alone can be employed not only in the respectively indicated combination but also in any other combination or taken alone without leaving the scope of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The novel features and characteristic of the disclosure are set forth in the appended claims. The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and together with the description, serve to explain the disclosed principles. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the figures to reference like features and components. Some embodiments of system and / or methods in accordance with embodiments of the present subject matter are now described below, by way of example only, and with reference to the accompanying figures.

[0038] The drawings show in:

[0039] Fig. 1 a schematic illustration of a vehicle comprising an audio system; and

[0040] Fig. 2 a schematic illustration of a method to operate a vehicle.

[0041] In the figures the same elements or elements having the same function are indicated by the same reference signs. DETAILED DESCRIPTION

[0042] In the present document, the word "exemplary" is used herein to mean "serving as an example, instance, or illustration". Any embodiment or implementation of the present subject matter described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

[0043] While the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawing and will be described in detail below. It should be understood, however, that it is not intended to limit the disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure.

[0044] The terms “comprises”, “comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion so that a setup, device or method that comprises a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. In other words, one or more elements in a system or apparatus preceded by “comprises” or “comprise” does not or do not, without more constraints, preclude the existence of other elements or additional elements in the system or method.

[0045] In the following detailed description of the embodiment of the disclosure, reference is made to the accompanying drawing that forms part hereof, and in which is shown by way of illustration a specific embodiment in which the disclosure may be practiced. This embodiment is described in sufficient detail to enable those skilled in the art to practice the disclosure, and it is to be understood that other embodiments may be utilized and that changes may be made without departing from the scope of the present disclosure. The following description is, therefore, not to be taken in a limiting sense.

[0046] Fig. 1 shows a schematic illustration of a vehicle 10 comprising an audio system 12.

[0047] Fig. 1 shows a schematic illustration of the vehicle 10 comprising an audio system 12. The vehicle 10 may comprise a cabin 32. The cabin 32 of the vehicle 10 may comprise seats 34 for passengers 24 of the vehicle 10. Some of the seats 34 may be occupied by the passengers 24 of the vehicle 10. The vehicle 10 may comprise an audio system 12. The audio system 12 may comprise a control unit 18 and a plurality of microphone devices 14, the microphone devices 14 being located at respective, different positions in the cabin 32 of the vehicle 10. The microphone devices 14 may be configured to record audio input signals 16 in the cabin 32 and to provide the respective audio input signals 16 to the control unit 18 of the audio system 12. The audio system 12 may comprise a plurality of loudspeaker devices 20, the loudspeaker devices 20 being located at respective loudspeaker positions in the cabin 32 of the vehicle 10. The speaker devices 20 may be configured to output respective output signals 22 provided by the control unit 18 to the respective speaker device 20. The microphone devices 14 may be configured to record the respective audio input signals 16 during an operation of the vehicle 10.

[0048] The control unit 18 may be configured to receive the audio input signals 16 and to identify language signals 26 in the audio input signals 16. The control unit 18 may be configured to identify a conversation of respective passengers 24 of a group 28 in the cabin 32 of the vehicle 10. The control unit 18 may be configured to identify the respective position of the respective passengers 24 based on the language signals 26 in the audio input signals 16 received from the microphone devices 14. An identification of a current speaker 30 of the group 28 may be carried out by means of speech analysis methods and / or localization methods.

[0049] The control unit 18 may comprise a module configured to detect speech signals in the audio input signals 16. The identification of passengers 24 may be performed by analysing voice characteristics of the audio input signals 16 and by localising the audio input signals 16. The voice detection module may be configured to determine a time series of the current speakers 30. The time series may comprise binary values indicating whether passengers 24 at estimated locations are speaking during a respective time slot of the time series.

[0050] The speech recognition module may provide the time series, giving temporal patterns of the current speakers 30 to a conversation recognition module of the control unit 18. The conversation recognition module may be configured to find mappings that may reflect conversations between the passengers 24 of the vehicle 10 based on the temporal patterns. The module may evaluate groups 28 of the passengers 24 having a conversation.

[0051] The control unit 18 may be configured to isolate individual speech signals from the audio input signals 16 received from the multiple in-cabin 32 microphone devices 14. The isolation may comprise filtering out background noise and other voices than the respective isolated voice of the current speaker 30. Based on the mapping of the previous speech recognition module, the control unit 18 may amplify the respective language signals 26 for respective audio output signals 22. The control unit 18 may provide the output signals 22 to loudspeaker devices 20 associated with passengers 24 belonging to the respective conversation group 28, except to the current speaker 30. The output signals 22 may be adjusted to a level of amplification which may be adjusted as a function of current cabin 32 noise and a distance between the location of the current speaker 30 and the location of the respective member of the group 28. The control unit 18 may provide smooth changes and modifications of the output signals 22 to keep the language signals 26 as natural as possible.

[0052] Fig. 2 shows a schematic illustration of a method to operate an audio system 12.

[0053] In a first step S1, microphone devices 14 of the audio system 12 of the vehicle 10 may record audio input signals 16 in a cabin 32 of the vehicle 10.

[0054] In a second step S2, the microphone devices 14 may provide the respective audio input signals 16 to a control unit 18 of the audio system 12.

[0055] In a third step S3, the control unit 18 may identify language signals 26 in the audio input signals 16 received from the microphone devices 14.

[0056] In a fourth step S4, the control unit 18 may identify a conversation of a respective group 28 of passengers 24 in the cabin 32 of the vehicle 10 based on the language signals 26. The control unit 18 may also identify respective positions of the respective passengers 24 in the cabin 32 of the vehicle 10 based on the language input signals.

[0057] In a fifth step S5, the control unit 18 identify a current speaker 30 of the group 28 and may isolate voice signals of the current speaker 30 of the group 28 of in the audio input signals 16.

[0058] In a sixth step S6, the control unit 18 may provide isolated the language signals 26 of the current speaker 30 to loudspeaker devices 20 associated with the respective position of the passengers 24 of the group 28.

[0059] Modern vehicles 10 have a large array of microphones installed in the cabin 32 to enable phone calls from all seats 34. They also have multiple speakers to cover all areas of the cabin 32. The proposed system leverages these existing microphones and speakers.

[0060] The system may be characterized as follows:

[0061] The control unit 18 may be configured to detect spoken language and locate seat 34 of speaking passenger 24

[0062] The control unit 18 may be configured to detect conversation by alternating triggers of detected spoken language from different seats 34.

[0063] The control unit 18 may be configured to aggregate the input signals from incabin 32 microphones to generate a best audio quality of the language signals 26, comprising the spoken language and to filter ambient noise.

[0064] The language signals 26, comprising the spoken language may be amplified on speaker devices 20 for only those seats 34 that are occupied by passengers 24 of the group 28 and not the speaking person to avoid confusing the speaker.

[0065] A level of amplification may be a function of in-cabin 32 noise level and distance between seats 34.

[0066] A 3D sound system may be used such that the amplified spoken language sounds as if it came from the location of the speaking passenger 24 and not from the speaker device 20.

[0067] The audio system 12 may run continuously in a background and may seamlessly perform an amplification when the conversation is detected. The audio system 12 may keep a sound profile as natural as possible such that the amplification is not even recognized as coming from the speakers in best case. All data processing may happens purely in-vehicle 10 and no data is transmitted to devices outside the vehicle 10 to guarantee a highest level of privacy.

[0068] The audio system 12 eases in-cabin 32 conversations

[0069] The audio system 12 may detect conversations and perform the amplification fully automated

[0070] Sound direction and level of amplification may be adjusted dynamically by the audio system 12 to make the conversation sound as natural as possible.

[0071] The audio system 12 may re-use existing microphone devices 14 and speaker devices 20 of speaker arrays in the vehicle 10 cabin 32 and may purely be configured as a software feature. Reference signs 10 12 14 16 18 20 22 24 26 28 30 32 34 vehicle audio system microphone device audio input signal control unit speaker device output signals passenger language signal group current speaker cabin seat

Claims

1. Vehicle (10) comprising an audio system (12), wherein,- the audio system (12) comprises multiple microphone devices (14), wherein a respective one of the microphone devices (14) is arranged at a respective microphone position in a cabin (32) of the vehicle (10) and is configured to provide respective audio input signals (16) recorded by the respective microphone device (14) to a control unit (18) of the audio system (12); and- the audio system (12) comprises multiple speaker devices (20), wherein a respective one of the speaker devices (20) is arranged at a respective speaker position in the cabin (32) of the vehicle (10) and is configured to output respective output signals (22) for the respective speaker device (20), received from the control unit (18) into the cabin (32) of the vehicle (10);characterized in thatthe control unit (18) is configured to- identify language signals (26) in the audio input signals (16) received from the microphone devices (14);- identify a conversation of a respective group (28) of passengers (24) in the cabin (32) of the vehicle (10) and a respective position of the respective passengers (24), based on the language signals (26);- isolate the language signals (26) of a current speaker (30) of the group (28) of passengers (24) in the audio input signals (16);- provide the output signals (22), comprising the isolated language signals (26) of the current speaker (30) to the speaker devices (20) associated with the respective position of the other passengers (24) of the group (28).

2. The vehicle (10) according to claim 1, characterized in thatthe control unit (18) is configured to adjust a level of amplification of the isolated language signal (26) in the respective output signal as a function of in-cabin noise and a distance between the position of the respective passenger (24) of the group (28) to the position of the current speaker (30) of the group (28).

3. The vehicle (10) according to claim 1 or 2, characterized in thatthe control unit (18) is configured to identify two or more conversations of respective groups (28) of passengers (24) in the cabin (32) of the vehicle (10) occurring in parallel.

4. The vehicle (10) according to any one of claims 1 to 3, characterized in thatthe control unit (18) is configured to provide the respective output signals (22) comprising the isolated language signals (26) to a predefined set of the speaker devices (20) associated with the location of the passenger (24) to emulate a source of the isolated language signals (26) in a direction of the current speaker (30).

5. The vehicle (10) according to any one of claims 1 to 4, characterized in thatthe control unit (18) is configured to identify the current speaker (30) by analyzing voice characteristics of the language signals (26).

6. The vehicle (10) according to any one of claims 1 to 5, characterized in thatthe control unit (18) is configured to identify the current speaker (30) by locating a source position of the language signals (26) in the audio input signals (16).

7. An audio system (12) for a vehicle (10), wherein- the audio system (12) comprises multiple microphone devices (14), wherein a respective one of the microphone devices (14) is arranged at a respective microphone position in a cabin (32) of a vehicle (10) and is configured to providerespective audio input signals (16) recorded by the respective microphone device (14) to a control unit (18) of the audio system (12);- the audio system (12) comprises multiple speaker devices (20), wherein a respective one of the speaker devices (20) is arranged at a respective speaker position in the cabin (32) of the vehicle (10) and is configured to output respective output signals (22) for the respective speaker device (20), received from the control unit (18) into the cabin (32) of the vehicle (10);characterized in thatthe control unit (18) is configured to- identify language signals (26) in the audio input signals (16) received from the microphone devices (14);- identify a conversation of a respective group (28) of passengers (24) in the cabin of the vehicle (10) and a respective position of the respective passengers (24); based on the language signals (26);- isolate the language signals (26) of a current speaker (30) of the group (28) of passengers (30) in the audio input signals (16);- provide the output signals (22), comprising the isolated language signals (26) of the current speaker (30) to the speaker devices (20) associated with the respective position of the other passengers (24) of the group (28).

8. A method to operate an audio system (12), comprising the following steps:- receiving respective audio input signals (16) recorded by respective microphone devices (14), arranged at respective microphone positions in a cabin (32) of the vehicle (10) by a control unit (18) of the audio system (12);- identifying language signals (26) in the audio input signals (16) received from the microphone devices (14) by the control unit (18);- identifying a conversation of a respective group (28) of passengers (24) in the cabin (32) of the vehicle (10) and a respective position of the respective passengers (24), based on the language signals (26) by the control unit (18);- isolating the language signals (26) of a current speaker (30) of the group of passengers (24) in the audio input signals (16) by the control unit (18); and - providing the output signals (22), comprising the isolated language signals (26) of the current speaker (30) to the speaker devices (20) associated with the respective position of the other passengers (24) of the group (28).

9. A computer program product comprising program code means for performing a method according to claim 8.

10. A non-transitory computer-readable storage medium comprising at least the computer program product according to claim 9.16

Images