Methods and systems for active noise reduction in vehicles

Abstract

The invention relates to a method for noise suppression for an occupant of a motor vehicle, wherein an in-ear earphone (1) is used (S1), wherein ambient noise is detected by at least one microphone (2) and noise compensation (S2) is performed by means of a computing unit (3) to partially eliminate the ambient noise by generating anti-noise, wherein the computing unit (3) executes a pre-trained machine learning model and thereby makes a selection (S3) as to which ambient noises are allowed to pass through to the occupant and which are eliminated, wherein the computing unit (3) is connected to a control unit (4) of the vehicle and the machine learning model makes the selection based on current information from the control unit (4) of the vehicle, wherein the computing unit (3) decides to allow the speech of a passenger to pass through.where an acoustic signature of the passenger is known to the machine learning model, and wherein the passenger is prompted to provide a speech sample by the control unit (4) of the vehicle or by the computing unit (3) when the passenger is unknown to the machine learning model in order to derive an acoustic signature of the passenger and to store him as known by assigning his acoustic signature.

Description

[0001] The invention relates to a method for noise suppression for an occupant of a motor vehicle, as well as a system for noise suppression for an occupant while driving in a motor vehicle.

[0002] Vehicles require significant material and financial investment in development and production to effectively dampen ambient and road noise. Physical damping materials, such as sound-absorbing mats or, as a further development, acoustic metamaterials, require considerable installation space within the vehicle, add weight, and increase production costs and scope.

[0003] Active noise cancellation (ANC) in-ear headphones are well-known and are now also used as clinically certified hearing aids. Currently, the use of headphones while driving is prohibited. However, this restriction does not apply to medical hearing aids. Medically approved hearing aids based on in-ear headphones, which can be controlled via a smartphone and can also play additional audio from the smartphone, are currently available. Active noise cancellation systems for vehicles are also currently available.

[0004] CN 109686356 A relates to an integrated ANC amplifier system for motor vehicles with active noise cancellation, comprising an amplifier base plate and a vehicle computer, wherein the ANC module for active noise cancellation of the entire vehicle is integrated on the surface of the amplifier base plate, the ANC module is powered directly via the OBD port, the ANC module communicates directly with the vehicle's CAN bus, the entertainment audio signal output by the vehicle computer is mixed with the noise-canceling audio signal output by the ANC module, and the audio signal is mixed by the amplifier base plate and then output to the speaker to suppress the sound wave and thus achieve the goal of noise cancellation.

[0005] DE 10 2022 125 547 A1 relates to a motor vehicle and a method for summarizing a conversation in a motor vehicle. DE 10 2022 123 850 A1 describes a method for detecting a sound transducer located in the interior or surroundings of a motor vehicle, which is designed to be worn in and / or on a person's ear, and for operating the detected sound transducer depending on the current and / or future role of the person to whom the detected sound transducer is assigned. Furthermore, DE 10 2024 110 269 A1 discloses a method for headphone conversation recognition.

[0006] The object of the invention is to provide noise suppression for an occupant of a motor vehicle with a view to the safest possible driving operation.

[0007] The invention is defined by the features of the independent claims. Advantageous further developments and embodiments are the subject of the dependent claims.

[0008] A first aspect of the invention relates to a method for noise suppression for an occupant of a motor vehicle, wherein an in-ear earphone is used which is designed to output acoustic signals, wherein ambient noise is detected by at least one microphone and noise compensation is carried out by means of a processing unit to partially eliminate the ambient noise by means of a processing unit, in that acoustic signals for generating anti-sound are determined by the processing unit and the in-ear earphone is controlled for output, wherein a pre-trained machine learning model is executed by the processing unit and the machine learning model makes a selection as to which ambient noises are allowed to pass through to the occupant and which are eliminated.where the computing unit is connected to a control unit of the vehicle and the selection is made using a machine learning model based on current information from the vehicle's control unit.

[0009] An in-ear hearing aid is preferably used as the in-ear device. An in-ear hearing aid is a special type of in-ear headphone that includes a plug-in element which can be inserted into the user's ear canal. This creates a seal in the ear canal. The system described uses such in-ear hearing aids for sound insulation in a vehicle. In addition to classic active noise cancellation (ANC), which involves playing a sound wave with a π-phase shift that counteracts the original wave from the ambient noise, artificial intelligence is also used through machine learning to optimize the ANC and to allow only desired sound information to pass through.

[0010] The machine learning model is primarily a deep neural network that is trained during vehicle development using camera data, rain sensor data, vehicle data (such as speed, engine speed, gear selection, tires used, and tire pressure), acoustic speech profiles of different people and languages, map data such as road type (highway, city, off-road) and surface material, special sounds such as sirens or car horns, and a continuous stream of microphone recordings from the vehicle under test in a wide variety of driving situations. The output of the neural network is a signal to generate a continuous anti-noise signal that filters out driving noise and unwanted background noise while allowing conversations and sounds important to the driver to reach the inner ear.

[0011] According to an advantageous embodiment, the control unit's information includes at least one of the following, each currently determined: camera data about the vehicle's surroundings, data from a vehicle's rain sensor, the vehicle's speed, the engine speed of the vehicle's drive system, the selected gear, design information about the vehicle's wheels and / or tires, the tire pressure of at least one tire of the vehicle, an acoustic voice profile of a passenger, location data in conjunction with map data comprising at least one of the following: the type of road being traveled, the characteristics of a roadway being traveled, a detected siren of an emergency vehicle, or a detected activation of a horn by another road user. The type of road being traveled indicates, in particular, whether a highway, a city center, an off-road area, or something else is being traversed.The property of the roadway, on the other hand, specifies in particular a surface condition, especially a material of the road surface.

[0012] According to the invention, the computing unit decides whether to allow the speech of a passenger to pass through if an acoustic signature of the passenger is known to the machine learning model.

[0013] According to the invention, the passenger is prompted to provide a voice sample by the vehicle's control unit or by the computing unit if the passenger is unknown to the computing unit, in order to derive an acoustic signature of the passenger and to store him / her as known by assigning his / her acoustic signature.

[0014] According to a further advantageous embodiment, a respective acoustic signature is stored by the vehicle's control unit and transmitted to the computing unit.

[0015] According to another advantageous embodiment, the quality and quantity of noise compensation is dynamically determined by the machine learning model.

[0016] Another aspect of the invention relates to a noise suppression system for an occupant while traveling in a motor vehicle, comprising an in-ear earphone, a microphone, and a processing unit, wherein the at least one microphone is configured to detect ambient noise, wherein the in-ear earphone is configured to output acoustic signals, and wherein the processing unit is configured to detect anti-noise and to output corresponding acoustic signals via the in-ear earphone, thereby at least partially eliminating the ambient noise, and to execute a pre-trained machine learning model and thereby select which ambient noises are allowed to pass through to the occupant and which are eliminated, wherein the processing unit is connected to a control unit of the vehicle and is further configured toto make the selection based on information from the vehicle's control unit using a machine learning model.

[0017] Advantages and preferred further developments of the proposed system result from an analogous and substantive transfer of the above statements made in connection with the proposed procedure.

[0018] Further advantages, features and details will become apparent from the following description, in which - possibly with reference to the drawing - at least one embodiment is described in detail.

[0019] They show: Fig. 1: A method according to an embodiment of the invention. Fig. 2: A flowchart for the selection of speech signals according to an embodiment of the invention. Fig. 3: An exemplary situation in which a system according to an embodiment of the invention is used to perform the process of Fig. 1. To implement concretely.

[0020] Fig. Figure 1 shows a method for noise suppression for an occupant of a motor vehicle, wherein an in-ear earphone 1 S1 is used, which is configured to output acoustic signals, wherein at least one microphone 2 detects ambient noise and a noise compensation S2 is performed by means of a computing unit 3 to partially eliminate the ambient noise, by which the computing unit 3 determines acoustic signals to generate anti-noise and controls the in-ear earphone 1 for output, wherein the computing unit 3 executes a pre-trained machine learning model and the machine learning model makes a selection S3 as to which ambient noises are allowed to reach the occupant and which are eliminated.wherein the computing unit 3 is connected to a control unit 4 of the vehicle and the selection is made by the machine learning model based on current information from the control unit 4 of the vehicle.

[0021] Fig. Figure 2 shows a flowchart that processing unit 3 follows to decide whether or not background speech noise should be allowed to pass through. Specifically, processing unit 3 is responsible for allowing speech from passengers inside the vehicle to reach the user with the in-ear headphones 1. In the first step, J1, an interior camera visually records each occupant of the vehicle and attempts to identify them using facial recognition. In query step Q1, it is checked whether processing unit 3 recognizes a particular occupant. Speech should only be allowed to pass through the in-ear headphones 1 from known occupants. If the check in query step Q1 fails, the process continues along path n to step J2, where a new speech profile is created.For this purpose, in step J2, a text to be read aloud is displayed to processing unit 3, or an acoustic message is output for repetition. The occupant, in the sense of the passenger, who is still unknown to processing unit 3 at this point, repeats the acoustic output or reads the output text aloud. Their speech sample is recorded, and an acoustic signature of the occupant is generated. Then, step J3 can proceed, in which the acoustic signature is passed to the machine learning model. Using the acoustic signature, the model can then decide in the future whether the passenger is known and whether their speech should be allowed through the in-ear headphone 1. If, however, query Q1 is already positive, branch y can proceed to the further query Q2, which checks whether the acoustic signature already exists.If this is the case, the process can proceed directly to step J3 via y, which is described above. However, if this is not the case, branch n continues with step J2 as described above.

[0022] Fig.Figure 3 shows a noise cancellation system for an occupant while traveling in a motor vehicle, comprising a left and a right in-ear earphone 1 for each wearer, a microphone 2, and a processing unit 3. A continuous stream of sound information is picked up by the microphone 2 and fed to the machine learning model running on the processing unit 3. The machine learning model, implemented as an artificial neural network, determines which ambient noises are eliminated at each in-ear earphone 1 by means of inverse acoustic signals and optimizes the anti-noise by dynamically adjusting the acoustic signals at each in-ear earphone 1. For this purpose, the machine learning model is connected to a control unit 4 of the vehicle to receive current information from the control unit 4, in particular vehicle states and / or environmental conditions.For this purpose, sensor data from the vehicle is transmitted to the machine learning model via the vehicle's control unit 4. This includes localization data such as satellite-based positioning (GPS) and / or initial sensor data, as well as map information, acoustic signatures of other potential passengers, and predefined external noises to be allowed through, such as sirens like a police siren. The following example illustrates how the machine learning model works: The vehicle is traveling in sixth gear at 130 km / h on a highway. It is raining, the road is wet, and the vehicle is equipped with M&S winter tires inflated to a pressure of 2.2 bar. A passenger, who is experiencing this vehicle for the first time, is sitting next to the driver.Both the driver and the front passenger wear in-ear headphones 1, each with a processing unit 3 connected via Bluetooth to the vehicle's control unit 4. Each in-ear headphone 1 is equipped with one or more microphones 2 that can pick up sound directly at the ears. The vehicle's localization unit, in conjunction with an external map stored in the cloud or locally in the vehicle, provides the neural network with information that the vehicle is on a highway. Ideally, the map data also includes information about the road surface. The neural network receives information about the road surface from the vehicle's camera data, which is processed using image processing. The rain sensor provides the neural network with information about the rain and its intensity.Additionally, the neural network receives information about the time and frequency of windshield wiper movements via the vehicle's control unit 4, which acts as the central on-board computer. Camera data tells the neural network when and at what distance the vehicle is overtaken by a truck, and that another car is driving in front of the vehicle and at what distance. Furthermore, the neural network knows the type and pressure of each tire via the central on-board computer and tire sensors. The interior camera allows control unit 4 to recognize the driver and thus provide the neural network with a previously stored voice profile. Control unit 4 does not recognize a passenger who is entering the vehicle for the first time based on the camera data. Therefore, after entering the vehicle, control unit 4 asks the passenger to read aloud a short text displayed on the passenger-side screen.The voice is recorded via microphones 2, analyzed by control unit 4, and a voice profile of the passenger is generated. This profile, along with their image information, is stored for future journeys. This voice profile is also passed to the neural network as input. The final, continuous input to the neural network is the data from microphones 2, which are connected to in-ear headphones 1. The neural network continuously calculates an anti-sound from this information, which is played into the ears of the driver and passenger via the in-ear headphones 1. This process cancels out unwanted driving noises and other unwanted sounds such as windshield wipers, ventilation, and other noises through negative interference of the sound waves from the anti-sound, while desired sounds, such as the voices of the driver and passenger, are not dampened but can even be amplified if desired.The same applies to desired external sounds, such as the horns of other road users or the sirens of emergency vehicles. Additionally, acoustic information from the control unit 4, such as music, radio broadcasts, telephone conversations, or similar, can be played through the in-ear headphones 1. Furthermore, frequencies that are less easily perceived by the wearer of the in-ear headphones 1 (this can be verified in a separate hearing test) are amplified, which corresponds to the original functionality of the in-ear headphones 1 as a hearing aid.

[0023] Although the invention has been further illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived from them by a person skilled in the art without departing from the scope of protection of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that the embodiments mentioned as examples are truly only examples and are not to be understood in any way as limiting, for example, the scope of protection, the possible applications, or the configuration of the invention.Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without leaving the scope of protection defined by the claims and their legal equivalents, such as further explanations in the description.

Claims

[1] A method for noise suppression for an occupant of a motor vehicle, wherein an in-ear earphone (1) is used (S1) designed to output acoustic signals, wherein at least one microphone (2) detects ambient noise and a noise compensation (S2) is performed by means of a computing unit (3) to partially eliminate the ambient noise by means of the computing unit (3) determining acoustic signals to generate anti-noise and controlling the in-ear earphone (1) for output, wherein the computing unit (3) executes a pre-trained machine learning model and the machine learning model makes a selection (S3) as to which ambient noises are allowed to pass through to the occupant and which are eliminated,wherein the computing unit (3) is connected to a control unit (4) of the vehicle and the selection is made by the machine learning model based on current information from the control unit (4) of the vehicle, wherein the computing unit (3) decides to allow the speech of a passenger if an acoustic signature of the passenger is known to the machine learning model, and wherein the passenger is prompted to provide a speech sample by the control unit (4) of the vehicle or by the computing unit (3) if the passenger is unknown to the machine learning model in order to derive an acoustic signature of the passenger and store it as known by assigning its acoustic signature. [2] Method according to claim 1, wherein the information of the control unit (4) comprises at least one of the following currently determined: camera data about an environment of the motor vehicle, data from a rain sensor of the motor vehicle, a speed of the motor vehicle, a rotational speed of a drive of the motor vehicle, a gear engaged, design information about wheels and / or tires of the motor vehicle, a tire pressure of at least one tire of the motor vehicle, an acoustic speech profile of the passenger, location data in conjunction with map data comprising at least one of: type of road travelled, characteristic of a roadway travelled, a detected siren of an emergency vehicle, a detected activation of a signal horn of another road user. [3] Method according to one of the preceding claims, wherein a respective acoustic signature is stored by the control unit (4) of the vehicle and transferred to the machine learning model. [4] Method according to any of the preceding claims, wherein the quality and quantity of the noise compensation is dynamically determined by the machine learning model. [5] Method according to any of the preceding claims, wherein an in-ear hearing aid is used as the in-ear earphone (1). [6] A noise suppression system for an occupant while traveling in a motor vehicle, comprising an in-ear earphone (1), a microphone (2), and a processing unit (3), wherein the at least one microphone (2) is configured to detect ambient noise, wherein the in-ear earphone (1) is configured to output acoustic signals, and wherein the processing unit (3) is configured to detect anti-noise and to output corresponding acoustic signals via the in-ear earphone (1), thereby at least partially eliminating the ambient noise, and to execute a pre-trained machine learning model and thereby select which ambient noises are allowed to pass through to the occupant and which are eliminated, wherein the processing unit (3) is connected to a control unit (4) of the vehicle and is further configured toThe machine learning model makes the selection based on information from the vehicle's control unit (4), wherein the computing unit (3) decides whether to allow a passenger's speech to pass through if the passenger's acoustic signature is known to the machine learning model, and wherein the passenger is prompted to provide a speech sample by the vehicle's control unit (4) or by the computing unit (3) if the passenger is unknown to the machine learning model, in order to derive an acoustic signature of the passenger and store it as known by assigning its acoustic signature.

Images