Method for determining head-related transfer function

Abstract

The invention relates to a method for determining a head-related transfer function (HRTF) of at least one person (2) using a vehicle (1), the head-related transfer function (HRTF) being calculated as a function of a feature of the person (2) captured by a camera (4, 6, 7). The invention is characterized in that said feature is captured at least partially in sequence in time by a plurality of cameras (4, 6, 7), at least one of said cameras (4) being arranged to capture the person (2) in the outside area of the vehicle (1) when the person (2) approaches and opens the vehicle (1).

Description

Technical Field

[0001] The invention relates to a method for determining a head-related or outer ear-related transfer function of the type specified in the preamble of claim 1. Furthermore, the invention relates to a method for outputting audio data taking into account the head-related transfer function thus determined. Background Art

[0002] The so-called head-related or outer ear-related transfer function is often also denoted by the abbreviation HRTF, which originates from the English term "Head Related Transfer Function" and has been adopted as a term in other languages. The HRTF describes the outer ear transfer function, which, in addition to the transit time difference between the two ears of a person, also plays a decisive role in the localization of sound sources. Since each person's anatomy is different, the HRTF is unique for each person. The shape of the head, which influences the diffraction of sound in the head and the distance between the ears, as well as the individual shape of the pinna and the ear canal are mainly relevant here. If the HRTF of a person is known, a virtual positioning of sound sources in space can be achieved, thereby realizing the localization of sound sources at a virtually predetermined spatial position. This function is often also referred to as 3D sound.

[0003] However, in many applications, nine or more loudspeakers are required for this purpose. However, if the HRTF of a person is known, only two loudspeakers located in the right and left areas of the person are needed to virtually produce this 3D sound effect. In order to know the HRTF, the basic method known in the general field of audio technology is to perform complex measurements of test signals in each ear canal with the help of a miniature microphone.

[0004] Now, if the anatomical structure or geometry of the head and the auricle and their spatial relative relationship can be optically captured, it may be simpler and more efficient. DE 10 2020 108 449 A1 describes a method for providing at least one user-specific binaural sound signal to a vehicle occupant, wherein the user-specific HRTF is calculated based on the anatomical structure of each person. The anatomical structure of each person is captured here by an interior camera arranged in the vehicle, so that an optimized sound experience with more speakers can be achieved simply and comfortably.

[0005] US2013 / 0194107 A1 also describes the acquisition and storage of HRTFs for generating virtual 3D sound effects through a signal processor. Here, a camera is also used to acquire the anatomical structure on which the calculation is based.

[0006] In practice, it is extremely difficult to capture all relevant anatomical structures by means of cameras installed in the vehicle (e.g., cameras installed in the area in front of the vehicle user), because these cameras generally cannot reliably and sufficiently accurately capture all relevant areas of the head of the corresponding person. Although it is now possible to ask the person to move and turn the head accordingly in front of the camera, this is relatively cumbersome in practice and is considered to reduce comfort if it is necessary to do so every time the 3D sound effect is activated. Summary of the invention

[0007] The object of the present invention is therefore to specify an improved method for determining a head-related transfer function for at least one vehicle occupant. In addition, the object of the present invention is to specify an improved method for outputting audio data.

[0008] According to the invention, this object is achieved by a method having the features of claim 1, and in particular the features of the characterizing part of claim 1. Advantageous refinements and improvements of the method according to the invention are derived from the associated dependent claims. In addition, claim 11 specifies a method for outputting an audio signal, which allows a simple design when using the head-related transfer function determined by the method according to claim 1. Advantageous refinements are also derived from the associated dependent claims.

[0009] The method according to the invention for determining an HRTF for at least one vehicle user detects features of a person, in particular anatomical features of the head and ears, by means of a camera, from which the HRTF is calculated in a manner known per se. The invention provides that a plurality of cameras detect the features at least partially sequentially in time, wherein at least one of the cameras is arranged such that it detects the person in the area outside the vehicle when the person approaches and opens the vehicle.

[0010] Therefore, the method according to the invention allows the capture of a person in the area outside the vehicle when the person approaches and gets on the vehicle. Therefore, an initial basis for the features used to determine the HRTF is provided long before the vehicle is started. Due to the fact that the person is still in the external area (i.e. the external environment of the vehicle) at this moment, good images can be captured by the camera to analyze the corresponding anatomical features. Preferably, the camera can take multiple images continuously here to record processes such as the person approaching the vehicle, opening the door or getting on the vehicle. Therefore, the person can be captured at different angles and generally in different lighting scenarios with different projected shadows. These data can then be processed accordingly and form a valuable initial basis in order to construct a model that is as accurate as possible of the ear and the head area related to the HRTF or the entire head based on the camera data.

[0011] In principle, an interior camera can be used for this purpose, which has a view outward through the vehicle window. This has the advantage that the camera is well protected against vandalism. However, it is particularly advantageous to use an exterior camera which is arranged in the area outside the vehicle. This can be a camera which is provided specifically for this purpose or is particularly preferably an existing camera. Thus, for example, a camera can be used which originally serves to perform facial recognition of approaching persons in order to ensure access to the vehicle based on the recognized face or other biometric features.

[0012] The camera used for this purpose can be arranged in particular in the area of ​​the B-pillars. Additionally or alternatively, it is also conceivable to use a camera in the area of ​​the A-pillars or in the area of ​​the rearview mirrors. In particular, such cameras can also be used if actual rearview mirrors are omitted and replaced by cameras. If a plurality of cameras are already installed in the vehicle exterior area, images can also be generated by different cameras in order to expand the options for possible lighting scenarios, viewing angles of relevant human anatomical features, etc.

[0013] Another particularly advantageous embodiment of the method according to the invention then provides that a further camera detects features of the person in the vehicle interior. In particular, a vehicle interior camera known per se and generally available can be used for this purpose. This can be, for example, a central camera in a roof control unit or a camera that is originally used to detect gaze direction, detect gestures, monitor the driver's attention, etc.

[0014] In particular, according to an extremely advantageous development of the method according to the invention, provision can be made, for example, for the first side of the head to be recorded when the person opens the door and then enters the vehicle as intended, using a camera in the vehicle dashboard, and for the second side of the head to be recorded when or after entering the vehicle, using a camera in a roof control unit centrally arranged between the seats. In the case of left-hand drive, such a camera can, for example, first record the left side of the head when the driver enters the vehicle, and then record the right side of the head with a central camera, in particular integrated in the roof control unit. These data can then be combined with a front view of the person recorded after entering the vehicle by means of the camera in the dashboard and with images recorded from the vehicle exterior area before entering the vehicle, in order to achieve an ideal modeling of the head or of the head area and ears associated with the HRTF. Based on this, the HRTF can then be calculated simply and efficiently and can be stored accordingly, and can be directly accessed when the 3D audio data are to be played back in the vehicle, when the person is recognized again, for example by facial recognition or by a personalized key, a personalized smartphone, a smartwatch, etc.

[0015] Prior acquisition in the area outside the vehicle also allows efficient acquisition of the person in any situation, so that by comparison with the stored data it can be determined whether the person's anatomy has changed, which generally occurs over time. In the case of unchanged anatomy, no additional features need to be acquired to recalculate the HRTF. If the anatomy of the area relevant to the HRTF has changed accordingly, acquisition can be performed according to the above steps to recalculate the HRTF.

[0016] According to another very advantageous embodiment of the method according to the invention, provision can be made here for illuminating the person's head with near-infrared light during the acquisition, which also applies in particular to acquisition in the area outside the vehicle. This also allows efficient acquisition of relevant features of the respective person under poor visibility conditions, for example at dusk or in the dark, so that the features required for the HRTF can be efficiently acquired and the function can be reliably calculated.

[0017] If the system finds that more data is needed after the person gets on the bus because not all perspectives are captured correctly, especially when the person gets on the bus for the first time, the method can also require the person to turn his or her head from one side to the other after getting on the bus when there is such a need, so as to further improve the feature capture for HRTF calculation.

[0018] Advantageously, the method can be used to improve audio systems installed in vehicles. For this purpose, the data obtained using the HRTF can be transmitted in real time via Ethernet to a central audio amplifier, for example. This processes the audio signal accordingly via algorithms adapted to the vehicle cabin in a DSP (digital audio processor) and sends it via cables to the loudspeakers in the vehicle seats.

[0019] Using the method according to claim 9, a plurality of loudspeakers in a vehicle can then be advantageously controlled.

[0020] If the HRTF of a person is known, a spatial virtual localization of the sound source can be achieved. In this case, the signal is convolved with the HRIR (Head Related Impulse Response) of the corresponding sound incident direction, which is known per se in the prior art, to produce a localization from that direction.

[0021] 3D sound is the future trend, but it requires 9 or more speakers. If a person's HRTF is known, 3D sound can be virtually generated with only two speakers. Therefore, two speakers can be used to produce sound effects similar to those of 9 speakers.

[0022] As mentioned above, HRTF can now be calculated based on the collected features, thereby generating virtual 3D sound effects to ensure that the people in the car get the best 3D sound experience. In order to achieve this at a low hardware cost, the audio data output method of the present invention can stipulate that the output of audio data is performed by an audio processor based on the HRTF calculated for the corresponding person, wherein two speakers integrated in the vehicle seat of the corresponding person are used in the vehicle. Therefore, unlike the prior art method described in the preface, this method only uses two speakers for each person to achieve virtual 3D sound effects. Here, these speakers are integrated in the vehicle seat of the corresponding person. According to a particularly advantageous design, they are placed in the seat back at approximately the shoulder height of a person of average size. Ideally, they are connected to the audio processor via a cable to transmit the virtual 3D sound effects generated by the audio processor in real time, while avoiding the delay that may be caused by the wireless connection, so that the corresponding person can get the best sound experience based on his or her individual HRTF. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Further advantageous designs of the method according to the invention also result from the following embodiments which are described in detail with reference to the drawings, wherein:

[0024] Figure 1 is a schematic side view of a vehicle showing a person approaching;

[0025] Figure 2 is a top view of a vehicle, showing the vehicle interior and people inside the interior. DETAILED DESCRIPTION

[0026] When Figure 1 and Figure 2 When listening to music in the vehicle 1 shown in the figure, the signal source (e.g. music) is not directly introduced into the inner ear, but experiences different reflection patterns at the body, head and auricle depending on the direction of incidence of the sound. Here, the superposition of direct sound and reflection causes interference, which the human brain may interpret as a signal direction. The so-called head-related transfer function (sometimes also called outer ear-related transfer function) also describes the transfer from the outer ear area to the ear canal. It is also generally referred to as HRTF, which comes from the English term "Head Related Transfer Function". For the sound generated by Figure 1 and Figure 2 The transfer function and the difference in transit time between the two ears are crucial for localizing the sound source for the corresponding person 2 shown in the figure. Since each person has a different anatomy, the HRTF is unique for each person. Related to this is the individual head shape, which influences the diffraction of the sound at the head and the distance between the ears. In addition, the individual shape of the pinna and the ear canal also plays a role.

[0027] In order to now allow a high-quality 3D sound experience by simple means through spatial virtual localization of sound sources, it is now important to know the individual HRTF of the respective person 2. The audio signal can then be convolved with the so-called head-related impulse response (HRIR) of the respective incident direction, thereby generating a localization from said direction. Figure 2 The digital audio processor (DSP) 3 shown thus allows a 3D sound effect to be generated for the respective person 2 .

[0028] In order to now easily and efficiently collect the individual HRTF of each person 2, such as Figure 1 As shown in the diagram of , when a person 2 approaches a vehicle 1 in order to open it, the person's face or head is captured by means of a first camera 4 in the so-called B-pillar of the vehicle 1. The camera 4 is usually already present and is used to unlock the vehicle 1, for example for authorized persons, by means of facial recognition. First data can now be captured by the camera under different viewing angles toward the head of the person 2 and under different lighting conditions in order to achieve facial recognition on the one hand and to define the coordinates of facial features such as eyes, nose, mouth, ears and head shape on the other hand by means of image analysis methods and mathematical calculations. Preferably, the data is captured here by means of near-infrared light, which is reflected by the face in order to ensure adequate facial recognition in all lighting conditions. First data for calculating the individual HRTF are therefore provided by means of this first data set.

[0029] When a door 5, for example the driver's door, is opened and when subsequently getting into the vehicle, a display in the instrument cluster of the vehicle 1 is displayed. Figure 2 The second camera 6, which can be seen in the diagram of , then captures the anatomical structures on the left side of the head of the person 2 when getting on the vehicle. The system for calculating the HRTF thus obtains another data set. The anatomical data on the right side of the head can now be captured by another camera 7 in the roof control unit, which is generally arranged on the roof, i.e. in its roof area. Finally, the second camera 6 optionally captures the head or face of the person 2 from the front to generate more data.

[0030] If the vehicle sound system now asks whether it is necessary to enable personalized high-quality sound effects and is confirmed accordingly, a 3D audio signal is output based on the unchanged individual characteristics of the vehicle interior, and an individual HRTF is generated specifically for the corresponding person 2. If the existing data is not sufficient to determine a reliable individual HRTF at this time, the vehicle's multimedia system can also ask the person 2 to slowly turn his head from left to right again. In this way, more data can be collected by the two cameras 6 and 7 in the vehicle interior and compared with the previously collected data or supplemented. Ideally, the data collected in this way is then transmitted to the central audio amplifier 8 in real time via the in-vehicle Ethernet so that it can be processed or processed accordingly there by an algorithm matched to the vehicle interior. Through the DSP 3 mentioned above, the audio signal is then transmitted to the speakers 10 and 11 preferably connected by a cable 9. They are integrated in the vehicle seat 12 of the person 2 and in particular in its seat back 13, roughly at the shoulder height of a person of average size. By using only two such speakers 9, 10 at shoulder height in the seat back 13, a virtual 3D soundscape can be generated, bringing an unparalleled auditory enjoyment. If the corresponding algorithm and HRTF are not used, the speakers 10, 11 can also be used for simple stereo playback, allowing the person 2 to experience sound effects similar to those from behind.

[0031] When approaching and getting on board, the anatomy of the approaching person 2 can now be automatically acquired by the first camera 4 in the B-pillar of the vehicle 1 and compared with the stored data. Since the anatomy of a person and therefore his HRTF can change over time, readjustments can be made accordingly. If improved algorithms are required at a later time, the entire system can also be upgraded, in particular remotely online. Through the multimedia system of the vehicle 1, personalized high-quality sound effects can then be enabled for all or only selected persons 2, so that 3D sound effects are used whenever a person 2 is recognized in the vehicle, for example based on facial recognition or also based on a personalized car key, smartphone, smartwatch, etc. that has been registered in the vehicle 1 and is bound to the person. Therefore, for example, it is also possible to allow only certain persons 2 to enjoy high-quality sound effects and, for example, to prohibit other users of the vehicle 1 accordingly.

Claims

1. A method for determining a head-related transfer function (HRTF) of at least one person (2) using a vehicle (1), wherein: The head-related transfer function (HRTF) is calculated according to the features of the person (2) collected by the cameras (4, 6, 7), Its characteristics are: The features are captured at least partially sequentially in time by a plurality of cameras (4, 6, 7), wherein at least one of the cameras (4) is arranged to capture the person (2) in an area outside the vehicle (1) when the person (2) approaches and opens the vehicle (1).

2. The method according to claim 1, characterized in that: An external camera (4) of the vehicle (1) is used as a camera (4) configured to capture images of the person (2) in an external area.

3. The method according to claim 2, characterized in that: The camera used as an exterior camera (4) is arranged in the region of a B-pillar and / or an A-pillar or in the region of a rearview mirror of the vehicle (1), wherein the rearview mirror can also be formed by the camera.

4. The method according to any one of claims 1 to 3, characterized in that: Other cameras (6, 7) capture features of the person (2) in the interior of the vehicle (1).

5. The method according to claim 4, characterized in that: The camera (6) in the dashboard is used to capture the first side of the head when getting on the vehicle, and the camera (7) in the roof control unit located in the center between the seats is used to capture the second side of the head when or after getting on the vehicle.

6. The method according to any one of claims 1 to 5, characterized in that: During the acquisition, the head of the person (2) is illuminated with near infrared light.

7. The method according to any one of claims 1 to 6, characterized in that: The features and the head-related transfer function (HRTF) of the corresponding person (2) are stored, wherein when the person (2) uses the vehicle (1) again, the features are re-collected and compared with the stored features, wherein if the features change, the head-related transfer function is adjusted.

8. The method according to any one of claims 1 to 7, characterized in that: As required, after getting on the vehicle, the person (2) is asked to turn his head from side to side while capturing images to improve the feature capture for calculating the head-related transfer function (HRTF).

9. The method according to any one of claims 1 to 8, characterized in that: The transfer function is used to control a digital audio processor and condition the audio signal.

10. The method according to claim 9, characterized in that: At least two speakers are controlled using the audio signal.

11. A method for outputting 3D audio data through an audio processor (3) based on a determined head-related transfer function (HRTF), wherein: Exactly two loudspeakers (10, 11) are used as sound sources and are integrated into the vehicle seat (12) of the respective person (2).

12. The method according to claim 11, characterized in that: The two loudspeakers (10, 11) are integrated into the seat back (13) at shoulder height for an average-sized person (2).

Images