Method for generating and reproducing a binaural recording

Abstract

A method for providing a binaural recording for a listener (34) having a head (38), wherein the method is carried out with a hearing system (12), wherein the binaural recording is heard using a hearing device (36), and wherein the binaural recording consists of a left binaural ear signal intended for a left ear of the listener (34) and a right binaural ear signal intended for a right ear of the listener (34), comprising the method steps: Providing the binaural recording consisting of the left binaural ear signal and the right binaural ear signal to the hearing system (12), Determining a head orientation (18) of the listener (34); Determining a source direction (24) of the provided binaural recording with respect to the head orientation (18); detecting a change in the head orientation (28) to a new head orientation (30); and Adjusting the provided binaural recording taking into account the determined source direction (24) of the binaural recording and the detected new head orientation (30).

Description

[0001] Human listeners are able to identify the location or origin of a detected sound in terms of direction and distance. To do this, people estimate the location of a source by using information derived from one ear (monaural information) and comparing information received at both ears (difference information or binaural information). This information includes time and level differences between the two ears, spectral information, temporal analysis, correlation analysis, and pattern matching.

[0002] A head-referred transfer function (HRTF) is a response that characterizes how an ear receives a sound from a point in space. As a sound reaches the listener, the size and shape of the head, the ears, the ear canal, and the density of the head transform the sound and influence its perception, amplifying some frequencies and attenuating others. HRTFs can vary considerably from person to person. A pair of HRTFs for two ears can be used to synthesize a binaural tone that appears to come from a specific point in space. Thus, for a listener listening to the binaural recording with headphones, a 3D stereo sound sensation is created, while the listener perceives the sound as if they were actually present in the room with the sound source.

[0003] Binaural recordings, made with artificial heads or in-ear microphones and played back with headphones, are a well-established technique for capturing and reproducing much of the spatial information that humans perceive in an acoustic scene. However, during normal listening with headphones, the signals do not change with head movements, so sound sources in the acoustic scene appear to rotate with the listener's head, which impairs the listening experience. As a result, the listener encounters difficulties locating the sound source and perceives it as being in the head.

[0004] When two speakers are used to play back the binaural recording, the signal from one speaker will also be heard by the contralateral ear (superimposition), thereby reducing the desired effect of the binaural recording. Therefore, superimposition filters are used to reduce this effect and create a realistic sound experience. Essentially, superimposition filters take a portion of the left speaker signal and feed it to the right speaker signal, combining it with the right speaker signal to cancel out the portion of the signal heard by the left ear, and vice versa. However, similar to listening through headphones, the signals do not change with the listener's head movements, resulting in similar adverse effects on the listening experience.

[0005] However, if the binaural recordings were made with in-ear microphones, the movement of a sound engineer's head during recording is linked to the binaural recording. Therefore, the listener of such a binaural recording encounters difficulties locating the sound source, as it is perceived as non-static.

[0006] US 9848273 B1 describes a hearing system with one or more hearing aids configured to be worn by a user. Each hearing aid includes a signal source providing an electrical input signal representing a sound from a virtual source. A filter implements a head-related transfer function (HRTF) to add spatialization information associated with a virtual location of the virtual source to the electrical signal and outputs a filtered electrical signal containing the spatialization information.

[0007] US 7333622 B2 describes a different approach for recording and playing back either live or pre-recorded three-dimensional sound. The method uses multiple microphones, a head tracker, and special signal processing techniques to combine the signals picked up by the microphones.

[0008] US 2018 / 0 210 695 A1 describes a method for rendering audio for playback via headphones, comprising receiving digital audio content, receiving binaural rendering metadata generated by an authoring tool that processes the received digital audio content, receiving playback metadata generated by a playback device, and combining the binaural rendering metadata and playback metadata to optimize playback of the digital audio content via the headphones.

[0009] The object of the present invention is to provide a more realistic and improved listening experience for a listener of a binaural recording. In particular, the object of the present invention is to overcome deficiencies of the prior art, such as limited source localization and a lack of externalization, i.e., the perception of sources within the listener's head.

[0010] This problem is solved by the independent claims. Advantageous embodiments are specified in the subclaims.

[0011] In particular, the present invention provides a method for providing a binaural recording for a listener with a head, wherein the method is carried out with a hearing system, wherein the binaural recording is heard using a hearing device, and wherein the binaural recording consists of a left binaural ear signal intended for a left ear of the listener and a right binaural ear signal intended for a right ear of the listener, and comprises the following method steps: providing the binaural recording consisting of the left binaural ear signal and the right binaural ear signal to the hearing system, determining a head orientation of the listener, determining a source direction of the provided binaural recording with respect to the head orientation,Detecting a change in head orientation to a new head orientation and adjusting the provided binaural recording taking into account the determined source direction of the binaural recording and the detected, new head orientation. In short, the method determines the orientation and changes in the orientation of the listener's head and adjusts the binaural recording accordingly.

[0012] The present invention also provides a method for recording a binaural recording, the method being carried out with a hearing system, the binaural recording being recorded using a recording device worn on a head of a sound engineer, and the binaural recording consisting of a left binaural ear signal received by the recording device near and / or at a left ear of the sound engineer, and a right binaural ear signal received by the recording device near and / or at a right ear of the sound engineer, and comprising the following method steps: recording the binaural recording consisting of the left binaural ear signal and the right binaural ear signal by means of the recording device, determining a head orientation of the sound engineer,Determining a source direction of the recorded binaural recording with respect to the head orientation, detecting a change in head orientation to a new head orientation, and adjusting the recorded binaural recording taking into account the determined source direction of the binaural recording and the detected new head orientation. In short, the method determines the orientation and changes in the orientation of the sound engineer's head and adjusts the binaural recording accordingly.

[0013] Furthermore, the present invention provides a method for providing a binaural recording to a listener, wherein the binaural recording is recorded by the second method mentioned above and the binaural recording is provided to the listener by the first method mentioned above. In other words, the method is a combination of the aforementioned methods.

[0014] The basic idea of ​​the invention is to analyze the binaural recording to determine the source direction in the original scene. The binaural recording is then modified to account for changes in the head orientation of the listener and / or the sound engineer, taking the source direction into account.

[0015] This basic idea can be incorporated into the process of listening to a binaural recording, resulting in a more realistic and enhanced listening experience. For the listener, the perceived source direction relative to the environment appears fixed, even though the listener moves their head during listening. Therefore, source localization, externalization, and realism of binaural recordings are improved.

[0016] Furthermore, the basic idea can be incorporated into the process of recording a binaural recording using a recording device worn on a sound engineer's head. Alternatively, a recording device can also be worn on a movable and / or rotatable dummy head. In this case, the method offers a way to eliminate the trajectories or concomitant effects of the movements and / or rotations of the sound engineer's head. For the listener of such binaural recordings, the perceived source direction appears to be fixed with respect to the environment, even though the sound engineer's head / dummy head has moved during recording. This enhances the listening experience. Furthermore, the method paves the way for simpler and more cost-effective binaural recording techniques.

[0017] It's also possible to incorporate the basic idea into both the process of recording a binaural recording and the process of listening to the binaural recording, thus combining the benefits of both. An enhanced and more realistic listening experience can be created while using a simple and inexpensive recording technique.

[0018] The methods are computationally simple and can be performed in real time. Because the methods adapt the binaural recording either at the location where it is recorded and / or at the location where a listener hears the binaural recording, the methods do not require any special signal processing or storage technologies between the recording process and the listening process. No special sound cards, data formats, software and / or computer platforms are required between recording and playback. The binaural recording can be saved in a common stereo format, e.g. MP3, on common electronic devices such as smartphones or other storage media such as USB sticks, memory cards or CDs. This also makes it possible to share the binaural recordings via a website and / or a server.Furthermore, the methods do not require special recording techniques that feature microphone arrays with more than two microphones used to sample the sound field during recording. Therefore, the methods are easy to apply and incur no additional costs. The methods result in an improved listening experience. An listening experience is a way in which the listener perceives sounds. It may involve determining the location of the sound source. The listener's brain uses subtle differences in loudness, timbre, and timing between the two ears to enable localization of the sound source. A listening experience in a real environment, such as listening to a concert in a music hall, is often more valued than listening to a recording of the concert on a listening device such as headphones or a stereo system.

[0019] According to the invention, a listener can be any type of person who hears sounds, e.g. music, speech or noises.

[0020] A binaural recording is a recording of sound signals that create a realistic listening experience when listening to the binaural recording. The realistic listening experience can include localization of the sound source. According to the invention, the binaural recording can be a recording of any type of sound. For example, it can be a recording of music, such as a concert; speech, such as an audiobook or a conference call; noises, or a combination of these, such as the soundtrack for a film.

[0021] The binaural recording consists of a left binaural ear signal and a right binaural ear signal. Binaural recordings can be heard through hearing aids worn on the listener's head. This makes it possible for different sound signals to be delivered to the listener's right and left ear, allowing the listener to perceive a realistic listening experience. Typical hearing aids worn on the listener's head are headphones, earbuds, in-ear or on-ear earphones, and the like. It is also possible for the hearing aid to be part of a headset, e.g., for a telephone, or part of a virtual reality headset. In addition, binaural recordings can be played back through loudspeakers using heterodyne suppression filters. The heterodyne suppression filters can be applied directly to the binaural recording.When listening to a binaural recording, the left binaural ear signal is the signal delivered to the left speaker and / or earphone near the listener's left ear, and is therefore intended for the listener's left ear. The right binaural ear signal is the signal delivered to the right speaker and / or earphone near the listener's right ear, and is therefore intended for the listener's right ear.

[0022] Binaural recordings can be created using various binaural recording techniques, e.g., using a dummy head, a Jecklin disc, and / or a pair of in-ear or on-ear microphones. According to the invention, it is not necessary to use a special recording technique with more than two microphones. When recording a binaural recording, the left binaural ear signal is the signal received by the recording device near and / or at the left ear of the sound engineer or the dummy head, or by the left microphone of the Jecklin disc. The right binaural ear signal is the signal received by the recording device near and / or at the right ear of the sound engineer or the dummy head, or by the right microphone of the Jecklin disc.

[0023] According to the invention, a sound engineer can be any person who makes a binaural recording using a recording device worn on the sound engineer's head. However, according to the invention, a sound engineer should also be understood to mean a dummy head to which a recording device is attached, or a Jecklin disc with two microphones. The dummy head or Jecklin disc can be mounted on a tripod, a stand, or a rotating device. The important feature is that the recording device on the sound engineer's head or on the dummy head / Jecklin disc can be moved and / or rotated together with the head. In the case of a dummy head or Jecklin disc, the rotational movement / movement can be achieved by displacing or rotating the dummy head or Jecklin disc.

[0024] Typical recording devices worn on the sound engineer's head are in-ear or on-ear microphones. It is also possible for the recording device to be part of a camera mounted on the dummy head and / or on a tripod to which the dummy head is attached. Alternatively, two microphones can be used on either side of a Jecklin disc.

[0025] According to the invention, head orientation is the orientation of the listener's head during listening and / or the orientation of the sound engineer's head, the dummy head, or the Jecklin disc during the recording process. The orientation can refer to any axis of the head with respect to an external environment of the head or with respect to another frame of reference. Preferably, the head orientation is the orientation of an axis perpendicular to an interaural junction of the head. For a human head, this axis corresponds to a line of sight. For a dummy head and a Jecklin disc, the interaural junction refers to the connection between the two microphones used for recording.

[0026] The source direction is the direction of the sound source. During the recording of a binaural recording, the source direction in the surrounding area corresponds to the source direction of the binaural recording. When listening to a binaural recording, the source direction is the perceived direction of the sound. The source direction can be the direction of a single source when only one source is emitting sound. It is also possible for the source direction to be a combined direction of a multi-source scene in which several sources are emitting sound. Typically, the source can be placed independently of the listener and / or the sound engineer. Therefore, it can be located behind the listener and / or the sound engineer, next to the listener and / or the sound engineer, or in front of the listener and / or the sound engineer.

[0027] Preferably, the sound source is located in front of the listener and / or the sound engineer.

[0028] The methods include the step of determining a head orientation. This head orientation may be an orientation relative to the external environment of the listener or the sound engineer. However, the step of determining a head orientation may also be a definition of an internal frame of reference of the head. Preferably, an initial head orientation defines an origin. All subsequent head orientations may be measured relative to this orientation. Alternatively, an average of various head orientations over a given time interval may be used to define an origin.

[0029] In a further step, the source direction of the binaural recording is determined relative to the head orientation. In the case of a listener, the source direction is the perceived source direction. In the case of a sound engineer, the source direction of the binaural recording corresponds to the source direction in the environment. The source direction of the binaural recording can be determined by analyzing the binaural recording. Determining the source direction is independent of any knowledge of the sound engineer's or the listener's environment. After this step, the source direction relative to the head orientation is known.

[0030] The methods then detect a change in head orientation to a new head orientation. Generally, this can be a small change in orientation or a large change. The change can occur quickly or slowly. The change in head orientation can occur in any direction. It is possible that the change is detected by measuring the head orientation according to set time intervals. After this step, the amount of change in head orientation and the new head orientation with respect to the head's internal frame of reference are known. Alternatively or additionally, the amount of change in head orientation and the new head orientation with respect to a previous head orientation are known. It is also possible that after this step, the amount of change in head orientation and the new head orientation with respect to an external frame of reference are known.

[0031] In a further step, the methods adapt the binaural recording, taking into account the source direction of the binaural recording and the new head orientation. This step causes the binaural recording to be modified according to the head movement and the determined source direction.

[0032] If these steps are implemented in the recording process, the effects are as follows: If a sound engineer moved his or her head during recording, or if a dummy head supporting the recording device moved during recording, this movement will be reflected in the binaural recording. The final step of the procedure allows this movement to be taken into account in the binaural recording. By adjusting the binaural recording, the influence of the movement on the binaural recording can be compensated. This creates a binaural recording that sounds as if the sound engineer's head had not moved during recording.

[0033] When the steps in the listening process are performed, the effects are as follows: When a listener moves his or her head, the sound usually moves with the head, creating the impression that the sound is inside the listener's head. The final step of the process allows the head movement to be taken into account and the movement to be reflected in the binaural recording. By adjusting the binaural recording, the influence of the movement can be taken into account. Therefore, a binaural recording is produced that sounds as if the source direction is fixed in the listener's environment, ensuring an enhanced listening experience.

[0034] According to a preferred embodiment of the invention, the step of detecting a change in head orientation to a new head orientation comprises detecting a rotational movement of the head. The head of the listener or the sound engineer can, in principle, move laterally or be rotated. However, a rotational movement of the head has a greater influence on the binaural recording than a lateral movement of the head. Preferably, the rotational movement of the head is a rotational movement about an axis perpendicular to the ground. In other words, it is a rotational movement of the head comparable to the movement of the head when crossing a busy road or to shaking the head during a disagreement. Alternatively or additionally, a rotational movement about another axis, for example a head nod, can also be considered.

[0035] According to a preferred embodiment of the invention, the step of determining a source direction of the binaural recording with respect to the head orientation and the step of adapting the binaural recording taking into account the source direction of the binaural recording and the new head orientation comprise calling up a map that relates the head orientation to a difference between the left binaural ear signal and the right binaural ear signal for a predetermined source direction. The steps of determining the source direction of the binaural recording and adapting the binaural recording taking into account the source direction of the binaural recording and the new head orientation both comprise calling up a map. According to the invention, a map can be a multidimensional map or an arrangement of data in a complex structure.The map creates a link between the head orientation and the binaural recording. The binaural recording consists of a left binaural ear signal and a right binaural ear signal that differ from each other. The way in which the left and right binaural ear signals differ depends on the head orientation relative to the source direction. This information can be stored in the map. The map can contain information about the difference in amplitude or level, known as the interaural level difference. (ILD^) and / or provided as a phase difference, referred to as interaural phase difference (IPD). Therefore, the map allows the source direction to be determined relative to the head orientation, independent of any knowledge of the sound engineer's or listener's environment. This includes only the binaural recording and the map.

[0036] For this purpose, according to a preferred embodiment of the invention, the step of determining a source direction of the binaural recording with respect to the head orientation of the listener or the sound engineer comprises determining a difference between the left binaural ear signal and the right binaural ear signal and comparing this difference with the characteristic map relating the head orientation to the difference between the left binaural ear signal and the right binaural ear signal for a given source direction. To determine the source direction, the method determines the difference between the left binaural ear signal and the right binaural ear signal. This can be an interaural level difference (ILD) of the binaural recording and / or an interaural phase difference (IPD) of the binaural recording. This difference can then be compared with the value ILD^ and / or IPD^ in the characteristic map. This allows the source direction to be determined. Preferably, this determination of the source direction includes a minimization approach in which the values ILD^ and / or IPD^ The values ​​in the characteristic map that best match the ILD and / or IPD values ​​of the binaural recording can be determined. This allows for quick and reliable determination of the source direction.

[0037] The step of adapting the binaural recording taking into account the source direction of the binaural recording and the new head orientation preferably includes retrieving the characteristic map. To this end, according to a preferred embodiment of the invention, the step of adapting the binaural recording taking into account the source direction of the binaural recording and the new head orientation comprises determining the new head orientation with respect to the source direction of the binaural recording and comparing this head orientation value with a characteristic map that relates the head orientation to the difference between the left binaural ear signal and the right binaural ear signal for a given source direction.To determine the type of adjustment of the binaural recording with respect to the head movement of the sound engineer or listener, the new head orientation relative to the source direction of the binaural recording is determined. This value can then be compared with the characteristic map to determine the type of adjustment of the binaural recording. The characteristic map can display the values. ILD^ and / or IPD^ for different head orientations and therefore contains information on how the binaural recording can be adjusted to account for head movement.

[0038] According to a preferred embodiment of the invention, the characteristic map relating the head orientation to the difference between the left binaural ear signal and the right binaural ear signal for a given source direction relates the head orientation to the difference in amplitude and / or to the phase difference between the left binaural ear signal and the right binaural ear signal for a frequency range of the right and left binaural ear signals and for a given source direction. Preferably, the characteristic map creates a link between the head orientation and the difference in amplitude ILD^ and / or the difference in phase IPD^ between the left binaural ear signal and the right binaural ear signal. Preferably, the map contains both information, the values ILD^ and IPD^ which allows for better determination of the source direction and also better adjustment of the binaural recording. The values ILD^ and / or IPD^ can be different for different frequencies of the binaural ear signal, so that the map shows the values ILD^ and / or IPD^ for a range of frequencies, e.g., from 10 Hz to 20 kHz, which covers the acoustic range of most people. The map contains the values ILD^ and / or IPD^ as a function of head orientation for a given source direction. The source direction can be specified relative to an internal reference frame of the listener's or sound engineer's head. For example, the map can be created for a source direction of 0 degrees, which means that the sound source coincides with the origin of the head's internal reference frame. The values ILD^ and / or IPD^ in the map are small for head alignments close to 0 degrees. At a head alignment of 0 degrees, the values ​​are ILD^ and / or IPD^ Zero. This means that in the step of adjusting the binaural recording to account for the new head orientation, the binaural recording will not be modified at all if the new head orientation matches the head orientation determined in the first step. In other words, if the listener or sound engineer does not move their head, the binaural recording will not be modified.

[0039] According to a preferred embodiment of the invention, the characteristic map relating the head orientation to the difference between the left binaural ear signal and the right binaural ear signal for a given source direction is adaptable to the head of the listener or the sound engineer and / or to the ears of the listener or the sound engineer and / or to an environment of the listener or the sound engineer. The method comprises two steps, which include calling up the characteristic map. The information stored in the characteristic map about ILD^ and IPD^ is important for determining the source direction and adjusting the binaural recording. The specific values ​​stored in the map for ILD^ and IPD^ can be generated using an analytical spherical head model and / or pre-measured, head-related transfer functions. The head-related transfer functions can be measured for a dummy head and / or the head of the sound engineer and / or the listener. This opens up the possibility of adapting the map to the specific head and / or ear shape of the listener or sound engineer. In particular, the shape of the outer ear (pinna) is important and influences the perception of sounds. Therefore, if the information in the map is specifically adapted for a listener or sound engineer, the listening experience is significantly improved. It is also possible to adapt the map to the listener's or sound engineer's environment to achieve a more realistic listening experience.

[0040] The present invention also provides a hearing system, wherein the hearing system is designed to carry out the first-mentioned method, comprising a hearing device and a head movement tracking device, wherein the head movement tracking device is designed to determine a head orientation and / or a change in the head orientation of the listener, and / or wherein the hearing system is designed to carry out the second-mentioned method with a recording device and a head movement tracking device, wherein the head movement tracking device is designed to determine a head orientation and / or a change in the head orientation of the sound engineer

[0041] The hearing aid can be any type of hearing aid suitable for listening to a binaural recording. Examples include loudspeakers with crosstalk suppression filters, headphones, earbuds, in-ear, or on-ear earphones. The recording device can be any type of recording device capable of recording different sound signals near and / or at the right and left ears of the sound engineer. The sound engineer can be a person, a dummy head with a recording device attached, or a Jecklin disc with two microphones. Examples include in-ear or on-ear microphones. Alternatively, the recording device can be part of a camera mounted on the dummy head, or the recording device can consist of two microphones on either side of a Jecklin disc.

[0042] The hearing system also includes a head tracking device. This can be, for example, an electromagnetic tracking system with 6 degrees of freedom, a consumer virtual reality headset, or a tracking device based on inertial measurement units. It is also possible to use an optical head tracking device, such as a camera or set of cameras. Some of these determine absolute positions from which the orientation of the head can be calculated, while others determine the orientation of the head. It is also possible for the head tracking device to be an acoustic head tracking device.

[0043] The acoustic head tracking device consists of a head-mounted microphone array with multiple microphones. Preferably, the array comprises four microphones arranged at the vertices of a tetrahedron. The acoustic head tracking device tracks the movement of the head in the acoustic field, i.e., it uses differences in the acoustic signals received by the microphones to determine the head orientation. The differences in the acoustic signals can be a time delay between the different signals from the microphones. Together with information about the geometry of the microphone array, the source position of the acoustic signal and the head orientation can be determined.

[0044] According to a preferred embodiment of the invention, the hearing device and / or recording device is configured to be worn by a listener and / or a sound engineer. The hearing device can be any type of hearing device that can be worn by a listener and can provide different sound signals for the listener's right ear and left ear. For example, hearing devices are headphones, earphones, in-ear, or on-ear earphones. Examples of recording devices are in-ear or on-ear microphones. It is also possible for the recording device and the hearing device to be integrated into the same device. Furthermore, it is possible for the sound engineer and the listener to be the same person.

[0045] According to a preferred embodiment of the invention, the hearing system comprises a memory device for storing a characteristic map relating the head orientation to a difference between the left binaural ear signal and the right binaural ear signal for a given source direction. The aforementioned method comprises two steps in which the characteristic map can be retrieved. The information stored in the characteristic map about the values ILD^ and IPD^ can be used to determine the source direction and to adjust the binaural recording. Therefore, the hearing system has a memory device for storing this map. It is also possible to store several different maps. The different maps can be specific to the listener and / or the sound engineer using the hearing system. Alternatively or additionally, the different maps can be specific to different environments in which binaural recordings are recorded and / or listened to. It is also possible to store the map on a server, with the hearing system being connected to this server via wireless communication technology.

[0046] These and other aspects of the invention will become apparent from the embodiments described below and will be explained with reference thereto. Individual features presented in the embodiments may, alone or in combination, constitute an aspect of the present invention. Features of the various embodiments may be transferred from one embodiment to another.

[0047] They show: Fig. 1 is a flowchart of the steps of the methods according to a preferred embodiment of the invention; Fig. 2 shows a sound engineer with a recording device making a binaural recording with two different head orientations according to a preferred embodiment of the invention; Fig. 3 shows a listener with a hearing device listening to a binaural recording with two different head orientations according to a preferred embodiment of the invention; Fig. 4 is a functional diagram of the methods according to a preferred embodiment of the invention; and Fig. 5 a representation of two different characteristic maps according to a preferred embodiment of the invention.

[0048] Fig. 1 shows a flowchart of the steps of the method for providing a binaural recording to a listener, which is applied in a hearing system, according to a preferred embodiment of the invention. Furthermore, Fig. 1 shows the steps of the method for recording a binaural recording, which is used in a hearing system, according to a preferred embodiment of the invention. The steps of the two methods are the same, regardless of whether they are incorporated into the process for listening to a binaural recording or into the process for recording a binaural recording.

[0049] Fig. Figure 2 shows a sound engineer 10 with a hearing system 12 according to a preferred embodiment of the invention. The hearing system 12 has a recording device 14, in this embodiment of the invention a headset with two in-ear microphones. A head tracking device 16 is integrated into the headset, which is configured to determine an orientation 18 of the head 20 of the sound engineer 10. The sound engineer 10 is in the process of recording a binaural recording of sounds emitted by a sound source. The sound engineer 10 in Fig. 2 has two different orientations of the head 20.

[0050] The individual steps of the procedure for recording a binaural recording are described below with reference to the flow chart in Fig. 1 and the sound engineer 10 in Fig. 2 briefly described.

[0051] The first step of the method, S100, is to determine the orientation 18 of the head 20 of the sound engineer 10. This orientation 18 of the head 20 of the sound engineer 10 is on the left side of Fig. 2. In this embodiment of the invention, the orientation 18 defines an origin of a reference frame and is therefore zero degrees.

[0052] In a next step of the method, S200, a source direction 24 of the binaural recording is determined with respect to the head orientation 18 of the sound engineer 10. The source direction 24 is a direction toward the sound source 22. It is determined without any knowledge of the environment of the sound engineer 10, but by analyzing the binaural recording. After this step, the angle 26 between the orientation 18 of the head 20 of the sound engineer 10 and the source direction 24 is known.

[0053] In a further step of the method, S300, a change in the head orientation 28 is detected. This change in the head orientation 28 leads to a new orientation 30 of the head 20 of the sound engineer 10. This new orientation 30 of the head 20 of the sound engineer 10 is on the right side of Fig. 2 shown.

[0054] In the final step of the method, S400, the binaural recording is adjusted taking into account the source direction 24 of the binaural recording and the new head orientation 30. Therefore, this step involves determining the angle 32 between the new head orientation 30 and the source direction 24.

[0055] Fig. Figure 3 shows a listener 34 with a hearing system 12 according to another preferred embodiment of the invention. The hearing system 12 has a hearing device 36, in this embodiment of the invention a headset. A head tracking device 16 is integrated into the headset and is configured to determine an orientation 18 of the head 38 of the listener 34. The listener 34 is currently listening to a binaural recording. The listener 34 in Fig. 3 shows two different orientations of the head 38.

[0056] In the following, the individual process steps for providing a binaural recording for a listener 34 are described with reference to the flow chart in Fig. 1, the listener 34 in Fig. 3, the function diagram in Fig. 4 and the map in Fig. 5 described.

[0057] After the orientation 18 of the head 38 of the listener 34 has been determined in the first method step S100, as shown on the left side of the Fig. As shown in Figure 3, in the next step S200, the source direction 24 of the binaural recording is determined with reference to the head orientation 18 of the listener 34. The source direction 24 is a direction toward the perceived sound source 22. It is determined by analyzing the binaural recording.

[0058] Referring to Fig. 4, for the purpose of analyzing the binaural recording, the time domain signal x intended for the left ear of the listener 34 l(k) and the time domain signal x intended for the right ear of the listener 34 r (k) of the binaural recording is first converted into a frequency-domain signal using a discrete Fourier transform (DFT). The signals in the frequency domain are represented by X r (λ, µ) and X l (λ, µ).

[0059] The analysis involves determining an interaural level difference (ILD) and an interaural phase difference (IPD) of the binaural recording: ILD(λ,μ)=|Xr(λ,μ)XI(λ,μ)| ILD(λ,μ)=arg(Xr(λ,μ)XI(λ,μ))=ϕr(λ,μ)−ϕl(λ,μ), where X r,l (λ,µ) denotes the binaural ear signal in a frequency bin µ at a time λ.

[0060] ILD(λ,µ) and IPD (λ,µ) can be represented in a map 40 with ILD^ and IPD^ be compared.

[0061] According to Fig. 5, the characteristic map 40 creates a link between the head alignment 18, 30 and the difference in amplitude ILD^ and the phase difference IPD^. The specific values ​​for ILD^ and IPD^, stored in the map 40 can be generated by an analytical spherical head model and / or by pre-measured head-related transfer functions (HRTFs). The HRTFs describe the modification of the signal by the head 38 and / or the ears of the listener 34 depending on the direction of sound incidence: Xl(λ,μ)=Hl(λ,μ)⋅S(λ,μ) Xr(λ,μ)=Hr(λ,μ)⋅S(λ,μ), where H i (λ,µ) with i ∈ {l,r} represents the HRTFs for the left and right ear, which can be divided into amplitude and phase components as follows: Hi(λ,μ)=|Hi(λ,μ)|⋅eiϕi(λ,μ).

[0062] The map 40 contains the information about measured and / or analytically calculated HRTFs as a function of φ, the angle between the orientation 18, 30 of the head and the source direction 24: ILD^(μ,φ)=|H^r(μ,φ)||H^l(μ,φ)| IPD^(μ,φ)=arg(H^r(μ,φ)H^l(μ,φ))=ϕ^(μ,φ)−ϕ^l(μ,φ)

[0063] In map 40, not only the resulting values ILD^ and IPD^ are stored, but also the HRTFs.

[0064] Fig. 5 shows two examples of maps 40. On the left are the values ILD^ (above) and IPD^ (below) are shown, which were determined from measured HRTFs. On the right are the values ILD^ (above) and IPD^ (below) determined by an analytical model.

[0065] Determining the source direction 24, S200, involves a minimization approach in which ILD^ and IPD^ in map 40 that best match the ILD and IPD values ​​of the binaural recording. Specifically, the minimization has the following form: φorig(λ,μ)=arg minφILD^(μ,φ)ILD(λ,μ)+ILD(λ,μ)ILD^(μ,φ)−2cos(IPD(λ,μ)−IPD(μ,φ)) where φ orig corresponds to the angle 26 between the orientation 18 of the head 38 of the listener 34 and the source direction 24.

[0066] In a further method step, S300, a change in the head orientation 28 to a new orientation 30 of the head 38 of the listener 34 is detected. Therefore, the angle 32 between the new head orientation 30 and the source direction 24 is known. This new orientation 30 of the head 38 of the listener 34 is on the right side of Fig. 3 shown.

[0067] In the final process step, S400, the binaural recording is adjusted taking into account the source direction 24 of the binaural recording and the new head orientation 30. The characteristic map 40 is also called up. The binaural recording is modified as follows: Yi(λ,μ)=Xi(λ,μ)⋅Gi(λ,μ), where Gi(λ,µ) are complex coefficients that represent the ILD and IPD values ​​of the binaural recording X i (λ,µ) according to Gi(λ,μ)=GiILD(λ,μ)⋅GiIPD(λ,μ). modify

[0068] The phase modifications are GrIPD(λ,μ)=e−iΔIPD(λ,μ)2 GlIPD(λ,μ)=e−iΔIPD(λ,μ)2, where the value ΔIPD(λ,µ) is determined by calling the map 40: ΔIPDG(λ,μ)=IPD^(μ,φdest(λ,μ))−IPD^(μ,φorig(λ,μ)), where φ dest denotes the angle 32 between the new head orientation 30 and the source direction 24, which was determined in the previous method step (S300).

[0069] The size of the complex coefficient modifies the ILD values ​​of the binaural recording and results directly from the HRTFs in the map according to GiILD(λ,μ)=|H^i(μ,φdest(λ,μ))||H^i(μ,φorig(λ,μ))|

[0070] The modifications become less invasive for smaller values ​​of Δφ, and the signal is not modified at all for Δφ = 0.

[0071] After adjusting the binaural signal, the modified signal Y i (λ, µ) is transformed back into the time domain by applying an inverse discrete Fourier transform (IDFT). List of reference symbols 10 sound engineers 12 hearing system 14 Mounting device 16 Head tracking device 18 Head alignment 20 Sound engineer's head 22 Sound source 24 Source direction 26 Angle between head orientation and source direction 28 Change of head orientation 30 New head alignment 32 Angle between new head orientation and source direction 34 listeners 36 hearing aid 38 Listener's head 40 map

Claims

[1] A method for providing a binaural recording for a listener (34) having a head (38), the method being carried out with a hearing system (12), the binaural recording being heard using a hearing device (36), and the binaural recording consisting of a left binaural ear signal intended for a left ear of the listener (34) and a right binaural ear signal intended for a right ear of the listener (34), comprising the method steps: Providing the binaural recording consisting of the left binaural ear signal and the right binaural ear signal to the hearing system (12), Determining a head orientation (18) of the listener (34); Determining a source direction (24) of the provided binaural recording with respect to the head orientation (18); detecting a change in the head orientation (28) to a new head orientation (30); and Adjusting the provided binaural recording taking into account the determined source direction (24) of the binaural recording and the detected new head orientation (30). [2] A method for recording a binaural recording, the method being carried out with a hearing system (12), the binaural recording being recorded using a recording device (14) worn on a head (20) of a sound engineer (10), and the binaural recording consisting of a left binaural ear signal received by the recording device (14) near and / or at a left ear of the sound engineer (10), and a right binaural ear signal received by the recording device (14) near and / or at a right ear of the sound engineer (10), comprising the method steps: Recording the binaural recording consisting of the left binaural ear signal and the right binaural ear signal by means of the recording device (14), Determining a head orientation (18) of the sound engineer (10); Determining a source direction (24) of the recorded binaural recording with respect to the head orientation (18); detecting a change in the head orientation (28) to a new head orientation (30); and Adjusting the recorded binaural recording taking into account the determined source direction (24) of the binaural recording and the detected new head orientation (30). [3] The method of claim 1 or 2, wherein the step of detecting a change in the head orientation (28) to a new head orientation (30) comprises detecting a rotational movement of the head (20, 38). [4] Method according to one of the preceding claims, wherein the step of determining a source direction (24) of the binaural recording with respect to the head orientation (18) and the step of adapting the binaural recording taking into account the source direction (24) of the binaural recording and the new head orientation (30) comprises calling up a map (40), the map (40) relating the head orientation (18, 30) to a difference between the left binaural ear signal and the right binaural ear signal for a given source direction (24). [5] A method according to any one of the preceding claims, wherein the step of determining a source direction (24) of the binaural recording with respect to the head orientation (18) comprises determining a difference between the left binaural ear signal and the right binaural ear signal and comparing this difference with a map (40), the map (40) relating the head orientation (18, 30) to the difference between the left binaural ear signal and the right binaural ear signal for a given source direction (24). [6] Method according to one of the preceding claims, characterized byin that the step of adapting the binaural recording taking into account the source direction (24) of the binaural recording and the new head orientation (30) comprises determining the new head orientation (30) with respect to the source direction (24) of the binaural recording and comparing this value of the head orientation with a characteristic map (40), wherein the characteristic map (40) relates the head orientation (18, 30) to the difference between the left binaural ear signal and the right binaural ear signal for a predetermined source direction (24). [7] Method according to one of claims 4 to 6, characterized byin that the characteristic field (40) relating the head orientation (18, 30) to the difference between the left binaural ear signal and the right binaural ear signal for a predetermined source direction (24) relates the head orientation (18, 30) to the difference in amplitude and / or in phase between the left binaural ear signal and the right binaural ear signal for a frequency range of the right and left binaural ear signals and for a predetermined source direction (24). [8] Method according to one of claims 4 to 7, characterized bythat the characteristic field (40) which relates the head orientation (18, 30) to the difference between the left binaural ear signal and the right binaural ear signal for a given source direction (24) is adaptable to the head (38, 20) of the listener (34) or the sound engineer (10) and / or to the ears of the listener (34) or the sound engineer (10) and / or to an environment of the listener (34) or the sound engineer (10). [9] A method of providing a binaural recording to a listener (34), wherein the binaural recording is recorded by a method according to any one of claims 2 to 8 and the binaural recording is provided to the listener (34) by a method according to any one of claims 1 and 3 to 8. [10] Hearing system (12), wherein the hearing system (12) is designed to carry out the method according to one of claims 1 or 3 to 8, with a hearing device (36) and a head movement tracking device (16), wherein the head movement tracking device (16) is designed to determine a head orientation (18, 30) and / or a change in the head orientation (28) of the listener (34), and / or wherein the hearing system (12) is designed to carry out the method according to one of claims 2 to 8 with a recording device (14) and a head movement tracking device (16), wherein the head movement tracking device (16) is designed to determine a head orientation (18, 30) and / or a change in the head orientation (28) of the sound engineer (10). [11] Hearing system (12) according to claim 10, wherein the hearing device (36) and / or the recording device (14) is designed to be worn by a listener (34) and / or a sound engineer (10). [12] Hearing system (12) according to claim 10 or 11, wherein the hearing system (12) comprises a memory device for storing a characteristic map (40), the characteristic map (40) relating the head orientation (18, 30) to a difference between the left binaural ear signal and the right binaural ear signal for a given source direction (24).

Images