Audio transducer implementation enhancements

Abstract

A method for generating an audio signal for a device equipped with a transparent mode, the method comprising: acquiring at least two external audio signals from at least two microphones located on the device; determining, based on the at least two external audio signals, at least one of: a sound direction or diffusivity or distance between a first microphone and a second microphone of the at least two microphones; modifying at least one of the at least two external audio signals based on at least one of the determined sound direction or diffusivity or distance; and rendering the modified at least one audio signal.

Description

Technical Field

[0001] The present application relates to apparatus and methods for audio transducer enhancement, but is not limited to audio transducer enhancement for head mounted units and headphones related to spatial aspects. Background Art

[0002] ANC (active noise cancellation) and pass-through / transparency features are increasingly being implemented in a variety of devices. For example, ANC and pass-through applications can be implemented in common devices such as headphones. In addition, ANC and pass-through implementations can be used in vehicles (e.g., cars) or apparel (e.g., motorcycle helmets, personal protective equipment (or PPE)). ANC actively (using electronic devices, microphones, and speaker elements) attenuates sounds from external sound sources for the user. The pass-through / transparency mode, in turn, actively plays external sound sources to the user so that the user can hear the surroundings (e.g., headphone users can hear the cars around them, hear and talk to other people in the same space). Ideally, in transparency mode, the user can clearly hear the surroundings (e.g., for the headphone user, it is as if they are not wearing a headphone).

[0003] Many devices allow users to choose the amount of outside sound they hear, so the device can gradually switch between ANC and Transparency modes.

[0004] When using an audio signal from an external or outer microphone, the transparency mode in headphones does not produce a pure pass through signal because the generated pass through signal is different from the audio signal experienced by the user because the external microphone is not in the user's ear canal. In addition, the difference between the generated pass through audio signal and the "experienced audio signal" may be due to the inner speaker or transducer not being in the same location as the external microphone, and the physical design of the device causing filtering effects (its shape, size, and weight affect the audio signal).

[0005] For example, if the device is a set of headphones, the external microphone is not in the ear canal of the headphone user. Additionally, in the automotive example device, the microphone is mounted on the outside of the vehicle and the speaker is either in a normal location (in the door, dashboard, etc.) or located on the driver's seat. Summary of the invention

[0006] According to a first aspect, a method for generating an audio signal for a device equipped with a transparency mode is provided, the method comprising: obtaining at least two external audio signals from at least two microphones located on the device; determining at least one of the following based on the at least two external audio signals: a sound direction or diffuseness or a distance between a first microphone and a second microphone of the at least two microphones; modifying at least one of the at least two external audio signals based on at least one of the determined sound direction or diffuseness or distance; and rendering at least one modified audio signal.

[0007] The at least two microphones located on the device may be located on a side of the device.

[0008] The at least two microphones located on the device may be located on opposite sides of the device.

[0009] The method may further include acquiring at least one internal audio signal, and modifying at least one of the at least two external audio signals is further based on the at least one internal audio signal.

[0010] The method may further include modifying the at least one of the at least two external audio signals based on a frequency profile to modify the at least two external audio signals more in lower frequencies.

[0011] The method may also include determining a distance between at least one of the at least two microphones located on the device and an associated speaker, wherein modifying at least one of the at least two external audio signals may be further based on the distance between at least one of the at least two microphones located on the device and the associated speaker.

[0012] The device may include one of: a smart phone; a headset; a vehicle equipped with the at least two microphones; a helmet equipped with the at least two microphones; and personal protective equipment equipped with the at least two microphones.

[0013] The first microphone and the second microphone of the at least two microphones may be a left device microphone and a right device microphone.

[0014] The first microphone may be a first group of microphones, and the second microphone may be a second group of microphones.

[0015] Based on the determined at least one of: sound direction or diffuseness or distance, modifying at least one of the at least two external audio signals may cause a user of the device to more correctly perceive the modified audio signal direction or diffuseness.

[0016] According to a second aspect, a device for generating an audio signal for a device equipped with a transparency mode is provided, the device comprising at least one processor and at least one memory, the memory storing instructions which, when executed by the at least one processor, cause the system to at least perform: obtaining at least two external audio signals from at least two microphones located on the device; determining at least one of the following based on the at least two external audio signals: a sound direction or diffusion or a distance between a first microphone and a second microphone of the at least two microphones; modifying at least one of the at least two external audio signals based on the determined at least one of the following: a sound direction or diffusion or a distance; and rendering at least one modified audio signal.

[0017] The at least two microphones located on the device may be located on a side of the device.

[0018] The at least two microphones located on the device may be located on opposite sides of the device.

[0019] The device may be caused to perform acquisition of at least one internal audio signal, and the device caused to perform modification of at least one of the at least two external audio signals may be further caused to perform modification of at least one of the at least two external audio signals based on the at least one internal audio signal.

[0020] The apparatus may be further caused to perform modifying the at least one of the at least two external audio signals based on the frequency profile to modify the at least two external audio signals more in lower frequencies.

[0021] The device may be further enabled to determine a distance between at least one of the at least two microphones located on the device and the associated speaker, wherein the device enabled to modify at least one of the at least two external audio signals may be further enabled to modify at least one of the at least two external audio signals based on the distance between at least one of the at least two microphones located on the device and the associated speaker.

[0022] The device may include one of: a smart phone; a headset; a vehicle equipped with the at least two microphones; a helmet equipped with the at least two microphones; and personal protective equipment equipped with the at least two microphones.

[0023] The device may be a part of the apparatus.

[0024] The apparatus may include the device.

[0025] The first microphone and the second microphone of the at least two microphones may be a left device microphone and a right device microphone.

[0026] The first microphone may be a first group of microphones, and the second microphone may be a second group of microphones.

[0027] The device may be configured to modify at least one of the at least two external audio signals based on the determined at least one of the sound direction or diffuseness or distance so that a user of the device more correctly perceives the modified audio signal direction or diffuseness.

[0028] According to a third aspect, a device for generating an audio signal for a device equipped with a transparency mode is provided, the device comprising a means configured to perform the following operations: obtain at least two external audio signals from at least two microphones located on the device; determine at least one of the following based on the at least two external audio signals: the sound direction or diffusion or distance between a first microphone and a second microphone of the at least two microphones; modify at least one of the at least two external audio signals based on the determined at least one of the following: the sound direction or diffusion or distance; and render at least one modified audio signal.

[0029] The at least two microphones located on the device may be located on a side of the device.

[0030] The at least two microphones located on the device may be located on opposite sides of the device.

[0031] The module may be configured to acquire at least one internal audio signal, and the module configured to modify at least one of the at least two external audio signals may be further configured to modify at least one of the at least two external audio signals based on the at least one internal audio signal.

[0032] The module may be further configured to modify the at least one of the at least two external audio signals based on a frequency profile to modify the at least two external audio signals more in lower frequencies.

[0033] The module can be further configured to determine a distance between at least one of the at least two microphones located on the device and the associated speaker, wherein the module configured to modify at least one of the at least two external audio signals can be further configured to modify at least one of the at least two external audio signals based on the distance between at least one of the at least two microphones located on the device and the associated speaker.

[0034] The device may include one of: a smart phone; a headset; a vehicle equipped with the at least two microphones; a helmet equipped with the at least two microphones; and personal protective equipment equipped with the at least two microphones.

[0035] The device may be a part of the apparatus.

[0036] The apparatus may include the device.

[0037] The first microphone and the second microphone of the at least two microphones may be a left device microphone and a right device microphone.

[0038] The first microphone may be a first group of microphones, and the second microphone may be a second group of microphones.

[0039] The module configured to modify at least one of the at least two external audio signals based on the determined at least one of: sound direction or diffuseness or distance may be such that a user of the device more correctly perceives the modified audio signal direction or diffuseness.

[0040] According to a fourth aspect, a device for generating an audio signal for a device equipped with a transparency mode is provided, the device comprising: an acquisition circuit configured to acquire at least two external audio signals from at least two microphones located on the device; a determination circuit configured to determine at least one of the following based on the at least two external audio signals: a sound direction or diffusion or distance between a first microphone and a second microphone of the at least two microphones; a modification circuit configured to modify at least one of the at least two external audio signals based on at least one of the determined: sound direction or diffusion or distance; and a rendering circuit configured to render at least one of the modified audio signals.

[0041] According to a fifth aspect, there is provided a computer program [or a computer-readable medium] comprising instructions, wherein the instructions are used to cause an apparatus to generate an audio signal for a device equipped with a transparency mode, causing the apparatus to perform at least the following operations: obtaining at least two external audio signals from at least two microphones located on the device; determining at least one of the following based on the at least two external audio signals: a sound direction or diffuseness or a distance between a first microphone and a second microphone of the at least two microphones; modifying at least one of the at least two external audio signals based on the determined at least one of the following: a sound direction or diffuseness or a distance; and rendering at least one modified audio signal.

[0042] According to a sixth aspect, a non-transitory computer-readable medium comprising program instructions is provided, wherein the instructions are used to cause an apparatus for generating an audio signal for a device equipped with a transparency mode to perform at least the following operations: obtaining at least two external audio signals from at least two microphones located on the device; determining at least one of the following based on the at least two external audio signals: a sound direction or diffuseness or a distance between a first microphone and a second microphone of the at least two microphones; modifying at least one of the at least two external audio signals based on at least one of the determined: sound direction or diffuseness or distance; and rendering at least one modified audio signal.

[0043] According to the seventh aspect, a device for generating an audio signal for a device equipped with a transparency mode is provided, comprising: a module for acquiring at least two external audio signals from at least two microphones located on the device; a module for determining at least one of the following based on the at least two external audio signals: the sound direction or diffusion or distance between a first microphone and a second microphone of the at least two microphones; a module for modifying at least one of the at least two external audio signals based on at least one of the determined: sound direction or diffusion or distance; and a module for rendering at least one modified audio signal.

[0044] According to an eighth aspect, a computer-readable medium comprising instructions is provided, wherein the instructions are used to enable an apparatus for generating an audio signal for a device equipped with a transparency mode to perform at least the following operations: obtaining at least two external audio signals from at least two microphones located on the device; determining at least one of the following based on the at least two external audio signals: a sound direction or diffusion or a distance between a first microphone and a second microphone of the at least two microphones; modifying at least one of the at least two external audio signals based on at least one of the determined: sound direction or diffusion or distance; and rendering at least one modified audio signal.

[0045] An apparatus comprises modules for performing the actions of the method as described above.

[0046] An apparatus is configured to perform the actions of the method as described above.

[0047] A computer program comprises instructions for causing a computer to execute the method as described above.

[0048] A computer program product stored on a medium may cause an apparatus to perform the method described herein.

[0049] An electronic device may include the apparatus described herein.

[0050] A chipset may include the apparatus described herein.

[0051] The embodiments of the present application are intended to solve the problems associated with the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] For a better understanding of the present application, reference will now be made by way of example to the accompanying drawings, in which:

[0053] Figure 1 to Figure 3 An example device is shown with respect to positioning of an external microphone and an internal transducer (or an ideally placed microphone) Figure 1a Headphones and Figure 1b Example differences within the capabilities of a vehicle in which the direction-of-arrival of a sound source is from behind or at an angle to the front relative to the device;

[0054] Figure 4 and Figure 5 An example diagram is shown, which shows the use of Figure 1 to Figure 3 The example shown in is the audio amplitude values ​​captured and played back by the external microphone and internal transducer relative to a range of sound source arrival directions;

[0055] Figure 6 schematically illustrates an example device including internal and external microphone and speaker transducers according to some embodiments;

[0056] Figure 7 and Figure 8 According to some embodiments Figure 6 Flow chart of the operation of the example apparatus shown in ;

[0057] Fig. 9 Schematically illustrates a balanced left and right on-axis position or size difference according to some embodiments;

[0058] Figures 10a to 10cThe position or size difference on-axis, parallel-to-axis and off-axis is shown;

[0059] Fig.11 It is shown that Fig. 9 Example phase differences caused by position or size differences on the left and right axes of the balance shown;

[0060] Fig.12 The difference in position or size on the left and right axes of imbalance is schematically shown;

[0061] Fig.13 It is shown that Fig.12 Example phase differences caused by position or size differences on the left and right axes of the imbalance shown;

[0062] Fig.14 A flow chart illustrating the operation of an example apparatus according to some embodiments;

[0063] Fig.15 Schematically illustrates the balance left and right parallel-axis position or size difference according to some embodiments;

[0064] Fig.16 The difference in position or size of the unbalanced left and right parallel shafts is schematically shown;

[0065] Fig.17 and 18 Example graphs showing the effect of a more stable and improved spatially transparent audio signal (overcoming acoustic leakage) with band-dependent directional / diffusivity correction; and

[0066] Fig.19 Example apparatus suitable for implementing some embodiments is schematically shown. DETAILED DESCRIPTION

[0067] As previously mentioned, current transparency or pass-through modes for suitable devices equipped with ANC functionality can produce a transparent audio signal that differs from the ideal or intended audio signal. This difference is due to the relative differences between the external or outboard microphone and the internal transducer or speaker, and further filtering aspects of the headset itself. In addition, in cases where the transparency method only considers the audio signal from one of the two microphones, there is also a difference in quality between the ideal and generated transparent audio signal.

[0068] Furthermore, when listening to music at loud volumes, the natural representation of ambient audio signals in transparency mode may not be sufficient to enable users to accurately detect all sound sources around them.

[0069] In order to render spatial audio, several features need to be implemented correctly. In particular, the features that should be implemented correctly in order to produce a high-quality audio signal are directionality and diffuseness. Directionality is important for safety reasons, such as being able to hear obstacles or hazards (such as the direction of a car), and diffuseness also provides the user with valuable clues about the distance of sound objects and the intelligibility of speech.

[0070] The upcoming Immersive Voice and Audio Services (IVAS) standard and immersive voice applications are configured to provide immersive audio communications. This type of communication system is more immersive, which means that the mixing of far-end ambience sounds and the local ambience environment can result in a confusing output audio signal. Therefore, improving the clarity of the local spatial audio environment is a research topic currently being investigated.

[0071] In the following examples, the device shown is a headphone device, but it will be appreciated that the same methods and apparatus for implementing the embodiments may be applied to other devices.

[0072] You can refer to Figure 1a and 1b A first example of the difference between real and ideal microphone positions and their effect on transparency mode is shown in FIG. Figure 1a and 1b , an example audio source 109 generates an audio signal from behind a user 101. The user 101 is shown on the left, relative to Figure 1a The headset 105, and Figure 1b 155, and external or external microphones (shown as left external microphones 106, 156 and right external microphones 107, 157) in a straight line with the audio source. However, the user 101 is located on the right side relative to the ideal microphone positions (shown as left ideal microphones 116, 166 and right ideal microphones 117, 167), which are located inside the ear canal and are actually in the audio shadow relative to the audio source 109 and are blocked by the ear 107 or the vehicle body 158. Therefore, the sound coming from behind the user's ear is blocked by the earlobes. However, the sound reaching Figure 1a and 1b The sound from the earcup microphones in the positions shown is not obscured. If the earcup microphones are used to generate pass-through sound, the user cannot distinguish the sound from the rear from the sound from the front by the shadowing effect. This makes it more difficult than necessary to perceive the direction from the rear when wearing a headset and in a car.

[0073] Figure 2 Another example of the difference between real and ideal microphone positions and their effect on transparency mode is shown. Figure 2 1, the example audio source 209 generates an audio signal from the front of the user 101 at a certain angle. The real microphone 106 / 107 and the ideal microphone 116 / 117 positions are similar to the positions described in the example shown in Figure 1. In this example, the sound from the front right arrives without masking and therefore reaches both microphones on the earmuffs at the same level. The same sound is masked relative to the ideal left microphone 116 in the left ear. Therefore, the level difference of the captured sound in the microphones is very small, but for the user (not wearing a headset-ideal user microphone position), the level difference is very large. If the microphones on the earmuffs are used to generate direct sound, the user cannot use the masking effect to determine the direction of the sound source.

[0074] also, Figure 3 An example is also shown where the difference between positions affects not only the shading effect but also the time difference. Figure 3 , an example audio source 300 generates an audio signal at an angle from the front of the user 101. The positions of the real microphones 106 / 107 and the ideal microphones 116 / 117 are similar to those described with respect to the example shown in FIG. In this example, it is shown that when a (thick) headset is used, the microphones 106 / 107 are far away from the ears, so the time difference between the sound reaching the left and right microphones from one direction is significantly greater than the time difference between the sound reaching the microphones 116 / 117 in the ears when the left and right ears are reached. This difference shown in distances 301 and 303 makes the direction perception different when using headsets of different thicknesses in the transparent mode. In addition, when the two microphones are close to each other, the diffusion of the sound they capture is less than when they are far apart. Therefore, the position difference (caused by the thickness of the headset) also affects the diffusion. The thicker the headset, the more diffusion in the transparent signal should be reduced to make the audio sound like when there is no headset.

[0075] Figure 4 and Figure 5 The graphs in further demonstrate this effect. In these examples, white noise was played to a mockup artificial head in an anechoic chamber, with ideally positioned microphones and simulated headphones positioned about 3 cm outside the head. One microphone was on the right side (right microphone) and two microphones were on the left side (left microphone and front microphone).

[0076] Figure 4The signals in left microphone 410 and right microphone 420, for example, from an ideal microphone are shown as white noise is played to the left and right microphones from 36 directions in the horizontal plane around the device, with 10 degrees between directions. Sounds from the front are in box 401, and sounds from the back are in box 403. As can be seen, the masking effect changes significantly as the sound moves around the head.

[0077] Figure 5 The same situation is shown but for a left microphone 510 and a right microphone 520 on a headset. Sound from the front is within box 501 and sound from the back is within box 503. As can be seen, the audio signal is less noticeable as the sound moves around the head due to the lack of masking effects.

[0078] In other words, because the microphone is not located where the ear canal is, the head has little effect on the microphone signal, resulting in Figure 5 The signal shown. This graph only has the left and right microphone signals that are typically used for the transparent signal. This signal can be improved to be more like an "ideal" audio signal that a human would hear using binauralization using three microphones using known methods. The level difference between the sound coming from the front and back (the red and blue boxes in the graph) is Figure 5 Insignificant, but in Figure 4 is more correct.

[0079] As mentioned above, similar issues arise with other devices. For example, in the case of a vehicle, the microphone may be mounted on the exterior of the vehicle, with the speaker located in a conventional location (in the door, dashboard, etc.) or configured as a soundbar or located within the seat of the user (e.g., driver). Similarly, a motorcycle helmet or head-mounted PPE protective device may be a device where the microphone is mounted on the exterior but the microphone is mounted inside the helmet / PPE.

[0080] In some embodiments, a headset (or suitable apparatus and method) with a transparency mode is provided, wherein microphone signals are compared to estimate the sound direction and / or diffuseness and / or distance between left and right ear cups / ear cup speakers (or more generally microphone and speaker locations), and the transparency signal is modified so that the user perceives the direction and diffuseness more correctly. In some embodiments, the (headphone) transparency signal is modified more in the low frequencies.

[0081] In some embodiments, suitable devices (e.g., headphones, vehicles, or clothing) and methods with a transparency mode are also provided, wherein microphone signals from two microphone locations (e.g., ear cups in a headset) are compared to estimate sound direction and / or diffuseness, and the transparency signal is modified so that the user perceives the direction and diffuseness more correctly. In some embodiments, the (headphone) transparency signal is modified more in the low frequencies.

[0082] In some embodiments, suitable apparatus and methods are also provided with a transparency mode, wherein microphone signals are compared to estimate sound direction and / or diffuseness, and the transparency signal is modified so that the direction of the sound source is intentionally clear, especially when the internal signal (music) from a device connected to the device is loud. In some embodiments, the transparency signal is modified more in the low frequencies.

[0083] Furthermore, in some embodiments, suitable apparatus and methods are provided with a transparency mode, wherein microphone signals from two microphone locations (e.g., ear cups on a headset) are compared to estimate sound direction and / or diffuseness, and the transparency signal is modified so that the direction of the sound source is deliberately clear, especially when the internal signal (music) from a device connected to the device or apparatus is loud. In some embodiments, the two (ear cup) microphones are compared to achieve a perceptually better estimate. In some embodiments, the (headphone) transparency signal is modified more in the low frequencies.

[0084] In addition, in some embodiments, suitable apparatus and methods (e.g., headphones) are provided, which are configured to reduce the phase difference and dispersion of at least two microphone signals and create a transparent signal from the modified microphone signals using information about the distance from the external microphone to the internal speaker (e.g., the thickness of the headphone). The amount by which the spatial parameter is modified (decreased) is based on the distance amount.

[0085] In the following examples, the concepts discussed herein can be implemented in any suitable headset or head-mounted device (headset). In some embodiments, the headset can be in-ear or over-the-ear. The headset can have a headband or no headband (e.g., it can be an earplug or earphone, which is at least partially located in the ear canal or against or adjacent to the ear canal). In an embodiment where two cup-shaped microphones are used to create a transparent signal, the microphone signal is transmitted to two earmuffs. In a headset with a headband, this can be implemented using a cable, and in a headset without a headband, a suitable wireless transmission, such as Bluetooth, is adopted.

[0086] about Figure 6, shows an example device in which some embodiments can be implemented. In this example, a user 601 wears a headset 603 equipped with a headband 631, a left earmuff 635, and a right earmuff 633. The left earmuff 635 in this example includes at least one left external (external) microphone 615 for recording sounds from outside the headset, and a speaker 617 configured to play or output a clear signal and sounds from a properly connected (wireless or wired) device. The device can be, for example, a mobile phone, a laptop, a PC, or any suitable electronic device. In some embodiments, the left earmuff 635 can also include at least one built-in microphone 619, which is configured to record sounds from inside the headset, between the left speaker 617 and the user's eardrum.

[0087] Similarly, the right ear cup 633 in this example includes at least one right external (external) microphone 605 that records sound from outside the headset, and at least one right speaker 607 that is configured to play or output a pass-through signal and sound from an appropriately connected device. In some embodiments, the right ear cup 633 may also include at least one internal microphone 609 that is configured to record sound from inside the headset, between the right speaker 607 and the user's eardrum.

[0088] In some embodiments, a device (e.g., a headset) with a transparency mode is provided in which microphone signals are compared to estimate sound direction and / or diffuseness, and the generated transparency signal is modified so that the user more correctly perceives direction and diffuseness.

[0089] In some embodiments, the audio signal from the microphone in the left ear cup (or more generally the "left" side microphone) can be used to create the left pass-through signal, and the microphone in the right ear cup (or more generally the "right" side microphone) can be used to create the right pass-through signal. In this way, no signal transmission is required between the two "sides" of the microphones or ear cups. In these embodiments, the total minimum number of microphones is four, two on each side (or in each ear cup).

[0090] In such an embodiment, the device (headphone) uses at least two microphone signals to analyze the sound direction, for example using the method in US9456289, and / or the diffusion as in GB1619573.7. The diffusion can be estimated as the D / A ratio (direct-to-ambient). In some embodiments, these parameters can be analyzed in frequency bands. In some embodiments, there can be 20-50 frequency bands, but the embodiments discussed herein can be applied to implementations with one or more frequency bands. In some embodiments, a smaller number of frequency bands can help reduce the time required for analyzing parameters.

[0091] In some embodiments, the left / right direction is estimated and the front / back ambiguity is left unresolved. This allows the level and / or phase of the through signal to be corrected for left / right separation and ignore the front / back separation. The front / back direction separation can be used to include the effects of occlusion (e.g., occlusion by the earlobe may be the largest single contributor to the head, but its effect is not as strong as occlusion by the human head in the case of left-right separation). In addition, solving only the left-right direction problem can be achieved with fewer microphones. For example, a minimum of two microphones can be used to determine the left-right separation.

[0092] The right external microphone audio signal can be equalized in a frequency band and fed to the same (ear cup) speaker to create a transparent signal. A similar approach can be applied to the left side, where the left external microphone audio signal can be equalized in a frequency band and fed to the same left (ear cup) speaker or a "left" virtual speaker in a soundbar to create a transparent signal for the left channel.

[0093] Since different frequencies acoustically leak to the user's ears differently through the headphones or vehicle or helmet or head-mounted device, equalization of the external microphone audio signal can be achieved. Through equalization, the transparent signal compensates for the part of the audio signal that the headphones passively attenuated from the leakage signal. The passive attenuation of the headphones is higher at higher frequencies, so the transparent signal level is higher at higher frequencies. For the same reason, the modifications implemented in some embodiments are applied more (and the modification level is higher) to the lower frequencies, where the leaked sound constitutes a large part of the lower frequencies.

[0094] In some embodiments, equalization is controlled based on the detected direction. Equalization is applied so that the level difference in the frequency band between the left ear cup and the right ear cup transparent signal corresponds to the binaural signal from the detected direction. The level difference of the binaural signal can be obtained or determined from a stored database or similar storage form. The database can be a general database or a database personalized for the current user.

[0095] In a more complex implementation, the diffusion of the audio signal is taken into account. The level difference between the left and right microphone audio signals (or earmuffs in headphones) is modified as the product of the D / A ratio and the level difference for a dry sound from the detected direction. The level difference for dry sound is called ILD (Inter-aural Level Difference), and the values ​​for different directions can be found from a known database. Alternatively, in some embodiments, the level difference measured or estimated by the user himself can be used. The determination of the level difference measured or estimated by the user himself can be implemented according to any known method.

[0096] In some embodiments, the audio signal may also be modified with device specific values ​​based on the device (e.g., headphones or headset / earbuds, vehicle, etc.). In some embodiments, if the sound environment is estimated to be completely diffuse (typically a D / A ratio of zero or close to zero), the product, i.e., the final level difference should be zero, and if the sound environment is completely directional (a D / A ratio of one or close to one), the product, in other words, the final level difference, should be the same as the dry level difference.

[0097] The method is further shown in Figure 7 In the operation shown.

[0098] With respect to step 701, the operation of receiving a microphone signal from one side (L or R) is shown, which in a headset may be one ear cup.

[0099] Then, with respect to step 703, the operation of dividing the signal into time-frequency tiles is shown.

[0100] As shown in step 705, it is an operation of estimating the sound direction in at least one block.

[0101] Furthermore, an operation of estimating a D / A ratio in at least one block is shown in step 707. This estimation operation is an optional step.

[0102] Furthermore, step 709 shows an operation of searching / calculating (or otherwise acquiring or determining) a level and / or phase difference from a database (or a suitable storage module) for at least one block direction.

[0103] Then, as shown in step 711, there is the operation of modifying the pass-through signal based on the acquired or found level (and phase) differences and optionally using the D / A ratio.

[0104] Step 713 also illustrates converting the modified signal back to a time domain representation.

[0105] Finally, step 715 shows the operation of using the modified time domain signal as a same-side (which may be the same-side ear cup in a headphone) transparent signal after implementing other known modifications such as equalization.

[0106] In some embodiments, the diffuseness of the transparent signal can also be modified based on the measured diffuseness by decorrelating or correlating the transparent signal so that its diffuseness matches the diffuseness measured from the external microphone. Decorrelation of the signals can be achieved using known decorrelators, and correlation of the signals can be achieved, for example, by mixing a stereo transparent signal with its mono downmix.

[0107] In addition, in some embodiments, the transparency (i.e., pass-through) signal should be presented to the user with very little latency. Direction and D / A ratio analysis may take some time (e.g., 20 milliseconds or longer), depending on the method used. Typically, important directions and D / A ratios (e.g., emergency vehicle sounds in the audio) do not change quickly, while less important directions (e.g., ambient noise directions) may change rapidly. Therefore, the system can use the directions and D / A ratios from earlier audio samples and use them to adjust the current samples in the transparent signal without causing major problems. However, some direction detection methods are very fast. For example, the level difference of the microphone signal can be directly mapped to the required level difference in the transparent signal using measurement data from a headset, where the measurement data is stored in a table. Additionally or alternatively, a higher sampling rate (e.g., 192kHz) can be used for analysis to reduce latency.

[0108] In addition to or in lieu of level changes, the phase of the pass signal may be changed to match a value in the database. Phase changes may be achieved using known methods. Typically, phase changes are more important at lower frequencies (<1.5kHz), while level changes are more important at higher frequencies (>1.5kHz).

[0109] Additionally, in some embodiments, changes are applied to the through signal such that the changes minimally modify the original through signal. In these embodiments, the focus is on the difference in level and phase of the left and right through signals. Thus, minimal modifications are made to both signals such that the difference is the same as in the database.

[0110] More detailed example implementations are presented below.

[0111] In this example, the two microphones are located on or near the left / right axis on the device. Therefore, this example can solve problems such as Figure 2 The problem shown in Figure 1, where the audio source comes from an angle, does not help with the front / back determination problem shown in Figure 1. The difference in sound coming from the front or back occurs very little compared to the difference in sound coming from the right or left. The device in this example has two microphones m=1,2, which produce the following signals:

[0112] x m (k),m=1,2

[0113] In this example, microphone 1 is used to create a left-ear transparent audio signal, and microphone 2 is used to create a right-ear transparent audio signal.

[0114] In some embodiments, the first operation is to filter the microphone audio signal using a filterbank so that it can be processed in a frequency band, for example using:

[0115] https: / / researchgate.net / publication / 225876013_Low_Delay_Filter-Banks_for_Speech_and_Audio_Processing

[0116] Other filter banks or time-frequency domain transforms may be used (although as noted above, very low latency methods such as those described above are preferred.) Low latency methods are employed because a user of a headset (or more generally a device) can typically hear both the source and the transparent signal via the speakers or headset, and any significant delay between the two may result in a situation where the user hears the source twice.

[0117] The filtered signal with B-band is:

[0118]

[0119] In the low frequency band, the direction is analyzed by the time difference between the signals.

[0120]

[0121] where τ b is the time difference in each frequency band, N i is an index showing the limits of the ith analysis window. The analysis window should be very small to reduce any latency. Typically, the analysis window can be 10 ms or shorter.

[0122] The time difference is converted into effective distance

[0123]

[0124] where F s is the sampling frequency and v is the speed of sound.

[0125] The effective distance can be converted into the direction of each low-frequency band.

[0126]

[0127] Where d is the distance between the microphones and b is the estimated distance between the sound source and the nearest microphone. Typically, b can be set to a fixed value. For example, it has been found that b = 2 meters provides stable results. The direction determined is the azimuth direction.

[0128] However, the direction is ambiguous because there are two microphones. In certain embodiments with three or more microphones, the ambiguity problem can be solved as described in US9456289.

[0129] At higher frequencies, the time difference between the arrival of the sound at the two microphones can be large compared to the wavelength of the sound, so it is better to use level-based directional analysis. The energy of the microphone signal in a frequency band can be calculated as follows:

[0130]

[0131] Energy difference between microphone signals:

[0132]

[0133] This can be compared to a database to find the closest value. The database contains energy differences for known sound directions. The comparison then provides a directional estimate for each higher frequency band.

[0134] Once the direction of each frequency band is determined, the audio signal in the frequency band is modified to better reflect human hearing without headphones. The direction is used to search the HRTF (Head Related Transfer Function) database for the left and right ear levels and phase differences corresponding to that direction in each frequency band.

[0135] HRTFs are usually provided in the form of frequency response bins, with frequency indices for both sides (left and right) in different directions. The average energy of the HRTF for the left ear l and direction α in frequency band b (the frequency band used here is the same as the previous filter bank) is:

[0136]

[0137] Among them, M b is the index of the limit of the frequency response bin showing frequency band b, HRTF l,α The function is readily available in the known HRTF database. The level difference between the left and right HRTF is:

[0138]

[0139] In these embodiments, the through signal is configured to have the same level difference in each frequency band. For example, if the direction in the estimated frequency band is α, then it is as follows:

[0140]

[0141] Where g is the gain that amplifies one of the signals and attenuates the other. In these embodiments, the method is configured to modify the signals as little as possible overall, since generally all modifications result in artifacts. The result of the above equation is:

[0142]

[0143] Audio signal

[0144]

[0145] is modified in the frequency band so that the modified audio signal has the same level and phase difference as indicated by the HRTF database. In some cases, only one of the two is used, and in most typical cases, the phase difference is used for the low frequency band and the level difference is used for the high frequency band.

[0146] For example, if microphone 1 is used to create a direct signal for the left side or channel (or ear), and microphone 2 is used to create a direct signal for the right side or channel (or ear), and the level difference between the left and right sides is as described before, and the phase difference in the HRTF database is:

[0147] phase_diff(b,α)

[0148]

[0149] In some embodiments, diffuseness is also taken into account. Diffusion can be measured using known methods, typically the correlation between microphone signals. Diffusion is typically measured as a D / A ratio. If the audio scene around the user is very diffuse, it is difficult to hear any clear audio direction, and the above modifications can be reduced or completely ignored. For example, where diffuseness has been estimated using known methods and can be used as a D / A ratio for each frequency band, if the D / A ratio is zero, the audio is very diffuse, and if the D / A is 1, the audio is very directional (the opposite of diffuse) and the D / A ratio can take any value between 0 and 1 to correspond to various diffuseness levels.

[0150] In some embodiments, the previous equation can be modified as follows:

[0151]

[0152] In some embodiments where the microphones are very close to each other, the diffuseness estimation method may underestimate the amount of diffuseness that the user hears without the headset.The diffuseness can then be added to the signal by decorrelating the signal using known methods.

[0153] In some embodiments, at lower frequencies (<1.5kHz) and increasingly towards frequencies below this, greater modification is applied because at lower frequencies the transparent signal only makes up a small portion of the audio heard by the user because at low frequencies ambient sound leaks through the headset. Leakage depends on the headset, typically over-ear headsets leak less than in-ear headphones, but this also depends on how well the headset fits. Typically, the modification factor can be 1.5-2.0 for low frequency gain and phase.

[0154] In some embodiments, the modified signal may then be input to an inverse filter bank, which may be as simple as summing all the signals, or more complex where a time-frequency domain transform is applied. The result is used to create a through signal.

[0155] In some embodiments, further processing may be applied before or after these operations, such as analog to digital (A / D) conversion, digital to analog (D / A) conversion, compression, equalization (EQ), etc.

[0156] In some cases with many sample rates (especially <48kHz), using a full sample delay for the phase may not be accurate enough. Therefore, a fractional delay can be applied.

[0157] Although in the above examples, the smoothing operation is not described for reasons of simplicity and clarity, typical implementations may implement smoothing.

[0158] In some embodiments, the headset may have a transparency mode in which microphone signals from both ear cups are compared to estimate sound direction and / or diffuseness, and the transparency signal is modified so that the user more correctly perceives direction and diffuseness.

[0159] In terms of implementation, this is similar to the example implementations described above. The difference between these embodiments and the above embodiments is that when microphones from two ear cups are used, the paired distance of the microphones is more similar to the distance between the user's ears, so the estimate of direction and / or diffuseness is more similar to the direction and / or diffuseness that a human (not wearing a headset) would perceive. For left and right ear pass-through audio signals, two microphones from the left and right ear cups are used. In these embodiments, the minimum total number of microphones is two, one in each ear cup.

[0160] In these embodiments, optimal performance is experienced when both ear cups of the headset are connected via a wired connection so that latency is negligible when processing microphone signals from both ear cups.

[0161] The method is further shown in Figure 8 In the operation shown.

[0162] With respect to step 801, the operation of receiving microphone signals from two sides (L and R) is shown, which for a headset may be two ear cups (L and R).

[0163] Then, with respect to step 803, the operation of dividing the signal into time-frequency blocks is shown.

[0164] As shown in step 805, it is an operation of estimating the sound direction in at least one block.

[0165] Furthermore, an operation of estimating a D / A ratio in at least one block is shown in step 807. This estimation operation is an optional step.

[0166] Furthermore, step 809 shows an operation of searching / calculating (or otherwise acquiring or determining) a level and / or phase difference from a database (or a suitable storage module) for at least one block direction.

[0167] Then, as shown in step 811, it is an operation to modify the through signals (L and R) based on the acquired or found level (and phase) differences and optionally using the D / A ratio.

[0168] Step 813 also shows converting the modified signal back to a time domain representation.

[0169] Finally, step 815 shows the operation of using the modified time domain signal as a transparent signal for both sides or ear cups after implementing other known modifications (such as equalization).

[0170] The processing equations are the same as discussed above, but the microphone positions are different.

[0171] In some embodiments, the device (headphones) is configured with a transparency mode in which the microphone signal is compared to the estimated sound direction and / or diffuseness and the transparency audio signal is modified so that the direction of the sound source is intentionally clear, particularly when the internal signal (music) of the device connected to the device is loud.

[0172] Because hearing the surrounding ambient sounds can be crucial, for example in traffic. Even without a headset, the ability to determine or hear audio signals coming from the correct direction (for example in a large city where buildings are very echoing) can be difficult. Any reduction in the ability to determine direction due to the headset can be problematic. Users can be distracted by loud music playing, for example, from a headset or in a vehicle or helmet. The louder the music, the more difficult it is to hear the dangerous sound sources around you in real life.

[0173] While many methods have been proposed for giving users "super" hearing of sound directions, few are suitable for use in transparency mode due to the ultra-low latency requirements. In some embodiments, one proposed low-latency method is presented below:

[0174] In some embodiments, a microphone signal (typically one of the signals from both sides or earmuffs) is divided into frequency bands using a low-delay filter bank. A larger microphone is selected for each frequency band. The larger signal is used as a binaural pass-through signal (+ other known modifications adopted), but a level difference is introduced to the left and right ear pass-through signals in each frequency band. The level difference is the same as the level difference in the original microphone signal, or the level difference is based on a detected direction similar to the previous embodiment described above. This modification may in some cases result in artifacts in the pass-through audio signal, so the modified signal is typically mixed with a normal or regular (unmodified) pass-through signal. The mix depends on the loudness of the music from the user device. The louder the music, the more modified the signal is used in the mix. The modification achieves a reduction in diffuseness, thereby improving the user's directional hearing.

[0175] In some embodiments, this can be achieved based on the above equation.

[0176]

[0177] In these embodiments, the equation replaces the D / A ratio with an ML value, where ML represents the music level and may be set to a value of 1 for a high music level and a value of 0 for a low music level.

[0178] In some embodiments, the modifications performed here may need to be applied in a greater manner to lower frequencies, where the leaked sound constitutes a large portion of the lower frequencies.

[0179] In very dispersed conditions, the sound may seem to be coming from the wrong direction due to strong reflections. In this case, applying the modification may result in poor performance. Therefore, in some embodiments the modification may be limited so that in dispersed conditions (dispersion is measured by the D / A ratio), the amount of modified signal in the mix is ​​limited.

[0180] In some embodiments, the device or headphones have a transparency mode in which microphone signals from the sides or ear cups are compared to estimated sound direction and / or diffuseness, and the transparency signals are modified so that the direction of the sound source is intentionally clear, particularly when the internal signal (music) from a device connected to the device or headphones is loud.

[0181] In this way, embodiments may be similar to the "two-side" or earcup modifications discussed above, but when using microphones from the sides or earcups, the paired distances of the microphones are more similar to the distance between the user's ears, and thus the estimates of directionality and / or diffuseness are more similar to the perception of a user not wearing headphones.

[0182] In a similar manner, this embodiment is better when the two sides (or ear cups of headphones) are connected with a wired connection so that the latency of using the microphone signals from the two sides or ear cups is negligible. In some embodiments, wireless connection between device sides can be achieved, such as in-ear headphones, but performance may be worse due to the additional latency.

[0183] As mentioned above, "thickness" refers to the distance between the microphone and the speaker element on a line parallel to the axis defined by the user's ear. Therefore, "thickness" in this application does not refer to the thickness of the headset, or more generally the thickness of the entire device.

[0184] In the following embodiments, there is benefit even if the microphones are not perfectly located on a line parallel to the axis defined by the user's ears, but the results applied in these cases are less than optimal.

[0185] So, for example, the thickness relative to Fig. 9 Shown. Fig. 9 A user 901 is shown wearing a headset 961. The headset 961 includes a headband 951 and left and right ear cups 903, 913. The left ear cup 903 has a left external microphone 905 and a left internal speaker 907 therein, separated by a left thickness 909. The right ear cup 913 has a right external microphone 915 and a right internal speaker 917 therein, separated by a right thickness 919. In this example, the thickness is defined relative to an axis 911 defined by the user's ear.

[0186] also, Figures 10a to 10c Examples are shown in, for example, Fig.10a Microphones 1001 and 1003 are located on axis 1005 defined by the user's ears. Fig.10b Microphones 1001 and 1003 are located on an axis 1015 parallel to an axis 1005 defined by the user's ears. Fig.10c Center microphones 1001 and 1003 are located on an axis 1025 that is non-parallel to an axis 1005 defined by the user's ears.

[0187] about Fig.11 , shows an example simulation result after signal modification. In the example shown, there are two sinusoidal signals (dashed lines) 1001 and 1011, which have the same frequency but different phases. However, by using the mid / side modification, they are closer to each other in phase. Therefore, Fig.11The dashed lines 1003 and 1013 and the solid lines 1005 and 1015 shown in FIG. 1 show where the modified signals are closer in phase at different modification levels.

[0188] In some embodiments, the left thickness value and the right thickness value may be different, such as Fig.11 shown. Fig.11 A user 901 is shown wearing a headset 961. The headset 961 includes a headband and a left ear cup 1203 and a right ear cup 1213. The left ear cup 1203 has a left external microphone 905 and a left internal speaker 907 therein, which are separated by a left thickness 1209. The right ear cup 1213 has a right external microphone 915 and a right internal speaker 917 therein, which are separated by a right thickness 1219. In this example, the left thickness 1209 is smaller than the right thickness 1219 and is defined relative to an axis 911 defined by the user's ear.

[0189] like Fig.13 As shown, the difference between the left thickness and the right thickness may cause the audio signal to be modified differently. Fig.13 In, with Fig.11 Compared to the audio signals of the right microphone signals 1311, 1313, 1315, the modification is more than the modification of the left microphone audio signals 1301, 1303, 1305 because the right thickness is greater than the left thickness.

[0190] In the simplest form, these further embodiments also control equalization based on the detected direction. The equalization is achieved in that the difference in level of the frequency band between the left ear cup transparent signal and the right ear cup transparent signal corresponds to the binaural signal from the detected direction. The level difference of the binaural signal can be found from a stored database or suitable storage. The database can be general or personalized for the current user.

[0191] The method is further shown in Fig.14 In the operation shown.

[0192] With respect to step 1401, the operation of receiving microphone signals from two ear cups (L and R) is shown.

[0193] Then, with respect to step 1403, the operation of dividing the signal into time-frequency blocks is shown.

[0194] As shown in step 1405, the operation is to estimate the sound direction in at least one block.

[0195] In addition, an operation of estimating a D / A ratio in at least one block is also shown in step 1407. This estimation operation is an optional step.

[0196] In addition, step 1409 also shows an operation of searching / calculating (or otherwise acquiring or determining) a level and / or phase difference from a database (or a suitable storage module) for at least one block direction.

[0197] Then, as shown in step 1411, the operation is to modify the see-through signal in one ear so that the level difference in at least one block is the same as the level difference in the database.

[0198] In some embodiments, the diffuseness of the transparent signal may also be modified based on the measured diffuseness by decorrelating or correlating the transparent signal so that its diffuseness matches the diffuseness measured from the external microphone. Decorrelation of the signals may be achieved in any suitable manner, such as using a decorrelator, and correlation of the signals may be achieved, for example, by mixing the stereo transparent signal with its mono downmix.

[0199] Wherein, the greater the level difference between the left and right ears, the closer to the ear canal it is measured. Conversely, the greater the phase difference, the further away from the ear canal it is measured. Thus, in some embodiments, the modification of the pass-through signal increases the level difference for thicker headphones and decreases the phase difference for thicker headphones. The amount of increase and decrease may be frequency dependent and may be found from a database that has been measured (or modeled) and stored in the memory of the headphones.

[0200] One possible implementation is to use mid / side coding. For example, when the microphone and speaker in a headset are Fig.15 As shown, the left audio signal (l) from the left ear cup microphone 905 and the right audio signal (r) from the right ear cup microphone 915 are converted to a mid / side representation:

[0201]

[0202]

[0203] The mid / side representation is converted back to the left speaker 907 audio signal (L) and the right speaker 917 audio signal (R).

[0204] The distance between the center line 1501 and the left or right microphone 905 or 915 is distance a 1503, and the distance between the center line 1501 and the left or right speaker 907 or 917 is distance b 1505.

[0205]

[0206] Additionally, some equalization of the speaker signal is required because headphones acoustically allow sound to pass through them in different amounts at different frequencies.

[0207] Using a mid / side representation is effective because it both improves the perceived direction of the sound to be closer to reality and modifies the coherence of the sound to better correspond to what the user would hear if the microphone was not too far from the speaker (the farther the microphone is from the user's ear, the more coherence needs to be increased). The mid / side representation is also very simple to compute and therefore does not consume much processing power or headphone battery. In some embodiments, other implementations are possible but will take up more processor resources.

[0208] In some embodiments, the microphone can be Fig.16 The asymmetrical placement shown, such that the distance a L and a R Therefore, the distance between the center line 1501 and the left and right microphones 905 and 915 is the distance a. L 1603 and a R 1613, the distance between the center line 1501 and the left and right speakers 907 and 917 is distance b 1505.

[0209] In this case, a different calculation is required according to the following formula:

[0210]

[0211] Therefore, the rendering of the microphone signals may be different when the microphones are located outside the vehicle and the speakers are in conventional positions (in the doors, dashboard, etc.) or when the speakers are located in the driver's seat. With a traditional positional speaker configuration, the microphone signals can be rendered almost "as is" to the speakers, but for the speakers in the seats, the stereo image should be significantly narrower. The mid-side example discussed above can be applied to suitable microphone selection. For normal speaker placement, microphones widely distributed on the outer surface of the vehicle can be selected, but for the seat speaker example, microphones closer to each other can be selected.

[0212] about Fig.17 and 18 , shows an example graph, where:

[0213] The solid line with an “o” indicates the amount of leakage;

[0214] The solid line with “*” indicates the level difference between the L (left) ear and the R (right) ear without headphones;

[0215] The solid line with an “x” indicates the level difference between the L and R ears wearing the headphones;

[0216] The dotted line with “*” indicates the phase difference between the L ear and the R ear without wearing headphones;

[0217] The dotted line with an "x" represents the phase difference between the L ear and the R ear wearing the headphone.

[0218] The through signal is ideally such that when it is acoustically combined with the leaked sound (the leaked sound has the characteristics of a line with x's), the combination has the characteristics of a line with *'s.

[0219] about Fig.19 , is an example electronic device that can be used as any of the device parts of the above-mentioned system. The device can be any suitable electronic device or device. For example, in some embodiments, the device 2000 is a mobile device, a user device, a tablet computer, a computer, an audio playback device, etc. The device can be configured to implement an encoder or a renderer or any functional block as described above.

[0220] In some embodiments, the device 2000 includes at least one processor or central processing unit 2007. The processor 2007 may be configured to execute various program codes, such as the methods described herein.

[0221] In some embodiments, the device 2000 includes a memory 2011. In some embodiments, at least one processor 2007 is coupled to the memory 2011. The memory 2011 can be any suitable storage module. In some embodiments, the memory 2011 includes a program code portion for storing program codes that can be implemented on the processor 2007. In addition, in some embodiments, the memory 2011 can also include a stored data portion for storing data (e.g., processed or to be processed data) according to the embodiments described herein. Whenever necessary, the implemented program code stored in the program code portion and the data stored in the stored data portion can be retrieved by the processor 2007 through the memory-processor coupling.

[0222] In some embodiments, the device 2000 includes a user interface 2005. In some embodiments, the user interface 2005 can be coupled to a processor 2007. In some embodiments, the processor 2007 can control the operation of the user interface 2005 and receive input from the user interface 2005. In some embodiments, the user interface 2005 can enable a user to input commands to the device 2000, for example, via a keypad. In some embodiments, the user interface 2005 can enable a user to obtain information from the device 2000. For example, the user interface 2005 can include a display configured to display information from the device 2000 to the user. In some embodiments, the user interface 2005 can include a touch screen or touch interface that can enable information to be input to the device 2000 and can display information to the user of the device 2000. In some embodiments, the user interface 2005 can be a user interface for communication.

[0223] In some embodiments, the device 2000 includes an input / output port 2009. In some embodiments, the input / output port 2009 includes a transceiver. In such embodiments, the transceiver can be coupled to the processor 2007 and configured to enable communication with other devices or electronic devices, for example, via a wireless communication network. In some embodiments, the transceiver or any suitable transceiver or transmitter and / or receiver module can be configured to communicate with other electronic devices or devices via a wire or wired coupling.

[0224] The transceiver can communicate with another device via any suitable known communication protocol. For example, in some embodiments, the transceiver can use a suitable Universal Mobile Telecommunications System (UMTS) protocol, a wireless local area network (WLAN) protocol such as, for example, IEEE 802.X, a suitable short-range radio frequency communication protocol such as Bluetooth, or an infrared data communication path (IRDA).

[0225] Input / output port 2009 may be configured to receive signals.

[0226] In some embodiments, device 2000 may function as at least part of a renderer.Input / output port 2009 may be coupled to a headset (which may be a head-tracked or non-tracked headset) or the like.

[0227] In general, various embodiments of the present invention may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, although the present invention is not limited thereto. Although various aspects of the present invention may be illustrated and described in block diagrams, flow charts, or using some other graphical representations, it is fully understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof, as non-limiting examples.

[0228] Embodiments of the present invention can be implemented by computer software that can be executed by a data processor (e.g., in a processor entity) of a mobile device, or by hardware, or by a combination of software and hardware. In addition, in this regard, it should be noted that any block of the logic flow in the figure can represent a program step, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software can be stored in a physical medium such as a memory chip or a memory block implemented in a processor, a magnetic medium such as a hard disk or a floppy disk, and an optical medium such as a DVD and its data variant CD.

[0229] The memory may be of any type suitable for the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. The data processor may be of any type suitable for the local technical environment and may include one or more of a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a gate-level circuit, and a processor based on a multi-core processor architecture, as non-limiting examples.

[0230] Embodiments of the present invention may be practiced in various components such as integrated circuit modules. The design of integrated circuits is essentially a highly automated process. Complex and powerful software tools are available to convert a logic level design into a semiconductor circuit design ready to be etched and formed on a semiconductor substrate.

[0231] Programs, such as those offered by Synopsys Inc. of Mountain View, Calif., and Cadence Design Inc. of San Jose, Calif., use well-established design rules along with a library of pre-stored design blocks to automatically route conductors and locate components on a semiconductor chip. Once the design of a semiconductor circuit is complete, the resulting design in a standardized electronic format (e.g., Opus, GDSII, etc.) can be transferred to a semiconductor fabrication facility or "fab" for manufacturing.

[0232] As used in this application, the term "circuitry" may refer to one or more or all of the following:

[0233] (a) pure hardware circuit implementation (e.g. implementation in analog and / or digital circuits only) and

[0234] (b) a combination of hardware circuitry and software, such as (where applicable):

[0235] (i) a combination of analog and / or digital hardware circuitry and software / firmware, and

[0236] (ii) any portion of a hardware processor (including a digital signal processor), software and memory with software that works together to enable a device such as a mobile phone or server to perform various functions, and

[0237] I A hardware circuit and / or a processor (eg, a microprocessor or portion of a microprocessor) that requires software (eg, firmware) for operation, but the software may not be present when it is not needed for operation.

[0238] This definition of circuitry applies to all uses of the term in this application, including in any claims. As another example, as used in this application, the term circuitry also covers implementations of only a hardware circuit or processor (or multiple processors) or portions of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example and where applicable to a particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device, or a similar integrated circuit in a server, cellular network device, or other computing or network device.

[0239] The term "non-transitory" as used herein refers to the limitation of the medium itself (ie, tangible, not a signal), not the limitation of data storage persistence (eg, RAM vs. ROM).

[0240] As used herein, “at least one of: ” and “at least one of ” and similar expressions, where a list of two or more elements is connected by “and” or “or”, mean at least any one element, or at least any two or more elements, or at least all elements.

[0241] The foregoing description provides a complete and informative description of exemplary embodiments of the present invention by way of exemplary and non-limiting examples. However, various modifications and adaptations may become apparent to those skilled in the relevant arts when the foregoing description is read in conjunction with the accompanying drawings and the appended claims. However, all such and similar modifications to the present teachings will still fall within the scope of the present invention as defined in the appended claims.

Claims

1. A method for generating an audio signal for a device equipped with a transparency mode, the method comprising: acquiring at least two external audio signals from at least two microphones located on the device; determining, based on the at least two external audio signals, at least one of: a sound direction or a diffuseness or a distance between a first microphone and a second microphone of the at least two microphones; modifying at least one of the at least two external audio signals based on the determined at least one of: sound direction, diffuseness, or distance; as well as The modified at least one audio signal is rendered.

2. The method according to claim 1, wherein: The at least two microphones located on the device are located on a side of the device.

3. The method according to claim 1, wherein: The at least two microphones located on the device are located on opposite sides of the device. 4 . The method of claim 1 , further comprising acquiring at least one internal audio signal, and modifying at least one of the at least two external audio signals is further based on the at least one internal audio signal.

5. The method according to claim 1, comprising: The at least one of the at least two external audio signals is modified further based on the frequency profile to modify the at least two external audio signals more in lower frequencies.

6. The method according to claim 1, further comprising: Determining a distance between at least one of the at least two microphones located on the device and an associated speaker, wherein modifying at least one of the at least two external audio signals is based on the distance between at least one of the at least two microphones located on the device and the associated speaker.

7. The method according to claim 1, wherein: The device comprises one of the following: Smartphone; Headphones; a vehicle equipped with the at least two microphones; a helmet equipped with said at least two microphones; and A personal protective equipment equipped with said at least two microphones.

8. The method according to claim 1, wherein: The first microphone and the second microphone of the at least two microphones are a left device microphone and a right device microphone.

9. The method according to claim 1, wherein: The first microphone is a first group of microphones, and the second microphone is a second group of microphones.

10. The method according to claim 1, wherein: Based on the determined at least one of: sound direction, diffuseness, or distance, at least one of the at least two external audio signals is modified so that a user of the device more correctly perceives the modified audio signal direction or diffuseness.

11. An apparatus for generating an audio signal for a device equipped with a transparency mode, the apparatus comprising at least one processor and at least one memory, the at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus to at least: acquiring at least two external audio signals from at least two microphones located on the device; determining, based on the at least two external audio signals, at least one of: a sound direction or a diffuseness or a distance between a first microphone and a second microphone of the at least two microphones; modifying at least one of the at least two external audio signals based on the determined at least one of: sound direction, diffuseness, or distance; and The modified at least one audio signal is rendered.

12. The device according to claim 11, wherein The at least two microphones located on the device are located on a side of the device.

13. The device according to claim 11, wherein: The at least two microphones located on the device are located on opposite sides of the device.

14. The apparatus of claim 11, further caused to acquire at least one internal audio signal, and the apparatus caused to modify at least one of the at least two external audio signals based on the at least one internal audio signal.

15. The apparatus of claim 11, caused to modify the at least one of the at least two external audio signals further based on a frequency profile to modify the at least two external audio signals more in lower frequencies.

16. The apparatus of claim 11, further caused to determine a distance between at least one of the at least two microphones located on the device and the associated speaker, wherein: The apparatus is caused to modify at least one of the at least two external audio signals based on the distance between at least one of the at least two microphones located on the device and the associated speaker.

17. The device according to claim 11, wherein: The device comprises one of the following: Smartphone; Headphones; a vehicle equipped with the at least two microphones; a helmet equipped with said at least two microphones; and A personal protective equipment equipped with said at least two microphones.

18. The device according to claim 11, wherein The first microphone and the second microphone of the at least two microphones are a left device microphone and a right device microphone.

19. The device according to claim 11, wherein: The first microphone is a first group of microphones, and the second microphone is a second group of microphones.

20. The apparatus of claim 11, being configured to modify at least one of the at least two external audio signals based on at least one of the determined sound direction, diffuseness, or distance, such that a user of the device more correctly perceives the modified audio signal direction or diffuseness.

Images