Audio-tactile signal generator

Abstract

An audio-tactile signal generator for a haptic system is described, the haptic system including an amplifier coupled to a tactile actuator. The audio-tactile signal generator includes an audio input configured to receive an audio signal; a tactile input configured to receive a tactile signal; and a controller configured to receive the tactile signal, at least one of an amplifier state, and a tactile actuator state. A mixer is coupled to the audio input and the tactile input. The mixer has an output configured to be coupled to the tactile actuator. The controller controls the mixer to process the audio signal depending on at least one of a characteristic of the tactile signal, the amplifier state, and the tactile actuator state. The mixer is configured to mix the tactile signal and a processed audio signal and output the mixed tactile signal and the processed audio signal.

Description

Technical Field

[0001] The present disclosure relates to an audio-haptic signal generator and a method of generating an audio-haptic signal. Background Art

[0002] Human-machine interaction (HMI) is increasingly relying on cross-modal user interaction, which involves multiple sensory modalities. In particular, cross-modal auditory and tactile feedback can be used to enhance touch interfaces. A typical example is a "virtual button", in which button presses are detected by a touch sensor and user feedback is provided by vibration or tactile feedback and usually by sound. Suitable tactile and auditory feedback designs make it possible to simulate real physical buttons. The combination of tactile and auditory feedback can be called audio-tactile. Such virtual buttons can be used on some commercial smartphones. Physical buttons have reliability issues due to, for example, the ingress of dust and water, which worsen over time and add physical limitations to the design of the device. On the other hand, virtual buttons are more reliable and easy to integrate, and they have more functions and are reprogrammable. Summary of the Invention

[0003] Various aspects of the present disclosure are defined in the accompanying claims. In a first aspect, an audio-tactile signal generator for a haptic system is provided, the haptic system comprising an amplifier coupled to a tactile actuator, the audio-tactile signal generator comprising: an audio input configured to receive an audio signal; a tactile input configured to receive a tactile signal; a controller configured to receive the tactile signal, at least one of an amplifier state, and a tactile actuator state; a mixer coupled to the audio input and the tactile input and having a mixer output configured to be coupled to the tactile actuator; wherein the controller is configured to control the mixer to process the audio signal depending on at least one of a characteristic of the tactile signal, the amplifier state, and the tactile actuator state; and wherein the mixer is configured to mix the tactile signal and a processed audio signal and output the mixed audio-tactile signal.

[0004] In one or more embodiments, the controller may be further configured to control the mixer to process the audio signal depending on characteristics of the audio signal.

[0005] In one or more embodiments, the controller may further include an amplifier sensor input configured to be coupled to the amplifier, wherein the controller is further configured to determine an amplifier state based on a sensor signal received on the amplifier sensor input and to control the mixer to process the audio signal depending on the amplifier state.

[0006] The amplifier state may include at least one of: an amplifier boost voltage, an amplifier voltage clipping level, an amplifier current clipping level, an amplifier load current, and an amplifier die temperature.

[0007] In one or more embodiments, the controller may further include a tactile actuator sensor input, which is configured to be connected to a tactile actuator, wherein the controller is further configured to determine a tactile actuator state based on a tactile actuator sensor signal received on the tactile actuator sensor input, and control the mixer to process the audio signal depending on the tactile actuator state.

[0008] The haptic actuator state may include at least one of: a haptic actuator resonant frequency, a haptic actuator offset, a haptic actuator velocity, a haptic actuator acceleration, and a haptic actuator voice coil temperature estimate.

[0009] In one or more embodiments, the mixer can be configured to process the audio signal by at least one of: time shifting the audio signal relative to the tactile signal, applying gain to the audio signal, applying a frequency-dependent gain to the audio signal, applying a frequency-dependent phase shift to the audio signal, and high-pass filtering the audio signal.

[0010] In one or more embodiments, the mixer may include an audio processor coupled to a summer, wherein an audio processor input is coupled to the audio input, an audio processor output is coupled to a first input of the summer, and a haptic signal input is coupled to a second input of the summer.

[0011] In one or more embodiments, the audio processor may include at least one of a delay adjuster, a gain adjuster, and a phase adjuster.

[0012] In one or more embodiments, the audio processor may include a series arrangement of at least two of a delay adjuster, a gain adjuster, and a phase adjuster coupled between the audio processor input and the audio processor output.

[0013] In one or more embodiments, the controller may include: a state parameter calculator coupled to an amplifier sensor input and a tactile sensor input and having a state parameter calculator output, wherein the state parameter calculator is configured to determine an amplifier state based on an amplifier sensor signal received on the amplifier sensor input and to determine a tactile state based on a tactile sensor signal received on the tactile sensor input;

[0014] an amplitude-frequency detector coupled to the audio input and the tactile input and having an amplitude-frequency detector output; at least one of a delay controller, a gain controller, and a phase controller; wherein the delay controller comprises a first delay controller input, a second delay controller input, and a delay controller output, the first delay controller input being coupled to the state parameter calculator output, the second delay controller input being coupled to the amplitude-frequency detector output, and the delay controller output being coupled to the delay adjuster; the gain controller comprises a first gain controller input, a second gain controller input, and a gain controller output, the first gain controller input being coupled to the state parameter calculator output, the second gain controller input being coupled to the amplitude-frequency detector output, and the gain controller output being coupled to the gain adjuster; and the phase controller comprises a first phase controller input, a second phase controller input, and a phase controller output, the first phase controller input being coupled to the state parameter calculator output, the second phase controller input being coupled to the amplitude-frequency detector output, and the phase controller output being coupled to the delay adjuster.

[0015] Embodiments of the audio-haptic signal generator may be included in a haptic system comprising an amplifier having an input coupled to the output of the mixer and an output configured to be coupled to a haptic actuator.

[0016] Embodiments of the audio-tactile signal generator may be included in a human-machine interface (eg, a touchpad or touch screen).

[0017] In a second aspect, a method for generating an audio-tactile signal for a tactile system is provided, the tactile system including an amplifier that drives a tactile actuator, the method comprising: receiving an audio signal; receiving a tactile signal; processing the audio signal depending on at least one of a characteristic of the tactile signal, an amplifier state, and a tactile actuator state; and mixing the processed audio signal and the tactile signal.

[0018] In one or more embodiments, the method may comprise processing the audio signal in dependence on a characteristic of the audio signal.

[0019] In a third aspect, an audio-tactile signal generator is provided, comprising: an audio input configured to receive an audio signal; a tactile input configured to receive a tactile signal; a controller having a first controller input connected to the tactile input; an audio processor connected to the audio input and a controller output; an adder connected to the output of the audio processor and the tactile input and configured to combine the tactile signal and a processed audio signal; wherein the controller is configured to control the audio processor to process the audio signal depending on characteristics of the tactile signal; and the audio-tactile signal generator is configured to output the combined tactile signal and the processed audio signal.

[0020] In one or more embodiments, the controller may further include an amplifier sensor input and a state parameter calculator, wherein the amplifier sensor input is configured to be connected to the amplifier, and the state parameter calculator is connected to the amplifier sensor input, wherein the controller is further configured to determine an amplifier state, and the audio processor is configured to process the audio signal depending on the amplifier state.

[0021] In one or more embodiments, the controller may further include a tactile actuator sensor input and a state parameter calculator, wherein the tactile actuator sensor input is configured to be connected to the tactile actuator, and the state parameter calculator is connected to the tactile actuator sensor input, wherein the controller is further configured to determine an amplifier state, and the audio processor is configured to process the audio signal depending on the amplifier state.

[0022] In one or more embodiments, the controller may include a second controller input coupled to the audio input, and wherein the controller may be further configured to control the audio processor to process the audio signal depending on characteristics of the audio signal.

[0023] In one or more embodiments, the audio processor may include at least one of a delay adjuster, a gain adjuster, and a phase adjuster. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In the drawings and description, like reference numerals refer to like features. Embodiments are now described in detail, by way of example only, as illustrated in the accompanying drawings, in which:

[0025] Figure 1 An audio-tactile signal generator according to an embodiment is shown.

[0026] Figure 2Frequency responses of audio and haptic signals providing an example virtual button effect are shown.

[0027] Figure 3A shows a graph showing Figure 2 Example haptic signals and audio signals are provided, which combine to provide a virtual button effect.

[0028] Figure 3B The combined haptic actuator signal and the audio signal are shown.

[0029] Figure 3C An example combined haptic actuator signal and delayed audio signal are shown.

[0030] Figure 4 An audio-tactile signal generator according to an embodiment is shown.

[0031] Figure 5 Another audio-tactile signal generator according to an embodiment is shown.

[0032] Figure 6 An audio-tactile signal generator according to an embodiment is shown.

[0033] Figure 7 A method of controlling a haptic actuator according to an embodiment is shown. DETAILED DESCRIPTION

[0034] Figure 1 An audio-tactile signal generator 100 according to an embodiment is shown. Audio-tactile signal generator 100 includes a controller 110 and a mixer 120. An audio signal input 102 can be connected to a first input of mixer 120. A tactile signal input 104 can be connected to a second input of mixer 120. Haptic signal input 104 can be connected to an input of controller 110. Controller 110 can have an amplifier sensor input 112 and a tactile actuator sensor input 114. Controller 110 can have a control output 106 connected to a control input of mixer 120.

[0035] In operation, the mixer output 108 can be connected to an amplifier 130, which can be, for example, a Class D audio amplifier, a Class A amplifier, or a Class AB amplifier. The amplifier output 116 can be connected to a haptic actuator 140, which can be, for example, an electric actuator (such as a linear resonant actuator (LRA)) or a piezoelectric actuator. The haptic signal input 112 can be connected to the amplifier 130. The haptic actuator 140 can be connected to a haptic actuator sensor input 114.

[0036] To achieve an audio-haptic effect, an audio signal can be received on audio signal input 102, and a haptic signal can be received on haptic signal input 104. In this context, an audio signal can be considered an auditory signal that is intended to produce a specific audible sound that is intended to be heard by a user primarily when actuating a haptic actuator. The audio signal can generally be designed to avoid generating vibrations that could interfere with the intended vibrations caused by the haptic signal.

[0037] The haptic signal is a signal primarily intended to produce a specific tactile sensation or tactile feedback when used to drive a haptic actuator. There is no desired audible effect.

[0038] In operation, the controller 110 may receive the audio signal, the haptic signal, the amplifier sensor signal, and / or the haptic actuator sensor signal.

[0039] The amplifier sensor signal may, for example, directly indicate an amplifier state or be used to determine an amplifier state. The amplifier state may include one or more of: an amplifier boost voltage, a voltage limit level, a current limit level, a current drain value which may indicate battery current for a portable device, and an amplifier integrated circuit die temperature.

[0040] The haptic actuator sensor signal can, for example, directly indicate a haptic actuator state or be used to determine the haptic actuator state. For an LRA, the haptic actuator state can include one or more of the following: resonant frequency, LRA offset, LRA velocity or acceleration value, or LRA voice coil temperature value. The haptic actuator sensor signal can, for example, be a signal representing current flowing into the haptic actuator.

[0041] The controller 110 may apply a control signal to the mixer 120 depending on one or more of the haptic actuator state, the amplifier state, and the characteristics of the haptic signal so that the output generated by the mixer does not exceed system limitations. The controller 110 controls the mixer 120, which may process the audio signal depending on the control signal.

[0042] Example system constraints may include limiting the amplifier output voltage to not exceed the amplifier clipping level and limiting the current draw to not exceed the battery current limit. Additional constraints may include limiting the LRA excursion to not exceed predefined limits to prevent one or more of mechanical damage and audible mechanical artifacts, and limiting the LRA voice coil temperature to mitigate thermal damage.

[0043] Characteristics of the audio signal or haptic signal may include, for example, one or more of the following: instantaneous signal value, time-varying amplitude, frequency spectrum, or peak value. Processing the audio signal may include applying a delay to the audio signal relative to the haptic signal. Processing the audio signal may include applying a time-varying gain to the audio signal. Processing the audio signal may include applying a frequency-dependent amplitude variation to the audio signal. Processing the audio signal may include applying a frequency-dependent phase variation to the audio signal. Mixer 120 may process the signal by applying a frequency-dependent gain and phase shift to apply high-pass filtering to the audio signal. Mixer 120 may combine the conditioned or processed audio signal with the haptic signal and may output the mixed conditioned audio and haptic signal output at mixer output 108. Amplifier 130, also referred to as a haptic driver, may amplify the mixed conditioned audio and haptic signal and may drive haptic actuator 140 using the combined signal.

[0044] It should be understood that in some examples, controller 110 can control mixer 120 to process the audio signal determined solely by the characteristics of the haptic signal. In these examples, amplifier sensor input 112 and haptic actuator sensor input 114 can be omitted. In other examples, controller 110 can adjust the audio signal determined solely by one of the amplifier state or the haptic actuator state. In these examples, the connection from haptic signal input 104 to controller 110 and to one of amplifier sensor input 112 and haptic actuator sensor input 114 can be omitted.

[0045] The audio-tactile signal generator 100 can be implemented in hardware or a combination of hardware and software, such as by software executable on a microcontroller or digital signal processor. In some examples, the amplifier 130 and the audio-tactile signal generator 100 can be included on the same device.

[0046] The present inventors have recognized that a haptic actuator (e.g., an LRA) that is typically designed only to produce haptic effects can be controlled by the audio-haptic signal generator 100 to simultaneously output a combined audio and haptic signal, for example, to achieve an audio haptic effect, without overdriving the LRA and without degrading haptic performance. This can avoid the need for additional speakers or the use of expensive multi-function sensors. The present inventors have also recognized that by adjusting or processing the audio signal, the haptic performance of the haptic actuator (e.g., an LRA) can be unaffected by the additional audio signal. The present inventors have further recognized that the user's experience of signal modification to the auditory signal component of the audio-haptic feedback signal can be more robust than the haptic signal component of the audio-haptic feedback signal.

[0047] refer to Figure 2 、 3A , 3B, and 3C may additionally understand example operations of the audio haptic signal generator 100 for a virtual click audio haptic signal. Figure 2 An example spectrum 150 of a haptic signal 152 and an audio signal 154 for a typical virtual button effect is shown. The x-axis represents frequency on a logarithmic scale from 50 Hz to 20 kHz. The y-axis represents amplitude in dB. The haptic signal 152 is concentrated in low frequencies below 1 kHz, particularly below 300 Hz. The audio signal 154 is concentrated in higher frequencies, typically above 2 kHz.

[0048] Figure 3A Graphs of audio and haptic virtual click signals 160 are shown. The haptic signal is shown as line 162. The audio signal is shown as line 164. The y-axis represents the normalized amplitude between -1 and +1. In the example depicted, amplifier voltage clipping occurs for sample values outside this range. The x-axis represents time, which varies between 0 seconds and 50 milliseconds.

[0049] Figure 3B An audio-haptic signal 170 is shown. Line 172 represents the combined audio signal 162 and haptic signal 164 produced by simple mixing. This results in amplifier clipping, as shown by regions 174 and 176.

[0050] Figure 3C An audio-haptic signal 180 is shown. Line 182 represents the combined audio signal 162 and haptic signal 164 generated by mixing by audio-haptic signal generator 100. Haptic signal generator 100 can determine, based on the detected amplitude of the haptic signal, that amplifier limiting may occur if the audio signal is mixed simultaneously, and therefore delays the audio signal (by 6 milliseconds in the example) before mixing. As a result, amplifier limiting is avoided, and since the delay is relatively small, there may not be a noticeable difference to the user.

[0051] Figure 4An audio-haptic signal generator 200 according to an embodiment is shown. Audio-haptic signal generator 200 includes a controller 210 and a mixer 230. Mixer 230 includes an audio processor 220 and an adder 234. Audio processor 220 may include a delay adjuster 222 having an input connected to audio signal input 202. Delay controller 222 may have an output 224 connected to gain adjuster 226. Controller 226 may have an output 228 connected to phase adjuster 232. Phase adjuster 232 may have an output 208 connected to a first input of adder 234. A second input of adder 234 may be connected to haptic signal input 204. Controller 210 may have an output 206 connected to delay adjuster 222, gain adjuster 226, and phase adjuster 232. Controller 210 can have a first input connected to audio signal input 202. Controller 210 can have a second input connected to haptic signal input 204. The output of adder 234 can be connected to mixer output 212. Mixer output 212 can be connected to an amplifier (not shown) that can drive a haptic actuator (e.g., a linear resonant actuator) (not shown).

[0052] The controller 210 controls the audio processor 220 to adjust the audio signal depending on at least one of the characteristics of the haptic signal and the audio signal. The characteristics of the haptic signal and the audio signal may include, for example, one or more of the following: instantaneous signal value, time-varying amplitude, spectrum, or peak value.

[0053] Audio processor 220 can process the audio signal by applying a delay to the audio signal relative to the haptic signal using delay adjuster 222. Delay adjuster 222 can be implemented as a delay circuit element in hardware or a delay circuit element in software combined with hardware. In some examples, the delay adjuster can delay the audio signal until the amplitude of the haptic signal is below a predetermined threshold.

[0054] The audio processor 220 can process the audio signal by applying a time-varying gain to the audio signal or applying a frequency-dependent amplitude variation to the audio signal using a gain adjuster 226. In some examples, the gain adjuster 226 can apply a smooth, sample-based gain control, which is typically implemented in a dynamic range controller or limiter. Alternatively, or in addition, in some examples, the gain adjuster 226 can include: a first stage having a variable gain implemented using an analysis filter bank or a fast Fourier transform (FFT) or a filter bank with a variable gain per subband, followed by a second stage including a variable gain stage; and a third stage including a synthesis filter bank or an inverse fast Fourier transform (iFFT) or similar technique.

[0055] Audio processor 220 can process the audio signal by applying a frequency-dependent phase change to the audio signal using phase adjuster 232. In some examples, phase adjuster 232 can include an all-pass filter for adjusting the audio signal so that the audio signal is out of phase with respect to the haptic signal when the effect of the haptic signal alone approaches one of the system limits.

[0056] Mixer 234 can combine the processed audio signal output from audio processor 220 with the haptic signal and can output the mixed processed audio and haptic signal at mixer output 212. The amplifier (not shown), which can also be referred to as a haptic driver, can amplify the mixed conditioned audio and haptic signal and drive the haptic actuator (not shown) with the combined signal.

[0057] Audio-haptic signal generator 200 may be implemented in hardware or a combination of hardware and software, such as software executable on a microcontroller or digital signal processor.

[0058] The audio-haptic signal generator 200 can output a combined audio and haptic signal simultaneously, for example, to produce an audio-haptic effect without overdriving a haptic actuator (e.g., an LRA) that is typically designed only for haptic output while maintaining the haptic performance. Additionally, by adjusting the audio signal, the haptic performance of the haptic actuator (e.g., an LRA) can be unaffected by the additional audio signal. The audio-haptic signal generator 200 can process the audio signal based on both the audio signal characteristics and the haptic signal characteristics. This can allow the controller 210 to more accurately adjust the delay, gain, and phase of the audio signal than by using only the haptic signal.

[0059] Figure 5An audio-haptic signal generator 300 according to an embodiment is shown. Audio-haptic signal generator 300 includes a controller 310 and a mixer 330. Mixer 330 includes an audio processor 320 and an adder 334. An audio input 302 can be connected to an input of audio processor 320. An output 308 of audio processor 320 can be connected to a first input of adder 334. A haptic signal input 304 can be connected to a second input of adder 334. An output of adder 334 can be connected to mixer output 312. A haptic signal input 304 can be connected to a first input of controller 310. Audio signal input 302 can be connected to a second input of controller 310. Controller 310 can have an amplifier sensor input 314 and a haptic actuator sensor input 316. Controller 310 can have a control signal output 306, which is connected to a control input of audio processor 320.

[0060] In operation, mixer output 312 can be connected to amplifier 340, which can be, for example, a Class D audio amplifier. Amplifier output 322 can be connected to haptic actuator 350, which can be, for example, a linear resonant actuator (LRA). Haptic signal input 314 can be connected to amplifier 340. Haptic actuator 350 can be connected to haptic actuator sensor input 316.

[0061] To achieve audio-haptic effects, an audio signal can be received on audio signal input 302, and a haptic signal can be received on haptic signal input 304. The audio signal is an audible signal primarily intended to produce an auditory output from the haptic actuator. The haptic signal is primarily intended to produce a tactile sensation or haptic feedback from the haptic actuator.

[0062] In operation, the controller 310 may receive the haptic signal, the audio signal, the amplifier sensor signal, and the haptic actuator sensor signal.

[0063] The amplifier sensor signal may, for example, directly indicate an amplifier state or be used to determine an amplifier state.The haptic actuator sensor signal may, for example, directly indicate a haptic actuator state or be used to determine a haptic actuator state.

[0064] Controller 310 may apply control signals to audio processor 320 depending on one or more of the haptic actuator state, the amplifier state, and characteristics of the haptic signal and the audio signal so that the output generated by mixer 330 does not exceed the system limits.

[0065] The processing of the audio signal by the audio processor 320 may include applying a delay to the audio signal relative to the tactile signal. The processing of the audio signal by the audio processor 320 may include applying a time-varying gain to the audio signal. The processing of the audio signal by the audio processor 320 may include applying a frequency-dependent amplitude change to the audio signal. The processing of the audio signal by the audio processor 320 may include applying a frequency-dependent phase change to the audio signal. The adder 334 may combine the processed audio signal with the tactile signal and may output the mixed processed audio and tactile signal at the mixer output 312. The amplifier 340, which may also be referred to as a tactile driver, may amplify the mixed adjusted audio and tactile signal and may drive the tactile actuator 350 using the combined signal.

[0066] Audio-haptic signal generator 300 enables mixing of audio-haptic signals without overdriving the system or degrading haptic performance. Audio-haptic signal generator 300 enables haptic actuators (such as LRAs) originally designed only for haptic output to be used with audio-haptic signals. In addition, because controller 310 controls audio processor 320 based on the characteristics of the audio and haptic signals, amplifier characteristics, and haptic actuator state, control of the audio signal can be more precise while remaining within the system limitations of the amplifier and haptic actuator.

[0067] Figure 6 An audio-haptic signal generator 400 according to an embodiment is shown. The audio-haptic signal generator 400 includes a controller 410 and a mixer 430. The mixer 430 includes an audio processor 420 and an adder 434.

[0068] The audio processor 420 can include a delay controller 422 having an input connected to the audio signal input 402. The delay controller 422 can have an output 424 connected to a gain adjuster 426. The controller 426 can have an output 428 connected to a phase adjuster 432. The phase adjuster 432 can have an output connected to the audio processor output 408. The audio processor output 408 can be connected to a first input of a mixing module 434. A second input of an adder 434 can be connected to the haptic signal input 404.

[0069] Controller 410 can have an amplitude and frequency detector 440 connected to audio signal input 402 and haptic signal input 404. Amplitude and frequency detector output 448 can be connected to delay controller 442. The output of delay controller 442 can be connected to delay control output 406. Amplitude and frequency detector output 448 can be connected to gain controller 444. The output of gain controller 444 can be connected to gain control output 406'. Amplitude and frequency detector output 448 can be connected to phase controller 446. The output of phase controller 446 can be connected to phase control output 406".

[0070] The delay control output 406 may be connected to a delay adjuster 422. The gain control output 406' may be connected to a gain adjuster 426. The phase control output 406" may be connected to a phase adjuster 432.

[0071] The controller 410 can have an amplifier sensor input 414 and a haptic actuator sensor input 416 connected to a state parameter calculator 450. The state parameter calculator 450 can have an output 456 connected to the delay controller 442, the gain controller 444, and the phase controller 446.

[0072] In operation, the mixer output 412 can be connected to an amplifier (not shown), which is connected to a haptic actuator (e.g., a linear resonant actuator). To obtain an audio-haptic effect, an audio signal can be received on the audio signal input 402, and a haptic signal can be received on the haptic signal input 404. An amplitude and frequency detector 440 can determine characteristics of the audio signal and characteristics of the haptic signal. The characteristics can include instantaneous signal value, time-varying amplitude, peak amplitude, and frequency spectrum. The amplitude and frequency detector 440 can output the characteristics of the audio signal and the characteristics of the haptic signal to a delay controller 442, a gain controller 444, and a phase controller 446.

[0073] The state parameter calculator 450 may receive an amplifier sensor signal from the amplifier (not shown) on an amplifier sensor input 414 . The state parameter calculator 450 may receive a tactile sensor signal from the tactile actuator (not shown) on a tactile actuator sensor input 416 .

[0074] The state parameter calculator 450 can determine amplifier state values and haptic actuator state values based on the corresponding amplifier sensor values and haptic actuator sensor values. In some examples, the state parameter calculator 450 can determine estimates of LRA offset and voice coil temperature based on the haptic actuator current value. In other examples, the state parameter calculator 450 can determine estimates of LRA offset and voice coil temperature based on the amplifier current value and an amplifier behavioral model. The state parameter calculator 450 can provide the haptic state values and amplifier state values to the delay controller 442, the gain controller 444, and the phase controller 446.

[0075] The delay controller 442 can generate a delay control output value at the delay control output 406 that is determined based on the audio signal characteristics and haptic signal characteristics provided by the amplitude and frequency detector 440 and the haptic actuator state and amplifier state provided by the state parameter calculator 450. For example, when the sum of the instantaneous signal value of the audio signal and the instantaneous signal value of the haptic signal exceeds the amplifier voltage clipping level, the delay controller 442 can control the delay adjuster 422 to apply a delay to the audio signal so that the peak of the audio signal can be shifted in time.

[0076] Gain controller 444 can generate a gain control output value at gain control output 406 ′ that is determined based on the audio signal characteristics, the haptic signal characteristics, the haptic actuator state, and the amplifier state. Gain adjuster 426 can adjust the gain of the audio signal based on the gain control output value.

[0077] For example, when the sum of the instantaneous signal value of the audio signal and the instantaneous signal value of the haptic signal exceeds the amplifier voltage clipping level, the delay controller 444 may control the gain adjuster 426 to apply attenuation corresponding to a gain value less than 0 dB.

[0078] Phase controller 446 can generate a phase control output value at phase control output 406″ that is determined based on the audio signal characteristics, the haptic signal characteristics, the haptic actuator state, and the amplifier state. Phase adjuster 432 can adjust the phase of the audio signal depending on the phase control output value.

[0079] For example, when the sum of the instantaneous signal value of the audio signal and the instantaneous signal value of the tactile signal exceeds the amplifier voltage limiting level, the phase controller 446 can control the phase adjuster 432 to apply full-pass filtering to the audio signal so that the peak of the audio signal can be shifted in time.

[0080] Summer 434 can combine the processed audio signal output from audio processor 420 with the haptic signal and can output the mixed processed audio and haptic signal output at mixer output 412. The amplifier (not shown), which can also be referred to as a haptic driver, can amplify the mixed adjusted audio and haptic signal and drive the haptic actuator (not shown) with the combined signal.

[0081] Audio-haptic signal generator 400 may be implemented in hardware or a combination of hardware and software, such as software executable on a microcontroller or digital signal processor.

[0082] The audio-haptic signal generator 400 can output a combined audio and haptic signal simultaneously, for example, to produce an audio-haptic effect without overdriving a haptic actuator (e.g., an LRA) that is typically designed only for haptic output while maintaining the haptic performance. Additionally, by adjusting the audio signal, the haptic performance of the haptic actuator (e.g., an LRA) can be unaffected by the additional audio signal. The audio-haptic signal generator 400 can process the audio signal based on both the audio signal characteristics and the haptic signal characteristics, as well as the amplifier state and the haptic actuator state. This can enable the controller 410 to more accurately adjust the delay, gain, and phase of the audio signal than by using only the haptic signal, the amplifier state, or the haptic actuator state.

[0083] In some examples, one or two of the delay adjuster 422, gain adjuster 426, and phase adjuster 432 can be omitted along with the corresponding delay controller 442, gain controller 444, and phase controller 446. In some examples, the delay adjuster 422, gain adjuster 426, and phase adjuster 432 can be arranged in a different order between the audio input 402 and the audio processor output 408.

[0084] Figure 7A method 500 for generating an audio haptic signal for a haptic system including an amplifier and a haptic actuator according to an embodiment is shown. In step 502, an audio signal is received. In step 504, a haptic signal is received. In step 506, the audio signal is processed based on at least one of characteristics of the audio signal, characteristics of the haptic signal, an amplifier state, and a haptic actuator state. In step 508, the processed audio signal is mixed with the haptic signal. Method 500 can enable the audio signal and the haptic signal to be combined and used by the amplifier to drive a haptic actuator intended solely for the haptic signal without overdriving the system or degrading the haptic performance. This can avoid the need for a separate speaker for the audio haptic effect. In some examples, method 500 can be applied in real time. In some examples, method 500 can be applied to generate the audio-haptic signal for later use.

[0085] An audio-tactile signal generator for a haptic system is described, the haptic system including an amplifier coupled to a haptic actuator. The audio-tactile signal generator includes an audio input configured to receive an audio signal; a tactile input configured to receive a tactile signal; and a controller configured to receive the tactile signal, at least one of an amplifier state, and a tactile actuator state. A mixer is coupled to the audio input and the tactile input. The mixer has an output configured to be coupled to the tactile actuator. The controller controls the mixer to process the audio signal based on at least one of a characteristic of the tactile signal, the amplifier state, and the tactile actuator state. The mixer is configured to mix the tactile signal with a processed audio signal and output the mixed tactile signal and the processed audio signal.

[0086] Embodiments of the linear resonant actuator controller and embodiments of the method of controlling a linear resonant actuator can be included in mobile devices such as smartphones, smart watches, portable medical devices, wearable devices, laptops, tablets, etc. Embodiments can be included in any device that uses an HMI with haptic and audio feedback. For example, the device can be a control panel for industrial control, automotive control, or home control systems.

[0087] The audio-tactile signal generator described herein can be implemented, for example, in a touchpad using a single tactile actuator for many application areas such as automotive and the Internet of Things (IoT), where physical buttons can be replaced by virtual buttons.

[0088] Although the appended claims are directed to particular combinations of features, it should be understood that the scope of the disclosure of the present invention also includes any novel feature or any novel combination of features disclosed herein explicitly or implicitly, or any generalization thereof, regardless of whether it relates to the same invention as the invention currently claimed in any claim and regardless of whether it alleviates any or all of the same technical problems as the present invention.

[0089] Features that are described in the context of separate embodiments may also be provided in combination in a single embodiment. Conversely, various features that are, for brevity, described in the context of a single embodiment may also be provided separately or in any suitable subcombination.

[0090] In some example embodiments, the above-described instruction sets / method steps are implemented as functions and software instructions embodied as executable instruction sets, which are implemented on a computer or machine programmed and controlled by the executable instructions. Such instructions are loaded for execution on a processor (such as one or more CPUs). The term processor includes a microprocessor, a microcontroller, a processor module or subsystem (including one or more microprocessors or microcontrollers), or other control or computing devices. A processor can refer to a single component or multiple components.

[0091] The applicants give full notice that new claims may be formulated to such features and / or combinations of such features during the prosecution of the present application or of any further application derived therefrom.

[0092] For the sake of completeness, it is also pointed out that the term "comprising" does not exclude other elements or steps, the term "a or an" does not exclude a plurality, a single processor or other unit may perform the functions of several means recited in the claims, and the reference signs in the claims should not be construed as limiting the scope of the claims.

Claims

1. An audio-tactile signal generator for a tactile system, characterized in that: The haptic system includes an amplifier coupled to a haptic actuator, the audio-haptic signal generator including: an audio input configured to receive an audio signal; a tactile input configured to receive a tactile signal; a controller configured to receive at least one of the haptic signal, an amplifier state, and a haptic actuator state; a mixer coupled to the audio input and the tactile input and having a mixer output configured to be coupled to the amplifier; wherein The controller is further configured to control the mixer to process the audio signal depending on at least one of a characteristic of the haptic signal, the amplifier state, and the haptic actuator state; and wherein the mixer is configured to mix the haptic signal and the processed audio signal and output a mixed audio-haptic signal; The controller further includes an amplifier sensor input configured to be coupled to the amplifier, wherein the controller is further configured to determine an amplifier state based on a sensor signal received on the amplifier sensor input and to control the mixer to process the audio signal depending on the amplifier state.

2. The audio-tactile signal generator according to claim 1, characterized in that The controller is further configured to control the mixer to process the audio signal depending on characteristics of the audio signal.

3. The audio-tactile signal generator according to claim 1, characterized in that The amplifier state includes at least one of the following: an amplifier boost voltage, an amplifier voltage limit level, an amplifier current limit level, an amplifier load current, and an amplifier die temperature.

4. The audio-tactile signal generator according to claim 1, wherein: The controller further includes a tactile actuator sensor input, which is configured to be connected to a tactile actuator, wherein the controller is further configured to determine a tactile actuator state based on a tactile actuator sensor signal received on the tactile actuator sensor input, and control the mixer to process the audio signal depending on the tactile actuator state; wherein the tactile actuator state includes at least one of the following: tactile actuator resonant frequency, tactile actuator offset, tactile actuator velocity, tactile actuator acceleration, and tactile actuator voice coil temperature estimation.

5. The audio-tactile signal generator according to claim 1, characterized in that The mixer is configured to process the audio signal by at least one of: time-shifting the audio signal relative to the haptic signal, applying gain to the audio signal, applying frequency-dependent gain to the audio signal, applying frequency-dependent phase shift to the audio signal, and high-pass filtering the audio signal.

6. The audio-tactile signal generator according to claim 1, characterized in that The mixer includes an audio processor coupled to an adder, and wherein an audio processor input is connected to the audio input, an audio processor output is connected to a first input of the adder, and a haptic signal input is connected to a second input of the adder, wherein the audio processor includes at least one of the following: a delay adjuster, a gain adjuster, a phase adjuster.

7. The audio-tactile signal generator according to claim 6, characterized in that At least two of the delay adjuster, the gain adjuster, and the phase adjuster are arranged in series and coupled between an audio processor input and an audio processor output.

8. The audio-tactile signal generator according to claim 6, characterized in that The controller includes a state parameter calculator coupled to an amplifier sensor input and a tactile sensor input and having a state parameter calculator output, wherein the state parameter calculator is configured to determine an amplifier state based on an amplifier sensor signal received on the amplifier sensor input and to determine a tactile state based on a tactile sensor signal received on the tactile sensor input; an amplitude-frequency detector coupled to the audio input and the tactile input and having an amplitude-frequency detector output; at least one of a delay controller, a gain controller, and a phase controller; wherein the delay controller comprises a first delay controller input, a second delay controller input and a delay controller output, the first delay controller input being coupled to the state parameter calculator output, the second delay controller input being coupled to the amplitude-frequency detector output, and the delay controller output being coupled to the delay adjuster; The gain controller includes a first gain controller input, a second gain controller input, and a gain controller output, wherein the first gain controller input is coupled to the state parameter calculator output, the second gain controller input is coupled to the amplitude-frequency detector output, and the gain controller output is coupled to the gain adjuster; The phase controller includes a first phase controller input, a second phase controller input, and a phase controller output, wherein the first phase controller input is connected to the state parameter calculator output, the second phase controller input is connected to the amplitude-frequency detector output, and the phase controller output is connected to the delay adjuster.

9. A tactile system, characterized in that An audio-tactile signal generator comprising the audio-tactile signal generator of any one of claims 1 to 8 and an amplifier having an input coupled to the output of the mixer and an output configured to be coupled to a tactile actuator.

10. A human-machine interface, characterized in that: Comprising the haptic system of claim 9.

11. A method for generating an audio-tactile signal for a tactile system, characterized in that: The haptic system comprises the haptic system of claim 9, and the method comprises: receiving audio signals; receiving tactile signals; processing the audio signal depending on at least one of a characteristic of the haptic signal, an amplifier state, and a haptic actuator state; and mixing the processed audio signal and the haptic signal; The method further comprises: receiving sensor signals; determining the amplifier state based on the sensor signal; The processing of the audio signal depending on at least one of a characteristic of the haptic signal, the amplifier state, and the haptic actuator state comprises: The audio signal is processed depending on at least one of a characteristic of the haptic signal and a state of the haptic actuator, and the amplifier state.

12. The method according to claim 11, characterized in that The method further comprises: The audio signal is processed depending on a characteristic of the audio signal.

13. An audio-tactile signal generator, characterized in that: include: an audio input configured to receive an audio signal; a tactile input configured to receive a tactile signal; a controller comprising a first controller input coupled to the tactile input; an audio processor coupled to the audio input and the controller output; a summer coupled to the output of the audio processor and the haptic input and configured to combine the haptic signal and the processed audio signal; wherein the controller is configured to control the audio processor to process the audio signal depending on characteristics of the haptic signal; And the audio-haptic signal generator is configured to output a combined haptic signal and a processed audio signal.

14. The audio-tactile signal generator according to claim 13, wherein: The controller also includes an amplifier sensor input and a state parameter calculator, wherein the amplifier sensor input is configured to be coupled to the amplifier, and the state parameter calculator is coupled to the amplifier sensor input, wherein the controller is further configured to determine an amplifier state, and the audio processor is configured to process the audio signal depending on the amplifier state.

15. The audio-tactile signal generator according to claim 13, wherein: The controller also includes a tactile actuator sensor input and a state parameter calculator, wherein the tactile actuator sensor input is configured to be connected to the tactile actuator and the state parameter calculator is connected to the tactile actuator sensor input, wherein the controller is further configured to determine an amplifier state and the audio processor is configured to process the audio signal depending on the amplifier state.

16. The audio-tactile signal generator according to claim 13, characterized in that The controller further comprises a second controller input coupled to the audio input, and wherein the controller is additionally configured to control the audio processor to process the audio signal depending on characteristics of the audio signal.

17. The audio-tactile signal generator according to claim 13, characterized in that The audio processor includes at least one of the following: a delay adjuster, a gain adjuster, and a phase adjuster.

Images