Method and apparatus for outputting haptic signal to haptic transducer

Abstract

Embodiments described herein relate to methods and apparatus for outputting a haptic signal to a haptic transducer. A method for triggering a haptic signal to be output to a haptic converter, comprising: receiving an audio signal for output through an audio output converter; determining whether the audio signal contains a haptic trigger based on an indication of a rate of change of an amplitude of the audio signal; and, in response to determining that the audio signal includes a haptic trigger, triggering the haptic signal to be output to the haptic transducer.

Description

[0001] This application is a divisional application of the invention patent application with the application date of April 4, 2019, application number 201980023336.0, and name “Method and device for outputting tactile signals to a tactile transducer”. Technical Field

[0002] Embodiments described herein relate to methods and apparatus for outputting haptic signals to a haptic transducer based on received audio signals that are output through a speaker. Background Art

[0003] For some applications, it may be desirable to generate haptic effects on mobile devices and other devices, such as user interfaces for smart home devices or vehicles. In some examples, it may be desirable to synchronize these haptic effects with specific audio events to enhance the user experience. In some cases (e.g., where audio source material is predefined in the system (e.g., a specific song of a known type)), it is conceivable that a series of haptic events designed to be output along with a song can be stored and replayed independently of the known audio.

[0004] However, there may be examples of audio playback that does not have predictable or predefined properties. For example, in a gaming application, the rendered audio output may include a combination of stored elements (e.g., background music) and other sounds (perhaps dependent on the gameplay). The exact timing and output of the latter category of sounds may depend on the user's actions in the game. Due to such user control, the output of a stored series of haptic events may be inappropriate. Summary of the invention

[0005] According to some embodiments, a method for triggering a tactile signal is provided, the tactile signal being output to a tactile transducer. The method comprises: receiving an audio signal for output through an audio output transducer; determining whether the audio signal contains a tactile trigger based on an indication of a rate of change of an amplitude of the audio signal, and in response to determining that the audio signal contains a tactile trigger, triggering the tactile signal to be output to the tactile transducer.

[0006] In some embodiments, the determining step includes: generating a first weighted sum representation of the audio signal using a first number of samples of the modified version of the audio signal; generating a second weighted sum representation of the audio signal using a second number of samples of the modified version of the audio signal, wherein the second number of samples is greater than the first number of samples; and comparing the first weighted sum representation to the second weighted sum representation. This comparison can provide an indication of the rate of change of the amplitude of the audio signal.

[0007] In some embodiments, the comparing step includes determining that the audio signal includes a haptic trigger when the first weighted sum representation exceeds the second weighted sum representation. In some embodiments, the comparing step includes determining that the audio signal includes a haptic trigger when the first weighted sum representation exceeds the second weighted sum representation plus a predetermined offset.

[0008] In some embodiments, the method further includes adjusting a first set of weighting coefficients used to generate the first weighted sum representation, and / or adjusting a second set of weighting coefficients used to generate the second weighted sum representation based on desired tactile trigger characteristics.

[0009] In some embodiments, the method further includes adjusting the first number of samples and / or the second number of samples based on desired tactile trigger characteristics.

[0010] In some embodiments, the method further includes determining whether the audio signal includes a haptic trigger based on a rate of change of an amplitude of the audio signal.

[0011] According to some embodiments, a method is provided, comprising: receiving an audio signal for output through an audio output transducer; determining a tactile signal based on the audio signal and at least one criterion, the tactile signal for output to the tactile transducer; and dynamically adjusting the at least one criterion based on a parameter, wherein the parameter comprises one or more of: information about the content of media playback played back together with the audio signal, information about the content of the audio signal, information about historical tactile events associated with the audio signal, and a battery charge status of the device.

[0012] In some embodiments, the method also includes: in response to the audio signal satisfying a first criterion among the at least one criterion, determining that the audio signal includes a tactile trigger; and, in response to determining that the audio signal includes the tactile trigger, outputting the tactile signal to the transducer.

[0013] The first criterion may include a threshold rate of change of the audio signal.

[0014] In some embodiments, the method also includes determining the tactile signal based on the tactile trigger and a second criterion of the at least one criterion.

[0015] In some embodiments, the method also includes generating the tactile signal by the tactile trigger based on a third criterion of the at least one criterion.

[0016] The third criterion may include one or more of the following: a maximum amplitude, an allowable dynamic range, and an allowable frequency range.

[0017] In some embodiments, the adjusting step includes adjusting the at least one criterion to reduce the repetition rate of tactile signal events derived from the audio signal in response to the parameter indicating that future repetition rates of tactile signal events should be reduced.

[0018] In some embodiments, the parameter indicates whether the audio signal includes speech, and the adjusting step includes: in response to the parameter indicating that the audio signal includes speech, adjusting the at least one criterion to reduce the repetition rate of haptic signal events.

[0019] In some embodiments, the parameter includes a battery charge state of the device, and the adjusting step includes adjusting the at least one criterion to reduce a repetition rate of tactile signal events in response to the parameter indicating that the battery charge state is below a threshold level.

[0020] In some embodiments, the parameters include information about the content of media playback played back together with the audio signal; and the adjusting step includes adjusting the at least one criterion to select the tactile signal based on the content of the media playback.

[0021] The media playback may include video data.

[0022] In some embodiments, the parameters include information about historical tactile events associated with the audio signal; and the adjusting step includes adjusting the at least one criterion to optimize expected future output of repeated tactile events.

[0023] According to some embodiments, a method for generating a tactile signal is provided, the tactile signal being output to a tactile transducer. The method comprises: receiving an audio signal, the audio signal being output through an audio output transducer; determining whether the audio signal contains a tactile trigger; in response to determining that the audio signal contains the tactile trigger, mapping energy at frequencies outside an expected frequency range in the tactile trigger to frequencies within the expected frequency range in the tactile signal by undersampling data in the audio signal containing the tactile trigger, so that energy at frequencies above the expected frequency range in the tactile trigger is converted to frequencies within the expected frequency range to generate the tactile signal.

[0024] In some embodiments, the method also includes adjusting a dynamic range of the tactile trigger to generate the tactile signal.

[0025] In some embodiments, the mapping step further includes distorting audio signal energy in the haptic trigger at frequencies below the expected frequency range to generate corresponding harmonic energy within the expected frequency range in the haptic signal.

[0026] According to some embodiments, a processor for triggering a tactile signal is provided, the tactile signal being output to a tactile transducer. The processor includes: an input configured to receive an audio signal for output through an audio output transducer; a tactile trigger detection block configured to determine whether the audio signal contains a tactile trigger based on an indication of a rate of change of an amplitude of the audio signal; and a tactile signal generator configured to trigger the tactile signal to be output to the tactile sensor in response to determining that the audio signal contains a tactile trigger.

[0027] According to some embodiments, a processor is provided for outputting a first tactile signal to a tactile transducer in a device. The processor includes: an input configured to receive an audio signal for output through an audio output transducer; a tactile signal determination block configured to determine a tactile signal based on the audio signal and at least one criterion, the tactile signal for output to the tactile transducer, wherein the processor is configured to dynamically adjust at least one criterion based on a parameter, wherein the parameter includes one or more of the following items: information about the content of media playback played back together with the audio signal, information about the content of the audio signal, information about historical tactile events associated with the audio signal, and a battery charge status of the device.

[0028] According to some embodiments, a processor for generating a tactile signal is provided, the tactile signal being output to a tactile transducer. The processor includes: an input configured to receive an audio signal, the audio signal being output through an audio output transducer; a tactile trigger detection block configured to determine whether the audio signal contains a tactile trigger; and a tactile signal generator configured to: in response to determining that the audio signal contains the tactile trigger, map energy at frequencies outside an expected frequency range in the tactile trigger to frequencies within the expected frequency range in the tactile signal by undersampling data in the audio signal containing the tactile trigger, so that energy at frequencies in the tactile trigger that are higher than the expected frequency range is converted to frequencies within the expected frequency range to generate the tactile signal.

[0029] According to some embodiments, an electronic device including a processor is provided, the processor being configured to trigger a tactile signal to be output to a tactile transducer. The processor includes: an input configured to receive an audio signal for output through an audio output transducer; a tactile trigger detection block configured to determine whether the audio signal contains a tactile trigger based on an indication of a rate of change of an amplitude of the audio signal; and a tactile signal generator configured to trigger the tactile signal to be output to the tactile transducer in response to determining that the audio signal contains a tactile trigger.

[0030] According to some embodiments, an electronic device including a processor is provided, wherein the processor is configured to output a first tactile signal to a tactile transducer in the device. The processor includes: an input configured to receive an audio signal for output through an audio output transducer; a tactile signal determination block configured to determine a tactile signal based on the audio signal and at least one criterion, wherein the processor is configured to dynamically adjust the at least one criterion based on a parameter, wherein the parameter includes one or more of the following items: information about the content of the media playback played back together with the audio signal, information about the content of the audio signal, information about historical tactile events associated with the audio signal, and a battery charge status of the device.

[0031] According to some embodiments, an electronic device including a processor is provided, the processor being configured to generate a tactile signal for output to a tactile transducer. The processor includes: an input configured to receive an audio signal for output through an audio output transducer; a tactile trigger detection block configured to determine whether the audio signal contains a tactile trigger; and a tactile signal generator configured to map energy at frequencies outside an expected frequency range in the tactile trigger to frequencies within the expected frequency range in the tactile signal by undersampling the tactile trigger data in response to determining that the audio signal contains the tactile trigger, so that energy at frequencies above the expected frequency range in the tactile trigger is converted to frequencies within the expected frequency range to generate the tactile signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] For a better understanding of the embodiments of the present disclosure, and to show how the same may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings, in which:

[0033] Figure 1 An example schematic diagram is illustrated showing a portion of a device 100 having an audio output transducer and a tactile output transducer;

[0034] Figure 2 An example of a method for triggering a haptic signal H according to some embodiments is shown. OUT In one embodiment of the processor, the tactile signal H OUT is output to a tactile transducer;

[0035] Figure 3 is an example flow chart illustrating a method for determining whether an audio signal includes a haptic trigger according to some embodiments;

[0036] Figure 4a illustrating an example audio signal, a first weighted sum representation of the audio signal, and a second weighted sum representation of the audio signal;

[0037] Figure 4b An example audio signal and a rate of change of the audio signal are illustrated;

[0038] Figure 5 illustrates an example of a processor for triggering a haptic signal that is output to a haptic transducer in a device according to some embodiments;

[0039] Figure 6 An example of a processor including an audio output transducer and a tactile transducer according to some embodiments is illustrated, wherein the processor outputs a tactile signal to the tactile transducer;

[0040] Figure 7 is an example flow chart illustrating a method for triggering a haptic signal, the haptic signal being output to a haptic transducer, according to some embodiments;

[0041] Figure 8 is an example flow chart illustrating a method for outputting a first tactile signal to a tactile transducer in a device according to some embodiments;

[0042] Fig. 9 is an example flow chart illustrating a method for outputting a tactile signal to a tactile transducer according to some embodiments. DETAILED DESCRIPTION

[0043] The following description sets forth example embodiments according to the present disclosure. Other example embodiments and implementations will be apparent to those of ordinary skill in the art. In addition, those of ordinary skill in the art will recognize that a variety of equivalent technologies may be used in place of or in conjunction with the embodiments discussed below, and all such equivalents are considered to be encompassed by the present disclosure.

[0044] Figure 11 is an example schematic diagram showing a portion of a device 100 with corresponding audio output transducers 102 and tactile output transducers 104. Device 100 may include any electronic device, such as: a portable device; a battery-powered device; a computing device; a communication device; a gaming device; a mobile phone; a personal media player; a laptop, tablet or notebook computing device, a smart watch, a virtual reality (VR) device or an augmented reality (AR) device, or a smart home device. For clarity, Figure 1 Elements of the device 100 that are not relevant to the present disclosure are not shown, but those skilled in the art will appreciate that, except for the elements in FIG. Figure 1 Device 100 may include other elements and components in addition to those shown.

[0045] The device 100 includes a processor 106, which may be, for example, an application processor. The processor 106 interfaces with a signal processor 108, which may be, for example, a digital signal processor (DSP). The signal processor 108 interfaces with an audio output amplifier 110, which is configured to output an audio signal to drive the audio output transducer 102. The signal processor 108 also interfaces with a tactile output amplifier 112, which is configured to output a tactile signal to drive the tactile transducer 104.

[0046] Processor 106 executes and provides an operating environment for device 100 to allow software applications (such as games) to be executed by device 100. Such applications typically receive user input. User input may include one or more of the following: touch user input and / or gesture user input that can be detected by a touch-sensitive surface (e.g., a touch screen) (not shown) of the device; dynamic user input (such as rotating or tilting the device) that can be detected by a sensor (such as an accelerometer or gyroscope of the device) (also not shown); and audio user input (such as spoken commands) that can be detected by a sensor (such as a microphone of the device) (also not shown). In response to detecting user input, the application can be operated to generate appropriate output at the device. For example, the application can be operated to update the image displayed on the display (not shown) of the device and to output appropriate audio effects by the audio output transducer 102. The application can also be operated to provide appropriate tactile output by the tactile output transducer 104 in response to detecting user input.

[0047] For example, in a driving game application, a user may interact with the touch screen of the device 100, for example, by touching a graphical representation of an accelerator pedal or brake pedal presented on the touch screen of the device, thereby accelerating or braking a vehicle in the game. This touch is detected by the touch screen and causes an image displayed on the display of the device to be updated by the application to give a visual indication that the vehicle is accelerating or braking. The update of the image displayed on the display may be accompanied by an appropriate audio effect, which is output by the output audio transducer 102, such as an engine revving effect when the accelerator pedal is touched, or a tire squealing effect when the brake pedal is touched.

[0048] To turn the vehicle, the user may rotate the device in a counterclockwise direction to turn left, or in a clockwise direction to turn right. In response to such kinetic input being detected by the device's accelerometer and / or gyroscope, the image displayed on the display is updated by the application, as appropriate, to give a visual indication of the vehicle's movement. In addition, appropriate audio effects, such as tire squealing effects, may be output via the output audio transducer 102.

[0049] Audio user input (such as voice commands) can additionally be used to, for example, perform special functions, such as firing a weapon. When a valid voice command or other audio user input is detected, the image displayed on the display can be updated, if appropriate, to give a visual indication of the effect of the command, and appropriate audio effects, such as a shooting sound effect, can be output through the output audio transducer 102.

[0050] The application may also be configured to provide additional sensory information to the user using the tactile output transducer 104. In the driving game application embodiment discussed above, in addition to updating the image displayed on the display and outputting appropriate audio effects, the device 100 may also provide tactile output in response to user input. For example, the tactile output transducer 104 may be actuated to cause the device 100 to vibrate while updating the image displayed on the display and outputting an audio effect through the output audio transducer 102, thereby providing additional sensory information to the user.

[0051] In such an application, tactile output can be designed into the application at the beginning. In other words, the designer of the application can identify the user input, visual output, and audio output to be associated with the tactile output, and manually program the application to provide tactile output in response to the identified user input or in combination with the identified visual output or the identified audio output. As should be understood, this manual programming is a time-consuming process during the programming stage. Alternatively, some simple processing (e.g., filtering or correction) can be applied to the audio signal, and this processed signal can be used to drive the tactile transducer. However, the disadvantage of this method is that the tactile output reflects the amplitude of the audio output, and loses the desired effect of making the tactile response only highlight or emphasize certain events. In addition, making the tactile circuit continuously active is unfavorable to the battery life in the mobile device.

[0052] Therefore, it may be desirable to be able to selectively scan the audio signal for relevant events and derive appropriate tactile outputs. In some applications (such as gaming applications), the user's experience of events (such as shooting on the screen, or, for example, other "loud" virtual events, or events with specific characteristics) can be enhanced by tactile responses. Therefore, these events can be considered "tactile triggers". Therefore, in some embodiments, tactile triggers are characterized by portions of the audio signal where large energy changes or high-speed changes in energy occur.

[0053] Figure 2 An example of a method for triggering a haptic signal H according to some embodiments is shown. OUT The processor 200, the tactile signal H OUT is output to the tactile transducer 201. In some embodiments, the tactile signal H OUT The signal may be driven to the tactile transducer 201 after being amplified by the tactile amplifier 208 .

[0054] The processor 200 comprises an input 202 for receiving an audio signal A IN , the audio signal A IN Used to output through the audio output transducer 203. Audio signal A IN The audio signal may be driven through an audio output transducer 203 after being amplified by an audio amplifier 207. The output transducer 203 may be part of the same device as the processor 200, or may be an external transducer on a separate device, such as a speaker or a headset. Thus, a device including the processor 200 may include an output suitable for connection to such an external transducer.

[0055] The processor 200 further includes a tactile trigger detection block 204. The tactile trigger detection block 204 is configured to receive an audio signal A IN , and based on the audio signal AIN The rate of change of the amplitude of the audio signal A is determined by IN Whether to include a haptic trigger. A haptic trigger can be considered a portion of an audio signal that has some characteristic that indicates it is suitable for output along with a haptic signal.

[0056] For example, if the rate of change of the amplitude of the audio signal is above a first threshold, such a rate of change may indicate that the amplitude of the audio signal A is IN An audio event occurs in the video, which can be the basis for a tactile event, thereby highlighting or emphasizing the audio event to the user.

[0057] Then, the tactile trigger detection block 204 may indicate to the tactile signal generator 205 that the audio signal A IN In response to receiving an indication that the audio signal contains a tactile trigger, the tactile signal generator 205 may trigger the tactile signal H OUT , the tactile signal H OUT is output to the tactile transducer 201 .

[0058] In some embodiments, the indication includes an indication of the type of audio event that occurred, for example, the indication may include information indicating that the audio signal includes a gunshot sound. Thus, the tactile signal generator 205 may generate an appropriate tactile signal H based on the indicated type of audio event in the audio signal that caused the tactile trigger, for example, by selecting a predefined tactile signal from the memory 209. OUT .

[0059] In some embodiments, the indication may indicate that the audio signal A IN Therefore, the tactile signal generator 205 can be based on the audio signal A IN The rate of change of the amplitude is used to generate the appropriate tactile signal H OUT For example, if the audio signal A IN The rate of change of the amplitude of is low, for example, lower than a second threshold (the second threshold is higher than the first threshold), the tactile signal generator 205 may generate a first type of tactile signal H OUT For example, the first type of haptic signal may include a beep or piecewise linear envelope (PWLE) type waveform, which are examples of simple and efficient methods of storing haptic waveforms.

[0060] However, if the audio signal A IN is high (eg, higher than a second threshold), the tactile signal generator 205 may generate a second type of tactile signal H OUT The second type of tactile signal H OUTThe second type of tactile signal may include a waveform that is shorter and more impulsive than the first type of tactile signal. For example, the second type of tactile signal may include a "more aggressive" or "clicky" type waveform that is stored in the device (or memory) as a waveform file composed of sampled discrete data.

[0061] In some embodiments, the indication may include an audio signal A IN In other words, the tactile trigger detection block 204 may output an audio signal A to the tactile signal generator in response to detecting that the audio signal contains a tactile trigger. IN In this embodiment, the tactile signal generator can generate a tactile signal by processing an audio signal A containing a tactile trigger. IN To generate a tactile signal. The applied processing can be as later referred to Figure 6 as described.

[0062] In other embodiments, the tactile signal generator 205 may receive an audio signal A IN As a separate input from the indication. In this embodiment, the indication can indicate to the tactile signal generator 205 when the audio signal A IN Contains tactile triggers, so when the audio signal A IN Generate tactile signal H OUT .

[0063] In some embodiments, in particular from the audio signal A IN Generate tactile signal H OUT In the embodiment, the audio signal A IN The haptic signal may be output to the audio amplifier 207 after being delayed by the delay block 206. In particular, the delay block 206 may be configured to introduce a delay into the path between the input 202 and the audio amplifier 207, which is the same as the delay introduced by the processing in the haptic signal generator 205 between the input 202 and the haptic amplifier 208.

[0064] Figure 3 Methods for determining whether an audio signal includes a haptic trigger according to some embodiments are illustrated. For example, Figure 3 The method exemplified in Figure 2 To illustrate one embodiment of this method, refer to Figure 4a An example audio signal 400 of FIG.

[0065] In step 301, haptic trigger detection block 204 generates a first weighted sum representation 402 of modified version 401 of audio signal 400 using a first number of samples of modified version 401 of audio signal 400. First weighted sum representation 402 provides a fluctuating representation of energy in modified version 401 of audio signal 400.

[0066] In step 302 , haptic trigger detection block 204 generates a second weighted sum representation 404 of modified version 401 of audio signal 400 using a second number of samples of modified version 401 of audio signal 400 , wherein the second number of samples is greater than the first number of samples used in step 301 .

[0067] When more samples are used to generate the weighted sum representation of the signal, the representation is less responsive to the presence of peaks and valleys in the audio signal. Thus, the second weighted sum representation 404 is less responsive to peaks and valleys in the modified version 401 of the audio signal 400 than the first weighted sum representation 402.

[0068] In step 303, haptic trigger detection block 204 may compare first weighted sum representation 402 to second weighted sum representation 404. This comparison may indicate a rate of change of the amplitude of audio signal 400.

[0069] In step 304 , haptic trigger detection block 204 may determine whether audio signal 400 contains a haptic trigger based on the comparison made in step 303 .

[0070] For example, when the first weighted sum representation 402 exceeds the second weighted sum representation 404 (eg Figure 4a ), the tactile trigger detection block 204 may determine that the audio signal 400 includes a tactile trigger.

[0071] In some embodiments, haptic trigger detection block 204 may determine that the audio signal contains a haptic trigger when first weighted sum representation 402 exceeds second weighted sum representation 404 plus a predetermined offset.

[0072] It should be understood that during some applications, the processor can adjust the sensitivity of the tactile output. For example, during some applications, the processor 200 can adjust the first set of weighting coefficients (the first set of weighting coefficients can include one or more coefficients) used to generate the first weighted sum representation and / or adjust the second set of weighting coefficients (the second set of weighting coefficients can include one or more coefficients) used to generate the second weighted sum representation based on the desired tactile trigger characteristics. In other words, the weighted sum can be changed so that the rate of change of the amplitude of the audio signal is required to be higher or lower to cause a tactile trigger.

[0073] Adjustment of the first weighting coefficient or the second weighting coefficient may also adjust the frequency response, which is contained in the corresponding weighted sum representation. For example, the second weighted sum may be configured to filter out more high frequencies than the first weighted sum; or, to create a high pass filtering effect so that the second weighted sum ignores any bass content. In this way, comparison of the weighted sums may search for a haptic trigger within a desired frequency range.

[0074] In some embodiments, the haptic trigger detection block can determine whether the audio signal no longer contains a haptic trigger based on an indication of a rate of change of the amplitude of the audio signal. In some embodiments, a comparison between first weighted sum representation 402 and second weighted sum representation 404 can determine the length of the haptic signal or the amplitude of the haptic signal. For example, when the difference between first weighted sum representation 402 and second weighted sum representation 404 is greater, the haptic signal is longer or the amplitude of the haptic signal is higher.

[0075] In response to determining that the audio signal no longer contains a haptic trigger, the haptic trigger detection block can indicate this condition to haptic signal generator 205 .

[0076] In some embodiments, when first weighted sum representation 402 drops back below second weighted sum representation 404 , the haptic trigger detection block can determine that the audio signal no longer contains a haptic trigger.

[0077] In some embodiments, haptic trigger detection block 204 can determine that the audio signal no longer contains a haptic trigger when first weighted sum representation 402 drops a predetermined offset below second weighted sum representation 404. In this embodiment, some hysteresis can be used.

[0078] It should be understood that in some embodiments, multiple different weighted and represented and associated offsets can be used to detect different types of haptic triggers corresponding to different types of audio events. A haptic signal can then be generated based on which type of haptic trigger is present.

[0079] In order to trigger the haptic signal output to the haptic transducer so that the haptic signal is synchronized with the audio signal containing the haptic trigger, the processing used to generate the haptic signal may have time constraints. This factor can effectively limit the range and functionality of decision points that can be used to register a haptic trigger based on changes in energy levels in the audio signal.

[0080] Figure 4b An example is illustrated in which the rate of change of an audio signal is used to determine whether the audio signal contains a haptic trigger according to some embodiments.

[0081] In this embodiment, an audio signal 408 and a rate of change 410 of the audio signal are illustrated. When the rate of change 410 of the audio signal is above a predetermined threshold 412, a haptic trigger may be detected. This detection is illustrated by a marker 414.

[0082] Figure 5 An example of a method for triggering a haptic signal H according to some embodiments is shown. OUT The processor 500, the tactile signal H OUT is output to the tactile transducer 201 in the device. Figure 2 Features similar to those illustrated in FIG. 1 have been given the same reference numerals. In some embodiments, the tactile signal H OUT The tactile signal may be driven through the tactile transducer 201 after being amplified by the tactile amplifier 208 .

[0083] The processor 500 comprises an input 501 for receiving an audio signal A IN , for output through the audio output transducer 203. Audio signal A IN The audio signal may be driven through the audio output transducer 203 after being amplified by the audio amplifier 207 .

[0084] Processor 500 includes a tactile signal determination block 502. The tactile signal determination block can be configured to determine a tactile signal based on the audio signal and at least one criterion for output to the transducer. In some embodiments, tactile signal determination block 502 includes: Figure 2 In these embodiments, in response to the audio signal satisfying a first criterion in the at least one criterion, determining that the audio signal includes a tactile trigger, and in response to determining that the audio signal includes a tactile trigger, outputting a tactile signal to the transducer. Figure 2 As described, the first criterion may thus comprise a threshold rate of change of the amplitude of the audio signal, which threshold rate of change may be represented by a comparison between a first weighted sum representation of the audio signal and a second weighted sum representation of the audio signal.

[0085] In this embodiment, the at least one criterion may include a second criterion. For example, the second criterion may include a second threshold as described above. In other words, if the audio signal A IN is high (eg, higher than a second threshold), the tactile signal generator 205 may generate a second type of tactile signal H OUT The second type of tactile signal H OUTThe second criterion may include a waveform that is shorter and more pulsating than the first type of tactile signal. For example, the second type of tactile signal may include a click waveform. Alternatively, the second criterion may include a relationship between a value of a rate of change of the amplitude of the audio signal and the amplitude of the determined tactile signal. In other words, in some embodiments, when the rate of change is higher, the amplitude of the generated tactile signal is higher.

[0086] In some embodiments, a haptic signal may be generated from a haptic trigger based on a third criterion in the at least one criterion by the haptic signal determination block 502. For example, the third criterion may include one or more of the following: a maximum amplitude of the haptic signal, an allowable dynamic range of the haptic signal, and an allowable frequency range of the haptic signal. Thus, a haptic signal may be generated from an audio signal such that the haptic signal satisfies the third criterion.

[0087] It should be understood that other methods may be utilized to generate a haptic signal from an audio signal that may use different criteria to determine whether a haptic signal will be output and what the haptic signal will be. Figure 2 The embodiment illustrated in can be considered as a fast-reacting decision phase, which can therefore provide a haptic signal quickly to maintain synchronization with the audio signal.

[0088] The processor 500 may then be configured to dynamically adjust at least one criterion based on a parameter, wherein the parameter includes one or more of the following: information about the content of the media playback played back together with the audio signal, information about the content of the audio signal, information about historical tactile events associated with the audio signal, and a battery charge state of the device. Thus, this process of dynamically adjusting the criterion based on the parameter determination may be performed over a longer period of time than determining the tactile signal, thereby allowing for greater flexibility in the type of information that may be used to determine the tactile signal.

[0089] In other words, the haptic signal determination block 502 can include processing with lower latency than the processing required to dynamically adjust at least one criterion. Therefore, this higher latency processing can be used to: a) apply more processing power to determining the haptic signal, and b) employ algorithms and techniques that are more suitable for larger audio data sets. By adjusting at least one criterion, the higher latency processing can effectively train or adapt the lower latency processing without affecting the latency of the output haptic signal.

[0090] The criteria described above may be adjusted by the processor to reduce the repetition rate of haptic signal events derived from the audio signal. For example, in some cases, the parameters may indicate that the repetition rate of haptic signal events that occur is too high (e.g., the user may have indicated through user input that the haptic events are too frequent) and that future repetition rates should be reduced.

[0091] For example, where the parameter includes a threshold rate of change of the amplitude of the audio signal, the adjustment may include adjusting the number of samples used for the first weighted and / or second weighted and representation of the audio signal. Alternatively, the adjustment may include increasing or decreasing the offset to reduce or increase the frequency at which the tactile trigger is detected.

[0092] In some embodiments, the processor includes a speech detection block 503, which is configured to determine whether the audio signal contains speech. Therefore, the parameter can include information about the content of the audio signal, which indicates whether the audio signal contains speech. It should be understood that in some embodiments, the processor 500 can receive an indication of whether the audio signal contains speech from an external processor.

[0093] In these embodiments, the processor can be configured to adjust at least one criterion to reduce the repetition rate of haptic signal events in response to a parameter indicating that the audio signal contains speech. This adjustment can be configured to prevent plosive consonants (such as "p") in the speech of the audio signal from causing an unwanted haptic response. In some embodiments, at least one criterion can be adjusted to turn off generation of any haptic signal in response to the speech indicated by the parameter.

[0094] In some embodiments, the parameter includes a battery charge state of the device. In these embodiments, the processor can adjust at least one criterion to reduce the repetition rate of tactile signal events in response to the parameter indicating that the battery state is below a threshold level. In other words, if the battery of the device is low, the tactile response can be reduced to save battery power for applications that are considered more important to the operation of the device. For example, in response to the parameter indicating that the battery of the device is below a threshold level, at least one criterion can be adjusted to reduce one or more of the following items: the intensity, length, or time density of the tactile signal output to the tactile transducer. Therefore, such adjustment of at least one criterion can provide more sophisticated management of the trade-off between low battery situations and the quality of user experience.

[0095] In some embodiments, the parameters include information about the content of the media playback played back together with the audio signal. For example, the audio signal can be played together with some video data, such as for a game application. In these embodiments, the processor can be configured to adjust at least one criterion to select a haptic signal based on the content of the media playback (e.g., the program content of the media playback). In other words, if the game content is a military-based game, the criteria can be adjusted to distinguish between shooting represented by an "on-screen" image (e.g., a muzzle flash) in the video data and "off-screen" shooting that is not represented by an image in the video data. Therefore, the criteria for generating a haptic signal can be adjusted to generate different haptic signals for two situations, even if the haptic triggers in the two situations may be very similar.

[0096] In some embodiments, the parameters include information about historical haptic events associated with the audio signal. For example, haptic history block 504 can store a data history about haptic signals output during a particular audio stream. Haptic history block 504 can include a process configured to, for example, find repeated haptic signals output during the audio signal. Haptic history block 504 can then output a parameter to adjust at least one criterion to optimize any expected future output of repeated haptic signals.

[0097] In some embodiments, it may be desirable that the audio signal A IN Time synchronization with some other media content (e.g., video data). In the case of adding a haptic signal derived from an audio signal, it may be necessary to control a third synchronization variable. The haptic signal may be delayed, even slightly delayed, relative to the input audio of the system. However, if the audio signal can be arranged to arrive at the processor before other media content, the output audio and haptic signal from the processor can be aligned with the other media content.

[0098] Figure 6 The processor 600 includes an audio output transducer 203 and a tactile transducer 201, wherein the processor 600 outputs a tactile signal H to the tactile transducer 201. OUT .and Figure 2 Features similar to those illustrated in FIG. 1 have been given the same reference numerals. In some embodiments, the tactile signal H OUT The tactile signal may be driven through the tactile transducer 201 after being amplified by the tactile amplifier 208 .

[0099] The processor 600 comprises an input 601 configured to receive an audio signal for output via an audio output transducer 203 (eg, a speaker).

[0100] Processor 600 includes a tactile trigger detection block 602 configured to determine whether an audio signal contains a tactile trigger. The tactile trigger detection block 602 may be configured to Figure 2 Alternatively, other methods of detecting a tactile trigger in an audio signal may be performed by tactile trigger detection block 602 .

[0101] The haptic trigger detection block 204 may then indicate to the haptic signal generator 205 that the audio signal A IN In some embodiments, the indication includes an audio signal A IN In other words, the tactile trigger detection block 204 can output an audio signal containing a tactile trigger to the tactile signal generator 205. In some embodiments, the tactile signal generator includes a IN input, and the indication includes information indicating the location of the tactile trigger within the audio signal.

[0102] The tactile signal generator 205 receives the audio signal A IN Indication of a tactile trigger is included, mapping energy at frequencies outside the expected frequency range in a received tactile trigger (received from a tactile trigger detection block or received as part of an audio signal) to frequencies within the expected frequency range to generate a tactile signal. Specifically, the tactile signal generator maps the frequencies by undersampling data in the audio signal including the tactile trigger so that energy at frequencies in the tactile trigger that are higher than the expected frequency range is converted to frequencies within the expected frequency range, thereby generating a tactile signal.

[0103] The expected frequency range may include frequencies that, when input into tactile transducer 201, result in a tactile event that may be felt by the user. Mapping higher frequencies by undersampling the tactile trigger means that, in a tactile event resulting from the tactile signal, the user will feel the energy in the tactile trigger. By undersampling the data in the audio signal containing the tactile trigger, the tactile signal generator may thus create a tactile signal that represents the audio signal containing the tactile trigger, but that is within a frequency range suitable for reproduction by tactile transducer 201.

[0104] The tactile signal generator 205 may be configured to output a tactile signal to a tactile transducer.

[0105] In some embodiments, the haptic signal generator can be configured to adjust the dynamic range of the haptic trigger to generate the haptic signal. In other words, although a typical 16-bit audio signal can have a dynamic range of up to and exceeding 90 dB, the user's sensitivity to distinguishing haptic events caused by the haptic signal is much smaller, such as 20 dB to 30 dB. Subsequently, if low-level audio material is found to form part of the audio signal during the haptic trigger, the dynamic range of the haptic trigger can be adjusted to suit the user's sensitivity to the generated haptic events.

[0106] In some embodiments, the tactile signal generator can be further configured to distort the audio signal energy at frequencies below the expected frequency range in the tactile trigger so that the corresponding energy is generated within the expected frequency range in the tactile signal. Similarly, the user will not feel the low-frequency content below the expected frequency range. Therefore, the tactile signal generator can correct or otherwise distort the low frequency of the tactile trigger to intentionally produce high-level harmonics within the expected frequency band so that the user can feel the tactile event.

[0107] It should be understood that the processors 200, 500 and 600 may be implemented as devices (such as Figure 1 At least a portion of the signal processor 108 in the device 100 illustrated in FIG.

[0108] Figure 7 A method for triggering a tactile signal is illustrated, the tactile signal being output to a tactile transducer. The method may be executed by a processor, for example Figure 2 The processor 200 or any suitable circuitry in.

[0109] At step 701 , the method includes receiving an audio signal for output via an audio output transducer.

[0110] In step 702 , the method includes determining whether an audio signal includes a haptic trigger based on an indication of a rate of change of an amplitude of the audio signal.

[0111] At step 703 , the method includes triggering a haptic signal in response to determining that the audio signal includes a haptic trigger, the haptic signal being output to a haptic transducer.

[0112] Figure 8 A method for outputting a first tactile signal to a tactile transducer in a device is illustrated. The method may be executed by a processor, for example Figure 5 The processor 500 or any suitable circuitry in.

[0113] In step 801 , the method includes receiving an audio signal for output via an audio output transducer.

[0114] At step 802 , the method includes determining a haptic signal based on the audio signal and at least one criterion, the haptic signal for output to a haptic transducer.

[0115] In step 803, the method includes dynamically adjusting at least one criterion based on a parameter, wherein the parameter includes one or more of the following items: information about the content of media playback played back together with the audio signal, information about the content of the audio signal, information about historical tactile events associated with the audio signal, and a battery charge status of the device.

[0116] Fig. 9 A method for generating a tactile signal for output to a tactile transducer is illustrated. The method may be executed by a processor, for example Figure 6 The processor 600 or any suitable circuitry in.

[0117] In step 901 , the method includes receiving an audio signal for output via an audio output transducer.

[0118] In step 902 , the method includes determining whether an audio signal includes a haptic trigger.

[0119] In step 903, the method includes, in response to determining that the audio signal includes a tactile trigger, mapping energy at frequencies outside the expected frequency range in the tactile trigger to frequencies within the expected frequency range in the tactile signal by undersampling data in the audio signal including the tactile trigger, so that energy at frequencies above the expected frequency range in the tactile trigger is converted to frequencies within the expected frequency range in the tactile signal. Therefore, a method and apparatus for outputting a tactile signal to a tactile transducer based on a received audio signal is provided, and the received audio signal is output through a speaker.

[0120] It should be noted that the embodiments mentioned above illustrate rather than limit the present invention, and those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. The word "comprising" does not exclude the presence of elements or steps other than those listed in the claims, "a" or "an" does not exclude a plurality, and a single feature or other unit may implement the functions of several units described in the claims. Any reference numeral or reference mark in the claims should not be interpreted as limiting the scope of the claims. Terms such as amplification or gain include the possibility of applying a scaling factor less than 1 to a signal.

[0121] Of course, it should be understood that various embodiments of analog conditioning circuits as described above, or blocks or portions thereof, may be integrated together with other blocks or portions thereof or with other functionality of a host device on an integrated circuit such as a smart codec.

[0122] Thus, those skilled in the art will recognize that some aspects of the apparatus and methods described above may be embodied as, for example, processor control code located on a non-volatile carrier medium such as a disk, CD-ROM or DVD-ROM, a programmed memory such as a read-only memory (firmware) or on a data carrier such as an optical signal carrier or an electrical signal carrier. For many applications, embodiments of the present invention will be implemented on a DSP (digital signal processor), an ASIC (application specific integrated circuit) or an FPGA (field programmable gate array). Thus, the code may include conventional program code or microcode or, for example, code for setting up or controlling an ASIC or FPGA. The code may also include code for dynamically configuring a reconfigurable apparatus such as a reprogrammable logic gate array. Similarly, the code may include code for a hardware description language such as Verilog TM The code may be distributed among multiple coupled components that communicate with each other, as will be appreciated by those skilled in the art. Where appropriate, the embodiments may also be implemented using code that runs on a field (re)programmable analog array or similar device to configure analog hardware.

[0123] It should be understood, especially by those of ordinary skill in the art who have the benefit of this disclosure, that various operations described herein, especially those associated with the accompanying drawings, may be implemented by other circuits or other hardware components. The order in which each operation of a given method is performed may be changed, and various elements of the systems illustrated herein may be added, reordered, combined, omitted, modified, etc. It is intended that the present disclosure includes all such modifications and changes, and therefore, the above description should be considered illustrative rather than limiting.

[0124] Similarly, although the present disclosure refers to specific embodiments, certain modifications and changes may be made to those embodiments without departing from the scope and coverage of the present disclosure. In addition, any benefits, advantages, or solutions to problems described herein with respect to specific embodiments are not intended to be construed as key, required, or essential features of an element.

[0125] Likewise, other embodiments will be apparent to those of ordinary skill in the art having the benefit of this disclosure, and such embodiments are deemed to be encompassed herein.

Claims

1. A method for outputting a first tactile signal to a tactile transducer in a device, comprising: receiving an audio signal, wherein the audio signal is output through an audio output transducer; determining a tactile signal for output to the tactile transducer based on the audio signal and at least one criterion; as well as The at least one criterion is dynamically adjusted based on a parameter, wherein the parameter includes one or more of: information about content of media played back along with the audio signal, information about content of the audio signal, information about historical tactile events associated with the audio signal, and a battery charge status of the device.

2. The method according to claim 1, further comprising: In response to the audio signal satisfying a first criterion of the at least one criterion, determining that the audio signal includes a haptic trigger, and In response to determining that the audio signal includes the haptic trigger, the haptic signal is output to the transducer. The method of claim 2 , wherein the first criterion comprises a threshold rate of change of the audio signal.

4. The method according to claim 2, further comprising: The haptic signal is determined based on the haptic trigger and a second criterion of the at least one criterion.

5. The method according to claim 2, further comprising: The haptic signal is generated from the haptic trigger based on a third criterion of the at least one criterion.

6. The method of claim 5, wherein the third criterion comprises one or more of: a maximum amplitude; an allowable dynamic range; and an allowable frequency range.

7. The method of claim 1, wherein the adjusting step comprises adjusting the at least one criterion to reduce a repetition rate of haptic signal events derived from the audio signal in response to the parameter indicating that future repetition rates of haptic signal events should be reduced.

8. The method of claim 7, wherein the parameter indicates whether the audio signal contains speech, and wherein the adjusting step comprises: In response to the parameter indicating that the audio signal includes speech, the at least one criterion is adjusted to reduce a repetition rate of haptic signal events.

9. The method of claim 7, wherein the parameter comprises a battery charge state of the device, and wherein the adjusting step comprises: In response to the parameter indicating that the battery state of charge is below a threshold level, the at least one criterion is adjusted to reduce a repetition rate of haptic signal events.

10. The method of claim 1, wherein the parameters include information about content of media playback played back along with the audio signal; and wherein the adjusting step includes adjusting the at least one criterion to select a haptic signal based on the content of the media playback. The method of claim 10 , wherein the media playback comprises video data.

12. The method of claim 1, wherein the parameters include information about historical haptic events associated with the audio signal; and wherein the adjusting step includes adjusting the at least one criterion to optimize an expected future output of repeated haptic events.

13. A processor for outputting a first tactile signal to a tactile transducer in a device, comprising: an input configured to receive an audio signal for output via an audio output transducer; A tactile signal determination block is configured to determine a tactile signal based on the audio signal and at least one criterion, the tactile signal being used to be output to the tactile transducer, wherein the processor is configured to dynamically adjust the at least one criterion based on a parameter, wherein the parameter includes one or more of the following items: information about the content of media playback played back together with the audio signal, information about the content of the audio signal, information about historical tactile events associated with the audio signal, and a battery charge status of the device.

14. The processor of claim 13, wherein the processor is configured to adjust the at least one criterion to reduce a repetition rate of haptic signal events during the audio signal in response to the parameter indicating that future repetition rates of haptic signal events should be reduced.

15. The processor of claim 14, wherein the parameter indicates whether the audio signal contains speech, and wherein the processor is configured to adjust the at least one criterion to reduce a repetition rate of haptic signal events in response to the parameter indicating that the audio signal contains speech.

16. The processor of claim 14, wherein the parameter comprises a battery charge state of the device, and the processor is configured to adjust the at least one criterion to reduce a repetition rate of tactile signal events in response to the parameter indicating that the battery charge state is below a threshold level.

17. The processor of claim 13, wherein the parameters include information about content of the media playback played back together with the audio signal; and wherein the processor is configured to adjust the at least one criterion to select a haptic signal based on the content of the media playback.

Images