Method and apparatus for generating haptic effects from audio content

Abstract

A method and apparatus for generating (23) haptic effects from an audio signal (201) is provided herein. To this end, classification information representative of the type of audio content associated with the audio signal (201) is acquired. And selecting a detection parameter according to the classification information (221). One or more components of the audio signal (201) are detected (22) according to the detection parameter (221). One or more characteristics representative of the detected component are determined (212). A haptic effect is generated from the characteristic (212).

Description

Technical Field

[0001] The present invention generally relates to the field of haptics, and more particularly to generating haptic data representing a haptic effect to be rendered by a haptic device. The present invention also relates to methods and apparatus for processing data representing audio content to generate data representing one or more haptic effects. Background Art

[0002] This section is intended to introduce the reader to various aspects of the art, which may be related to various aspects of at least one exemplary embodiment of the present invention described and / or claimed below. These descriptions are believed to be helpful in providing the reader with background information to facilitate a better understanding of various aspects of the present invention.

[0003] Haptic technology broadly refers to any technology that reproduces the sense of touch in a user interface by applying force, vibration, motion, and other sensations (such as temperature) in addition to visual and audio information content when presenting multimedia to provide information to the end user.

[0004] Haptic feedback covers a wide range of possible stimulation implementation methods, but is mainly divided into haptic haptic technology and kinesthetic haptic technology: Haptic haptic technology (or haptic haptic effect) refers to sensations such as vibration, friction or micro-deformation, while kinesthetic haptic feedback (or kinesthetic haptic effect) refers to the sensation of providing force, which can stimulate mechanical stimulation as well as stimulation related to the position and movement of the body.

[0005] The presentation of tactile feedback or tactile effects is achieved using a tactile device (also called a tactile presentation device), which corresponds to an arrangement of one or more tactile actuators. For example, vibrotactile effects can be achieved by using tactile devices such as eccentric rotating mass (ERM), linear resonant actuators (LRA), and wide bandwidth actuators such as voice coil motors (VCM) or piezoelectric actuators (PZT). Kinesthetic effects can be presented by actuators applying forces that resist limb movement, which are felt only in muscles and tendons, not in the skin. Other examples of tactile devices include resistive force feedback devices, active force feedback devices, and skin indentation devices.

[0006] Although there are currently a large number of haptic devices that can experience haptic effects, the content, such as audio and / or video content, that includes haptic effects to be presented is quite limited. The lack of haptic content limits people's interest in haptic devices and limits the user's experience when consuming content. Summary of the invention

[0007] The following section presents a simplified overview of at least one example embodiment to provide a basic understanding of some aspects of the present invention. This overview is not a broad overview of the embodiments. It is not intended to identify the key or important elements of the embodiments. The following overview merely presents some aspects of at least one example embodiment in a simplified form as a prerequisite for a more detailed description provided elsewhere in this article.

[0008] According to a first aspect of the present invention, there is provided a method for generating a tactile effect, the method comprising:

[0009] - obtaining classification information representing a type of audio content associated with the audio signal;

[0010] - selecting a detection parameter group from the plurality of detection parameter groups according to the classification information using the mapping information, the mapping information mapping each detection parameter group from the plurality of detection parameter groups to an audio content type of the determined list of different types of audio content;

[0011] - detecting at least one component of the audio signal according to the set of detection parameters;

[0012] - determining at least one characteristic representing at least one component; and

[0013] - generating said haptic effect in dependence on at least one characteristic.

[0014] In an embodiment, the method further comprises obtaining a user preference parameter set representing user preferences, and the haptic effect is further generated according to the user preference parameter set.

[0015] In an embodiment, the method further comprises:

[0016] - receiving mode information representing a rendering mode for rendering a haptic effect;

[0017] -Selecting a user preference parameter group from a plurality of user preference parameter groups according to the mode information.

[0018] In another embodiment, the values ​​of the detection parameters of the detection parameter set are based on the user preference parameter set.

[0019] In another embodiment, the at least one component belongs to a component group comprising:

[0020] - a component corresponding to an instant in the time representation of the audio signal; and

[0021] - components corresponding to sub-bands of a frequency band associated with the frequency representation of the audio signal.

[0022] In another embodiment, the frequency representation is obtained by applying a Fast Fourier Transform to the time representation of the audio signal.

[0023] In another embodiment, the detection parameter group includes at least parameters belonging to a parameter group, the parameter group including:

[0024] - represents the parameter determining the amplitude;

[0025] - parameters representing the time window;

[0026] - parameters representing the limits of the frequency interval;

[0027] - a parameter representing a frequency threshold; and

[0028] -Parameter representing the spectrum power threshold

[0029] In another embodiment, the at least one characteristic belongs to a group of characteristics comprising:

[0030] -characteristics representing amplitude values;

[0031] - characteristics representing frequency values;

[0032] -The characteristic of the amplitude power difference between the 2 power spectral density representatives representing the audio signal.

[0033] In an embodiment, the method further comprises obtaining actuator information representative of a type of haptic actuator presenting the haptic effect, the haptic effect being further generated based on the actuator information.

[0034] In another embodiment, the mapping information is stored in memory.

[0035] According to a second aspect of the present invention, there is provided a device for generating a haptic effect, wherein the device comprises a memory associated with at least one processor configured to implement the method according to the first aspect of the present invention.

[0036] According to a third aspect of the present invention, there is provided a computer program product comprising instructions, which, when executed by one or more processors, causes the one or more processors to perform the method according to the first aspect of the present invention.

[0037] According to a fourth aspect of the present invention, there is provided a non-transitory storage medium having program code instructions for executing the method according to the first aspect of the present invention.

[0038] The specific nature of at least one embodiment and other objects, advantages, features and uses of at least one embodiment will become apparent from the following description of examples taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Reference will now be made, by way of example, to the accompanying drawings which show example embodiments of the present invention, and in which:

[0040] Figure 1 is a schematic representation of a communication network for transmitting audio content according to at least one example embodiment;

[0041] Figure 2 According to at least one example embodiment, Figure 1 An example of a scheme for generating a haptic effect of a signal of an audio content;

[0042] Figure 3 is a representative embodiment according to at least one example embodiment Figure 1 An example of a detection scheme for a specific component within an audio signal;

[0043] Figure 4 According to at least one example embodiment, Figure 1 A schematic block diagram of the steps of a method for generating tactile effects from audio content;

[0044] Figure 5 is a schematic block diagram of an example of a system that implements various aspects and example embodiments.

[0045] Similar drawing symbols may be used in different drawings to represent similar components. DETAILED DESCRIPTION

[0046] At least one example embodiment will be described more fully below with reference to the accompanying drawings, in which examples of at least one example embodiment are shown. However, the example embodiments may be implemented in many alternative forms and should not be construed as being limited to the examples described herein. Therefore, it should be understood that this document is not intended to limit the example embodiments to the specific forms disclosed. On the contrary, this document is intended to cover all modifications, equivalents, and alternatives that fall within the spirit and scope of the present invention.

[0047] The example embodiments described below may be combined with each other, or each example embodiment may be implemented alone, for example, as an alternative embodiment.

[0048] At least one of the aspects relates to a method and apparatus for generating tactile data representing one or more tactile effects from audio data of an audio signal representing audio content. The tactile data can be transmitted to a tactile device with tactile rendering capabilities. The tactile device can also be configured to render audio content from the audio signal describing the audio content.

[0049] The haptic device includes, for example, one or more haptic actuators, such as an LRA and / or a VCM.

[0050] The tactile data is obtained by processing the audio data according to the detection parameter set. The detection parameter set is determined according to classification information representing the type of audio content, for example, selected from a plurality of detection parameter sets. Such classification information is, for example, received together with the audio data or determined by processing the audio data. Such processing includes, for example, classifying the audio data using a machine learning method. The detection parameter set may be selected based on mapping information retrieved, received or obtained from a memory storing mapping information. The mapping information maps each of the plurality of detection parameter sets to a different type of audio content.

[0051] Processing the audio data includes, for example, detecting one or more components (eg, temporal and / or specific frequency components) of the audio signal and extracting parameters or characteristics representative of each of the one or more components from the audio signal.

[0052] This processing enables the generation of haptic data from characteristics of the audio data detected or determined according to a set of detection parameters, which are particularly suitable for the type of audio signal. It enables one or more haptic effects to be presented together with the audio content, which are related and consistent with the audio content.

[0053] Figure 1 is a schematic diagram illustrating a communication network 100 for communicating audio content according to at least one example embodiment.

[0054] The network 100 corresponds to, for example, a wide area network (WAN), a local area network (LAN) or a wireless local area network (WLAN), or a combination of one or more of these networks. According to other examples, the network 100 corresponds to a radio broadcast network, a television broadcast network, a cellular network or a satellite network, or any combination thereof.

[0055] The network 100 enables the transmission of audio content from a first device 11 (eg, a transmitter and / or encoder) to a second device 12 (eg, a receiver and / or decoder).

[0056] The audio content transmitted from the first device 11 to the second device 12 may be in any form. For example, the audio content may be transmitted as an analog audio signal or as a digital audio signal. In the following, the term "audio signal" will be used to represent a signal that transmits data representing the audio content.

[0057] Once received by the second device 12, the audio signal may be converted from analog to digital or vice versa, as known to those skilled in the art.

[0058] The audio signal may be represented in time space and / or frequency space. The audio signal may be converted from a time space representation to a frequency space representation, for example using a Fast Fourier Transform (FFT), or inversely converted from a frequency space representation to a time space representation, for example using an inverse FFT, as known to those skilled in the art.

[0059] The audio signal is represented, for example, by a series of samples. The samples in the time-space representation are represented by amplitude values ​​associated with time. The samples in the time-space representation are represented by data representing amplitudes associated with frequencies (or frequency ranges). The audio signal represented by a series of samples, for example, results from a sampling process at a determined sampling rate or frequency (e.g., 44100 Hz).

[0060] According to certain embodiments, the audio signal may be associated with a video signal corresponding to video content. For example, the audio signal may correspond to audio content of a movie, a video game, or virtual reality (VR) content.

[0061] The second device 12 is configured to process the received audio signal, in particular to generate data representing one or more haptic effects from the audio signal (or a portion of the received audio signal). The processing performed by the second device 12 may, for example, include analog / digital conversion, frequency filtering (e.g., low-pass filtering, band-pass filtering), time filtering (e.g., selecting a time window), downsampling, converting time space to frequency space, converting frequency space to time space, etc.

[0062] The following references Figure 5 Example embodiments of the first device 11 and / or the second device 12 are described.

[0063] The second device 12 is communicatively coupled to a third device 13 corresponding to a haptic device. According to a specific and non-limiting example, the third device 13 is also configured to present audio content, and the third device 13 corresponds to headphones, XR headphones, or over-ear headphones, for example. According to another specific and non-limiting example, the third device 13 is configured to present only haptic effects (and no audio content), and the third device 13 corresponds to, for example, a haptic vest, a haptic chair, a palm pad, a mouse, etc.

[0064] Figure 2 is an example of a generation scheme representing one or more haptic effects from audio signal 201 according to at least one embodiment.

[0065] The following embodiments will be described below with reference to an audio signal 201 corresponding to a digital audio signal represented in time space received at an input of the second device 12. Of course, the principles herein are not limited to this example but extendable to any audio signal.

[0066] The audio signal 201 corresponds to an audio signal transmitted by the first device 11 or only a part thereof. A part of the audio signal may correspond to a part associated with a determined frequency band, the audio signal transmitted by the first device 11 being transmitted in a larger frequency spectrum than the frequency spectrum corresponding to the determined frequency band.

[0067] Blocks 21 to 23 represent the general operations / steps of a process / method and the processing units that implement the operations / steps of the process / method. The processing units of blocks 21 to 23 can be implemented as a single component or as separate components of a combination of hardware and software known to those skilled in the art.

[0068] According to a specific and non-limiting example, the data representing the audio signal 201 includes information or a flag representing the presence or absence of tactile data representing a tactile effect to be presented in the data. Such information corresponds, for example, to a specific bit in the header, a value of "0" for the bit indicating that the data of the audio signal 201 does not include any tactile data, and a value of "1" for the bit indicating that the data of the audio signal 201 includes or includes tactile data describing a tactile effect to be presented. According to this example, the operations of blocks 21 to 23 are performed only when the information represents the absence of tactile data (e.g., a bit with a value of "1").

[0069] According to another specific and non-limiting example, classification information representing the type of audio content is associated with the audio signal. The classification information is, for example, contained in the data representing the audio signal 201 or received as metadata together with the data representing the audio signal 201 (e.g., in a channel different from the channel of the audio signal 201).

[0070] The classification information is associated with the audio signal on a time scale, i.e., the classification information may vary over time, since the type of audio content may vary over time. The classification information may, for example, be associated with each time segment of the audio segment, the duration of each time segment being fixed or varying to adapt to the audio content and the type of audio / noise contained in the audio content.

[0071] The classification information corresponds, for example, to an identification value encoded on, for example, 4 or 8 bits. The type of audio content identified by the classification information, for example using the image information, belongs to a determined list of different types, including, for example: voice, music, silence, explosion, dialogue, gun noise, etc.

[0072] Sub-genres may be identified within a genre, e.g., for the genre "Music," sub-genres may include, for example: classical, pop, rock, electronic, jazz, metal, hard rock, etc.

[0073] The classification information is obtained, for example, by speech recognition methods known to those skilled in the art. Such methods can be applied to the audio content itself and, if applicable, to video content associated with the audio content, the video content providing additional information to identify the type of the audio content (e.g., identification of people talking to each other provides information to identify the type of "conversation" or "speech" or "sound", etc.).

[0074] The classification information is obtained, for example, by implementing a machine learning method, a deep learning method, an artificial intelligence (e.g., via a neural network), as is known to those skilled in the art. Examples of audio classification methods are described in "Audio Classification Method Based on Machine Learning" (author Feng Rong et al., published in IEEE, September 21, 2017, DOI: 10.1109 / ICITBS.2019.98) or "Comparative Analysis of Machine Learning Algorithms for Audio Signals Classification" (author Poonam Mahana and Gurbhej Singh, published in IJCSNS (International Journal of Computer Science and Network Security), Vol. 15, No. 6, June 2015).

[0075] The processing of the audio content (and possibly the video content associated therewith) is for example carried out by the first device 11 and the classification information is added to the data stream transmitted to the second device 12 .

[0076] According to a variant of the embodiment, when the second device 12 does not receive the classification information with the audio signal 201 , the determination of the classification information is performed by the second device 12 or by another device communicatively coupled to the second device 12 .

[0077] In the operation of block 21, one or more processes are applied to the data representing the audio signal 201. The one or more processes include, for example:

[0078] Downsampling, for example from a 44100 Hz sampling rate to a 2400, 4800 or 8000 Hz sampling rate; and / or

[0079] Converting the audio signal 201 from the time domain to the frequency domain, for example by using a Fourier transform or a fast Fourier transform; and / or

[0080] Low-pass filtering, for passing the portion of the audio signal 201 having a frequency lower than a determined cut-off frequency (or frequency cut-off); and / or

[0081] Bandpass filtering for passing a portion of the audio signal 201 within a certain frequency range; and / or

[0082] Buffer data using a First In First Out (FIFO) buffer; and / or

[0083] When the second device 12 does not receive the classification information of the audio signal 201 from the first device 11, the classification information is determined.

[0084] In the operation of block 22, data 211 generated by the processing performed in the operation of block 21 is further processed.

[0085] A detection parameter group 221 is determined according to the classification information. For example, according to the classification information, that is, according to the type of the audio content, the detection parameter group 221 is selected from the parameter group list or the parameter group set.

[0086] The detection parameter sets are, for example, stored in a remote storage device communicatively coupled to the second device 12, and a mapping between each set and a corresponding type of audio content is associated with the list, and the mapping information is, for example, recorded in a look-up table (LUT). If applicable, the detection parameter sets with the LUT are, for example, stored in a memory of a device that performs a semantic recognition method or any other method to determine classification information.

[0087] According to this example, the set of detection parameters selected according to the classification information is received by the second device 12 together with the audio content, for example in a channel separate from a channel transmitting data representing the audio content.

[0088] According to another example, the detection parameter sets are stored, for example, in a memory of the second device 12, wherein the mapping information registered in the LUT establishes a relationship between each set and a corresponding type of audio content associated with the list.

[0089] Each detection parameter group comprises, for example, one or more detection parameters, the type and / or value associated with each detection parameter depending on the type of audio content identified by the classification information. The same detection parameter may belong to multiple detection parameter groups of the manifest.

[0090] The data 211 is advantageously processed to detect one or more specific components of the audio signal represented by the data 211 based on detection parameters selected in accordance with the classification information.

[0091] The one or more components detected or determined in the audio signal using the detection parameters correspond to, for example:

[0092] one or more transients, a transient being represented by a sudden change or variation in the audio signal, a transient corresponding to, for example, a portion of the audio signal having an abrupt change in amplitude (e.g., an amplitude peak or a series of amplitude peaks); and / or

[0093] One or more frequency components / portions of the audio signal, each frequency component / portion of the audio signal corresponding to a sub-band of a frequency band associated with the audio signal (e.g., a portion of the audio signal having a frequency below a determined cut-off frequency or having a frequency interval or band belonging to a determined frequency).

[0094] Transients can be detected in a time representation or a frequency representation of the audio signal, whereas frequency components can be detected in a frequency representation (frequency domain) of the audio signal.

[0095] The one or more detection parameters for detecting the one or more components correspond to, for example, one or more of the following parameters:

[0096] represents a parameter determining the magnitude; and / or

[0097] a parameter representing a time window, ie a time period having a determined duration; and / or

[0098] Parameters representing the limits of the frequency interval, i.e. the lower and upper limits of the frequency interval; and / or

[0099] a parameter representing a frequency threshold, i.e. a determined frequency value, also called frequency cutoff or cutoff frequency; and / or

[0100] Parameter representing the spectrum power threshold.

[0101] The following will refer to Figure 3 An exemplary method of detecting or determining one or more components from an audio signal and a set of detection parameters 221 is described.

[0102] One or more characteristics 212 representing each detected component are determined and transmitted to the processing unit of block 23 for use in synthesizing a haptic effect.

[0103] The type of characteristic advantageously depends on the component it represents.

[0104] For example, when the detected component corresponds to an instant, the one or more characteristics 212 may correspond to one or more of the following characteristics or parameters:

[0105] a parameter representing the amplitude, such as the maximum amplitude value detected within a certain time window; and / or

[0106] A parameter representing the difference between 2 amplitude spectral densities.

[0107] When the detected components correspond to frequency components, the one or more characteristics 212 may correspond to one or more of the following characteristics or parameters:

[0108] a parameter representing frequency, such as a maximum frequency value and / or a maximum frequency value within a frequency range associated with a frequency representation of a signal (eg, an envelope of at least a portion of an audio signal); and / or

[0109] Parameters representing detected frequency components, such as parameters representing samples of a frequency sub-portion of the audio signal, which parameters correspond, for example, to coefficients of a Fourier transform or a fast Fourier transform.

[0110] In the operation of block 23 , data 233 representing one or more haptic effects is determined, generated, or synthesized based on the one or more characteristics 212 received from the processing unit of block 22 .

[0111] For example, one or more haptic effects are generated by providing one or more characteristics 212 to one or more haptic effect synthesis models (or rules).

[0112] According to a specific embodiment, values ​​of one or more parameters of a haptic effect synthesis model used to generate data representing a haptic effect depend on user preference parameters 231 representing user preferences (ie, preferences of a user consuming audio content and experiencing haptic effects).

[0113] The user preference parameters are received, for example, from a third device 13 (e.g., a haptic device) that is communicatively coupled or connected to the second device 12, for example, via a wired connection (e.g., USB) or via a wireless connection (e.g., Bluetooth or Wifi). To achieve this goal, the third device 13 (e.g., a haptic device) includes, for example, a human-machine interface (HMI) configured to set one or more parameters. Such an HMI includes, for example, one or more buttons and / or an adjustment wheel.

[0114] According to another embodiment, the user preference parameters are set via an HMI associated with the second device 12, for example, implemented by the second device 12. The HMI corresponds to, for example, a graphical HMI displayed on a display coupled to the second device 12 or included in the second device 12. The user preference parameters are input, for example, using peripheral devices such as a keyboard and / or a mouse, or using a touch interface associated with a display screen.

[0115] The user may, for example, set a value for each of at least a portion of the parameters used by a haptic effect synthesis model implemented to generate a haptic effect via the HMI.

[0116] According to another embodiment, the user can select a presentation mode among a plurality of presentation modes, for example via an HMI associated with the third device 13 (e.g. a tactile device) and / or with the second device 12, the values ​​of the parameters assigned to the model varying between each presentation mode.

[0117] An example of a presentation mode is as follows:

[0118] Default mode, which corresponds to a balanced mode, enhancing the listening of audio content by presenting smooth haptic effects, for example, to increase immersion;

[0119] Entertainment mode, e.g. designed for beat-based music and / or action-filled entertainment content: haptic effects can be used to enhance the role of bass and beats to increase the sense of presence of dynamic music and sound effects;

[0120] Game mode, e.g. designed for game sound effects.

[0121] When a rendering mode is selected, the second device 12 receives information representing, for example, the rendering mode that has been selected. The second device 12 uses this information to retrieve parameter values ​​associated with the rendering mode from a memory to synthesize a haptic effect.

[0122] According to another example, the second device 12 directly receives the value associated with the selected presentation mode, for example from the third device 13 (eg a haptic device).

[0123] According to a variant embodiment, the value of one or more detection parameters for detecting one or more components in the audio signal 201 is based on a user preference parameter. For example, a threshold for detecting specificity in the audio signal is adjusted based on the user preference parameter.

[0124] According to another specific embodiment, actuator information 232 representing the type of haptic actuator used to render the haptic effect is received or obtained by the second device 12, such as by the processing unit of block 23. The actuator information 232 is advantageously used by the processing unit of block 23 to generate a haptic effect according to the actuator information.

[0125] The actuator information 232 is received by, for example, the third device 13 (eg, a haptic device). The actuator information 232 includes, for example, data representing the haptic rendering capability of the third device 13 (eg, a haptic device), such as:

[0126] Data representing the number of haptic actuators included in a third device 13 (eg, a haptic device); and / or

[0127] Data representing the spatial distribution of haptic actuators of a third device 13 (eg, a haptic device); and / or

[0128] Data representing the type of actuator and / or the type of third device 13 (eg, haptic device) (eg, LRA, VCM, ERM, PZT).

[0129] For example, when the third device 13 (e.g., a haptic device) is connected to the second device 12, for example via USB, the actuator information 232 (and the user preference parameters 231 when received from the third device 13 (e.g., a haptic device)) is automatically received from the third device 13 (e.g., a haptic device).

[0130] When the third device 13 (e.g., haptic device) is connected to the second device 12, the third device 13 (e.g., haptic device) may automatically initiate a process for transmitting the actuator information 232 through a dedicated application programming interface (Application Program Interface(s); API). This process corresponds to the so-called "Plug and Play" (PnP) process, for example, and the third device 13 (e.g., haptic device) and the second device 12 correspond to PnP devices.

[0131] According to another embodiment, the actuator information 232 is obtained or retrieved from the memory of the second device 12 or from a remote system, for example, when an identifier identifying the third device 13 (e.g., a tactile device) is received, and the identifier is, for example, automatically transmitted by the third device 13 (e.g., a tactile device) to the second device 12 or transmitted by the third device 13 (e.g., a tactile device) after the second device 12 sends a request.

[0132] Figure 3 is a schematic representation of the detection of one or more components in the audio signal 201 according to at least one example embodiment.

[0133] Figure 3 The operations / steps of implementing one or more exemplary processes / methods in the processing unit of block 22 are shown. Such operations / steps may be implemented, for example, in one or more processing units 31, 32. Processing units 31 and 32 may be implemented as a single component or as separate components and a combination of hardware and software known to those skilled in the art.

[0134] Processing unit 31 advantageously represents a first part of the process performed by the processing unit of block 22 to detect one or more transients (if any) in the audio signal.

[0135] Processing unit 32 advantageously represents a second portion of the process performed by the processing unit of block 22 to detect one or more frequency sub-portions (if any) in the audio signal.

[0136] The detection of one or more transients is based on input data 301, which represents the audio signal 201. The input data 301 corresponds, for example, to data of the audio signal 201 itself or to data of an audio signal that has been processed (eg downsampled, buffered).

[0137] The detection of one or more frequency sub-portions is based on input data 302 representing the audio signal 201. The input data 302 may for example correspond to data of the audio signal 201 itself or to data of an audio signal that has been processed (e.g. time-frequency converted, low-pass filtered, band-pass filtered, downsampled, buffered).

[0138] The input data 301 and 302 may be the same or different, ie different processing (if any) may be applied to the audio signal 201 to obtain the input data 301 and 302 .

[0139] According to a first exemplary embodiment, the envelope of the audio signal 201 is obtained (eg extracted) from data representing the audio signal (eg from a representation of the audio signal in the time domain).

[0140] The envelope is obtained by any method known to a person skilled in the art, such as described in the paper "Improved estimation of the amplitude envelope of time-domain signals using true envelope cepstral smoothing" by Marcelo Caetano and Xavier Rodet, published in May 2011 at the IEEE International Conference on Acoustics, Speech and Signal Processing (HAL ID: hal-00604385), or in the paper "Aheuristic approach to obtain signal envelope with a simple software implementation" by Cecilia Gisele Jarne, published in July 2018 at Anales AFA.

[0141] Instantaneous detection

[0142] Each instant is detected based on the rate of change of the envelope of the audio signal, for example by deriving a function representing the envelope to obtain a function representing the rate of change.

[0143] For example, a transient is detected whenever the derivative of the envelope is greater than a determined threshold value, the determined threshold value corresponding to a detection parameter 311 .

[0144] According to a variant embodiment, the detection of the instant is based on another detection parameter 311 corresponding to the time window. According to this variant embodiment, the audio signal 201 is segmented into time segments, each time segment having a duration corresponding to the time window parameter, each time segment having at most one instant in it, by comparing the derivative of the audio signal associated with said time segment with a determined threshold parameter.

[0145] The characteristic 312 representing the detected instant corresponds to an amplitude value and a time, which corresponds to the amplitude of the audio signal and the corresponding time associated with the derivative at the point where the derivative is greater than a determined threshold.

[0146] Detection of frequency components / sub-components

[0147] The detection of frequency components / sub-portions of the audio signal 201 is achieved, for example, using a time window as a detection parameter 321 .

[0148] For example, in each time segment of the envelope of the audio signal having a duration equal to the time window, a (first) characteristic 312 corresponding to the maximum value of the amplitude of the envelope of each time segment is obtained.

[0149] A frequency domain representation of each time segment of the envelope is obtained, for example via FFT, and a (second) characteristic 322 corresponding to, for example, the highest frequency (maximum frequency within the time segment) is obtained. According to another embodiment, a bandpass filter is performed on the frequency domain representation of the time segment, and the extracted (second) characteristic 322 corresponds to the highest frequency (maximum frequency) within the bandpass filtered signal.

[0150] Haptic Effect Generation / Synthesis

[0151] When a transient is detected, a haptic effect associated / related to the transient is generated based on the characteristic 312 received by the processing unit of block 23 .

[0152] For example, data representing a haptic effect associated with the instant is generated, the data comprising a magnitude value of the haptic effect determined from the magnitude value obtained from the processing unit 31 .

[0153] The magnitude value of the haptic effect is determined, for example, by applying a weighting factor or coefficient to the received magnitude value, the value of the weighting factor being based on user preference parameter 231 , for example.

[0154] The haptic effect representing the instant is synchronized with the audio content, for example, by using the characteristic 312 representing time.

[0155] The haptic effect associated with the instant corresponds, for example, to a peak having an amplitude obtained as a function of an amplitude corresponding to a characteristic obtained from the detected instant. The duration of the haptic effect is set, for example, according to a time parameter included in user preference parameters 231 .

[0156] When a frequency component is detected, a haptic effect associated / related to this frequency component is generated based on the characteristic 322 received by the processing unit of block 23 .

[0157] For example, the tactile effect may be represented by a sinusoidal function, where the frequency corresponds to the (frequency) characteristic 322 obtained from the processing unit of block 32, and the amplitude of the signal corresponds to the (amplitude) characteristic 322 obtained from the processing unit of block 32, and a weighting factor or coefficient (e.g., depending on user preference parameter 231) is, for example, applied to the (amplitude) characteristic 322 to obtain the amplitude of the tactile effect.

[0158] Different functions may be used to generate tactile effects (for instantaneous and / or frequency components), the function used to generate a signal representing the tactile effect depending, for example, on the type of third device 13 (e.g., tactile device) and / or on the tactile rendering capabilities of an actuator included in the third device 13 (e.g., tactile device).

[0159] For example, if the third device 13 (e.g., a haptic device) includes one or more voice coils, the first function may be used to generate a haptic effect corresponding to a transient from characteristic 312, and the second function may be used to generate a haptic effect corresponding to a frequency component from characteristic 322. If the third device 13 (e.g., a haptic device) includes one or more LRAs, a third function may be used to generate a haptic effect corresponding to a transient from characteristic 312, and a fourth function may be used to generate a haptic effect corresponding to a frequency component from characteristic 322. According to this example, the first, second, third, and fourth functions are different.

[0160] According to the second exemplary embodiment, data or signals representing the haptic effect are obtained by applying a low pass filter to the audio signal 201 .

[0161] Detection of frequency components / sub-components

[0162] A first low pass filter is applied to the audio signal 201 with a first frequency cutoff corresponding to the detection parameter 321 , for example equal to 400 or 450 Hz.

[0163] The first low-pass signal obtained from the first low-pass filtering is used as a drive signal for obtaining or generating a haptic effect by the processing unit of block 23, for example. The characteristic 322 transmitted by the processing unit of block 32 to the processing unit of block 23 corresponds, for example, to a parameter representing a sample set of the first low-pass signal.

[0164] Instantaneous detection

[0165] A second low pass filter is applied to the audio signal 201 with a second frequency cutoff higher than the first frequency cutoff (eg equal to 450, 550 or 600 Hz), the second frequency cutoff also corresponding to the detection parameter 321 .

[0166] As known to those skilled in the art, the power spectrum (Power Spectrum), also known as PSD (Power Spectral Density), is determined for the first low-pass signal (PSD1) and the second low-pass signal (PSD2) obtained from the second low-pass filtering.

[0167] For example, when the difference between PSD2 and PSD1 is greater than or above a determined threshold, denoted as “T” (which is received as detection parameter 321 ), ie, when PSD2 − PSD1 > T, a transient is detected.

[0168] The characteristic 312 obtained from the detected instant corresponds, for example, to the amplitude power difference associated with the detection of “PSD2 - PSD1 > T”.

[0169] For example, the above operation may be repeated for each time segment of the audio signal 201 , the duration of which is indicated or signaled by the detection parameter.

[0170] According to a variant embodiment, the Root Mean Square (RMS) is used instead of PSD.

[0171] Haptic Effect Generation / Synthesis

[0172] When a frequency component is detected, data representing the first low-pass signal is used by the processing unit of block 23 as a driving signal for synthesizing a corresponding haptic effect.

[0173] When a transient is detected, the amplitude-power difference is used to generate or synthesize a corresponding haptic effect to determine the amplitude of the haptic effect, such as by applying a weighting factor or coefficient to the received amplitude-power difference, the value of the weighting factor being, for example, based on user preference parameter 231 .

[0174] For the first example embodiment, different functions may be used to generate tactile effects (for instantaneous and / or frequency components), the function used to generate a signal representing the tactile effect depending, for example, on the type of the third device 13 (e.g., a tactile device) and / or the tactile rendering capabilities of an actuator included in the third device 13 (e.g., a tactile device).

[0175] Figure 4is a schematic block diagram of steps of a method of generating one or more haptic effects from audio content according to at least one example embodiment.

[0176] In a first step 41 , classification information representative of the type of audio content associated with an audio signal is obtained, for example received from a remote device or system or retrieved from a memory.

[0177] In the second step 42 , a detection parameter set is selected from a plurality of detection parameter sets according to the classification information obtained in the first step 41 .

[0178] In a third step 43 , at least one component of the audio signal is detected according to the set of detection parameters selected in the second step 42 .

[0179] In a fourth step 44 at least one characteristic representative of the at least one component detected in the third step 43 is determined or obtained from the at least one component.

[0180] In a fifth step 45 , one or more haptic effects are generated depending on the at least one characteristic determined in the fourth step 44 .

[0181] Figure 5 is a schematic block diagram of an example of a system 5 that implements various aspects and example embodiments.

[0182] The system 5 may be embodied as one or more devices including various components described below. In various embodiments, the system 5 may be configured to implement one or more aspects described in the present invention.

[0183] Examples of devices that may form all or part of system 5 include personal computers, laptops, smartphones, tablet computers, digital multimedia set-top boxes and their associated processing systems, head-mounted display devices (HMDs, perspective glasses), tactile sensors or actuators, "caves" (systems including multiple displays), servers, tactile encoders, tactile decoders, post-processors that process the output of tactile decoders, pre-processors that provide input to tactile encoders, network servers, set-top boxes, wireless (e.g., Bluetooth) connected wearable tactile devices, game controllers, mice, mouse pads, keyboards, palm pads, chairs, tables, XR headsets, headphones, bracelets, head and / or lumbar support devices or chairs, any other device for processing tactile data or tactile signals, or other communication devices. The components of system 5 may be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or separate components. For example, in at least one embodiment, the processing and encoder / decoder components of system 5 may be distributed across multiple ICs and / or discrete components. In various embodiments, system 5 may be communicatively coupled to other similar systems or other electronic devices via, for example, a communications bus or through dedicated input and / or output ports.

[0184] The system 5 may include at least one processor 51 configured to execute instructions loaded therein for implementing, for example, various aspects described in the present invention. The processor 51 may include embedded memory, input and output interfaces, and various other circuits known in the art. The system 5 may include at least one memory 52 (e.g., a volatile storage device and / or a non-volatile storage device). The system 5 may include a storage device 54, which may include a non-volatile memory and / or a volatile memory, including but not limited to an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 54 may include an internal storage device, an additional storage device, and / or a network accessible storage device.

[0185] The system 5 may include an encoder / decoder module 53 configured to, for example, process data to provide encoded / decoded tactile signals or data, and the encoder / decoder module 53 may include its own processor and memory. The encoder / decoder module 53 may represent a module that may be included in a device to perform encoding and / or decoding functions. As is well known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 53 may be implemented as a separate component of the system 5 or may be incorporated into the processor 51 as a combination of hardware and software known to those skilled in the art.

[0186] Program code to be loaded onto the processor 51 or the encoder / decoder module 53 to perform various aspects described in the present invention may be stored in the storage device 54 and subsequently loaded onto the memory 52 for execution by the processor 51. According to various embodiments, one or more of the processor 51, the memory 52, the storage device 54, and the encoder / decoder module 53 may store one or more various items during the execution of the processes described in the present invention. Such stored items may include, but are not limited to, data representing audio content, data representing video content, haptic-related data, bitstreams, matrices, variables, and intermediate or final results from equations, formulas, operations, and arithmetic logic processing.

[0187] In some embodiments, memory within the processor 51 and / or encoder / decoder module 53 may be used to store instructions and provide working memory for processes that may be performed during data processing, encoding, or decoding.

[0188] However, in other embodiments, memory external to the processing device (e.g., the processing device may be a processor) may be used for one or more of these functions. The external memory may be memory 52 and / or storage device 54, such as dynamic volatile memory and / or non-volatile flash memory. In at least one embodiment, fast external dynamic volatile memory such as RAM may be used as working memory for data processing.

[0189] As indicated at block 55, input to the components of system 5 may be provided via various input devices. Such input devices include, but are not limited to, (i) an RF section that may receive an RF signal transmitted over the air, such as by a broadcast device, (ii) a composite input terminal, (iii) a USB input terminal, (iv) a phone connector (also known as a phone jack, audio jack, headphone jack, or jack plug) input terminal, and / or (v) an HDMI input terminal.

[0190] In various embodiments, the input device of block 55 may have associated corresponding input processing components known in the art. For example, the RF section may be associated with the following components required: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) band limiting the frequency band again to a narrower frequency band to select a signal frequency band that (for example) may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing debugging, and (vi) demultiplexing to select the desired datagram stream. The RF section of various embodiments may include one or more components to perform these functions, such as a frequency selector, a signal selector, a frequency band limiter, a channel selector, a filter, a downconverter, a demodulator, a debugger, and a demultiplexer. The RF section may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or to the baseband.

[0191] In one set-top box embodiment, the RF section and its associated input processing components can receive RF signals transmitted through wired (e.g., cable) media. The RF section can then perform frequency selection by filtering, downconverting, and filtering again to a desired frequency band.

[0192] Various embodiments may rearrange the order of the above-described (and other) components, remove some of these components, and / or add other components that perform similar or different functions.

[0193] Adding components may include inserting components between existing components, such as inserting an amplifier and an analog-to-digital converter. In various embodiments, the RF portion may include an antenna.

[0194] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 5 to other electronic devices via the USB and / or HDMI connections. It should be understood that various aspects of input processing may be implemented by, for example, analog-to-digital conversion, time-domain to frequency-domain conversion, downsampling, bandpass or lowpass filtering, Reed-Solomon debugging, etc., for example, in a separate input processing IC or in the processor 51, as desired. Similarly, various aspects of USB or HDMI interface processing may be implemented in a separate interface IC or in the processor 51, as desired. The processed stream may be provided to various processing components, including, for example, the processor 51 and the encoder / decoder module 53, which operates in conjunction with the memory and storage components to process the data stream as needed for presentation on the output device.

[0195] The various components of the system 5 may be disposed in an integrated housing in which the various components may be interconnected and transmit data therebetween using appropriate connection arrangements, such as internal buses known in the art, including I2C buses, wiring, and printed circuit boards.

[0196] System 5 may include a communication interface 56 that enables communication with other devices via a channel 560. Communication interface 56 may include, but is not limited to, a transceiver configured to transmit and receive data via channel 560. Communication interface 56 may include, but is not limited to, a modem or a network card, and channel 560 may be implemented, for example, within a wired and / or wireless medium.

[0197] In various embodiments, a Wi-Fi network such as IEEE 802.11 may be used to stream data to system 5. Wi-Fi signals for these embodiments may be received via a channel 560 suitable for Wi-Fi communications and communication interface 56. Channel 560 for these embodiments may typically connect to an access point or router that provides access to external networks including the Internet to allow streaming applications and other over-the-top communications.

[0198] Other embodiments may provide streaming data to system 5 using a set-top box or computer that transmits data via the HDMI connection of block 55 .

[0199] Still other embodiments may use the RF connection of block 55 to provide streaming data to system 5 .

[0200] Streamed data may be used as a means of communicating information used by system 5. Communication information may include data encoded into a container such as a binary stream or a haptic effect file.

[0201] It should be understood that the communication can be accomplished in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc. can be used to send information to a corresponding data processing device.

[0202] The system 5 may provide output signals to various output devices, including a display 570 , a speaker 580 , and other peripheral devices 590 such as haptic devices / actuators.

[0203] Control signals may be transmitted between the system 5 and the display 570, speaker 580 or other peripheral device 590 using signaling such as AV.Link, Consumer Electronics Control (CEC), audio protocols, Universal Serial Bus (USB), Haptics Industry Forum-Universal Haptic Protocol (HIF UHP), or other communication protocols that enable device-to-device control with or without user intervention.

[0204] The output devices may be communicatively coupled to the system 5 via dedicated connections through corresponding interfaces 57 , 58 , and 59 .

[0205] Alternatively, the output device may be connected to the system 5 via the communication interface 56 using the channel 560. The display 570, the speaker 580 and / or the haptic device (actuator) may be integrated into a single unit with the other components of the system 5 in an electronic device (eg, a television).

[0206] In various embodiments, the (display) interface 57 may include a display driver, such as a timing controller (TCon) chip.

[0207] The display 570, speaker 580, and / or haptic device (actuator) may alternatively be separate from one or more other components. In various embodiments where the display 570, speaker 580, and / or haptic device (actuator) may be external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0208] exist Figures 1 to 5 In the present invention, various methods are described, and each method includes one or more steps or actions for implementing the described method. Unless a specific order of steps or actions is required for the correct operation of the method, the order and / or use of specific steps and / or actions can be modified or combined.

[0209] Some examples are described with respect to block diagrams and / or operational flow charts. Each block represents a circuit component, module, or portion of program code that includes one or more executable instructions for implementing a specific logical function. It should also be noted that in other embodiments, the functions annotated in the blocks may not occur in the order indicated. For example, two blocks shown in succession may actually be executed approximately simultaneously, or the blocks may sometimes be executed in the reverse order, depending on the functions involved.

[0210] The embodiments and aspects described herein may be implemented, for example, by methods or processes, devices, computer programs, data streams, bit streams, or signals. Even if only discussed in the context of a single form of embodiment (e.g., discussed only as a method), the embodiments of the features discussed may also be implemented in other forms (e.g., devices or computer programs).

[0211] These methods can be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device.

[0212] In addition, these methods can be implemented through instructions executed by a processor, and such instructions (and / or data values ​​generated by the implementation) can be stored on a computer-readable storage medium. The computer-readable storage medium can take the form of a computer-readable program product, which is embodied in one or more computer-readable media and has a computer-readable program code embodied thereon that can be executed by a computer. Considering the inherent ability to store information therein and provide the inherent ability to retrieve information therefrom, the computer-readable storage medium used herein can be considered a non-transitory storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. It should be understood that although the following provides more specific examples of computer-readable storage media to which the present embodiment can be applied, as those skilled in the art will readily appreciate, it is merely an illustrative and non-exhaustive list: portable computer disk; hard disk; read-only memory (ROM); erasable programmable read-only memory (EPROM or flash memory); portable compact disk read-only memory (CD-ROM); optical storage device; magnetic storage device; or any suitable combination of the foregoing.

[0213] These instructions may form an application program tangibly embodied on a processor-readable medium.

[0214] Instructions may be, for example, hardware, firmware, software, or a combination thereof. For example, instructions may be found in an operating system, a separate application, or a combination of both. Thus, a processor may be represented, for example, as a device configured to perform a process and a device including a processor-readable medium (e.g., a storage device) having instructions for performing the process. Furthermore, in addition to instructions, or in lieu of instructions, the processor-readable medium may also store data values ​​generated by an embodiment.

[0215] The device may be implemented, for example, with appropriate hardware, software, and firmware. Examples of such devices include personal computers, laptops, smartphones, tablets, digital multimedia set-top boxes, digital television receivers, personal video recording systems, networked home appliances, head-mounted display devices (HMDs, perspective glasses), projectors (projectors), "caves" (systems including multiple displays), servers, video and / or tactile encoders, video and / or tactile decoders, tactile engines, post-processors for processing the output of a video decoder, pre-processors for providing input to a video encoder, network servers, set-top boxes, wirelessly connected wearable tactile devices, such as Bluetooth-connected wearable tactile devices, game controllers, mice, mouse pads, keyboards, palm pads, chairs, desks, XR headsets, headphones, bracelets, head and / or lumbar support devices or chairs, and any other device for processing tactile data or signals representing one or more tactile feedback or effects, or other communication devices. It should be noted that the device may be portable.

[0216] Computer software may be implemented by processor 51, or by hardware, or by a combination of hardware and software. As a non-limiting example, embodiments may also be implemented by one or more integrated circuits. Memory 52 may be of any type suitable for the technical environment, and may be implemented using any appropriate data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. As a non-limiting example, processor 51 may be of any type suitable for the technical environment, and may include one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0217] It will be apparent to one skilled in the art that embodiments may generate a variety of signals formatted to carry information that may, for example, be stored or transmitted. Such information may include, for example, instructions for executing a method or data generated by one of the described embodiments. For example, a signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using a radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted via a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0218] The terms used herein are used only for the purpose of describing specific embodiments and are not intended to be limiting. As used herein, the singular forms "a", "an", and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "include / comprise" and / or "including / comprising" when used in this specification may specify stated characteristics, integers, steps, operations, elements and / or components, for example, but do not exclude the presence or addition of one or more other characteristics, integers, steps, operations, components, components and / or groups thereof. In addition, when a component is referred to as "responsive" or "connected" to another component, it may be directly responsive or connected to the other component, or there may be intermediate components. In contrast, when a component is referred to as "directly responsive" or "directly connected" to other components, there are no intermediate components.

[0219] It should be understood that the use of any of the symbols / terms " / ", "and / or", and "at least one of" such as in the case of "A / B", "A and / or B", and "at least one of A and B" may be intended to cover selection of only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C", such wording is intended to cover selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). As will be apparent to one of ordinary skill in this and related arts, this may be extended to as many items as listed.

[0220] Various numerical values ​​may be used in the present invention. Specific values ​​may be used for example purposes and the described aspects are not limited to these specific values.

[0221] It should be understood that although the terms first, second, etc. may be used herein to describe various components, these components are not limited to these terms. These terms are only used to distinguish one element from another element. For example, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component without departing from the teachings of the present invention. There is no implicit ordering between the first component and the second component.

[0222] References to "one example embodiment" or "one implementation" and other variations are often used to convey that a particular feature, structure, characteristic, etc. (described in connection with an embodiment / implementation) is included in at least one embodiment / implementation. Therefore, the appearances of the phrase "in one embodiment" or "in one implementation" and any other variations appearing in various places of the present invention are not necessarily all referring to the same embodiment.

[0223] Similarly, references herein to "according to an exemplary embodiment / instance / implementation" or "in an exemplary embodiment / instance / implementation" and other variations are often used to convey that a particular characteristic, structure, or feature (in conjunction with the embodiment / instance / implementation description) may be included in at least one embodiment / instance / implementation. Therefore, the expressions "according to an embodiment / instance / implementation" or "in an embodiment / instance / implementation" appearing in different places in the specification do not necessarily all refer to the same embodiment / instance / implementation, nor do separate or alternative embodiments / instances / implementations necessarily exclude other embodiments / instances / implementations.

[0224] The reference numerals appearing in the claims are for illustration only and have no limiting effect on the scope of the claims.Embodiments / examples and variations of the present invention may be employed in any combination or sub-combination, although not explicitly described.

[0225] When the figures are presented as flow charts, it should be understood that they also provide block diagrams of the corresponding devices. Similarly, when the figures are presented as block diagrams, it should be understood that they also provide flow charts of the corresponding methods / processes.

[0226] Although some figures include arrows on communication paths to illustrate a primary direction of communication, it should be understood that communication can occur in the opposite direction to the depicted arrows.

[0227] Additionally, the present invention may refer to "obtaining" various information. Obtaining information may include, for example, one or more of receiving information, determining information, estimating information, calculating information, or retrieving information from a memory.

[0228] Additionally, the application may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, or estimating information.

[0229] Additionally, the present invention may refer to "receiving" various information. Like "accessing," receiving is a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from memory). Furthermore, "receiving" generally involves operations such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, or estimating information in various ways.

[0230] A variety of embodiments have been described herein. However, it should be understood that various modifications may be made. For example, elements of different embodiments may be combined, supplemented, modified, or removed to produce other embodiments. In addition, it will be understood by those of ordinary skill that other structures and processes may replace the disclosed structures and processes, and that the resulting embodiments will perform at least substantially the same functions in at least substantially the same manner to achieve at least substantially the same results as the disclosed embodiments. Therefore, the present invention contemplates these and other embodiments.

Claims

1. A method for generating a tactile effect, the method comprising: obtaining (41) classification information representing a type of audio content associated with an audio signal (201); selecting (42) a detection parameter set (221) from a plurality of detection parameter sets according to the classification information using mapping information, the mapping information mapping each detection parameter set from the plurality of detection parameter sets to a determined audio content type from a list of different types of audio content; detecting (43) at least one component of the audio signal (201) based on the set of detection parameters (221); determining (44) at least one characteristic (212) representative of the at least one component; as well as The haptic effect is generated (45) based on the at least one characteristic (212).

2. The method of claim 1, further comprising obtaining a user preference parameter set (231) representing user preferences, wherein the haptic effect is further generated according to the user preference parameter set (231).

3. The method of claim 2, further comprising: receiving mode information, the mode information representing a presentation mode for presenting the haptic effect; The user preference parameter group is selected from a plurality of user preference parameter groups according to the mode information (231).

4. The method according to claim 2 or 3, wherein: The values ​​of the detection parameters of the detection parameter set (221) are based on the user preference parameter set (231).

5. The method according to any one of claims 1 to 4, wherein: The at least one component belongs to a component group, the component group comprising: a component corresponding to an instant in a time representation of the audio signal (201); and A component corresponding to a sub-band of a frequency band associated with a frequency representation of the audio signal (201).

6. The method of claim 5, wherein: The frequency representation is obtained by applying a Fast Fourier Transform to the time representation of the audio signal (201).

7. The method according to any one of claims 1 to 6, wherein: The detection parameter group (221) at least includes parameters belonging to a parameter group, and the parameter group includes: represents the parameter that determines the amplitude; Parameters representing the time window; Parameters representing the limits of the frequency interval; a parameter representing a frequency threshold; and Parameter representing the spectrum power threshold.

8. The method according to any one of claims 1 to 7, wherein: The at least one characteristic (212) belongs to a characteristic group, the characteristic group comprising: Characteristic representing the magnitude value; Characteristics representing frequency values; The 2 power spectral densities representing the audio signal represent the characteristics of the amplitude power difference between them.

9. The method of any one of claims 1 to 8, further comprising obtaining actuator information (232) representative of a type of haptic actuator presenting the haptic effect, the haptic effect being further generated based on the actuator information (232).

10. The method according to any one of claims 1 to 9, wherein: The image information is stored in the memory.

11. A device (5) for generating a tactile effect, wherein: The device comprises a memory (52) associated with at least one processor (51), the at least one processor (51) being configured to implement the method according to any one of claims 1 to 10.

12. A non-transitory processor-readable medium storing instructions for causing a processor to execute at least the steps of the method according to any one of claims 1 to 10.

13. A computer program product comprising program code instructions for executing the method according to any one of claims 1 to 10 when the program is executed on a computer.

Images