Block-based structure for haptic data

Abstract

Methods, devices, and data streams for generating, transmitting, and decoding haptic data are provided. The present principles generally relate to the field of encoding haptic data. In particular, the present principles relate to a block-based structure for haptic data that allows parallel processing of data chunks for scalable network distribution, such as streaming applications.

Description

1. Technical Field

[0001] The present principles generally relate to the field of encoding haptic data. In particular, the present principles relate to a block-based structure for haptic data that allows parallel processing of data chunks for scalable network distribution such as streaming applications. 2. Background technology

[0002] This section is intended to introduce the reader to various aspects of the art, which may be related to various aspects of the present principles described and / or claimed below. It is believed that this discussion is helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present principles. Therefore, it should be understood that these statements should be read in this light, and not as admissions of prior art.

[0003] A tactile sequence is a set of data encoded for presentation based on touch and positioning in space, just as a video sequence is a set of encoded data for presentation using vision. A tactile sequence encodes temporal data, for example represented as a track associated with a tactile device. A tactile device can present tactile sensations of different modalities and positioning in space, such as vibration, force, position, velocity, or temperature. There are formats for encoding tactile sequences, mainly proprietary formats. They are intended to be read from local memory. However, based on recent developments in tactile devices and with the continued development of tactile data formats, such as the MPEG (Moving Picture Experts Group) standardization process for encoded representations of tactile sensations, a format for encoding tactile data suitable for streaming applications and other network distribution is now needed. According to the present principles, a block-based structure for tactile data allows data to be processed in parallel by blocks. 3. Summary of the invention

[0004] A simplified summary of the present principles is presented below to provide a basic understanding of some aspects of the present principles. This summary is not an extensive review of the present principles. It is not intended to identify the key or important elements of the present principles. The following summary merely presents some aspects of the present principles in a simplified form as a preface to the more detailed description provided below.

[0005] The present principles relate to a method for encoding tactile data for a tactile sequence. The method includes obtaining binary tactile data representing a tactile effect and metadata describing the tactile effect. The tactile effect is decomposed into time events and frequency bands, and they are grouped into tracks. The tracks and metadata are encoded in access units, and the metadata includes sensory data pointing to a subset of the tracks and experience data pointing to a subset of the sensory data. In one embodiment, the access unit is a network abstraction layer unit. In another embodiment, the access unit is constructed depending on whether the access unit includes metadata or frequency band data. In yet another embodiment, the metadata includes information describing a tactile effect library, an avatar, or a device.

[0006] The present principles also relate to apparatuses implementing the above-mentioned methods.The present principles also relate to methods and apparatuses for decoding and processing haptic data encoded according to the above-mentioned methods. 4. Description of the drawings

[0007] The present disclosure will be better understood, and other specific features and advantages will appear, upon reading the following description, which refers to the accompanying drawings, in which:

[0008] - Figure 1 illustrates a possible organization of data for a haptic sequence according to the present principles;

[0009] - Figure 2 illustrates how a haptic signal may be decomposed into two frequency bands and / or reconstructed from two frequency bands;

[0010] - Figure 3 An example architecture of a processing engine 30 that may be configured to implement the present principles is shown;

[0011] - Figure 4 illustrates a representation of a data structure compatible with the present principles;

[0012] - Figure 5 An example of a block-based structure for haptic data in accordance with the present principles is shown.

[0013] - Figure 6 A high-level syntax for a block-based structure for haptic data encoded in a data stream in accordance with the present principles is described;

[0014] - Figure 7 The diagram shows how to describe Figure 4 and 5 An example of the syntax of a NAL unit for the described experience type;

[0015] - Figure 8 The diagram shows how to describe Figure 4 and 5 Examples of syntax for NAL units of the described perceptual types;

[0016] - Fig. 9 shows the NAL unit structure for describing an effects library proposed herein according to the present principles;

[0017] - Fig.10 It is proposed to describe the NAL unit structure to encode Figure 4 and 5 An example of the syntax of the described metadata track;

[0018] - Fig.11 It is proposed to describe the NAL unit structure to encode Figure 4 and 5Examples of syntax for the metadata bands described;

[0019] - Fig.12 An example of band data structured as NAL units in accordance with the present principles is shown. 5. Specific implementation methods

[0020] The present principle will be described more fully below with reference to the accompanying drawings, in which examples of the present principle are shown. However, the present principle can be embodied in a variety of alternative forms and should not be construed as being limited to the examples set forth herein. Therefore, although the present principle allows for various modifications and alternative forms, specific examples thereof are shown by the examples in the accompanying drawings and will be described in detail herein. However, it should be understood that it is not intended to limit the present principle to the specific form disclosed, but on the contrary, the present disclosure will cover all modifications, equivalents and substitutes that fall within the spirit and scope of the present principle as defined by the claims.

[0021] The terms used herein are only for the purpose of describing specific examples, and are not intended to limit this principle. As used herein, the singular forms "one", "an" and "the" are intended to also include plural forms, unless the context clearly indicates otherwise. It will also be understood that the terms "include", "include", "include" and / or "include" when used in this specification specify the presence of the features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their combinations. In addition, when an element is referred to as "response" or "connected" to another element, it can directly respond or be connected to another element, or there can be an intermediate element. In contrast, when an element is referred to as "direct response" or "direct connection" to other elements, there is no intermediate element. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items, and can be abbreviated as " / ".

[0022] It will be understood that although the terms first, second, etc. can be used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the teaching of the present principles, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element.

[0023] Although some diagrams include arrows on communication paths to illustrate a primary direction of communication, it is to be understood that communication can occur in the opposite direction of the depicted arrows.

[0024] Some examples are described with reference to block diagrams and operational flow charts, wherein each block represents a circuit element, module, or code portion, which includes one or more executable instructions for implementing (one or more) specified logical functions. It should also be noted that in other implementations, the (one or more) functions marked in the box may not occur in the order marked. For example, two blocks shown in succession may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order, depending on the functions involved.

[0025] The phrases "according to an example" or "in an example" mentioned herein mean that a particular feature, structure, or characteristic described in conjunction with the example may be included in at least one implementation of the present principles. The phrases "according to an example" or "in an example" appearing in various places in the specification do not necessarily refer to the same example, nor are they necessarily separate or alternative examples that are mutually exclusive with other examples.

[0026] Reference numerals appearing in the claims are by way of illustration only and shall have no limiting effect on the scope of the claims.The present examples and variations may be employed in any combination or sub-combination although not explicitly described.

[0027] Figure 1 A possible organization of data for a tactile sequence according to the present principles is illustrated 10. Figure 1 In the example of , the data includes high-level metadata information 11 describing the overall tactile experience defined in the sequence. A list of avatars 12 (i.e., body representations) is also provided. Avatars are referenced in the file to specify the desired locations on the body for tactile stimulation. The tactile data is described, for example, by a list of perceptions 13. These perceptions correspond to tactile signals associated with specific sensory modalities (i.e., vibration, force, position, velocity, temperature, ...).

[0028] The perception includes a list of tracks 14. In a first embodiment, the track is a signal that can be used directly by the haptic device associated with the track. In another embodiment, the track can be decomposed into frequency bands. Each frequency band defines a portion of the signal within a given frequency range. The frequency bands can be described by a list of haptic effects, each haptic effect including a list of keyframes. The haptic signal in the track can then be reconstructed by combining the data in the frequency bands. Figure 2 It illustrates how a haptic signal 20 may be decomposed into and / or reconstructed from two frequency bands 21 and 22. There are many methods to decompose a signal into frequency bands and to retrieve a signal by adding frequency bands.

[0029] Different types of haptic bands can be considered, for example, transient bands, curve bands, vector wave bands, and wavelet bands. A band consists of a series of "effects", each of which is defined by a list of "keyframes". The data contained in the effects and keyframes is interpreted differently for different types of haptic bands and encoding modalities. For example:

[0030] -For transient strips, each effect stores a set of keyframes that define position, amplitude, and frequency. A keyframe represents a transient event.

[0031] - For a curve band 21, each effect stores a set of keyframes defining the position and amplitude. The keyframes represent the control points of the curve. The type of interpolation function used to generate the band (e.g., cubic or linear) is specified in the band's metadata.

[0032] -For Vector Wave bands, the effect stores a set of keyframes that define position, amplitude, and frequency.

[0033] - For wavelet band 22, the effect stores the content of one wavelet block. It contains a keyframe for each coefficient of the wavelet transformed and quantized signal, where only amplitude values ​​are used. The coefficients are scaled to the range [-1, 1]. Additionally, the original maximum amplitude as well as the maximum number of used bits are stored in the keyframe.

[0034] One possible method for encoding the tactile data of a tactile sequence uses descriptive tactile files (e.g., .IVS and .AHAP files) or waveform files (e.g., .wav files) as input. The encoding of these two types of file formats follows two different approaches. For descriptive input files, the encoder first analyzes the data and then transcodes them into, for example, MPEG format. For waveform signals, the data is processed using signal analysis methods to generate key frames, and either interpolation is performed between key frames, or wavelet encoding is used to generate a binary coded stream. However, with this approach, the data is stored sequentially (bands and tracks) as in a file. Therefore, retrieving a track at a given time requires loading the entire track, accessing and sampling the first band, reading the entire first band, and then reading and sampling the second band, and so on. As a result, this encoding method is not suitable for streaming applications and other network distributions.

[0035] According to the present principles, a method for encapsulating tactile data in a format suitable for scalable network transmission is proposed. According to the present principles, tactile data of a tactile sequence is prepared for a data structure that can be described by network abstraction layer unit (NALu) packetization. In this specification, NAL units are used as examples. Other comparable data structures suitable for scalable network transmission can be used. As another embodiment of the present principles, an access unit (AU) structure corresponding to an elementary stream (ES) divided into data slices is proposed. In this embodiment, the ES is structured into AUs ("coded samples"), which can be mapped to NALu ("network packets") for networking purposes.

[0036] Figure 4A representation of a data structure compatible with the present principles is illustrated. Two types of data are represented: i) description data or metadata 41 describing how the signal is encoded (number of perceptions, number of tracks, body model, device, ...), and ii) encoded binary data 42 (bands, effects and keyframes). According to the present principles as described below, the network abstraction layer concept is applied to tactile data. The following semantic tags are set:

[0037] Access Unit (AU) = decodable tactile data packet from an elementary stream (ES)

[0038] NAL unit (NALu) = Network Ready Packet (packet from AU)

[0039] According to this principle, semantic tags are based on the following properties:

[0040] Haptic effects (FX) are sliced ​​in access units (AU). AU is the lowest, independently decodable information. It is a kind of "sample" of a track.

[0041] Haptic effects are broken down into time events (keyframes) and frequency bands, grouped into tracks,

[0042] Each track is a separate channel,

[0043] Perception is a set of tracks, because a multiplex can be encoded,

[0044] An experience is a set of perceptions in different streams, similar to multilingual audio,

[0045] AUs are grouped into NALus as different types of Media Aware Network Elements (MANEs) for network management (duplicate, remove, ...).

[0046] Based on this, two embodiments are possible: fixed length AU or variable length AU.

[0047] In both cases, AUs are encapsulated into Network Aware Packets (NALu) with the following properties:

[0048] - There are two types of NAL units:

[0049] NAL unit metadata (experience, perception, track) -> applies to all / several AUs

[0050] NAL unit data band -> dedicated to AU

[0051] -NALu consists of NALu header + NALu payload

[0052] The NALu header should be of fixed length, usually 1 byte.

[0053] NALu should indicate how important the packet is to be properly managed by the network (repetition for random access, removal for scalability, ...)

[0054] Figure 5 An embodiment of a block-based structure for tactile data according to the present principles is shown. NALu corresponds to the header and the payload,

[0055] - The NALu header is encoded with a given number of bits (e.g. 8 or 10 or 16) and includes for example:

[0056] NAL type: corresponds to haptic metadata (e.g., experience, perception, track) and data (data band),

[0057] The level (encoded on 2 bits, for example) is added information that specifies the level of a band. It specifies its importance for decoding, similar to the scalable approach in audio and video, and

[0058] Res: Field reserved for future use.

[0059] -NALu payload is structured depending on its type (metadata or band data): The metadata payload includes information describing the haptic effect library, avatar or device. In an alternative, a specific NALu type can be specified for each type to simplify browsing, which involves experience, perception and track metadata specified in MPEG WD.

[0060] Additional information can be added, such as:

[0061] Phase ID (e.g. 1, 2a, or 2b), and / or

[0062] • Profile and level of haptics (eg simple, primary, extended).

[0063] The band data payload includes:

[0064] AU type: A new flag to indicate whether the AU is an independently decodable AU (RAU) or another part of the previous AU (DAU). This information is useful for random access;

[0065] TS: timestamp corresponding to the start of this AU, with reference to the global haptic experience clock;

[0066] Perceptual / track / band IDs correspond to the IDs of each element;

[0067] Nb FX: The number of haptic effects (FX) to be encoded in this payload. In the alternative or in addition, the length of the remaining payload may be encoded;

[0068] Data packet, consisting of several coded effects (FX), with relative timestamp (RTS), type and coded data;

[0069] o RTS: relative time to the previous TS timestamp (= delta / increment). Allows to reduce the number of bits for encoding timing information. In an alternative, the RTS can be replaced by a global timestamp and replace the previous TS bytes,

[0070] o Type: the type of encoding method of the data according to MPEGWD (usually transient, curve, vector, wavelet),

[0071] o FXi: effect data of the ith encoding according to the encoding method (type) and the current WD specification;

[0072] Byte alignment: Padding bits are used to ensure that the payload is an integer multiple of bytes.

[0073] This format is provided as an example. The number of bits can be adapted depending on the implementation of the present principles. Specific codes for synchronization purposes can be added. Profile and level information can also be added.

[0074] It is possible to add constraints to the generation of the file to ensure it is decodable:

[0075] -The tactile encoding flow starts with the NALu metadata experience,

[0076] - the first data band AU is RAU,

[0077] -RAU starts with a keyframe (e.g. wavelet),

[0078] - the metadata AU needs to be repeated regularly for random access (e.g. every second),

[0079] - At least a level 0 is required (and means the current band is a baseline), higher levels indicate supplementary bands that can be skipped for low bitrate applications (a kind of scalability).

[0080] Figure 6 to 1 3 shows another embodiment of a block-based structure of tactile data according to the present principles. Details of NAL units according to the present principles are provided through these figures. In these NAL units, some information elements are added to simplify the encapsulation of the network protocol. For example, byte padding is used to add some bits so that the total number of bits is a multiple of the byte. The cyclic redundancy check (CRC) is a function calculated on the transmitted bits. In the event of a transmission error, the CRC calculated on the receiver side is different from the transmitted CRC. The simulation bit is a bit (such as a "synchronization bit") that is added when the bit combination in the binary stream is equal to the number of reserved bits. If this happens, some bits are added to the bit stream to prevent the error.

[0081] Figure 6 A high-level syntax for a block-based structure of tactile data encoded in a data stream in accordance with the present principles is described. In this embodiment, a NAL unit begins with a header portion that includes, for example, the type of NAL unit and the length of the payload (e.g., in bytes). Adding the length of the payload in the header of the NAL unit is useful for network applications. When a NAL unit is received, the application first reads the header and obtains the number of bits required to read for the payload. Therefore, the memory size can be used to handle parallel reads and / or detect bit loss (as a checksum). In accordance with the present principles, the payload includes a metadata portion and a data band portion.

[0082] Figure 7 The diagram shows how to describe Figure 4 and 5 An example of the syntax of a NAL unit of the described experience type. Like any NAL unit, the experience NAL unit consists of a header and a payload. Avatars are described in these NAL units.

[0083] Figure 8 The diagram shows how to describe Figure 4 and 5 An example of the syntax of a NAL unit of the described perceptual type. Like any NAL unit, a perceptual NAL unit consists of a header and a payload. Figure 8 In an example of , the metadata NAL unit payload includes an effects library. In a variant, the effects library is stored in the payload of the band data.

[0084] Fig. 9 The NAL unit structure for describing the effect library proposed in this article is shown. The effect library is not included in the payload of the band data, but is separated into a dedicated NAL unit. Therefore, the effect library can be sent separately from the payload, and thereby simplify the parallelization process and increase error robustness. In addition, this separated structure allows the effect to be changed during the experience (similar to the update of the library), and is convenient for the use of editing tools or servers. For example, a common NAL unit can be stored for each expert or application, and only needs to be indexed when the effect is used. When separated from the payload, the effect library is constructed differently. Some syntax elements available in the stream are repeated here (in the effect library NAL unit), such as curve type or band type, to ensure that the effect library NAL unit is independent of the current experience.

[0085] Fig.10 It is proposed to describe the NAL unit structure to encode Figure 4 and 5An example of the syntax of the described metadata track. Like any NAL unit, the experience NAL unit consists of a header and a payload.

[0086] Fig.11 It is proposed to describe the NAL unit structure to encode Figure 4 and 5 An example of the syntax of the described metadata band. Like any NAL unit, the experience NAL unit includes a header and a payload. Rather than being included in the payload of the band data, it is separated into a dedicated NAL unit. Thus, the metadata band can be sent separately from the payload and thereby simplify parallelization or increase error robustness.

[0087] Fig.12 An example of band data structured as a NAL unit is shown.

[0088] Figure 3 An example architecture of a processing engine 30 which may be configured to implement the present principles is shown. The device may be linked with other devices via their bus 31 and / or via an I / O interface 36.

[0089] The device 30 comprises the following elements linked together by a data and address bus 31:

[0090] - a microprocessor 32 (or CPU), which is for example a DSP (or digital signal processor);

[0091] - ROM (or read-only memory) 33;

[0092] - RAM (or random access memory) 34;

[0093] - storage interface 35;

[0094] - an I / O interface 36 for receiving data to be transmitted from an application; and

[0095] -power supply( Figure 2 not shown), such as batteries.

[0096] According to one example, the power supply is external to the device. In each of the memories mentioned, the term "register" used in the specification may correspond to an area of ​​small capacity (a few bits) or a very large area (e.g., an entire program or a large amount of received or decoded data). ROM 33 includes at least programs and parameters. ROM 33 may store algorithms and instructions to perform the technology according to the present principles. When turned on, CPU 32 uploads the program to RAM and executes the corresponding instructions.

[0097] RAM 34 includes in registers the program executed by CPU 32 and uploaded after the device 30 is turned on, input data in registers, intermediate data in different states of the method in registers, and other variables in registers used to execute the method.

[0098] The implementations described herein may be implemented in, for example, a method or process, an apparatus, a computer program product, a data stream, or a signal. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method or device), the implementations of the features discussed may also be implemented in other forms (e.g., programs). The apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The method may be implemented in, for example, an apparatus (such as, for example, a processor), which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as, for example, a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0099] The device 30 is linked, for example, via a bus 31, to a set of sensors 37 and a set of presentation devices 38. The sensors 37 may be, for example, cameras, microphones, temperature sensors, inertial measurement units, GPS, humidity sensors, IR or UV light sensors, or wind sensors. The presentation devices 38 may be, for example, displays, speakers, vibrators, heaters, fans, etc.

[0100] According to an example, the device 30 is configured to implement a method according to the present principles and belongs to the set comprising:

[0101] -mobile device;

[0102] -communication equipment;

[0103] -Gaming equipment;

[0104] - Tablet (or tablet);

[0105] - Laptop;

[0106] -Still picture camera;

[0107] -Camcorder.

[0108] The implementation described herein can be implemented in, for example, a method or process, a device, a computer program product, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (e.g., discussed only as a method or device), the implementation of the discussed features can also be implemented in other forms (e.g., programs). The device can be implemented in, for example, appropriate hardware, software, and firmware. The method can be implemented in, for example, a device (such as, for example, a processor), which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as, for example, a smart phone, a tablet computer, a computer, a mobile phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0109] The implementation of the various processes and features described herein can be embodied in various different devices or applications, in particular, for example, devices or applications associated with data encoding, data decoding, view generation, texture processing, and other processing of images and related texture information and / or depth information. Examples of such devices include encoders, decoders, post-processors for processing outputs from decoders, pre-processors for providing inputs to encoders, video encoders, video decoders, video codecs, network servers, set-top boxes, laptops, personal computers, cellular phones, PDAs, and other communication devices. As should be clear, the device can be mobile and even installed in a mobile vehicle.

[0110] Additionally, the method may be implemented by instructions executed by a processor, and such instructions (and / or data values ​​produced by the implementation) may be stored on a processor-readable medium, such as, for example, an integrated circuit, a software carrier, or other storage device, such as, for example, a hard disk, a compact disk ("CD"), an optical disk (such as, for example, a DVD, commonly referred to as a digital versatile disk or a digital video disk), a random access memory ("RAM"), or a read-only memory ("ROM"). The instructions may form an application program tangibly embodied on a processor-readable medium. The instructions may be, for example, in hardware, firmware, software, or a combination. The instructions may be found, for example, in an operating system, a separate application, or a combination of both. Thus, a processor may be characterized as both, for example, a device configured to perform a process and a device that includes a processor-readable medium (such as a storage device) having instructions for performing the process. Furthermore, in addition to or in lieu of instructions, the processor-readable medium may store data values ​​produced by the implementation.

[0111] As will be apparent to one skilled in the art, implementations may generate a variety of signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry as data the rules for writing or reading the grammar of the described embodiments, or to carry as data the actual grammar values ​​written by the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or a baseband signal. The formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

[0112] Many implementations have been described. Nevertheless, it will be appreciated that various modifications may be made. For example, elements of different implementations may be combined, supplemented, modified, or removed to produce other implementations. Additionally, it will be appreciated by those of ordinary skill in the art that other structures and processes may replace those disclosed, and that the resulting implementations will perform at least substantially the same (one or more) functions in at least substantially the same (one or more) manners to achieve at least substantially the same (one or more) results as the disclosed implementations. Therefore, the present application contemplates these and other implementations.

Claims

1. A method for encoding tactile data of a tactile sequence, the method comprising: - obtaining binary haptic data representing a haptic effect and metadata describing the haptic effect; - Decompose haptic effects into time events and frequency bands and group them into tracks; and - Encoding tracks and metadata in access units, the metadata including sensory data pointing to a subset of tracks and experience data pointing to a subset of sensory data. The method of claim 1 , wherein the access unit is a network abstraction layer unit.

3. A method according to claim 1 or 2, wherein the access units are constructed depending on whether they include metadata or frequency band data. 4 . The method of claim 1 , wherein the metadata comprises information describing a haptic effects library, an avatar, or a device.

5. A device for encoding tactile data of a tactile sequence, the device comprising a processor configured to: - obtaining binary haptic data representing a haptic effect and metadata describing the haptic effect; - Decompose haptic effects into time events and frequency bands and group them into tracks; and - Encoding tracks and metadata in access units, the metadata including sensory data pointing to a subset of tracks and experience data pointing to a subset of sensory data. The apparatus of claim 5 , wherein the access unit is a network abstraction layer unit.

7. The apparatus according to claim 5 or 6, wherein the access units are constructed depending on whether they include metadata or frequency band data.

8. The device of one of claims 5 to 7, wherein the metadata comprises information describing a haptic effects library, an avatar or a device.

9. A method for decoding tactile data of a tactile sequence, the method comprising: - obtain a set of tracks and metadata encoded in access units, the metadata including sensory data pointing to a subset of the tracks and experience data pointing to a subset of the sensory data; and - Accessing experience data in a track pointed to by the sensory data associated with the experience data.

10. The method of claim 9, wherein the access unit is a network abstraction layer unit.

11. The method according to claim 9 or 10, wherein the access units are constructed depending on whether they include metadata or frequency band data.

12. The method of one of claims 9 to 11, wherein the metadata comprises information describing a haptic effects library, an avatar, or a device.

13. A device for decoding tactile data of a tactile sequence, the device comprising a processor configured to: - obtaining a set of tracks encoded in access units and metadata, the metadata comprising sensory data pointing to a subset of the tracks and experience data pointing to a subset of the sensory data; and - Accessing experience data in a track pointed to by the sensory data associated with the experience data. The apparatus of claim 13 , wherein the access unit is a network abstraction layer unit.

15. The apparatus according to claim 13 or 14, wherein the access units are constructed depending on whether they include metadata or frequency band data.

16. The device of one of claims 13 to 15, wherein the metadata comprises information describing a haptic effects library, an avatar, or a device.

Images