Block-based structures for haptic data

Abstract

Methods, devices, and data streams for generating, transmitting, and decoding haptic data are provided. The present principles relate generally to the area of ​​haptic data encoding. In particular, the present principles relate to a block-based structure for haptic data that enables chunks of data to be processed in parallel for scalable network delivery, such as in streaming applications.

Description

[Technical Field]

[0001] The present principles relate generally to the area of ​​haptic data encoding, and in particular to a block-based structure for haptic data that allows chunks of data to be processed in parallel for scalable network delivery, such as in streaming applications. [Background technology]

[0002] This section is intended to introduce the reader to various aspects of art that may be related to various aspects of the present principles, which are described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present principles. Accordingly, it should be understood that these statements are to be read in this light, and not as admissions of prior art.

[0003] A haptic sequence is a set of data encoded for rendering based on contact and spatial positioning, similar to a video sequence being a set of data encoded for rendering using vision. A haptic sequence encodes temporal data, represented as a trajectory associated with a haptic device, for example. A haptic device can render different modalities of contact and positioning in space, such as vibration, force, position, velocity, or temperature. Formats for encoding haptic sequences exist, primarily proprietary formats, which are meant to be read from local memory. However, recent developments in haptic devices and ongoing developments in haptic data formats, such as the Moving Pictures Experts Group (MPEG) standardization process for coded representation of haptics, currently require a format for encoding haptic data suitable for streaming applications and other network distribution. According to the present principles, a block-based structure for haptic data allows for parallel processing of data by chunks. Summary of the Invention

[0004] The following presents a simplified summary of the present principles to provide a basic understanding of some aspects of the present principles. This summary is not an extensive overview of the present principles. It is not intended to identify key or critical elements of the present principles. The following summary merely presents some aspects of the present principles in a simplified form as a prelude to the more detailed description provided below.

[0005] The present principles relate to a method for encoding haptic data for a haptic sequence. The method includes obtaining binary haptic data representing haptic effects and metadata describing the haptic effects. The haptic effects are separated into time events and frequency bands and grouped into trajectories. The trajectories and metadata are encoded into access units, where the metadata includes sensory data pointing to a subset of the trajectories and experiential data pointing to a subset of the sensory data. In one embodiment, the access units are network abstraction layer units. In another embodiment, the access units are structured according to whether they include metadata or band data. In yet another embodiment, the metadata includes information describing a haptic effect library, an avatar, or a device.

[0006] The present principles also relate to devices for implementing the above methods.The present principles also relate to methods and devices for decoding and processing haptic data encoded according to the above methods. [Brief explanation of the drawings]

[0007] The present disclosure will be better understood, and other particular features and advantages will become apparent, on reading the following description, in which reference is made to the accompanying drawings, in which: [Figure 1] 1 illustrates a possible organization of data for a haptic sequence in accordance with the present principles. [Figure 2] 1 illustrates how a haptic signal can be separated into and / or reconstructed from two frequency bands. [Figure 3] 1 illustrates an exemplary architecture of a processing engine 30 that may be configured to implement the present principles. [Figure 4] A representation of a data structure compatible with the present principles is given below. [Figure 5] 1 shows an example of a block-based structure for haptic data in accordance with present principles. [Figure 6] A high level syntax for a block-based structure for haptic data encoded within a data stream in accordance with the present principles is described. [Figure 7] 6 shows an example of syntax for describing NAL units of the experience types described with reference to FIGS. [Figure 8] 6 shows an example of syntax for describing NAL units of the perceptual type described with reference to FIGS. [Figure 9] 1 illustrates the NAL unit structure proposed herein for describing an effects library, in accordance with the present principles. [Figure 10] An example of a syntax describing the NAL unit structure for encoding the metadata trail described with reference to FIGS. 4 and 5 is proposed. [Figure 11] An example of a syntax describing the NAL unit structure for encoding the metadata bands described with reference to FIGS. 4 and 5 is presented. [Figure 12] 1 shows an example of band data structured as NAL units in accordance with the present principles. DETAILED DESCRIPTION OF THE INVENTION

[0008] The present principles are described more fully below with reference to the accompanying drawings, in which examples of the present principles are shown. However, the present principles may be embodied in many alternative forms and should not be construed as limited to the examples set forth herein. Accordingly, while the present principles are susceptible to various modifications and alternative forms, specific examples thereof are shown by way of example in the drawings and are described in detail herein. It is to be understood, however, that there is no intention to limit the present principles to the particular forms disclosed; on the contrary, the present disclosure is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present principles as defined by the appended claims.

[0009] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the present principles. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It will be further understood that as used herein, the terms "comprises," "comprising," "includes," and / or "including" specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Furthermore, when an element is referred to as "responsive to" or "connected to" another element, it may be directly responsive to or connected to the other element, or intervening elements may be present. In contrast, when an element is referred to as "directly responsive to" or "directly connected to" another element, there are no intervening elements present. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ".

[0010] In this specification, terms such as "first," "second," etc. may be used to describe various elements, but it should be understood that these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element without departing from the teachings of the present principles.

[0011] Some of the figures include arrows on communication paths to indicate the primary direction of communication, however, it should be understood that communication may occur in the opposite direction to the depicted arrow.

[0012] Some examples are described with reference to block diagrams and operational flowcharts, in which each block represents circuit elements, modules, or portions of code, with each block including one or more executable instructions for implementing specified logical functions. It should also be noted that in other implementations, the functions noted in the blocks may occur out of the order noted. For example, two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may be executed in the reverse order, depending on the functionality involved.

[0013] As used herein, "by one example" or "in one example" means that a particular feature, structure, or characteristic described in connection with this example may be included in at least one implementation of the present principles. The appearances of the phrase "by one example" or "in one example" in various places in this specification do not necessarily all refer to the same example, and are not necessarily mutually exclusive of other examples in separate or alternative examples.

[0014] Reference signs appearing in the claims are merely for illustrative purposes and shall have no limiting effect on the scope of the claims. Although not explicitly stated, the present embodiments and variations may be used in any combination or subcombination.

[0015] FIG. 1 shows a possible organization 10 of data for a haptic sequence in accordance with the present principles. In the example of FIG. 1, the data includes high-level metadata information 11 that describes the overall haptic experience defined in the sequence. A list of avatars 12 (i.e., body representations) is also provided. The avatars are referenced within the file to specify the desired location of haptic stimulation on the body. The haptic data is described, for example, by a list of sensations 13. These sensations correspond to haptic signals associated with specific sensory modalities (i.e., vibration, force, position, velocity, temperature...).

[0016] The perception includes a list of trajectories 14. In a first embodiment, the trajectories are signals directly usable by the haptic device associated with the trajectory. In another embodiment, the trajectories can be separated into frequency bands. Each band defines a portion of the signal within a given frequency range. The bands may be described using a list of haptic effects, each of which contains a list of keyframes. The haptic signal within the trajectory can then be reconstructed by combining the data within the bands. Figure 2 shows how a haptic signal 20 can be separated into and / or reconstructed from two frequency bands 21 and 22. There are many ways to separate a signal into bands and extract the signal by adding bands.

[0017] Different types of haptic bands can be considered, for example, transient bands, curve bands, vector wave bands and wavelet bands. A band contains a series of "effects", each defined by a list of "keyframes". The data contained in the effects and keyframes is interpreted differently for different types of haptic bands and coding modalities, for example: For transient bands, each effect stores a set of keyframes that define the position, amplitude, and frequency. A keyframe represents a transient event. - For curve bands 21, each effect stores a set of keyframes that define the position and amplitude. The keyframes represent the control points of the curve. The type of interpolation function used to generate the band (e.g. cubic or linear) is specified in the band's metadata. -For vector wave bands, the effect stores a set of keyframes that define the position, amplitude and frequency. For wavelet waveband 22, the effect stores the contents of one wavelet block. The wavelet block contains keyframes for all coefficients of the wavelet-transformed and quantized signal, and only amplitude values ​​are used. The coefficients are scaled to the range [-1,1]. Additionally, the original maximum amplitude is stored in the keyframe along with the maximum number of bits used.

[0018] Possible methods for encoding haptic data for a haptic sequence take as input descriptive haptic files (e.g., .IVS and .AHAP files) or waveform files (e.g., .wav files). Encoding these two types of file formats follows two distinct approaches. In the case of descriptive input files, the encoder first analyzes the data and then transcodes them, for example, into MPEG format. In the case of waveform signals, the data is processed using signal analysis methods to generate keyframes and interpoles between keyframes, or wavelet coding is used to generate a binary encoded stream. However, in such approaches, the data is stored sequentially (bands and trajectories), similar to a file. Therefore, retrieving a trajectory at a given time requires loading the entire trajectory, accessing and sampling the first band, reading the entire first band, then reading the second band and sampling it, and so on. As a result, such encoding methods are not suitable for streaming applications and other network distributions.

[0019] According to the present principles, a method is proposed for encapsulating haptic data in a format suitable for scalable network transmission. According to the present principles, haptic data of a haptic sequence is prepared for a data structure that can be represented by Network Abstraction Layer units (NALu) packetization. In this description, NAL units are used as an example. Other equivalent data structures suitable for scalable network transmission may be used. As another embodiment of the present principles, an Access Unit (AU) structure is proposed that corresponds to an elementary stream (ES) into slices of data. In this embodiment, the ES is structured into AUs ("coded samples") that can be mapped to NALu for networking purposes ("network packets").

[0020] Figure 4 shows a representation of a data structure compatible with the present principles. Two types of data are represented: i) descriptive data or metadata 41 describing how the signal is encoded (some perceptions, some trajectories, body model, device...), and ii) encoded binary data 42 (bands, effects, and keyframes). The concept of a network abstraction layer is adapted to haptic data according to the present principles, as explained below. The following semantic tags are set: Access Unit (AU) = a packet of decodable haptic data from an Elementary Stream (ES). ●NAL unit (NALu) = Network Ready packet (packet from AU)

[0021] According to the present principles, semantic tags are based on the following properties: Haptic effects (FX) are sliced ​​into Access Units (AUs). An AU is the lowest level of independently decodable information. It is a kind of trajectory "sample." ●Haptic effects are separated into time events (keyframes) and frequency bands, and grouped into trajectories. Each trajectory is a separate channel. • Perception is a set of trajectories, multiplexed together, that can be coded. • The experience is a set of perceptions in different streams, corresponding to multilingual audio. ●AUs are packetized into NALu as different types of media-aware network elements (MANE) for network management (repetition, removal, ...).

[0022] Based on this, two embodiments are possible: fixed length AUs or variable length AUs.

[0023] In both cases, the AU is encapsulated in a Network Aware Packet (NALu) that has the following properties: There are two types of NAL units: NAL unit metadata (experience, perception, trajectory) -> applies to all AUs / some AUs NAL unit DataBand->AU only -NALu consists of NALu header + NALu payload NALu headers should be of a fixed length, typically one byte. NALu should indicate packet importance for proper management by the network (repetition for random access, elimination for scalability, etc.)

[0024] 5 shows one embodiment of a block-based structure for haptic data in accordance with the present principles. NALu corresponds to the header and payload. The NALu header is coded with a given number of bits (e.g. 8 or 10 or 16) and contains, for example: NAL type: corresponds to haptic metadata (e.g., experience, perception, trajectory) and data (DataBand), The level (e.g., coded on 2 bits) is additional information to specify the level of the band, its importance for decoding, similar to scalable methods in audio and video; Reserved: Fields reserved for future use. NALu payloads are structured according to their type (metadata or band data): Metadata payloads contain information describing haptic effect libraries, avatars, or devices. Alternatively, a specific NALu type may be assigned to each type to simplify browsing, with respect to experience, perception, and trajectory metadata specified in MPEG WD.

[0025] Additional information can be added, for example: Phase ID (e.g., 1, 2a, or 2b), and / or • Tactile profile and level (e.g., simple, primary, extended).

[0026] The band data payload includes: AU type: a new flag to indicate whether an AU is an independently decodable AU (RAU) or a separate part of the previous AU (DAU). This information is useful for random access; ● TS: timestamp corresponding to the start of this AU, with reference to the global haptic experience clock; ● Perception ID / Trajectory ID / Band ID correspond to the ID of each element; Nb FX: The number of haptic effects (FX) to be coded in this payload. Alternatively or additionally, the length of the remaining payload may be coded; ● A data packet made up of several coded effects (FX) and having relative time stamps (RTS), a type and coded data; ○ RTS: relative time (= delta / increment) with respect to the previous TS timestamp. Allows to reduce the number of bits for encoding timing information. Alternatively, this RTS can be replaced by a global timestamp, replacing the previous TS byte; Type: type of coding method for the data according to MPEG WD (typically transient, curvilinear, vector, wavelet), ○FXi: the ith coded effect data according to coding method (type) and current WD specification; ●Byte Align: A bit inserted to ensure an integer multiple of bytes for the payload.

[0027] This format is provided as an example. The number of bits can be adapted according to the implementation of the present principles. Specific codes used for synchronization purposes may be added. Profile and level information may also be added.

[0028] To ensure that a file is decryptable, you can add constraints to the creation of the file: - the haptic coded stream starts with the NALu metadata experience, The first DataBand AU is a RAU, The RAU starts with a keyframe (e.g., a wavelet), -MetaData AUs must be repeated periodically (e.g., every second) for random access, -At least one level 0 is required (and at least one level 0 means the current band is the baseline), higher levels indicate supplementary bands that can be skipped for low bitrate applications (a kind of scalability).

[0029] Figures 6-13 show another embodiment of a block-based structure for haptic data according to the present principles. Details of NAL units according to the present principles are provided throughout these figures. In these NAL units, some information elements are added to facilitate encapsulation by network protocols. For example, byte insertion is used to add some bits to have a total number of bits that is a multiple of a byte. A cyclic redundancy check (CRC) is a function calculated on the transmitted bits. If a transmission error occurs, the CRC calculated at the receiver side differs from the transmitted CRC. Emulation bits are bits added when a bit combination in the binary stream equals a reserved number of bits (e.g., "sync bits"). If an error occurs, some bits are added to the bit stream to prevent this error.

[0030] FIG. 6 illustrates a high-level syntax for a block-based structure for haptic data encoded into a data stream in accordance with the present principles. In this embodiment, a NAL unit begins with a header portion that includes, for example, the type of NAL unit and the length of the payload (e.g., in bytes). Adding the payload length to the NAL unit header is useful for network applications. When receiving a NAL unit, an application first reads the header to obtain the number of bits to be read for the payload. This memory size can therefore be used to handle parallel reads and / or to detect bit loss (as a checksum). In accordance with the present principles, the payload includes a metadata portion and a data band portion.

[0031] Figure 7 shows an example of syntax for describing the experience-type NAL units described with reference to Figures 4 and 5. Like any NAL unit, experience NAL units include a header and a payload. These NAL units describe avatars.

[0032] Figure 8 shows an example of syntax for describing a perceptual-type NAL unit as described with reference to Figures 4 and 5. Like any NAL unit, an experiential NAL unit includes a header and a payload. In the example of Figure 8, the metadata NAL unit payload includes a library of effects. In a variant, the effects library is stored in the payload of the band data.

[0033] Figure 9 shows the NAL unit structure proposed herein for describing an effect library. Instead of being included in the payload of band data, the effect library is separated as a dedicated NAL unit. Therefore, the effect library can be transmitted separately from the payload, thus facilitating the parallelization process and increasing error robustness. Additionally, such a separate structure allows for effects to be modified during an experience (as with library updates) and for easier use by editing tools or servers. For example, a generic NAL unit can be stored per artist or application and only needs to be indexed when the effect is used. The way in which the effect library is constructed when separated from the payload is different. Some syntax elements available in the stream are repeated here (within the effect library NAL unit), such as curve type or band type, to ensure that the effect library NAL unit is independent of the current experience.

[0034] Figure 10 proposes an example of a syntax describing the NAL unit structure for encoding the metadata trajectories described with respect to Figures 4 and 5. Like any NAL unit, an empirical NAL unit includes a header and a payload.

[0035] Figure 11 proposes an example of syntax describing the NAL unit structure for encoding the metadata band described with reference to Figures 4 and 5. Like any NAL unit, the empirical NAL unit comprises a header and a payload. Instead of being included in the payload of the band data, the metadata band is separated as a dedicated NAL unit. Thus, the metadata band can be transmitted separately from the payload, thus facilitating parallelization and / or increasing error robustness.

[0036] FIG. 12 shows an example of band data structured as NAL units.

[0037] 3 shows an exemplary architecture of a processing engine 30 that may be configured to implement the present principles. This device may be linked to other devices via a bus 31 and / or via an I / O interface 36.

[0038] The device 30 includes the following elements linked together by a data and address bus 31: a microprocessor 32 (or CPU), for example a DSP (or Digital Signal Processor), -ROM (or Read Only Memory) 33, RAM (or Random Access Memory) 34, - storage device interface 35, an I / O interface 36 for receiving data to be transmitted from an application, and - a power source (not shown in Figure 2), e.g. a battery;

[0039] According to one example, the power source is external to the device. In each of the mentioned memories, the word "register" as used herein may correspond to a small area (a few bits) or a very large area (e.g., an entire program or a large amount of received or decoded data). The ROM 33 contains at least programs and parameters. The ROM 33 can store algorithms and instructions for performing techniques according to the present principles. When switched on, the CPU 32 uploads the program in the RAM and executes the corresponding instructions.

[0040] The RAM 34 contains in registers the programs executed by the CPU 32 and uploaded after switching on the device 30, input data in registers, intermediate data for different states of the methods in registers, and other variables used for the execution of the methods in registers.

[0041] The implementations described herein may be realized in, for example, a method or process, an apparatus, a computer program product, a data stream, or a signal. Even when discussed in the context of only a single form of implementation (e.g., discussed only as a method or device), the discussed feature implementations may also be implemented in other forms (e.g., a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The method may be performed in an apparatus such as a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include, for example, communication devices such as computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users.

[0042] The device 30 is linked, for example, via a bus 31, to a set of sensors 37 and a set of rendering devices 38. The sensors 37 may be, for example, a camera, a microphone, a temperature sensor, an inertial measurement unit, a GPS, a humidity measurement sensor, an IR or UV sensor, or a wind sensor. The rendering devices 38 may be, for example, a display, a speaker, a vibrator, a heat source, a fan, etc.

[0043] By way of example, the device 30 is configured to implement the method according to the present principles and belongs to the set comprising: -Mobile devices, -communication devices, -Gaming devices, -tablet (or tablet computer), -Laptop, -still camera, -Video camera.

[0044] The implementations described herein may be realized in, for example, a method or process, an apparatus, a computer program product, a data stream, or a signal. Even when discussed in the context of only a single form of implementation (e.g., discussed only as a method or device), the implementation of the discussed features may also be implemented in other forms (e.g., a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The method may be performed in an apparatus such as a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices such as, for example, smartphones, tablets, computers, mobile phones, handheld / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users.

[0045] Implementations of the various processes and features described herein may be embodied in a variety of different devices or applications, particularly devices or applications associated with, for example, data encoding, data decoding, view generation, texture processing, and other processing of images and associated texture and / or depth information. Examples of such devices include encoders, decoders, post-processors that process output from decoders, pre-processors that provide input to encoders, video coders, video decoders, video codecs, web servers, set-top boxes, laptops, personal computers, mobile phones, PDAs, and other communication devices. As should be clear, the devices may be mobile and installed in mobile vehicles.

[0046] Additionally, methods may be implemented by instructions executed by a processor, and such instructions (and / or data values ​​produced by an implementation) may be stored on a processor-readable medium such as, for example, an integrated circuit, a software carrier, or other storage device such as, for example, a hard disk, a compact diskette ("CD"), an optical disk (e.g., a DVD, often referred to as a digital versatile disc or digital video disc), a random access memory ("RAM"), or a read-only memory ("ROM"). The instructions may form an application program tangibly embodied on the processor-readable medium. The instructions may be, for example, hardware, firmware, software, or a combination. The instructions may be found, for example, in an operating system, a separate application, or a combination of the two. A processor may therefore be characterized as both, for example, a device configured to execute a process and a device that includes a processor-readable medium (such as a storage device) having instructions for executing a process. Furthermore, a processor-readable medium may store data values ​​produced by an implementation in addition to or in place of instructions.

[0047] As will be apparent to those skilled in the art, implementations may result in a variety of signals formatted to carry information that may be, for example, stored or transmitted. The information may include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal may be formatted to carry, as data, rules for writing or reading the syntax of a described embodiment, or to carry, as data, the actual syntax values ​​written by a described embodiment. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. The signal may be transmitted over a variety of different wired or wireless links, as is known. The signal may be stored on a processor-readable medium.

[0048] Many implementations have been described. Nevertheless, it will be understood that various modifications may be made. For example, elements of different implementations may be combined, supplemented, modified, or deleted to produce other implementations. Additionally, those skilled in the art will understand that other structures and processes may be substituted for those disclosed, with the resulting implementation performing at least substantially the same function in at least substantially the same way to achieve at least substantially the same results as the disclosed implementations. Accordingly, these and other implementations are contemplated by this application.

Claims

1. 1. A method for encoding haptic data of a haptic sequence, comprising: - obtaining binary haptic data representing a haptic effect and metadata describing said haptic effect; - separating the haptic effects into time events and frequency bands and grouping them into trajectories; encoding said trajectories and said metadata, which comprises sensory data pointing to a subset of the trajectories and experiential data pointing to a subset of the sensory data, into an access unit.

2. The method of claim 1 , wherein the access unit is a network abstraction layer unit.

3. 3. The method of claim 1 or 2, wherein the access units are structured according to whether they contain metadata or whether they contain band data.

4. The method of any one of claims 1 to 3, wherein the metadata includes information describing a haptic effects library, an avatar, or a device.

5. 1. A device for encoding haptic data of a haptic sequence, the device comprising a processor, the processor comprising: - obtaining binary haptic data representing a haptic effect and metadata describing said haptic effect; - separating the haptic effects into time events and frequency bands and grouping them into trajectories; A device configured for encoding said trajectories and said metadata comprising sensory data pointing to a subset of the trajectories and experiential data pointing to a subset of the sensory data into an access unit.

6. The device of claim 5 , wherein the access unit is a network abstraction layer unit.

7. 7. A device according to claim 5 or 6, wherein the access units are structured according to whether they contain metadata or whether they contain band data.

8. The device of any one of claims 5 to 7, wherein the metadata includes a haptic effects library, an avatar, or information describing the device.

9. 1. A method for decoding haptic data of a haptic sequence, comprising: - obtaining a set of trajectories and metadata containing sensory data pointing to a subset of the trajectories and experiential data pointing to a subset of the sensory data, encoded in an access unit; - accessing experiential data in a trajectory pointed to by sensory data associated with said experiential data.

10. The method of claim 9 , wherein the access unit is a network abstraction layer unit.

11. 11. The method of claim 9 or 10, wherein the access units are structured according to whether they contain metadata or whether they contain band data.

12. The method of any one of claims 9 to 11, wherein the metadata includes information describing a haptic effects library, an avatar, or a device.

13. 1. A device for decoding haptic data of a haptic sequence, the device comprising a processor, the processor comprising: - obtaining a set of trajectories and metadata containing sensory data pointing to a subset of the trajectories and experiential data pointing to a subset of the sensory data, encoded in an access unit; - A device configured to access experiential data within a trajectory pointed to by sensory data associated with said experiential data.

14. The device of claim 13 , wherein the access unit is a network abstraction layer unit.

15. 15. A device according to claim 13 or 14, wherein the access units are structured according to whether they contain metadata or whether they contain band data.

16. The device of any one of claims 13 to 15, wherein the metadata includes a haptic effects library, an avatar, or information describing the device.

Images