Method and device for sending tactile data

Abstract

A method and apparatus for transmitting haptic data (512) representative of a haptic effect is provided. To achieve this, haptic device data (511) representative of haptic presentation capabilities of a haptic presentation device (51) is acquired. The haptic data (512) is determined by processing received haptic data (500) representative of a haptic effect in accordance with haptic device data (511) to adapt presentation of the haptic effect to haptic presentation capabilities. The haptic data (512) is transmitted to a haptic presentation device (51).

Description

Technical Field

[0001] The present application relates generally to the field of tactile technology, and in particular to the processing of tactile-related data. The present application relates to the communication of tactile-related data between a tactile engine and a tactile device configured to present one or more tactile effects on one or more parts of a human body. The present application also relates to methods and devices for sending tactile data representing tactile effects so that the tactile effects are presented by the tactile device on a part of a person's body. The present application also relates to a communication protocol between a tactile (rendering) engine and a tactile device including one or more tactile actuators. Background Art

[0002] This section is intended to introduce the reader to various aspects of the art, which may be related to various aspects of at least one exemplary embodiment of the present application described and / or claimed below. Such discussion is considered to be helpful in providing background information to the reader so that the reader can better understand various aspects of the present application.

[0003] Haptic technology broadly refers to any technology that recreates the sensation of touch in a user interface by applying force, vibration, motion, and other sensations (e.g., temperature) to provide information to the end user, for example, in addition to visual and audio information when presenting multimedia content.

[0004] Haptic feedback covers a wide range of possible stimulation embodiments, but is mainly divided into haptic haptic technology and kinesthetic haptic technology: haptic haptic feedback (or haptic haptic effects) refers to sensations such as vibration, friction, or micro-deformation, while kinesthetic haptic feedback (or kinesthetic haptic effects) refers to sensations that provide a sense of force, which can stimulate mechanical stimulation as well as stimulation related to the orientation and movement of the body.

[0005] The presentation of tactile feedback or tactile effects is obtained using a tactile device (also called a tactile presentation device), which corresponds to the arrangement of one or more tactile actuators. For example, a vibration tactile effect can be obtained by using an eccentric rotating mass (ERM), a linear resonant actuator (LRA), and a wide bandwidth actuator (such as a voice coil motor (VCM) or a piezoelectric actuator (PZT)). Kinesthetic effects can be presented by an actuator applying a force that hinders the movement of a limb, and such an effect is only felt on muscles and tendons, not on the skin. Other examples of tactile devices include resistive force feedback devices, active force feedback devices, and skin indentation devices.

[0006] There are a wide range of haptic devices with various arrangements of one or more actuators forming the haptic devices. When haptic-related data is gathered in a container to be sent to a haptic engine (also referred to as a haptic rendering engine) for rendering a haptic effect on a human body, the type of haptic rendering device that will be used to render the haptic effect on a human body is generally unknown, which may cause the haptic engine and one or more associated haptic rendering devices to incorrectly render the haptic effect. Summary of the invention

[0007] The following section presents a brief overview of at least one exemplary embodiment to provide a basic understanding of some aspects of the present application. This overview is not an extensive review of the exemplary embodiments. It is not intended to be used to identify the key or important elements of the embodiments. The following overview merely presents certain aspects of at least one exemplary embodiment in a brief form as a prelude to a more detailed description provided elsewhere in this document.

[0008] According to a first aspect of the present application, there is provided a method for sending tactile data representing a tactile effect, the method comprising:

[0009] - obtaining haptic device data representing haptic rendering capabilities of the haptic rendering device;

[0010] - determining haptic data by processing received haptic data representing the haptic effect according to the haptic device data to adapt the rendering of the haptic effect to the haptic rendering capabilities; and

[0011] -Send haptic data.

[0012] In an exemplary embodiment, the received haptic data includes a set of information including:

[0013] - target body part information representing the target body part of the body model targeted by the haptic effect, and / or

[0014] - reference haptic device information representing a reference haptic device intended to render the haptic effect, and / or

[0015] - rendering information representing rendering parameters of the haptic effect,

[0016] Processing includes:

[0017] - comparing the haptic device data to the information set; and

[0018] - When a result of the comparison represents at least a difference between the haptic device data and the information set, adjusting the received haptic data according to the result.

[0019] In an exemplary embodiment, the processing includes:

[0020] - comparing the target body part information with first information of a body part contained in the haptic device data and representing a body model associated with the haptic rendering device; and

[0021] -When the result of the comparison represents a difference between the first information and the target body part information, adjusting the target body part information according to the result.

[0022] In a further exemplary embodiment, the body model comprises a first plurality of body parts and groups of a second plurality of body parts, each group of the groups of the second plurality of body parts comprising at least a body part of the first plurality of body parts,

[0023] Adjusting is performed based on the first information, the target body part information, and relationship information representing a relationship between at least some of the first plurality of body parts and at least some of the groupings of the second plurality of body parts.

[0024] In another exemplary embodiment, the reference haptic device information includes spatial distribution information representing a spatial distribution of a haptic actuator group included in the reference haptic device in a determined multi-dimensional space and position information representing a position in the determined multi-dimensional space.

[0025] Processing includes:

[0026] - comparing the spatial distribution information with second information contained in the haptic device data and representing the spatial distribution of the set of haptic actuators included in the haptic rendering device in the determined multidimensional space;

[0027] - determining at least a tactile actuator in a tactile actuator group of a tactile presentation device according to the spatial distribution information, the second information and the position information; and

[0028] - adjusting the reference haptic device information according to at least a determined haptic actuator of the set of haptic actuators of the haptic rendering device.

[0029] In a further exemplary embodiment, the reference haptic device information further includes first distance information representing a distance relative to the location, the processing further includes determining second distance information based on the spatial distribution information and the second information, and the adjusting includes replacing the first distance information with the second distance information.

[0030] In additional exemplary embodiments, the presentation parameter includes a presentation parameter representing frequency, a presentation parameter representing amplitude and / or a presentation parameter representing phase.

[0031] Processing includes:

[0032] - comparing the presentation information with information representative of the type of tactile presentation device;

[0033] - When a result of the comparison represents at least a difference between the presentation capabilities associated with the type and the presentation information, filtering at least part of the presentation parameters according to the result.

[0034] In another exemplary embodiment, the presentation parameter comprises a presentation parameter representing an application program interface call,

[0035] Processing includes:

[0036] - comparing the presentation information with information representing the capability of the tactile presentation device to process the application program interface;

[0037] - When the result of the comparison shows that the tactile presentation device cannot process the application program interface, converting at least part of the presentation parameters according to the result, the conversion being based on information representing the type of the tactile presentation device.

[0038] In an exemplary embodiment, the haptic device data includes at least one of the following:

[0039] - first data representing the number of body parts targeted by the tactile presentation means; and / or

[0040] - second data representing an identifier for each body part; and / or

[0041] - third data representing a set of tactile perception patterns, the set of tactile perception patterns comprising at least a tactile perception pattern; and / or

[0042] - fourth data representing the number of tactile actuators of the tactile presentation device; and / or

[0043] - fifth data representing the spatial distribution of the tactile actuators; and / or

[0044] - sixth data representing whether the tactile actuators are all of the same type; and / or

[0045] - seventh data representative of the actuator type of each actuator; and / or

[0046] - eighth data representing tactile presentation parameters of each actuator; and / or

[0047] - ninth data representing the capability of processing the application program interface; and / or

[0048] - Tenth data representing the format of the haptic device data.

[0049] In an exemplary embodiment, the third data is used for each body part, and the fourth data, the fifth data, the sixth data, the seventh data, and the eighth data are used for each tactile perception pattern in the tactile perception pattern set.

[0050] In a further exemplary embodiment, the haptic device data is received from a haptic presentation device.

[0051] According to a second aspect of the present application, a device for transmitting haptic data representing a haptic effect is provided, wherein the device comprises a memory associated with at least a processor, the processor being configured to implement the method according to the first aspect of the present application.

[0052] According to a third aspect of the present application, a computer program product comprising instructions is provided. When the program is executed by one or more processors, the one or more processors are caused to execute the method according to the first aspect of the present application.

[0053] According to a fourth aspect of the present application, a non-transitory storage medium is provided, carrying program code instructions for executing the method according to the first aspect of the present application.

[0054] The specific nature of at least one exemplary embodiment and other objects, advantages, features and uses of the at least one exemplary embodiment will become apparent from the following description of the examples taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Reference will now be made, by way of example, to the accompanying drawings which show exemplary embodiments of the present application, in which:

[0056] Figure 1 shows a schematic representation of a body model according to at least one exemplary embodiment;

[0057] Figure 2 According to at least one exemplary embodiment, Figure 1 Schematic hierarchical representation of the body model;

[0058] Figure 3 shows a schematic representation of a spatial arrangement of haptic actuators of a haptic device in a determined multi-dimensional space according to at least one exemplary embodiment;

[0059] Figure 4 According to at least one exemplary embodiment, Figure 3 The spatial arrangement of the tactile device is converted into the spatial arrangement of another tactile device;

[0060] Figure 5 An example of a communication scheme for haptic-related data according to at least one example embodiment is shown;

[0061] Figure 6 A schematic block diagram showing steps of a method for sending tactile data according to at least one exemplary embodiment;

[0062] Figure 7 A schematic block diagram of an example of a system in which various aspects and exemplary embodiments are implemented is illustrated.

[0063] Like reference numerals may be used in different drawings to denote similar components. DETAILED DESCRIPTION

[0064] At least one exemplary embodiment will be described more fully below with reference to the accompanying drawings, in which examples of at least one exemplary embodiment are illustrated. However, the exemplary embodiments may be embodied in many alternative forms and should not be construed as being limited to the examples set forth herein. Therefore, it should be understood that there is no intention to limit the exemplary embodiments to the specific forms disclosed. On the contrary, the disclosure is intended to cover all modifications, equivalents, and alternatives that fall within the spirit and scope of the present application.

[0065] At least one of the various aspects generally relates to methods and apparatus for transmitting haptic data representing a haptic effect for rendering the haptic effect by a haptic device receiving the haptic data, such haptic device hereinafter referred to as a haptic rendering device.

[0066] The haptic rendering device is configured to render a haptic effect from data describing the haptic effect received, for example, from a haptic engine. The haptic rendering device comprises a haptic actuator group, which comprises one or more haptic actuators. The haptic rendering device is associated with one or more body parts of a person, i.e., the haptic actuators of the haptic rendering device are arranged or located on the body part or parts in such a way that the body part or parts feel the haptic effect.

[0067] The haptic device data representing the haptic rendering capabilities of the haptic rendering device is obtained, for example, by a haptic engine. The haptic device data may be retrieved, for example, from a memory, received from the haptic rendering device, or from a remote device (eg, a server or a computer).

[0068] The haptic data to be sent by the haptic engine to the haptic rendering device is determined or obtained by processing haptic data, e.g. received in a container by the haptic engine and representing a haptic effect to be rendered on one or more body parts of the user (according to the haptic device data).

[0069] Such processing enables the presentation of the haptic effect described in the received haptic data to be adapted, converted or adjusted to the haptic presentation capabilities of the haptic presentation device.

[0070] The tactile data obtained from the processing is sent to a tactile presentation device to present a tactile effect on a person wearing or touching the tactile presentation device.

[0071] A container may correspond, for example, to a bitstream, a network packet, or a file (eg, a text file or a binary file).

[0072] A binary structure such as a binary file (e.g., ISOBMFF file) is an example of a container. Binary files define a syntax for parsing and understanding the file structure. They have a beginning and an end, and usually hold self-contained information. Binary files are usually transmitted and stored as a whole.

[0073] However, for transmission over a network, such as using the Synchronous Collaboration Transport Protocol (SCTP), Interactive Real-Time Protocol (IRTP), Efficient Transport Protocol (ETP), Real Time Network Protocol (RTNP), or Real Time application-level Protocol for distributed Interactive media (RTP / I), binary files can be further segmented into smaller file units, such as transport packets.

[0074] Figure 1 Illustrated is a schematic representation of a body part 10 according to at least one exemplary embodiment.

[0075] Figure 1 The body model 10 represents a human body and is used to identify various parts of the human body.

[0076] The body model 10 includes a first grouping of body parts and a second grouping of body parts.

[0077] Each body part in the first set of body parts belongs to one or more of the groups in the second set of body parts, and each of the groups in the second set of body parts includes one or more body parts in the groups in the second set of body parts.

[0078] The body parts in the first set of body parts may, for example, correspond to the groups in the second set of groupings of body parts.

[0079] For example, the second set of body part groupings includes the following groupings:

[0080] - a grouping called "head" 101, including the following body parts: head and neck;

[0081] - a group called "torso" 102, comprising only one body part, the torso;

[0082] - a group called "waist" 103, comprising only one body part, namely the waist;

[0083] - a grouping called "legs", including the following body parts: upper leg 104, lower leg 105 and foot 106;

[0084] - a grouping called "arm", including the following body parts: upper arm 107, lower arm 108 and hand;

[0085] - a grouping called "hand" 12, comprising the following body parts: palm 109 and fingers;

[0086] - a grouping called "fingers", including the following body parts: thumb 110, index finger 111, middle finger 112, ring finger 113 and little finger 114;

[0087] - A grouping called "knuckles" includes the following body parts: first knuckle, second knuckle, and third knuckle;

[0088] - A group called "foot", which includes the following body parts: plant and toes;

[0089] - a group called "lower limbs", corresponding to the lower part of the body model and including the following body parts: waist and legs;

[0090] - a group called "upper limbs", corresponding to the higher part of the body model 10 and comprising the following body parts: head, chest and arms;

[0091] a group called "right part", corresponding to the right half of the body model 10 and comprising the following body parts: head, chest, waist, right arm and right leg;

[0092] a group called “left part” 11 , corresponding to the left half of the body model 10 and comprising the following body parts: head, chest, waist, left arm and left leg;

[0093] a group called “front”, corresponding to the front of the body model 10 and comprising the following body parts: front head, front chest, front waist, front arms and front legs;

[0094] a group called "back", corresponding to the back of the body model 10, and including the following body parts: back of the head, back of the chest, back of the waist, back of the arms and back of the legs; and

[0095] A group called "all" corresponds to the overall body model 10 and brings together all groups and body parts, namely: head, chest, waist, arms and legs.

[0096] Of course, the groups and body parts constituting the body model are not limited to the above examples. The number of groups and the number of body parts are also not limited to the above examples and can be any number.

[0097] Each grouping of body parts can be identified with, for example, an identifier corresponding to a grammatical element, such as one of the following grammatical elements: "all", "left", "right", "front", "back", "upper limb", "lower limb", "head", "torso", "waist", "legs", "arms", "hands", "feet", "fingers / toes".

[0098] Similarly, each body part may be an identifier having, for example, an identifier corresponding to a grammatical element, which may be the same as the grammatical element identifying the grouping when the body part also corresponds to a grouping. For example, the following grammatical elements may be used to identify body parts when necessary: ​​"head", "torso", "waist", "leg", "arm", "hand", "foot", "fingers / toes", "thumb", "index finger", "middle finger", "ring finger", "little finger", "palm", "sole", "knuckle", "first knuckle", "second knuckle", "third knuckle", "upper arm / leg", "lower arm / leg".

[0099] According to a variant, each body part and each grouping may be identified by an identifier corresponding to a numerical value (for example: a binary value or a hexadecimal value).

[0100] The one or more identifiers of the one or more groups are used to signal (i.e., indicate) in the container which body part or parts are targeted by the haptic effect. If necessary, one or more target body parts are signaled using one or more identifiers of the one or more target body parts in addition to the one or more identifiers of the one or more groups.

[0101] In order to signal one or more body parts (referred to as target body parts) in a container containing information to be sent, for example, from a transmitter to a receiver, from a tactile data generator to a tactile rendering engine, or from an encoder to a decoder, data identifying the target body parts is written or encoded into the container with reference to each group containing the target body parts.

[0102] For example, based on the example where the target body part corresponds to the third knuckle of each finger of two hands, the data used to signal this information may correspond to: "third knuckle of hand".

[0103] According to another example, in order to signal the 3 knuckles of the left hand thumb, the data for signaling this information may correspond to: “left hand thumb”.

[0104] According to another example, in order to identify the third knuckle of the thumb of the right hand, the first data for signaling this information may correspond to:

[0105] "Right hand thumb third" or "Right hand thumb third knuckle", depending on how the body parts are grouped in the body model.

[0106] The use of groupings of body parts, taking advantage of the symmetry of the body model 10 (right / left and front / back), enables clear and unambiguous identification of each body part of the body model that may be targeted by a haptic effect.

[0107] In addition, the use of groupings of body parts enables efficient encoding or signaling of information identifying the targeted body parts, particularly when the targeted body parts include multiple body parts of the same type (e.g., knuckles, both hands, the same part of the leg), as the amount of data used to encode or signal the information is reduced.

[0108] According to a specific embodiment, the signal transmission of the targeted body part located on a specific side of the body model (i.e., on the left-hand side or on the right-hand side) is obtained by writing data identifying the side (i.e., the left side or the right side) with reference to each group of "left side" and "left side" of each body part into a container.

[0109] According to a further specific embodiment, the signal transmission of the targeted body part located on a specific face of the body model (i.e., the front or the back (or dorsal) face) is obtained by writing data identifying the face (i.e., the front or the back) with reference to each grouping of "front" and "back" of each body part into a container.

[0110] According to a further specific embodiment, data for signaling a target body part (or for encoding data identifying a target body part) includes a first identifier identifying the target body part (e.g., a grammatical element describing the target body part) and a second identifier identifying one or more groups including the target body part (e.g., one or more grammatical elements describing the one or more groups including the target body part).

[0111] According to another specific embodiment, the data for signaling the target body part comprises a third identifier identifying a logical operator. The logical operator corresponds, for example, to:

[0112] - logical operator NOT; and / or

[0113] - logical operator AND; and / or

[0114] - Logical operator OR; and / or

[0115] - logical operator XOR; and / or

[0116] - Logical operator NOR.

[0117] For example, the upper arm and lower arm of one or both arms may be signaled and / or encoded into the container with the following data: "left arm NOT hand" or left arm, and "arm NOT hand" for both arms.

[0118] The use of the logical operator NOT enables the exclusion of one or more body parts from a grouping. The exclusion is signaled using the identifier of the logical operator NOT and the identifier of the excluded body part. In the above example, the body part or group "hand" is excluded from the group "arm" to signal and / or encode the identification of the lower arm and upper arm of the arm.

[0119] Another way to signal this information is: "ARM UP AND LOWER".

[0120] The use of logical operators may be more efficient than identifying each target body part individually. For example, to identify all hands except the first knuckle of each finger, the following first data may be used to signal and / or encode information into a container: "hand NOT first knuckle".

[0121] The different embodiments described above can be combined with each other in any combination.

[0122] Figure 2 Illustrated is a schematic representation 20 of the relationship between groupings and body parts of the body model 10 according to at least one exemplary embodiment.

[0123] Figure 2 Portions of the groupings and body parts of the body model 10 and connections or links between the groupings and / or body parts are shown. Figure 2 The associations and correlations between the groups and body parts are shown in a hierarchical manner.

[0124] When a group (e.g., group "A") or a body part is included in another group (e.g., group "B"), the group (e.g., group "A") or body part is said to be subordinate to or belong to the other group (e.g., group "B"). Figure 2 , it is shown in a hierarchical manner with two boxes belonging to two different layers (corresponding to the layers of the hierarchical levels) interconnected by a line.

[0125] For example, the box referenced 201 corresponds to the grouping “ALL” and belongs to the highest hierarchical level (eg: level 0 ) in the structure or tree 20 .

[0126] The four boxes 211, 212, 213, 214 form a layer having a hierarchical level (e.g., level 1) directly below level 0. For example, box 211 represents a group "left", box 212 represents a group "right", box 213 represents a group "front", and box 214 represents a group "back". Each of the group "left", the group "right", the group "front", and the group "back" belongs to the group "all", or in other words, the group "all" includes the group "left", the group "right", the group "front", and the group "back".

[0127] As shown in the structure or tree 20, each of the groupings "left", "right", "front", and "back" includes groups represented by boxes with reference numerals 221, 222, 223, 224, and 225. Boxes 221, 222, 223, 224, and 225 form a layer having a hierarchical level (e.g., level 2) directly below level 1.

[0128] Box 221 represents, for example, the group "head", box 222 represents, for example, the group "chest", box 223 represents, for example, the group "arm", box 224 represents, for example, the group "leg", and box 225 represents, for example, the group "waist".

[0129] In a layer having a hierarchical level directly below level 2 (e.g., level 3), three boxes 231, 232, 233 are connected to (and subordinate to) box 223. The groups and / or body parts represented by boxes 231 to 233 are included in the group "arm" 223. Box 231 represents, for example, the body part "upper arm", box 232 represents, for example, the body part "lower arm", and box 233 represents, for example, the group "hand" (which may also correspond to a body part).

[0130] In a layer having a hierarchical level directly below level 3 (e.g., level 4), two boxes 241 and 242 are connected to (and subordinate to) box 233. The groups and body parts represented by boxes 241 and 242 are included in the group "hand" 233. Box 241 represents, for example, the body part "palm", and box 242 represents, for example, the group "fingers".

[0131] In a layer having a hierarchical level directly below level 4 (e.g., level 5), five boxes 251, 252, 253, 254, and 255 are connected to (and subordinate to) box 242. The respective groups represented by boxes 251 to 255 are included in the group "fingers" 242. Box 251 represents, for example, a group "thumbs" (which may also correspond to body parts according to variants), box 252 represents, for example, a group "index fingers" (which may also correspond to body parts according to variants), box 253 represents, for example, a group "middle fingers" (which may also correspond to body parts according to variants), box 254 represents, for example, a group "ring fingers" (which may also correspond to body parts according to variants), and box 255 represents, for example, a group "little fingers" (which may also correspond to body parts according to variants).

[0132] When boxes 251 to 255 represent groups, namely, group "thumb", group "index finger", group "middle finger", group "ring finger", and group "pinky finger", the structure or tree 20 includes an additional layer having a hierarchical level directly below level 5 (e.g., level 6), which includes boxes 261, 262, and 263, which are connected to (and subordinate to) each of the boxes 251 to 255. For clarity, in Figure 2 Only the connections between blocks 261 to 263 on the one hand and block 254 on the other hand are shown.

[0133] Box 261 represents, for example, the body part "first knuckle", box 262 represents, for example, the body part "second knuckle", and box 263 represents, for example, the body part "third knuckle".

[0134] According to a variant, the body part "first knuckle", the body part "second knuckle" and the body part "third knuckle" are grouped in the group "knuckles", the group "knuckles" are connected to (or subordinate to) each group in the groups "fingers" 251 to 255, and the body part "first knuckle", the body part "second knuckle" and the body part "third knuckle" are connected to (or subordinate to) the group "knuckles".

[0135] Figure 3 The figure shows a schematic representation of the arrangement of a set of actuators of a haptic device on a body part 3 in a determined multi-dimensional space according to at least one exemplary embodiment.

[0136] Body part 3 and body model (e.g. Figure 1 Corresponding to the elements of the body model 10).

[0137] exist Figure 3 In the figure, body part 3 is represented by a cylinder, and only the front of the cylinder is visible. Of course, the form of the body part is not limited to the cylinder, but can be any form.

[0138] according to Figure 1 By way of non-limiting example, a haptic device associated with a body part includes 40 haptic actuators, Figure 3 Only 20 tactile actuators 300 to 304, 310 to 314, 320 to 324 and 330 to 334 are visible.

[0139] Figure 3 The 20 tactile actuators of the tactile device are associated with the front side of the body part 3 , and the 20 tactile actuators (not shown) of the tactile device are associated with the back (or dorsal) side of the body part 3 .

[0140] according to Figure 3 In the example of, the spatial distribution of each haptic actuator is signaled into a container (intended to be sent to a decoder and / or a haptic (rendering) engine) relative to a Cartesian 3D space (X, Y, Z), which can be associated with body part 3 or with a body model to which body part 1 belongs, for example. According to this example, the haptic device signaled in the container to be sent to the haptic engine can be called a reference haptic device, because its corresponding haptic device is intended or attempted to present a haptic effect. The reference haptic device can be different from the haptic rendering device actually used to present the haptic effect on the person wearing the haptic rendering device.

[0141] Spatial distribution, for example, is represented by a vector (in [N x ,N y ,N z ]) is transmitted in the following signaling format:

[0142] -First information (the first element of the vector 'N x ') represents the number of tactile actuators according to a first dimension of the 3D space (according to the X-axis, which represents, for example, the vertical axis); N x may be viewed as the resolution of the haptic actuators according to the first dimension, i.e. the number of haptic actuators per haptic device according to the first dimension;

[0143] - Second information (the second element of the vector 'N y ') represents the number of tactile actuators according to the second dimension of the 3D space (according to the Y axis, which represents, for example, the horizontal axis); N y may be viewed as the resolution of the haptic actuators according to the second dimension, i.e. the number of haptic actuators per haptic device according to the second dimension; and

[0144] - The third information (the third element of the vector 'N z ') represents the number of tactile actuators according to the third dimension of the 3D space (according to the Z axis, which represents, for example, the depth axis); N zIt can be seen as the resolution of the haptic actuators according to the third dimension, ie the number of haptic actuators per haptic device according to the third dimension.

[0145] The information representing the maximum resolution of the reference haptic device (ie, the total number of haptic actuators of each haptic device) can be obtained from the second data in the form of N x *N y *N z get.

[0146] The spatial distribution of the tactile actuators of the reference tactile device can be signaled or encoded as follows: [5,4,2] to obtain a total of 40 tactile actuators (5*4*2) according to 5 tactile actuators according to the X-axis, 4 tactile actuators according to the Y-axis, and 2 tactile actuators according to the Z-axis.

[0147] For each face of the body part, the haptic actuators are arranged according to 4 columns and 5 rows.

[0148] The first column (on the left hand side) includes five haptic actuators 300, 301, 302, 303, and 304. The second column is to the right of the first column and includes five haptic actuators 310, 311, 312, 313, and 314. The third column is to the right of the second column and includes five haptic actuators 320, 321, 322, 323, and 324. The fourth column is to the right of the third column and includes five haptic actuators 330, 331, 332, 333, and 334.

[0149] The tactile actuators are arranged according to a matrix on each side of the body part, the matrix comprising 5 rows and 4 columns.

[0150] According to another example, the spatial distribution of the haptic device includes a single actuator signaled or encoded as [1, 1, 1].

[0151] According to a specific and non-limiting embodiment, the vector [N x ,N y ,N z ]'s third element 'N z 'Less than or equal to 2, that is, N z =1 or N z = 2. According to this embodiment, the maximum number of layers of actuators according to the Z axis (depth) is equal to 2, wherein the first layer corresponds to the front and the second layer corresponds to the back of the body part (or haptic device).

[0152] The signaling / encoding spatial distribution as explained previously enables, for example, a decoder and / or a haptic (rendering) engine of a receiving container to obtain information about a reference haptic device via the specific spatial distribution of one or more haptic actuators, thereby enabling identification of the reference haptic device.

[0153] Furthermore, the signaling / encoding spatial distribution as explained previously enables a decoder and / or a haptic (rendering) engine of the receiving container to adapt the haptic effect to be presented by a haptic rendering device (used by a person to whom the haptic effect is presented, which device may be different from a reference haptic device) to the specific spatial distribution of the one or more haptic actuators signaled in the container.

[0154] According to a specific embodiment, in addition to the second data, third data representing a specific position in the 3D space is added to the container.

[0155] According to a first example, the specific position represents the position of a specific haptic actuator in a group of haptic actuators of a reference haptic device.

[0156] According to a second example, the specific location represents, for example, a point between 2 haptic actuators in a haptic actuator group of a reference haptic device.

[0157] The third data is added to the container, for example when the haptic device comprises a plurality of actuators (e.g. when the spatial distribution is signaled / encoded with data different from [1,1,1]) and / or when only some of the haptic actuators of the (reference) haptic device are intended for rendering the haptic effect.

[0158] According to a first example, the position or orientation of a specific haptic actuator is expressed as a vector (in [C x ,C y ,C z ]) is transmitted or encoded in the following signaling format:

[0159] -First information (the first element of the vector 'C x ') represents the index of the row / line to which the specific tactile actuator belongs, i.e. according to the first dimension of the 3D space (according to the X-axis);

[0160] - Second information (the second element of the vector 'C y ') represents the index of the column to which the specific tactile actuator belongs, i.e. according to the second dimension of the 3D space (according to the Y axis); and

[0161] - The third information (the third element of the vector 'C z ') represents the index of the face (front or back) to which the tactile actuator belongs, i.e. according to the third dimension of the 3D space (according to the Z axis).

[0162] For example, to identify Figure 3 The tactile actuator referenced by 323 in the figure, the third information corresponds to: [3, 2, 0]; in order to identify Figure 3For the tactile actuator referenced as 311, the third information corresponds to: [1, 1, 0]; and in order to identify a tactile actuator located at the same orientation as the tactile actuator but behind / backward, the third information corresponds to: [3, 2, 1].

[0163] Therefore, by signaling the spatial distribution of the reference haptic devices by identifying a specific haptic actuator (eg, haptic actuator 323 ), the signaling / encoding of information corresponds to: [5, 4, 2][3, 2, 0].

[0164] According to a second example, the position of a point is represented by a vector (with [C x ,C y ,C z ]) is signaled or encoded as follows:

[0165] The first information (the first element of the vector 'C x ') represents a floating point value corresponding to the position of a point according to a first dimension (according to the X-axis) of the 3D space; the first information is, for example, included between the minimum index of the row / line and the maximum index of the row / line;

[0166] The second information (the second element of the vector 'C y ') represents a floating point value corresponding to the position of a point according to a second dimension (according to the Y axis) of the 3D space; the second information is, for example, contained between the minimum index of the column and the maximum index of the column; and

[0167] The third information (the third element of the vector 'C z ') represents a floating point value corresponding to the orientation of a point according to a third dimension (according to the Z axis) of the 3D space; the third information is contained between 0 and 1, for example.

[0168] According to another specific embodiment, in addition to the third data, fourth data representing a distance relative to a specific location identified by the third data (ie, a specific haptic actuator or a point between two haptic actuators) is added to the container. The distance is defined in 3D space.

[0169] For example, when presenting a haptic effect through a grouping of haptic actuators, incorporating the distance enables identification of the grouping of haptic actuators in the group of haptic actuators forming a reference haptic device.

[0170] Adding the fourth data enables optimizing the encoding of information by minimizing the amount of data used to signal / encode the information. Using distances expressed in 3D space is more efficient than identifying each tactile actuator in the tactile actuator group individually via the third data as explained above for a specific tactile actuator.

[0171] The distance to a specific haptic actuator is represented by a vector (in [Rx ,R y ,R z ]) is transmitted or encoded in the following signaling format:

[0172] -First information (the first element of the vector 'R x ') represents the number of selected tactile actuators that deviate from the position of a specific tactile actuator or point according to a first dimension of 3D space (according to the X-axis);

[0173] - Second information (the second element of the vector 'R y ') represents the number of selected tactile actuators that deviate from the position of a particular tactile actuator or point according to a second dimension of 3D space (according to the Y-axis); and

[0174] - The third information (the third element of the vector 'R z ') represents the number of selected tactile actuators that deviate from the position of a specific tactile actuator or point according to the third dimension of 3D space (according to the Z axis).

[0175] For example, in order to identify the haptic actuator group including the haptic actuators 322 , 323 , and 324 by using the third information identifying the specific haptic actuator 323 , the distance may be signaled or encoded as: [1, 0, 0].

[0176] Therefore, if [C x ,C y ,C z ] corresponds to third data identifying the position of a specific tactile actuator, and [R x ,R y ,R z ] Corresponding to the fourth data defining the distance, the tactile actuator group defined as presenting the tactile effect will include a tactile actuator having a position [P x ,P y ,P z ] of each tactile actuator, the orientation [P x ,P y ,P z ] verified as:

[0177] C x -R x ≤P x ≤C x +R x ;and

[0178] C y -R y ≤P y ≤C y +R y ;and

[0179] C z-R z ≤P z ≤C z +R z .

[0180] According to a further embodiment, data representing the haptic animation is signaled and / or encoded into the container.

[0181] A haptic animation corresponds to a haptic effect or a series of haptic effects presented via a plurality of haptic actuators according to a path, such as defined by a starting or initial point and an ending or termination point.

[0182] The haptic animation may correspond to a haptic effect or to a series of haptic effects distributed over one or more parts of the user's body that receive such haptic effect(s) and presented via a series of haptic actuators arranged on one or more parts of the user's body.

[0183] The starting point corresponds to, for example, a first specific haptic actuator or to a first haptic actuator group. The ending point corresponds to, for example, a second specific haptic actuator or to a second haptic actuator group. The first haptic actuator and the second haptic actuator belong to, for example, the same haptic device or to two different haptic devices.

[0184] As explained previously, a first specific haptic actuator (or a first grouping of haptic devices) is advantageously signaled into the container using the second data and optionally the third and fourth data.

[0185] As explained previously, a second specific haptic actuator (or a second grouping of haptic devices) is advantageously signaled into the container using the second data and optionally the third and fourth data.

[0186] The haptic (rendering) engine of the receiving container parses the data contained in the container to read data representing the haptic animation, specifically data representing the starting point and the ending point (ie, the first haptic actuator and the second haptic actuator).

[0187] The haptic (rendering) engine determines or calculates a path of haptic actuators based on the first haptic actuator and the second haptic actuator, the path starting from the first haptic actuator and ending at the second haptic actuator. The path includes multiple haptic actuators belonging to one or more haptic devices arranged on different parts of the user's body that receive the haptic effect.

[0188] The path is further determined according to the haptic effect to be presented, such as according to the duration of the haptic effect, according to the magnitude of the haptic effect at the starting point, and according to the magnitude of the haptic effect at the ending point.

[0189] The multidimensional space for defining data representing the target haptic device is not limited to Figure 3 , i.e., 3D space (X, Y, Z), but is extended to any multidimensional space, such as 2D space (θ, Y), such as a 2D multidimensional space corresponding to a polar 2D space.

[0190] According to a specific embodiment, the spatial distribution of the reference haptic devices is defined or represented by data representing active areas in a set of areas, each active area including one or more haptic actuators.

[0191] According to this embodiment, the haptic (rendering) engine receives the following data from a reference haptic device to describe the spatial distribution of the haptic actuators of the reference haptic device:

[0192] - data representing a set of regions of a reference tactile device: the data representing, for example, a matrix having 'N1' (N1 is an integer) regions or cells, the matrix being defined with 'C1' (C1 is an integer) columns and 'R1' (R1 is an integer) rows, N1 being equal to C1*R1; and

[0193] - Data identifying one or more regions or one or more cells in the matrix containing tactile actuators, a region or cell containing one or more tactile actuators being referred to as an active region or an active cell.

[0194] For example, the number of haptic actuators per active area is not defined in the container received by the haptic (rendering) engine. According to this example, Figure 3 Each of the elements 300, 301, 302, 303 and 304, 310, 311, 312, 313 and 314, 320, 321, 322, 323 and 324, and 330, 331, 332, 333 and 334 illustrated in the figure can correspond to an active area and represent a single tactile actuator or a grouping of tactile actuators, the number of tactile actuators for each active area is not contained in the container and is not known to the tactile (rendering) engine.

[0195] According to a variant, the number of tactile actuators per active area is defined in a container received by the tactile (rendering) engine, this information being encoded in data representing the spatial distribution of the tactile actuators.

[0196] When only a portion of the active areas of a reference haptic device are intended for rendering haptic effects, data representing the active areas targeted for rendering haptic effects is encoded and / or signaled into a container to be sent to a haptic (rendering) engine, with each active area identified, for example, by an index of a column and an index of a row containing the active area.

[0197] Figure 4Transforming from data representing a spatial distribution of haptic actuators of a first haptic device to data representing a spatial distribution of haptic actuators of a second haptic device according to at least one example embodiment is illustrated.

[0198] Figure 4 The left side shows a spatial distribution 41 of haptic actuators of a first haptic device, and the right side shows a spatial distribution 42 of haptic actuators of a second haptic device different from the first haptic device.

[0199] The first haptic device corresponds, for example, to a reference haptic device, data representative of which are signaled or encoded into a container sent to the haptic (rendering) engine. The second haptic device corresponds, for example, to a haptic rendering device, i.e. a haptic device implemented on the user side, for rendering the haptic effect described by the data contained in the container received by the haptic (rendering) engine.

[0200] according to Figure 4 As a non-limiting example, the haptic actuator of the first haptic device and the haptic actuator of the second haptic device are associated with or arranged on the same body part.

[0201] The spatial distribution 41 of the first tactile device is, for example, represented by the following second data: x ,N1 y ,N1 z ] defined in 3D space.

[0202] The spatial distribution 42 of the second tactile device is, for example, represented by the following second data: x ,N2 y ,N2 z ] defined in 3D space.

[0203] The first tactile device corresponds to, for example, Figure 3 A haptic device is described which comprises 40 haptic actuators distributed in a space defined by [5, 4, 2].

[0204] The second tactile device includes, for example, six tactile actuators, of which two tactile actuators 401 and 411 are arranged at the upper front portion and two tactile actuators ( Figure 4 ) is arranged in the upper back part, and 2 tactile actuators 400, 410 are located in the side body part.

[0205] The second haptic device includes 6 haptic actuators distributed in a space defined by [2, 2, 2].

[0206] The container received by the haptic (rendering) engine or decoder may include the following data or information:

[0207] - first data representing the spatial distribution of the first tactile device, namely [N1 x ,N1 y ,N1 z ];

[0208] - second data representing the determined position (ie, a specific haptic actuator or point in the haptic actuator group or in the first haptic device in the 3D space), namely [C1 x ,C1 y ,C1 z ];and

[0209] - The third data representing the distance, i.e. [R1 x ,R1 y ,R1 z ].

[0210] When a second tactile device associated with the tactile rendering engine to render the tactile effect is different from the first tactile device that is signaled into the container, the tactile rendering engine converts the second data and the third data to determine which tactile actuator(s) of the second tactile device correspond to the tactile actuators of the first tactile device identified by the second data and the third data.

[0211] To achieve this purpose, the tactile rendering engine converts the second data and the third data contained in the container into a determined tactile actuator (denoted as [C2 x ,C2 y ,C2 z ]) and the distance representing the second tactile device (denoted as [R1 x ,R1 y ,R1 z ]).

[0212] Parameter C2 x 、C2 y and C2 z For example, it is obtained through the following equation: The tactile rendering engine knows that N2 x 、N2 y and N2 z (These parameters are, for example, stored in the memory of the haptic rendering engine or retrieved from a second haptic device connected to the haptic (rendering) engine) and the haptic (rendering) engine retrieves N1 from the received container x 、N1 y and N1 z , C1 x 、C1 y and C1 z and R1 x 、R1 y and R1 z :

[0213]

[0214] Applying equations 1 to 6 to Figure 4 A non-limiting example of where [N1 x, N1 y ,N1 z ]=[5,4,2]、[C1 x ,C1 y ,C1 z ]=[3,2,0](tactile actuator 123), [R1 x ,R1 y ,R1 z ]=[1,1,0] and [N2 x, N2 y ,N2 z ]=[2,2,2], we get [C2 x ,C2 y ,C2 z ]=[1,1,0](haptic actuator 311) and [R2 x ,R2 y ,R2 z ]=[0,0,0].

[0215] Equations 1 to 6 are for example stored in a memory of a haptic (rendering) engine. According to a variant, equations 1 to 6 are encoded to be included and transmitted with data representative of the first haptic device.

[0216] Naturally, the equations used to calculate the transformation depend on the multidimensional space used to represent the data.

[0217] According to a specific embodiment, the spatial distribution 42 of the second haptic device is defined or represented by data representing active areas in a set of areas, each active area comprising one or more haptic actuators.

[0218] According to this embodiment, the haptic (rendering) engine receives the following data from the second haptic device to describe the spatial distribution 42 of the haptic actuators:

[0219] - data representing a set of regions associated with the body part to which the second tactile device is associated: for example, the data represents a matrix having 'N2' (N2 is an integer) regions or cells, the matrix being defined with 'C2' (C is an integer) columns and 'R2' (R2 is an integer) rows, N2 being equal to C2*R2; and

[0220] - Data identifying one or more regions or one or more cells in the matrix containing tactile actuators, a region or cell containing one or more tactile actuators being referred to as an active region or an active cell.

[0221] For example, the number of haptic actuators per active area is not defined in the container received by the haptic (rendering) engine. According to this example, Figure 4 Each element 400 , 401 , 410 , and 411 illustrated in corresponds to an active area and represents a single haptic actuator or a grouping of haptic actuators, the number of haptic actuators per active area is not contained in the container and is not known by the haptic (rendering) engine.

[0222] According to a variant, the number of haptic actuators per active area is defined in a container received by the haptic (rendering) engine, this information being encoded in data representative of the spatial distribution of the haptic actuators.

[0223] According to this embodiment, the haptic (rendering) engine compares or maps the first data, the second data and / or the third data representing the first haptic device onto a matrix (or data identifying one or more active areas of the first rendering device intended to render the haptic effect) and one or more active areas representing the spatial distribution of each haptic actuator of the second haptic device to determine which active area(s) of the second haptic device corresponds to the haptic actuator of the first haptic device identified by the second data and the third data.

[0224] Figure 5 A communication scheme 5 of haptic-related data according to a particular embodiment of the present principles is schematically shown.

[0225] Figure 5 The operation of a communication system and a communication method implemented by such a system is shown.

[0226] Figure 5 A haptic (rendering) engine 50 is shown in communication connection with a haptic rendering device 51. The haptic (rendering) engine 50 is connected with the haptic rendering device 51, for example, via a wired or wireless connection.

[0227] The tactile presentation device 51 is connected to the tactile (presentation) engine 50 through, for example, a USB input terminal, a telephone connector (also called a telephone jack, an audio jack, a headphone jack, or a jack head) input terminal, or an HDMI input terminal.

[0228] According to another example, the tactile presentation device 51 or The communication interface is connected to the haptic (rendering) engine 50 .

[0229] The data exchange between the tactile presentation device 51 and the tactile (presentation) engine 50 can be carried out within the framework of protocol communication, and the data is exchanged through the application program interface (API) implemented on both sides of the system 5 (i.e., in the tactile (presentation) engine 50 and in the tactile presentation device 51).

[0230] Haptic (rendering) engine 50 is advantageously configured to receive haptic data 500 from a remote device (eg, a cloud server, a computer, an encoder, a haptic data generator, etc.), for example, over a LAN or WLAN network.

[0231] The tactile data 500 is, for example, signaled in the form of encoded data and / or encoded in a container, the container corresponding to, for example, a file, a bit stream, a data packet, a data frame.

[0232] The received haptic data 500 may include, for example, data representing a haptic effect to be presented by the haptic device, such as:

[0233] - the type of tactile (and / or sensory) effect (e.g., vibrotactile effect, force effect, hardness effect, texture effect, temperature effect, wind effect, pain effect, olfactory effect, light-based effect, etc.); and / or

[0234] - the amplitude, phase and / or frequency of one or more signals representing a haptic effect; and / or

[0235] - the start time of the haptic effect; and / or

[0236] - the duration of the haptic effect; and / or

[0237] - an identifier of the haptic effect, which can be used by the system receiving the data to retrieve (eg, from a memory of the system) parameters of the haptic effect; and / or

[0238] - information representing one or more body parts to which the haptic effect described in the received haptic data is directed, such information being referred to as target body part information; and / or

[0239] - Information representing a reference haptic device that is expected to present a haptic effect on the user's body, such information is referred to as reference haptic device information.

[0240] The target body part information is for example obtained by using a reference to the body model 10 Figure 1 and Figure 2 The described syntax is defined in the received tactile data, ie, a target body part may be defined by reference to a grouping of one or more body parts to which the target body part belongs.

[0241] According to a variant, the target body part information corresponds to, for example, a unique identifier for each target body part, and the haptic (rendering) engine retrieves information about the one or more target body parts from a look-up table (LUT) stored in a memory of the haptic engine and from one or more identifiers included in the received haptic data 500. The body model can be represented, for example, as a skeleton composed of joints and segments, a mesh, or as body parts and groups of body parts, wherein the group of body parts includes one or more body parts, each of which is identified by a specific identifier.

[0242] The reference haptic device information may include, for example, spatial distribution information about haptic actuators forming the reference haptic device, such as the reference Figure 3 described.

[0243] According to a variant, the reference haptic device information may further include:

[0244] - Information identifying the location (i.e., a specific haptic actuator or a point between two haptic actuators), such as a reference Figure 3 described; and optionally

[0245] -Information representing distance when more than one haptic actuator is expected to render the haptic effect.

[0246] According to another variation, the reference haptic device information may further include information identifying one or more active areas, each active area including one or more haptic actuators.

[0247] Container 500 is received by a module that implements a parsing and / or decoding process 501 to parse, read and / or decode the encoded data contained in container 500 to obtain decoded haptic data. Module 501 implementing the parsing / decoding process may be hardware, software or a combination of hardware and software.

[0248] The haptic (rendering) engine 50 is further advantageously configured to obtain haptic device data 511 representing the haptic rendering capabilities of the haptic rendering device 51 .

[0249] For example, when the haptic presentation device 51 is connected to the haptic engine 50 , such as via USB, the haptic device data 511 is automatically received from the haptic presentation device 51 .

[0250] When the haptic presentation device 51 is connected to the haptic engine 50, the haptic presentation device 51 automatically starts a process for sending haptic device data through one or more dedicated APIs. This process corresponds to, for example, a so-called "Plug and Play (PnP)" process, and the haptic presentation device 51 and the haptic engine 50 correspond to PnP devices.

[0251] According to another example, tactile device data is obtained or retrieved from the memory of the tactile engine 50 or from a remote system, for example, upon receiving an identifier identifying the tactile presentation device 51, which is, for example, automatically sent by the tactile presentation device 51 to the tactile engine 50 or sent by the tactile presentation device 51 after the tactile engine 50 sends a request.

[0252] The haptic device data includes, for example, at least part of the following information and / or data:

[0253] - information and / or data representing a name of the haptic device and / or representing an identifier of the haptic device 51: such information may correspond, for example, to a string like “chair”, “palmrest” or “XR headset”; and / or

[0254] - information and / or data representing the manufacturer of the haptic device, for example in the form of a string; and / or

[0255] - information and / or data representative of the version of the haptic device: such information may correspond, for example, to a string of characters, such as "1.1"; and / or

[0256] - information and / or data representative of the number of body parts associated with or targeted by the haptic device 51 : such information may, for example, correspond to an integer; and / or

[0257] - Information and / or data representing the name or identifier of each body part associated with or targeted by the haptic device: such information may correspond, for example, to a list or string or to a list of integers, for example according to the same naming convention used for one or more body parts targeted in the haptic data 500 (e.g., using the same convention for the body parts targeted in the haptic data 500). Figure 1 and Figure 2 grammar described herein) to identify one or more body parts; and / or

[0258] - Information and / or data representing one or more tactile perception patterns associated with each body part: Such information may correspond, for example, to a list of integers. A tactile perception pattern corresponds, for example, to a type of tactile feedback corresponding to a tactile effect. For example, a tactile perception pattern / tactile feedback may correspond to one of the following: 1. vibrotactile (vibration and texture), 2. force (in terms of time), 3. force (in terms of space), 4. temperature, 5. smell, 6. air, etc.; and / or

[0259] (The following information and / or data are for example for each tactile perception mode)

[0260] - information and / or data representative of spatial distribution information of one or more tactile actuators comprised in the tactile device 51: such information corresponds, for example, to a list of integers, each integer corresponding to, for example, the number of tactile actuators per dimension of the multidimensional space (for example, 3 integers when the multidimensional space corresponds to a 3D space), or such information corresponds, for example, to a list of identifiers of one or more active areas. Such information advantageously corresponds to a reference Figure 3 and Figure 4 The spatial distribution of the description; and / or

[0261] - information and / or data representative of whether all tactile actuators of the tactile device are of the same type: such information corresponds, for example, to a Boolean value of "true" when all tactile actuators are of the same type and a Boolean value of "false" when the tactile actuators are not all of the same type; and / or

[0262] - Information and / or data representative of the type of tactile actuator and of the characteristics of the tactile actuator: if all tactile actuators are of the same type, such information is signaled once; and when the tactile actuators are not all of the same type, such information is signaled for each type of tactile actuator. Such information corresponds, for example, to a list of integers, the first integer identifying the type (of the type list) and the other integers indicating the values ​​of a list of determined parameters (which may depend on the type of the tactile actuator). The syntax of this information is, for example, for each type: [actuator type, minimum value (e.g. of amplitude and / or frequency), maximum value, resonance, vibration axis (1 for X-axis, 2 for Y-axis, 3 for Z-axis), ...]. The actuator type may be, for example: 0. None, 1. LRA, 2. VCA, 3. ERM, 4. Piezoelectric, 5. Servo, 6. Pneumatic, 7. Passive force, 8. Active force, 9. Peltier cell; and / or

[0263] - information and / or data representing the capabilities of a processing application program interface (API), such as an API representing a transient signal; and / or

[0264] - Information and / or data representative of a format or structure of the haptic device data, such information corresponding to, for example, an integer identifying the format / structure in a list of known formats / structures.

[0265] In a variation, the information representing the processing API capabilities is derived from information representing the format or structure of the haptic device data.

[0266] The haptic device data 511 is stored, for example, in a memory of the haptic (rendering) engine 50 .

[0267] When one or more differences occur in the comparison, at least a portion of the received haptic data 500 is advantageously compared with the haptic device data 511 to process the received haptic data. The comparison and processing process 502 is implemented, for example, by a corresponding module 502 of the haptic engine 50, which can be hardware, software, or a combination of hardware and software.

[0268] The information and / or data compared to one another advantageously refer to the same components or elements, for example:

[0269] - comparing data of haptic device data 511 relating to each body part associated with or targeted by the haptic device 51 with data of the container 500 relating to one or more body parts targeted by the haptic effect; and / or

[0270] - comparing data of the haptic device data 511 relating to the configuration (eg spatial distribution) of the haptic actuators of the haptic device 51 with data of the container 500 relating to a reference haptic device; and / or

[0271] - Compare data of the tactile device data 511 related to the rendering capabilities of the tactile device 51 (e.g., type of tactilely perceived melody / tactile feedback, type and characteristics of one or more tactile actuators) with data of the container 500 related to the tactile effect itself (type of tactilely perceived melody / tactile feedback, type of modulation (frequency, amplitude, base) applied to one or more basic signals to obtain the tactile effect, rendering parameters related to, for example, frequency and / or amplitude, etc.).

[0272] The processing applied to received haptic data 500 advantageously depends on the results of one or more comparison operations.

[0273] The purpose of the process applied to the received tactile data 500 based on the results of one or more comparison operations is to convert and / or adjust at least a portion of the received tactile data 500 to conform to the tactile rendering capabilities of the tactile rendering device 51, which are described or represented by the tactile device data 511.

[0274] According to the first example

[0275] In a first operation, one or more body parts targeted by the tactile effect and identified or signaled in the received tactile data 500 are compared with one or more body parts associated with or targeted by one or more tactile actuators of the tactile rendering device 51 (and identified and signaled in the tactile device data 511).

[0276] If one or more body parts identified in the received haptic data 500 correspond to (or match) one or more body parts identified in the haptic device data 511 , then the received haptic data 500 generated by the first operation is not processed.

[0277] If one or more body parts identified in the received tactile data 500 do not correspond to (or do not completely match) one or more body parts identified in the tactile device data 511, then in a second operation, information contained in the received tactile data and related to one or more body parts identified in the received tactile data 500 (referred to as target body part information) is adjusted or converted based on the comparison result.

[0278] For example, when using the reference Figure 1 and Figure 2 When one or more body parts are identified (in both the received haptic data 50 and the haptic device data 511 ) using the described syntax, the first plurality of body parts and the second plurality of body parts of the body model 10 are referenced.

[0279] Adjustments or conversions applied to the target body part information are performed or processed based on information related to one or more body parts contained in the tactile device data 511, the target body part information, and rules determined based on the relationship between each body part and each grouping of body parts.

[0280] Rules such as parsing Figure 2 The tree 20 is determined based on the relationship between each body part and each grouping of each body part.

[0281] The rules are stored, for example, in a memory of the haptic engine 50 and are updated when modifications or changes are made to the tree 20. Modifications correspond, for example, to adding or removing body parts or groups of body parts, adding or removing relationships between different groups and / or body parts.

[0282] For example, one or more depth levels (corresponding to reference Figure 2 The described hierarchical levels) are associated with each body part contained in the tree 20 for one or more groups to which each body part belongs.

[0283] The following depth levels are associated with the body part “first knuckle” 261, for example:

[0284] - a first depth level equal to 1, reference group 254 "Ring finger";

[0285] - a second depth level equal to 2, reference group 242 "fingers";

[0286] - A third depth level equal to 3, reference grouping 233 "Hands".

[0287] The following depth levels are for example associated with the body part “palm” 241 : A first depth level equal to 1 , with reference to the grouping 254 “hand”.

[0288] As an example, consider the target body part information identifying the body part “first knuckle” 261 in the received haptic data 500 and the information identifying the body part “palm” 241 in the haptic device data 511 .

[0289] The results of the comparison between each information show the differences.

[0290] The first grouping common to body parts "palm" 241 and "first knuckle" 261 in tree 20 is group "hand" 254. The depth level between body part "first knuckle" 261 and group "hand" is 3. The depth level between body part "palm" 241 and group "hand" is 1.

[0291] A maximum depth level may be associated with each body part to determine whether a haptic effect must be presented when the body part targeted by the haptic effect does not match the body part associated with haptic device 51 .

[0292] For example, a rule may be to present a haptic effect only when the difference between the depth levels for a common grouping is less than or equal to a maximum depth level.

[0293] According to this example, if the maximum depth level is 2, the haptic effect for “first knuckle” 261 must be presented on body part “palm” 241 because the difference in depth levels for group “hand” is equal to two.

[0294] The target body part information (eg, “first knuckle”) in the received tactile data 500 is converted into the body part “palm” and the tactile data 512 generated by the conversion is sent to the tactile device 51 .

[0295] According to another example, a rule may be to present a haptic effect only when the depth level associated with the target body part (for both the target body part and the first grouping of body parts identified in haptic device data 511) is less than or equal to a maximum depth level.

[0296] According to this another example, if the maximum depth level is 2, then the haptic effect for "first knuckle" 261 will not be presented on body part "palm" 241 because the depth level for group "hand" is greater than 2.

[0297] According to a further example, a rule may be to reduce the amplitude of the haptic effect when the depth level associated with the target body part (for both the target body part and the first grouping of body parts identified in the haptic device data 511) is greater than a maximum depth level.

[0298] According to this further example, the target body part information (e.g., "first knuckle") in the received tactile data 500 is converted into the body part "palm", the amplitude of the tactile effect defined in the received tactile data 500 is adjusted to a reduced level, and the tactile data 512 obtained by converting the received tactile data is sent to the tactile device 51 to present the adjusted tactile effect.

[0299] Of course, other rules may be defined to determine whether a haptic effect must be presented on a body part associated with haptic device 51 (and identified in haptic device data 511 ) that is different from the body part identified as the target body part in received haptic data 500 .

[0300] According to the second example

[0301] In a first operation, data / information representing the spatial distribution of each tactile actuator (of a reference tactile device) signaled in the received tactile data 500 is compared with data / information representing the spatial distribution of each tactile actuator (of a tactile presentation device 51) signaled in the tactile device data 511.

[0302] If the spatial distribution of reference haptic devices signaled in the received haptic data 500 corresponds to (or matches) the spatial distribution of haptic presentation devices 51 signaled in the haptic device data 511 , then the received haptic data 500 resulting from the first operation is not processed.

[0303] If the spatial distribution of the reference tactile devices signaled in the received tactile data 500 does not correspond to (or does not match) the spatial distribution of the tactile presentation devices 51 signaled in the tactile device data 511, information contained in the received tactile data 500 and related to one or more tactile actuators targeted to present a tactile effect (referred to as reference tactile device information) is adjusted or converted based on the comparison result in the second operation.

[0304] In the second operation, the tactile actuators in the tactile actuator group of the tactile presentation device 51 are determined based on the spatial distribution information of the reference tactile device (signaled in the received tactile data 500), the spatial distribution information of the tactile presentation device 51 (signaled in the tactile device data 511), and the position information of the tactile actuator representing the reference tactile device or a point between two tactile actuators (the position information is signaled in the received tactile data 500).

[0305] The tactile actuator of the tactile presentation device is determined by referring to Figure 4 The described equations 1, 2 and 3 are obtained.

[0306] Identify the data of the haptic actuator / point of the reference haptic device in the received haptic data 500 (eg, parameter C1 x 、C1 y and C1 z ) is advantageously converted into (or replaced with) data identifying a haptic actuator of a haptic rendering device in haptic data 512 sent to haptic device 51 (e.g., parameter C2 x 、C2 y and C2 z ).

[0307] When the reference tactile device information further includes first distance information (e.g., as reference Figure 4 Explained by R1 x 、R1 y and R1 z Define) to use parameter C1 x 、C1 y and C1 z When the identified tactile actuator / point is targeted at the tactile actuator group, the processing of the received tactile data further includes determining the second distance information, for example by referring to Figure 4 Equation 4, Equation 5 and Equation 6 described.

[0308] Data defining the first distance information in the received tactile data 500 (eg, parameter R1 x 、R1 y and R1 z ) is advantageously converted into (or replaced by) data defining the second distance information in the tactile data 512 sent to the tactile device 51 (eg, parameter R2 x , R2 y and R2 z ).

[0309] According to the third example

[0310] In a first operation, data / information representing the tactile effect signaled in the received tactile data 500 (e.g., the type of tactile effect, rendering parameters such as amplitude or frequency, etc.) is compared with data / information representing the rendering capabilities of the tactile actuator (or tactile rendering device 51) signaled in the tactile device data 511.

[0311] In a second operation, when the comparison indicates that the rendering capabilities of the haptic actuator are unable to enable the haptic rendering device 51 to render the haptic effect described in the received haptic data 500 , the data / information representing the haptic effect is processed.

[0312] For example, if a haptic effect is defined with a frequency parameter that describes a frequency variation (e.g., over space and / or over time) and an amplitude parameter that describes the variation (e.g., over space and / or over time), and if the type of one or more actuators signaled in the haptic device data 511 is only configured (or capable of) presenting the amplitude parameter, the frequency portion of the haptic data is filtered and only the amplitude portion is sent with the haptic data 512 to the haptic presentation device 51.

[0313] For example, if the haptic actuator of the haptic presentation device 51 corresponds to an LRA, only the amplitude modulation is presented (while the frequency modulation is filtered and not presented).

[0314] According to another example, if the haptic actuator of the haptic presentation device 51 corresponds to the ERM, only the amplitude parameters having a value or level greater than a threshold value (e.g., 0.5) are retained in the haptic data 512 sent to the haptic presentation device 51, which means that the amplitude parameters having a value or level less than the threshold value are filtered from the received haptic data 500 to obtain the haptic data 512 to be sent. As a result, the ERM vibrates only when the amplitude parameter describing the haptic effect is greater than the threshold value.

[0315] According to another example, if the haptic actuator of the haptic presentation device 51 corresponds to a voice coil, amplitude modulation and frequency modulation are presented simultaneously.

[0316] According to a further example, when a haptic effect to be presented is defined by an application program interface (API) or by calling an API in received haptic data:

[0317] - the haptic rendering engine 50 sends parameters (such as frequency parameters and / or amplitude parameters) representing the API of the haptic rendering device 51 to the haptic rendering device, and the haptic rendering device 51 is capable of rendering the haptic effect described by the API; and

[0318] - The haptic rendering engine 50 converts the parameters of the haptic effect described by the API call into parameters of a signal that the haptic rendering device 51 can render (e.g., a short signal in PCM for the voice coil to receive a PCF signal, or a short amplitude pulse for an LRA).

[0319] The API call associated with the determined signal may be encoded into the received haptic data 500 in the following form:

[0320] - an identifier identifying an API call stored in the memory of the haptic (rendering) engine 50; or

[0321] - A set of instructions to be executed by the haptic (rendering) engine 50 .

[0322] The API call may be used, for example, to render a transient state (eg, a haptic effect corresponding to an impact).

[0323] The API call is represented and synthesized, for example, by a sinusoidal function of only one period with a determined duration (eg 22 ms).

[0324] Figure 6 A schematic block diagram illustrating steps of a method of transmitting haptic data representing a haptic effect according to at least one example embodiment is shown.

[0325] In a first step 61 , haptic device data representing haptic rendering capabilities of the haptic rendering device is acquired, for example received from the haptic rendering device.

[0326] In a second step 62 , haptic data is determined by processing the received haptic data representing the haptic effect based on the haptic device data acquired in step 61 to adapt the presentation of the haptic effect by the haptic presentation device to the haptic presentation capabilities of the haptic presentation device.

[0327] In a third step 63 , the haptic data is sent to a haptic rendering device.

[0328] In an optional fourth step, the haptic effect described by the haptic data is presented by a haptic presentation device.

[0329] Figure 7 A schematic block diagram illustrating an example of a system 7 in which various aspects and exemplary embodiments are implemented is shown.

[0330] The system 7 may be embedded as one or more devices, including various components described below. In various embodiments, the system 7 may be configured to implement one or more aspects described in this application.

[0331] Examples of equipment that may constitute all or part of system 7 include a personal computer, a laptop computer, a smart phone, a tablet computer, a digital multimedia set-top box, a digital television receiver, a personal video recording system, a connected home appliance, a connected vehicle and its associated processing system, a head-mounted display device (HMD, see-through glasses), a tactile sensor or actuator, a "cave" (a system including multiple displays), a server, a tactile encoder, a tactile decoder, a post-processor that processes output from a tactile decoder, a pre-processor that provides input to a tactile encoder, a web server, a set-top box, a wireless (e.g., ), a wearable haptic device, a game controller, a mouse, a mouse pad, a keyboard, a hand rest, a chair, a desk, an XR headset, a head-mounted headset, a bracelet, a head and / or lumbar support device or a seat, any other device for processing tactile data or tactile signals, or any other communication device. The elements of system 7 can be implemented individually or in combination in a single integrated circuit (IC), multiple ICs and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 7 can be distributed across multiple ICs and / or discrete components. In various embodiments, system 7 can be communicatively connected to other similar systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports.

[0332] The system 7 may include at least one processor 71 configured to execute instructions loaded therein to implement, for example, various aspects described in the present application. The processor 71 may include embedded memory, input and output interfaces, and various other circuits known in the art. The system 7 may include at least one memory 72 (e.g., a volatile memory device and / or a non-volatile memory device). The system 7 may include a storage device 74, which may include a non-volatile memory and / or a volatile memory, including but not limited to an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 74 may include an internal storage device, an attached storage device, and / or a network accessible storage device.

[0333] The system 7 may include an encoder / decoder module 73, which is configured to, for example, process data to provide encoded / decoded tactile signals or data, and the encoder / decoder module 73 may include its own processor and memory. The encoder / decoder module 73 may represent a (one or more) module that may be included in the device to perform encoding and / or decoding functions. As is known, the device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 73 may be implemented as a separate element of the system 73, or may be incorporated into the processor 71 as a combination of hardware and software known to those skilled in the art.

[0334] Program code to be loaded into the processor 71 or the encoder / decoder 73 to perform various aspects described in the present application may be stored in the storage device 74 and subsequently loaded into the memory 72 for execution by the processor 71. According to various embodiments, during the execution of the processes described in the present application, one or more of the processor 71, the memory 72, the storage device 74, and the encoder / decoder module 73 may store one or more of various items. Such stored items may include, but are not limited to, tactile-related data, encoded / decoded data for identifying one or more body parts of a body model and / or one or more groups of one or more body parts, bit streams, matrices, variables, and intermediate or final results of equations, formulas, operations, and arithmetic logic processing.

[0335] In several embodiments, memory internal to the processor 71 and / or encoder / decoder module 73 may be used to store instructions and provide working memory for processes that may be performed during encoding or decoding.

[0336] However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 71 or the encoder / decoder module 73) is used for one or more of these functions. The external memory may be the memory 72 and / or the storage device 74, such as a dynamic volatile memory and / or a non-volatile flash memory. In at least one embodiment, a fast external dynamic volatile memory such as RAM may be used as working memory for video encoding and decoding operations, such as for MPEG-V.

[0337] As indicated in box 75, input to the elements of system 7 may be provided through various input devices. Such input devices include, but are not limited to, (i) an RF section that may receive an RF signal over the air, such as by a broadcaster, (ii) a composite input terminal, (iii) a USB input terminal, (iv) a phone connector (also known as a phone jack, audio jack, headphone jack, or jack plug) input terminal, and / or (v) an HDMI input terminal.

[0338] In various embodiments, the input device of block 75 has associated corresponding input processing elements, as known in the art. For example, the RF portion may be associated with elements necessary for: (i) selecting a desired frequency (also referred to as selecting a signal, or band-limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) band-limiting the frequency band again to a narrower frequency band to select a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired packet stream. The RF portion of various embodiments may include one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF portion may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or a baseband.

[0339] In one set-top box embodiment, the RF section and its associated input processing elements can receive RF signals transmitted over a wired (e.g., cable) medium. The RF section can then perform frequency selection by filtering, down-converting, and filtering again to a desired frequency band.

[0340] Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions.

[0341] Adding an element may include inserting an element between existing elements, such as, for example, inserting an amplifier and an analog-to-digital converter.In various embodiments, the RF portion may include an antenna.

[0342] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 7 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, within a separate input processing IC or within the processor 71, as necessary. Similarly, various aspects of USB or HDMI interface processing may be implemented, for example, within a separate interface IC or within the processor 71, as necessary. The demodulated, error-corrected, and demultiplexed streams may be provided to various processing elements, including, for example, the processor 71 and the encoder / decoder 73, which operate in conjunction with memory and storage elements to process the data streams for presentation on an output device, as necessary.

[0343] The various elements of the system 7 may be provided within an integrated housing. Within the integrated housing, the various elements may be interconnected and data transferred between them using a suitable connection arrangement 75, such as an internal bus (including an I2C bus), wiring, and printed circuit boards known in the art.

[0344] The system 7 may include a communication interface 76 that enables communication with other devices via a communication channel 760. The communication interface 76 may include, but is not limited to, a transceiver configured to send and receive data over the communication channel 760. The communication interface 76 may include, but is not limited to, a modem or a network card, and the communication channel 760 may be implemented, for example, within a wired and / or wireless medium.

[0345] In various embodiments, a Wi-Fi network such as IEEE 802.11 may be used to stream data to the system 7. The Wi-Fi signals of these embodiments may be received via a communication channel 760 suitable for Wi-Fi communications and a communication interface 76. The communication channel 760 of these embodiments may typically connect to an access point or router that provides access to external networks including the Internet to allow streaming applications and other over-the-top communications.

[0346] Other embodiments may provide streaming data to system 7 using a set top box that delivers the data through the HDMI connection of input box 75.

[0347] Other embodiments may use the RF connection of input box 75 to provide streaming data to system 7.

[0348] Streamed data may be used as a way of signaling information used by the system 7. The signaling information may, for example, comprise data encoded in a container such as a binary stream or a haptic effect file.

[0349] It should be appreciated that signaling may be implemented in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc. may be used to signal information to a corresponding decoder.

[0350] System 7 can provide signals to various output devices, including display 770 , speaker 780 , and other peripheral devices 790 such as haptic devices / actuators.

[0351] In various embodiments, control signals may be communicated between the system 7 and the display 770, speakers 780, or other peripheral devices 790 using signaling such as AV.Link (audio / video link), consumer electronics control (CEC), audio protocols, Universal Serial Bus (USB), Haptic Industry Forum-Universal Haptic Protocol (HIF UHP), or other communication protocols that enable device-to-device control with or without user intervention.

[0352] The output devices may be communicatively connected to the system 7 via dedicated connections through respective interfaces 77 , 78 and 79 .

[0353] Alternatively, the output device may be connected to the system 7 using the communication channel 760 via the communication interface 76. The display 770, speaker 780 and / or actuator 790 may be integrated with other components of the system 7 in a single unit in an electronic device such as, for example, a television.

[0354] In various embodiments, the display interface 77 may include a display driver, such as, for example, a timing controller (T Con) chip.

[0355] For example, if the RF portion of input 75 is part of a separate set-top box, then display 770, speaker 780, and / or actuator 790 may optionally be separate from one or more of the other components. In various embodiments where display 770, speaker 780, and / or actuator 790 may be external components, output signals may be provided via dedicated output connections, including, for example, an HDMI port, a USB port, or a COMP output.

[0356] exist Figure 1-7 In the present invention, various methods are described herein, and each method includes one or more steps or actions to implement the described method. Unless a specific order of steps or actions is required for the correct operation of the method, the order and / or use of specific steps and / or actions can be modified or combined.

[0357] Some examples are described about block diagrams and / or operational flow charts. Each box represents a portion of a circuit element, module or code, which includes one or more executable instructions for implementing (one or more) specified logical functions. It should also be noted that, in other embodiments, the (one or more) functions marked in the box may not occur in the order indicated. For example, depending on the functions involved, two boxes shown in succession can actually be executed substantially concurrently, or sometimes these boxes can be executed in the opposite order.

[0358] The embodiments and aspects described herein may be implemented in, for example, a method or process, an apparatus, a computer program, a data stream, a bit stream, or a signal. Even if only discussed in the context of a single form of embodiment (e.g., discussed only as a method), the embodiments of the features discussed may also be implemented in other forms (e.g., an apparatus or a computer program).

[0359] The method may be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device.

[0360] In addition, the method can be implemented by instructions executed by a processor, and such instructions (and / or data values ​​generated by the implementation) can be stored on a computer-readable storage medium. The computer-readable storage medium can take the form of a computer-readable program product implemented in one or more computer-readable media and having a computer-readable program code implemented thereon that can be executed by a computer. Considering the inherent ability to store information therein and the inherent ability to provide information retrieval therefrom, the computer-readable storage medium used herein can be considered as a non-transient storage medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or apparatus, or any suitable combination of the foregoing. It should be appreciated that, although more specific examples of computer-readable storage media to which the present embodiment can be applied are provided below, as those of ordinary skill in the art will readily appreciate, it is merely illustrative and not exhaustive: portable computer floppy disk; hard disk; read-only memory (ROM); erasable programmable read-only memory (EPROM or flash memory); portable compact disk read-only memory (CD-ROM); optical storage device; magnetic storage device; or any suitable combination of the foregoing.

[0361] The instructions may form an application program tangibly embodied on a processor-readable medium.

[0362] For example, instructions may be in hardware, firmware, software, or a combination. For example, instructions may be found in an operating system, a separate application, or a combination of both. Thus, a processor may be characterized as, for example, a device configured to perform a process and a device including a processor-readable medium (such as a storage device) having instructions for performing the process. Additionally, in addition to or in lieu of instructions, a processor-readable medium may store data values ​​generated by an embodiment.

[0363] The device may be implemented in, for example, appropriate hardware, software, and firmware. Examples of such devices include personal computers, laptops, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, head-mounted display devices (HMDs, perspective glasses), projectors (projectors), "cave" (systems including multiple displays), servers, video and / or haptic encoders, video and / or haptic decoders, post-processors that process output from a video decoder, pre-processors that provide input to a video encoder, web servers, set-top boxes, wirelessly connected wearable haptic devices (e.g., The device may include a wearable haptic device, a connected wearable haptic device, a game controller, a mouse, a mouse pad, a keyboard, a hand rest, a chair, a desk, an XR headset, a headphone, a wristband, a head and / or lumbar support device or a seat, and any other device for processing haptic data or signals representing one or more haptic feedback or effects, or other communication devices. It should be clear that the equipment can be mobile.

[0364] The computer software may be implemented by the processor 71 or by hardware, or by a combination of hardware and software. As a non-limiting example, embodiments may also be implemented by one or more integrated circuits. The memory 72 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology (such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples). The processor 71 may be of any type suitable for the technical environment and may encompass one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture, as non-limiting examples.

[0365] As will be apparent to one of ordinary skill in the art, embodiments may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method or data generated by one of the described embodiments. For example, a signal may be formatted to carry a bit stream of the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0366] The terms used herein are only used to describe the purpose of a particular embodiment and are not intended to be limited. As used herein, the singular forms "one", "a kind of" and "the" may also be intended to include plural forms unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the terms "include / comprise" and / or "including / comprising" may specify the existence of stated, for example, features, integers, steps, operations, elements and / or components, but do not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. Moreover, when an element is referred to as "in response to" or "connected to" another element, it may directly respond to or be connected to another element, or there may be an intermediate element. In contrast, when an element is referred to as "directly responding to" or "directly connected to" another element, there is no intermediate element.

[0367] It should be appreciated that use of any of the symbols / terms " / ", "and / or", and "at least one of" may be intended to encompass selection of only the first listed option (A), or only the second listed option (B), or both options (A and B), such as in the case of "A / B", "A and / or B", and "at least one of A and B". As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C", such wording is intended to encompass selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This may be extended to as many items as listed, as will be apparent to one of ordinary skill in this and related arts.

[0368] Various numerical values ​​may be used in the present application. Specific values ​​may be used for illustrative purposes and the described aspects are not limited to these specific values.

[0369] It will be understood that although the terms first, second, etc. can be used to describe various elements in this article, these elements are not limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the teaching of this application, the first element can be referred to as the second element, and similarly, the second element can be referred to as the first element. There is no suggestion of sorting between the first element and the second element.

[0370] References to "an exemplary embodiment" or "exemplary embodiments" or "one implementation" or "implementation" and other variations thereof are frequently used to convey that a particular feature, structure, characteristic, etc. (described in conjunction with an embodiment / implementation) is included in at least one embodiment / implementation. Thus, the appearances of the phrases "in an exemplary embodiment" or "in an exemplary embodiment" or "in one implementation" or "in an implementation" and any other variations appearing in various places in this application do not necessarily all refer to the same embodiment.

[0371] Similarly, references herein to "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" and other variations thereof are frequently used to convey that a particular feature, structure, or characteristic (described in conjunction with an exemplary embodiment / example / implementation) may be included in at least one exemplary embodiment / example / implementation. Therefore, the expressions "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" appearing in various places in the specification do not necessarily all refer to the same exemplary embodiment / example / implementation, nor are separate or alternative exemplary embodiments / examples / implementations necessarily mutually exclusive of other exemplary embodiments / examples / implementations.

[0372] Reference numerals appearing in the claims are for illustration purposes only and have no limiting effect on the scope of the claims.The present embodiments / examples and variations may be employed in any combination or sub-combination although not explicitly described.

[0373] When a figure is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding device. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0374] While some diagrams include arrows on communication paths to illustrate a primary direction of communication, it should be understood that communication can occur in the opposite direction to the depicted arrows.

[0375] Various embodiments relate to decoding. As used in this application, "decoding" may encompass, for example, all or part of a process performed on a received tactile signal (which may include a received bitstream encoding one or more tactile signals) to produce a final output suitable for presenting a tactile effect or for further processing in a reconstructed tactile feedback or effect. In various embodiments, such a process includes one or more of the processes typically performed by a decoder. In various embodiments, for example, such a process also includes or optionally includes a process performed by a decoder of the various embodiments described in this application.

[0376] Various embodiments relate to encoding. In a manner similar to the above discussion of "decoding", "encoding" as used in this application may encompass, for example, all or part of a process performed on an input tactile signal to produce an encoded bitstream. In various embodiments, such a process includes one or more of the processes typically performed by an encoder. In various embodiments, such a process also includes or optionally includes a process performed by an encoder of the various embodiments described in this application.

[0377] Additionally, this application may refer to "obtaining" various pieces of information. Obtaining information may include, for example, one or more of: estimating the information, calculating the information, or retrieving the information from a memory.

[0378] Further, this application may refer to "accessing" various pieces of information. Obtaining information may include, for example, one or more of: receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, or estimating information.

[0379] Additionally, this application may refer to "receiving" various pieces of information. Like "accessing," receiving is a broad term. Receiving information may include, for example, one or more of: accessing information or retrieving information (e.g., from a memory). Further, during operations such as, for example, storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, or estimating information, "receiving" is generally involved in one way or another.

[0380] Moreover, as used herein, the word "signal" is especially used to indicate something to a corresponding decoder, etc. In this way, in an embodiment, the same parameter can be used on the encoder side and the decoder side. Therefore, for example, the encoder can transmit (explicitly signal) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters and other parameters, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select specific parameters. By avoiding the transmission of any actual function, bit saving is achieved in various embodiments. It should be appreciated that signaling can be completed in a variety of ways. For example, in various embodiments, one or more grammatical elements, signs, etc. are used to signal information to the corresponding decoder. Although the verb form of the word "signal (signal)" is related to the above, the word "signal" can also be used as a noun in this article.

[0381] A number of embodiments have been described. However, it should be understood that various modifications may be made. For example, elements of different embodiments may be combined, supplemented, modified, or removed to produce other embodiments. In addition, it will be understood by those of ordinary skill that other structures and processes may replace the disclosed structures and processes, and that the resulting embodiments will perform at least substantially the same (one or more) functions in at least substantially the same (one or more) manners to achieve at least substantially the same (one or more) results as the disclosed embodiments. Thus, the present application contemplates these and other embodiments.

Claims

1. A method of sending haptic data (512) representing a haptic effect, the method comprising the steps of: - acquiring (61) tactile device data (511) representing the tactile presentation capabilities of the tactile presentation device (51); - determining (62) the haptic data by processing received haptic data (500) representing the haptic effect according to the haptic device data (511) to adapt the rendering of the haptic effect to the haptic rendering capability; and - sending (63) said tactile data (512).

2. The method of claim 1, wherein the received tactile data comprises a set of information, the set of information comprising: - target body part information representing the target body part of the body model (10) targeted by the haptic effect, and / or - reference haptic device information representing a reference haptic device for rendering the haptic effect, and / or - rendering information representative of rendering parameters of said haptic effect, The processing includes: - comparing said haptic device data with said information set; and - When a result of the comparison represents at least a difference between the haptic device data (511) and the information set, adjusting the received haptic data (500) according to the result.

3. The method according to claim 2, wherein the processing comprises: - comparing the target body part information with first information of a body part of the body model (10) contained in the tactile device data (511) and associated with the tactile presentation device (51); and - When a result of the comparison represents a difference between the first information and the target body part information, adjusting the target body part information according to the result.

4. The method according to claim 3, wherein: The body model (10) comprises a first plurality of body parts and a second plurality of groupings of body parts, each grouping of the second plurality of groupings of body parts comprising at least a body part of the first plurality of body parts, The adjusting is based on the first information, the target body part information, and relationship information representing a relationship between at least some of the first plurality of body parts and at least some of the groupings of the second plurality of body parts.

5. The method according to any one of claims 2 to 4, wherein the reference tactile device information comprises spatial distribution information representing the spatial distribution of the tactile actuator group (300 to 304, 310 to 311, 320 to 324, 330 to 334) included in the reference tactile device in a determined multidimensional space, and position information representing the position in the determined multidimensional space, The processing includes: - comparing the spatial distribution information with second information contained in the tactile device data (511) and representing the spatial distribution of the tactile actuator group included in the tactile presentation device (51) in the determined multidimensional space; - determining at least one tactile actuator in the tactile actuator group of the tactile presentation device (51) according to the spatial distribution information, the second information and the position information; and - adjusting the reference haptic device information according to the at least determined haptic actuator of the haptic actuator group of the haptic presentation device (51).

6. The method of claim 5, wherein the reference tactile device information further comprises first distance information representing a distance relative to the position, the processing further comprises determining second distance information based on the spatial distribution information and the second information, and the adjusting comprises replacing the first distance information with the second distance information.

7. The method according to any one of claims 2 to 6, wherein the rendering parameters include rendering parameters representing frequency, rendering parameters representing amplitude and / or rendering parameters representing phase, The processing includes: - comparing said presentation information with information representative of the type of said tactile presentation device; - When a result of said comparison represents at least a difference between the presentation capabilities associated with said type and said presentation information, filtering at least part of said presentation parameters according to said result.

8. The method according to any one of claims 2 to 6, wherein the presentation parameters comprise presentation parameters representing an application program interface call, The processing includes: - comparing the presentation information with information representative of the capability of the tactile presentation device to process an application program interface; - When the result of the comparison shows that the tactile presentation device cannot process the application program interface, converting at least part of the presentation parameters according to the result, the conversion being based on information representing the type of the tactile presentation device.

9. The method of claim 1 or 2, wherein the haptic device data (511) comprises at least one of the following: - first data representing the number of body parts targeted by the tactile presentation device (51); and / or - second data representing an identifier for each of said body parts; and / or - third data representing a set of tactile perception patterns, the set of tactile perception patterns comprising at least a tactile perception pattern; and / or - fourth data representing the number of tactile actuators of the tactile presentation device (51); and / or - fifth data representing the spatial distribution of said tactile actuators; and / or - sixth data representing whether the tactile actuators are all of the same type; and / or - seventh data representative of the actuator type of each of said actuators; and / or - eighth data representing a tactile rendering parameter of each of said tactile actuators; and / or - ninth data representing the capability of processing the application program interface; and / or - tenth data representing the format of the haptic device data (511).

10. The method according to claim 9, wherein the third data is used for each of the body parts, and the fourth data, the fifth data, the sixth data, the seventh data and the eighth data are used for each tactile perception pattern in the tactile perception pattern set.

11. The method according to any one of claims 1 to 10, wherein the haptic device data (511) is received from the haptic presentation device (51).

12. A device (7) for transmitting tactile data representative of a tactile effect, wherein the device (7) comprises a memory (72) associated with at least a processor (71), the processor (71) being configured to implement the method according to any one of claims 1 to 11.

13. A non-transitory processor-readable medium storing therein instructions for causing a processor to execute at least the steps of the method according to any one of claims 1 to 11.

14. A computer program product comprising program code instructions for executing the method according to any one of claims 1 to 11 when said program is executed on a computer.

Images