Method and apparatus for signaling / parsing haptic data

Abstract

A method and apparatus are provided for signaling / parsing data representative of an image between a body part (1) of a body model for which a haptic effect is directed and a haptic device comprising a set of haptic actuators (100 to 104, 110 to 114, 120 to 124, 130 to 134) configured to present a haptic effect on at least part of the body part. Signaling / parsing the data includes writing or reading first and second data into or from a container, the first data representing a body part and the second data representing a spatial distribution of the groups of haptic actuators (100 to 104, 110 to 114, 120 to 124, 130 to 134) in the determined multi-dimensional space.

Description

Technical Field

[0001] The present application relates generally to the field of haptic technology, and in particular to the processing of haptic-related data. The present application relates to representing and / or signaling data regarding a mapping between body parts of a body model and a haptic device configured to present one or more haptic effects, and to parsing such data. The present application also relates to methods and apparatus for encoding / decoding data representing a haptic device configured to present a haptic device on one or more body parts of a person. Background Art

[0002] This section is intended to introduce the reader to various aspects of the art, which may be related to various aspects of at least one exemplary embodiment of the present application described and / or claimed below. Such discussion is considered to be helpful in providing background information to the reader so that the reader can better understand various aspects of the present application.

[0003] Haptic technology broadly refers to any technology that recreates the sensation of touch in a user interface by applying force, vibration, motion, and other sensations (e.g., temperature) to provide information to the end user, for example, in addition to visual and audio information when presenting multimedia content.

[0004] Haptic feedback covers a wide range of possible stimulation embodiments, but is mainly divided into haptic haptic technology and kinesthetic haptic technology: Haptic haptic feedback (or haptic haptic effects) refers to sensations such as vibration, friction, or micro-deformation, while kinesthetic haptic feedback (or kinesthetic haptic effects) refers to sensations that provide a sense of force, which can stimulate mechanical stimulation as well as stimulation related to the position and movement of the body.

[0005] The presentation of tactile feedback or tactile effects is obtained using a tactile device, which corresponds to the arrangement of one or more tactile actuators. For example, a vibratory tactile effect can be obtained by using an eccentric rotating mass (ERM), a linear resonant actuator (LRA), and a wide bandwidth actuator (such as a voice coil motor (VCM) or a piezoelectric actuator (PZT)). Kinesthetic effects can be presented by an actuator applying a force that hinders the movement of a limb, and such an effect is only felt on muscles and tendons, not on the skin. Other examples of tactile devices include resistive force feedback devices, active force feedback devices, and skin indentation devices.

[0006] There are a wide range of haptic devices with various arrangements of one or more actuators forming the haptic device. When gathering haptic-related data in a container to be sent to a haptic rendering engine to render a haptic effect on a human body, the type of haptic device that will be used to render the haptic effect on a human body is generally unknown, which may cause the rendering engine to incorrectly render the haptic effect with the associated one or more haptic devices. Summary of the invention

[0007] The following section presents a brief overview of at least one exemplary embodiment to provide a basic understanding of some aspects of the present application. This overview is not an extensive review of the exemplary embodiments. It is not intended to be used to identify the key or important elements of the embodiments. The following overview merely presents certain aspects of at least one exemplary embodiment in a brief form as a prelude to a more detailed description provided elsewhere in this document.

[0008] According to a first aspect of the present application, there is provided a method of signaling a mapping between a body part of a body model targeted by a haptic effect and a haptic device comprising a haptic actuator group into a container, the haptic actuator group being configured to present the haptic effect on at least part of the body part, the method comprising:

[0009] - writing first data representing the body part into the container;

[0010] -Writing second data representing the spatial distribution of the tactile actuator group in the determined multi-dimensional space into the container.

[0011] In an exemplary embodiment, the method further includes: when the tactile actuator group includes a plurality of tactile actuators, writing third data representing a position of a determined tactile actuator in the tactile actuator group in the determined multi-dimensional space into the container.

[0012] In an exemplary embodiment, the method further includes writing fourth data representing a distance relative to the determined haptic actuator into the container, the distance being defined in the determined multi-dimensional space.

[0013] In a further exemplary embodiment, the determined haptic actuator corresponds to a starting point of the haptic animation, and the method further includes writing fifth data representing another determined haptic actuator corresponding to an ending point of the haptic animation into the container.

[0014] In another exemplary embodiment, the determined multidimensional space corresponds to:

[0015] - a Cartesian three-dimensional space, wherein the first data comprises first information representing the number of tactile actuators according to a first dimension of the three-dimensional space, second information representing the number of tactile actuators according to a second dimension of the three-dimensional space, and third information representing the number of tactile actuators according to a third dimension of the three-dimensional space, the maximum number of tactile actuators according to the third information being equal to 2;

[0016] - A two-dimensional space, wherein the first data comprises first information representative of the number of angular areas according to a first dimension of the two-dimensional space and second information representative of the number of tactile actuators according to a second dimension of the two-dimensional space for each of said angular areas.

[0017] In a further exemplary embodiment, the method further comprises writing data representing a haptic device type associated with the haptic device, with reference to a spatial distribution of the haptic device type, into the container.

[0018] In additional exemplary embodiments, the method further includes writing sixth data representing the haptic effect into the container.

[0019] According to a second aspect of the present application, there is provided a method of parsing data representing a mapping between a body part of a body model targeted by a haptic effect and a haptic device including a haptic actuator group configured to present a haptic effect on at least part of the body part from a container, the method comprising:

[0020] - reading first data representing a body part from a container;

[0021] - Reading second data representing the spatial distribution of the tactile actuator group in the determined multi-dimensional space from the container.

[0022] In an exemplary embodiment, the method further includes: when the tactile actuator group includes a plurality of tactile actuators, reading third data representing a position of a determined tactile actuator in the tactile actuator group in the determined multi-dimensional space from the container.

[0023] In an exemplary embodiment, the method further comprises: reading fourth data representing a distance relative to the determined haptic actuator from the container, the distance being defined in the determined multi-dimensional space.

[0024] In a further exemplary embodiment, the determined haptic actuator corresponds to a starting point of the haptic animation, and the method further includes reading fifth data representing another determined haptic actuator corresponding to an ending point of the haptic animation from the container.

[0025] In another exemplary embodiment, the determined multidimensional space corresponds to:

[0026] - a Cartesian three-dimensional space, wherein the first data comprises first information representing the number of tactile actuators according to a first dimension of the three-dimensional space, second information representing the number of tactile actuators according to a second dimension of the three-dimensional space, and third information representing the number of tactile actuators according to a third dimension of the three-dimensional space, the maximum number of tactile actuators according to the third information being equal to 2;

[0027] - A two-dimensional space, wherein the first data comprises first information representative of the number of angular areas according to a first dimension of the two-dimensional space and second information representative of the number of tactile actuators according to a second dimension of the two-dimensional space for each of said angular areas.

[0028] In a further exemplary embodiment, the method further comprises reading, from the container, data representing a haptic device type associated with the haptic device, with reference to a spatial distribution of the haptic device type.

[0029] In another exemplary embodiment, the method further includes converting the third data and / or the fourth data according to the second data and data representing a spatial distribution of a haptic actuator group of another haptic device.

[0030] In additional exemplary embodiments, the method further includes reading sixth data representing the haptic effect from the container.

[0031] According to a third aspect of the present application, there is provided a device for signaling a mapping between a body part of a body model targeted by a tactile effect and a tactile device comprising a tactile actuator group to a container, wherein the device comprises a memory associated with at least a processor, and the processor is configured to implement the method according to the first aspect of the present application.

[0032] According to a fourth aspect of the present application, there is provided a device for parsing from a container data representing a mapping between a body part of a body model targeted by a tactile effect and a tactile device comprising a tactile actuator group, wherein the device comprises a memory associated with at least a processor, the processor being configured to implement the method according to the second aspect of the present application.

[0033] According to a fifth aspect of the present application, there is provided a container formatted to contain data representing a mapping between a body part of a body model targeted by a haptic effect and a haptic device comprising a haptic actuator group, the haptic actuator group being configured to present the haptic effect on at least part of the body part, the data comprising:

[0034] - first data representing a body part;

[0035] - Second data representing the spatial distribution of the tactile actuator group in the determined multi-dimensional space.

[0036] According to a sixth aspect of the present application, there is provided a bitstream carrying data representing a mapping between a body part of a body model targeted by a haptic effect and a haptic device comprising a haptic actuator group, the haptic actuator group being configured to present a haptic effect on at least part of the body part, the data comprising:

[0037] - first data representing a body part;

[0038] - Second data representing the spatial distribution of the tactile actuator group in the determined multi-dimensional space.

[0039] According to a seventh aspect of the present application, a computer program product comprising instructions is provided. When the program is executed by one or more processors, the one or more processors execute the method according to the first aspect of the present application.

[0040] According to an eighth aspect of the present application, a non-transitory storage medium is provided, carrying program code instructions for executing the method according to the first aspect of the present application.

[0041] According to a ninth aspect of the present application, a computer program product comprising instructions is provided. When the program is executed by one or more processors, the one or more processors execute the method according to the second aspect of the present application.

[0042] According to the tenth aspect of the present application, a non-transitory storage medium is provided, carrying instructions of a program code for executing the method according to the second aspect of the present application.

[0043] The specific nature of at least one exemplary embodiment and other objects, advantages, features and uses of the at least one exemplary embodiment will become apparent from the following description of the examples taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Reference will now be made, by way of example, to the accompanying drawings which show exemplary embodiments of the present application, in which:

[0045] Figure 1 shows a schematic representation of a spatial arrangement of haptic actuators of a haptic device in a first multi-dimensional space according to at least one exemplary embodiment;

[0046] Figure 2 shows a schematic representation of a spatial arrangement of haptic actuators of a haptic device in a second multi-dimensional space according to at least one exemplary embodiment;

[0047] Figure 3 According to at least one exemplary embodiment, Figure 1 The spatial arrangement of the tactile device is converted into the spatial arrangement of another tactile device;

[0048] Figure 4shows a schematic representation of a body model according to at least one exemplary embodiment;

[0049] Figure 5 A representative embodiment according to at least one exemplary embodiment is shown. Figure 1 or Figure 2 Examples of signaling / parsing schemes or examples of encoding / decoding schemes for spatially arranged data;

[0050] Figure 6 A method of transmitting a representation through a communication network according to at least one exemplary embodiment is shown. Figure 1 or Figure 2 two remote devices of spatially arranged data;

[0051] Figure 7 A carrying representative according to at least one exemplary embodiment is shown Figure 1 or Figure 2 Examples of containers for spatially arranged data;

[0052] Figure 8 A method for representing a Figure 4 A schematic block diagram of the steps of a method for signaling data of a mapping between one or more body parts of a body model and a haptic device to a container;

[0053] Fig. 9 A method of parsing a representation from a container according to at least one exemplary embodiment is shown. Figure 4 A schematic block diagram of the steps of a method for mapping data between one or more body parts of a body model and a haptic device;

[0054] Fig.10 A schematic block diagram of an example of a system in which various aspects and exemplary embodiments are implemented is illustrated.

[0055] Like reference numerals may be used in different drawings to denote similar components. DETAILED DESCRIPTION

[0056] At least one exemplary embodiment will be described more fully below with reference to the accompanying drawings, in which examples of at least one exemplary embodiment are illustrated. However, the exemplary embodiments may be embodied in many alternative forms and should not be construed as being limited to the examples set forth herein. Therefore, it should be understood that there is no intention to limit the exemplary embodiments to the specific forms disclosed. On the contrary, the disclosure is intended to cover all modifications, equivalents, and alternatives that fall within the spirit and scope of the present application.

[0057] At least one of the aspects generally relates to methods and apparatus for signaling information representing a mapping between a body part of a body model targeted by a haptic effect and a haptic device into a container. The haptic device includes a set of haptic actuators configured to present a haptic effect on a body part intended to receive the haptic effect.

[0058] The body model is represented, for example, by a skeleton composed of joints and segments, a mesh, or by body parts and groups of body parts, wherein a group of body parts includes one or more body parts and each is identified by a specific identifier.

[0059] Signaling the mapping information includes writing first data representing a body part targeted by the haptic effect into a container, the first data identifying the body part. Signaling also includes writing second data representing a spatial distribution of a set of haptic actuators of a haptic device on the body part in a determined multi-dimensional space (e.g., in a three-dimensional (3D) space or in a two-dimensional (2D) space) into the container.

[0060] One or more aspects of the present principles also relate to corresponding methods and apparatus for parsing information representing a mapping between a body part and a haptic device from a container.

[0061] Exemplary embodiments of the present principles are directed to methods and apparatus for encoding information representing data of a mapping between a body part and a haptic device into a container; and also directed to corresponding methods and apparatus for decoding information representing data of a mapping between a body part and a haptic device from a container.

[0062] In short, the present invention relates to signaling / parsing and / or encoding / decoding haptic-related data to enable presentation of one or more haptic effects by a haptic device arranged on a human body (i.e., on at least a portion of a human body). The haptic-related data may correspond to data representing one or more portions of a body model representing a human body to which a haptic effect is to be applied and to data representing the haptic device, and more specifically to an arrangement of actuators forming the haptic device.

[0063] Using the second data to indicate how one or more actuators forming the haptic device are spatially arranged or distributed (on the body part targeted by the haptic effect) enables a haptic rendering engine receiving the second data to optimize rendering of the haptic effect.

[0064] Receiving the second data may also enable the haptic rendering engine to determine the type of haptic device signaled by the second data (and from which parameters defining the haptic effect to be rendered may, for example, be calculated), and to adjust / convert the parameters of the haptic effect based on the haptic device available at the rendering stage (which may be different from the haptic device signaled in the container).

[0065] A container may correspond, for example, to a bitstream, a network packet, or a file (eg, a text file or a binary file).

[0066] A binary structure such as a binary file (e.g., ISOBMFF file) is an example of a container. Binary files define a syntax for parsing and understanding the file structure. They have a beginning and an end, and usually hold self-contained information. Binary files are usually transmitted and stored as a whole.

[0067] However, for transmission over a network, such as using the Synchronous Collaboration Transport Protocol (SCTP), Interactive Real-Time Protocol (IRTP), Efficient Transport Protocol (ETP), Real Time Network Protocol (RTNP), or Real Time application-level Protocol for distributed Interactive media (RTP / I), binary files can be further segmented into smaller file units, such as transport packets.

[0068] Figure 1 A schematic representation of an arrangement of an actuator group of a haptic device on a body part 1 according to at least one exemplary embodiment is illustrated.

[0069] Body part 1 corresponds to a body model (e.g. Figure 4 Elements of the body model described.

[0070] The body model represents a human body and is used to identify various parts of the human body.

[0071] The body part 1 corresponds, for example, to the trunk, a part of the arm or a part of the leg.

[0072] exist Figure 1 In the figure, body part 1 is represented by a cylinder, and only the front of the cylinder is visible. Of course, the form of the body part is not limited to a cylinder, but can be any form.

[0073] according to Figure 1 By way of non-limiting example, a haptic device associated with a body part includes 40 haptic actuators, Figure 1 Only 20 tactile actuators 100 to 104, 110 to 114, 120 to 124 and 130 to 134 are visible.

[0074] Figure 1 The 20 tactile actuators of the tactile device are associated with the front side of the body part 1 , and the 20 tactile actuators (not shown) of the tactile device are associated with the back (or dorsal) side of the body part 1 .

[0075] according to Figure 1 In an example, the spatial distribution of the tactile actuators is signaled into a container (intended for transmission to a decoder and / or a tactile rendering engine) relative to a Cartesian 3D space (X, Y, Z), which Cartesian 3D space can, for example, be associated with body part 1 or with a body model to which body part 1 belongs.

[0076] Spatial distribution, for example, is represented by a vector (in [N x ,N y ,N z ]) is transmitted in the following signaling format:

[0077] -First information (the first element of the vector 'N x ') represents the number of tactile actuators according to a first dimension of the 3D space (according to the X-axis, which represents, for example, the vertical axis); N x may be viewed as the resolution of the haptic actuators according to the first dimension, i.e. the number of haptic actuators per haptic device according to the first dimension;

[0078] - Second information (the second element of the vector 'N y ') represents the number of tactile actuators according to the second dimension of the 3D space (according to the Y axis, which represents, for example, the horizontal axis); N y may be viewed as the resolution of the haptic actuators according to the second dimension, i.e. the number of haptic actuators per haptic device according to the second dimension; and

[0079] - The third information (the third element of the vector 'N z ') represents the number of tactile actuators according to the third dimension of the 3D space (according to the Z axis, which represents, for example, the depth axis); N z It can be seen as the resolution of the haptic actuators according to the third dimension, ie the number of haptic actuators per haptic device according to the third dimension.

[0080] The information representing the maximum resolution of the haptic device (ie, the total number of haptic actuators of each haptic device) can be obtained from the second data in the form of N x *N y *N z get.

[0081] The spatial distribution of the tactile actuators of the tactile device can be signaled or encoded as follows: [5,4,2] to obtain a total of 40 tactile actuators (5*4*2) according to 5 tactile actuators according to the X-axis, 4 tactile actuators according to the Y-axis, and 2 tactile actuators according to the Z-axis.

[0082] For each face of the body part, the haptic actuators are arranged according to 4 columns and 5 rows.

[0083] The first column (on the left hand side) includes five haptic actuators 100, 101, 102, 103, and 104. The second column is to the right of the first column and includes five haptic actuators 110, 111, 112, 113, and 114. The third column is to the right of the second column and includes five haptic actuators 120, 121, 122, 123, and 124. The fourth column is to the right of the third column and includes five haptic actuators 130, 131, 132, 133, and 134.

[0084] The tactile devices are arranged according to a matrix on each side of the body part, the matrix comprising 5 rows and 4 columns.

[0085] According to another example, the spatial distribution of the haptic device includes a single actuator signaled or encoded as [1, 1, 1].

[0086] According to a specific and non-limiting embodiment, the vector [N x ,N y ,N z ]'s third element 'N z 'Less than or equal to 2, that is, N z =1 or N z = 2. According to this embodiment, the maximum number of layers of actuators according to the Z axis (depth) is equal to 2, wherein the first layer corresponds to the front and the second layer corresponds to the back of the body part (or haptic device).

[0087] The signaling / encoding spatial distribution as explained previously enables, for example, a decoder of a receiving container and / or a haptic rendering engine to obtain information about the haptic device via the specific spatial distribution of one or more haptic actuators, thereby enabling identification of the haptic device.

[0088] Furthermore, the signaling / encoding spatial distribution as explained previously enables a decoder and / or a haptic rendering engine of the receiving container to adapt the haptic effect to be rendered by the haptic device to the specific spatial distribution of the signaled one or more haptic actuators in the container.

[0089] According to a particular embodiment, in addition to the second data, third data representing the position of a specific haptic actuator in the group of haptic actuators of the haptic device is added to the container.

[0090] For example, third data is added to the container when the haptic device includes multiple actuators (e.g., when the spatial distribution is signaled / encoded with data different from [1,1,1]) and / or when only some of the haptic actuators of the haptic device are used to render the haptic effect.

[0091] The location or position of a specific haptic actuator is represented by a vector (in [C x ,C y ,C z ]) is transmitted or encoded in the following signaling format:

[0092] -First information (the first element of the vector 'C x ') represents the index of the row to which the specific tactile actuator belongs, i.e. according to the first dimension of the 3D space (according to the X-axis);

[0093] - Second information (the second element of the vector 'C y ') represents the index of the column to which the specific tactile actuator belongs, i.e. according to the second dimension of the 3D space (according to the Y axis); and

[0094] - The third information (the third element of the vector 'C z ') represents the index of the face (front or back) to which the tactile actuator belongs, i.e. according to the third dimension of the 3D space (according to the Z axis).

[0095] For example, to identify Figure 1 The tactile actuator referenced by 123 in the figure, the third information corresponds to: [3, 2, 0]; in order to identify Figure 1 For the tactile actuator referenced by 111, the third information corresponds to: [1, 1, 0]; and for identifying a tactile actuator located at the same position as the tactile actuator but behind / backward, the third information corresponds to: [3, 2, 1].

[0096] Therefore, by identifying a specific haptic actuator (eg, haptic actuator 123 ) to signal the spatial distribution of the haptic devices, the signaling / encoding of information corresponds to: [5, 4, 2][3, 2, 0].

[0097] According to another particular embodiment, in addition to the third data, fourth data representing a distance relative to a specific haptic actuator identified via the third data are added to the container. The distance is defined in a 3D space.

[0098] For example, when presenting a haptic effect through a grouping of haptic actuators, incorporating the distance enables identification of the grouping of haptic actuators in the group of haptic actuators forming the haptic device.

[0099] Adding the fourth data enables optimizing information encoding by minimizing the amount of data used to signal / encode the information. Using distances expressed in 3D space is more efficient than identifying each tactile actuator in the tactile actuator group individually via the third data as explained above for a specific tactile actuator.

[0100] The distance to a specific haptic actuator is represented by a vector (in [R x ,R y ,R z ]) is transmitted or encoded in the following signaling format:

[0101] -First information (the first element of the vector 'R x ') represents the amount of the selected tactile actuator that deviates from the specific tactile actuator according to the first dimension of the 3D space (according to the X-axis);

[0102] - Second information (the second element of the vector 'R y ') represents the amount by which the selected tactile actuator deviates from the specific tactile actuator according to a second dimension of the 3D space (according to the Y-axis); and

[0103] - The third information (the third element of the vector 'R z ') represents the amount by which the selected tactile actuator deviates from a specific tactile actuator according to the third dimension of the 3D space (according to the Z-axis).

[0104] For example, to identify the haptic actuator group including the haptic actuators 122 , 123 , and 124 by using the third information identifying the specific haptic actuator 123 , the distance may be signaled or encoded as: [1, 0, 0].

[0105] Therefore, if [C x ,C y ,C z ] corresponds to third data identifying the location of a specific tactile actuator, and [R x ,R y ,R z ] Corresponding to the fourth data defining the distance, the tactile actuator group presenting the tactile effect will include a tactile actuator having a position [P x ,P y ,P z ] for each tactile actuator, the position [P x ,P y ,P z ] verified as:

[0106] C x -R x ≤P x ≤C x +R x ;and

[0107] C y -R y ≤P y ≤C y +R y ;and

[0108] C z -R z ≤P z ≤C z +R z .

[0109] According to a further embodiment, data representing the haptic animation is signaled and / or encoded into the container.

[0110] A haptic animation corresponds to a haptic effect or a series of haptic effects presented via a plurality of haptic actuators according to a path, such as defined by a starting or initial point and an ending or termination point.

[0111] The haptic animation may correspond to a haptic effect or to a series of haptic effects distributed over one or more parts of the user's body that receive such haptic effect(s) and presented via a series of haptic actuators arranged on one or more parts of the user's body.

[0112] The starting point corresponds to, for example, a first specific haptic actuator or to a first haptic actuator group. The ending point corresponds to, for example, a second specific haptic actuator or to a second haptic actuator group. The first haptic actuator and the second haptic actuator belong to, for example, the same haptic device or to two different haptic devices.

[0113] As explained previously, a first specific haptic actuator (or a first grouping of haptic devices) is advantageously signaled into the container using the second data and optionally the third and fourth data.

[0114] As explained previously, a second specific haptic actuator (or a second grouping of haptic devices) is advantageously signaled into the container using the second data and optionally the third and fourth data.

[0115] The haptic rendering engine of the receiving container parses the data contained in the container to read data representing the haptic animation, specifically data representing the starting point and the ending point (ie, the first haptic actuator and the second haptic actuator).

[0116] The rendering engine determines or calculates a path of haptic actuators based on the first haptic actuator and the second haptic actuator, the path starting from the first haptic actuator and ending at the second haptic actuator. The path includes multiple haptic actuators belonging to one or more haptic devices arranged on different parts of the user's body that receive the haptic effect.

[0117] The path is further determined according to the haptic effect to be presented, such as according to the duration of the haptic effect, according to the magnitude of the haptic effect at the starting point, and according to the magnitude of the haptic effect at the ending point.

[0118] Figure 2 A schematic representation of an arrangement of an actuator group of a haptic device on a body part 2 in a second multi-dimensional space according to at least one exemplary embodiment is illustrated.

[0119] Body Part 2 and Figure 1 , and the tactile device is also associated with this body part 2.

[0120] according to Figure 2 By way of non-limiting example, the haptic device associated with body part 2 includes 40 haptic actuators, Figure 2 Only 20 tactile actuators 200 to 204, 210 to 214, 220 to 224 and 230 to 234 are visible.

[0121] according to Figure 2 In an example, the spatial distribution of the haptic actuators is signaled to a container (which is intended to be sent to a decoder and / or a haptic rendering engine) relative to a 2D space (θ, Y), which can be associated with a body part 2 or with a body model to which the body part 2 belongs. The 2D space can be assimilated to a polar coordinate 2D space, for example. The 2D space can be assimilated to a polar 2D space, for example.

[0122] For example, the spatial distribution is represented by a vector ([N θ ,N Y ]) is transmitted in the following signaling format:

[0123] -First information (the first element of the vector 'N θ ') represents the number of angular regions into which the body part is segmented or divided according to the first dimension of the 2D space; and

[0124] - Second information (the second element of the vector 'N Y ') represents the number of angular regions into which the body part is segmented or divided according to the second dimension of the 2D space.

[0125] according to Figure 2 As an example, the spatial distribution of the tactile actuators of the tactile device can be signaled or encoded as follows: [8,5] with 8 angular areas 20 to 27 and 5 tactile actuators according to the longitudinal axis of each angular area, resulting in a total of 40 tactile actuators (8*5).

[0126] according to Figure 2In the example of , only the tactile actuators of the angular areas 26 , 27 , 20 and 21 are visible, corresponding to the front side of body part 2 .

[0127] According to a specific embodiment, in addition to the second data representing the spatial distribution of the haptic devices 200 to 204 , 210 to 214 , 220 to 224 , and 230 to 234 , third data representing the position of a specific haptic actuator in the haptic actuator group of the haptic device is added to the container.

[0128] For example, when the haptic device includes a plurality of actuators and / or when only a portion of the haptic actuators of the haptic device are used to render the haptic effect, the third data is added to the container.

[0129] The location or position of a specific haptic actuator is represented by a vector (in [C θ ,C Y ]) is transmitted or encoded in the following signaling format:

[0130] -First information (the first element of the vector 'C θ ') represents the index of the angular region to which a specific tactile actuator belongs; and

[0131] - Second information (the second element of the vector 'C Y ') represents the index of a specific tactile actuator in the angular region identified as the first information, where the index of the actuator in a given angular region starts at 0 (eg, the actuator at the bottom), and the second information refers to the second dimension of the 2D space.

[0132] For example, to identify Figure 2 The tactile actuator with reference number 223, the third information corresponds to: [0,3], the angular area 20 corresponds to the reference angular area with index 0, the angular area 21 with index 1, and so on, until the angular area 27 with index 7; and in order to identify Figure 2 For the tactile actuator referenced as 211, the third information corresponds to: [7, 1].

[0133] Therefore, by identifying a specific haptic actuator (eg, haptic actuator 123 ) to signal the spatial distribution of the haptic devices, the signaling / encoding of information corresponds to: [8, 5][0, 3].

[0134] According to another particular embodiment, in addition to the third data, fourth data representing a distance relative to a specific haptic actuator identified via the third data are added to the container. The distance is defined in a 2D space.

[0135] For example, when presenting a haptic effect through a grouping of haptic actuators, incorporating the distance enables identification of the grouping of haptic actuators in the group of haptic actuators forming the haptic device.

[0136] The distance to a specific haptic actuator is represented by a vector (in [R θ ,R Y ]) is transmitted or encoded in the following signaling format:

[0137] -First information (the first element of the vector 'R θ ') represents the number of angular regions selected around the reference angular region according to the first dimension of the 2D space; and

[0138] - Second information (the first element of the vector 'R Y ') represents the number of selected tactile actuators that deviate from a specific tactile actuator according to the second dimension of the 2D space.

[0139] For example, to identify the haptic actuator group including the haptic actuators 222 , 223 , and 224 by using the third information identifying the specific haptic actuator 223 , the distance may be signaled or encoded as: [0, 1].

[0140] Therefore, if [C θ ,C Y ] corresponds to third data identifying the location of a specific tactile actuator, and [R θ ,R Y ] Corresponding to the fourth data defining the distance, the tactile actuator group presenting the tactile effect will include a tactile actuator having a position [P θ ,P y ] for each tactile actuator, the position [P θ ,P y ] verified as:

[0141] C θ -R θ ≤P θ ≤C θ +R θ ;and

[0142] C Y -R Y ≤P Y ≤C Y +R Y

[0143] Figure 3 Transforming from data representing a spatial distribution of haptic actuators of a first haptic device to data representing a spatial distribution of haptic actuators of a second haptic device according to at least one example embodiment is illustrated.

[0144] Figure 3The left side shows a spatial distribution 31 of haptic actuators of a first haptic device, and the right side shows a spatial distribution 32 of haptic actuators of a second haptic device different from the first haptic device.

[0145] The haptic actuator of the first haptic device and the haptic actuator of the second haptic device are associated with or arranged on the same body part.

[0146] The spatial distribution 31 of the first tactile device is, for example, represented by the following second data: x ,N1 y ,N1 z ] defined in 3D space.

[0147] The spatial distribution 32 of the second tactile device is, for example, represented by the following second data: x ,N2 y ,N2 z ] defined in 3D space.

[0148] The first tactile device corresponds to, for example, Figure 1 A haptic device is described which comprises 40 haptic actuators distributed in a space defined by [5, 4, 2].

[0149] The second tactile device includes six tactile actuators, two of which are arranged at the upper front portion, and two of which are arranged at the upper front portion. Figure 3 ) is arranged in the upper back part, and 2 tactile actuators 300, 310 are located in the side body part.

[0150] The second haptic device includes 6 haptic actuators distributed in a space defined by [2, 2, 2].

[0151] The container received by the haptic rendering engine or decoder may include the following data or information:

[0152] - second data representing the spatial distribution of the first tactile device, namely [N1 x ,N1 y ,N1 z ];

[0153] - third data representing a determined tactile actuator in the tactile actuator group of the first tactile device, namely [C1 x ,C1 y ,C1 z ];and

[0154] - The fourth data representing the distance, namely [R1 x ,R1 y ,R1 z ].

[0155] When a second tactile device associated with the tactile rendering engine to render the tactile effect is different from the first tactile device that is signaled into the container, the tactile rendering engine converts the third data and the fourth data to determine which tactile actuator(s) of the second tactile device correspond to the tactile actuators of the first tactile device identified by the third data and the fourth data.

[0156] To achieve this purpose, the haptic rendering engine converts the third data and the fourth data contained in the container into a determined haptic actuator (denoted as [C2 x ,C2 y ,C2 z ]) and the distance representing the second tactile device (denoted as [R1 x ,R1 y ,R1 z ]).

[0157] Parameter C2 x 、C2 y and C2 z For example, it is obtained through the following equation: The tactile rendering engine knows that N2 x 、N2 y and N2 z (These parameters are, for example, stored in a memory of the tactile rendering engine or retrieved from a second tactile device connected to the tactile rendering engine) and the tactile rendering engine retrieves N1 from the received container x 、N1 y and N1 z , C1 x 、C1 y and C1 z and R1 x 、R1 y and R1 z :

[0158]

[0159] Applying equations 1 to 6 to Figure 3 A non-limiting example of where [N1 x, N1 y ,N1 z ]=[5,4,2]、[C1 x ,C1 y ,C1 z ]=[3,2,0](tactile actuator 123), [R1 x ,R1 y ,R1 z ]=[1,1,0] and [N2 x, N2 y ,N2 z]=[2,2,2], we get [C2 x ,C2 y ,C2 z ]=[1,1,0](haptic actuator 311) and [R2 x ,R2 y ,R2 z ]=[0,0,0].

[0160] Equations 1 to 6 are stored, for example, in a memory of a haptic rendering engine. According to a variant, equations 1 to 6 are encoded to be included and transmitted with first data, second data, third data and fourth data.

[0161] Naturally, the equations used to calculate the transformation depend on the multi-dimensional space used to represent the second data, the third data and the fourth data.

[0162] Figure 4 Illustrated is a schematic representation of a body model 4 according to at least one exemplary embodiment.

[0163] Figure 4 The body model 4 represents the human body and is used to identify various parts of the human body

[0164] The body model 4 includes a first group of body parts and a second group of body parts. Each body part in the first group of body parts belongs to one or more groups in the second group of body parts, and each group in the second group of body parts includes one or more body parts in the second group of body parts.

[0165] The body parts in the first set of body parts may, for example, correspond to the groups in the second set of groupings of body parts.

[0166] For example, the second set of body part groupings includes the following groupings:

[0167] - a grouping called "head" 401, including the following body parts: head and neck;

[0168] - a group called "torso" 402, including only one body part, the torso;

[0169] - a group called "waist" 403, comprising only one body part, namely the waist;

[0170] - a grouping called "Legs", including the following body parts: upper leg 404, lower leg 405, and foot 406;

[0171] - a grouping called "arm", including the following body parts: upper arm 407, lower arm 408 and hand 409;

[0172] - a grouping called "hand" 42, comprising the following body parts: palm 409 and fingers 410 to 414;

[0173] - a grouping called "fingers", including the following body parts: thumb 4101, index finger 411, middle finger 412, ring finger 413 and little finger 414;

[0174] - A grouping called "knuckles" includes the following body parts: first knuckle, second knuckle, and third knuckle;

[0175] - A group called "foot", which includes the following body parts: plant and toes;

[0176] - a grouping called "lower limbs", corresponding to the lower part of the body model and including the following body parts: waist and legs;

[0177] - a group called "upper limbs", corresponding to the higher part of the body model 40 and comprising the following body parts: head, chest and arms;

[0178] a group called "right part", corresponding to the right half of the body model 40 and comprising the following body parts: head, chest, waist, right arm and right leg;

[0179] a group called “left part” 41 , corresponding to the left half of the body model 4 and comprising the following body parts: head, chest, waist, left arm and left leg;

[0180] a group called “front”, corresponding to the front of the body model 4 and comprising the following body parts: front head, front chest, front waist, front arms and front legs;

[0181] a group called "back", corresponding to the back of the body model 4, and including the following body parts: back of the head, back of the chest, back of the waist, back of the arms and back of the legs; and

[0182] A group called "all" corresponds to the overall body model 4 and brings together all groups and body parts, namely: head, chest, waist, arms and legs.

[0183] Of course, the groups and body parts constituting the body model are not limited to the above examples. The number of groups and the number of body parts are also not limited to the above examples and can be any number.

[0184] Each grouping of body parts can be identified with, for example, an identifier corresponding to a grammatical element, such as one of the following grammatical elements: "all", "left", "right", "front", "back", "upper limb", "lower limb", "head", "torso", "waist", "legs", "arms", "hands", "feet", "fingers / toes".

[0185] Similarly, each body part may be an identifier having, for example, an identifier corresponding to a grammatical element, which may be the same as the grammatical element identifying the grouping when the body part also corresponds to a grouping. For example, the following grammatical elements may be used to identify body parts when necessary: ​​"head", "torso", "waist", "leg", "arm", "hand", "foot", "fingers / toes", "thumb", "index finger", "middle finger", "ring finger", "little finger", "palm", "sole", "knuckle", "first knuckle", "second knuckle", "third knuckle", "upper arm / leg", "lower arm / leg".

[0186] According to a variant, each body part and each grouping may be identified by an identifier corresponding to a numerical value (for example: a binary value or a hexadecimal value).

[0187] The one or more identifiers of the one or more groups are used to signal (i.e., indicate) in the container via the first data which body part or parts are targeted by the haptic effect. If necessary, the one or more targeted body parts are signaled using the one or more identifiers of the one or more targeted body parts in addition to the one or more identifiers of the one or more groups.

[0188] In order to signal or encode data representing one or more body parts (referred to as targeted body parts) in a container containing information to be sent, for example, from a transmitter to a receiver, from a tactile data generator to a tactile rendering engine, or from an encoder to a decoder, first data identifying the targeted body parts can be written or encoded into the container with reference to each grouping that includes the targeted body parts.

[0189] For example, where the targeted body part corresponds to the third knuckle of each finger 410 to 414 of the two hands, the first data for signaling this information may correspond to: “third knuckle of the hand”.

[0190] According to another example, in order to signal the three knuckles of the thumb of the left hand, the first data for signaling this information may correspond to: "left hand thumb"

[0191] According to another example, in order to identify the third joint of the right hand thumb, the first data used to signal the information may correspond to: “right hand thumb third” or “right hand thumb third joint”, depending on how the body parts are grouped in the body model 4 .

[0192] The use of groupings of body parts, taking advantage of the symmetry of the body model 4 (right / left and front / back), enables clear and unambiguous identification of each body part of the body model that may be targeted by a haptic effect.

[0193] In addition, the use of groupings of body parts enables efficient encoding or signaling of information identifying the targeted body parts, particularly when the targeted body parts include multiple body parts of the same type (e.g., knuckles, both hands, the same part of the leg), as the amount of data used to encode or signal the information is reduced.

[0194] According to a specific embodiment, the signal transmission of the targeted body part located on a specific side of the body model 4 (i.e., on the left-hand side or on the right-hand side) is obtained by writing data identifying the side (i.e., the left side or the right side) with reference to each group of "left side" and "left side" of each body part into a container.

[0195] According to a further specific embodiment, the signal transmission of the targeted body part located on a specific face of the body model (i.e., the front or the back (or dorsal) face) is obtained by writing data identifying the face (i.e., the front or the back) with reference to each grouping of "front" and "back" of each body part into a container.

[0196] According to a further specific embodiment, first data for signaling a targeted body part (or encoding data identifying a targeted body part) includes a first identifier (e.g., a grammatical element describing the targeted body part) that identifies the targeted body part, and a second identifier that identifies one or more groups that include the targeted body part (e.g., one or more grammatical elements that describe one or more groups that include the targeted body part).

[0197] In another specific embodiment, the first data includes a third identifier identifying a logical operator. The logical operator corresponds to, for example:

[0198] - logical operator NOT; and / or

[0199] - logical operator AND; and / or

[0200] - Logical operator OR; and / or

[0201] - logical operator XOR; and / or

[0202] - Logical operator NOR.

[0203] For example, the upper arm and the lower arm of one or both arms may be signaled and / or encoded into the container with the following first data:

[0204] "left arm NOT hand" or left arm, and

[0205] The two arms are “arms NOT hands”.

[0206] The use of the logical operator NOT enables the exclusion of one or more body parts from a grouping. The exclusion is signaled using the identifier of the logical operator NOT and the identifier of the excluded body part. In the above example, the body part or group "hand" is excluded from the group "arm" to signal and / or encode the identification of the lower arm and upper arm of the arm.

[0207] Another way to signal this information is: "ARM UP AND LOWER".

[0208] The use of logical operators may be more efficient than identifying each targeted body part individually. For example, to identify all hands except the first knuckle of each finger, the following first data may be used to signal and / or encode information into a container: "hand NOT first knuckle".

[0209] The different embodiments described above can be combined with each other in any combination.

[0210] Figure 5 Schematically illustrated are schematic diagrams of signaling / parsing and / or encoding / decoding schemes for haptic-related data according to certain embodiments of the present principles.

[0211] The haptic-related data 501 advantageously comprises first data identifying the targeted body part and first data representing the set of haptic actuators forming the haptic device intended to present the haptic effect in a determined multidimensional space (e.g. Figure 1 3D space or Figure 2 The tactile device whose spatial distribution is encoded into the container 502 may be referred to as a reference tactile device.

[0212] According to a variant, the haptic-related data 501 also include third data representing the position of a determined haptic actuator of the group of haptic actuators in the determined multidimensional space.

[0213] According to another variant, the haptic-related data 501 further comprise fourth data representing a distance relative to a determined haptic actuator represented by the third data, the distance being defined in a determined multi-dimensional space.

[0214] According to additional variants, the haptic-related data 501 also includes data representing the haptic effect to be presented by the haptic device on the targeted body part. The data representing the haptic effect to be applied to the targeted body part includes, for example:

[0215] - the type of tactile (and / or sensory) effect (e.g., vibrotactile effect, force effect, hardness effect, texture effect, temperature effect, wind effect, pain effect, olfactory effect, light-based effect, etc.); and / or

[0216] - the amplitude, phase and / or frequency of one or more signals representing a haptic effect; and / or

[0217] - the start time of the haptic effect; and / or

[0218] - the duration of the haptic effect; and / or

[0219] - An identifier of the haptic effect, which can be used by the system receiving the data to retrieve (eg, from the system's memory) the parameters of the haptic effect.

[0220] The haptic related data 501 are signaled and / or encoded in the form of encoded data via a signaling process and / or encoding process 51 in the module into a container, which corresponds to, for example, a file, a bit stream, a data packet, a data frame.

[0221] The container 502 is sent to a module implementing a parsing and / or decoding process 52 to parse, read and / or decode the encoded data contained in the container 502 to obtain decoded haptic-related data 503. The modules implementing the signaling / encoding process and the parsing / decoding process may be hardware, software, or a combination of hardware and software.

[0222] The signaling process 51 includes a writing operation, according to which first data identifying the targeted body part and second data indicating the spatial distribution of the reference tactile device are written into the container 502 .

[0223] During the encoding process, the write operation of the signaling process can be interpreted as an encoding operation.

[0224] The parsing process 52 may, for example, correspond to a reverse operation of the signaling process 51 , in which the information and data contained in the container 502 are parsed and read to obtain the data and information contained in the container 502 .

[0225] In the decoding process, the reading operation of the parsing process may be interpreted as a decoding operation to decode the data of the container 502 encoded in the container 502 by the encoding process.

[0226] according to Figure 6 In the example shown in FIG. 1 , in the context of transmission between two remote devices 61 and 62 (of the type of device 10) via a communication network 60, the device 61 includes a device configured to implement the Figures 1 to 5 and Figure 8 The means for signaling and / or encoding data described herein, and the device 62 comprises means configured to implement the Figures 1 to 5 and Fig. 9 The described tools for parsing and / or decoding data.

[0227] According to an example, the network 60 is a LAN or WLAN network adapted to broadcast and / or stream the haptic related data from the device 61 to parsing / decoding / rendering devices including the device 62 .

[0228] For transmission via the device 61 , the signal carries the container 7 .

[0229] Figure 7 An example of a container 7 is shown when haptic-related data is sent via a packet-based transmission protocol. Figure 7 An example structure of a haptic related data stream is shown 7. The structure is included in a container that organizes the stream in separate syntax elements.

[0230] The structure may include a header part 71, which is a set of data common to each syntax element of the stream. For example, the header part includes metadata about the syntax elements, describing the properties and role of each syntax element.

[0231] The structure may include a payload including grammatical elements 72 identifying the body part targeted by the haptic effect and grammatical elements 72 characterizing a reference haptic effect, such as a spatial distribution or maximum resolution, haptic actuators or groupings of haptic actuators of a haptic device intended to render the haptic effect.

[0232] Figure 8 A schematic block diagram illustrating steps of a method for signaling into a container a mapping between a body part of a body model targeted by a haptic effect and a haptic device including a set of haptic actuators configured to present the haptic effect on at least a portion of the body part according to at least one exemplary embodiment.

[0233] The targeted body part is defined with reference to the body model, e.g. Figure 4 The body model described or any representation of a body model known to those skilled in the art.

[0234] Figure 8Also shown is a schematic block diagram of the steps of a method of encoding data representing a map according to a specific and non-limiting embodiment.The writing operations described with reference to the signaling method may be interpreted as encoding operations for the encoding method.

[0235] In a first step 81 , first data representing a body part are written into a container.

[0236] In a second step 82, second data representing the spatial distribution of the tactile actuator group in the determined multidimensional space (or representing the maximum resolution) is written into the container. The multidimensional space corresponds to, for example, the reference Figure 1 The 3D space or reference described Figure 2 Described 2D space.

[0237] In an exemplary embodiment, the method further includes, when the tactile actuator group includes a plurality of tactile actuators, writing third data representing a position of a determined tactile actuator in the tactile actuator group in the determined multi-dimensional space into the container.

[0238] In an exemplary embodiment, the method further includes writing fourth data representing a distance relative to the determined haptic actuator into the container, the distance being defined in the determined multi-dimensional space.

[0239] In a supplementary embodiment, the method also includes writing data representing a tactile animation into a container. The data representing the tactile animation include, for example, data representing the starting point of the tactile animation and data representing the ending point of the tactile animation. The starting point may correspond to a determined tactile actuator of the first tactile device or a tactile actuator group corresponding to the first tactile device. The data representing the starting point is advantageously written in the form of second data, third data, and / or fourth data. In the same manner, the ending point may correspond to another determined tactile actuator of the first tactile device (or a second tactile device different from the first tactile device) or another tactile actuator group corresponding to the first tactile device (or the second tactile device). The data representing the ending point is advantageously written in the form of second data, third data, and / or fourth data.

[0240] In another exemplary embodiment, the method further includes writing data representing a haptic device type associated with the haptic device, with reference to a spatial distribution of the haptic device type, into the container.

[0241] In additional exemplary embodiments, the method further includes writing sixth data representing the haptic effect into the container.

[0242] Fig. 9A schematic block diagram of steps of a method for parsing, from a received container, data representing a mapping between a body part of a body model targeted by a haptic effect and a haptic device comprising a set of haptic actuators configured to present the haptic effect on at least a portion of the body part according to at least one exemplary embodiment is shown.

[0243] The body part to be targeted is defined with reference to the body model, e.g. Figure 4 The body model described or any representation of a body model known to those skilled in the art.

[0244] Fig. 9 Also shown is a schematic block diagram of the steps of a method of decoding data representing a map according to a specific and non-limiting embodiment.The reading operations described with reference to the parsing method may be interpreted as decoding operations for the decoding method.

[0245] In a first step 91, first data representing the body part are read from a container.

[0246] In a second step 82, second data representing the spatial distribution of the tactile actuator group in a determined multidimensional space (or representing the maximum resolution) are read from the container. The multidimensional space corresponds to, for example, the reference Figure 1 The 3D space or reference described Figure 2 Described 2D space.

[0247] In an exemplary embodiment, the method further includes, when the tactile actuator group includes a plurality of tactile actuators, reading third data representing a position of a determined tactile actuator in the tactile actuator group in the determined multi-dimensional space from the container.

[0248] In an exemplary embodiment, the method further includes reading fourth data representing a distance relative to the determined haptic actuator from the container, the distance being defined in the determined multi-dimensional space.

[0249] In a supplementary embodiment, the method also includes reading data representing a tactile animation from a container. The data representing the tactile animation include, for example, data representing the starting point of the tactile animation and data representing the ending point of the tactile animation. The starting point may correspond to a determined tactile actuator of the first tactile device or to a tactile actuator group of the first tactile device. The data representing the starting point is advantageously included in the container in the form of second data, third data, and / or fourth data. In the same manner, the ending point may correspond to another determined tactile actuator of the first tactile device (or a second tactile device different from the first tactile device) or to another tactile actuator group of the first tactile device (or the second tactile device). The data representing the ending point is advantageously included in the container in the form of second data, third data, and / or fourth data.

[0250] In another exemplary embodiment, the method further includes reading, from the container, data representing a haptic device type associated with the haptic device, with reference to a spatial distribution of the haptic device type.

[0251] In additional exemplary embodiments, the method further includes reading sixth data representing the haptic effect from the container.

[0252] Fig.10 A schematic block diagram illustrating an example of a system in which various aspects and exemplary embodiments are implemented is shown.

[0253] The system 10 may be embedded as one or more devices, including various components described below. In various embodiments, the system 10 may be configured to implement one or more aspects described in this application.

[0254] Examples of equipment that may constitute all or part of system 10 include a personal computer, a laptop computer, a smart phone, a tablet computer, a digital multimedia set-top box, a digital television receiver, a personal video recording system, a connected home appliance, a connected vehicle and its associated processing system, a head-mounted display device (HMD, see-through glasses), a tactile sensor or actuator, a "cave" (a system including multiple displays), a server, a tactile encoder, a tactile decoder, a post-processor that processes output from a tactile decoder, a pre-processor that provides input to a tactile encoder, a web server, a set-top box, a wireless (e.g., ), a wearable haptic device, a game controller, a mouse, a mouse pad, a keyboard, a hand rest, a chair, a desk, an XR headset, a head-mounted headset, a bracelet, a head and / or lumbar support device or a seat, any other device for processing tactile data or tactile signals, or any other communication device. The elements of system 10 can be implemented individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 10 can be distributed across multiple ICs and / or discrete components. In various embodiments, system 10 can be communicatively connected to other similar systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports.

[0255] The system 10 may include at least one processor 1001, which is configured to execute instructions loaded therein, for example, to implement various aspects described in the present application. The processor 1001 may include embedded memory, input and output interfaces, and various other circuits known in the art. The system 10 may include at least one memory 1002 (e.g., a volatile memory device and / or a non-volatile memory device). The system 10 may include a storage device 1004, which may include a non-volatile memory and / or a volatile memory, including but not limited to an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 1004 may include an internal storage device, an attached storage device, and / or a network accessible storage device.

[0256] The system 10 may include an encoder / decoder module 1003, which is configured to, for example, process data to provide encoded / decoded tactile signals or data, and the encoder / decoder module 1003 may include its own processor and memory. The encoder / decoder module 1003 may represent a module (or modules) that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 1003 may be implemented as a separate element of the system 93, or may be incorporated into the processor 1001 as a combination of hardware and software known to those skilled in the art.

[0257] Program code to be loaded into the processor 1001 or the encoder / decoder 1003 to perform various aspects described in the present application may be stored in the storage device 1004 and subsequently loaded into the memory 1002 to be executed by the processor 1001. According to various embodiments, during the execution of the processes described in the present application, one or more of the processor 1001, the memory 1002, the storage device 1004, and the encoder / decoder module 1003 may store one or more of various items. Such stored items may include, but are not limited to, tactile-related data, encoded / decoded data of one or more body parts of the recognized body model and / or one or more groups of one or more body parts, bit streams, matrices, variables, and intermediate or final results of equations, formulas, operations, and operation logic processing.

[0258] In several embodiments, memory internal to the processor 1001 and / or encoder / decoder module 1003 may be used to store instructions and provide working memory for processes that may be performed during encoding or decoding.

[0259] However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 1001 or the encoder / decoder module 1003) is used for one or more of these functions. The external memory may be the memory 1002 and / or the storage device 1004, for example, dynamic volatile memory and / or non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store the operating system of the television. In at least one embodiment, a fast external dynamic volatile memory such as RAM may be used as working memory for video encoding and decoding operations, for example for MPEG-V.

[0260] As indicated in block 1005, input to the elements of system 10 may be provided through various input devices. Such input devices include, but are not limited to, (i) an RF section that may receive an RF signal transmitted over the air, such as by a broadcaster, (ii) a composite input terminal, (iii) a USB input terminal, (iv) a phone connector (also known as a phone jack, audio jack, headphone jack, or jack plug) input terminal, and / or (v) an HDMI input terminal.

[0261] In various embodiments, the input device of block 1005 has associated corresponding input processing elements, as known in the art. For example, the RF portion may be associated with elements necessary for: (i) selecting a desired frequency (also referred to as selecting a signal, or limiting the signal band to within a band), (ii) down-converting the selected signal, (iii) limiting the band again to a narrower band to select a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired packet stream. The RF portion of various embodiments may include one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF portion may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or a baseband.

[0262] In one set-top box embodiment, the RF section and its associated input processing elements can receive RF signals transmitted over a wired (e.g., cable) medium. The RF section can then perform frequency selection by filtering, down-converting, and filtering again to a desired frequency band.

[0263] Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions.

[0264] Adding an element may include inserting an element between existing elements, such as, for example, inserting an amplifier and an analog-to-digital converter.In various embodiments, the RF portion may include an antenna.

[0265] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 10 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, in a separate input processing IC or in the processor 1001, as necessary. Similarly, various aspects of USB or HDMI interface processing may be implemented, for example, in a separate interface IC or in the processor 1001, as necessary. The demodulated, error-corrected, and demultiplexed streams may be provided to various processing elements, including, for example, the processor 1001 and the encoder / decoder 1003, which operate in conjunction with memory and storage elements to process the data streams for presentation on an output device, as necessary.

[0266] The various elements of system 10 may be provided within an integrated housing. Within the integrated housing, the various elements may be interconnected and data transferred between them using a suitable connection arrangement 1005, such as an internal bus (including an I2C bus), wiring, and printed circuit boards known in the art.

[0267] The system 10 may include a communication interface 1006 that enables communication with other devices via a communication channel 1060. The communication interface 1006 may include, but is not limited to, a transceiver configured to send and receive data over the communication channel 1060. The communication interface 1006 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented, for example, within a wired and / or wireless medium.

[0268] In various embodiments, a Wi-Fi network such as IEEE 802.11 may be used to stream data to the system 10. The Wi-Fi signals of these embodiments may be received via a communication channel 1060 suitable for Wi-Fi communications and the communication interface 1006. The communication channel 1060 of these embodiments may typically connect to an access point or router that provides access to external networks including the Internet to allow streaming applications and other over-the-top communications.

[0269] Other embodiments may provide streamed data to system 10 using a set top box that delivers the data through the HDMI connection of input box 1005 .

[0270] Other embodiments may use the RF connection of input box 1005 to provide streaming data to system 10 .

[0271] Streamed data may be used as a means of signaling information used by system 10. The signaling information may, for example, include data encoded in a container such as a binary stream or a haptic effect file.

[0272] It should be appreciated that signaling may be implemented in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc. may be used to signal information to a corresponding decoder.

[0273] System 10 can provide signals to various output devices, including display 1070 , speaker 1080 , and other peripheral devices 1090 such as haptic devices / actuators.

[0274] In various embodiments, control signals may be communicated between the system 10 and the display 1070, speakers 1080, or other peripheral devices 1090 using signaling such as AV.Link (audio / video link), consumer electronics control (CEC), audio protocols, universal serial bus (USB), Haptic Industry Forum-Universal Haptic Protocol (HIF UHP), or other communication protocols that enable device-to-device control with or without user intervention.

[0275] Output devices may be communicatively connected to system 10 via dedicated connections through respective interfaces 1007 , 1008 , and 1009 .

[0276] Alternatively, output devices may be connected to system 10 via communication interface 1006 using communication channel 1060. Display 1070, speaker 1080, and / or actuator 1090 may be integrated with other components of system 10 in a single unit in an electronic device such as, for example, a television.

[0277] In various embodiments, the display interface 1007 may include a display driver, such as, for example, a timing controller (TCon) chip.

[0278] For example, if the RF portion of input 1005 is part of a separate set-top box, then display 1070, speaker 1080, and / or actuator 1090 may optionally be separate from one or more of the other components. In various embodiments where display 1070, speaker 1080, and / or actuator 1090 may be external components, output signals may be provided via dedicated output connections, including, for example, an HDMI port, a USB port, or a COMP output.

[0279] exist Figure 1-10In the present invention, various methods are described herein, and each method includes one or more steps or actions to implement the described method. Unless a specific order of steps or actions is required for the correct operation of the method, the order and / or use of specific steps and / or actions can be modified or combined.

[0280] Some examples are described about block diagrams and / or operational flow charts. Each box represents a portion of a circuit element, module or code, which includes one or more executable instructions for implementing (one or more) specified logical functions. It should also be noted that, in other embodiments, the (one or more) functions marked in the box may not occur in the order indicated. For example, depending on the functions involved, two boxes shown in succession can actually be executed substantially concurrently, or sometimes these boxes can be executed in the opposite order.

[0281] The embodiments and aspects described herein may be implemented in, for example, a method or process, an apparatus, a computer program, a data stream, a bit stream, or a signal. Even if only discussed in the context of a single form of embodiment (e.g., discussed only as a method), the embodiments of the features discussed may also be implemented in other forms (e.g., an apparatus or a computer program).

[0282] The method may be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device.

[0283] In addition, the method can be implemented by instructions executed by a processor, and such instructions (and / or data values ​​generated by the implementation) can be stored on a computer-readable storage medium. The computer-readable storage medium can take the form of a computer-readable program product implemented in one or more computer-readable media and having a computer-readable program code implemented thereon that can be executed by a computer. Considering the inherent ability to store information therein and the inherent ability to provide information retrieval therefrom, the computer-readable storage medium used herein can be considered as a non-transient storage medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or apparatus, or any suitable combination of the foregoing. It should be appreciated that, although more specific examples of computer-readable storage media to which the present embodiment can be applied are provided below, as those of ordinary skill in the art will readily appreciate, it is merely illustrative and not exhaustive: portable computer floppy disk; hard disk; read-only memory (ROM); erasable programmable read-only memory (EPROM or flash memory); portable compact disk read-only memory (CD-ROM); optical storage device; magnetic storage device; or any suitable combination of the foregoing.

[0284] The instructions may form an application program tangibly embodied on a processor-readable medium.

[0285] For example, instructions may be in hardware, firmware, software, or a combination. For example, instructions may be found in an operating system, a separate application, or a combination of both. Thus, a processor may be characterized as, for example, a device configured to perform a process and a device including a processor-readable medium (such as a storage device) having instructions for performing the process. Additionally, in addition to or in lieu of instructions, a processor-readable medium may store data values ​​generated by an embodiment.

[0286] The device may be implemented in, for example, appropriate hardware, software, and firmware. Examples of such devices include personal computers, laptops, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, head-mounted display devices (HMDs, perspective glasses), projectors (projectors), "cave" (systems including multiple displays), servers, video and / or haptic encoders, video and / or haptic decoders, post-processors that process output from a video decoder, pre-processors that provide input to a video encoder, web servers, set-top boxes, wirelessly connected wearable haptic devices (e.g., The device may include a wearable haptic device, a connected wearable haptic device, a game controller, a mouse, a mouse pad, a keyboard, a hand rest, a chair, a desk, an XR headset, a headphone, a wristband, a head and / or lumbar support device or a seat, and any other device for processing haptic data or signals representing one or more haptic feedback or effects, or other communication devices. It should be clear that the equipment can be mobile.

[0287] The computer software may be implemented by the processor 91 or by hardware, or by a combination of hardware and software. As a non-limiting example, embodiments may also be implemented by one or more integrated circuits. The memory 1002 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology (such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples). The processor 1001 may be of any type suitable for the technical environment and may encompass one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture, as non-limiting examples.

[0288] As will be apparent to one of ordinary skill in the art, embodiments may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method or data generated by one of the described embodiments. For example, a signal may be formatted to carry a bit stream of the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0289] The terms used herein are only used to describe the purpose of a particular embodiment and are not intended to be limited. As used herein, the singular forms "one", "a kind of" and "the" may also be intended to include plural forms unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the terms "include / comprise" and / or "including / comprising" may specify the existence of stated, for example, features, integers, steps, operations, elements and / or components, but do not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. Moreover, when an element is referred to as "in response to" or "connected to" another element, it may directly respond to or be connected to another element, or there may be an intermediate element. In contrast, when an element is referred to as "directly responding to" or "directly connected to" another element, there is no intermediate element.

[0290] It should be appreciated that use of any of the symbols / terms " / ", "and / or", and "at least one of" may be intended to encompass selection of only the first listed option (A), or only the second listed option (B), or both options (A and B), such as in the case of "A / B", "A and / or B", and "at least one of A and B". As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C", such wording is intended to encompass selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This may be extended to as many items as listed, as will be apparent to one of ordinary skill in this and related arts.

[0291] Various numerical values ​​may be used in the present application. Specific values ​​may be used for illustrative purposes and the described aspects are not limited to these specific values.

[0292] It will be understood that although the terms first, second, etc. can be used to describe various elements in this article, these elements are not limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the teaching of this application, the first element can be referred to as the second element, and similarly, the second element can be referred to as the first element. There is no suggestion of sorting between the first element and the second element.

[0293] References to "an exemplary embodiment" or "exemplary embodiments" or "one implementation" or "implementation" and other variations thereof are frequently used to convey that a particular feature, structure, characteristic, etc. (described in conjunction with an embodiment / implementation) is included in at least one embodiment / implementation. Thus, the appearances of the phrases "in an exemplary embodiment" or "in an exemplary embodiment" or "in one implementation" or "in an implementation" and any other variations appearing in various places in this application do not necessarily all refer to the same embodiment.

[0294] Similarly, references herein to "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" and other variations thereof are frequently used to convey that a particular feature, structure, or characteristic (described in conjunction with an exemplary embodiment / example / implementation) may be included in at least one exemplary embodiment / example / implementation. Therefore, the expressions "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" appearing in various places in the specification do not necessarily all refer to the same exemplary embodiment / example / implementation, nor are separate or alternative exemplary embodiments / examples / implementations necessarily mutually exclusive of other exemplary embodiments / examples / implementations.

[0295] Reference numerals appearing in the claims are for illustration purposes only and have no limiting effect on the scope of the claims.The present embodiments / examples and variations may be employed in any combination or sub-combination although not explicitly described.

[0296] When a figure is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding device. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0297] While some diagrams include arrows on communication paths to illustrate a primary direction of communication, it should be understood that communication can occur in the opposite direction to the depicted arrows.

[0298] Various embodiments relate to decoding. As used in this application, "decoding" may encompass, for example, all or part of a process performed on a received tactile signal (which may include a received bitstream encoding one or more tactile signals) to produce a final output suitable for presenting a tactile effect or for further processing in a reconstructed tactile feedback or effect. In various embodiments, such a process includes one or more of the processes typically performed by a decoder. In various embodiments, for example, such a process also includes or optionally includes a process performed by a decoder of the various embodiments described in this application.

[0299] Various embodiments relate to encoding. In a manner similar to the above discussion of "decoding", "encoding" as used in this application may encompass, for example, all or part of a process performed on an input tactile signal to produce an encoded bitstream. In various embodiments, such a process includes one or more of the processes typically performed by an encoder. In various embodiments, such a process also includes or optionally includes a process performed by an encoder of the various embodiments described in this application.

[0300] Additionally, this application may refer to "obtaining" various pieces of information. Obtaining information may include, for example, one or more of: estimating the information, calculating the information, or retrieving the information from a memory.

[0301] Further, this application may refer to "accessing" various pieces of information. Obtaining information may include, for example, one or more of: receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, or estimating information.

[0302] Additionally, this application may refer to "receiving" various pieces of information. Like "accessing," receiving is a broad term. Receiving information may include, for example, one or more of: accessing information or retrieving information (e.g., from a memory). Further, during operations such as, for example, storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, or estimating information, "receiving" is generally involved in one way or another.

[0303] Moreover, as used herein, the word "signal" is especially used to indicate something to a corresponding decoder, etc. In this way, in an embodiment, the same parameter can be used on the encoder side and the decoder side. Therefore, for example, the encoder can transmit (explicitly signal) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters and other parameters, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select specific parameters. By avoiding the transmission of any actual function, bit saving is achieved in various embodiments. It should be appreciated that signaling can be completed in a variety of ways. For example, in various embodiments, one or more grammatical elements, signs, etc. are used to signal information to the corresponding decoder. Although the verb form of the word "signal (signal)" is related to the above, the word "signal" can also be used as a noun in this article.

[0304] A number of embodiments have been described. However, it should be understood that various modifications may be made. For example, elements of different embodiments may be combined, supplemented, modified, or removed to produce other embodiments. In addition, it will be understood by those of ordinary skill that other structures and processes may replace the disclosed structures and processes, and that the resulting embodiments will perform at least substantially the same (one or more) functions in at least substantially the same (one or more) manners to achieve at least substantially the same (one or more) results as the disclosed embodiments. Thus, the present application contemplates these and other embodiments.

Claims

1. A method for signaling a mapping between a body part (1) of a body model targeted by a haptic effect and a haptic device comprising a set of haptic actuators (100 to 104, 110 to 114, 120 to 124, 130 to 134) into a container (502), wherein the set of haptic actuators (100 to 104, 110 to 114, 120 to 124, 130 to 134) is configured to present the haptic effect on at least part of the body part (1), the method comprising: - writing (81) first data representing said body part (1) into said container (502); - Writing (82) second data representing the spatial distribution of the tactile actuator group (100 to 104, 110 to 114, 120 to 124, 130 to 134) in the determined multi-dimensional space into the container (502).

2. The method according to claim 1, further comprising: When the tactile actuator group (100 to 104, 110 to 114, 120 to 124, 130 to 134) includes multiple tactile actuators, third data representing the position of a determined tactile actuator in the tactile actuator group in the determined multidimensional space is written into the container (502).

3. The method according to claim 2, further comprising: Fourth data representing a distance relative to the determined tactile actuator is written into the container (502), the distance being defined in the determined multi-dimensional space.

4. The method according to any one of claims 1 to 3, wherein the determined tactile actuator corresponds to a starting point of a tactile animation, and the method further comprises writing fifth data representing another determined tactile actuator corresponding to an end point of the tactile animation into the container (502).

5. The method according to any one of claims 1 to 4, wherein the determined multidimensional space corresponds to: - a Cartesian three-dimensional space, wherein the first data comprises first information representing the number of tactile actuators according to a first dimension of the three-dimensional space, second information representing the number of tactile actuators according to a second dimension of the three-dimensional space, and third information representing the number of tactile actuators according to a third dimension of the three-dimensional space, the maximum number of tactile actuators according to the third information being equal to 2; -Two-dimensional space, wherein the first data includes first information representing the number of angular areas (20 to 27) according to a first dimension of the two-dimensional space and second information representing the number of tactile actuators according to a second dimension of the two-dimensional space according to each of the angular areas (20 to 27).

6. The method according to any one of claims 1 to 5, further comprising writing sixth data representing the haptic effect into the container (502).

7. A method of parsing from a container (502) data representing a mapping between a body part (1) of a body model targeted by a haptic effect and a haptic device comprising a set of haptic actuators (100 to 104, 110 to 114, 120 to 124, 130 to 134), the set of haptic actuators being configured to render the haptic effect on at least part of the body part, the method comprising: - reading (91) first data representing said body part from said container (502); - reading (92) from the container (502) second data representing the spatial distribution of the group of tactile actuators (100 to 104, 110 to 114, 120 to 124, 130 to 134) in the determined multidimensional space.

8. The method according to claim 7, further comprising: When the tactile actuator group (100 to 104, 110 to 114, 120 to 124, 130 to 134) includes multiple tactile actuators, third data representing the position of a determined tactile actuator in the tactile actuator group in the determined multidimensional space is read from the container.

9. The method according to claim 8, further comprising: Fourth data representing a distance relative to the determined tactile actuator is read from the container (502), the distance being defined in the determined multi-dimensional space.

10. The method according to any one of claims 7 to 9, wherein the determined tactile actuator corresponds to a starting point of a tactile animation, and the method further comprises reading from the container fifth data representing another determined tactile actuator corresponding to an ending point of the tactile animation.

11. The method according to any one of claims 7 to 10, further comprising converting the third data and / or the fourth data according to the second data and data representing a spatial distribution of a haptic actuator group of another haptic device.

12. The method according to any one of claims 7 to 11, wherein the determined multidimensional space corresponds to: - a Cartesian three-dimensional space, wherein the first data comprises first information representing the number of tactile actuators according to a first dimension of the three-dimensional space, second information representing the number of tactile actuators according to a second dimension of the three-dimensional space, and third information representing the number of tactile actuators according to a third dimension of the three-dimensional space, the maximum number of tactile actuators according to the third information being equal to 2; -Two-dimensional space, wherein the first data includes first information representing the number of angular areas (20 to 27) according to a first dimension of the two-dimensional space and second information representing the number of tactile actuators according to a second dimension of the two-dimensional space according to each of the angular areas (20 to 27).

13. The method according to any one of claims 7 to 12, further comprising reading sixth data representing the haptic effect from the container (502).

14. A device (10) for signaling a mapping between a body part of a body model targeted by a tactile effect and a tactile device comprising a tactile actuator group to a container (502), wherein the device (10) comprises a memory (1002, 1004) associated with at least a processor (1001), and the processor (1001) is configured to implement the method according to any one of claims 1 to 6.

15. A device (10) for parsing from a container (502) data representing a mapping between a body part of a body model targeted by a tactile effect and a tactile device comprising a tactile actuator group, wherein the device (10) comprises a memory (1002, 1004) associated with at least a processor (1001), the processor (1001) being configured to implement a method according to any one of claims 7 to 13.

Images