Representation format of tactile objects

Abstract

A haptic rendering device and a corresponding rendering method enable rendering of haptic effects defined in a haptic signal that includes information representing an immersive scene description. The immersive scene includes information representing at least one element of the scene and information representing haptic objects, including a type of haptic effect, at least one parameter of the haptic effect, and a haptic volume or surface on which the haptic effect is active. The parameter of the haptic effect may be a haptic texture map. A corresponding syntax is proposed.

Description

[Technical Field]

[0001] At least one of the present embodiments relates generally to haptics, and in particular to defining a representation format for haptic objects in an immersive scene based on glTF™ (Graphics Language Transmission Format). [Background technology]

[0002] A fully immersive user experience is offered to the user through an immersive system based on feedback and interaction. The interaction may use traditional control methods to meet the user's needs. Current visual and auditory feedback provides a satisfying and realistic sense of immersion. Additional feedback can be provided by haptic effects that allow human users to perceive the virtual environment with their senses and thus obtain a better experience of full immersion with improved realism. However, haptics is one area where there is still room for advancement to improve the overall user experience in immersive systems.

[0003] Typically, an immersive system can include a 3D scene representing a virtual environment, with virtual objects located within the 3D scene. To enhance user interaction with elements of the virtual environment, haptic feedback through simulation of haptic actuators can be used. Such interaction is based on the concept of "haptic objects" that correspond to physical phenomena communicated to the user. In the context of an immersive scene, haptic objects enable the provision of haptic effects by defining appropriate haptic actuator stimuli to mimic physical phenomena on a haptic rendering device. Different types of haptic actuators enable the restoration of different types of haptic feedback.

[0004] An example of a haptic object is an explosion. An explosion can be rendered through vibration and heat, thus combining different haptic effects for the user to enhance realism. An immersive scene typically includes multiple haptic objects, for example, using a first haptic object associated with a global effect and a second haptic object associated with a local effect.

[0005] The principles described herein apply to any immersive environment that uses haptics, such as augmented reality, virtual reality, mixed reality, or haptic-enhanced video (or omnidirectional / 360° video) rendering, and more generally to any haptic-based user experience. Accordingly, such example scenes of immersive environments are considered immersive scenes.

[0006] Haptic sensation refers to the sense of touch and includes two dimensions: tactile sensation and kinesthetic sensation. The first dimension relates to tactile sensations such as friction, roughness, hardness, and temperature, and is sensed through mechanoreceptors in the skin (Merkel cells, Ruffini endings, Meissner's corpuscles, and Pacinian corpuscles). The second dimension relates to the senses of force / torque, position, and motion / velocity provided by muscles, tendons, and mechanoreceptors in joints. Haptic sensation also contributes to the proprioceptive system (i.e., perception of one's own body) and is therefore involved in the perception of self-motion. Thus, the perception of acceleration, velocity, or any body model can be assimilated as a haptic effect. The frequency range varies depending on the type of modality, but is approximately 0-1 kHz. Most existing devices capable of rendering haptic signals generate vibrations. Examples of such haptic actuators include linear resonant actuators (LRAs), eccentric rotating masses (ERMs), and voice-coil linear motors. These actuators may be integrated into a haptic rendering device such as a haptic suit, but may also be integrated into a smartphone or game controller.

[0007] Several formats have been defined for encoding haptic signals, either relating to high-level descriptions using XML-like formats (e.g., MPEG-V), parametric representations using json-like formats, such as Apple Haptic Audio Pattern (AHAP) or Immersion Corporation's HAPT format, or waveform encoding (ongoing standardization of IEEE 1918.1.1 for haptic and kinesthetic signals). The HAPT format has recently been included in the MPEG ISOBMFF file format specification (ISO / IEC 14496 part 12).

[0008] Additionally, the GL Transmission Format (glTF™) is a royalty-free specification for the efficient transmission and loading of 3D scenes and models by applications. The format defines an extensible, common public format for 3D content tools and services that streamlines authoring workflows and enables interoperable use of content across the industry.

[0009] Additionally, modern 3D engines are capable of mapping textures onto 3D objects. These textures contain information related to various appearance-related parameters such as the object's color, as well as information about its geometry, such as normal or bump maps that aid modern visual rendering algorithms in the rendering process, as well as complex parameters such as diffuse, emissive, and glossiness that also determine how the object is rendered.

[0010] The embodiments described below are designed with the above in mind. Summary of the Invention

[0011] The embodiments relate to a haptic rendering device and a corresponding rendering method that allows rendering haptic effects defined in a haptic signal that includes information representing an immersive scene description. A corresponding syntax is proposed and defined as an extension of the glTF™ format.

[0012] A first aspect of at least one embodiment is directed to a signal for rendering an immersive scene that includes information representing a scene description including at least one information representing at least one element of the scene and information representing a haptic object including a type of haptic effect, at least one parameter of the haptic effect, and a haptic volume or surface on which the haptic effect is active.

[0013] A second aspect of at least one embodiment is directed to an apparatus including: a processor configured to acquire information representing a scene description including at least one information representing at least one element of a scene and information representing a haptic object including a type of haptic effect, at least one parameter of the haptic effect, and a haptic volume or surface on which the haptic effect is active; detect a collision between a position of a user or a body part of a user and the haptic volume; and prepare data generated based on the at least one parameter of the haptic effect for rendering an immersive scene.

[0014] A third aspect of at least one embodiment is directed to a method including: acquiring information representing a scene description including at least one information representing at least one element of a scene; and information representing a haptic object including a type of haptic effect, at least one parameter of the haptic effect, and a haptic volume or surface on which the haptic effect is active; detecting a collision between a position of a user or a body part of a user and the haptic volume; and preparing data generated based on the at least one parameter of the haptic effect for rendering an immersive scene.

[0015] According to a fourth aspect of at least one embodiment, there is provided a computer program comprising program code instructions executable by a processor, the computer program implementing at least the steps of the method according to the third aspect.

[0016] According to a fifth aspect of at least one embodiment, there is provided a computer program product stored on a non-transitory computer readable medium and comprising program code instructions executable by a processor, the computer program product implementing at least the steps of the method according to the third aspect.

[0017] In an alternative embodiment, at least one parameter of the haptic effect is a haptic texture map. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 1 is a block diagram of an example system in which various aspects and embodiments may be implemented. [Figure 2] 1 is an exemplary flowchart of a process for rendering a haptic feedback description file according to at least one embodiment. [Figure 3] 1 illustrates an example data structure for an immersive scene description file containing haptic objects, according to at least one embodiment. [Figure 4] 1 shows an example of a 3D scene with haptic objects. [Figure 5] 5 illustrates an example of a glTF™-based data structure corresponding to the scene of FIG. 4, according to at least one embodiment. [Figure 6A] 1 illustrates an example of a 3D object according to an embodiment that uses a haptic texture map. [Figure 6B] 10 illustrates an example of a temperature haptic effect according to one embodiment using a haptic texture map. [Figure 6C] 10 illustrates an example of a velocity-hardness haptic effect according to one embodiment using a haptic texture map. [Figure 7A]1 illustrates an example of a haptic object including a haptic texture map according to at least one embodiment. [Figure 7B] 1 illustrates an example of a haptic object including a haptic texture map according to at least one embodiment. [Figure 8A] 1 illustrates an example of a scene description representing a haptic object including a haptic texture map, according to at least one embodiment. [Figure 8B] 1 illustrates an example of a scene description representing a haptic object including a haptic texture map, according to at least one embodiment. [Figure 9] Illustrates various haptic effect properties for continuous effects. DETAILED DESCRIPTION OF THE INVENTION

[0019] Haptic objects may relate to the global environment, such as a gentle breeze, or to local effects, such as a punch to the chest. In the first case, the haptic effect is rendered for the fully immersive scene; in the latter case, the haptic effect is activated (and therefore effective) only in a determined subspace of the immersive scene (hereafter referred to as the haptic volume). The haptic volume may be limited to a 2D surface, typically the surface of an object or a simple 2D plane (e.g., the floor). In addition, some haptic rendering devices, such as haptic suits, can provide localized haptic effects (e.g., vibrations in the chest) at precise locations on the user.

[0020] 1 shows a block diagram of an example system in which various aspects and embodiments may be implemented. In the illustrated immersive system, a user, Alice, uses a haptic rendering device 100 to interact with a server 180 that hosts an immersive scene 190 over a communications network 170. This immersive scene 190 may contain various data and / or files representing different elements required for its rendering: a scene description 191, audio data, video data, a 3D model, and haptic objects 192.

[0021] The haptic rendering device comprises a processor 101. The processor 101 may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, application specific integrated circuits (ASICs), field programmable gate array (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, etc. The processor may perform data processing such as haptic signal decoding, input / output processing, and / or any other functionality that enables the device to operate in an immersive system.

[0022] The processor 101 may be coupled to an input unit 102 configured to communicate user interactions. Multiple types of inputs and modalities can be used for that purpose. Physical keypads and touch-sensitive surfaces are typical examples of inputs adapted for this application, although voice control can also be used. In addition, the input unit may include a digital camera capable of capturing still images or video. The processor 101 may be coupled to a display unit 103 configured to output visual data to be displayed on a screen. Multiple types of displays can be used for that purpose, such as a liquid crystal display (LCD) or an organic light-emitting diode (OLED) display unit. The processor 101 may also be coupled to an audio unit 104 configured to render sound data that is converted into audio waves through an adapted transducer, such as a loudspeaker. The processor 101 may be coupled to a communication interface 105 configured to exchange data with external devices. Communication preferably uses wireless communication standards that provide mobility for haptic rendering devices, such as cellular (e.g., LTE) communication, Wi-Fi communication, etc. Processor 101 may access information from and store data in memory 106, which may comprise multiple types of memory, including random access memory (RAM), read-only memory (ROM), a hard disk, a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, or any other type of memory storage device. In one embodiment, processor 101 may access information from and store data in memory that is not physically located on the device, such as a server, a home computer, or another device.

[0023] The processor 101 may be coupled to a haptic unit 107 configured to provide haptic feedback to the user, the haptic feedback being described in a haptic object 192 that is part of a scene description 191 of an immersive scene 190. The haptic feedback 191 describes the type of feedback to be provided according to a syntax described further below. Such a description file is typically communicated to the haptic rendering device 100 from a server 180. The haptic unit 107 may comprise a single haptic actuator or multiple haptic actuators arranged at multiple locations on the haptic rendering device. Different haptic units may have different numbers of actuators and / or the actuators may be arranged differently on the haptic rendering device.

[0024] The processor 101 may receive power from a power source 108 and may be configured to distribute and / or control the power to other components in the device 100. The power source may be any suitable device for powering the device. By way of example, the power source may include one or more dry batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0025] While this diagram depicts the processor 101 and the other elements 102-108 as separate components, it will be understood that these elements may be integrated together in an electronic package or chip. It will be understood that the haptic rendering device 100 may include any subcombination of the elements described herein while remaining consistent with an embodiment. The processor 101 may further be coupled to other peripherals or units not shown in FIG. 1 , which may include one or more software and / or hardware modules that provide additional features, functionality, and / or wired or wireless connectivity. For example, the peripherals may include sensors such as a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, a Bluetooth module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, etc. For example, the processor 101 may be coupled to a localization unit configured to locate the haptic rendering device within its environment. The localization unit may integrate not only a GPS chipset that provides longitude and latitude position relative to the current location of the haptic rendering device, but also other motion sensors such as an accelerometer and / or electronic compass that provide localization services.

[0026] Typical examples of haptic rendering device 100 include a haptic suit, a smartphone, a game controller, a haptic glove, a haptic chair, a haptic prop, a motion platform, etc., although any device or configuration of devices that provides similar functionality may be used as haptic rendering device 100 while conforming to the principles of the present disclosure.

[0027] In at least one embodiment, the device does not include a display unit, but does include a haptic unit. In such an embodiment, the device does not visually render the scene, but only renders haptic effects. However, the device may prepare data for display so that another device, such as a screen, can perform the display. Examples of such devices include a haptic suit or a motion platform.

[0028] In at least one embodiment, the device does not include a haptic unit but does include a display unit. In such an embodiment, the device does not render haptic effects, but only visually renders the scene. However, the device may prepare data for rendering haptic effects so that another device, such as a haptic prop, can perform the haptic rendering. Examples of such devices include a smartphone, a head-mounted display, or a laptop.

[0029] In at least one embodiment, the device does not include a display unit or a haptic unit. In such an embodiment, the device does not visually render a scene or render haptic effects. However, the device may prepare data for display so that another device, such as a screen, can perform the display, or may prepare data for rendering haptic effects so that another device, such as a haptic prop, can perform the haptic rendering. Examples of such devices include a desktop computer, an optical media player, or a set-top box.

[0030] In at least one embodiment, the immersive scene 190 and associated elements are hosted directly in the memory 106 of the haptic rendering device 100, allowing for local rendering and interaction.

[0031] 1 as separate elements, the principles described herein also apply when these elements are integrated directly into the scene description and are not separate elements. Any mix between the two options is also possible, where some elements are integrated into the scene description and others are separate files.

[0032] FIG. 2 shows an exemplary flowchart of a process for rendering a haptic feedback description file according to at least one embodiment. Such a process 200 is typically implemented in a haptic rendering device 100 and executed by a processor 101 of such a device. In step 201, the processor obtains a description of the immersive scene (191 in FIG. 1). This may be done, for example, by receiving it from a server over a communications network, by reading it from an external storage device or local memory, or by any other means. The processor analyzes the scene description file to extract haptic objects (192 in FIG. 1) that allow for determining parameters related to the haptic effect, more specifically, the haptic volume associated with the haptic effect. In step 202, the processor monitors the position of the user (or, for more accurate detection, of the user's body parts) within the immersive scene to detect intersections (object collisions) with the haptic volume during interaction. Collision detection may be performed, for example, by a dedicated physics engine specialized for this task. If such an intersection is detected, in step 203 the processor extracts parameters from the haptic object that allow selecting which haptic signal needs to be applied to which actuator or set of actuators. In step 204 the processor controls the haptic unit to apply the selected haptic signal to the haptic actuator or set of actuators, thus rendering a haptic feedback according to the information of the haptic object.

[0033] Haptic effects are described above as being collision-triggered, but may also be event-triggered, such as the rising sun (which increases the ambient temperature), or an explosion (where vibrations can simulate shock waves), or an incoming communication or other situation relative to the entire immersive scene.

[0034] As mentioned above, some devices do not perform rendering themselves but delegate this task to other devices, in which case data is prepared for rendering of visual elements and / or haptic effects and sent to the device that performs the rendering.

[0035] In a first example, immersive scene 191 may include an outdoor campsite virtual environment in which a user can move an avatar representing themselves. A first haptic feedback may be a breeze generated by a fan present somewhere within the virtual environment. A second haptic feedback may be a temperature of 30°C when the avatar is in close proximity to a campfire. This effect would be rendered by a heating element in a haptic suit worn by a user performing process 200. However, this second feedback is only activated when the user's position is detected to be within the haptic volume of a second haptic object. In this case, the haptic volume represents the distance to the fire at which the user feels the temperature.

[0036] In another example, immersive scene 191 may include a video of a fight between two boxers, and if the user is wearing a haptic suit, the haptic effect may be a strong vibration to the user's chest when one of the wrestlers is hit by a punch.

[0037] FIG. 3 illustrates an example data structure for an immersive scene, according to at least one embodiment. This embodiment is based on the glTF™ file format. The core of glTF™ is a JSON file that describes the structure and organization of a scene containing a 3D model. This diagram shows the relationships between the elements that make up this structure. In this context, scene 300 is the top-level element that collects all other elements. Most notably, it contains an array of nodes. Each node 301 can contain child nodes, creating a hierarchy. Nodes may reference meshes, cameras, or skins, and local geometric transformations may be associated with nodes. Mesh 310 corresponds to the geometry data needed to render the mesh. Vertex skinning is performed using skin 320, so that the vertices of the mesh are influenced by the bones of a skeleton based on its pose. Camera 325 determines the projection matrix. Animation 340 may be applied to node properties. Buffer 355 contains data used for the 3D model's geometry, animation, and skinning. Buffer Views 350 add structural information to the buffer data, and Accessors 345 define the exact type and layout of the buffer view. Materials 360 determine how objects should be rendered based on physical material properties. Textures 365 allow for defining the appearance of objects. Images 370 define the image data used for textures, and Samplers 380 describe the wrapping and scaling of textures. All these elements of a glTF™ file make it possible to define a traditional immersive scene with any haptic feedback.

[0038] Thus, in at least one embodiment, the glTF™ file further includes a haptic object 330 that describes the haptic feedback to be rendered. In an alternative embodiment, the haptic object includes a haptic texture map 335, and data for the haptic texture map may be stored with other textures 365. Such haptic objects are described herein.

[0039] FIG. 4 shows an example of a 3D scene with haptic objects. Each volume is an area where the user can feel the corresponding effect. In one example, a sphere corresponds to a 378 Hz vibration, and a cube corresponds to a 10 Newton pressure effect. This is the kind of information that needs to be stored in the haptic object as part of the immersive scene description. At runtime, the user navigates within the immersive scene through their device 100 of FIG. 1. Depending on the type of immersive application, navigation may correspond to different types of navigation. In a virtual reality example, navigation relates to the movement of an avatar (e.g., a 3D object) that represents the user within the immersive scene and is under the user's control. In an augmented reality application example, navigation relates to the user's physical movement in the real world, which is tracked by a positioning system to determine the user's corresponding position within the immersive scene. In an omnidirectional video example, navigation relates to the user's viewpoint within 360° space.

[0040] During navigation within an immersive scene, a collision with a haptic object may occur when the user's (or their avatar's) position collides with the haptic object's haptic volume, or in other words, when the user's position is within the boundaries of the haptic volume. In such a situation, a corresponding haptic effect is triggered in the haptic rendering device. In the example of Figure 4, when the user collides with the sphere, the haptic rendering device will render vibrations at a frequency of 378 Hz until the collision is resolved.

[0041] However, haptic objects do not necessarily correspond to visible 3D objects, so they can be associated with a volume (a haptic volume) that does not have any visible representation, such that a collision occurs when the user's position is "inside" the volume.

[0042] Thus, in at least one embodiment, the immersive scene includes at least one haptic object characterized by a type of haptic effect, information characterizing the haptic signal to be applied, and information describing a volume within the scene where the haptic effect is active and should be applied by a haptic rendering device. In one alternative embodiment, the information characterizing the haptic signal is a reference to a file containing the haptic signal. In one alternative embodiment, the haptic volume is the entire immersive scene, such that the haptic effect is global and independent of the user's position. In one alternative embodiment, the haptic volume corresponds to the geometric shape of a virtual object with which the haptic object is associated.

[0043] FIG. 5 illustrates an example of a glTF™-based data structure corresponding to the scene of FIG. 4 , according to at least one embodiment. Scene 500 includes a top-level node 501 that includes three child nodes 510, 520, and 530. The first child node 510 corresponds to the sphere object in the lower left of FIG. 4 . Node 510 includes transformation parameters 511 that define the node's translation t1, rotation r1, and scaling s1, as well as a mesh 512 that includes the complete geometry of the sphere object (i.e., the set of vertices and faces). Node 510 also includes a haptic object 513, whose type 514 is determined to be vibration with a frequency 515 of 378 Hz and an intensity 516 of 0.5. The haptic object's shape 517 references the shape of the node and thus will use the geometry defined by mesh 512, which defines the sphere. Thus, the haptic effects of haptic object 513 will be active within the volume of this sphere. The second child node 520 corresponds to the cube object at the top of Figure 4. Node 520 includes transformation parameters 521 that define the node's translation t2, rotation r2, and scaling s2, as well as a mesh 522 that contains the cube object's complete geometry (i.e., set of vertices and faces). Node 520 also includes a haptic object 523, whose type 524 is determined to be pressure with a value 525 of 10. The haptic object's shape 526 references the shape of node 520 and therefore uses the geometry defined by mesh 522, which defines the cube. Thus, the haptic effects of haptic object 523 will be active within the volume of this cube. The third child node 530 corresponds to the cylindrical object at the bottom right of Figure 4. This node does not include any haptic objects and therefore has no associated haptic effects.

[0044] Table 1 shows an example of syntax for defining haptic objects according to at least one embodiment. More specifically, this table shows JSON syntax based on the extension mechanism of glTF™. A haptic object extension designed according to at least one embodiment is identified in this example syntax as "IDCC_Haptics." The list of haptic effects in the table includes, but is not exhaustive, vibration, pressure, and temperature effects. Other types of haptic effects may be defined based on the same principles (wind, rain, snow, electricity, or any combination of effects). Tables 11 and 12 describe syntax for pressure and temperature effects, respectively. The syntax for describing haptic effects is defined in specific JSON schemas (some examples are provided below) and then instantiated in the "properties" of nodes in a scene description file, as described below. A "shapeType" is also associated with haptic objects, allowing for the description of haptic volumes. It can be a primitive volume (sphere or cube) scaled according to the node's scale property (thus allowing for ellipsoidal or parallelepiped volumes), or it can be defined as a custom mesh. In the latter case, the custom mesh is defined by the node's existing mesh property and corresponds to the geometry of the visible object. Defining one of the primitive volumes allows determining the haptic volume without relying on any visible elements.

[0045] [Table 1]

[0046] In at least one embodiment, in addition to the spherical and cubic primitive volumes, additional volumes conventionally used by 3D physics engines may be used, such as a 2D plane, an ellipsoid, a parallelepiped, or a capsule (a capsule is made from two hemispheres joined together by a cylinder). The syntax, not shown in the tables, involves adding the additional primitive volumes to additional enumerated values ​​and additional parameters to define these conventional shapes.

[0047] Usually a single effect is defined, but multiple effects can exist and be combined, for example pressure and temperature can be combined to represent weather conditions (cold rain).

[0048] Table 2 shows an example of a glTF™-based syntax for defining vibrotactile effects, according to at least one embodiment.

[0049] [Table 2]

[0050] Vibration haptic effects can be defined according to parameters including frequency (vibration at a constant frequency of a sinusoidal signal) and intensity (amplitude of vibration), or according to a haptic signal (a waveform similar to an audio signal) if a more complex effect is desired. In the first case, the effect parameters can be defined directly within the vibration syntax and conveyed by the “frequency” and “intensity” syntax elements, as shown in the table. In the second case, the effect is determined according to a signal defined by data embedded in a glTF™ buffer corresponding to an accessor index. Such data is typically loaded from an external file, such as a waveform audio file (.wav format), a haptic file format (OHM format), or any other file format adapted to convey vibration signals. A default value may be determined and should be used if a corresponding syntax element is not present. For example, the default value for the vibration effect defined in Figure 7 is to vibrate at half intensity at a frequency of 250 Hz.

[0051] Table 3 shows an example of a scene definition including a vibrating teapot, according to at least one embodiment. This example shows how to define a simple scene including a 3D object associated with a vibration effect.

[0052] [Table 3-1]

[0053] [Table 3-2]

[0054] In this example, the scene contains a single node named "teapot," which represents a unique 3D object in the scene. The geometry of this node is loaded from the "teapot.bin" file via a set of bufferViews. The material defines how the mesh is represented, and the translation defines the object's position in the virtual environment. A haptic object is also associated with this node. This haptic object corresponds to a vibration effect (the "vibration" syntax element) at a frequency of 250 Hz (the "frequency" syntax element) with an intensity of 70% (the "intensity" syntax element). The haptic volume is defined as the mesh of the node (the "shapeType" syntax element = 2), and therefore the mesh of the teapot. Therefore, if such an object were present in an immersive scene, vibrations would be rendered when the user's position collides with the geometry of the teapot—in other words, when the user "touches" the teapot.

[0055] Table 4 shows an example of a scene definition including haptic objects and associated haptic volumes, according to at least one embodiment.

[0056] [Table 4]

[0057] As mentioned earlier, haptic volumes are not necessarily visible. For brevity and simplicity, this example does not include definitions of other nodes containing other objects not associated with haptic volumes, and only includes two haptic effects with invisible haptic volumes. The scene contains a single node named "haptic_example." This demonstrates the use of the "IDCC_haptics" extension, version (2.0) of the glTF™ specification, and does not use a buffer to load resources. The first haptic effect is a vibrating haptic object configured to vibrate at a frequency of 378 Hz and half intensity (0.5). This effect is not associated with a visible object, but rather with an invisible haptic volume located at position p = (-1.8, 0.7, -0.7), a cube of size 1.2. The second haptic effect is a pressure haptic object configured to exert a force of 10 Newtons. This effect is not associated with a visible object, but rather with an invisible haptic volume, which is a sphere located at position p'=(-2.9,0.0,0.0) and of size 1.0 (the default value, since it is not specified).

[0058] In one embodiment, a haptic object is associated with a mesh-based virtual object, but is comprised of a volume larger than the volume defined by the mesh. For example, a virtual object may correspond to a fireplace represented by a mesh with texture and animation, and the haptic object may include a temperature haptic effect comprised of a spherical haptic volume larger than the bounding box of the fireplace mesh. This configuration allows a user approaching the virtual fireplace to feel the heat before touching (colliding with) the fireplace.

[0059] In one embodiment, a scene includes multiple overlapping haptic objects with concentric volumes that have different haptic effects. For example, a set of haptic objects may use concentric spheres located around a fireplace, with the volumes having sizes that decrease with increasing temperature values. Using this technique, a user would feel a gradual increase in heat as they approached the fireplace. Because the user is about to collide with multiple spheres, the smallest one (i.e., the one closest to the fire) will be selected.

[0060] Table 5 shows an example that enables a gradual effect for a vibrotactile effect, according to at least one embodiment. Indeed, rather than defining multiple overlapping haptic objects, one embodiment proposes interpolation between a minimum and maximum value across the span of the haptic volume. This table shows only the syntax elements that are added to the vibrotactile effect definition according to Table 2. First, an "interpolation" flag syntax element is added to the vibrotactile effect definition. This flag allows for the value of the haptic effect to be interpolated and determines how the interpolation is performed. The interpolation may be any linear, exponential, or nonlinear function. Second, the "min" and "max" syntax elements allow for the range of the interpolation to be defined by defining a scaling factor that is applied to the expected value.

[0061] [Table 5-1]

[0062] [Table 5-2]

[0063] Table 6 shows an example of a vibrotactile effect using interpolation.

[0064] [Table 6]

[0065] In this example, the haptic volume is a cube of size 2.0 placed at the origin of the virtual environment. The haptic effect is linearly interpolated between 0.4 and 1.0, or more precisely, between 0.4 x 1.0 (the first value is the "minimum" scaling value for "interpolation" and the second value is "intensity") and 1.0 x 1.0 (the first value is the default "maximum" scaling value for "interpolation" and the second value is "intensity"). The interpolation is done according to the distance to the center of the haptic volume. Thus, at the origin, the haptic effect intensity is 0.4. At a location equal to (1.0, 1.0, 1.0), which is the center of the cube, the intensity is 0.4. At a location equal to (0.5, 0.5, 0.5), the intensity is 0.7.

[0066] In at least one embodiment, the type of interpolation is defined by a parameter of the haptic object that allows one to choose between at least linear and custom, in which case the function is determined by an additional parameter.

[0067] Table 7 shows an example of a glTF™-based syntax for vibrohaptic effects according to at least one embodiment that uses haptic signals stored in a file.

[0068] [Table 7]

[0069] This embodiment builds on the exemplary syntax shown in Table 2 for a vibrohaptic effect and adds a reference to a file that stores the haptic signal that is applied to render the effect. This allows for defining more sophisticated haptic effects than simply using a fixed-frequency sinusoidal signal. In addition, an intensity parameter can also be applied to the haptic signal. This allows for a unique haptic signal file to be shared and applied at different levels of intensity to different haptic objects. In an example using haptic objects with concentric volumes, the haptic objects can share the same haptic signal file and provide a gradual effect with increasing intensity. The file format is adapted for storing haptic signals. Examples of formats for such use are Waveform audio (WAV), Object Haptic Metadata (OHM), Apple Haptic Audio Pattern (AHAP), or Immersion Corporation's HAPT format. The same principles apply to other types of haptic effects as well.

[0070] Table 8 shows an example of a glTF™-based syntax for vibrohaptic effects that defines the location of the effect, according to at least one embodiment. This embodiment builds on the Object Haptic Metadata (OHM) format and targets applying haptic effects to defined locations on the user's body.

[0071] [Table 8-1]

[0072] [Table 8-2]

[0073] Therefore, in at least one embodiment, it is proposed to add a syntax element to the above-proposed syntax that allows specifying where an effect should be applied. This can be done in two steps: first, by determining a geometric model (i.e., a body model) that represents the spatial acuity of haptic perception; and second, by determining where the haptic effect should be applied on the body model. The geometric model may be selected as a generic model from a set of standard predefined models. In this case, the model is based on the mesh of a human body. The geometric model may also be determined as a custom geometric model by specifying its geometric shape. This allows for adaptation to non-standard haptic rendering devices, such as haptic chairs. In this case, the spatial acuity of haptic perception is limited by the exact location of the actuators on the rendering device. In the proposed syntax, the geometric model is identified by an "avatar_ID." The location where the effect should be applied is selected by using a "body_part_mask" syntax element, corresponding to a binary mask specifying a body part that determines the set of associated vertices, or by using a "vertices" syntax element specifying the vertices to be stimulated.

[0074] Table 9 shows an example of a glTF™-based syntax for defining a geometric model when using vibrohaptic effects that defines the location of the effect, according to at least one embodiment. This syntax defines an identifier for the geometric model, "id," an "lod" value that specifies the level of detail (and therefore resolution) of the geometric model, and the "type" of the haptic effect to be rendered. This syntax therefore allows for specifying the exact location where the haptic effect should be applied.

[0075] [Table 9-1]

[0076] [Table 9-2]

[0077] Table 10 shows an example of a glTF™-based syntax for vibrohaptic effects using channels to carry haptic signals, according to at least one embodiment. This embodiment adds the concept of channels to the syntax proposed above. In fact, a waveform audio or OHM file may contain multiple channels to carry multiple haptic signals associated with multiple haptic objects. In this case, the syntax further includes information indicating the channels to be used.

[0078] [Table 10-1]

[0079] [Table 10-2]

[0080] Table 11 shows an example of glTF™-based syntax for pressure haptic effects, according to at least one embodiment. A pressure haptic effect can be defined simply by a numeric pressure value expressed in Newtons in the "value" syntax element. If the "value" syntax element is not present, the default value "0.0", which corresponds to no pressure, should be used. All of the embodiments described above in connection with vibrohaptic effects apply equally to pressure haptic effects.

[0081] [Table 11]

[0082] Table 12 shows an example of glTF™-based syntax for a temperature haptic effect, according to at least one embodiment. A temperature haptic effect can be defined simply by a numeric temperature value expressed in degrees Celsius in the "value" syntax element. If the "value" syntax element is not present, the default value "20.0", which corresponds to no pressure, should be used. All of the embodiments described above in connection with vibrohaptic effects apply equally to temperature haptic effects.

[0083] [Table 12]

[0084] 6A shows an example of a 3D object according to an embodiment using haptic texture maps. 3D object 1700 represents a metal bottle 1710 with a black soft rubber holder 1720 to insulate a user's hand from the temperature of the bottle. Traditionally, texture files can be used to describe the color, diffuse, emissive, normal, occlusion, roughness, metallic, and specular gloss of an object material, allowing proper (physically based) rendering by a rendering engine based on the texture files.

[0085] In addition to displaying a representation of the 3D bottle, the rendering can benefit from a force feedback device that allows the user to feel the shape of the bottle and its different components from the geometric description.

[0086] According to one embodiment, this rendering is enhanced by the use of haptic texture maps to describe the haptic properties of the object. Haptic texture maps allow for simulating different harshness and temperatures of the bottle in FIG. 6A by defining different parameters for different haptic properties of specific regions of the 3D object. For example, metal bottle 1710 is rendered as a hard, cool metal bottle with a softer, warmer rubber holder 1720 by using additional texture information encoded using a similar principle based on texture maps. The temperature texture map shown in FIG. 6B determines the temperature on the surface of the object (metal parts are cooler than plastic parts), and the temperature texture map shown in FIG. 6C indicates that the metal parts are hard while the rubber holder is soft.

[0087] Using these haptic texture maps, when a user touches an object, the location of the haptic texture is determined, the associated haptic information is retrieved, and the corresponding haptic effect is rendered. This mechanism makes it possible to define 3D objects with complex surfaces that have heterogeneous haptic data for different types of haptic features.

[0088] According to one embodiment, the glTF™-based syntax for defining haptic effects includes a haptic texture map for defining the haptic effect. Different haptic features may be considered and may need to be differentiated. In one embodiment, the haptic features listed in Table 13 are considered.

[0089] [Table 13]

[0090] Dynamic stiffness, stroke spectrum response, and stick-slip do not directly encode tactile values, but use table indices. The ID corresponds to a file where the coefficients of the autoregressive filter are stored. The ID models the vibrations measured in the material during a short contact (dynamic stiffness) or stroke (stroke spectrum response or stick-slip transient), as shown in Table 14.

[0091] [Table 14]

[0092] Table 15 shows an example of syntax for defining haptic texture map properties for a haptic object, according to at least one embodiment. This syntax allows parameters to be determined for different types of haptic effects. - "rate-hardness" allows to determine the surface hardness, in other words it is defined as the initial rate of change of force with respect to the penetration velocity, and is used to simulate both stiffness and damping behavior with better stability. The values ​​are stored in an 8-bit texture and cover values ​​from 0 to 10000 N.s-1 / ms-1, with a resolution of 40 N.s-1 / ms-1. - "contact-area-spread-rate" is defined as the rate at which the contact area spreads across the finger surface as the finger presses against the surface. The value is stored in an 8-bit texture and is 0.1 N.cm 2 Resolution: 0~25.6N.cm 2 Covers values ​​of - "local-surface-orientation" allows to determine the curvature of the shape. The values ​​are stored in a 24-bit texture (3 x 8 bits depending on x, y and z direction) and cover values ​​from 0 to 180 degrees with a resolution of 0.002 degrees. - "local-indentation" allows to determine the surface relief or fine details. The values ​​are stored in an 8-bit texture, with a resolution of 0.04 mm, covering values ​​from -5 mm to +5 mm. - "kinetic-friction" allows to determine the coefficient of kinetic friction, i.e. the force due to friction between each object. The value is stored in an 8-bit texture, with a resolution of 0.04, covering values ​​from -5 to +5. -"static-friction" allows to determine the static friction coefficient, i.e. the force required to make objects slide against each other. The values ​​are stored in an 8-bit texture, with a resolution of 0.04, covering values ​​from -5 to +5. - "temperature" allows to determine the absolute temperature of an object. The values ​​are stored in an 8-bit texture, with a resolution of 0.5°C, covering values ​​from -50°C to +75°C. - "relative-temperature" allows to determine the temperature relative to the user (e.g. 37.5°C). The value is stored in an 8-bit texture and covers values ​​from -25.4°C to +25.4°C with a resolution of 0.2°C. - "dynamic-stiffness" allows to determine the compliance of the object in terms of vibration, i.e. the transient vibration when the user strokes the object. The value is stored in an 8-bit texture, covering values ​​from 0 to 255, and the value is an id in an index table. - "stroke-spectral-response" allows to determine the vibration caused by friction between two objects. The values ​​are stored in an 8-bit texture, covering values ​​from 0 to 255, and the value is an id in an index table. -"stick-slip" allows to determine the vibration phenomenon finally observed at the transition between stiction and sliding. The value is stored in an 8-bit texture, covering values ​​from 0 to 255, and the value is an id in the index table.

[0093] [Table 15-1]

[0094] [Table 15-2]

[0095] [Table 15-3]

[0096] 7A and 7B illustrate an example of a haptic object including a haptic texture map, according to at least one embodiment. The object represents a teapot, and the haptic effect is related to the temperature of the teapot. The geometry of the teapot is defined by a corresponding mesh. A haptic temperature effect is defined by a haptic texture map applied to the geometry of the object, which in this example defines the bottom of the teapot as hot, the lid of the teapot as cold, and the sides of the teapot as varying from hot to cold. In these illustrations, areas with hot temperatures are represented as areas with lighter shades of gray, while darker shades of gray represent areas with cooler temperatures. In other words, lighter is hotter and darker is cooler. However, the values ​​shown in the illustrations do not reflect the temperatures defined in Table 1 and have been arbitrarily selected to obtain an understandable depiction.

[0097] 8A and 8B show an example of a scene description representing a haptic object including a haptic texture map, according to at least one embodiment. The scene corresponds to a teapot with a temperature-related haptic map as described in FIGS. 7A and 7B. The scene description syntax spans FIGS. 8A and 8B. Starting at the end of the glTF™ description file in FIG. 8B, the scene description 2001 includes a single node called teapot. Thus, the set of nodes 2010 is a single child node called teapot. The geometry is defined in 2020 as a first mesh with translations for positioning the object within the scene. The single child node also includes a haptic object 2030 that includes two effects: a vibration effect 2031 and a temperature haptic map 2032. The haptic map 2032 is defined to provide a haptic effect related to "temperature" and uses the texture image at index "0," thus the first texture image, in the list of texture files 2040. The vibration effect is defined directly by its parameters, with a vibration frequency of 250 Hz and an intensity of 0.7 applied to the mesh of the object since the ShapeType parameter is equal to 2. Other sections of the scene description file relate to the mesh defining the geometry of the object 2050, the material 2060 defining the visual appearance of the surface by the defaultMat material in Figure 8A, the buffer view 2070, the buffer 2075 storing the data, the version number 2080, the description of the buffer 2085 and the list of extensions used 2090.

[0098] A first example syntax for conveying haptic objects is described above, and a second example syntax according to at least one embodiment is described below. This second example syntax allows for a more precise and optimized description of signals. For example, a haptic effect can be defined once and then referenced multiple times, optionally with some variation, to generate a haptic signal. It also includes more signal parameters to provide a more complete solution for generating any type of signal.

[0099] Table 16 shows an example of a first level syntax for an extension that describes a global haptic experience, according to at least one embodiment. It provides a description of the haptic object, lists different avatars (i.e., body representations), and defines the required signals. Shape attributes are also added.

[0100] [Table 16-1]

[0101] [Table 16-2]

[0102] [Table 16-3]

[0103] The syntax shown in Table 16 is based on the following elements: -description: A string description of the signal. -avatars: A list of all avatars used in this haptic experience. This refers to the avatar schema described below. -signals: a list of all signals attached to the haptic object. This array references the signal schema described below. -trigger: This keyword can be used to specify the event that will trigger the haptic object. -shape: defines the shape of the haptic object. -accessors: an array of information and references to bufferViews. This refers to the glTF accessor schema defined in the glTF 2.0 official specification. -bufferViews: Part of a buffer. This refers to the glTF bufferView schema defined in the glTF 2.0 official specification. -buffers: References to raw data. This refers to the glTF buffer schema defined in the glTF 2.0 official specification.

[0104] In addition to the syntax described above, haptic signals may be described as shown in the syntax in Table 17. This syntax includes a string description of the signal, some metadata information (e.g., signal type, encoder type, sampling rate, etc.), a reference to an avatar, and the signal's data. If the signal contains PCM data, it can be accessed by referencing a file or an accessor to a buffer. For descriptive content, a list of all required effects is defined at this level. A list of channels finally completes the signal.

[0105] [Table 17-1]

[0106] [Table 17-2]

[0107] The syntax shown in Table 17 is based on the following elements: -description: A string description of the signal. -signal_type: Specifies the type of tactile stimulus (vibration, temperature, force, etc.). -encoder: Specifies the type of encoder used to store the signal. "Raw" means that the signal file is referenced without any type of encoding. "Descriptive" is used when the signal is described using only glTF extensions (for example, it can be translated from an IVS or AHAP file). "PCM_Lossy" and "PCM_Lossless" indicate that the signal is encoded using a dedicated encoder (here, the AAC and ALS codecs, respectively). -sampling_rate: The sampling rate of the signal. -bit_depth: The bit depth of the referenced data. -nb_channels: Number of channels in the signal. -nb_samples_per_channel: Number of samples in each channel. -nb_reduced_samples_per_channel: Number of samples in each channel after downsampling. -avatar_id: The id of the avatar mentioned above, using the avatar schema described below. -signal_file: Path to the file containing the haptic data. This can be any type of file, including wav, ahap, ivs, aac, or other data formats. -signal_accessor: The accessor id of the data in the buffer. - effect_list: A list of all haptic effects used in the signal. This refers to the haptic effect schema described below. -channels: A list of channels for the signal. This refers to the haptic channel schema described below.

[0108] Haptic effects may be described using the syntax shown in Table 18. This syntax defines basic effects that can be referenced in the timeline of a haptic channel. This allows an effect to be described once and then referenced multiple times in different channels. Effects can be described using different properties. We have defined five types of effects: Continuous, Periodic, Transient, PCM, or Timeline. Continuous and Periodic effects can be defined using one or several properties. For example, intensity, attack time, fade time, attack level, and decay level can be used to define a simple effect (similar to an IVS). More advanced effects can be described using envelope properties, which allow a curve to be defined by specifying key points. Transient effects may be defined with just intensity and sharpness values. PCM effects may simply reference raw data stored in a buffer. Properties such as intensity, attack_time, fade_time, and envelope can be used as multipliers for these effects. A timeline effect is simply a timed reference to a previously defined base effect.

[0109] [Table 18-1]

[0110] [Table 18-2]

[0111] [Table 18-3]

[0112] The syntax shown in Table 18 is based on the following elements: -id: The id of the effect. -effect_type: Specifies the type of haptic effect. Effect types include continuous effects (e.g., aperiodic effects used for force feedback), periodic effects (e.g., sine wave effects used for vibration), transient effects (e.g., short, small vibration effects that feel like a tap), PCM (i.e., raw signal data), or timeline effects that reference other existing effects. -PCM_data: Accessor to the effect's raw data. -intensity: Intensity of the effect. If the effect uses PCM data, envelope data, or a timeline, this property can be used as a multiplier. -sharpness: Defines the sharpness of the effect. -duration: The duration of the effect. -attack_time: Duration of the effect's attack phase. -fade_time: Duration of the effect's decay phase. -release_time: The time it takes for the sustained intensity envelope to reach 0 after the event ends. -attack_level:Intensity of the signal at the beginning. -decay_level:Intensity at the end of the signal. -envelope: An array of keyframes that define the envelope of the signal. -wave_frequency: Frequency of the periodic effect. -waveform: Waveform for periodic effects. -timeline: The effect timeline.

[0113] The various haptic effect properties of the continuous effect are shown in Figure 9.

[0114] The haptic channel extension provides specific metadata information for each channel of the signal. As shown in the syntax in Table 19, it includes accessors to a description, gain, blend weighting (for finally merging the signals together), body part masks (which follow the same rules as OHM for positioning effects), and a list of vertices (to provide more precise body localization). For descriptive content, an effect timeline is used to reference and organize, in time, the effects defined at the signal level. Finally, the property timeline can be used as an additional way to adjust intensity and sharpness parameters in time.

[0115] [Table 19-1]

[0116] [Table 19-2]

[0117] The syntax shown in Table 19 is based on the following elements: -id: The id of the channel. -description: Channel description. -gain: The gain applied to all effects on this channel. -mixing_weight: An optional weighting for blending the channels together. -body_part_mask: A body mask that specifies where on the body this channel's effect should be applied. -vertices: The avatar vertices that define where on the body this channel's effect should be applied. -effect_timeline: The channel's effect timeline, which uses the haptic reference schema defined below. - properties_timeline: Properties timeline. The properties defined here are used as multipliers to change the magnitude or sharpness of the channels of the signal over time.

[0118] As shown in the syntax in Table 20, haptic references may be used within a timeline to reference haptic effects defined at the signal level. This requires only the id and start time of the effect. This also provides the possibility to override properties of the referenced effect. This feature allows the same effect to be used multiple times in different channels with slight variations.

[0119] [Table 20-1]

[0120] [Table 20-2]

[0121] [Table 20-3]

[0122] The syntax shown in Table 20 is based on the following elements: -id: The id of the referenced effect -starting_time: The start time of the reference effect on the timeline -effect_type -wave_frequency -waveform -intensity -sharpness -duration -attack_time -fade_time -release_time -attack_level -decay_level

[0123] All of these parameters, except for "id" and "starting time", are optional parameters that can be used to override properties of the referenced effect. They are the same as those defined for the effect schema in Table 18. An example is to reuse a haptic effect, but with a lower intensity. As already mentioned, this allows optimizing the definition of an entire scene by reusing some parameters.

[0124] This extension can be used with the channel timeline property to adjust the intensity or sharpness parameters, as shown in the syntax in Table 21. It is used as a multiplier. The property can be defined using a single value or a curve defined by key points.

[0125] [Table 21]

[0126] The syntax shown in Table 21 is based on the following elements: -property_type: The type of the property. This specifies whether the property is a single value or a curve, and whether it should be applied to intensity or sharpness. -value: The value of the property. -curve: An array of keyframes defined by values ​​and timestamps.

[0127] Haptic avatars are used as body representations for haptic effects. Different types of avatars can be defined, and can reference custom meshes from buffers to determine their specific geometry, as shown in the syntax in Table 22.

[0128] [Table 22-1]

[0129] [Table 22-2]

[0130] The syntax shown in Table 22 is based on the following elements: -id: The id of the avatar. -lod: Avatar level of detail. -Type: Predefined types of avatars, including vibration, pressure and temperature. Other avatars can be described using "custom" types and meshes. -mesh: Avatar mesh.

[0131] The syntax shown in Table 23 defines a first example of a haptic object that uses the signals given in the companion file "vibration.wav".

[0132] [Table 23-1]

[0133] [Table 23-2]

[0134] The syntax shown in Table 24 defines a second example of a haptic object that contains a fully descriptive signal.

[0135] [Table 24-1]

[0136] [Table 24-2]

[0137] [Table 24-3]

[0138] The term "user" is used throughout this specification. This is meant to cover animals as well as human users. An example use case concerns notifying a dog that it has entered a restricted area. To that end, the haptic rendering device may take the form of a vibrating dog collar. When the dog enters the restricted area, a vibration is provided. In this case, the body model uses an appropriate mesh.

[0139] Although various embodiments have been described separately, the embodiments can be combined in any manner while respecting the principles of the present disclosure.

[0140] Although the embodiments relate to haptic effects, those skilled in the art will appreciate that the same principles can be applied to other effects, such as sensory effects, and thus including smells and tastes, and therefore the appropriate syntax will determine the appropriate parameters associated with these effects.

[0141] References to "one embodiment" or "embodiment" or "one implementation" or "implementation," as well as other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with an embodiment is included in at least one embodiment. Thus, appearances of "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation" appearing in various places throughout this application, as well as other variations, are not necessarily all referring to the same embodiment.

[0142] Additionally, the application may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from memory.

[0143] Additionally, the present application or its claims may refer to "obtaining" various information. Obtaining, like "accessing," is intended to be a broad term. Obtaining information may include, for example, one or more of receiving information, accessing information, or retrieving information (e.g., from memory or optical media storage). Furthermore, "obtaining" generally involves in some way, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0144] For example, in the case of "A / B," "A and / or B," and "at least one of A and B," it should be understood that the use of any of the following " / ," "and / or," and "at least one of" is intended to encompass selection of only the first listed alternative (A), or selection of only the second listed alternative (B), or selection of both alternatives (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such phrases are intended to encompass selection of only the first listed alternative (A), or selection of only the second listed alternative (B), or selection of only the third listed alternative (C), or selection of only the first and second listed alternatives (A and B), or selection of only the first and third listed alternatives (A and C), or selection of only the second and third listed alternatives (B and C), or selection of all three alternatives (A and B and C). This can be expanded as many times as the number of items listed, as would be readily apparent to one of ordinary skill in this and related arts.

Claims

1. An immersive scene with haptic effects, Obtaining an immersive scene description based on a data structure consisting of a JSON file defining a scene graph of nodes and referencing binary data for geometric shapes, said immersive scene description including nodes, which themselves: a data structure representing at least one virtual object of the immersive scene; a data structure representing at least one haptic object in the immersive scene, Type of haptic effect, At least one parameter of the haptic effect; and a haptic volume or surface in which the haptic effect will be activated, the haptic volume or surface being determined by the geometric shape of the virtual object contained in the same node; a data structure representing at least one haptic object, the data structure including: obtaining an immersive scene description, Detecting a collision between a position of a representation of a user or a user's body part and said haptic volume or surface; preparing data for rendering the immersive scene, the data being generated based on the at least one virtual object of the haptic effect, and, in response to detecting the collision, a type of the haptic effect and the at least one parameter.

2. The method of claim 1 , wherein the type of haptic effect is selected from the set including vibration, pressure, temperature, and motion.

3. The method of claim 1 or 2, wherein the parameters of the haptic effect describe signals to be applied to a haptic actuator to render the haptic effect.

4. The method of any one of claims 1 to 3, wherein the parameters of the haptic effect include an identification of a file containing the haptic signal to be applied.

5. 5. The method of claim 1, wherein the haptic volume points to the at least one virtual object of the immersive scene and is determined by the surface of the geometric shape of the at least one virtual object of the immersive scene.

6. The method according to any one of claims 1 to 4, wherein the haptic volume is selected from the set comprising a 2D plane, a sphere, an ellipsoid, a cube, a parallelepiped, and a capsule.

7. The method according to any one of claims 1 to 6, wherein the at least one virtual object of the immersive scene is selected from the set comprising: a 3D object, a 2D or 3D video, and an omnidirectional video.

8. The method of any one of claims 1 to 7, wherein at least one of the parameters of the haptic effect is a texture map.

9. 1. A device comprising a processor, the processor comprising: Immersive scenes with haptic effects Obtaining an immersive scene description based on a data structure consisting of a JSON file defining a scene graph of nodes and referencing binary data for geometric shapes, said immersive scene description including nodes, which themselves: a data structure representing at least one virtual object of the immersive scene; a data structure representing at least one haptic object in the immersive scene, Type of haptic effect, At least one parameter of the haptic effect; and a haptic volume or surface in which the haptic effect will be activated, the haptic volume or surface being determined by the geometric shape of the virtual object contained in the same node; a data structure representing at least one haptic object, the data structure including: obtaining an immersive scene description, Detecting a collision between a position of a representation of a user or a user's body part and said haptic volume or surface; preparing data for rendering the immersive scene, the data being generated based on the at least one virtual object and, in response to detecting the collision, a type of the haptic effect and the at least one parameter of the haptic effect.

10. The device of claim 9 , wherein the type of haptic effect is selected from the set including vibration, pressure, temperature, and movement.

11. 11. The device of claim 9 or 10, wherein the parameters of the haptic effect describe signals to be applied to a haptic actuator to render the haptic effect.

12. The device of any one of claims 9 to 11, wherein the parameters of the haptic effect include an identification of a file containing the haptic signal to be applied.

13. 13. The device of claim 9, wherein the haptic volume points to the at least one virtual object of the immersive scene and is determined by the surface of the geometric shape of the at least one virtual object of the immersive scene.

14. The device according to any one of claims 9 to 12, wherein the haptic volume is selected from the set comprising a 2D plane, a sphere, an ellipsoid, a cube, a parallelepiped, and a capsule.

15. The device of any one of claims 9 to 14, wherein the at least one virtual object of the immersive scene is selected from the set comprising: a 3D object, a 2D or 3D video, and an omnidirectional video.

16. The device of any one of claims 9 to 15, wherein at least one of the parameters of the haptic effect is a texture map.

17. 17. The device of claim 9, wherein the device is further configured to render the haptic effect by applying a haptic signal to a haptic actuator according to at least one of the parameters of the haptic effect.

Images