Tactile data processing method and related equipment

Abstract

The invention discloses a tactile data processing method and related equipment, and belongs to the technical field of render.The tactile data processing method comprises the steps that tactile data are obtained based on data source position related information in an action object or a trigger object in an interaction behavior; obtaining target tactile data from the tactile data according to trigger information associated with the trigger object; wherein the trigger information comprises at least one of collision information, collision position information and collision mode information.

Description

Technical Field

[0001] This application belongs to the field of rendering technology, and particularly relates to a method for processing tactile data and related devices. Background Art

[0002] The rendering of tactile signals is based on interactivity extension. Interaction is defined based on behavior, and behavior includes a trigger object and an action object. The tactile feedback action (ACTION_SET_HAPTIC) in the action object is used to trigger tactile signal feedback. However, ACTION_SET_HAPTIC only supports directly playing pre-stored tactile signals, which limits the flexibility of tactile feedback. Summary of the Invention

[0003] Embodiments of this application provide a method for determining a rendering object and related devices, which can solve the problem that the tactile feedback action in the existing solution only supports directly playing pre-stored tactile signals, limiting the flexibility of tactile feedback.

[0004] In a first aspect, a method for processing tactile data is provided, which is executed by a rendering end and includes:

[0005] Obtaining tactile data based on the data source location-related information in the action object or the trigger object in the interaction behavior;

[0006] Obtaining target tactile data from the tactile data according to the trigger information associated with the trigger object;

[0007] Wherein, the trigger information includes at least one of whether there is a collision, collision position information, and collision mode information

[0008] In a second aspect, a device for processing tactile data is provided, including:

[0009] A first obtaining module, configured to obtain tactile data based on the data source location-related information in the action object or the trigger object in the interaction behavior;

[0010] A second obtaining module, configured to obtain target tactile data from the tactile data according to the trigger information associated with the trigger object;

[0011] Wherein, the trigger information includes at least one of whether there is a collision, collision position information, and collision mode information

[0012] In a third aspect, an electronic device is provided, which includes a processor and a memory. The memory stores programs or instructions that can run on the processor. When the programs or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0013] In a fourth aspect, an electronic device is provided, including a processor and a communication interface. The processor is configured to obtain haptic data based on information related to the data source location in the action object or trigger object in the interaction behavior.

[0014] Target haptic data is obtained from the haptic data according to the trigger information associated with the trigger object.

[0015] The trigger information includes at least one of whether there is a collision, collision position information, and collision mode information.

[0016] In a fifth aspect, a readable storage medium is provided. Programs or instructions are stored on the readable storage medium. When the programs or instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0017] In a sixth aspect, a chip is provided. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to run programs or instructions to implement the method described in the first aspect.

[0018] In a seventh aspect, a computer program / program product is provided. The computer program / program product is stored in a storage medium. The computer program / program product is executed by at least one processor to implement the steps of the method described in the first aspect.

[0019] In the embodiments of the present application, haptic data is obtained based on information related to the data source location in the action object or trigger object in the interaction behavior; target haptic data is obtained from the haptic data according to the trigger information associated with the trigger object; the trigger information includes at least one of whether there is a collision, collision position information, and collision mode information. Through the above solution, different target haptic data can be selected from the haptic data based on the trigger information related to the trigger object relationship and then rendered, so that different haptic feedback characteristics can be obtained based on different target haptic data, improving the flexibility of haptic feedback. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It shows a schematic diagram of accessing haptic data in the embodiments of the present application;

[0021] Figure 2 It shows a schematic flowchart of the method for processing haptic data in the embodiments of the present application;

[0022] Figure 3Schematic diagram of a module of a tactile data processing device according to an embodiment of the present application;

[0023] Figure 4 Block diagram of the structure of an electronic device according to an embodiment of the present application;

[0024] Figure 5 Block diagram of the structure of a terminal according to an embodiment of the present application. Detailed implementation manners

[0025] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, rather than all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art fall within the protection scope of the present application.

[0026] The terms "first", "second", etc. in the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are usually of the same category, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "or" in the present application means at least one of the connected objects. For example, "A or B" covers three scenarios, namely, Scenario 1: including A and not including B; Scenario 2: including B and not including A; Scenario 3: including both A and B. The character " / " generally indicates an "or" relationship between the associated objects before and after.

[0027] The term "indicate" in the present application can be either a direct indication (or an explicit indication) or an indirect indication (or an implicit indication). Among them, a direct indication can be understood as that the sender clearly informs the receiver of specific information, operations to be performed, or request results, etc. in the sent indication; an indirect indication can be understood as that the receiver determines the corresponding information according to the indication sent by the sender, or makes a judgment and determines the operations to be performed or request results, etc. according to the judgment result.

[0028] To enable those skilled in the art to better understand the embodiments of the present application, the following explanations are made first.

[0029] 1. The synthesis or rendering process of the scene.

[0030] The rendering engine can synthesize a scene based on a glTF (3D model file format) file or render the scene based on the glTF file. The ways for the engine to obtain the glTF file include: the server side and the client side interact with the glTF file through the glTF status stream transmission, or obtain the glTF file through the program files stored on the server side and the client side.

[0031] For haptic scene rendering, the engine can obtain haptic signal feedback, and then play the haptic signal through one or more operations among encoding, transmission, and decoding of the haptic signal by a haptic signal actuator.

[0032] 2. The Moving Pictures Experts Group (MPEG) has made an extension (haptic extension) in haptic signal rendering for glTF (3D model file format);

[0033] MPEG_haptic extension provides a method for accessing haptic data. MPEG_haptic includes a set of haptic objects (hapticObjects), as shown in Table 1. As shown in Table 2, the accessor property in hapticObjects further points to the haptic data source.

[0034] Table 1

[0035]

[0036] Table 2

[0037]

[0038] The specific access method for haptic data is as Figure 1 shown. The hapticObjects in MPEG_haptic point to the accessor, and the accessor further points to a circular buffer object. The circular buffer object provides the data position information for accessing the circular buffer on the one hand, and reflects the property information of the haptic data rendering by referring to the MPEG_media object on the other hand.

[0039] 3. MPEG has also made an extension related to haptic materials for glTF (MPEG_haptic_material extension);

[0040] The MPEG_haptic_material extension provides tactile properties for materials. MPEG_haptic_material contains the materials attribute. Among them, MPEG_haptic_material is shown in Table 3, and MPEG_haptic_material.material is shown in Table 4.

[0041] Table 3

[0042]

[0043] The materials attribute points to a set of material objects, and each material object points to the hapticObject in MPEG_haptic that provides the data source through the haptic attribute, and indicates the relevant material tactile properties (such as stiffness, friction, etc.) and texture coordinate related information.

[0044] Table 4

[0045]

[0046] 4. The Interactivity extension in glTF (3D model file format);

[0047] glTF is used for engines and applications to efficiently transmit and load 3D scenes and models. glTF defines an extensible publishing format, which simplifies the creation workflow and interactive services by enabling the interoperable use of 3D content across the industry.

[0048] The Moving Picture Expert Group (MPEG) has extended glTF interactively and initially designed the relevant standards for interactivity. For tactile scenarios, the relevant standards for interactivity are mainly used to describe the interaction between the avatar and the haptic source, based on which tactile signals are triggered and generated.

[0049] Interactivity is defined based on behavior, which includes trigger nodes (i.e., trigger node, which can also be described as the trigger object) and action nodes (i.e., action node, which can also be described as the action object). Among them, the trigger node includes collision condition, proximity condition, and visibility condition. Among them, the action node includes haptic feedback actions (i.e., ACTION_SET_HAPTIC), activation nodes (ACTION_ACTIVATE), etc. ACTION_SET_HAPTIC is implemented based on the HapticActionNode object or the HapticActionMedia object.

[0050] The hapticObject attribute in HapticActionNode (shown in Table 5) points to the hapticObject in MPEG_haptic, and the mediaIndex attribute in HapticActionMedia (shown in Table 6) further points to the accessor in the above hapticObject to access the haptic data.

[0051] Table 5

[0052]

[0053] Table 6

[0054]

[0055]

[0056] The objects in the embodiments of the present application include at least one of the following: scene, node, mesh, material, vertices, face, primitive, etc.

[0057] Next, in conjunction with the accompanying drawings, through some embodiments and their application scenarios, the method for processing haptic data and related devices provided by the embodiments of the present application will be described in detail.

[0058] As Figure 2 shown, the embodiments of the present application provide a method for processing haptic data, which is executed by a rendering end. The rendering end includes a rendering engine or a display engine. The method includes:

[0059] Step 201: Obtain haptic data based on the information related to the data source location in the action object or the trigger object in the interaction behavior.

[0060] Optionally, the action object and the trigger object are objects in the behavior (i.e., the above-mentioned interaction behavior) defined by the interaction standard, and the interaction standard is a standard for describing the interaction between the avatar and the haptic source.

[0061] Optionally, the action object includes a haptic action media object, the haptic action media object includes a media index, and the media index is associated with the information related to the data source location;

[0062] Alternatively, the trigger object includes a node or a primitive, and the attributes of the node or the primitive are associated with the information related to the data source location.

[0063] Optionally, the information related to the data source location includes information for indicating the location of the haptic data (such as the haptic data source).

[0064] As an implementation, obtaining haptic data based on the information related to the data source location in the action object includes: following the interactivity processing model, the rendering end obtains haptic data according to the haptic feedback action (ACTION_SET_HAPTIC), and at least the mediaIndex included in the HapticActionMedia object can point to the haptic data in MPEG_haptic.

[0065] Optionally, ACTION_SET_HAPTIC can also be associated with a specific haptic processing object, and this object can indicate information such as the haptic data source based on the haptic material.

[0066] As an implementation, obtaining haptic data based on the information related to the data source location in the trigger object includes: based on the property associated with the node / primitive in the trigger object, indicating that the node / primitive is associated with the haptic material property, and this haptic material property can point to the haptic data source corresponding to the haptic material.

[0067] Optionally, the above-mentioned haptic data source includes haptic data, and the haptic data includes texture perception data.

[0068] Step 202: Obtain target haptic data from the haptic data according to the trigger information associated with the trigger object;

[0069] Wherein, the triggering information includes at least one of whether there is a collision, collision position information, and collision mode information.

[0070] Optionally, the above target tactile data is tactile data that needs to be rendered.

[0071] In the embodiments of the present application, based on the information related to the data source position in the action object or the triggering object in the interaction behavior, tactile data is obtained; according to the triggering information associated with the triggering object, target tactile data is obtained from the tactile data; wherein, the triggering information includes at least one of whether there is a collision, collision position information, and collision mode information. Through the above solution, different target tactile data can be selected from the tactile data based on the triggering information of the triggering object relationship and then rendered, so that different tactile feedback characteristics can be obtained based on different target tactile data, improving the flexibility of tactile feedback.

[0072] Optionally, the obtaining of the tactile data includes:

[0073] Obtaining metadata information associated with the tactile data based on the attribute information in the action object or the triggering object;

[0074] According to the metadata information and based on the information related to the data source position in the action object or the triggering object in the interaction behavior, tactile data is obtained from the tactile data source;

[0075] Wherein, the metadata information includes at least one of the storage format information of the tactile data, the resolution information of the tactile data, and the tactile type information associated with the tactile data.

[0076] Here, based on the above metadata information, corresponding tactile data is obtained from the tactile data source, such as obtaining tactile data of a corresponding tactile type or resolution.

[0077] For example, attributes can be added to the object associated with ACTION_SET_HAPTIC to indicate the metadata information associated with the tactile data source, or the metadata can be indicated through the hapticmaterial attribute associated with the node / primitive in the trigger object.

[0078] Optionally, the method in the embodiments of the present application further includes:

[0079] Determining the obtaining method of the tactile data according to the first attribute information associated with the triggering object or the action object;

[0080] Wherein, the first attribute information includes at least one of the following:

[0081] Item 1: Acquisition method identification information;

[0082] Item 2: Tactile feedback type;

[0083] Item 3: Tactile material type;

[0084] Among them, the acquisition method includes at least one of the following:

[0085] Based on the data source location-related information in the action object, obtain the tactile data source;

[0086] Based on the data source location-related information in the trigger object, obtain the tactile data source.

[0087] Exemplarily, the object associated with ACTION_SET_HAPTIC (such as HapticActionNode or HapticActionMedia) can indicate the acquisition method of the tactile data through a property, and this property can indicate whether the tactile data is obtained according to the trigger object.

[0088] For the above Item 1, it is indicated whether the tactile data is obtained based on the data source location-related information in the trigger object through the acquisition method identification information. For example, this acquisition method identification information is represented by a flag. Flag = 0 indicates that the tactile data is obtained according to the data source location-related information in the trigger object, and flag = 1 indicates that the tactile data is obtained according to the data source location-related information in the action object, such as obtained by HapticActionNode or HapticActionMedia.

[0089] Optionally, the above flag can be an existing use Collider flag or a newly added flag.

[0090] For the above Item 2: The acquisition method of the tactile data can be known according to the tactile feedback type indicated by the HapticActionNode. For example, for preset types of tactile feedback such as stiffness / friction / texture perception / vibration perception, the tactile data source is obtained according to the data source location-related information in the trigger object.

[0091] For the above Item 3: For example, if the materialHapticModality of the object (HapticActionNode) related to the feedback action is not empty or is indicated as a preset material type, the tactile data is obtained according to the data source location-related information in the trigger object.

[0092] Optionally, obtaining target haptic data from the haptic data according to the trigger information associated with the trigger object includes:

[0093] Obtaining texture coordinates corresponding to the collision positions between sub-objects in the trigger object;

[0094] Obtaining target haptic data from the haptic data according to the texture coordinates.

[0095] The sub-objects in the above trigger object include mesh vertices.

[0096] In an embodiment of the present application, if a mesh is used to implement the geometry of a haptic source, the mesh can be attached with a haptic material, and the mesh can be directly used in a trigger object for interactivity to determine collision conditions.

[0097] When a trigger object (such as a collision trigger) in the interactivity behavior is activated, the rendering engine determines the mesh vertices corresponding to the touch points of (the haptic source and the avatar), and accordingly obtains the texture coordinates corresponding to the mesh vertices of the touch points. Then, target haptic data corresponding to the texture coordinates is selected from the haptic data, achieving the purpose of extracting discrete haptic data based on the texture coordinates, and further improving the flexibility of haptic feedback.

[0098] Optionally, obtaining target haptic data from the haptic data according to the trigger information associated with the trigger object includes:

[0099] Obtaining texture coordinates associated with the sub-objects that have collided in the trigger object;

[0100] Obtaining target haptic data from the haptic data according to the texture coordinates.

[0101] The above sub-objects that have collided include primitives or nodes.

[0102] In an embodiment of the present application, the geometry of the haptic source is composed of multiple relatively simple primitives, and these primitives are used to determine the collision condition in the trigger object of the interactivity.

[0103] When the trigger object (such as a collision trigger) in the interactivity behavior is activated, the rendering engine obtains the texture coordinates corresponding to the node / primitive in the corresponding trigger object. The rendering engine obtains the target haptic data from the haptic material data source according to one or more texture coordinates and generates a haptic feedback signal, achieving the purpose of extracting discrete haptic data based on the texture coordinates, and thus being able to improve the flexibility of the haptic feedback.

[0104] Optionally, obtaining the texture coordinates associated with the sub-object that has a collision in the trigger object includes:

[0105] Obtaining the texture coordinates according to the attribute information of the sub-object that has a collision.

[0106] As an implementation manner, the attribute information of the sub-object that has a collision includes texture coordinates or vertex information of the mesh, and the vertex information of the mesh corresponds to the texture coordinates.

[0107] For example, the property associated with the node / primitive in the trigger object can indicate a certain texture coordinate of the haptic material corresponding to the node / primitive. When the rendering engine determines that the node / primitive has a collision, the corresponding texture coordinates can be obtained.

[0108] Optionally, the property associated with the node / primitive in the trigger object needs to be defined standardly. For example, in the trigger object, there is a texture coordinate list, and the node list / primitive list in the trigger object corresponds to the texture coordinate list, and each node / primitive corresponds to one of the texture coordinates.

[0109] Optionally, a certain texture coordinate of the haptic material can also be indicated indirectly. For example, the property associated with the node / primitive in the trigger object can indicate a certain vertex of the mesh to which the haptic material is attached, and the vertex of the mesh can be mapped to the texture coordinate of the haptic material.

[0110] Optionally, according to the trigger information associated with the trigger object, obtaining target haptic data from the haptic data includes:

[0111] Determining a rendering method according to the second attribute information associated with the action object;

[0112] In the case where the rendering method indicates obtaining target haptic data from the haptic data according to the texture coordinate, obtaining target haptic data from the haptic data according to the trigger information associated with the trigger object;

[0113] Wherein, the second attribute information includes at least one of the following:

[0114] A1: Rendering method identification information, and this rendering method identification can be an existing useCollider flag or an identification;

[0115] A2: Haptic feedback type; for example, for preset types of haptic feedback such as stiffness / friction / texture / vibrating texture, the rendering end extracts target haptic data from the haptic data according to the texture coordinate for rendering feedback.

[0116] A3: Haptic material type. For example, when the haptic material attribute (materialHapticModality) of the object (HapticActionNode) related to the feedback action is not empty or is indicated as a preset material type, the rendering end extracts target haptic data from the haptic data according to the texture coordinate for rendering feedback

[0117] In the embodiments of the present application, the object associated with ACTION_SET_HAPTIC (such as HapticActionNode or HapticActionMedia) can indicate the rendering method of the haptic data through a property, and this property can indicate whether the rendering end extracts target haptic data from the haptic data according to the texture coordinate for rendering feedback.

[0118] Optionally, the collision method includes at least one of the following: swiping across the surface; tapping; pressing; contact point movement direction, contact point movement speed.

[0119] Optionally, the method of the embodiments of the present application further includes:

[0120] The collision mode is obtained according to a user input modality attribute of a trigger object.

[0121] In the embodiment of the present application, the collision mode is obtained based on the user input mode attribute of the trigger object, and the rendering end can trigger corresponding tactile feedback based on the collision mode.

[0122] Optionally, the acquiring of tactile data based on the information related to the data source location in the action object or the trigger object in the interactive behavior includes:

[0123] Based on the information related to the data source position in the action object or the trigger object in the interactive behavior and the collision mode, the tactile data is obtained.

[0124] In an embodiment of the present application, the rendering end can determine the tactile data source based on the information related to the data source position in the action object or trigger object in the interactive behavior, and then select the corresponding tactile data from the tactile data source according to the collision mode, so that the subsequent rendering end can feedback different tactile signals according to different collision modes.

[0125] Optionally, acquiring target tactile data from the tactile data according to the trigger information associated with the trigger object includes:

[0126] According to the collision mode, obtain the texture coordinate sequence;

[0127] According to the mapping coordinate sequence, target tactile data is acquired from the tactile data.

[0128] Exemplarily, the texture of the tactile material is mapped on the mesh in a preset direction (horizontally / vertically, etc.), the contact point passes across the material surface in different directions (horizontally / vertically, etc.), and the rendering end collects tactile signals according to different texture coordinate sorting (i.e., texture coordinate sequence) to generate feedback tactile signals (i.e., target tactile signals). Among them, the texture coordinate sorting may include multiple groups of different texture coordinates. In addition, in the embodiment of the present application, multiple different texture coordinate sorting methods may be pre-set, and different texture coordinate sorting methods correspond to different collision methods, and then the corresponding texture coordinate sorting method may be obtained according to the collision method.

[0129] Based on the above solution, the rendering end can extract discrete tactile signals (ie, target tactile signals) from the tactile signals according to the contact points and play them, thereby improving the flexibility of tactile feedback.

[0130] The tactile data processing method provided in the embodiment of the present application may be executed by a tactile data processing device. In the embodiment of the present application, the tactile data processing method is executed by a tactile data processing device as an example to illustrate the tactile data processing device provided in the embodiment of the present application.

[0131] As Figure 3 shown, an embodiment of the present application provides a processing device 300 for tactile data, including:

[0132] A first acquisition module 301, configured to acquire tactile data based on information related to the data source position in the action object or the trigger object in the interaction behavior;

[0133] A second acquisition module 302, configured to acquire target tactile data from the tactile data according to the trigger information associated with the trigger object;

[0134] Wherein, the trigger information includes at least one of whether there is a collision, collision position information, and collision mode information.

[0135] Optionally, the first acquisition module includes:

[0136] A first acquisition sub-module, configured to acquire metadata information associated with the tactile data based on the attribute information in the action object or the trigger object;

[0137] A second acquisition sub-module, configured to acquire tactile data from the tactile data source according to the metadata information and based on the information related to the data source position in the action object or the trigger object in the interaction behavior;

[0138] Wherein, the metadata information includes at least one of the storage format information of the tactile data, the resolution information of the tactile data, and the tactile type information associated with the tactile data.

[0139] Optionally, the device further includes:

[0140] A determination module, configured to determine the acquisition method of the tactile data according to the first attribute information associated with the trigger object or the action object;

[0141] Wherein, the first attribute information includes at least one of the following:

[0142] Acquisition method identification information;

[0143] Tactile feedback type;

[0144] Tactile material type;

[0145] Wherein, the acquisition method includes at least one of the following:

[0146] Acquire a tactile data source based on the information related to the data source position in the action object;

[0147] Acquire a tactile data source based on the information related to the data source position in the trigger object.

[0148] Optionally, the second acquisition module includes:

[0149] A third acquisition sub-module, configured to acquire texture coordinates corresponding to the collision positions between sub-objects in the trigger object;

[0150] A fourth acquisition sub-module, configured to acquire target tactile data from the tactile data according to the texture coordinates.

[0151] Optionally, the second acquisition module includes:

[0152] A fifth acquisition sub-module, configured to acquire texture coordinates associated with the sub-objects that collide in the trigger object;

[0153] A sixth acquisition sub-module, configured to acquire target tactile data from the tactile data according to the texture coordinates.

[0154] Optionally, the fifth acquisition sub-module is configured to acquire the texture coordinates according to the attribute information of the sub-objects that collide.

[0155] Optionally, the attribute information of the sub-objects that collide includes texture coordinates or vertex information of a mesh, and the vertex information of the mesh corresponds to the texture coordinates.

[0156] Optionally, the second acquisition module includes:

[0157] A determination sub-module, configured to determine a rendering method according to the second attribute information associated with the action object;

[0158] A seventh acquisition sub-module, configured to acquire target tactile data from the tactile data according to the trigger information associated with the trigger object when the rendering method indicates acquiring target tactile data from the tactile data according to texture coordinates;

[0159] Wherein, the second attribute information includes at least one of the following:

[0160] Rendering method identification information;

[0161] Tactile feedback type;

[0162] Tactile material type.

[0163] Optionally, the collision method includes at least one of the following: swiping across a surface; tapping; pressing; contact point movement direction, contact point movement speed.

[0164] Optionally, the device further includes:

[0165] A third acquisition module, configured to acquire the collision method according to the user input method attribute of the trigger object.

[0166] Optionally, the first acquisition module is configured to acquire tactile data based on the data source location-related information and the collision manner in the action object or the trigger object in the interaction behavior.

[0167] Optionally, the second acquisition module includes:

[0168] An eighth acquisition sub-module, configured to acquire a texture coordinate sequence according to the collision manner;

[0169] A ninth acquisition sub-module, configured to acquire target tactile data from the tactile data according to the texture coordinate sequence.

[0170] The device according to the embodiment of the present application acquires tactile data based on the data source location-related information in the action object or the trigger object in the interaction behavior; acquires target tactile data from the tactile data according to the trigger information associated with the trigger object; wherein, the trigger information includes at least one of whether there is a collision, collision position information, and collision manner information. Through the above solution, different target tactile data can be selected from the tactile data based on the trigger information of the trigger object relationship and then rendered, so that different tactile feedback characteristics can be obtained based on different target tactile data, improving the flexibility of tactile feedback.

[0171] The tactile data processing device provided by the embodiment of the present application can implement Figure 2 each process implemented by the method embodiment and achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0172] Optionally, as Figure 4 shown, the embodiment of the present application further provides an electronic device 400, including a processor 401 and a memory 402. A program or instruction that can run on the processor 401 is stored on the memory 402. For example, when the electronic device 400 is a terminal, when the program or instruction is executed by the processor 401, each step of the above method embodiment for processing tactile data is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be elaborated here.

[0173] The embodiment of the present application further provides an electronic device, including a processor and a communication interface. The communication interface is coupled to the processor, and the processor is configured to run a program or instruction to implement the steps in the method embodiment as Figure 2 shown. This device embodiment corresponds to the above method embodiment. Each implementation process and implementation manner of the above method embodiment can be applied to this electronic device embodiment, and the same technical effect can be achieved.

[0174] The above electronic device can be a terminal or other devices other than terminals, such as a server, a Network Attached Storage (NAS), etc.

[0175] Among them, the terminal can be a mobile phone, a tablet personal computer, a laptop computer, a notebook computer, a personal digital assistant (PDA), a handheld computer, a netbook, an ultra-mobile personal computer (UMPC), a mobile internet device (MID), an augmented reality (AR) device, a virtual reality (VR) device, a mixed reality (MR) device, a robot, a wearable device, a flight vehicle, a vehicle user equipment (VUE), a shipborne device, a pedestrian user equipment (PUE), a smart home (home devices with wireless communication functions, such as refrigerators, TVs, washing machines or furniture, etc.), a game console, a personal computer (PC), an automated teller machine or a self-service machine, etc., which are terminal-side devices. Wearable devices include: smart watches, smart bracelets, smart earphones, smart glasses, smart jewelry (smart bracelets, smart bracelets, smart rings, smart necklaces, smart anklets, smart ankle chains, etc.), smart wristbands, smart clothing, etc. Among them, the vehicle user equipment can also be referred to as a vehicle terminal, a vehicle controller, a vehicle module, a vehicle component, a vehicle chip or a vehicle unit, etc. It should be noted that the specific type of the terminal is not limited in the embodiments of the present application.

[0176] The server can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server. The cloud server can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), or cloud computing services based on big data and artificial intelligence platforms, etc.

[0177] Taking an electronic device as an example of the terminal, Figure 5 It is a schematic diagram of the hardware structure of a terminal for implementing an embodiment of the present application.

[0178] The terminal 500 includes, but is not limited to, at least some components such as a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, and a processor 510.

[0179] Those skilled in the art can understand that the terminal 500 may further include a power supply (such as a battery) for powering each component. The power supply can be logically connected to the processor 510 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. Figure 5 The terminal structure shown does not limit the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0180] It should be understood that in the embodiments of the present application, the input unit 504 may include a graphics processing unit (GPU) 5041 and a microphone 5042. The graphics processor 5041 processes the image data of a static picture or video obtained by an image acquisition device (such as a camera) in a video acquisition mode or an image acquisition mode, or may process the obtained point cloud data. The display unit 506 may include a display panel 5061, and the display panel 5061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 507 includes at least one of a touch panel 5071 and other input devices 5072. The touch panel 5071 is also called a touch screen. The touch panel 5071 may include two parts: a touch detection device and a touch controller. Other input devices 5072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated here.

[0181] In the embodiments of the present application, after the radio frequency unit 501 receives downlink data from a network-side device, it can be transmitted to the processor 510 for processing; in addition, the radio frequency unit 501 can send uplink data to the network-side device. Generally, the radio frequency unit 501 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc.

[0182] The memory 509 can be used to store software programs or instructions and various data. The memory 509 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area may store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 509 may include volatile memory or non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static RAM (SRAM), a dynamic RAM (DRAM), a synchronous DRAM (SDRAM), a double data rate SDRAM (DDR SDRAM), an enhanced SDRAM (ESDRAM), a synch link DRAM (SLDRAM), and a direct rambus RAM (DRRAM). The memory 509 in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.

[0183] The processor 510 may include one or more processing units; optionally, the processor 510 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above modem processor may not be integrated into the processor 510 either.

[0184] Among them, the processor 510 is configured to obtain tactile data based on the information related to the data source location in the action object or trigger object in the interaction behavior;

[0185] Obtain target tactile data from the tactile data according to the trigger information associated with the trigger object;

[0186] Among them, the trigger information includes at least one of whether there is a collision, collision position information, and collision mode information.

[0187] Optionally, the processor 510 is further configured to:

[0188] Obtain metadata information associated with the haptic data based on the attribute information in the action object or the trigger object; according to the metadata information

[0189] and based on the information related to the data source location in the action object or the trigger object in the interaction behavior, obtain haptic data from the haptic data source;

[0190] Wherein, the metadata information includes at least one of the storage format information of the haptic data, the resolution information of the haptic data, and the haptic type information associated with the haptic data.

[0191] Optionally, the processor 510 is further configured to:

[0192] Determine the acquisition method of the haptic data according to the first attribute information associated with the trigger object or the action object;

[0193] Wherein, the first attribute information includes at least one of the following:

[0194] Acquisition method identification information;

[0195] Haptic feedback type;

[0196] Haptic material type;

[0197] Wherein, the acquisition method includes at least one of the following:

[0198] Obtain a haptic data source based on the information related to the data source location in the action object;

[0199] Obtain a haptic data source based on the information related to the data source location in the trigger object.

[0200] Optionally, the processor 510 is further configured to:

[0201] Obtain the texture coordinates corresponding to the collision position between sub-objects in the trigger object;

[0202] Obtain target haptic data from the haptic data according to the texture coordinates.

[0203] Optionally, the processor 510 is further configured to:

[0204] Obtain the texture coordinates associated with the sub-object that has collided in the trigger object;

[0205] Obtain target haptic data from the haptic data according to the texture coordinates.

[0206] Optionally, the processor 510 is further configured to:

[0207] Obtain the texture coordinates according to the attribute information of the child object that has collided.

[0208] Optionally, the attribute information of the child object that has collided includes texture coordinates or vertex information of the mesh, and the vertex information of the mesh corresponds to the texture coordinates.

[0209] Optionally, the processor 510 is further configured to:

[0210] Determine the rendering method according to the second attribute information associated with the action object;

[0211] In the case where the rendering method indicates obtaining target tactile data from the tactile data according to the texture coordinates, obtain the target tactile data from the tactile data according to the trigger information associated with the trigger object;

[0212] Wherein, the second attribute information includes at least one of the following:

[0213] Rendering method identification information;

[0214] Tactile feedback type;

[0215] Tactile material type.

[0216] Optionally, the collision method includes at least one of the following: swiping across the surface; tapping; pressing; contact point movement direction, contact point movement speed.

[0217] Optionally, the processor 510 is further configured to:

[0218] Obtain the collision method according to the user input method attribute of the trigger object.

[0219] Optionally, the processor 510 is further configured to:

[0220] Obtain tactile data based on the information related to the data source location in the action object or the trigger object and the collision method in the interaction behavior.

[0221] Optionally, the processor 510 is further configured to:

[0222] Obtain a sequence of texture coordinates according to the collision method;

[0223] Obtain target tactile data from the tactile data according to the sequence of texture coordinates.

[0224] In an embodiment of the present application, based on the information related to the data source location in the action object or the trigger object in the interaction behavior, haptic data is obtained; according to the trigger information associated with the trigger object, target haptic data is obtained from the haptic data; wherein, the trigger information includes at least one of whether there is a collision, collision position information, and collision mode information. Through the above solution, different target haptic data can be selected from the haptic data based on the trigger information related to the trigger object relationship and then rendered, so that different haptic feedback characteristics can be obtained based on different target haptic data, improving the flexibility of haptic feedback.

[0225] It can be understood that the implementation processes of the various implementation manners mentioned in this embodiment can refer to the relevant descriptions of the method for processing haptic data in the method embodiment, and achieve the same or corresponding technical effects. To avoid repetition, they will not be elaborated here.

[0226] The embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements each process of the above method embodiment for processing haptic data, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0227] Wherein, the processor is the processor in the terminal described in the above embodiment. The readable storage medium includes computer-readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disks, or optical discs, etc. In some examples, the readable storage medium can be a non-transitory readable storage medium.

[0228] The embodiment of the present application further provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run a program or instruction to implement each process of the above method embodiment for processing haptic data, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0229] It should be understood that the chip mentioned in the embodiment of the present application can include a system-on-chip (also referred to as a system chip, chip system, or system-on-chip), or can also include an independent display chip, etc.

[0230] The embodiment of the present application further provides a computer program / program product, which is stored in a storage medium. The computer program / program product is executed by at least one processor to implement each process of the above method embodiment for processing haptic data, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0231] It should be noted that in this text, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, article or device comprising that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0232] From the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of a computer software product plus a necessary general hardware platform, and of course, can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes several instructions for causing a terminal or a network-side device to execute the methods described in various embodiments of the present application.

[0233] The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms of embodiments without departing from the purpose of the present application and the scope protected by the claims, and these embodiments are all within the protection scope of the present application.

Claims

1. A method for processing tactile data, which is executed by a rendering end, characterized in that, it includes: obtaining tactile data based on the information related to the data source position in the action object or the triggering object in the interaction behavior; obtaining target tactile data from the tactile data according to the triggering information associated with the triggering object; wherein, the triggering information includes at least one of whether there is a collision, collision position information, and collision mode information.

2. The method according to claim 1, characterized in that, the obtaining of tactile data includes: obtaining metadata information associated with the tactile data based on the attribute information in the action object or the triggering object; obtaining tactile data from a tactile data source according to the metadata information and based on the information related to the data source position in the action object or the triggering object in the interaction behavior; wherein, the metadata information includes at least one of the storage format information of the tactile data, the resolution information of the tactile data, and the tactile type information associated with the tactile data.

3. The method according to claim 1, characterized in that, it further includes: determining the obtaining method of the tactile data according to the first attribute information associated with the triggering object or the action object; wherein, the first attribute information includes at least one of the following: obtaining method identification information; tactile feedback type; tactile material type; wherein, the obtaining method includes at least one of the following: obtaining a tactile data source based on the information related to the data source position in the action object; obtaining a tactile data source based on the information related to the data source position in the triggering object.

4. The method according to claim 1, characterized in that, obtaining target tactile data from the tactile data according to the triggering information associated with the triggering object includes: obtaining the texture coordinates corresponding to the collision position between sub-objects in the triggering object; obtaining target tactile data from the tactile data according to the texture coordinates.

5. The method according to claim 1, characterized in that, obtaining target tactile data from the tactile data according to the triggering information associated with the triggering object includes: obtaining the texture coordinates associated with the sub-object that has a collision in the triggering object; obtaining target tactile data from the tactile data according to the texture coordinates.

6. The method according to claim 5, characterized in that, the obtaining of the texture coordinates associated with the sub-object that has a collision in the triggering object includes: obtaining the texture coordinates according to the attribute information of the sub-object that has a collision.

7. The method according to claim 6, characterized in that, the attribute information of the sub-object that has a collision includes texture coordinates or vertex information of a mesh, and the vertex information of the mesh corresponds to the texture coordinates.

8. The method according to any one of claims 1 to 7, characterized in that, obtaining target tactile data from the tactile data according to the triggering information associated with the triggering object includes: determining a rendering method according to the second attribute information associated with the action object. When the rendering method indicates obtaining target haptic data from the haptic data according to the texture coordinates, obtain the target haptic data from the haptic data according to the trigger information associated with the trigger object; Wherein, the second attribute information includes at least one of the following: Rendering method identification information; Haptic feedback type; Haptic material type.

9. The method according to claim 1, wherein, The collision method includes at least one of the following: swiping across the surface; tapping; pressing; contact point movement direction, contact point movement speed.

10. The method according to claim 1 or 9, wherein, further includes: Obtain the collision method according to the user input mode attribute of the trigger object.

11. The method according to claim 1, wherein, The obtaining haptic data based on the data source position related information in the action object or the trigger object in the interaction behavior includes: Obtain haptic data based on the data source position related information in the action object or the trigger object in the interaction behavior and the collision method.

12. The method according to claim 1, wherein, The obtaining the target haptic data from the haptic data according to the trigger information associated with the trigger object includes: Obtain a sequence of texture coordinates according to the collision method; Obtain the target haptic data from the haptic data according to the sequence of texture coordinates.

13. A processing device for haptic data, wherein, includes: A first obtaining module, configured to obtain haptic data based on the data source position related information in the action object or the trigger object in the interaction behavior; A second obtaining module, configured to obtain target haptic data from the haptic data according to the trigger information associated with the trigger object; Wherein, the trigger information includes at least one of whether there is a collision, collision position information, and collision method information.

14. The device according to claim 13, wherein, The first obtaining module includes: A first obtaining sub-module, configured to obtain metadata information associated with the haptic data based on the attribute information in the action object or the trigger object; A second obtaining sub-module, configured to obtain haptic data from the haptic data source according to the metadata information and based on the data source position related information in the action object or the trigger object in the interaction behavior; Wherein, the metadata information includes at least one of the storage format information of the haptic data, the resolution information of the haptic data, and the haptic type information associated with the haptic data.

15. The device according to claim 13, wherein, further includes: A determination module, configured to determine the obtaining method of the haptic data according to the first attribute information associated with the trigger object or the action object; Wherein, the first attribute information includes at least one of the following: Obtaining method identification information; Haptic feedback type; Haptic material type; Wherein, the obtaining method includes at least one of the following: Obtain a haptic data source based on the data source position related information in the action object; Obtain a haptic data source based on the data source position related information in the trigger object.

16. The device according to claim 13, wherein, the second acquisition module includes: a third acquisition sub-module, configured to acquire texture coordinates corresponding to a collision position between sub-objects in a trigger object; a fourth acquisition sub-module, configured to acquire target tactile data from the tactile data according to the texture coordinates.

17. The device according to claim 13, wherein, the second acquisition module includes: a fifth acquisition sub-module, configured to acquire texture coordinates associated with a sub-object that has collided in a trigger object; a sixth acquisition sub-module, configured to acquire target tactile data from the tactile data according to the texture coordinates.

18. The device according to claim 17, wherein, the fifth acquisition sub-module is configured to acquire the texture coordinates according to attribute information of the sub-object that has collided.

19. The device according to claim 18, wherein, the attribute information of the sub-object that has collided includes texture coordinates or vertex information of a mesh, and the vertex information of the mesh corresponds to the texture coordinates.

20. The device according to any one of claims 13 to 19, wherein, the second acquisition module includes: a determination sub-module, configured to determine a rendering method according to second attribute information associated with the action object; a seventh acquisition sub-module, configured to acquire target tactile data from the tactile data according to trigger information associated with the trigger object when the rendering method indicates acquiring target tactile data from the tactile data according to texture coordinates; wherein, the second attribute information includes at least one of the following: rendering method identification information; tactile feedback type; tactile material type.

21. The device according to claim 13, wherein, the collision method includes at least one of the following: swiping across a surface; tapping; pressing; contact point movement direction, contact point movement speed.

22. The device according to claim 13 or 21, wherein, further includes: a third acquisition module, configured to acquire the collision method according to a user input method attribute of a trigger object.

23. The device according to claim 13, wherein, the first acquisition module is configured to acquire tactile data based on information related to a data source position in an action object or a trigger object in an interaction behavior and a collision method.

24. The device according to claim 13, wherein, the second acquisition module includes: an eighth acquisition sub-module, configured to acquire a sequence of texture coordinates according to a collision method; a ninth acquisition sub-module, configured to acquire target tactile data from the tactile data according to the sequence of texture coordinates.

25. An electronic device, wherein, includes a processor and a memory, the memory stores a program or instruction that can run on the processor, and when the program or instruction is executed by the processor, the steps of the method for processing tactile data according to any one of claims 1 to 12 are implemented.

26. A readable storage medium, wherein, The program or instructions are stored on the readable storage medium, and when the program or instructions are executed by the processor, the steps of the method for processing tactile data as described in any one of claims 1 to 12 are implemented.

27. A chip, characterized in that the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or instructions to implement the steps of the method for processing tactile data as described in any one of claims 1 to 12.

Images