Tactile data processing method and related equipment

Abstract

The invention discloses a tactile data processing method and related equipment, and belongs to the technical field of rendering, and the tactile data processing method comprises the following steps: obtaining tactile feedback data according to at least one of the following items; tactile data and a correction value of the tactile data; the association relationship between the at least one piece of tactile data and the contact force or the contact depth; tactile data and priorities of objects associated with the tactile data; fused data of the at least two types of tactile data; tactile data is acquired by a target object, and the target object is an object with a tactile data acquisition function.

Description

Technical Field

[0001] This application belongs to the field of rendering technology, and particularly relates to a method for processing tactile data and related devices. Background Art

[0002] In the related art, a tactile actuator can simulate tactile signals, such as simulating the texture information of an object's surface. If an object is modeled into a 3D model, the tactile sensation in the virtual world can be transmitted to the real world through the actuator. However, when rendering tactile signals currently, only tactile feedback signals can be obtained based on pre-stored tactile signals, that is, only fixed tactile feedback signals can be obtained based on pre-stored tactile signals. As a result, the tactile actuator can only simulate fixed tactile feedback signals, making it difficult to truly present the tactile sensation of the virtual world. Summary of the Invention

[0003] Embodiments of this application provide a method for processing tactile data and related devices, which can solve the problem that it is difficult to truly present the tactile sensation of the virtual world in the existing solutions.

[0004] In a first aspect, a method for processing tactile data is provided, which is executed by a rendering end. The method includes:

[0005] Obtain tactile feedback data according to at least one of the following:

[0006] Tactile data and the correction value of the tactile data;

[0007] The association relationship between at least one tactile data and the contact force or contact depth;

[0008] Tactile data and the priority of the object associated with the tactile data;

[0009] The fusion data of at least two kinds of tactile data;

[0010] Tactile data collected by a target object, where the target object is an object with a tactile data collection function.

[0011] In a second aspect, a device for processing tactile data is provided, including:

[0012] A first acquisition module, configured to obtain tactile feedback data according to at least one of the following:

[0013] Tactile data and the correction value of the tactile data;

[0014] The association relationship between at least one tactile data and the contact force or contact depth;

[0015] Tactile data and the priority of the object associated with the tactile data;

[0016] The fusion data of at least two kinds of tactile data;

[0017] Tactile data collected by a target object, where the target object is an object with tactile data collection function.

[0018] In a third aspect, an electronic device is provided. The electronic device includes a processor and a memory. The memory stores a program or instruction that can run on the processor. When the program or instruction is executed by the processor, the steps of the method described in the first aspect are implemented.

[0019] In a fourth aspect, an electronic device is provided, including a processor and a communication interface. Wherein, the processor is configured to obtain tactile feedback data according to at least one of the following:

[0020] Tactile data and a correction value of the tactile data;

[0021] The association relationship between at least one tactile data and the contact force or contact depth;

[0022] Tactile data and the priority of the object associated with the tactile data;

[0023] The fusion data of at least two types of tactile data;

[0024] Tactile data collected by a target object, where the target object is an object with tactile data collection function.

[0025] In a fifth aspect, a readable storage medium is provided. The readable storage medium stores a program or instruction. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0026] In a sixth aspect, a chip is provided. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to run a program or instruction to implement the method described in the first aspect.

[0027] In a seventh aspect, a computer program / program product is provided. The computer program / program product is stored in a storage medium. The computer program / program product is executed by at least one processor to implement the steps of the method described in the first aspect.

[0028] In the embodiments of the present application, haptic feedback data is obtained according to at least one of the following: haptic data and a correction value of the haptic data; the association relationship between at least one haptic data and the contact force or contact depth; the priority of the haptic data and the object associated with the haptic data; the fusion data of at least two haptic data; the haptic data collected by a target object, where the target object is an object with a haptic data collection function. Through the above solution, the haptic data can be further processed (enhanced processing) to obtain haptic feedback data, or the haptic feedback data can be obtained based on the collected haptic data, rather than only obtaining the haptic feedback data based on the pre-stored haptic data. The haptic feedback data can reflect flexible haptic characteristics, and thus the haptic experience of the virtual world can be presented more realistically based on the haptic feedback data. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It shows a schematic diagram of accessing haptic data in the embodiments of the present application;

[0030] Figure 2 It shows a schematic flowchart of the method for processing haptic data in the embodiments of the present application;

[0031] Figure 3 It shows a schematic block diagram of the modules of the apparatus for processing haptic data in the embodiments of the present application;

[0032] Figure 4 It shows a schematic block diagram of the structure of an electronic device in the embodiments of the present application;

[0033] Figure 5 It shows a schematic block diagram of the structure of a terminal in the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, rather than all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application belong to the scope of protection of the present application.

[0035] The terms "first", "second", etc. in this application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances so that the embodiments of this application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "or" in this application means at least one of the connected objects. For example, "A or B" covers three scenarios, namely, Scenario 1: including A and not including B; Scenario 2: including B and not including A; Scenario 3: including both A and B. The character " / " generally indicates an "or" relationship between the associated objects before and after.

[0036] The term "indication" in this application can be either a direct indication (or an explicit indication) or an indirect indication (or an implicit indication). Among them, a direct indication can be understood as that the sender clearly informs the receiver of specific information, operations to be performed, request results, etc. in the sent indication; an indirect indication can be understood as that the receiver determines the corresponding information according to the indication sent by the sender, or makes a judgment and determines the operations to be performed or request results, etc. according to the judgment result.

[0037] To enable those skilled in the art to better understand the embodiments of this application, the following explanations are made first.

[0038] 1. The Moving Pictures Experts Group (MPEG) has made an extension (haptic extension) in haptic signal rendering for glTF (3D model file format);

[0039] MPEG_haptic extension provides a method for accessing haptic data. MPEG_haptic includes a set of haptic objects (hapticObjects), as shown in Table 1. As shown in Table 2, the accessor property in hapticObjects further points to the haptic data source.

[0040] Table 1

[0041]

[0042] Table 2

[0043]

[0044] The specific access method of haptic data is as Figure 1As shown, the hapticObjects in MPEG_haptic point to the accessor, which in turn points to the circular buffer object. The circular buffer object provides, on the one hand, the data position information for accessing the circular buffer, and on the other hand, reflects the property information of the haptic data rendering by referring to the MPEG_media object.

[0045] 2. MPEG also made an extension related to haptic materials (MPEG_haptic_material extension) to glTF;

[0046] The MPEG_haptic_material extension provides haptic characteristics for the materials. MPEG_haptic_material contains the materials attributes. Among them, MPEG_haptic_material is shown in Table 3, and MPEG_haptic_material.material is shown in Table 4.

[0047] Table 3

[0048]

[0049] The materials attribute points to a group of material objects. Each material object, in turn, points to the hapticObject in MPEG_haptic that provides the data source through the haptic attribute, and indicates the relevant material haptic characteristics (such as stiffness, friction, etc.) and texture coordinate related information.

[0050] Table 4

[0051]

[0052] 3. The Interactivity extension in glTF (3D model file format);

[0053] glTF is used for engines and applications to efficiently transmit and load 3D scenes and models. glTF defines an extensible publishing format, which simplifies the creation workflow and interactive services by enabling interoperable use of 3D content across the industry.

[0054] The Moving Picture Expert Group (MPEG) has interactively extended glTF and initially designed relevant standards for interactivity. For haptic scenarios, the relevant standards for interactivity are mainly used to describe the interaction between an avatar and a haptic source, based on which haptic signals are triggered and generated.

[0055] Interactivity is defined based on behavior, which includes a trigger condition node (i.e., trigger node) and an action node (i.e., action node). Among them, triggers include collision conditions, proximity conditions, and visibility conditions. Among them, actions include haptic feedback actions (i.e., ACTION_SET_HAPTIC), activate nodes (ACTION_ACTIVATE), etc. ACTION_SET_HAPTIC is implemented based on the HapticActionNode object or the HapticActionMedia object.

[0056] The hapticObject attribute in HapticActionNode (shown in Table 5) points to the hapticObject in MPEG_haptic, and the mediaIndex attribute in HapticActionMedia (shown in Table 6) further points to the accessor in the above hapticObject to access haptic data.

[0057] Table 5

[0058]

[0059] Table 6

[0060]

[0061]

[0062] The objects in the embodiments of this application include at least one of the following: scene, node, mesh, material, vertices, face, primitive, etc.

[0063] The following will, with reference to the accompanying drawings, elaborate on the method for processing tactile data and related devices provided by the embodiments of the present application through some embodiments and their application scenarios.

[0064] As Figure 2 shown, the embodiments of the present application provide a method for processing tactile data, which is executed by a rendering end. The rendering end includes a rendering engine or a display engine. The method includes:

[0065] Step 201: Obtain tactile feedback data according to at least one of the following:

[0066] The first item: Tactile data and a correction value of the tactile data.

[0067] Based on the correction value, the tactile data can be corrected so that the tactile feedback data can more realistically present the tactile sensations of the virtual world. The tactile data can be pre-stored tactile data or tactile data collected based on the following target object.

[0068] The second item: The correlation between at least one tactile data and the contact force or contact depth.

[0069] The contact force or contact depth may refer to the contact force or contact depth between an avatar and a tactile source.

[0070] Based on the above correlation, the obtained tactile feedback data can feedback the contact depth. For example, for the surface of the same object, assuming different forces are used to touch it, then based on the above correlation, tactile feedback data that can feedback different tactile sensations can be obtained.

[0071] The third item: Tactile data and the priority of the object associated with the tactile data.

[0072] Exemplarily, in the case where multiple objects all trigger tactile feedback, the rendering end can obtain the tactile data of the object with the highest priority as the tactile feedback data based on the priority of the object. Thus, the purpose of screening tactile feedback data according to the priority of the object is achieved.

[0073] The fourth item: The fusion data of at least two types of tactile data.

[0074] Optionally, the above at least two types of tactile data correspond to at least two data types. In the embodiments of the present application, different types of tactile data can be fused. For example, the above tactile data includes stiffness data, texture data, etc. Obtaining tactile feedback data based on the fusion data can make the tactile feedback data more realistically present the tactile sensations of the virtual world.

[0075] The fifth item: Tactile data collected by a target object, where the target object is an object with a tactile data collection function.

[0076] In the embodiments of the present application, a target object with a tactile data acquisition function is defined, and real-time acquisition of tactile data is achieved based on the target object. The target object includes at least one of a scene, a node, a mesh, a material, a vertex, a face, a primitive body, etc.

[0077] Optionally, the tactile data in the embodiments of the present application includes texture perception data.

[0078] The method of obtaining tactile feedback data based on at least one of the above can be described as obtaining tactile feedback data in an enhanced manner.

[0079] Whether to obtain tactile feedback data in the above enhanced manner can be indicated by an attribute in the glTF object. Optionally, this attribute identifies whether the haptic texture of a certain material changes according to the contact depth / contact force. For example, when this attribute identifies that the haptic texture of a certain material changes according to the contact depth or contact force, it indicates that the above enhanced manner is used to obtain tactile feedback data.

[0080] In the above solution of the embodiments of the present application, tactile feedback data is obtained according to at least one of the following: tactile data and a correction value of the tactile data; the association relationship between at least one tactile data and the contact force or contact depth; the priority of the tactile data and the object associated with the tactile data; the fusion data of at least two tactile data; the tactile data collected by the target object, where the target object is an object with a tactile data acquisition function. Through the above solution, tactile data can be further processed to obtain tactile feedback data or tactile feedback data can be obtained based on the collected tactile data, rather than only obtaining tactile feedback data based on pre-stored tactile data. This tactile feedback data can reflect flexible tactile characteristics, and thus a more realistic tactile feeling of the virtual world can be presented based on this tactile feedback data.

[0081] Optionally, obtaining tactile feedback data according to the tactile data corresponding to the target object and the correction value of the tactile data includes:

[0082] Obtaining the correction value according to the contact force or contact depth;

[0083] Performing a correction operation on the tactile data according to the correction value to obtain tactile feedback data.

[0084] In the embodiments of the present application, after the rendering end obtains the tactile data, the tactile data is corrected by a correction value. Specifically, the contact force or contact depth is mapped to a corresponding correction value, and the tactile data is corrected based on the correction value, so that the tactile feedback data can more realistically present the tactile sensations of the virtual world.

[0085] As an implementation manner, obtaining the correction value according to the contact force or contact depth includes:

[0086] According to the mapping function, the contact force or contact depth is mapped to a correction value;

[0087] Wherein, the contact force or contact depth is the input of the mapping function, and the correction value is the output of the mapping function.

[0088] Optionally, the relevant information of the mapping function is indicated by the first attribute information, and the first attribute information is carried in the object representing the tactile source or carried in the action object performing the tactile action.

[0089] The relevant information of the above mapping function includes but is not limited to the parameter items of the mapping function, the type of the mapping function, etc.

[0090] Exemplarily, the above object representing the tactile source includes the corresponding node or primitive in the trigger object.

[0091] Exemplarily, the above action object performing the tactile action can be the HapticActionNode object or the HapticActionMedia object in the interactivity model.

[0092] Optionally, the mapping function is a monotonically increasing function (non-monotonically decreasing function). For example, the output value of the mapping function shows a trend of linear increase, quadratic increase or exponential increase as the input depth value increases.

[0093] For example, for a material with the property of a spring, Hooke's law F = kx can be used, where F is the normal force (corresponding to the correction value), k is the stiffness coefficient, and x is the displacement (corresponding to the above contact force or contact depth). Such a material can adopt a linear increase function.

[0094] For another example, for a material with a relatively high rigidity, an exponential increase function can be used and remain constant after exceeding a certain depth value, so as to be able to simulate the properties of a rigid object with a weak buffering effect and a strong supporting effect.

[0095] Based on the above first attribute information, the rendering end can obtain relevant information of the mapping function, so that the contact force or contact depth can be mapped to a correction value based on the mapping function, and then the tactile data can be corrected based on the correction value to obtain tactile feedback data with a depth effect.

[0096] Optionally, correcting the tactile data according to the correction value to obtain tactile feedback data includes:

[0097] Performing weighted summation, weighted multiplication, or weighted addition and multiplication on the tactile data according to the correction value to obtain tactile feedback data.

[0098] Optionally, the relevant parameters of the correction operation are indicated by second attribute information, and the second attribute information is carried in the object representing the tactile source or in the action object performing the tactile action.

[0099] Exemplarily, the above object representing the tactile source includes the corresponding node or primitive in the trigger object.

[0100] Exemplarily, the above action object performing the tactile action can be a tactile action node object (HapticActionNode object) or a tactile action media object (HapticActionMedia object) in the interactivity model.

[0101] Optionally, the relevant parameters of the correction operation include a weighting coefficient.

[0102] As an implementation manner, the formula corresponding to the above weighted summation is shown in formula (1), the formula corresponding to the above weighted multiplication is shown in formula (2), and the formula corresponding to the above weighted addition and multiplication is shown in formula (3). Where v m represents the tactile feedback data, v represents the tactile data, m represents the correction value, c i (i = 1, 2,...) represents the weight parameter, Ω represents the lower bound of the tactile feedback value, O represents the upper bound of the tactile feedback value, and the clamp(a, b, c) function limits the value a within the lower bound b and the upper bound c. It should be understood that in order to ensure that the corrected tactile feedback value maintains the original numerical range, the clamp function is used to limit the correction result.

[0103] v m = clamp(c 1 v + c 2 m, Ω, O) (1)

[0104] v m = clamp(c 3 vm, Ω, O) (2)

[0105] v m = clamp(c 4 v(1 + c 5 m), Ω, O) (3)

[0106] In the embodiments of the present application, it is possible to indicate whether to enable the correction value to correct the tactile data through the glTF object property. When the glTF object property indicates that the correction value is enabled to correct the tactile data, the tactile feedback data is obtained according to the tactile data and the correction value of the tactile data.

[0107] Optionally, obtaining the tactile feedback data includes:

[0108] When there is an association relationship between the tactile data and the contact force or the contact depth, according to the association relationship, the target tactile data corresponding to the target contact force or the target contact depth is obtained;

[0109] According to the target tactile data, the tactile feedback data is obtained.

[0110] In the embodiments of the present application, the glTF object (such as node / mesh / primitive / surface / vertex) may include multiple tactile data sources (such as texture perception data sources), and there is an association relationship between the tactile data source and the contact force or the contact depth. After the rendering end calculates the contact depth (i.e., the target contact depth) or the contact force (i.e., the target contact force), the corresponding target tactile data is obtained based on the above association relationship. For example, the tactile signal node corresponds to data source #1, data source #2,... data source #n. For the contact depth from x1 to x2, the rendering end obtains data source #1; for the contact depth from x3 to x4, the rendering end obtains data source #2.

[0111] Optionally, the association relationship may be presented by the glTF extension. For example, by introducing the glTF extension, the mapping relationship between the contact depth or the contact force and the tactile data source is indicated.

[0112] Optionally, obtaining the tactile feedback data includes:

[0113] Among at least two objects, the tactile data associated with the object with the highest priority is selected as the tactile feedback data, and the at least two objects are objects with an association relationship.

[0114] In an embodiment of the present application, a plurality of interrelated objects (glTF objects) are set. When the plurality of interrelated objects all trigger haptic feedback, the rendering end obtains the haptic data associated with the object with the highest priority as the haptic feedback data.

[0115] The association relationship of the above at least two objects can be presented by the glTF extension. For example, the objects with an association relationship are included in a group of object lists.

[0116] Optionally, the priority of the object is related to at least one of the following:

[0117] Priority identifier; different objects correspond to different priority (priority) identifiers, and different priority identifiers correspond to different priorities;

[0118] The arrangement order of the objects, that is, different arrangement positions correspond to different priorities. For example, the object ranked first has the highest priority;

[0119] The contact depth associated with the object, such as different contact depths corresponding to different priorities;

[0120] The contact force associated with the object, such as different contact forces corresponding to different priorities.

[0121] In an embodiment of the present application, the priority of the object can be presented by the glTF extension.

[0122] Optionally, the method in the embodiment of the present application further includes:

[0123] Obtain at least two types of haptic data to be fused according to protocol agreements or the third attribute information of the object.

[0124] In an embodiment of the present application, after the rendering end obtains a variety of haptic data (such as stiffness data, friction data, texture sensation data, vibration texture sensation data, vibration sensation data), at least two types of haptic data to be fused can be selected based on protocol agreements or the above third attribute information.

[0125] Exemplarily, the above third attribute information can be indicated by the glTF property. This glTF property can be added to the HapticActionNode or HapticActionMedia. When interactivity triggers haptic feedback, the rendering end can determine which haptic data to fuse.

[0126] Optionally, the method in the embodiment of the present application further includes:

[0127] Fuse at least two types of haptic data according to a preset fusion method to obtain the fusion data;

[0128] Among them, the preset fusion method includes at least one of the following: weighted summation, weighted multiplication; weighted addition and multiplication.

[0129] The above fusion method or the weighting coefficient corresponding to the fusion method can be indicated by the glTF object property. The formulas corresponding to the weighted summation, weighted multiplication, and weighted addition and multiplication have been described in the above description and will not be elaborated here.

[0130] Optionally, the fusing at least two types of tactile data according to a preset fusion method to obtain the fused data includes:

[0131] Fusing the at least two types of tactile data according to a preset fusion method to obtain the fused data when the at least two types of tactile data meet a preset condition;

[0132] Among them, the preset condition includes: the at least two types of tactile data act on the same tactile actuator.

[0133] The above tactile actuator is identified by a haptic device, a body part mask, a body part target, an actuator target, a vertex, a surface, a primitive, etc.

[0134] Here, by fusing at least two types of tactile data corresponding to the same tactile actuator, the tactile actuator can simulate a tactile signal based on the fused tactile data, so as to more realistically present the tactile feeling of the virtual world.

[0135] Optionally, the method of the embodiment of the present application further includes:

[0136] Triggering a target object to collect tactile data, where the tactile data at least includes texture perception data.

[0137] As an implementation manner, the above target object may be an action object added in interactivity.

[0138] Optionally, the target object includes at least one of the following (it can be understood that the attribute information of the target object includes at least one of the following):

[0139] A tactile data acquisition function identifier, which is used to indicate that the target object is used for tactile data acquisition;

[0140] A tactile data type identifier;

[0141] Format information of tactile data;

[0142] Resolution information corresponding to the tactile data to be collected. For example, this information includes a high-resolution identifier or a low-resolution identifier;

[0143] Mesh identifier corresponding to the object to be collected or texture coordinates or mapping coordinates of the mesh corresponding to the object to be collected;

[0144] Storage location information of the tactile data. In the embodiments of the present application, the rendering engine triggers a glTF object for processing the tactile data collection by detecting an interactivity trigger, and calls a Medium Access Function (MAF) to store the tactile data.

[0145] In the above solution of the embodiments of the present application, tactile feedback data is obtained according to at least one of the following: tactile data and a correction value of the tactile data; the association relationship between at least one tactile data and the contact force or contact depth; the priority of the tactile data and the object associated with the tactile data; the fusion data of at least two tactile data; tactile data collected by a target object, where the target object is an object with a tactile data collection function. Through the above solution, it is possible to further process the tactile data to obtain tactile feedback data or to obtain tactile feedback data based on the collected tactile data, rather than only obtaining tactile feedback data based on pre-stored tactile data. The tactile feedback data can reflect flexible tactile characteristics, and thus, based on the tactile feedback data, the tactile experience of the virtual world can be presented more realistically.

[0146] In the method for processing tactile data provided by the embodiments of the present application, the execution subject may be a device for processing tactile data. In the embodiments of the present application, taking the device for processing tactile data as an example to execute the method for processing tactile data, the device for processing tactile data provided by the embodiments of the present application is described.

[0147] As Figure 3 shown, the embodiments of the present application provide a device 300 for processing tactile data, including:

[0148] A first acquisition module 301, configured to obtain tactile feedback data according to at least one of the following:

[0149] Tactile data and a correction value of the tactile data;

[0150] The association relationship between at least one piece of tactile data and the contact force or contact depth;

[0151] The priority of the tactile data and the object associated with the tactile data;

[0152] The fusion data of at least two pieces of tactile data;

[0153] The tactile data collected by the target object, where the target object is an object with the function of collecting tactile data.

[0154] Optionally, the first acquisition module includes:

[0155] The first acquisition sub-module is used to obtain the correction value according to the contact force or contact depth;

[0156] The second acquisition sub-module is used to perform a correction operation on the tactile data according to the correction value to obtain tactile feedback data.

[0157] Optionally, the first acquisition sub-module is used to map the contact force or contact depth to a correction value according to a mapping function;

[0158] Wherein, the contact force or contact depth is the input of the mapping function, and the correction value is the output of the mapping function.

[0159] Optionally, the relevant information of the mapping function is indicated by first attribute information, and the first attribute information is carried in the object representing the tactile source or in the action object performing the tactile action.

[0160] Optionally, the second acquisition sub-module is used to perform weighted summation, weighted multiplication or weighted addition and multiplication processing on the tactile data according to the correction value to obtain tactile feedback data.

[0161] Optionally, the relevant parameters of the correction operation are indicated by second attribute information, and the second attribute information is carried in the object representing the tactile source or in the action object performing the tactile action.

[0162] Optionally, the first acquisition module includes:

[0163] The third acquisition sub-module is used to obtain the target tactile data corresponding to the target contact force or target contact depth according to the association relationship when there is an association relationship between the tactile data and the contact force or contact depth;

[0164] The fourth acquisition sub-module is used to obtain the tactile feedback data according to the target tactile data.

[0165] Optionally, the first acquisition module is configured to select, from at least two objects, the haptic data associated with the object having the highest priority as the haptic feedback data, where the at least two objects are objects having an association relationship.

[0166] Optionally, the priority of the object is related to at least one of the following:

[0167] Priority identifier;

[0168] Arrangement order of the objects;

[0169] Contact depth associated with the object;

[0170] Contact force associated with the object.

[0171] Optionally, the device according to an embodiment of the present application further includes:

[0172] A second acquisition module, configured to acquire at least two types of haptic data for fusion according to protocol agreements or third attribute information of the object.

[0173] Optionally, the device according to an embodiment of the present application further includes:

[0174] A fusion module, configured to fuse at least two types of haptic data according to a preset fusion method to obtain the fusion data;

[0175] Wherein, the preset fusion method includes at least one of the following: weighted summation, weighted product; weighted addition and multiplication.

[0176] Optionally, the fusion module is configured to fuse the at least two types of haptic data according to a preset fusion method to obtain the fusion data when the at least two types of haptic data meet a preset condition;

[0177] Wherein, the preset condition includes: the at least two types of haptic data act on the same haptic actuator.

[0178] Optionally, the device according to an embodiment of the present application further includes:

[0179] A trigger module, configured to trigger a target object to collect haptic data.

[0180] Optionally, the target object includes at least one of the following:

[0181] Haptic data acquisition function identifier;

[0182] Haptic data type identifier;

[0183] Format information of the haptic data;

[0184] Resolution information corresponding to the haptic data to be collected;

[0185] The mesh identifier corresponding to the object to be collected or the texture coordinate information of the mesh corresponding to the object to be collected;

[0186] The storage location information of the tactile data.

[0187] The device according to the embodiments of the present application obtains tactile feedback data according to at least one of the following: tactile data and the correction value of the tactile data; the association relationship between at least one tactile data and the contact force or contact depth: the priority of the tactile data and the object associated with the tactile data: the fusion data of at least two tactile data: the tactile data collected by the target object, where the target object is an object with a tactile data collection function. Through the above solution, the tactile data can be further processed (enhanced processing) to obtain tactile feedback data or the tactile feedback data can be obtained based on the collected tactile data, rather than only obtaining the tactile feedback data based on the pre-stored tactile data. The tactile feedback data can reflect flexible tactile characteristics, and then based on the tactile feedback data, the tactile feeling of the virtual world can be presented more realistically.

[0188] The processing device for tactile data provided by the embodiments of the present application can implement Figure 2 each process implemented by the method embodiments and achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0189] Optionally, as Figure 4 shown, the embodiments of the present application further provide an electronic device 400, including a processor 401 and a memory 402. A program or instruction that can run on the processor 401 is stored on the memory 402. For example, when the electronic device 400 is a terminal, when the program or instruction is executed by the processor 401, each step of the above method embodiment for processing tactile data is implemented, and the same technical effects can be achieved. To avoid repetition, it will not be elaborated here.

[0190] The embodiments of the present application further provide an electronic device, including a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the steps in the method embodiment as Figure 2 shown. This device embodiment corresponds to the above method embodiment. Each implementation process and implementation manner of the above method embodiment can be applied to this electronic device embodiment, and the same technical effects can be achieved.

[0191] The above electronic device can be a terminal or other devices other than the terminal, such as a server, a Network Attached Storage (NAS), etc.

[0192] Among them, the terminal can be a mobile phone, a tablet personal computer, a laptop computer, a notebook computer, a personal digital assistant (PDA), a handheld computer, a netbook, an ultra-mobile personal computer (UMPC), a mobile internet device (MID), an augmented reality (AR) device, a virtual reality (VR) device, a mixed reality (MR) device, a robot, a wearable device, a flight vehicle, a vehicle user equipment (VUE), a shipborne device, a pedestrian user equipment (PUE), a smart home (home appliances with wireless communication functions, such as refrigerators, TVs, washing machines, or furniture, etc.), a game console, a personal computer (PC), a teller machine, or a self-service machine, etc., which are terminal-side devices. Wearable devices include: smart watches, smart bracelets, smart earphones, smart glasses, smart jewelry (smart bracelets, smart bracelets, smart rings, smart necklaces, smart anklets, smart ankle chains, etc.), smart wristbands, smart clothing, etc. Among them, the vehicle user equipment can also be referred to as a vehicle terminal, a vehicle controller, a vehicle module, a vehicle component, a vehicle chip, or a vehicle unit, etc. It should be noted that the specific type of the terminal is not limited in the embodiments of the present application.

[0193] The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server. The cloud server can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery network (CDN), or cloud computing services based on big data and artificial intelligence platforms, etc.

[0194] Taking an electronic device as an example of the terminal, Figure 5 It is a schematic diagram of the hardware structure of a terminal for implementing an embodiment of the present application.

[0195] The terminal 500 includes, but is not limited to, at least some components such as a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, and a processor 510.

[0196] Those skilled in the art can understand that the terminal 500 may further include a power source (such as a battery) for supplying power to each component. The power source may be logically connected to the processor 510 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. Figure 5 The terminal structure shown does not limit the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0197] It should be understood that in the embodiments of the present application, the input unit 504 may include a graphics processing unit (GPU) 5041 and a microphone 5042. The graphics processor 5041 processes the image data of a static picture or video obtained by an image acquisition device (such as a camera) in a video acquisition mode or an image acquisition mode, or may process the obtained point cloud data. The display unit 506 may include a display panel 5061, and the display panel 5061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 507 includes at least one of a touch panel 5071 and other input devices 5072. The touch panel 5071 is also called a touch screen. The touch panel 5071 may include two parts: a touch detection device and a touch controller. The other input devices 5072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated here.

[0198] In the embodiments of the present application, after the radio frequency unit 501 receives downlink data from a network side device, it may be transmitted to the processor 510 for processing; in addition, the radio frequency unit 501 may send uplink data to the network side device. Generally, the radio frequency unit 501 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc.

[0199] The memory 509 can be used to store software programs or instructions and various data. The memory 509 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area may store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 509 may include volatile memory or non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM), and a direct rambus random access memory (DRRAM). The memory 509 in the embodiments of the present application includes, but is not limited to, these and any other suitable types of memory.

[0200] The processor 510 may include one or more processing units; optionally, the processor 510 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above modem processor may not be integrated into the processor 510 either.

[0201] Among them, the processor 510 is used to obtain tactile feedback data according to at least one of the following:

[0202] Tactile data and a correction value of the tactile data;

[0203] The association relationship between at least one tactile data and the contact force or contact depth;

[0204] Tactile data and the priority of the object associated with the tactile data;

[0205] Fusion data of at least two pieces of tactile data;

[0206] Tactile data collected by a target object, where the target object is an object with a tactile data collection function.

[0207] Optionally, the processor 510 is further configured to:

[0208] Obtain the correction value according to the contact force or contact depth;

[0209] Perform a correction operation on the tactile data according to the correction value to obtain tactile feedback data.

[0210] Optionally, the processor 510 is further configured to:

[0211] Map the contact force or contact depth to a correction value according to a mapping function;

[0212] Wherein, the contact force or contact depth is the input of the mapping function, and the correction value is the output of the mapping function.

[0213] Optionally, the relevant information of the mapping function is indicated by first attribute information, and the first attribute information is carried in the object representing the tactile source or in the action object performing the tactile action.

[0214] Optionally, the processor 510 is further configured to:

[0215] Perform weighted summation, weighted multiplication, or weighted addition and multiplication processing on the tactile data according to the correction value to obtain tactile feedback data.

[0216] Optionally, the relevant parameters of the correction operation are indicated by second attribute information, and the second attribute information is carried in the object representing the tactile source or in the action object performing the tactile action.

[0217] Optionally, the processor 510 is further configured to:

[0218] In the case where there is an association relationship between the tactile data and the contact force or contact depth, obtain target tactile data corresponding to a target contact force or target contact depth according to the association relationship;

[0219] Obtain the tactile feedback data according to the target tactile data.

[0220] Optionally, the processor 510 is further configured to:

[0221] Select the tactile data associated with the object with the highest priority among at least two objects as the tactile feedback data, where the at least two objects are objects with an association relationship.

[0222] Optionally, the priority of the object is related to at least one of the following:

[0223] Priority identifier;

[0224] The arrangement order of the objects;

[0225] The contact depth associated with the object;

[0226] The contact force associated with the object.

[0227] Optionally, the processor 510 is further configured to:

[0228] Obtain at least two types of haptic data for fusion according to the protocol agreement or the third attribute information of the object.

[0229] Optionally, the processor 510 is further configured to:

[0230] Fuse at least two types of haptic data in a preset fusion manner to obtain the fusion data;

[0231] Wherein, the preset fusion manner includes at least one of the following: weighted summation, weighted product; weighted addition and multiplication.

[0232] Optionally, the processor 510 is further configured to:

[0233] Fuse at least two types of haptic data in a preset fusion manner to obtain the fusion data when the at least two types of haptic data meet the preset conditions;

[0234] Wherein, the preset conditions include: the at least two types of haptic data act on the same haptic actuator.

[0235] Optionally, the processor 510 is further configured to:

[0236] Trigger the target object to collect haptic data.

[0237] Optionally, the target object includes at least one of the following:

[0238] Haptic data collection function identifier;

[0239] Haptic data type identifier;

[0240] Format information of the haptic data;

[0241] Resolution information corresponding to the haptic data to be collected;

[0242] Mesh identifier corresponding to the object to be collected or texture coordinate information of the mesh corresponding to the object to be collected;

[0243] Storage location information of the haptic data.

[0244] In the embodiments of the present application, tactile feedback data is obtained according to at least one of the following: tactile data and a correction value of the tactile data; the association relationship between at least one piece of tactile data and the contact force or contact depth; the priority of the tactile data and the object associated with the tactile data; the fusion data of at least two pieces of tactile data; the tactile data collected by a target object, where the target object is an object with a tactile data collection function. Through the above solution, the tactile data can be further processed (enhanced processing) to obtain tactile feedback data, or the tactile feedback data can be obtained based on the collected tactile data, rather than only obtaining the tactile feedback data based on the pre-stored tactile data. The tactile feedback data can reflect flexible tactile characteristics, and based on this tactile feedback data, the tactile feeling of the virtual world can be presented more realistically.

[0245] It can be understood that the implementation processes of the implementation manners mentioned in this embodiment can refer to the relevant descriptions of the method for processing tactile data in the method embodiment, and achieve the same or corresponding technical effects. To avoid repetition, they will not be elaborated here.

[0246] The embodiments of the present application further provide a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements each process of the above-mentioned method embodiment for processing tactile data, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0247] Among them, the processor is the processor in the terminal described in the above embodiment. The readable storage medium includes computer-readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disks, or optical disks, etc. In some examples, the readable storage medium may be a non-transitory readable storage medium.

[0248] The embodiments of the present application further provide a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement each process of the above-mentioned method embodiment for processing tactile data, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0249] It should be understood that the chip mentioned in the embodiments of the present application may include a system-on-chip (also referred to as a system chip, chip system, or system-on-chip), or may include an independent display chip, etc.

[0250] The embodiments of the present application further provide a computer program / program product, which is stored in a storage medium. The computer program / program product is executed by at least one processor to implement each process of the above-mentioned method embodiment for processing tactile data, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0251] It should be noted that, in this text, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article or device comprising such element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, but may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0252] From the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of a computer software product plus a necessary general hardware platform, and of course, can also be implemented by hardware. This computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disc, etc.) and includes several instructions for causing a terminal or a network-side device to execute the methods described in various embodiments of the present application.

[0253] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms of embodiments without departing from the purpose of the present application and the scope protected by the claims. These embodiments are all within the protection scope of the present application.

Claims

1. A method for processing tactile data, which is executed by a rendering end, characterized in that, it includes: obtaining tactile feedback data according to at least one of the following: tactile data and a correction value of the tactile data; the correlation relationship between at least one tactile data and the contact force or contact depth; tactile data and the priority of the object associated with the tactile data; fusion data of at least two types of tactile data; tactile data collected by a target object, where the target object is an object with a tactile data collection function.

2. The method according to claim 1, characterized in that, obtaining tactile feedback data according to the tactile data corresponding to the target object and the correction value of the tactile data, includes: obtaining the correction value according to the contact force or contact depth; performing a correction operation on the tactile data according to the correction value to obtain tactile feedback data.

3. The method according to claim 2, characterized in that, the obtaining the correction value according to the contact force or contact depth includes: mapping the contact force or contact depth to a correction value according to a mapping function; wherein, the contact force or contact depth is the input of the mapping function, and the correction value is the output of the mapping function.

4. The method according to claim 3, characterized in that, the relevant information of the mapping function is indicated by first attribute information, and the first attribute information is carried in the object representing the tactile source or carried in the action object performing the tactile action.

5. The method according to any one of claims 2 to 4, characterized in that, performing a correction operation on the tactile data according to the correction value to obtain tactile feedback data, includes: performing weighted summation, weighted multiplication or weighted addition and multiplication processing on the tactile data according to the correction value to obtain tactile feedback data.

6. The method according to any one of claims 2 to 5, characterized in that, the relevant parameters of the correction operation are indicated by second attribute information, and the second attribute information is carried in the object representing the tactile source or carried in the action object performing the tactile action.

7. The method according to claim 1, characterized in that, obtaining tactile feedback data, includes: in the case where there is a correlation relationship between the tactile data and the contact force or contact depth, obtaining target tactile data corresponding to the target contact force or target contact depth according to the correlation relationship; obtaining the tactile feedback data according to the target tactile data.

8. The method according to claim 1, characterized in that, obtaining tactile feedback data, includes: selecting the tactile data associated with the object with the highest priority among at least two objects as the tactile feedback data, where the at least two objects are objects with a correlation relationship.

9. The method according to claim 8, characterized in that, the priority of the object is related to at least one of the following: priority identifier; the arrangement order of the objects; the contact depth associated with the object; the contact force associated with the object.

10. The method according to claim 1, characterized in that, it further includes: obtaining at least two types of tactile data for fusion according to protocol agreement or the third attribute information of the object.

11. The method according to claim 1, It is characterized in that It further includes: Fusing at least two types of tactile data according to a preset fusion method to obtain the fused data; Wherein, the preset fusion method includes at least one of the following: weighted summation, weighted multiplication; weighted addition and multiplication.

12. The method according to claim 11, It is characterized in that The fusing of at least two types of tactile data corresponding to the target object according to a preset fusion method to obtain the fused data includes: When the at least two types of tactile data meet a preset condition, fusing the at least two types of tactile data according to a preset fusion method to obtain the fused data; Wherein, the preset condition includes: the at least two types of tactile data act on the same tactile actuator.

13. The method according to claim 1, It is characterized in that It further includes: Triggering the target object to collect tactile data.

14. The method according to any one of claims 1-13, It is characterized in that The target object includes at least one of the following: Tactile data acquisition function identifier; Tactile data type identifier; Format information of tactile data; Resolution information corresponding to the tactile data to be collected; Mesh identifier of the object to be collected or texture coordinate information of the mesh corresponding to the object to be collected; Storage location information of tactile data.

15. A processing device for tactile data, It is characterized in that It includes: A first acquisition module, configured to acquire tactile feedback data according to at least one of the following: Tactile data and a correction value of the tactile data; Association relationship between at least one tactile data and contact force or contact depth; Priority of tactile data and the object associated with the tactile data; Fusion data of at least two types of tactile data; Tactile data collected by a target object, where the target object is an object with a tactile data acquisition function.

16. The device according to claim 15, It is characterized in that The first acquisition module includes: A first acquisition sub-module, configured to obtain the correction value according to the contact force or contact depth; A second acquisition sub-module, configured to perform a correction operation on the tactile data according to the correction value to obtain tactile feedback data.

17. The device according to claim 16, It is characterized in that The first acquisition sub-module is configured to map the contact force or contact depth to a correction value according to a mapping function; Wherein, the contact force or contact depth is the input of the mapping function, and the correction value is the output of the mapping function.

18. The device according to claim 17, It is characterized in that Relevant information of the mapping function is indicated by first attribute information, and the first attribute information is carried in an object representing a tactile source or carried in an action object performing a tactile action.

19. The device according to any one of claims 16 to 18, It is characterized in that The second acquisition sub-module is configured to perform weighted summation, weighted multiplication or weighted addition and multiplication processing on the tactile data according to the correction value to obtain tactile feedback data.

20. The device according to any one of claims 16 to 19, It is characterized in that The relevant parameters of the correction operation are indicated by the second attribute information, which is carried in the object representing the tactile source or in the action object performing the tactile action.

21. The device according to claim 15, wherein, the first acquisition module includes: a third acquisition sub-module, configured to, when there is an association relationship between the tactile data and the contact force or the contact depth, acquire target tactile data corresponding to the target contact force or the target contact depth according to the association relationship; a fourth acquisition sub-module, configured to obtain the tactile feedback data according to the target tactile data.

22. The device according to claim 15, wherein, the first acquisition module is configured to select, from at least two objects, the tactile data associated with the object having the highest priority as the tactile feedback data, and the at least two objects are objects having an association relationship.

23. The device according to claim 22, wherein, the priority of the object is related to at least one of the following: a priority identifier; the arrangement order of the objects; the contact depth associated with the object; the contact force associated with the object.

24. The device according to claim 15, wherein, it further includes: a second acquisition module, configured to acquire at least two types of tactile data to be fused according to a protocol agreement or third attribute information of an object.

25. The device according to claim 15, wherein, it further includes: a fusion module, configured to fuse at least two types of tactile data according to a preset fusion method to obtain the fusion data; wherein, the preset fusion method includes at least one of the following: weighted summation, weighted product; weighted addition and multiplication.

26. The device according to claim 25, wherein, the fusion module is configured to, when the at least two types of tactile data meet a preset condition, fuse the at least two types of tactile data according to a preset fusion method to obtain the fusion data; wherein, the preset condition includes: the at least two types of tactile data act on the same tactile actuator.

27. The device according to claim 15, wherein, it further includes: a trigger module, configured to trigger a target object to collect tactile data.

28. The device according to any one of claims 15-27, wherein, the target object includes at least one of the following: a tactile data acquisition function identifier; a tactile data type identifier; format information of the tactile data; resolution information corresponding to the tactile data to be collected; a mesh identifier corresponding to the object to be collected or texture coordinate information of the mesh corresponding to the object to be collected; storage location information of the tactile data.

29. An electronic device, wherein, it includes a processor and a memory, the memory stores a program or instruction that can run on the processor, and when the program or instruction is executed by the processor, the steps of the method for processing tactile data according to any one of claims 1 to 14 are implemented.

30. A readable storage medium, wherein, The program or instructions are stored on the readable storage medium, and when the program or instructions are executed by the processor, the steps of the method for processing tactile data as described in any one of claims 1 to 14 are implemented.

31. A chip, characterized in that the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or instructions to implement the steps of the method for processing tactile data as described in any one of claims 1 to 14.

Images