Haptic data processing method and related devices

Abstract

The present application belongs to the technical field of rendering. Disclosed are a haptic data processing method and related devices. The haptic data processing method in the embodiments of the present application comprises: on the basis of data source location related information in an action object or a trigger object in an interactive behavior, acquiring haptic data; and on the basis of trigger information associated with the trigger object, acquiring target haptic data from the haptic data, wherein the trigger information comprises at least one of whether a collision occurs, collision location information, and collision mode information.

Description

Tactile data processing method and related equipment

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese Patent Application No. 202311697139.1 filed in China on December 11, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application belongs to the field of rendering technology, and specifically relates to a tactile data processing method and related equipment. Background Art

[0004] Haptic signal rendering is based on the interaction extension. Interactions are defined based on behaviors, which include trigger objects and action objects. The haptic feedback action (ACTION_SET_HAPTIC) in the action object is used to trigger haptic signal feedback. However, ACTION_SET_HAPTIC only supports directly playing pre-stored haptic signals, limiting the flexibility of haptic feedback. Summary of the Invention

[0005] The embodiments of the present application provide a method for processing tactile data and related devices, which can solve the problem in related solutions that the tactile feedback action only supports direct playback of pre-stored tactile signals, limiting the flexibility of tactile feedback.

[0006] In a first aspect, a method for processing tactile data is provided, which is executed by a rendering end and includes:

[0007] Acquiring tactile data based on information related to a data source location in an action object or a trigger object in an interactive behavior;

[0008] acquiring target tactile data from the tactile data according to trigger information associated with the trigger object;

[0009] The trigger information includes at least one of the following: whether a collision occurs, collision location information, and collision mode information.

[0010] In a second aspect, a tactile data processing device is provided, comprising:

[0011] A first acquisition module is used to acquire tactile data based on information related to a data source location in an action object or a trigger object in an interactive behavior;

[0012] a second acquisition module, configured to acquire target tactile data from the tactile data according to trigger information associated with the trigger object;

[0013] The trigger information includes at least one of the following: whether a collision occurs, collision location information, and collision mode information.

[0014] In a third aspect, an electronic device is provided, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0015] In a fourth aspect, an electronic device is provided, comprising a processor and a communication interface, wherein the processor is configured to obtain tactile data based on information related to a data source location in an action object or a trigger object in an interactive behavior;

[0016] acquiring target tactile data from the tactile data according to trigger information associated with the trigger object;

[0017] The trigger information includes at least one of whether a collision occurs, collision position information, and collision mode information.

[0018] In a fifth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0019] In a sixth aspect, a chip is provided, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is configured to run a program or instruction to implement the method described in the first aspect.

[0020] In a seventh aspect, a computer program / program product is provided, wherein the computer program / program product is stored in a storage medium and is executed by at least one processor to implement the steps of the method described in the first aspect.

[0021] In an embodiment of the present application, tactile data is obtained based on information related to the data source location of an action object or a trigger object in an interactive behavior; target tactile data is obtained from the tactile data based on trigger information associated with the trigger object; wherein the trigger information includes at least one of whether a collision occurred, collision location information, and collision method information. Through the above scheme, different target tactile data can be selected from the tactile data based on the trigger information of the trigger object relationship and then rendered, thereby obtaining different tactile feedback characteristics based on different target tactile data, thereby improving the flexibility of tactile feedback. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] FIG1 is a schematic diagram showing access to tactile data in an embodiment of the present application;

[0023] FIG2 is a schematic flow chart showing a method for processing tactile data according to an embodiment of the present application;

[0024] FIG3 is a schematic diagram showing a module of a device for processing tactile data according to an embodiment of the present application;

[0025] FIG4 is a block diagram showing the structure of an electronic device according to an embodiment of the present application;

[0026] FIG5 shows a structural block diagram of a terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the accompanying drawings in the embodiments of this application to clearly describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0028] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same type, and do not limit the number of objects, for example, the first object can be one or more. In addition, "or" in this application represents at least one of the connected objects. For example, "A or B" covers three options, namely, Option 1: including A but not including B; Option 2: including B but not including A; Option 3: including both A and B. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0029] The term "indication" in this application can be either a direct indication (or explicit indication) or an indirect indication (or implicit indication). A direct indication can be understood as the sender explicitly informing the receiver of specific information, the operation to be performed, or the requested result, etc. in the instruction sent; an indirect indication can be understood as the receiver determining the corresponding information based on the instruction sent by the sender, or making a judgment and determining the operation to be performed or the requested result, etc. based on the judgment result.

[0030] In order to enable those skilled in the art to better understand the embodiments of the present application, the following description is first given.

[0031] 1. Scene synthesis or rendering process.

[0032] The rendering engine can synthesize scenes from glTF (3D model file format) files, or it can render scenes based on glTF files. The engine can obtain glTF files by exchanging glTF files between the server and client via the glTF state stream, or by obtaining glTF files from program files stored on the server and client.

[0033] For tactile scene rendering, the engine may obtain tactile signal feedback, and then the tactile signal executor may play the tactile signal through one or more operations of encoding, transmitting, and decoding the tactile signal.

[0034] 2. The Moving Pictures Experts Group (MPEG) has made a haptic extension to glTF (a 3D model file format);

[0035] The MPEG_haptic extension provides access methods for haptic data. MPEG_haptic includes a set of haptic objects (hapticObjects), as shown in Table 1. As shown in Table 2, the accessor properties in hapticObjects point to the haptic data source.

[0036] Table 1

[0037] Table 2

[0038] Figure 1 shows how haptic data is accessed. The hapticObjects in MPEG_haptic point to the accessor, which in turn points to the circular buffer object. The circular buffer object provides access to the circular buffer's data location information and reflects the rendering attributes of the haptic data by referencing the MPEG_media object.

[0039] 3. MPEG also made haptic material-related extensions to glTF (MPEG_haptic_material extension);

[0040] The MPEG_haptic_material extension provides haptic properties for materials. MPEG_haptic_material contains material properties. MPEG_haptic_material is shown in Table 3, and MPEG_haptic_material.material is shown in Table 4.

[0041] Table 3

[0042] The materials property points to a set of material objects. Each material object points to the hapticObject in MPEG_haptic that provides the data source through the haptic property, and indicates the relevant material tactile properties (such as stiffness, friction, etc.) and texture coordinate information.

[0043] Table 4

[0044] 4. Interactivity extension in glTF (3D model file format);

[0045] glTF is used for efficient transmission and loading of 3D scenes and models by engines and applications. glTF defines an extensible publishing format that simplifies authoring workflows and interactive services by enabling interoperable use of 3D content across the industry.

[0046] The Moving Picture Experts Group (MPEG) has extended glTF and initially designed standards for interactivity. For haptic scenarios, these standards primarily describe the interaction between avatars and haptic sources, triggering and generating haptic signals based on this interaction.

[0047] Interactivity is defined based on behavior. Behavior includes trigger condition nodes (i.e., trigger nodes, which can also be described as trigger objects) and action nodes (i.e., action nodes, which can also be described as action objects). The trigger node includes collision conditions, proximity conditions, and visibility conditions. The action node includes tactile feedback actions (i.e., ACTION_SET_HAPTIC), activation nodes (ACTION_ACTIVATE), etc. ACTION_SET_HAPTIC is implemented based on tactile action node objects (HapticActionNode objects) or tactile action media objects (HapticActionMedia objects).

[0048] The hapticObject attribute in HapticActionNode (as shown in Table 5) points to the hapticObject in MPEG_haptic, and the media index (mediaIndex) attribute in HapticActionMedia (as shown in Table 6) further points to the accessor in the above hapticObject, thereby accessing the tactile data.

[0049] Table 5

[0050] Table 6

[0051] The object in the embodiment of the present application includes at least one of the following: scene, node, mesh, material, vertices, face, primitive, etc.

[0052] The following describes in detail the tactile data processing method and related devices provided by the embodiments of the present application through some embodiments and their application scenarios in conjunction with the accompanying drawings.

[0053] As shown in FIG2 , an embodiment of the present application provides a method for processing tactile data, which is executed by a rendering end including a rendering engine or a display engine. The method includes:

[0054] Step 201: Acquire tactile data based on the data source location related information in the action object or trigger object in the interactive behavior.

[0055] Optionally, the action object and the trigger object are objects in a behavior defined by an interaction standard (ie, the above-mentioned interaction behavior), and the interaction standard is a standard for describing the interaction between an avatar and a tactile source.

[0056] Optionally, the action object includes a tactile action media object, the tactile action media object includes a media index, and the media index is associated with information related to a data source location;

[0057] Alternatively, the trigger object includes a node or a primitive, and attributes of the node or primitive are associated with information related to a data source location.

[0058] Optionally, the data source location related information includes information for indicating the location of tactile data (such as the tactile data source).

[0059] As an implementation method, obtaining tactile data based on the data source location information in the action object includes: following the interaction processing model, the rendering end obtains tactile data according to the tactile feedback action (ACTION_SET_HAPTIC), at least the mediaIndex contained in the HapticActionMedia object can point to the tactile data in MPEG_haptic.

[0060] Optionally, ACTION_SET_HAPTIC may also be associated with a specific haptic processing object, which may indicate information such as a haptic data source based on haptic material.

[0061] As an implementation method, based on the data source location related information in the trigger object, obtaining tactile data includes: indicating the haptic material attribute associated with the node / primitive based on the property associated with the node / primitive in the trigger object, and the haptic material attribute can point to the tactile data source corresponding to the haptic material.

[0062] Optionally, the tactile data source includes tactile data, and the tactile data includes texture data.

[0063] Step 202: acquiring target tactile data from the tactile data according to the trigger information associated with the trigger object;

[0064] The trigger information includes at least one of whether a collision occurs, collision position information, and collision mode information.

[0065] Optionally, the target tactile data is tactile data that needs to be rendered.

[0066] In an embodiment of the present application, tactile data is obtained based on information related to the data source location of an action object or a trigger object in an interactive behavior; target tactile data is obtained from the tactile data based on trigger information associated with the trigger object; wherein the trigger information includes at least one of whether a collision occurred, collision location information, and collision method information. Through the above scheme, different target tactile data can be selected from the tactile data based on the trigger information of the trigger object relationship and then rendered, thereby obtaining different tactile feedback characteristics based on different target tactile data, thereby improving the flexibility of tactile feedback.

[0067] Optionally, acquiring tactile data includes:

[0068] Acquiring metadata information associated with the tactile data based on attribute information in the action object or the trigger object;

[0069] Acquire tactile data from a tactile data source according to the metadata information and information related to a data source location in an action object or a trigger object in the interactive behavior;

[0070] The metadata information includes at least one of storage format information of the tactile data, resolution information of the tactile data, and tactile type information associated with the tactile data.

[0071] Here, based on the metadata information, corresponding tactile data is obtained from the tactile data source, such as tactile data of a corresponding tactile type or resolution.

[0072] For example, an attribute may be added to the object associated with ACTION_SET_HAPTIC to indicate metadata information associated with the haptic data source, or the metadata may be indicated through a haptic material attribute associated with a node / primitive in a trigger object.

[0073] Optionally, the method of the embodiment of the present application further includes:

[0074] determining a method for acquiring the tactile data according to first attribute information associated with the trigger object or the action object;

[0075] The first attribute information includes at least one of the following:

[0076] Item 1: Acquisition method identification information;

[0077] Item 2: tactile feedback type;

[0078] Item 3: Type of tactile material;

[0079] The acquisition method includes at least one of the following:

[0080] acquiring a tactile data source based on the data source location related information in the action object;

[0081] A tactile data source is acquired based on the data source position related information in the trigger object.

[0082] For example, an object associated with ACTION_SET_HAPTIC (eg, HapticActionNode or HapticActionMedia) may indicate a method for obtaining tactile data through a property. The property may indicate whether the tactile data is obtained according to a trigger object.

[0083] For the first item above, the acquisition method identification information indicates whether the tactile data is acquired based on the data source location related information in the trigger object. For example, the acquisition method identification information is represented by a flag, flag=0 indicates that the tactile data is acquired based on the data source location related information in the trigger object, and flag=1 indicates that the tactile data is acquired based on the data source location related information in the action object, such as acquired by HapticActionNode or HapticActionMedia.

[0084] Optionally, the flag may be an existing use collider flag or a newly added flag.

[0085] Regarding the second item above, the haptic data acquisition method can be determined based on the haptic feedback type indicated by the haptic action node (HapticActionNode). For example, for preset haptic feedback types such as stiffness, friction, texture, and vibration, the haptic data source is obtained based on the data source location information in the trigger object.

[0086] Regarding the third item above: for example, if the tactile material attribute (materialHapticModality) of the feedback action-related object (HapticActionNode) is not empty or indicates a preset material type, the tactile data is obtained according to the data source location related information in the trigger object.

[0087] Optionally, acquiring target tactile data from the tactile data according to the trigger information associated with the trigger object includes:

[0088] Get the texture coordinates corresponding to the collision position between sub-objects in the trigger object;

[0089] Target tactile data is acquired from the tactile data according to the mapping coordinates.

[0090] The sub-objects in the trigger object include mesh vertices.

[0091] In an embodiment of the present application, if a mesh is used to implement the geometry of a haptic source, the mesh may be attached with a haptic material, and the mesh may be directly used in a trigger object of an interaction to determine a collision condition.

[0092] When a trigger object (such as a collision trigger) in an interaction behavior is activated, the rendering engine determines the mesh vertex corresponding to the contact point (haptic source and avatar) and obtains the texture coordinates corresponding to the mesh vertex corresponding to the contact point. It then selects the target haptic data corresponding to the texture coordinates from the haptic data, extracting discrete haptic data based on the texture coordinates and improving the flexibility of haptic feedback.

[0093] Optionally, acquiring target tactile data from the tactile data according to the trigger information associated with the trigger object includes:

[0094] Get the texture coordinates associated with the sub-object that collides with the trigger object;

[0095] Target tactile data is acquired from the tactile data according to the mapping coordinates.

[0096] The sub-objects that collide include primitives or nodes.

[0097] In an embodiment of the present application, a plurality of relatively simple primitives are used to form the geometry of a haptic source. These primitives are used in a trigger object of interaction to determine a collision condition.

[0098] When a trigger object (such as a collision trigger) in an interactivity behavior is activated, the rendering engine obtains the texture coordinates corresponding to the node / primitive in the corresponding trigger object. Based on one or more texture coordinates, the rendering engine obtains the target tactile data from the haptic material data source and generates a tactile feedback signal. This achieves the purpose of extracting discrete tactile data based on the texture coordinates, thereby improving the flexibility of tactile feedback.

[0099] Optionally, obtaining the mapping coordinates associated with the sub-object that collides in the triggering object includes:

[0100] The mapping coordinates are obtained according to the attribute information of the sub-object that collides.

[0101] As an implementation manner, the attribute information of the sub-object that collides includes mapping coordinates or vertex information of a mesh, and the vertex information of the mesh corresponds to the mapping coordinates.

[0102] For example, the property associated with the node / primitive in the trigger object can indicate a certain texture coordinate of the haptic material corresponding to the node / primitive. When the rendering engine determines that the node / primitive collides, it can obtain the corresponding texture coordinate.

[0103] Optionally, properties associated with nodes / primitives in the trigger object need to be defined using a standard definition. For example, a trigger object may contain a texture coordinate list, with the node list / primitive list in the trigger object corresponding to the texture coordinate list, and each node / primitive corresponding to one of the texture coordinates.

[0104] Alternatively, a texture coordinate of the haptic material can be indirectly indicated. For example, a property associated with a node / primitive in a trigger object can indicate a vertex of the mesh to which the haptic material is attached, and the mesh vertex can be mapped to the texture coordinate of the haptic material.

[0105] Optionally, acquiring target tactile data from the tactile data according to the trigger information associated with the trigger object includes:

[0106] determining a rendering mode according to second attribute information associated with the action object;

[0107] In a case where the rendering mode indicates obtaining target tactile data from the tactile data according to the mapping coordinates, obtaining the target tactile data from the tactile data according to the trigger information associated with the trigger object;

[0108] The second attribute information includes at least one of the following:

[0109] A1: Rendering mode identification information, which can be the existing useCollider flag or a newly added flag;

[0110] A2: tactile feedback type; for example, for preset types of tactile feedback such as stiffness / friction / texture / vibration texture, the rendering end extracts target tactile data from the tactile data according to the mapping coordinates for rendering feedback.

[0111] A3: Haptic material type. For example, when the haptic material property (materialHapticModality) of the object (HapticActionNode) related to the feedback action is not empty or indicates a preset material type, the rendering end extracts the target haptic data from the haptic data based on the texture coordinates for rendering feedback.

[0112] In an embodiment of the present application, the object associated with ACTION_SET_HAPTIC (such as HapticActionNode or HapticActionMedia) can indicate the rendering method of tactile data through a property. This property can indicate whether the rendering end extracts target tactile data from the tactile data according to the mapping coordinates for rendering feedback.

[0113] Optionally, the collision mode includes at least one of the following: sliding across a surface; knocking; pressing; contact point movement direction; contact point movement speed.

[0114] Optionally, the method of the embodiment of the present application further includes:

[0115] The collision mode is obtained according to a user input modality attribute of a trigger object.

[0116] In the embodiment of the present application, the collision mode is obtained based on the user input mode attribute of the trigger object, and the rendering end can trigger corresponding tactile feedback based on the collision mode.

[0117] Optionally, the acquiring of tactile data based on the information related to the data source location in the action object or the trigger object in the interactive behavior includes:

[0118] Acquire tactile data based on the location information of the data source in the action object or trigger object in the interactive behavior and the collision mode.

[0119] In an embodiment of the present application, the rendering end can determine the tactile data source based on the information related to the data source position in the action object or trigger object in the interactive behavior, and then select the corresponding tactile data from the tactile data source according to the collision mode, so that the subsequent rendering end can feedback different tactile signals according to different collision modes.

[0120] Optionally, acquiring target tactile data from the tactile data according to the trigger information associated with the trigger object includes:

[0121] According to the collision mode, obtain the mapping coordinate sequence;

[0122] Target tactile data is acquired from the tactile data according to the mapping coordinate sequence.

[0123] Exemplarily, the texture of the tactile material is mapped on the mesh in a preset direction (horizontal / vertical, etc.), the contact point crosses the surface of the material in different directions (horizontal / vertical, etc.), and the rendering end collects the tactile signal according to different texture coordinate sorting (i.e., texture coordinate sequence) to generate a feedback tactile signal (i.e., target tactile signal). Among them, the texture coordinate sorting can include multiple groups of different texture coordinates. In addition, in the embodiment of the present application, a plurality of different texture coordinate sorting methods can be pre-set, and different texture coordinate sorting methods correspond to different collision methods, and then the corresponding texture coordinate sorting method can be obtained according to the collision method.

[0124] Based on the above solution, the rendering end can extract discrete tactile signals (ie, target tactile signals) from the tactile signals according to the contact points and play them, thereby improving the flexibility of tactile feedback.

[0125] The tactile data processing method provided in the embodiment of the present application can be executed by a tactile data processing device. In the embodiment of the present application, the tactile data processing device executing the tactile data processing method is used as an example to illustrate the tactile data processing device provided in the embodiment of the present application.

[0126] As shown in FIG3 , an embodiment of the present application provides a tactile data processing device 300 , including:

[0127] A first acquisition module 301 is configured to acquire tactile data based on information related to a data source location in an action object or a trigger object in an interactive behavior;

[0128] a second acquisition module 302, configured to acquire target tactile data from the tactile data according to the trigger information associated with the trigger object;

[0129] The trigger information includes at least one of whether a collision occurs, collision position information, and collision mode information.

[0130] Optionally, the first acquisition module includes:

[0131] A first acquisition submodule, configured to acquire metadata information associated with the tactile data based on attribute information in the action object or the trigger object;

[0132] A second acquisition submodule is configured to acquire tactile data from a tactile data source based on the metadata information and information related to a data source location in an action object or a trigger object in the interactive behavior;

[0133] The metadata information includes at least one of storage format information of the tactile data, resolution information of the tactile data, and tactile type information associated with the tactile data.

[0134] Optionally, the device further comprises:

[0135] a determination module, configured to determine a method for acquiring the tactile data according to first attribute information associated with the trigger object or the action object;

[0136] The first attribute information includes at least one of the following:

[0137] Acquisition method identification information;

[0138] Type of tactile feedback;

[0139] Type of tactile material;

[0140] The acquisition method includes at least one of the following:

[0141] acquiring a tactile data source based on the data source location related information in the action object;

[0142] A tactile data source is acquired based on the data source position related information in the trigger object.

[0143] Optionally, the second acquisition module includes:

[0144] The third acquisition submodule is used to obtain the mapping coordinates corresponding to the collision positions between sub-objects in the trigger object;

[0145] The fourth acquisition submodule is configured to acquire target tactile data from the tactile data according to the mapping coordinates.

[0146] Optionally, the second acquisition module includes:

[0147] The fifth acquisition submodule is used to obtain the mapping coordinates associated with the sub-object that collides with the trigger object;

[0148] The sixth acquisition submodule is configured to acquire target tactile data from the tactile data according to the mapping coordinates.

[0149] Optionally, the fifth acquisition submodule is used to acquire the mapping coordinates according to attribute information of the sub-object that collides.

[0150] Optionally, the attribute information of the sub-object that collides includes mapping coordinates or vertex information of a mesh, and the vertex information of the mesh corresponds to the mapping coordinates.

[0151] Optionally, the second acquisition module includes:

[0152] a determination submodule, configured to determine a rendering mode according to second attribute information associated with the action object;

[0153] a seventh acquisition submodule, configured to acquire target tactile data from the tactile data according to trigger information associated with the trigger object, when the rendering mode indicates acquiring target tactile data from the tactile data according to the mapping coordinates;

[0154] The second attribute information includes at least one of the following:

[0155] Rendering mode identification information;

[0156] Type of tactile feedback;

[0157] Tactile material type.

[0158] Optionally, the collision mode includes at least one of the following: sliding across a surface; knocking; pressing; contact point movement direction, contact point movement speed.

[0159] Optionally, the device further comprises:

[0160] The third acquisition module is configured to acquire the collision mode according to a user input mode attribute of the trigger object.

[0161] Optionally, the first acquisition module is used to acquire tactile data based on information related to a data source position and a collision mode in an action object or a trigger object in an interactive behavior.

[0162] Optionally, the second acquisition module includes:

[0163] The eighth acquisition submodule is used to obtain a mapping coordinate sequence according to the collision mode;

[0164] A ninth acquisition submodule is configured to acquire target tactile data from the tactile data according to the mapping coordinate sequence.

[0165] The device of an embodiment of the present application obtains tactile data based on information related to the data source location in an action object or a trigger object in an interactive behavior; and obtains target tactile data from the tactile data based on trigger information associated with the trigger object; wherein the trigger information includes at least one of whether there is a collision, collision location information, and collision method information. Through the above scheme, different target tactile data can be selected from the tactile data based on the trigger information of the trigger object relationship and then rendered, thereby obtaining different tactile feedback characteristics based on different target tactile data, thereby improving the flexibility of tactile feedback.

[0166] The tactile data processing device provided in the embodiment of the present application can implement each process implemented in the method embodiment of Figure 2 and achieve the same technical effect. To avoid repetition, it will not be described here.

[0167] Optionally, as shown in Figure 4, an embodiment of the present application also provides an electronic device 400, including a processor 401 and a memory 402, and the memory 402 stores programs or instructions that can be run on the processor 401. For example, when the electronic device 400 is a terminal, the program or instructions are executed by the processor 401 to implement the various steps of the above-mentioned tactile data processing method embodiment, and can achieve the same technical effect. To avoid repetition, they are not repeated here.

[0168] The present application also provides an electronic device including a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is configured to execute a program or instruction to implement the steps in the method embodiment shown in FIG2 . This device embodiment corresponds to the above-described method embodiment, and each implementation process and implementation method of the above-described method embodiment are applicable to this electronic device embodiment and can achieve the same technical effects.

[0169] The electronic device may be a terminal, or may be other devices other than a terminal, such as a server, a network attached storage (NAS), etc.

[0170] Among them, the terminal can be a mobile phone, tablet personal computer, laptop computer, notebook computer, personal digital assistant (PDA), handheld computer, netbook, ultra-mobile personal computer (UMPC), mobile Internet device (MID), augmented reality (AR), virtual reality (VR) equipment, mixed reality (MR) equipment, robot, wearable device, flight vehicle, vehicle user equipment (VUE), shipborne equipment, pedestrian user equipment (PUE), smart home (home appliances with wireless communication function, such as refrigerator, TV, washing machine or furniture, etc.), game console, personal computer (PC), ATM or self-service machine and other terminal-side devices. Wearable devices include: smart watches, smart bracelets, smart headphones, smart glasses, smart jewelry (smart bracelets, smart bracelets, smart rings, smart necklaces, smart anklets, smart anklets, etc.), smart wristbands, smart clothing, etc. Among them, vehicle-mounted devices can also be called vehicle-mounted terminals, vehicle-mounted controllers, vehicle-mounted modules, vehicle-mounted components, vehicle-mounted chips, or vehicle-mounted units, etc. It should be noted that the specific type of terminal is not limited in the embodiments of this application.

[0171] The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), or cloud computing services based on big data and artificial intelligence platforms.

[0172] Taking an electronic device as a terminal as an example, FIG5 is a schematic diagram of the hardware structure of a terminal implementing an embodiment of the present application.

[0173] The terminal 500 includes but is not limited to: a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509 and at least some of the components of the processor 510.

[0174] Those skilled in the art will appreciate that the terminal 500 may also include a power supply (such as a battery) to power various components. The power supply may be logically connected to the processor 510 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The terminal structure shown in FIG5 does not limit the terminal. The terminal may include more or fewer components than shown, or may combine certain components, or have different component arrangements, which will not be described in detail here.

[0175] It should be understood that in the embodiment of the present application, the input unit 504 may include a graphics processing unit (GPU) 5041 and a microphone 5042. The graphics processor 5041 processes the image data of a static picture or video obtained by an image acquisition device (such as a camera) in a video acquisition mode or an image acquisition mode, or may process the obtained point cloud data. The display unit 506 may include a display panel 5061, which may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 507 includes a touch panel 5071 and at least one of other input devices 5072. The touch panel 5071 is also called a touch screen. The touch panel 5071 may include two parts: a touch detection device and a touch controller. Other input devices 5072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0176] In the embodiment of the present application, after receiving downlink data from a network-side device, the radio frequency unit 501 may transmit the data to the processor 510 for processing. Furthermore, the radio frequency unit 501 may send uplink data to the network-side device. Typically, the radio frequency unit 501 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, and the like.

[0177] The memory 509 can be used to store software programs or instructions and various data. The memory 509 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 509 may include a volatile memory or a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM), and a direct memory bus random access memory (DRRAM). The memory 509 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0178] Processor 510 may include one or more processing units. Optionally, processor 510 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 510.

[0179] The processor 510 is configured to obtain tactile data based on information related to a data source location in an action object or a trigger object in an interactive behavior;

[0180] acquiring target tactile data from the tactile data according to trigger information associated with the trigger object;

[0181] The trigger information includes at least one of whether a collision occurs, collision position information, and collision mode information.

[0182] Optionally, the processor 510 is further configured to:

[0183] Acquiring metadata information associated with the tactile data based on attribute information in the action object or the trigger object;

[0184] Acquire tactile data from a tactile data source according to the metadata information and information related to a data source location in an action object or a trigger object in the interactive behavior;

[0185] The metadata information includes at least one of storage format information of the tactile data, resolution information of the tactile data, and tactile type information associated with the tactile data.

[0186] Optionally, the processor 510 is further configured to:

[0187] determining a method for acquiring the tactile data according to first attribute information associated with the trigger object or the action object;

[0188] The first attribute information includes at least one of the following:

[0189] Acquisition method identification information;

[0190] Type of tactile feedback;

[0191] Type of tactile material;

[0192] The acquisition method includes at least one of the following:

[0193] acquiring a tactile data source based on the data source location related information in the action object;

[0194] A tactile data source is acquired based on the data source position related information in the trigger object.

[0195] Optionally, the processor 510 is further configured to:

[0196] Get the texture coordinates corresponding to the collision position between sub-objects in the trigger object;

[0197] Target tactile data is acquired from the tactile data according to the mapping coordinates.

[0198] Optionally, the processor 510 is further configured to:

[0199] Get the texture coordinates associated with the sub-object that collides with the trigger object;

[0200] Target tactile data is acquired from the tactile data according to the mapping coordinates.

[0201] Optionally, the processor 510 is further configured to:

[0202] The mapping coordinates are obtained according to the attribute information of the sub-object that collides.

[0203] Optionally, the attribute information of the sub-object that collides includes mapping coordinates or vertex information of a mesh, and the vertex information of the mesh corresponds to the mapping coordinates.

[0204] Optionally, the processor 510 is further configured to:

[0205] determining a rendering mode according to second attribute information associated with the action object;

[0206] In a case where the rendering mode indicates obtaining target tactile data from the tactile data according to the mapping coordinates, obtaining the target tactile data from the tactile data according to the trigger information associated with the trigger object;

[0207] The second attribute information includes at least one of the following:

[0208] Rendering mode identification information;

[0209] Type of tactile feedback;

[0210] Tactile material type.

[0211] Optionally, the collision mode includes at least one of the following: sliding across a surface; knocking; pressing; contact point movement direction, contact point movement speed.

[0212] Optionally, the processor 510 is further configured to:

[0213] The collision mode is obtained according to a user input mode attribute of a trigger object.

[0214] Optionally, the processor 510 is further configured to:

[0215] Acquire tactile data based on the location information of the data source in the action object or trigger object in the interactive behavior and the collision mode.

[0216] Optionally, the processor 510 is further configured to:

[0217] According to the collision mode, obtain the mapping coordinate sequence;

[0218] Target tactile data is acquired from the tactile data according to the mapping coordinate sequence.

[0219] In an embodiment of the present application, tactile data is obtained based on information related to the data source location of an action object or a trigger object in an interactive behavior; target tactile data is obtained from the tactile data based on trigger information associated with the trigger object; wherein the trigger information includes at least one of whether a collision occurred, collision location information, and collision method information. Through the above scheme, different target tactile data can be selected from the tactile data based on the trigger information of the trigger object relationship and then rendered, thereby obtaining different tactile feedback characteristics based on different target tactile data, thereby improving the flexibility of tactile feedback.

[0220] It can be understood that the implementation process of each implementation method mentioned in this embodiment can refer to the relevant description of the tactile data processing method of the method embodiment, and achieve the same or corresponding technical effects. To avoid repetition, it will not be described here.

[0221] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned embodiment of the method for processing tactile data is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0222] The processor is the processor in the terminal described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk. In some examples, the readable storage medium may be a non-transitory readable storage medium.

[0223] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned tactile data processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0224] It should be understood that the chip mentioned in the embodiments of the present application may include a system-level chip (also referred to as a system chip, a chip system or a system-on-chip chip), and may also include an independent display chip, etc.

[0225] An embodiment of the present application further provides a computer program / program product, which is stored in a storage medium and is executed by at least one processor to implement the various processes of the above-mentioned tactile data processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0226] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0227] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of a computer software product plus a necessary general-purpose hardware platform, or of course, by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes a number of instructions for enabling a terminal or network-side device to execute the methods described in each embodiment of the present application.

[0228] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms of implementation methods without departing from the purpose of this application and the scope of protection of the claims. These implementation methods are all within the protection of this application.

Claims

1. A method for processing tactile data, executed by a rendering end, comprising: Acquiring tactile data based on information related to a data source location in an action object or a trigger object in an interactive behavior; acquiring target tactile data from the tactile data according to the trigger information associated with the trigger object; The trigger information includes at least one of whether there is a collision, collision position information, and collision mode information.

2. The method according to claim 1, wherein: The acquiring of tactile data comprises: Acquiring metadata information associated with the tactile data based on attribute information in the action object or the trigger object; Acquire tactile data from a tactile data source according to the metadata information and information related to a data source location in an action object or a trigger object in the interactive behavior; The metadata information includes at least one of storage format information of the tactile data, resolution information of the tactile data, and tactile type information associated with the tactile data.

3. The method according to claim 1, further comprising: Determining a method for acquiring the tactile data according to first attribute information associated with the trigger object or the action object; The first attribute information includes at least one of the following: Acquisition method identification information; Type of tactile feedback; Type of tactile material; The acquisition method includes at least one of the following: Acquire a tactile data source based on the data source location related information in the action object; A tactile data source is acquired based on the data source position related information in the trigger object.

4. The method according to claim 1, wherein: Acquiring target tactile data from the tactile data according to the trigger information associated with the trigger object, comprising: Get the texture coordinates corresponding to the collision position between sub-objects in the trigger object; According to the mapping coordinates, target tactile data is acquired from the tactile data.

5. The method according to claim 1, wherein: Acquiring target tactile data from the tactile data according to the trigger information associated with the trigger object, comprising: Get the texture coordinates associated with the sub-object that collides in the trigger object; According to the mapping coordinates, target tactile data is acquired from the tactile data.

6. The method according to claim 5, wherein: The step of obtaining the mapping coordinates associated with the sub-object that collides in the triggering object includes: The mapping coordinates are obtained according to the attribute information of the sub-object that collides.

7. The method according to claim 6, wherein: The attribute information of the sub-object that collides includes mapping coordinates or vertex information of a mesh, and the vertex information of the mesh corresponds to the mapping coordinates.

8. The method according to any one of claims 1 to 7, wherein: Acquiring target tactile data from the tactile data according to the trigger information associated with the trigger object, comprising: Determining a rendering mode according to second attribute information associated with the action object; In a case where the rendering mode indicates obtaining target tactile data from the tactile data according to the mapping coordinates, obtaining the target tactile data from the tactile data according to the trigger information associated with the trigger object; The second attribute information includes at least one of the following: Rendering mode identification information; Type of tactile feedback; Tactile material type.

9. The method according to claim 1, wherein: The collision mode includes at least one of the following: sliding across the surface; knocking; pressing; contact point movement direction, contact point movement speed.

10. The method according to claim 1 or 9, further comprising: The collision mode is obtained according to a user input mode attribute of a trigger object.

11. The method according to claim 1, wherein: The step of acquiring tactile data based on the information related to the data source location in the action object or the trigger object in the interactive behavior includes: Based on the information related to the data source position in the action object or the trigger object in the interactive behavior and the collision mode, the tactile data is obtained.

12. The method according to claim 1, wherein: The acquiring target tactile data from the tactile data according to the trigger information associated with the trigger object includes: According to the collision mode, obtain the texture coordinate sequence; According to the mapping coordinate sequence, target tactile data is acquired from the tactile data.

13. A tactile data processing device, comprising: A first acquisition module, configured to acquire tactile data based on information related to a data source location in an action object or a trigger object in an interactive behavior; a second acquisition module, configured to acquire target tactile data from the tactile data according to trigger information associated with the trigger object; The trigger information includes at least one of whether there is a collision, collision position information, and collision mode information.

14. The device according to claim 13, wherein: The first acquisition module includes: A first acquisition submodule, configured to acquire metadata information associated with the tactile data based on attribute information in an action object or a trigger object; A second acquisition submodule is used to acquire tactile data from a tactile data source according to the metadata information and information related to a data source location in an action object or a trigger object in an interactive behavior; The metadata information includes at least one of storage format information of the tactile data, resolution information of the tactile data, and tactile type information associated with the tactile data.

15. The apparatus according to claim 13, further comprising: a determination module, configured to determine a method for acquiring the tactile data according to first attribute information associated with the trigger object or the action object; The first attribute information includes at least one of the following: Acquisition method identification information; Type of tactile feedback; Type of tactile material; The acquisition method includes at least one of the following: Acquire a tactile data source based on the data source location related information in the action object; A tactile data source is acquired based on the data source position related information in the trigger object.

16. The device according to claim 13, wherein: The second acquisition module includes: The third acquisition submodule is used to obtain the map coordinates corresponding to the collision positions between sub-objects in the trigger object; The fourth acquisition submodule is used to acquire target tactile data from the tactile data according to the mapping coordinates.

17. The device according to claim 13, wherein: The second acquisition module includes: The fifth acquisition submodule is used to acquire the map coordinates associated with the sub-object that collides in the trigger object; The sixth acquisition submodule is used to acquire target tactile data from the tactile data according to the mapping coordinates.

18. The device according to claim 17, wherein: The fifth acquisition submodule is used to acquire the mapping coordinates according to the attribute information of the sub-object that collides.

19. The device according to claim 18, wherein: The attribute information of the sub-object that collides includes mapping coordinates or vertex information of a mesh, and the vertex information of the mesh corresponds to the mapping coordinates.

20. The device according to any one of claims 13 to 19, wherein: The second acquisition module includes: A determination submodule, used to determine a rendering mode according to second attribute information associated with the action object; a seventh acquisition submodule, configured to acquire target tactile data from the tactile data according to trigger information associated with the trigger object, when the rendering mode indicates acquiring target tactile data from the tactile data according to the mapping coordinates; The second attribute information includes at least one of the following: Rendering mode identification information; Type of tactile feedback; Tactile material type.

21. The device according to claim 13, wherein: The collision mode includes at least one of the following: sliding across the surface; knocking; pressing; contact point movement direction, contact point movement speed.

22. The apparatus according to claim 13 or 21, further comprising: The third acquisition module is used to acquire the collision mode according to the user input mode attribute of the trigger object.

23. The device according to claim 13, wherein: The first acquisition module is used to acquire tactile data based on the information related to the data source position in the action object or the trigger object in the interactive behavior and the collision mode.

24. The device according to claim 13, wherein: The second acquisition module includes: The eighth acquisition submodule is used to acquire a texture coordinate sequence according to the collision mode; The ninth acquisition submodule is used to acquire target tactile data from the tactile data according to the mapping coordinate sequence.

25. An electronic device comprising a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method for processing tactile data according to any one of claims 1 to 12 are implemented. 26 . A readable storage medium storing a program or an instruction, wherein the program or the instruction, when executed by a processor, implements the steps of the method for processing tactile data according to claim 1 .

27. A chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the steps of the method for processing tactile data as described in any one of claims 1 to 12.

Images