Haptic signal processing method and apparatus, and device

Abstract

The present application relates to the technical field of communications, and discloses a haptic signal processing method and apparatus, and a device. The haptic signal processing method in embodiments of the present application comprises: a first device acquires haptic signal processing-related information, wherein the haptic signal processing-related information comprises at least one of the following: first information, second information, and third information, the first information is used for representing spatial distribution characteristics of signal strength of a target haptic signal, the second information is perception-related information of the target haptic signal, the third information is used for indicating a trigger condition, and the trigger condition is a trigger condition for triggering an action object; and the first device performs haptic signal processing on the basis of the haptic signal processing-related information.

Description

Tactile signal processing methods, devices and equipment

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411486635.7, filed in China on October 23, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application belongs to the field of computer technology, and specifically relates to a tactile signal processing method, apparatus and device. Background Technology

[0004] Related technologies support basic haptic rendering capabilities. For example, the rendering engine can render haptic scenes based on Graphics Language Transmission Format (glTF) files. However, in haptic scenes, for signal sources that the target object is not actually touching, the engine cannot generate haptic signals from the target object to those signal sources, resulting in poor realism in the rendered haptic scenes. Summary of the Invention

[0005] This application provides a tactile signal processing method, apparatus, and device that can solve the problem of poor realism in rendered tactile scenes.

[0006] In a first aspect, a tactile signal processing method is provided, executed by a first device, the method comprising:

[0007] The first device acquires tactile signal processing related information, which includes at least one of the following: first information, second information, and third information. The first information is used to characterize the spatial distribution characteristics of the signal intensity of the target tactile signal, the second information is the perception-related information of the target tactile signal, and the third information is used to indicate the triggering condition, which is the triggering condition of the triggering action object.

[0008] The first device performs tactile signal processing based on the tactile signal processing related information.

[0009] Secondly, a tactile signal processing device is provided, comprising:

[0010] The processing module is used to acquire tactile signal processing related information, which includes at least one of the following: first information, second information, and third information. The first information is used to characterize the spatial distribution characteristics of the signal intensity of the target tactile signal, the second information is the perception-related information of the target tactile signal, and the third information is used to indicate the triggering condition, which is the triggering condition of the triggering action object.

[0011] The processing module is also used to perform tactile signal processing based on the tactile signal processing related information.

[0012] Thirdly, a tactile signal processing apparatus is provided, the apparatus being configured to perform the steps of the method described in the first aspect.

[0013] Fourthly, an electronic device is provided, comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method as described in the first aspect.

[0014] Fifthly, an electronic device is provided, including a processor and a communication interface, wherein,

[0015] The processor is configured to acquire tactile signal processing related information, the tactile signal processing related information including at least one of the following: first information, second information, and third information, the first information being used to characterize the spatial distribution characteristics of the signal intensity of the target tactile signal, the second information being perception-related information of the target tactile signal, and the third information being used to indicate triggering conditions, the triggering conditions being triggering conditions for triggering an action object;

[0016] The processor is also configured to perform tactile signal processing based on the tactile signal processing related information.

[0017] In a sixth aspect, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0018] In a seventh aspect, a chip is provided, the chip including a processor and a communication interface coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0019] Eighthly, a computer program / program product is provided, the computer program / program product being stored in a storage medium, the computer program / program product being executed by at least one processor to implement the method as described in the first aspect.

[0020] In this embodiment, a first device acquires tactile signal processing related information, which includes at least one of the following: first information, second information, and third information. The first information characterizes the spatial distribution characteristics of the signal intensity of the target tactile signal, the second information is perception-related information of the target tactile signal, and the third information indicates a triggering condition, which is a triggering condition for a triggering action object. The first device performs tactile signal processing based on the tactile signal processing related information. Thus, by utilizing at least one of the spatial distribution characteristics of the signal intensity of the target tactile signal, the perception-related information of the target tactile signal, and the triggering condition for a triggering action object, the descriptive function of the tactile scene is enhanced. Using this enhanced descriptive function to assist tactile signal processing can improve the realism of the rendered tactile scene. Attached Figure Description

[0021] Figure 1 is a schematic diagram of a tactile data access method in related technologies;

[0022] Figure 2 is a flowchart of a tactile signal processing method provided in an embodiment of this application;

[0023] Figure 3 is one of the schematic diagrams showing the spatial distribution of signal intensity of a signal source according to an embodiment of this application;

[0024] Figure 4 is a second schematic diagram of the spatial distribution of signal intensity of a signal source provided in an embodiment of this application;

[0025] Figure 5 is a schematic diagram of one of the tactile coding methods provided in the embodiments of this application;

[0026] Figure 6 is a second schematic diagram of a tactile coding method provided in an embodiment of this application;

[0027] Figure 7 is a third schematic diagram of a tactile coding method provided in an embodiment of this application;

[0028] Figure 8 is a schematic diagram of the structure of a tactile signal processing device provided in an embodiment of this application;

[0029] Figure 9 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0030] Figure 10 is a schematic diagram of the structure of a terminal provided in an embodiment of this application. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0032] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, the first object can be one or more. Furthermore, "or" in this application indicates at least one of the connected objects. For example, the scope of protection for "A or B" covers at least three scenarios: Scenario 1: including A but not B; Scenario 2: including B but not A; Scenario 3: including both A and B. In addition, the terms "A and / or B," "at least one of A and B," and "at least one of A or B" also cover at least the above three scenarios. The character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0033] The term "instruction" in this application can be either a direct instruction (or explicit instruction) or an indirect instruction (or implicit instruction). A direct instruction can be understood as one in which the sender explicitly informs the receiver of specific information, the operation to be performed, or the requested result, etc., in the instruction sent. An indirect instruction can be understood as one in which the receiver determines the corresponding information based on the instruction sent by the sender, or makes a judgment and determines the operation to be performed or the requested result, etc., based on the judgment result.

[0034] For ease of understanding, the following explains some aspects of the embodiments of this application:

[0035] 1. Scene compositing or rendering process

[0036] The rendering engine can synthesize or render a scene based on a Graphics Language Transmission Format (glTF) file. By rendering the haptic scene, the rendering engine can obtain haptic feedback signals. These haptic signals are then encoded, transmitted, and decoded before being played by a haptic signal actuator.

[0037] The rendering engine obtains glTF files in several ways: the server and client interact by transmitting and exchanging glTF files through glTF state streams, or by obtaining glTF files through program files stored on the server and client.

[0038] 2. The glTF Moving Picture Experts Group (MPEG) haptic extension

[0039] The MPEG haptic extension provides a method for accessing haptic data. MPEG haptic includes a set of haptic objects. The accessor property in haptic objects then points to the haptic data source.

[0040] Table 1

[0041] Table 2

[0042] In Tables 1 and 2 above, array refers to an array and integer refers to an integer.

[0043] The specific access method for haptic data is shown in Figure 1. In MPEG_haptic, hapticObjects points to the accessor, which in turn points to the circular buffer object. The circular buffer object provides the data location information for accessing the circular buffer, and on the other hand, it reflects the attribute information of the haptic data rendering by referencing the MPEG_media object.

[0044] 3. MPEG_haptic_material extension in glTF

[0045] The MPEG_haptic_material extension provides tactile properties for materials. MPEG_haptic_material includes the materials attribute, as shown in Table 3 below.

[0046] Table 3

[0047] The `materials` property points to a set of material objects. Each material object, through the `haptic` property, points to the `hapticObject` in `MPEG_haptic` which provides the data source, and indicates the relevant material tactile properties (such as stiffness, friction, etc.) and texture coordinates, as shown in Table 4 below.

[0048] Table 4

[0049] 4. Interactivity extension in glTF

[0050] glTF is used for the efficient transfer and loading of 3D scenes and models by engines and applications. glTF defines a scalable publishing format that simplifies the creation workflow and interactive services by enabling interoperability of 3D content across the industry.

[0051] MPEG has extended glTF with interactive features and has initially designed relevant standards for interactivity. For haptic scenarios, the relevant standards for interactivity are mainly used to describe the interaction between the avatar and the haptic source, based on which haptic signals are triggered and generated.

[0052] Interactivity is defined based on behavior, which includes a trigger object and an action object. The trigger object includes collision conditions, proximity conditions, and visibility conditions. The action object includes haptic feedback actions (ACTION_SET_HAPTIC), which are implemented based on haptic action node objects or haptic action media objects.

[0053] The hapticObject property in the HapticActionNode points to the hapticObject in MPEG_haptic, and the mediaIndex property in the HapticActionMedia points to the accessor in the aforementioned hapticObject, thereby accessing haptic data. Please refer to Tables 5 and 6 below.

[0054] Table 5

[0055] Table 6

[0056] It should be noted that the Object mentioned in the embodiments of this application can be a node, mesh, primitive, etc.

[0057] 5. MPEG PCM haptic encoding and decoding scheme

[0058] MPEG Pulse-code modulation (PCM) haptic coding consists of high-frequency signal coding (wavelet band) and low-frequency signal coding (curve band).

[0059] In low-frequency signal encoding, the current signal is only encoded when the signal strength difference between the current signal and the most recently encoded signal exceeds a certain threshold. This encoding method is based on Weber's Law. Weber's Law is an important principle in psychology that elucidates the subjective perception mechanism of human differences in stimulus intensity. Specifically, only when the relative difference between two consecutive stimuli exceeds this threshold (such as the Just Noticeable Difference, JND) will the difference be recognized by the human perceptual system, thus triggering the necessity of encoding.

[0060] During the low-frequency signal decoding process, the decoder interpolates between two adjacent coding points to obtain the decoded low-frequency signal.

[0061] The tactile signal processing method, apparatus, and device provided in this application will be described in detail below with reference to the accompanying drawings and through some embodiments and application scenarios.

[0062] Referring to Figure 2, which is a flowchart of a tactile signal processing method provided in an embodiment of this application, the tactile signal processing method includes the following steps:

[0063] Step 101: The first device acquires tactile signal processing related information, which includes at least one of the following: first information, second information, and third information. The first information is used to characterize the spatial distribution characteristics of the signal intensity of the target tactile signal, the second information is the perception-related information of the target tactile signal, and the third information is used to indicate the triggering condition, which is the triggering condition of the triggering action object.

[0064] Step 102: The first device performs tactile signal processing based on the tactile signal processing related information.

[0065] The target tactile signal may be a temperature signal, a vibration signal, etc. This embodiment does not limit the target tactile signal.

[0066] The first information can be used to describe the propagation characteristics of the target tactile signal.

[0067] The first information may include: at least one spatial boundary; or signal strength information at at least one spatial boundary; or at least one target parameter; or signal strength information corresponding to at least one target parameter; or at least one spatial location; etc. This embodiment does not limit the specific implementation of the first information. The target parameter is used to characterize the spatial distance between the target location and the signal source.

[0068] The second information may include: the signal strength or signal strength range of the target tactile signal; or, the perception threshold of the target tactile signal, wherein the perception threshold is used to characterize the minimum perceptible change value of the target tactile signal; etc. This embodiment does not limit the specific implementation of the second information.

[0069] The triggering condition is the triggering condition for the action object, and the action object may include action objects related to tactile signal processing.

[0070] In one implementation, relevant triggering conditions can be defined for entering / exiting a spatial boundary, and the first device can trigger signal feedback based on entering / exiting the spatial boundary. Taking the tactile signal as a temperature signal as an example, the temperature inside a spatial boundary changes relative to the temperature outside the spatial boundary. Entering the space can trigger temperature signal feedback.

[0071] In one implementation, the third information may include an identification attribute of a spatial boundary, which may be indicated by a mesh, line, surface, point, or primitive. This identification attribute may be used to identify "entry" or "exit," or it may be used as a trigger condition to indicate entering or exiting the spatial boundary.

[0072] For example, the third information may be carried in the triggering object.

[0073] In one embodiment, the first information and the third information may be related, and the third information may be used to indicate the triggering condition corresponding to the spatial boundary in the first information.

[0074] In one embodiment, the first device performs tactile signal processing based on the tactile signal processing related information, which may include: the first device performing tactile rendering based on the tactile signal processing related information.

[0075] In this embodiment, the execution end (i.e., the first device) can be a rendering engine, or it can be a rendering device, a scene compositing device, a scene compositing engine, a rendering engine, or a rendering device. Additionally, the glTF file can be recognized by the rendering engine or stored by a storage device.

[0076] In this embodiment, a first device acquires tactile signal processing related information, which includes at least one of the following: first information, second information, and third information. The first information characterizes the spatial distribution characteristics of the signal intensity of the target tactile signal, the second information is perception-related information of the target tactile signal, and the third information indicates a triggering condition, which is a triggering condition for a triggering action object. The first device performs tactile signal processing based on the tactile signal processing related information. Thus, by utilizing at least one of the spatial distribution characteristics of the signal intensity of the target tactile signal, the perception-related information of the target tactile signal, and the triggering condition for a triggering action object, the descriptive function of the tactile scene is enhanced. Using this enhanced descriptive function to assist tactile signal processing can improve the realism of the rendered tactile scene.

[0077] Optionally, the first information is the glTF attribute information in the glTF file of the graphics language transmission format;

[0078] And / or,

[0079] The second piece of information is the glTF attribute information in the glTF file;

[0080] And / or,

[0081] The third piece of information is the glTF attribute information in the glTF file.

[0082] In this embodiment, the spatial distribution characteristics of the signal intensity of the target tactile signal are indicated by the glTF attribute information in the glTF file, or the perception-related information of the target tactile signal is indicated by the glTF attribute information in the glTF file, or the triggering conditions are indicated by the glTF attribute information in the glTF file. This can enhance the descriptive function of the tactile scene and thus improve the realism of the rendered tactile scene.

[0083] Optionally, the first information includes at least one of the following:

[0084] At least one spatial boundary;

[0085] Signal strength information at at least one spatial boundary;

[0086] At least one target parameter;

[0087] Signal strength information corresponding to at least one target parameter;

[0088] The target parameter is used to characterize the spatial distance between the target location and the signal source.

[0089] The spatial boundaries can be indicated by a mesh, line, surface, point, or primitive.

[0090] For example, spatial boundaries can define an enclosed space.

[0091] The signal strength information can be the absolute value of the signal strength, the relative value of the signal strength, or a range of signal strength values, etc. The relative value of the signal strength can refer to the difference in signal strength relative to the signal source, the difference in signal strength relative to the nearby spatial boundary, or the difference in signal strength relative to a reference value, etc.

[0092] The signal source can be a signal source for the target tactile signal.

[0093] It should be noted that, for the signal source of the target tactile signal, the spatial limit or the target parameter used to characterize the spatial distance can be one. The spatial limit or target parameter can define the effective area of ​​the signal source, and the signal source signal can be limited within or outside the spatial limit or spatial distance.

[0094] Alternatively, the effective area of ​​the signal source can be defined as within the spatial boundary or spatial distance indicated by the first information, or the effective area of ​​the signal source can be defined as outside the spatial boundary or spatial distance indicated by the first information.

[0095] Optionally, the method further includes at least one of the following:

[0096] The first device determines the signal strength information of the target tactile signal at a first position based on the signal strength information of the target tactile signal at a first spatial boundary and the signal strength information of the target tactile signal at a second spatial boundary. The first position is the position between the first spatial boundary and the second spatial boundary. The at least one spatial boundary includes the first spatial boundary and the second spatial boundary, and the first spatial boundary is adjacent to the second spatial boundary.

[0097] The first device determines the signal strength information of the target tactile signal at a second position based on the signal strength information of the target tactile signal at a first spatial distance and the signal strength information at a second spatial distance. The second position is the position between the first spatial distance and the second spatial distance, and the first spatial distance and the second spatial distance are adjacent spatial distances determined based on the at least one target parameter.

[0098] The step of determining the signal strength information of the target tactile signal at a first position based on the signal strength information of the target tactile signal at a first spatial boundary and the signal strength information of the second spatial boundary may include: using the signal strength information of the spatial boundary closer to the first position between the first and second spatial boundaries as the signal strength information at the first position; or using the signal strength information of the spatial boundary farther from the first position between the first and second spatial boundaries as the signal strength information at the first position; or constructing a first-order or n-order function with the spatial boundaries as independent variables based on the signal strength information of the target tactile signal at the first and second spatial boundaries, and determining the signal strength information of the target tactile signal at the first position based on the constructed function; etc., this embodiment does not limit this step.

[0099] Optionally, the first information can also be used to indicate function-related information of the first-order or nth-order function with spatial limits as independent variables, such as the coefficients or type of the function.

[0100] In this embodiment, the first device determines the signal strength information of the target tactile signal at a first position based on the signal strength information of the target tactile signal at the first spatial boundary and the signal strength information of the second spatial boundary, thereby enabling the determination of the signal strength information of the position between two adjacent spatial boundaries.

[0101] The step of determining the signal strength information of the target tactile signal at the second position based on the signal strength information of the target tactile signal at the first spatial distance and the signal strength information of the second spatial distance can include: using the signal strength information of the spatial distance closer to the second position between the first and second spatial distances as the signal strength information at the second position; or using the signal strength information of the spatial distance farther from the second position between the first and second spatial distances as the signal strength information at the second position; or constructing a first-order or n-order function with spatial distance as the independent variable based on the signal strength information of the target tactile signal at the first and second spatial distances, and determining the signal strength information of the target tactile signal at the second position based on the constructed function; etc., this embodiment does not limit this step.

[0102] In addition, the first information can also be used to indicate function-related information of the first-order or nth-order function with spatial distance as the independent variable, such as the coefficients of the function or the function type.

[0103] In this embodiment, the first device determines the signal strength information of the target tactile signal at a second position based on the signal strength information of the target tactile signal at a first spatial distance and the signal strength information at a second spatial distance, thereby enabling the determination of the signal strength information of the position between adjacent spatial distances.

[0104] Optionally, the first information is carried by at least one of the following:

[0105] glTF objects associated with interactive triggers; glTF objects associated with haptic motion nodes; glTF objects associated with haptic sources;

[0106] And / or,

[0107] The second information is carried by at least one of the following:

[0108] glTF objects associated with haptic motion nodes; glTF objects associated with digital avatars; glTF objects associated with haptic signal actuators;

[0109] And / or,

[0110] The third information is carried through a glTF object associated with the interactive trigger.

[0111] For example, first information can be added to the glTF object of the haptic action node, or to the glTF object of the haptic source node.

[0112] Optionally, the first information is node-level glTF attribute information; or, the first information is mesh-level glTF attribute information; or, the first information is scene-level glTF attribute information.

[0113] And / or,

[0114] The second information is node-level glTF attribute information; or, the second information is mesh-level glTF attribute information; or, the second information is scene-level glTF attribute information.

[0115] And / or,

[0116] The third information is node-level glTF attribute information; or, the third information is mesh-level glTF attribute information; or, the third information is scene-level glTF attribute information.

[0117] The first information can be node-level glTF attribute information, and the spatial distribution characteristics of the target tactile signal intensity can be obtained by parsing the node; or, the first information can be grid-level glTF attribute information, and the spatial distribution characteristics of the target tactile signal intensity can be obtained by parsing the grid; or, the first information can be scene-level glTF attribute information, and the spatial distribution characteristics of the target tactile signal intensity can be obtained by parsing the scene.

[0118] The second information can be node-level glTF attribute information, and the perception-related information of the target tactile signal can be obtained by parsing the node; or, the second information can be grid-level glTF attribute information, and the perception-related information of the target tactile signal can be obtained by parsing the grid; or, the second information can be scene-level glTF attribute information, and the perception-related information of the target tactile signal can be obtained by parsing the scene.

[0119] The third information can be node-level glTF attribute information, and the triggering condition can be obtained by parsing the node; or, the third information can be mesh-level glTF attribute information, and the triggering condition can be obtained by parsing the mesh; or, the third information can be scene-level glTF attribute information, and the triggering condition can be obtained by parsing the scene.

[0120] Optionally, the second information includes at least one of the following:

[0121] The signal strength or signal strength range of the target tactile signal;

[0122] The perception threshold of the target tactile signal, which is used to characterize the minimum change value of the perceptible target tactile signal.

[0123] The perception threshold can also be described as the perception inertia threshold.

[0124] The signal strength or signal strength range in the second information may be correlated with the sensing threshold. Different sensing thresholds can be set for different signal strengths or signal strength ranges; or, only one sensing threshold can be set.

[0125] For example, the signal strength or signal strength range of the target tactile signal can be a first signal strength or a first signal strength range; the perception threshold of the target tactile signal can be the perception threshold corresponding to the first signal strength or the first signal strength range.

[0126] In one implementation, the second information may include:

[0127] A first signal strength or a first signal strength range, for example, a first temperature or a first temperature range T0;

[0128] The sensing threshold corresponding to the first signal strength or the first signal strength range, for example, the temperature sensing threshold p(T0) corresponding to the first temperature or the first temperature range.

[0129] It should be noted that multiple sets of (T0, p(T0)) can exist. For example, regarding the finger's temperature perception, when the temperature T0 is between [30, 34], the finger's sensory system has high sensitivity, p(T0) = 0.5; when the temperature T0 is between [35, 41], the finger's sensory system has moderate sensitivity, p(T0) = 0.7; and when the temperature T0 is between [23, 29], the finger's sensory system has moderate sensitivity, p(T0) = 0.8.

[0130] Optionally, the method further includes

[0131] The first device acquires a first value of the target tactile signal;

[0132] Wherein, if the difference between the first value and the reference value of the target tactile signal is less than the perception threshold, the first device does not encode the first value during the tactile signal processing.

[0133] and / or

[0134] If the difference between the first value and the reference value of the target tactile signal is greater than or equal to the perception threshold, the first device encodes the first value during the tactile signal processing.

[0135] Optionally, in some embodiments, when the difference between the first value and the reference value of the target tactile signal is less than or equal to the perception threshold, the first device does not encode the first value during the tactile signal processing.

[0136] and / or

[0137] If the difference between the first value and the reference value of the target tactile signal is greater than the perception threshold, the first device encodes the first value during the tactile signal processing.

[0138] The first value of the target tactile signal can be the value of the target tactile signal currently captured in real time. The reference value of the target tactile signal can be a verified reference value of the target tactile signal that has been adopted as a benchmark. The perception threshold can be determined based on the reference value, and a perception threshold can be uniquely determined by the reference value. For example, taking the second information including the perception threshold corresponding to the first signal strength or the range of the first signal strength as an example, the second information can be used to determine the range of the first signal strength in which the reference value is located, or the first signal strength closest to the reference value, and thus determine the corresponding perception threshold.

[0139] In this embodiment, by comparing the difference between the first value of the target tactile signal and the reference value of the target tactile signal with a perception threshold to determine whether to encode the first value, the amount of encoded data can be reduced as much as possible while ensuring that the signal is not distorted, thereby improving encoding efficiency.

[0140] Optionally, the second information is associated with the position of the tactile signal actuator of the target tactile signal, and / or the second information is associated with the position where the target tactile signal is applied.

[0141] The method of indicating the position of the tactile signal actuator or the position of action of the target tactile signal may include at least one of the following:

[0142] The position of the haptic signal actuator or the location of the target haptic signal is indicated by specifying glTF basic objects such as Node, Mesh, or Primitive. The position of the glTF basic object is related to the position of the haptic signal actuator or the location of the signal.

[0143] The position of the tactile signal actuator or the location where the target tactile signal is applied is indicated by indicating coordinate information. For example, it can indicate the relative coordinates within the node's coordinate system or the absolute coordinates relative to the world coordinate system.

[0144] This directly indicates actuator-related information or body part-related information. For example, it indicates actuator ID, actuator target, body part target, body part mask, or mesh vertices, etc.

[0145] In this embodiment, the second information is associated with the position of the tactile signal actuator of the target tactile signal, so that the perception-related information of the corresponding tactile signal can be determined according to the tactile signal actuator at different positions.

[0146] In this embodiment, the second information is associated with the position of the target tactile signal, so that the perception-related information of the corresponding tactile signal can be determined according to the different positions of the tactile signal.

[0147] It should be noted that while the current glTF standard supports basic haptic rendering, it is still incomplete. The propagation characteristics of haptic signals cannot yet be described by the glTF standard (e.g., the propagation characteristics of temperature), which can cause the generated haptic signals to fail to reflect the realistic sensations in the scene. This application's embodiments enhance the descriptive capabilities of glTF by designing first information. Furthermore, the glTF standard does not provide auxiliary encoding information for the encoding process of the generated haptic signals, and the haptic encoding process may still cause signal distortion. Factors such as the scene, haptic type, or actuator model can all affect the encoding method, thus impacting the user experience. This application's embodiments enhance the descriptive capabilities of glTF by designing second information. For example, for the low-frequency encoding process of PCM haptic encoding, the encoder encodes based on the encoding threshold in the second information, ensuring that the encoded signal is not distorted. This encoding threshold is related to factors such as haptic type, scene, and / or actuator type.

[0148] This application provides a description function for tactile scenes, providing more comprehensive auxiliary information for the generation and encoding of tactile signals.

[0149] The following examples will provide further explanation:

[0150] Example 1: Design glTF attribute information (such as primary information) to describe the propagation characteristics of tactile signals.

[0151] In one implementation, the spatial distribution characteristics of the signal strength associated with the signal source can be indicated by the corresponding glTF attribute information, and the rendering engine performs signal processing / rendering accordingly. The indication methods for the spatial distribution characteristics of the signal strength include at least one of the following:

[0152] Option 1: As shown in Figure 3, for a certain signal source, the relevant glTF attribute information (such as the first information) indicates (multiple) spatial boundaries, and / or, the signal strength related information corresponding to the spatial boundaries.

[0153] Optionally, spatial boundaries can be indicated by mesh, lines, surfaces, points, primitives, etc.

[0154] For example, spatial boundaries can define an enclosed space.

[0155] Optionally, signal strength-related information can be the absolute value of signal strength, the relative value of signal strength, the range of signal strength, etc. The relative value of signal strength can refer to the difference in signal strength relative to the signal source, the difference in signal strength relative to the nearby spatial boundary, or the difference in signal strength relative to a reference value, etc.

[0156] Option 2: As shown in Figure 4, for a certain signal source, the relevant glTF attribute information (such as the first information) indicates (multiple) distances (such as extents) from the signal source, and / or, the signal strength information corresponding to that distance.

[0157] Optionally, signal strength-related information can be the absolute value of signal strength, the relative value of signal strength, the range of signal strength, etc. The relative value of signal strength can refer to the difference in signal strength relative to the signal source, the difference in signal strength relative to nearby distances, the difference in signal strength relative to a reference value, etc.

[0158] Optionally, for a certain signal source, the spatial boundary or distance can be one, which can define the effective area of ​​the signal source, and the signal of the signal source can be limited within or outside the spatial boundary or distance.

[0159] Optionally, the effective area of ​​the signal source is within the spatial boundary or distance, or the area outside the spatial boundary or distance can be indicated by the corresponding glTF attribute information (such as the first information).

[0160] Optionally, the signal strength determination method between adjacent spatial boundaries includes at least one of the following:

[0161] Option 1: Determine the signal strength at the spatial boundary closer to or farther from the signal source as the signal strength between the adjacent spatial boundaries.

[0162] Option 2: Based on the signal strength at adjacent spatial boundaries, construct a first-order or n-order function with the spatial boundaries as the independent variable, and determine the signal strength between the adjacent spatial boundaries based on the function.

[0163] Among them, function-related information such as function coefficients or function type can be indicated by the corresponding glTF attribute information (such as the first information).

[0164] Optionally, the method for determining the signal strength between adjacent spatial distances includes at least one of the following:

[0165] Option 1: Determine the signal strength at a spatial distance closer to or farther from the signal source as the signal strength between the adjacent spatial distances.

[0166] Option 2: Based on the signal strength at adjacent spatial distances, construct a first-order or n-order function with spatial distance as the independent variable, and determine the signal strength between the adjacent spatial distances based on the function.

[0167] Among them, function-related information such as function coefficients or function type can be indicated by the corresponding glTF attribute information (such as the first information).

[0168] It should be noted that the signal strength between the signal source and the first spatial boundary or the first spatial distance can also be derived using the above method.

[0169] Optionally, the relevant glTF attribute information (such as the first information) may be carried by a glTF object related to the interactivity trigger, or by a glTF object related to the haptic action node, or by a glTF object related to the haptic source.

[0170] For example, in the haptic action node, add information describing characteristics related to signal propagation.

[0171] For example, in the haptic source node, add information describing the characteristics related to signal propagation.

[0172] Optionally, the relevant glTF attribute information (such as the first information) can be node-level, mesh-level, or scene-level attribute information.

[0173] For example, at the scene level, you can define signal source nodes and / or nodes related to the signal source propagation characteristics. This allows the rendering engine to identify the signal source and its propagation characteristics by parsing the scene.

[0174] For example, at the node-level, characteristics related to signal propagation can be defined.

[0175] For example, at the node-level, signal source identifiers can be defined.

[0176] In one implementation, relevant triggering conditions are defined for entering / exiting a spatial boundary, and the rendering engine triggers signal feedback based on entering / exiting the spatial boundary. For example, if the temperature inside a spatial boundary changes relative to the temperature outside the spatial boundary, entering that space can trigger temperature signal feedback.

[0177] The triggering conditions can be indicated by relevant glTF attribute information (such as third information).

[0178] Optionally, the triggering object includes spatial boundary-related properties, which can be indicated by a mesh, line, face, point, or primitive.

[0179] Optionally, the triggering object includes an identification attribute to identify whether "entering" or "exiting" the space boundary is a triggering condition.

[0180] Optionally, the relevant glTF attribute information (such as third information) can be carried by the glTF object associated with the interactivity trigger.

[0181] For example, based on the interaction extension, a corresponding trigger object can be added. In this way, the rendering engine can reuse the existing processing model to determine and process new trigger conditions.

[0182] Optionally, the relevant glTF attribute information (such as third information) can be node-level, mesh-level, or scene-level attribute information.

[0183] For example, at the scene level, one or more types of nodes can be defined, one type of node is used to represent spatial boundaries, and / or, another type of node is used to represent objects entering / exiting spatial boundaries.

[0184] For example, at the scene-level, you can define "entry / exit" indicators.

[0185] For example, at the node-level, shapes representing spatial boundaries (such as mesh, or primitive, etc.) can be defined.

[0186] Example 2: Design glTF attribute information (such as the second information) as auxiliary information for haptic encoding.

[0187] In one implementation, glTF attribute information (such as second information) indicates tactile (e.g., temperature) perception-related information (i.e., second information), and the rendering engine processes / renders the tactile signal based on the tactile perception-related information.

[0188] The tactile perception-related information includes at least one of the following:

[0189] A first signal strength or a first signal strength range, for example, a first temperature or a first temperature range T0;

[0190] Threshold information (i.e., sensing threshold) corresponding to a first signal strength or a first signal strength range, for example, temperature sensing inertia threshold information p(T0) corresponding to a first temperature or a first temperature range.

[0191] It should be noted that multiple sets of (T0, p(T0)) can exist. For example, regarding the finger's temperature perception, when the temperature T0 is between [30, 34], the finger's sensory system has high sensitivity, p(T0) = 0.5; when the temperature T0 is between [35, 41], the finger's sensory system has moderate sensitivity, p(T0) = 0.7; and when the temperature T0 is between [23, 29], the finger's sensory system has moderate sensitivity, p(T0) = 0.8.

[0192] In addition, the method of signal processing based on tactile perception-related information includes the following steps, and for more specific encoding methods, please refer to Example 3:

[0193] T r This represents a previously validated and adopted tactile signal as a benchmark, often referred to as a "reference signal".

[0194] T c This refers to the tactile signals captured in real time.

[0195] In the process of tactile encoding, it is necessary to determine an appropriate tactile perception inertia threshold, denoted as p(T). r This is a dynamically changing signal that depends on the reference signal T. r The variable. p(T) r The accuracy of tactile coding determines whether tactile coding algorithms can effectively distinguish tactile changes that are perceptible to humans.

[0196] When the tactile encoding algorithm receives a new tactile signal T c When doing this, T will be calculated first. c With the current reference signal T r The absolute difference d between them;

[0197] If the difference d is less than the inertia threshold p(T) r If the difference d reaches or exceeds the perception threshold p(T), the tactile encoding algorithm will determine that the current tactile perception change is not significant enough, and therefore choose to ignore the tactile signal and not perform encoding processing; conversely, if the difference d reaches or exceeds the perception threshold p(T), the algorithm will not perform encoding processing. r If the current tactile perception change is considered to have significant perceptual meaning, then the current signal value T is then... c Encode it.

[0198] Optionally, the tactile perception-related information can be associated with actuator position information or signal action position information.

[0199] The indication method of the actuator position information or signal action position information includes at least one of the following:

[0200] The actuator position information or signal action position information is indicated by pointing to glTF basic objects such as Node, Mesh, or Primitive. The position corresponding to the glTF basic object is related to the actuator position information or signal action position.

[0201] The position information of the actuator or the position information of the signal action is indicated by indicating coordinate information. For example, it can indicate the relative coordinates within the coordinate system of the node, or the absolute coordinates relative to the world coordinate system;

[0202] It can directly indicate information related to the Actuator or body parts. For example, it can indicate the Actuator ID, Actuator target, body part target, body part mask, or mesh vertices, etc.

[0203] Optionally, the relevant glTF attribute information (such as the second information) may be carried by a glTF object related to a haptic action node, or by a glTF object related to an avatar, or by a glTF object related to an executor.

[0204] Optionally, the relevant glTF attribute information (such as the second information) can be node-level, mesh-level, or scene-level attribute information.

[0205] Example 3:

[0206] To provide users with an immersive temperature-tactile experience, enabling them to perceive subtle temperature changes, this example proposes a multi-level temperature-tactile coding method based on the temperature-sensing inertia region. The 0th-order, 1st-order, and 2nd-order coding methods will be introduced below.

[0207] The 0th-order temperature tactile coding method based on the temperature-sensing inert region is as follows:

[0208] When the difference d does not exceed the perception threshold p(T) r When the user terminal plays the tactile signal value T, r If the difference d reaches or exceeds the perception threshold p(T) r The user terminal plays the tactile signal value T. c The encoding method is shown in Figure 5, with the dashed line representing the 0th-order temperature tactile encoding method.

[0209] The first-order temperature tactile coding method based on the temperature-sensing inert region is as follows:

[0210] When the difference d reaches or exceeds the perception threshold p(T) rAt time t, the signal value is encoded using first-order encoding. Assume the current time is t. c Temperature signal value T c Reaching or exceeding the perception threshold p(T) r If T , then a and b are calculated using the following system of equations: c =a*t c +b T r =a*t r +b

[0211] Among them, T r t is the reference signal temperature value. r Let be the time corresponding to the reference signal temperature value. Then, for any time t∈[t... r ,t c The user plays the tactile temperature value a*t+b, and the encoding method is shown in Figure 6. The dashed line represents the first-order temperature tactile encoding method.

[0212] The second-order temperature tactile coding method based on the temperature-sensing inert region is as follows:

[0213] When the difference d reaches or exceeds the perception threshold p(T) r At time t, the signal value is encoded in two orders. Assume the current time is t. c Temperature signal value T c Reaching or exceeding the perception threshold p(T) r If ), then a, b, and h are calculated using the following system of equations: T c =a*(t) c ) 2 +bt c +h T r =a*(t) r ) 2 +bt r +h T m =a*(t) m ) 2 +bt m +h

[0214] Among them, t m For [t] r ,t c The midpoint of the time interval, i.e. T m Let time t m The corresponding temperature value. Therefore, for any time t∈[t... r ,t c The user terminal plays the tactile temperature value a*(t). 2 +bt+h, the encoding method is shown in Figure 7, with the dashed line representing the second-order temperature tactile encoding method. Time t3, time t6, and t... 11The time intervals are [t1,t5], [t5,t8], and [t8,t5], respectively. 14 The intermediate time t m .

[0215] In this embodiment, the propagation characteristics of tactile signals are described by glTF attribute information; in addition, auxiliary information is provided by glTF attribute information to assist MPEG tactile encoding; thereby improving the realism of tactile scene rendering and improving the distortion problem of tactile signal encoding and decoding.

[0216] The tactile signal processing method provided in this application can be executed by a tactile signal processing device. This application uses the execution of the tactile signal processing method by a tactile signal processing device as an example to illustrate the tactile signal processing device provided in this application.

[0217] This application provides a tactile signal processing device. As an example, the tactile signal processing device may be a communication device or a component within a communication device, such as a chip. The communication device may be a terminal, a network-side device, or a server, etc.

[0218] The tactile signal processing device includes a receiving module, a transmitting module, and a processing module. These modules can be implemented in software or hardware. When implemented in hardware, the processing module can be implemented by a processor. For example, the processor can include general-purpose processors, special-purpose processors, such as a Central Processing Unit (CPU), microprocessor, Digital Signal Processor (DSP), Artificial Intelligence (AI) processor, Graphics Processing Unit (GPU), Application Specific Integrated Circuit (ASIC), Network Processor (NP), Field Programmable Gate Array (FPGA), or other programmable logic devices, gate circuits, transistors, discrete hardware components, etc. The receiving and transmitting modules can be implemented by a communication interface, which can include one or more of the following: transceiver, pins, circuits, bus, radio frequency unit, etc.

[0219] Specifically, referring to Figure 8, when the tactile signal processing device is the first device or a component of the first device, the tactile signal processing device 200 includes:

[0220] Processing module 201 is used to acquire tactile signal processing related information, the tactile signal processing related information including at least one of the following: first information, second information, and third information, the first information is used to characterize the spatial distribution characteristics of the signal intensity of the target tactile signal, the second information is the perception related information of the target tactile signal, and the third information is used to indicate the triggering condition, the triggering condition being the triggering condition of the triggering action object;

[0221] The processing module 201 is also used to perform tactile signal processing based on the tactile signal processing related information.

[0222] Optionally, the first information is the glTF attribute information in the glTF file of the graphics language transmission format;

[0223] And / or,

[0224] The second piece of information is the glTF attribute information in the glTF file;

[0225] And / or,

[0226] The third piece of information is the glTF attribute information in the glTF file.

[0227] Optionally, the first information includes at least one of the following:

[0228] At least one spatial boundary;

[0229] Signal strength information at at least one spatial boundary;

[0230] At least one target parameter;

[0231] Signal strength information corresponding to at least one target parameter;

[0232] The target parameter is used to characterize the spatial distance between the target location and the signal source.

[0233] Optionally, the processing module 201 is further configured to perform at least one of the following:

[0234] Based on the signal strength information of the target tactile signal at the first spatial boundary and the signal strength information of the second spatial boundary, the signal strength information of the target tactile signal at the first position is determined. The first position is the position between the first spatial boundary and the second spatial boundary. The at least one spatial boundary includes the first spatial boundary and the second spatial boundary, and the first spatial boundary is adjacent to the second spatial boundary.

[0235] Based on the signal strength information of the target tactile signal at a first spatial distance and the signal strength information at a second spatial distance, the signal strength information of the target tactile signal at a second position is determined. The second position is the position between the first spatial distance and the second spatial distance, and the first spatial distance and the second spatial distance are adjacent spatial distances determined based on the at least one target parameter.

[0236] Optionally, the first information is carried by at least one of the following:

[0237] glTF objects associated with interactive triggers; glTF objects associated with haptic motion nodes; glTF objects associated with haptic sources;

[0238] And / or,

[0239] The second information is carried by at least one of the following:

[0240] glTF objects associated with haptic motion nodes; glTF objects associated with digital avatars; glTF objects associated with haptic signal actuators;

[0241] And / or,

[0242] The third information is carried through a glTF object associated with the interactive trigger.

[0243] Optionally, the first information is node-level glTF attribute information; or, the first information is mesh-level glTF attribute information; or, the first information is scene-level glTF attribute information.

[0244] And / or,

[0245] The second information is node-level glTF attribute information; or, the second information is mesh-level glTF attribute information; or, the second information is scene-level glTF attribute information.

[0246] And / or,

[0247] The third information is node-level glTF attribute information; or, the third information is mesh-level glTF attribute information; or, the third information is scene-level glTF attribute information.

[0248] Optionally, the second information includes at least one of the following:

[0249] The signal strength or signal strength range of the target tactile signal;

[0250] The perception threshold of the target tactile signal, which is used to characterize the minimum change value of the perceptible target tactile signal.

[0251] Optionally, the processing module 201 is further configured to:

[0252] Obtain the first value of the target tactile signal;

[0253] Wherein, if the difference between the first value and the reference value of the target tactile signal is less than the perception threshold, the first value is not encoded during the tactile signal processing.

[0254] and / or

[0255] If the difference between the first value and the reference value of the target tactile signal is greater than or equal to the perception threshold, the first value is encoded during the tactile signal processing.

[0256] Optionally, the second information is associated with the position of the tactile signal actuator of the target tactile signal, and / or the second information is associated with the position where the target tactile signal is applied.

[0257] The tactile signal processing device provided in this application embodiment can implement the various processes implemented in the method embodiment of FIG2 and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0258] As shown in Figure 9, this application embodiment also provides an electronic device 300, including a processor 301 and a memory 302. The memory 302 stores a program or instructions that can run on the processor 301. When the program or instructions are executed by the processor 301, they implement the various steps of the above-described tactile signal processing method embodiment and achieve the same technical effect.

[0259] This application also provides an electronic device, including a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps in the method embodiment shown in FIG2. This device embodiment corresponds to the above method embodiment, and all implementation processes and methods of the above method embodiments can be applied to this terminal embodiment and can achieve the same technical effect.

[0260] The processor or processing circuit in this application embodiment may include general-purpose processors, special-purpose processors, etc., such as central processing units (CPUs), microprocessors, digital signal processors (DSPs), artificial intelligence (AI) processors, graphics processing units (GPUs), application-specific integrated circuits (ASICs), network processors (NPs), field-programmable gate arrays (FPGAs), or other programmable logic devices, gate circuits, transistors, discrete hardware components, etc. The communication interface in this application embodiment may include transceivers, pins, circuits, buses, etc.

[0261] The aforementioned electronic devices can be terminals or other devices besides terminals, such as servers, network attached storage (NAS), etc.

[0262] The terminal can be a mobile phone, tablet computer, laptop computer, notebook computer, personal digital assistant (PDA), handheld computer, netbook, ultra-mobile personal computer (UMPC), mobile internet device (MID), augmented reality (AR), virtual reality (VR) device, mixed reality (MR) device, robot, wearable device, flight vehicle, vehicle user equipment (VUE), shipboard equipment, pedestrian user equipment (PUE), smart home (home devices with wireless communication capabilities, such as refrigerators, televisions, washing machines, or furniture), game console, personal computer (PC), ATM or self-service machine, etc. Wearable devices include: smartwatches, smart bracelets, smart earphones, smart glasses, smart jewelry (smart bracelets, smart chains, smart rings, smart necklaces, smart anklets, smart anklets, etc.), smart wristbands, smart clothing, etc. Among these, in-vehicle devices can also be referred to as in-vehicle terminals, in-vehicle controllers, in-vehicle modules, in-vehicle components, in-vehicle chips, or in-vehicle units, etc. It should be noted that the embodiments in this application do not limit the specific type of terminal.

[0263] A server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server. A cloud server can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), or cloud computing services based on big data and artificial intelligence platforms.

[0264] Taking an electronic device as an example, Figure 10 is a schematic diagram of the hardware structure of a terminal implementing an embodiment of this application.

[0265] The terminal 400 includes, but is not limited to, at least some of the following components: radio frequency unit 401, network module 402, audio output unit 403, input unit 404, sensor 405, display unit 406, user input unit 407, interface unit 408, memory 409, and processor 410.

[0266] Those skilled in the art will understand that terminal 400 may also include a power supply (such as a battery) for powering various components. The power supply can be logically connected to processor 410 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The terminal structure shown in Figure 10 does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0267] It should be understood that, in this embodiment, the input unit 404 may include a graphics processor 4041 and a microphone 4042. The graphics processor 4041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 406 may include a display panel 4061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 407 includes at least one of a touch panel 4071 and other input devices 4072. The touch panel 4071 is also called a touch screen. The touch panel 4071 may include a touch detection device and a touch controller. Other input devices 4072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0268] In this embodiment, after receiving downlink data from the network-side device, the radio frequency unit 401 can transmit it to the processor 410 for processing; in addition, the radio frequency unit 401 can send uplink data to the network-side device. Typically, the radio frequency unit 401 includes, but is not limited to, antennas, amplifiers, transceivers, couplers, low-noise amplifiers, duplexers, etc.

[0269] The memory 409 can be used to store software programs or instructions, as well as various data. The memory 409 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 409 may include volatile memory or non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 409 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0270] Processor 410 may include one or more processing units; optionally, processor 410 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 410.

[0271] The processor 410 is used to acquire tactile signal processing related information, which includes at least one of the following: first information, second information, and third information. The first information is used to characterize the spatial distribution characteristics of the signal intensity of the target tactile signal, the second information is the perception-related information of the target tactile signal, and the third information is used to indicate the triggering condition, which is the triggering condition of the triggering action object.

[0272] The processor 410 is also configured to perform tactile signal processing based on the tactile signal processing related information.

[0273] Optionally, the first information is the glTF attribute information in the glTF file of the graphics language transmission format;

[0274] And / or,

[0275] The second piece of information is the glTF attribute information in the glTF file;

[0276] And / or,

[0277] The third piece of information is the glTF attribute information in the glTF file.

[0278] Optionally, the first information includes at least one of the following:

[0279] At least one spatial boundary;

[0280] Signal strength information at at least one spatial boundary;

[0281] At least one target parameter;

[0282] Signal strength information corresponding to at least one target parameter;

[0283] The target parameter is used to characterize the spatial distance between the target location and the signal source.

[0284] Optionally, the processor 410 is further configured to perform at least one of the following:

[0285] Based on the signal strength information of the target tactile signal at the first spatial boundary and the signal strength information of the second spatial boundary, the signal strength information of the target tactile signal at the first position is determined. The first position is the position between the first spatial boundary and the second spatial boundary. The at least one spatial boundary includes the first spatial boundary and the second spatial boundary, and the first spatial boundary is adjacent to the second spatial boundary.

[0286] Based on the signal strength information of the target tactile signal at a first spatial distance and the signal strength information at a second spatial distance, the signal strength information of the target tactile signal at a second position is determined. The second position is the position between the first spatial distance and the second spatial distance, and the first spatial distance and the second spatial distance are adjacent spatial distances determined based on the at least one target parameter.

[0287] Optionally, the first information is carried by at least one of the following:

[0288] glTF objects associated with interactive triggers; glTF objects associated with haptic motion nodes; glTF objects associated with haptic sources;

[0289] And / or,

[0290] The second information is carried by at least one of the following:

[0291] glTF objects associated with haptic motion nodes; glTF objects associated with digital avatars; glTF objects associated with haptic signal actuators;

[0292] And / or,

[0293] The third information is carried through a glTF object associated with the interactive trigger.

[0294] Optionally, the first information is node-level glTF attribute information; or, the first information is mesh-level glTF attribute information; or, the first information is scene-level glTF attribute information.

[0295] And / or,

[0296] The second information is node-level glTF attribute information; or, the second information is mesh-level glTF attribute information; or, the second information is scene-level glTF attribute information.

[0297] And / or,

[0298] The third information is node-level glTF attribute information; or, the third information is mesh-level glTF attribute information; or, the third information is scene-level glTF attribute information.

[0299] Optionally, the second information includes at least one of the following:

[0300] The signal strength or signal strength range of the target tactile signal;

[0301] The perception threshold of the target tactile signal, which is used to characterize the minimum change value of the perceptible target tactile signal.

[0302] Optionally, the processor 410 is further configured to:

[0303] Obtain the first value of the target tactile signal;

[0304] Wherein, if the difference between the first value and the reference value of the target tactile signal is less than the perception threshold, the first value is not encoded during the tactile signal processing.

[0305] and / or

[0306] If the difference between the first value and the reference value of the target tactile signal is greater than or equal to the perception threshold, the first value is encoded during the tactile signal processing.

[0307] Optionally, the second information is associated with the position of the tactile signal actuator of the target tactile signal, and / or the second information is associated with the position where the target tactile signal is applied.

[0308] It is understood that the implementation process of each implementation method mentioned in this embodiment can refer to the relevant description in Figure 2 of the method embodiment and achieve the same or corresponding technical effects. To avoid repetition, it will not be described again here.

[0309] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described tactile signal processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0310] The processor mentioned above is the processor in the terminal described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk. In some examples, the readable storage medium may be a non-transient readable storage medium.

[0311] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described tactile signal processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0312] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0313] This application also provides a computer program / program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described tactile signal processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0314] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0315] From the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of computer software products plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes several instructions to cause the terminal or network-side device to execute the methods described in the various embodiments of this application.

[0316] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other implementations under the guidance of this application without departing from the spirit and scope of the claims. All of these implementations are within the protection scope of this application.

Claims

1. A tactile signal processing method, comprising: The first device acquires tactile signal processing related information, which includes at least one of the following: first information, second information, and third information. The first information is used to characterize the spatial distribution characteristics of the signal intensity of the target tactile signal, the second information is the perception-related information of the target tactile signal, and the third information is used to indicate the triggering condition, which is the triggering condition of the triggering action object. The first device performs tactile signal processing based on the tactile signal processing related information.

2. The method according to claim 1, wherein, The first information is the glTF attribute information in the glTF file of the graphics language transmission format; And / or, The second piece of information is the glTF attribute information in the glTF file; And / or, The third piece of information is the glTF attribute information in the glTF file.

3. The method according to claim 1 or 2, wherein, The first information includes at least one of the following: At least one spatial boundary; Signal strength information at at least one spatial boundary; At least one target parameter; Signal strength information corresponding to at least one target parameter; The target parameter is used to characterize the spatial distance between the target location and the signal source.

4. The method according to claim 3, wherein, The method further includes at least one of the following: The first device determines the signal strength information of the target tactile signal at a first position based on the signal strength information of the target tactile signal at a first spatial boundary and the signal strength information of the target tactile signal at a second spatial boundary. The first position is the position between the first spatial boundary and the second spatial boundary. The at least one spatial boundary includes the first spatial boundary and the second spatial boundary, and the first spatial boundary is adjacent to the second spatial boundary. The first device determines the signal strength information of the target tactile signal at a second position based on the signal strength information of the target tactile signal at a first spatial distance and the signal strength information at a second spatial distance. The second position is the position between the first spatial distance and the second spatial distance, and the first spatial distance and the second spatial distance are adjacent spatial distances determined based on the at least one target parameter.

5. The method according to any one of claims 1-4, wherein, The first information is carried by at least one of the following: glTF objects associated with interactive triggers; glTF objects associated with haptic motion nodes; glTF objects associated with haptic sources; And / or, The second information is carried by at least one of the following: glTF objects associated with haptic motion nodes; glTF objects associated with digital avatars; glTF objects associated with haptic signal actuators; And / or, The third information is carried through a glTF object associated with the interactive trigger.

6. The method according to any one of claims 1-5, wherein, The first information is node-level glTF attribute information; or, the first information is mesh-level glTF attribute information; or, the first information is scene-level glTF attribute information. And / or, The second information is node-level glTF attribute information; or, the second information is mesh-level glTF attribute information; or, the second information is scene-level glTF attribute information. And / or, The third information is node-level glTF attribute information; or, the third information is mesh-level glTF attribute information; or, the third information is scene-level glTF attribute information.

7. The method according to any one of claims 1-6, wherein, The second information includes at least one of the following: The signal strength or signal strength range of the target tactile signal; The perception threshold of the target tactile signal, which is used to characterize the minimum change value of the perceptible target tactile signal.

8. The method according to claim 7, wherein, The method further includes: The first device acquires a first value of the target tactile signal; Wherein, if the difference between the first value and the reference value of the target tactile signal is less than the perception threshold, the first device does not encode the first value during the tactile signal processing. and / or If the difference between the first value and the reference value of the target tactile signal is greater than or equal to the perception threshold, the first device encodes the first value during the tactile signal processing.

9. The method according to any one of claims 1-8, wherein, The second information is associated with the position of the tactile signal actuator of the target tactile signal, and / or the second information is associated with the position where the target tactile signal is applied.

10. A tactile signal processing device, comprising: The processing module is used to acquire tactile signal processing related information, which includes at least one of the following: first information, second information, and third information. The first information is used to characterize the spatial distribution characteristics of the signal intensity of the target tactile signal, the second information is the perception-related information of the target tactile signal, and the third information is used to indicate the triggering condition, which is the triggering condition of the triggering action object. The processing module is also used to perform tactile signal processing based on the tactile signal processing related information.

11. The apparatus according to claim 10, wherein, The first information is the glTF attribute information in the glTF file of the graphics language transmission format; And / or, The second piece of information is the glTF attribute information in the glTF file; And / or, The third piece of information is the glTF attribute information in the glTF file.

12. The apparatus according to claim 10 or 11, wherein, The first information includes at least one of the following: At least one spatial boundary; Signal strength information at at least one spatial boundary; At least one target parameter; Signal strength information corresponding to at least one target parameter; The target parameter is used to characterize the spatial distance between the target location and the signal source.

13. The apparatus according to claim 12, wherein, The processing module is also used for at least one of the following: Based on the signal strength information of the target tactile signal at the first spatial boundary and the signal strength information of the second spatial boundary, the signal strength information of the target tactile signal at the first position is determined. The first position is the position between the first spatial boundary and the second spatial boundary. The at least one spatial boundary includes the first spatial boundary and the second spatial boundary, and the first spatial boundary is adjacent to the second spatial boundary. Based on the signal strength information of the target tactile signal at a first spatial distance and the signal strength information at a second spatial distance, the signal strength information of the target tactile signal at a second position is determined. The second position is the position between the first spatial distance and the second spatial distance, and the first spatial distance and the second spatial distance are adjacent spatial distances determined based on the at least one target parameter.

14. The apparatus according to any one of claims 10-13, wherein, The second information includes at least one of the following: The signal strength or signal strength range of the target tactile signal; The perception threshold of the target tactile signal, which is used to characterize the minimum change value of the perceptible target tactile signal.

15. The apparatus according to claim 14, wherein, The processing module is also used for: Obtain the first value of the target tactile signal; Wherein, if the difference between the first value and the reference value of the target tactile signal is less than the perception threshold, the first value is not encoded during the tactile signal processing. and / or If the difference between the first value and the reference value of the target tactile signal is greater than or equal to the perception threshold, the first value is encoded during the tactile signal processing.

16. An electronic device comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the tactile signal processing method as claimed in any one of claims 1-9.

17. A readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the tactile signal processing method as claimed in any one of claims 1-9.

18. A computer program / program product that, when executed by at least one processor, implements the steps of the tactile signal processing method as claimed in any one of claims 1-9.

19. A chip comprising a processor and a communication interface, wherein, The communication interface is coupled to the processor, which is used to run programs or instructions to implement the steps of the tactile signal processing method as described in any one of claims 1-9.

20. A communication device configured to perform the steps of the tactile signal processing method as described in any one of claims 1-9.

Images