Haptic-signal interpolation method and apparatus, and electronic device and storage medium

Abstract

The present application belongs to the technical field of haptic-signal coding and decoding. Provided are a haptic-signal interpolation method and apparatus, and an electronic device and a storage medium. The haptic-signal interpolation method comprises: by means of decoding a bit stream of a haptic signal, determining the interpolation type of a first haptic event component in a first haptic event (S210); and on the basis of the interpolation type of the first haptic event component, performing signal value interpolation between a signal value of the first haptic event component and a signal value of a second haptic event component following the first haptic event component, and obtaining the haptic signal (S220). The haptic-signal interpolation method can represent different types of haptic signals more accurately and flexibly, and save on overheads for haptic-signal representation, thereby improving the user experience of haptic media consumers and reducing the bandwidth consumption of haptic media transmissions.

Description

Tactile signal interpolation method, device, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 11, 2023, with application number 2023116936006 and invention name “Tactile signal interpolation method, device, electronic device and storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of encoding and decoding of tactile signals, and more specifically, to a tactile signal interpolation method, device, electronic device, and storage medium. Background Art

[0003] Immersive media content is often presented through various wearable or interactive devices. Therefore, in addition to traditional visual and auditory presentations, immersive media also offers a new form of tactile presentation. Tactile presentation uses a tactile presentation mechanism that combines hardware and software to allow users to receive information through their bodies, providing an embedded physical sensation and conveying key information about the system they are using. For example, a phone can vibrate to alert the user of a received message; this vibration is a type of tactile presentation. Tactile presentation can enhance auditory and visual presentations, improving the user experience.

[0004] In the related art, tactile presentation can be achieved through the transmission of tactile signals. Specifically, the tactile signal can be defined as a tactile event, and the tactile event can be decomposed into multiple tactile event components to achieve the transmission of the tactile signal.

[0005] However, this way of defining tactile signals may result in poor flexibility and excessive overhead in tactile signal representation, leading to problems such as poor user experience for tactile media consumers and excessive bandwidth consumption in tactile media transmission.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a tactile signal interpolation method, apparatus, electronic device, and storage medium, which can more accurately and flexibly represent different types of tactile signals and save the overhead of tactile signal representation, thereby improving the user experience of tactile media consumers and reducing bandwidth consumption of tactile media transmission.

[0008] In a first aspect, an embodiment of the present application provides a tactile signal interpolation method, which is applied to a decapsulation device or a signal processing device. The method includes:

[0009] determining an interpolation type of a first haptic event component in a first haptic event by decoding a bit stream of a haptic signal;

[0010] Based on the interpolation type of the first tactile event component, signal value interpolation is performed between the signal value of the first tactile event component and the signal value of a second tactile event component to obtain the tactile signal, where the second tactile event component is a tactile event component before or after the first tactile event component.

[0011] In a second aspect, an embodiment of the present application provides a tactile signal interpolation method, which is applied to a packaging device or a signal processing device, and the method includes:

[0012] Acquire tactile signals;

[0013] Based on the tactile signal, a first interpolation type is determined, and the first interpolation type is used to perform signal value interpolation between the signal value of the first tactile event component and the signal value of the second tactile event component in the first tactile event, where the second tactile event component is a tactile event component before or after the first tactile event component.

[0014] In a third aspect, an embodiment of the present application provides a tactile signal interpolation device, which is applied to a decapsulation device or a signal processing device, and the device includes:

[0015] a determining unit, configured to determine an interpolation type of a first haptic event component in a first haptic event by decoding a bit stream of a haptic signal;

[0016] An interpolation unit is used to interpolate signal values ​​between the signal value of the first tactile event component and the signal value of a second tactile event component based on the interpolation type of the first tactile event component, and obtain the tactile signal, where the second tactile event component is a tactile event component before or after the first tactile event component.

[0017] In a fourth aspect, an embodiment of the present application provides a tactile signal interpolation device, which is applied to a packaging device or a signal processing device, and the device includes:

[0018] an acquisition unit, configured to acquire a tactile signal;

[0019] A determination unit is used to determine a first interpolation type based on the tactile signal, wherein the first interpolation type is used to perform signal value interpolation between the signal value of the first tactile event component and the signal value of the second tactile event component in the first tactile event, and the second tactile event component is a tactile event component before or after the first tactile event component.

[0020] In a fifth aspect, an embodiment of the present application provides an electronic device, including:

[0021] a processor adapted to implement computer instructions; and,

[0022] A computer-readable storage medium stores computer instructions, wherein the computer instructions are suitable for being loaded by a processor and executing the method provided by the first aspect or the second aspect mentioned above.

[0023] In a sixth aspect, the present application provides a chip for implementing the method provided in the first or second aspect above. Specifically, the chip includes: a processor for calling and running a computer program from a memory, so that a device equipped with the chip executes the method provided in the first or second aspect above.

[0024] In the seventh aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are read and executed by a processor of a computer device, the computer device executes the method provided in the first aspect or the second aspect mentioned above.

[0025] In an eighth aspect, embodiments of the present application provide a computer program product or computer program, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in the first or second aspect described above.

[0026] In an embodiment of the present application, the decapsulation device determines the interpolation type of the first tactile event component in the first tactile event by decoding the bit stream of the tactile signal; based on the interpolation type of the first tactile event component, signal value interpolation is performed between the signal value of the first tactile event component and the signal value of the second tactile event component to obtain a tactile signal, wherein the second tactile event component is the tactile event component before or after the first tactile event component. In other words, the decapsulation device can not only flexibly restore different types of tactile signals through signal value interpolation, that is, more accurately and flexibly represent different types of tactile signals, but also reduce the information of the tactile event components that need to be carried in the bit stream, that is, save the overhead of tactile signal representation, thereby improving the user experience of tactile media consumers and reducing the bandwidth consumption of tactile media transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0028] FIG1 is a schematic diagram of three degrees of freedom provided in an embodiment of the present application.

[0029] FIG2 is a schematic diagram of three degrees of freedom+ provided in an embodiment of the present application.

[0030] FIG3 is a schematic diagram of six degrees of freedom provided in an embodiment of the present application.

[0031] FIG4 is a schematic diagram of a system architecture provided in an embodiment of the present application.

[0032] FIG5 is a schematic diagram of an immersive media system provided in an embodiment of the present application.

[0033] FIG6 is a schematic diagram of a data structure representing tactile information provided by an embodiment of the present application.

[0034] FIG7 is a schematic flowchart of a tactile signal interpolation method provided in an embodiment of the present application.

[0035] FIG8 is an example of the format of a bit stream provided by an embodiment of the present application.

[0036] FIG9 is an example of the interpolation principle provided by an embodiment of the present application.

[0037] FIG10 is a schematic flowchart of another tactile signal interpolation method provided in an embodiment of the present application.

[0038] FIG11 is a schematic block diagram of a tactile signal interpolation device provided in an embodiment of the present application.

[0039] FIG12 is a schematic block diagram of another tactile signal interpolation device provided in an embodiment of the present application.

[0040] FIG13 is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments provided by this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0042] To facilitate understanding of the technical solutions provided by this application, the relevant terms are explained below.

[0043] Haptics: The sensory experience obtained by the human body through contact, such as vibration, pressure, temperature, etc.

[0044] Haptics Signal: A signal used to represent a specific modal tactile experience and rendered on a specific device.

[0045] JavaScript Object Notation (JSON) is a lightweight data exchange format. It uses a text format that is completely independent of the programming language to store and represent data. Its simplicity and clear hierarchical structure make JSON an ideal data exchange language. It is easy for humans to read and write, as well as for machines to parse and generate, and it significantly improves network transmission efficiency.

[0046] Bit stream: also known as code stream, refers to the binary data stream after compression encoding.

[0047] Track: A track is a collection of media data within a media file, consisting of multiple time-sequential samples. A media file can consist of one or more tracks. For example, a media file can contain a video track, an audio track, and a subtitle track. Metadata information can also be included as a media type within a file in the form of a metadata media track.

[0048] Item: A project is a unit of encapsulation for non-sequential media data during the media file encapsulation process. For example, a static image can be encapsulated as an item.

[0049] Sample: A sample is a unit of media file encapsulation. A track consists of many samples, each of which corresponds to a specific timestamp. For example, a video media track can consist of many samples, with a sample typically representing a video frame. In this embodiment of the present application, a sample in a tactile media track can be one or more tactile signals.

[0050] Sub-sample: A sub-sample is a portion of the sample data.

[0051] Sample Number: The first sample in a track has a sample number of 1.

[0052] Sample Entry: This field indicates metadata related to all samples in a track. For example, a sample entry for a video track typically contains metadata related to decoder initialization.

[0053] Sample Group: It is used to group some samples in a track according to specific rules.

[0054] Dynamic adaptive streaming over HTTP (DASH): is an adaptive bitrate streaming technology that enables high-quality streaming media to be delivered over the Internet through traditional HTTP web servers.

[0055] Media Presentation Description (MPD): Media presentation description signaling in DASH, used to describe media segment information.

[0056] Representation: A combination of one or more media components in DASH. For example, a video file of a certain resolution can be considered a representation.

[0057] Adaptation Sets: In DASH, a collection of one or more video streams. An adaptation set can contain multiple representations.

[0058] Media Segment: A playable segment that conforms to a specific media format. Playback may require the cooperation of zero or more preceding segments and an initialization segment.

[0059] Degree of Freedom (DoF): refers to the freedom of movement supported by users when watching immersive media and generating content interaction.

[0060] Three degrees of freedom (3DoF): For example, as shown in Figure 2, this refers to the three degrees of freedom of the user's head rotating around the x-axis, y-axis, and z-axis.

[0061] 3DoF+: In addition to the 3DoF, the user also has limited freedom of movement along the x-, y-, and z-axes. For example, as shown in Figure 3, in addition to the 3DoF, the user also has the freedom to move forward, backward, left, and right along the x-, y-, and z-axes.

[0062] Six degrees of freedom (6DoF): In addition to the three degrees of freedom, the user also has the freedom to move freely along the x-axis, y-axis, and z-axis. For example, as shown in Figure 4, in addition to the three degrees of freedom, the user also has the freedom to move forward, backward, left, and right along the x-axis, y-axis, and z-axis.

[0063] Audio Video Coding Standard (AVS): AVS is a technical specification for compressing and encoding audio and video data, aiming to reduce the storage space and transmission bandwidth of audio and video data while maintaining good sound and video quality.

[0064] ISO Based Media File Format (ISOBMFF): is a media file encapsulation standard. The most typical ISOBMFF file is the MP4 file.

[0065] Smart Media Transport (SMT) specifies smart media transport technologies, including encapsulation formats, transport protocols, and signaling messages, for use in the transmission and delivery of multimedia data over heterogeneous packet switching networks.

[0066] Media asset: Any multimedia data entity associated with a unique identifier used to construct a multimedia presentation.

[0067] It should be noted that the terms used in the implementation methods of this application are only used to explain the embodiments of this application and are not intended to limit this application.

[0068] For example, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" is only a description of the combination relationship of enumerated objects, indicating that one or more items may exist. For example, at least one of the following: A, B, C can mean the following combinations: A exists alone, B exists alone, C exists alone, A and B exist at the same time, A and C exist at the same time, B and C exist at the same time, and A, B, and C exist at the same time. The term "multiple" refers to two or more. The character " / " generally indicates that the related objects before and after are in an "or" relationship.

[0069] For example, the term "corresponding" can indicate a direct or indirect correspondence between two items, an association between the two items, or a relationship between an indicator and the indicated, a configuration and the configured, and so on. The term "indication" can be a direct indication, an indirect indication, or an association. For example, "A indicates B" can mean that A directly indicates B, for example, B can obtain it through A; it can also mean that A indirectly indicates B, for example, A indicates C, and B can obtain it through C; it can also mean that A and B have an association. The terms "predefined" or "preconfigured" can mean that the corresponding code, table, or other relevant information that can be used for indication is pre-stored in the device, or it can refer to a protocol agreement. "Protocol" can refer to a standard protocol in this field. The term "when..." can be interpreted as "if," "if," "when," "in response to," and similar descriptions. Similarly, depending on the context, the phrase "if determined" or "if (stated condition or event) is detected" can be interpreted as "when determined" or "in response to determining" or "when (stated condition or event) is detected" or "in response to detecting (stated condition or event)" and similar descriptions. The terms "first", "second", "third", "fourth", "Ath", "Bth" and the like are used to distinguish different objects rather than to describe a specific order. The terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. Among them, digital video compression technology is mainly used to compress huge digital image video data for easy transmission and storage.

[0070] The following describes the application scenarios, technical problems to be solved, and inventive concepts of the embodiments of the present application.

[0071] The solution provided in this application may also relate to the field of vehicle-mounted technology.

[0072] The solution provided in this application may also involve coding standards or technologies.

[0073] The solution provided in this application can be applied to the field of digital video coding technology, for example, the field of image coding and decoding, the field of video coding and decoding, the field of hardware video coding and decoding, the field of dedicated circuit video coding and decoding, and the field of real-time video coding and decoding. In other words, the solution provided in this application can be combined with: Audio Video Coding Standard (AVS), the second generation AVS standard (AVS2) or the third generation AVS standard (AVS3). Specifically including but not limited to H.264 / Audio Video Coding (AVC) standard, H.265 / High Efficiency Video Coding (HEVC) standard and H.266 / Versatile Video Coding (VVC) standard. In other words, the solution provided in this application can be combined with other proprietary or industry standards, for example, including ITU-TH.261, ISO / IEC MPEG-1 Visual, ITU-TH.262 or ISO / IEC MPEG-2 Visual, ITU-TH.263, ISO / IEC MPEG-4 Visual, ITU-TH.264 (also known as ISO / IEC MPEG-4 AVC), and also includes Scalable Video Codec (SVC) and Multi-View Video Codec (MVC) extensions.

[0074] The client referred to in this application may be a user terminal. Optionally, the user terminal includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a vehicle terminal, an aircraft, etc.

[0075] FIG4 is a schematic diagram of an application scenario 100 applicable to an embodiment of the present application.

[0076] As shown in Figure 4, the application scenario 100 may include a terminal 110 and a server 120. The terminal 110 and the server 120 are directly or indirectly connected via a wired or wireless manner.

[0077] Among them, the server 120 can produce or collect tactile media signals according to the expected tactile effects, and generate an exchange format for the tactile signals. The exchange format for the tactile signals can also be called signal data that conforms to the exchange format. After the server 120 generates the exchange format for the tactile signals, it can encode the exchange format for the tactile signals to obtain a bit stream (also called a code stream) of the tactile signals, and encapsulate the bit stream of the tactile signals to obtain a media file or media segment of the tactile signals, and then transmit the media file or media segment to the client 110.

[0078] Correspondingly, after receiving the media file or media segment sent by the terminal server 120, the terminal 110 can first decapsulate and decode the media file or media segment to obtain the exchange format of the tactile signal, and then render and present the signal based on the exchange format of the tactile signal. The terminal 110 can be any electronic product that can interact with the user through one or more methods such as a keyboard, touchpad, touch screen, remote control, voice or handwriting.

[0079] Exemplarily, the terminal may include but is not limited to the above-mentioned mobile phones, tablet personal computers, laptop computers, notebook computers, personal digital assistants (PDAs), handheld computers, netbooks, ultra-mobile personal computers (UMPCs), mobile Internet devices (MIDs), augmented reality (AR), virtual reality (VR) devices, robots, wearable devices, flight vehicles, vehicle user equipment (VUE), shipborne equipment, pedestrian user equipment (PUE), smart homes (home appliances with wireless communication capabilities, such as refrigerators, televisions, washing machines or furniture, etc.), game consoles, personal computers (PCs), ATMs or self-service machines and other terminal-side devices. Wearable devices include: smart watches, smart bracelets, smart headphones, smart glasses, smart jewelry (smart bracelets, smart bracelets, smart rings, smart necklaces, smart anklets, smart anklets, etc.), smart wristbands, smart clothing, etc. Among them, vehicle-mounted equipment can also be called vehicle-mounted terminals, vehicle-mounted controllers, vehicle-mounted modules, vehicle-mounted components, vehicle-mounted chips, or vehicle-mounted units.

[0080] Server 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic "cloud" computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0081] It should be understood that the above-mentioned terminal 110 and server 120 are only examples. Other existing or future possible implementations of terminals or servers, if applicable to this application, should also be included in the scope of protection of this application and included here by reference.

[0082] The solution provided in this application involves immersive media content, and its related content is described below.

[0083] Immersive media content refers to media content that can bring an immersive experience to consumers. Immersive media content can be divided into 3DoF media, 3DoF+ media, and 6DoF media according to the user's degree of freedom when consuming media content.

[0084] FIG5 is a schematic diagram of an immersive media system provided in an embodiment of the present application.

[0085] As shown in FIG5 , the immersive media system includes a file encapsulator and a file decapsulator.

[0086] The desired haptic media effect (A) can be used to capture or produce an exchange format (B) for haptic signals, which can also be referred to as signal data conforming to the exchange format. The exchange format (B) is encoded to produce a bitstream (E). The bitstream (E) is then assembled into a media file (F) for file playback or a sequence of initialization segments and media fragments for streaming (Fs), depending on the specific file format. The file encapsulator can also include metadata in the file or fragment. The fragment Fs is delivered to the player using a delivery mechanism.

[0087] The file (F) output by the file encapsulator is the same as the file (F') input by the file decapsulator. The file decapsulator processes the media file (F') or the received fragments (F's) and decapsulates the bitstream (E') and parses it to obtain the interchange format (D'), which can be used for rendering and presentation to the user. Where applicable, the visual image is rendered and dynamically displayed on the screen of a head-mounted display or any other display device based on the current viewing position, viewing direction, or viewport determined by various types of sensors (such as the head), where the dynamic display can be achieved using position sensors or eye tracking sensors.

[0088] Immersive media content is often presented through various wearable or interactive devices. Therefore, in addition to traditional visual and auditory presentations, immersive media also features a new form of tactile presentation. Tactile presentation uses a tactile presentation mechanism that combines hardware and software to allow users to receive information through their bodies, providing an embedded physical sensation and conveying key information about the system they are using. For example, a phone can vibrate to alert the user of a received message; this vibration is a type of tactile presentation. Tactile presentation can enhance auditory and visual presentations, improving the user experience.

[0089] Tactile information presentation has been widely used so far and is generally categorized into the following: vibrotactile, which directly presents tactile sensations in the form of vibrations; kinematic tactile, which simulates the weight or pressure of an object; and electrotactile, which simulates feedback information about a specific texture through electrical stimulation. Specifically:

[0090] 1. Vibratory touch.

[0091] Vibration of a specific frequency and intensity is simulated by the terminal device's motor vibration. For example, in a shooting game, vibration is used to simulate the specific effects of device use.

[0092] 2. Kinematic touch.

[0093] Kinematic haptic systems simulate the weight or pressure of an object. For example, in a driving video game, the steering wheel might resist turning when moving at higher speeds or operating a heavier vehicle. This type of feedback directly affects the user's muscles. In the example of a driving game, the user must apply more force to get the desired response from the steering wheel.

[0094] 3. Electrotactile sensation.

[0095] Electrotactile presentation uses electrical pulses to deliver tactile stimulation to the nerve endings in a user's skin. Electrotactile presentation can create a highly realistic experience for users wearing suits or gloves equipped with electrotactile technology. Almost any sensation can be simulated using electrical pulses: temperature changes, pressure changes, the sensation of moisture.

[0096] The current state of tactile technology in the industry demonstrates that, from the vibration haptics commonly found in everyday devices to the diverse tactile experiences in niche areas, tactile experience itself has become a form of presentation that users are accustomed to. Furthermore, with the increasing prevalence of wearable and interactive devices, the tactile experience users experience when consuming media content will no longer be limited to basic vibration haptics, but will instead include a full range of haptic sensations, including vibration, pressure, speed, acceleration, temperature, humidity, and smell, offering a more realistic, tactile experience.

[0097] FIG6 is a schematic diagram of a data structure representing tactile information provided by an embodiment of the present application.

[0098] As shown in Figure 6, the highest-level data structure represented by tactile information is tactile experience, which is used to describe all tactile experience information in a file or bitstream. Tactile experience includes metadata information related to the tactile experience, possible device information, and one or more tactile patterns. A tactile pattern corresponds to a tactile signal of a certain type (such as vibration, pressure, temperature, etc.). The tactile pattern includes metadata information related to the tactile pattern, possible predefined knowledge events, and one or more tactile channels. A tactile channel contains all or part of the tactile signal of the corresponding tactile pattern. The tactile channel includes metadata information related to the tactile channel and one or more tactile events. Generally speaking, tactile signals corresponding to different rendering devices in the same tactile mode can be organized into different tactile channels. A tactile event is a basic tactile signal unit. A tactile event includes metadata information related to the tactile event and possible tactile event components. The tactile event component is the time domain or frequency domain component of the tactile event.

[0099] The data structure of each level is described below.

[0100] 1. Tactile experience.

[0101] The data structure of tactile experience is shown in Table 1.

[0102] Table 1

[0103] 2. Device information.

[0104] The data structure of device information is shown in Table 2.

[0105] Table 2

[0106] 3. Tactile mode.

[0107] The data structure of the tactile mode is shown in Table 3.

[0108] Table 3

[0109] 4. Tactile channel.

[0110] The data structure of the tactile channel is shown in Table 4.

[0111] Table 4

[0112] 5. Tactile events.

[0113] The data structure of the tactile event is shown in Table 5.

[0114] Table 5

[0115] It is worth noting that knowledge events are special tactile events whose data structure follows the data structure definition of tactile events. Knowledge events are usually predefined tactile events that can be repeatedly referenced to avoid repeated parsing of the same tactile event.

[0116] 6. Haptic event component.

[0117] The data structure of the tactile event component is shown in Table 6.

[0118] Table 6

[0119] From the above content, it can be seen that tactile presentation can be achieved through the transmission of tactile signals. Specifically, tactile signals can be defined as tactile events, and support can be provided to decompose tactile events into multiple tactile event components, thereby achieving the transmission of tactile signals. However, this way of defining tactile signals may lead to poor flexibility in tactile signal representation and excessive representation overhead, which in turn may lead to problems such as poor user experience for tactile media consumers and excessive bandwidth consumption for tactile media transmission. For example, there is no clear solution for how to restore the original tactile signal through the tactile event component, and how to derive the signal value between the various tactile event components.

[0120] In view of this, the embodiments of the present application provide a tactile signal interpolation method, device, electronic device and storage medium. Through signal value interpolation, not only can different types of tactile signals be flexibly restored, that is, different types of tactile signals can be represented more accurately and flexibly, but also the information of tactile event components that need to be carried in the bitstream can be reduced, that is, the overhead of tactile signal representation can be saved, thereby improving the user experience of tactile media consumers and reducing the bandwidth consumption of tactile media transmission.

[0121] The tactile signal interpolation method provided in the embodiment of the present application can be applied to tactile feedback-related products, the server side, the player side, and intermediate nodes of the immersive system, and the present application does not make any specific limitations on this.

[0122] The tactile signal interpolation method provided in the embodiment of the present application is described below.

[0123] Figure 2 shows a schematic flow chart of a tactile signal interpolation method 200 according to an embodiment of the present application. This method 200 can be executed by any electronic device with data processing capabilities. For example, the electronic device can be implemented as a terminal, such as the terminal 110 or signal processing device shown in Figure 4 . For ease of explanation, the following description of the tactile signal interpolation method 200 is based on an example of a decapsulation device executing the tactile signal interpolation method 200.

[0124] As shown in FIG2 , the haptic signal interpolation method 200 may include:

[0125] In step S210 , the decapsulation device determines an interpolation type of a first haptic event component in a first haptic event by decoding a bit stream of the haptic signal.

[0126] Exemplarily, a tactile signal is a signal used to convey a tactile experience, and is intended to present a perceptible tactile feedback to the user through a specific device. The tactile signal may include multiple modalities, such as vibration, pressure, temperature, etc., to simulate different tactile sensations. The tactile signal may be rendered on a specific device so that the user can feel the texture, shape, weight and other attributes of the virtual object or scene through touch, thereby enhancing the perception and cognition of the virtual environment. The bit stream of the tactile signal may be a bit stream obtained by encoding the exchange format of the tactile signal, and the bit stream of the tactile signal may also be referred to as the code stream of the tactile signal. The exchange format of the tactile signal is data of the tactile signal in a specific format. For example, the exchange format of the tactile signal may be JSON data.

[0127] In some embodiments, the first haptic event is decomposed into one or more haptic event components, and the first haptic event component is one of the one or more haptic event components into which the first haptic event is decomposed.

[0128] FIG8 is an example of the format of a bit stream provided by an embodiment of the present application.

[0129] As shown in FIG8 , the bit stream of the haptic signal may include one or more data units, each of which may include a header and at least one data packet. For example, the at least one data packet may include packets 1 to N. By decoding the header, relevant information about the at least one data packet may be determined. For example, by decoding the type field in the header, it may be determined that the at least one data packet includes a data packet carrying a data structure of a first haptic event, or that the at least one data packet includes a data packet carrying a data structure of a first haptic event component.

[0130] In some embodiments, the interpolation type of the first haptic event component includes at least one of the following: nearest neighbor interpolation, linear interpolation, cubic spline interpolation, Akma interpolation, Bessel interpolation, B-spline interpolation, Lagrange interpolation, Newton interpolation, and custom algorithm interpolation.

[0131] The various interpolation types are described below.

[0132] Nearest Neighbor Interpolation assumes that the value of each point is the same as the value of its nearest known point.

[0133] Linear interpolation assumes that the value between two known points is linear, that is, the midpoint between the two points is their average value.

[0134] Cubic Spline Interpolation uses a set of continuous cubic polynomial functions to connect known data points. The endpoints of each polynomial are tangent to two known data points, ensuring continuity and smoothness.

[0135] Akima interpolation can be used to estimate the values ​​of unknown data points between given data points. This interpolation method is based on linear interpolation, and its characteristic is that it takes into account the curvature changes between data points.

[0136] Bezier interpolation is an interpolation method based on Bezier curves. Bezier curves are parametric curves defined by a set of control points, used to smoothly connect known data points. In Bezier interpolation, by adjusting the positions and weights of the control points, a smooth curve passing through the known data points can be generated. This method is widely used in computer graphics, animation, and other fields to create smooth, continuous paths and shapes.

[0137] B-spline Interpolation: B-spline interpolation uses a B-spline curve (a parametric curve) to pass through known data points. A B-spline curve is a continuous curve defined by a series of control points.

[0138] Lagrange Interpolation: Lagrange interpolation uses Lagrange polynomials for interpolation. Lagrange polynomials are a special type of interpolation polynomial.

[0139] Newton Interpolation: Newton interpolation uses Newton difference polynomials for interpolation. This method is based on the Newton difference formula and uses known data points to calculate the polynomial for the interpolation points.

[0140] Custom Algorithm Interpolation: Custom algorithm interpolation uses a custom algorithm to perform interpolation calculations based on specific requirements or problems. This approach allows developers to implement specific interpolation logic as needed.

[0141] Of course, in other alternative embodiments, the interpolation type of the first tactile event component may also be other interpolation types. For example, including but not limited to: Hermite interpolation, which can be based on a method of interpolation of given node values ​​and derivative values, and can generate a smooth curve between nodes. Polynomial interpolation, which can use a polynomial function to interpolate through known data points. For example, an appropriate polynomial order can be selected based on the number of known data points. Radial Basis Function Interpolation, which can use a radial basis function to estimate the value of an unknown point. The radial basis function is a function that calculates weights based on distance and is used to interpolate between known data points.

[0142] In step S220 , the decapsulation device performs signal value interpolation between the signal value of the first haptic event component and the signal value of the second haptic event component based on the interpolation type of the first haptic event component, and obtains the haptic signal.

[0143] The second haptic event component is a haptic event component before or after the first haptic event component.

[0144] For example, the second trigger event component is a haptic event component located before or after the first haptic event component in the first haptic event.

[0145] Exemplarily, the decapsulation device performs signal value interpolation between the signal value of the first haptic event component and the signal value of the second haptic event component before or after the first haptic event component based on the interpolation type of the first haptic event component. For example, the decapsulation device can insert one or more signal values ​​and obtain a haptic signal based on the existing signal values ​​(i.e., the signal value of the haptic signal component in the first haptic event, which includes the signal value of the first haptic event component and the signal value of the second haptic event component). That is, in this example, the second haptic event component is a haptic event component in the first haptic event.

[0146] For example, the second haptic event component may be a haptic event component that precedes or follows the first haptic event component according to the decoding order of the haptic event components in the first haptic event. For example, the second haptic event component may be the first haptic event component that precedes or follows the first haptic event component according to the decoding order of the haptic event components in the first haptic event.

[0147] In this embodiment, the decapsulation device can not only flexibly restore different types of tactile signals through signal value interpolation, that is, more accurately and flexibly represent different types of tactile signals, but also reduce the information of tactile event components that need to be carried in the bitstream, that is, save the overhead of tactile signal representation, thereby improving the user experience of tactile media consumers and reducing the bandwidth consumption of tactile media transmission.

[0148] In some embodiments, the above step S210 may include steps S210-A and S210-B:

[0149] Step S210-A: The decapsulation device obtains a first identifier by decoding the bit stream, where the first identifier is used to indicate an interpolation type;

[0150] Step S210 -B: The decapsulation device determines the interpolation type of the first haptic event component based on the interpolation type indicated by the first identifier.

[0151] For example, if the value of the first identifier is a first numerical value, the interpolation type indicated by the first identifier may be the interpolation type corresponding to the first numerical value. Different numerical values ​​may correspond to different interpolation types. For example, the correspondence between the first numerical value and the interpolation type may be as follows:

[0152] 0: nearest neighbor interpolation.

[0153] 1: Linear interpolation.

[0154] 2: Cubic spline interpolation.

[0155] 3: Akima interpolation.

[0156] 4: Bezier interpolation.

[0157] 5: B-spline interpolation.

[0158] 6: Lagrange interpolation.

[0159] 7: Newton interpolation.

[0160] 8: Custom algorithm interpolation.

[0161] In some embodiments, the decapsulation device in step S210-A obtains the first identifier by decoding the bit stream, which may be implemented as follows:

[0162] The decapsulation device obtains the second identifier by decoding the bit stream; and obtains the first identifier by decoding the bit stream when the second identifier indicates the presence of the first identifier.

[0163] Exemplarily, the decapsulation device obtains a second identifier by decoding the bit stream, where the second identifier is used to indicate whether the first identifier exists in the bit stream; when the second identifier indicates that the first identifier exists, the decapsulation device obtains the first identifier by decoding the bit stream.

[0164] When the second identifier indicates that the first identifier does not exist, it means that the bitstream does not include the first identifier. In this case, the decapsulation device can determine the default interpolation type as the interpolation type of the first tactile event component, or it can be considered that it is not necessary or unnecessary to interpolate between the signal value of the first tactile event component and the signal value of the second tactile event component.

[0165] Of course, in other alternative embodiments, the second identifier can also be used to indicate whether to interpolate between the signal value of the first tactile event component and the signal value of the second tactile event component. For example, when the second identifier indicates to interpolate between the signal value of the first tactile event component and the signal value of the second tactile event component, the decapsulation device can directly determine the default interpolation type as the interpolation type of the first tactile event component. In other words, the decapsulation device can obtain the second identifier by decoding the bitstream, and can determine the interpolation type of the first tactile event component through the second identifier, without having to obtain the first identifier mentioned above by decoding the bitstream.

[0166] In some embodiments, the decapsulation device in step S210-A obtains the first identifier by decoding the bit stream, which may be implemented as follows:

[0167] The decapsulation device obtains the third identifier by decoding the bit stream; and obtains the first identifier by decoding the bit stream when the third identifier indicates that the first tactile event is a continuous event.

[0168] Exemplarily, the decapsulation device decodes the bitstream to obtain a third identifier, which is used to indicate whether the first tactile event is a continuous event. If the third identifier indicates that the first tactile event is a continuous event, the decapsulation device decodes the bitstream to obtain the first identifier. If the third identifier indicates that the first tactile event is not a continuous event, it means that the bitstream does not include the first identifier. In this case, the default interpolation type can be determined as the interpolation type of the first tactile event component, and it can also be considered that interpolation between the signal value of the first tactile event component and the signal value of the second tactile event component is unnecessary or unnecessary.

[0169] In some embodiments, the decapsulation device in step S210-A obtains the first identifier by decoding the bit stream, which may be implemented as follows:

[0170] The decapsulation device obtains a data structure of the first haptic event by decoding the bit stream, where the data structure of the first haptic event includes a first identifier.

[0171] Correspondingly, in step S210-B, the decapsulation device determines the interpolation type of the first haptic event component based on the interpolation type indicated by the first identifier, which may be implemented as follows:

[0172] The decapsulation device determines the interpolation type indicated by the first identifier as the interpolation type of some or all haptic event components in the first haptic event, where the some or all haptic event components include the first haptic event component.

[0173] Exemplarily, the decapsulation device determines the interpolation type indicated by the first identifier as the interpolation type of some tactile event components in the first tactile event, where the some tactile event components include the first tactile event component. For example, the some tactile event components may be all tactile event components except the last tactile event component in the first tactile event. For another example, the some tactile event components may be tactile event components at preset positions in the first tactile event. For example, the second tactile event component may be a tactile event component located at the first position, the middle position, or the last position according to the decoding order of the tactile event components in the first tactile event. In other words, the first tactile event component is any one of the tactile event components at the first position, the middle position, or the last position. Of course, the first position, the middle position, or the last position are merely examples of preset positions and should not be understood as limiting the embodiments of the present application.

[0174] Exemplarily, the decapsulation device determines the interpolation type indicated by the first identifier as the interpolation type of all haptic event components in the first haptic event. In other words, the first haptic event component can be any haptic event component in the first haptic event.

[0175] Exemplarily, the data structure of the first haptic event may also be referred to as attribute information of the first trigger event, an element of the first haptic event, or an exchange format of the first haptic event. The data structure of the first haptic event includes relevant information of the first trigger event, including but not limited to metadata of the first haptic event and a list of haptic event components of the first haptic event. Each haptic event component in the list of haptic event components can be used to determine a signal value at a position of the haptic signal.

[0176] In this embodiment, the data structure of the first tactile event includes a first identifier, which is equivalent to that the first identifier can be applied to one or more tactile event components in the first tactile event, which can reduce the amount of data in the bit stream used to determine the interpolation type of the tactile event component, thereby improving data transmission efficiency and saving transmission resources of the first identifier.

[0177] Illustratively, the data structure of the first haptic event may be as shown in Table 7.

[0178] Table 7

[0179] As shown in Table 7, the first identifier is the interpolation_type field, and the second identifier is the interpolation_flag field. When the interpolation_flag field in the data structure of the first tactile event takes a value of 1, the interpolation_type field indicated in the data structure of the first tactile event is effective for all tactile event components of the first tactile event. It is worth noting that knowledge events are special tactile events, and their data structure follows the data structure definition of tactile events. Knowledge events are usually predefined tactile events that can be repeatedly referenced to avoid repeated parsing of the same tactile event.

[0180] In some embodiments, the decapsulation device in step S210-A obtains the first identifier by decoding the bit stream, which may be implemented as follows:

[0181] The decapsulation device obtains a data structure of the first haptic event component by decoding the bitstream, where the data structure of the first haptic event component includes a first identifier. Accordingly, in step S210-B, the decapsulation device determines the interpolation type of the first haptic event component based on the interpolation type indicated by the first identifier, which can be implemented as follows:

[0182] The decapsulation device determines the interpolation type indicated by the first identifier as the interpolation type of the first haptic event component.

[0183] Exemplarily, the data structure of the first haptic event component may also be referred to as attribute information of the first haptic event component, an element of the first haptic event component, or an exchange format of the first haptic event component. The data structure of the first haptic event component includes a time domain component or a frequency domain component of a signal value of the haptic signal.

[0184] In this embodiment, the data structure of the first haptic event component includes the first identifier, which means that the first identifier is only applicable to the first haptic event, which can improve the flexibility of the interpolation type of the haptic event component in the first haptic event.

[0185] In some embodiments, the method 200 of the embodiment of the present application may further include the following steps:

[0186] The decapsulation device determines the interpolation type indicated by the first identifier as the interpolation type of the tactile event component after the first tactile event component, until the data structure in the first tactile event includes a new tactile event component with the first identifier or the last tactile event component in the first tactile event.

[0187] Exemplarily, the decapsulation device determines the interpolation type indicated by the first identifier as the interpolation type of the first tactile event component and the interpolation type of the tactile event components after the first tactile event component, until the data structure in the first tactile event includes a tactile event component with a new first identifier or the last tactile event component in the first tactile event. Specifically, if the decapsulation device decodes a third tactile event component including a new first identifier in the data structure after the first tactile event component according to the decoding order of the tactile event components in the first tactile event, the decapsulation device can determine the interpolation type indicated by the new first identifier as the interpolation type of the third tactile event component. If the decapsulation device does not send a new first identifier after the first tactile event component until the last tactile event component in the first tactile event is decoded according to the decoding order of the tactile event components in the first tactile event, the first identifier applies to all tactile event components after the first tactile event component. In other words, the interpolation types from the first haptic event component to the last haptic event component (for example, including the first haptic event component and the last haptic event component) are all the interpolation types indicated by the first identifier included in the data structure of the first haptic event component.

[0188] In this embodiment, the decapsulation device interrupts the application of the first identifier through the tactile event component of the new first identifier or the last tactile event component in the first tactile event. This not only reduces the amount of data in the bitstream used to determine the interpolation type of the tactile event component, thereby improving data transmission efficiency and saving transmission resources of the first identifier, but also improves the flexibility of the interpolation type of the tactile event component in the first tactile event.

[0189] Illustratively, the data structure of the first haptic event component may be as shown in Table 8.

[0190] Table 8

[0191] As shown in Table 8, the first identifier is the interpolation_type field, and the second identifier is the interpolation_flag field. Assuming that the first haptic event includes N haptic event components, if the interpolation_flag field in the data structure of the i-th haptic event component is 1, the corresponding interpolation_type field will remain in effect until a new interpolation_type field appears or until the last haptic event component among the N haptic event components.

[0192] In some embodiments, in step S220, the decapsulation device performs signal value interpolation between the signal value of the first haptic event component and the signal value of a second haptic event component before or after the first haptic event component based on the interpolation type of the first haptic event component, and obtains the haptic signal, which can be implemented as follows:

[0193] The decapsulation device interpolates between the signal value of the first haptic event component and the signal value of the second haptic event component based on the signal value of the first haptic event component and uses the interpolation type of the first haptic event component to obtain a haptic signal.

[0194] Exemplarily, the decapsulation device may utilize the interpolation type (i.e., interpolation method) of the first tactile event component, take the signal value of the first tactile event component as input, perform interpolation between the signal value of the first tactile event component and the signal value of the second tactile event component, and obtain a tactile signal. For example, assuming that the interpolation type of the first tactile event component is nearest neighbor interpolation, the decapsulation device may utilize nearest neighbor interpolation, take the signal value of the first tactile event component as the signal value of the nearest neighbor, and interpolate the signal value of the nearest neighbor between the signal value of the first tactile event component and the signal value of the second tactile event component to obtain a tactile signal.

[0195] Exemplarily, the decapsulation device may interpolate between the signal value of the first haptic event component and the signal value of the second haptic event component based on the signal value of the first haptic event component and the signal value of the second haptic event component using the interpolation type of the first haptic event component to obtain the haptic signal. For example, assuming that the interpolation type of the first haptic event component is linear interpolation, the decapsulation device may determine the average value of the signal value of the first haptic event component and the signal value of the second haptic event component, and interpolate the average value between the signal value of the first haptic event component and the signal value of the second haptic event component to obtain the haptic signal.

[0196] FIG9 is an example of the interpolation principle provided by an embodiment of the present application.

[0197] As shown in Figure 9, assuming that the first tactile event includes tactile event components 1 to 3, the interpolation type of the tactile event components is nearest neighbor interpolation, and the interpolation type of the tactile event component 2 is linear interpolation, the decapsulation device can use the nearest neighbor interpolation to take the signal value of the tactile event component 1 as the nearest neighbor signal value, and insert the nearest neighbor signal value between the signal value of the tactile event component 1 and the signal value of the tactile event component 1. In addition, the decapsulation device can determine the average value of the signal values ​​of the tactile event component 2 and the tactile event component 3, and insert a linear signal value between the signal value of the tactile event component 2 and the signal value of the tactile event component 3. Then, the decapsulation device obtains a tactile signal based on the signal values ​​of the tactile event components 1 to 3 and the two inserted signal values.

[0198] Of course, in other alternative embodiments, the decapsulation device may also interpolate between the signal value of the first tactile event component and the signal value of the second tactile event component based on the signal value of the second tactile event component and the interpolation type of the first tactile event component to obtain a tactile signal. For example, assuming that the interpolation type of the first tactile event component is nearest neighbor interpolation, the decapsulation device may use the nearest neighbor interpolation to take the signal value of the second tactile event component as the nearest neighbor signal value, and insert the nearest neighbor signal value between the signal value of the first tactile event component and the signal value of the second tactile event component to obtain a tactile signal. For example, the decapsulation device may determine, based on the application scenario of the first tactile event, whether to interpolate between the signal value of the first tactile event component or the signal value of the second tactile event component and the signal value of the second tactile event component using the interpolation type of the first tactile event component.

[0199] In some embodiments, before step S210, method 200 may further include the following steps:

[0200] The decapsulation device receives a tactile media file sent by the encapsulation device and decapsulates the tactile media file to obtain a bit stream; or receives a tactile media segment sent by the encapsulation device and decapsulates the tactile media segment to obtain a bit stream; or receives a tactile media track sent by the file encapsulation device and decapsulates the tactile media track to obtain a bit stream.

[0201] Exemplarily, the decapsulation device can receive three different types of tactile media encapsulation data: tactile media files, tactile media fragments, and tactile media tracks, all of which are sent by the encapsulation device. Regardless of whether it is a file, fragment, or track, it uses a certain encapsulation format, which is designed to facilitate data transmission and storage. When the decapsulation device receives these encapsulated data, it needs to perform a decapsulation operation to obtain the bit stream. Decapsulation is an inverse process, the purpose of which is to remove the encapsulation format to restore or extract the original data stream. The specific decapsulation method may vary depending on the encapsulation format.

[0202] The following is an exemplary description of the decoding process of the bit stream based on Table 9 and Table 10 in combination with the first identifier, the second identifier, and the third identifier involved in this application.

[0203] Table 9

[0204] The semantics of each field in Table 9 are as follows:

[0205] eventId: An identifier indicating a haptic event.

[0206] eventType: Indicates the type of tactile event. A value of 0 indicates an instantaneous event; a value of 1 indicates a continuous event; and a value of 2 indicates a reference event.

[0207] eventSementicsFlag: When the value is 0, it does not indicate the semantics of the tactile event; when the value is 1, it indicates the semantics of the tactile event.

[0208] semanticKeywords: indicates the semantics of the event. The semantics of a tactile event are event auxiliary information keywords that describe the content creator's intention.

[0209] referEventId: indicates the knowledge event identifier corresponding to the reference event.

[0210] interpolationFlag: Whether to indicate the interpolation type. When the value of this field is 1, it indicates the signal interpolation type; when the value of this field is 0, it does not indicate the signal interpolation type.

[0211] interpolationType: indicates the interpolation type. Possible values ​​are as follows:

[0212] 0: nearest neighbor interpolation;

[0213] 1: linear interpolation;

[0214] 2: cubic spline interpolation;

[0215] 3: Akima interpolation;

[0216] 4: Bezier interpolation;

[0217] 5: B-spline interpolation;

[0218] 6: Lagrange interpolation;

[0219] 7: Newton interpolation;

[0220] 8: Custom algorithm interpolation.

[0221] relativePosition: Indicates the time or space offset of the touch event.

[0222] duration: indicates the duration of the tactile event. This attribute exists when the event type is a continuous event.

[0223] Amplitude: indicates the maximum amplitude of the signal of a tactile event.

[0224] baseFrequency: Indicates the base frequency of the signal of the haptic event.

[0225] numComponents: Indicates the number of components of a tactile event. The components of a tactile event can be time-domain components or frequency-domain components of the tactile event.

[0226] Specifically, readEvent() is a function for processing tactile events, which is used to read and parse various attributes of a tactile event from an input. First, the unique identifier eventId of the tactile event is read and stored. Then, eventType is read, which represents the type of the tactile event. Next, eventSementicsFlag is read. This flag is usually used to indicate whether the tactile event has semantic information. If the value of eventSementicsFlag is true (non-zero), semantic keywords semanticKeywords can be further read and stored. Then, the function checks whether the value of eventType is equal to 2. If it is equal to 2, it means this is a reference event, and then the reference event ID referEventId can be read and stored. Reference events are usually used to describe the relationship between an event and other events. Next, the function reads interpolationFlag. If the value of interpolationFlag is 1, it indicates that there is an interpolation type, and then the interpolation type interpolationType can be read and stored. Interpolation is a mathematical method used to estimate new data points between two discrete data points. When eventType is equal to 1, the function can read and store the duration duration of the tactile event. Finally, the function reads and stores the amplitude value amplitude, the base frequency baseFrequency, and the number of components numComponents of the tactile event. These attributes provide more information about the tactile signal. Once all the attributes are read, the function processes each tactile event component through a for loop (from i = 0 to i < numComponents). Inside the loop, a function named readEventComponent() is called, which may be responsible for reading and processing a single tactile event component. Among them, the first identifier above is the interpolation_type field, the second identifier above is the interpolation_flag field, and the third identifier above is the eventType field.

[0227] Table 10

[0228] Among them, the semantics of each field in Table 10 are as follows:

[0229] referEventId: Indicates the identifier of the tactile event to which the tactile event component belongs.

[0230] interpolationFlag: Whether to indicate the interpolation type. When the value of this field is 1, it indicates the signal interpolation type; when the value of this field is 0, it does not indicate the signal interpolation type.

[0231] interpolationType: indicates the interpolation type. Possible values ​​are as follows:

[0232] 0: nearest neighbor interpolation;

[0233] 1: linear interpolation;

[0234] 2: cubic spline interpolation;

[0235] 3: Akima interpolation;

[0236] 4: Bezier interpolation;

[0237] 5: B-spline interpolation;

[0238] 6: Lagrange interpolation;

[0239] 7: Newton interpolation;

[0240] 8: Custom algorithm interpolation.

[0241] relative_position: indicates the time or space offset of the haptic event component relative to the haptic event.

[0242] relative_amplitude: indicates the amplitude ratio of the tactile event component relative to the tactile event, with a value range of 0 to 1.

[0243] relative_frequency: Indicates the frequency offset of the haptic event component relative to the haptic event.

[0244] Specifically, readEventComponent() is an auxiliary function used to read and process a single tactile event component of a tactile event. First, read and store the reference event ID referEventId. The reference event ID refers to the identifier of other events relative to the current event, which is used to describe the relationship between events. Next, read interpolationFlag. If the value of interpolationFlag is 1, it means that the component indicates an interpolation type, and then read and store the interpolation type interpolationType. The interpolation type defines the interpolation method and parameters. Then, the function reads relative_position. Relative_position represents the relative position information of the component in the tactile event. Next, the function reads relative_amplitude. Relative_amplitude represents the relative amplitude information of the component in the tactile event. Finally, the function reads relative_frequency. Relative_frequency represents the relative frequency information of the component in the tactile event. Among them, the first identifier above is the interpolation_type field, the second identifier above is the interpolation_flag field, and the third identifier above is the eventType field.

[0245] The preferred embodiments of the present application are described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the specific details in the embodiments mentioned above. Within the technical concept of the present application, the technical solution of the present application can be subjected to a variety of simple modifications, and these simple modifications all fall within the scope of protection of the present application. For example, the various specific technical features described in the specific embodiments mentioned above can be combined in any suitable manner unless there is any contradiction. In order to avoid unnecessary repetition, the present application will not further explain various possible combinations. For another example, the various different embodiments of the present application can also be arbitrarily combined, and as long as they do not violate the ideas of the present application, they should also be regarded as the contents disclosed in the present application.

[0246] It should also be understood that in the various method embodiments of the present application, the size of the serial numbers of the processes involved above does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0247] The following describes the tactile signal interpolation method provided in the embodiment of the present application from the perspective of a device for encoding or packaging tactile signals.

[0248] Figure 10 is a schematic flow chart of a tactile signal interpolation method 300 provided in an embodiment of the present application. This method 300 can be executed by any electronic device with data processing capabilities. For example, the electronic device can be implemented as a server, such as the server 120 or signal processing device shown in Figure 4 . For ease of explanation, the following description of this tactile signal interpolation method 300 is based on an example of a packaged device executing this tactile signal interpolation method 300.

[0249] As shown in FIG10 , the haptic signal interpolation method 300 may include:

[0250] Step S310, the packaging device obtains a tactile signal;

[0251] In step S320, the packaging device determines a first interpolation type based on the tactile signal. The first interpolation type is used to perform signal value interpolation between the signal value of the first tactile event component and the signal value of the second tactile event component in the first tactile event. The second tactile event component is a tactile event component before or after the first tactile event component.

[0252] In some embodiments, method 300 may further include:

[0253] The packaging device encodes the first identifier to obtain a bit stream of the haptic signal, where the first identifier is used to indicate a first interpolation type.

[0254] In some embodiments, method 300 may further include:

[0255] The packaging device encodes a second identifier or a third identifier, the second identifier is used to indicate whether the first identifier exists, and the third identifier is used to indicate whether the first tactile event is a continuous event.

[0256] In some embodiments, the packaging device encodes the first identifier to obtain a bit stream of the tactile signal, which can be implemented as follows:

[0257] The packaging device encodes the data structure of the first haptic event and the data structure of the first haptic event component to obtain a bit stream;

[0258] The data structure of the first tactile event includes the first identifier, or the data structure of the first tactile event component includes the first identifier.

[0259] In some embodiments, method 300 may further include:

[0260] The encapsulation device encapsulates the bit stream to obtain a tactile media file of the tactile signal;

[0261] The encapsulation device sends the haptic media file, the haptic media file segment in the haptic media file, or the haptic media track in the haptic media file to the decapsulation device.

[0262] In some embodiments, the interpolation type of the first haptic event component includes at least one of the following: nearest neighbor interpolation, linear interpolation, cubic spline interpolation, Akma interpolation, Bessel interpolation, B-spline interpolation, Lagrange interpolation, Newton interpolation, and custom algorithm interpolation.

[0263] It should be understood that method 300 can be understood as the reverse process of method 200, that is, in method 300, the packaging device produces or collects tactile signals according to the expected tactile media effect, and generates an exchange format for the tactile signals. For the continuous tactile signals, the interpolation type of the tactile event component can be selected by the tactile signal (for example, the signal value of the interpolation point), and the interpolation type is indicated in the tactile event or the tactile event component. The packaging device compresses the exchange format of the tactile signal into a bit stream of the tactile signal. The packaging device encapsulates the bit stream of the tactile signal into a tactile media file, which contains one or more tactile media tracks. Optionally, the packaging device can slice the tactile media file into multiple tactile media file segments. More specifically, the specific scheme of method 300 can be found in the relevant content of method 200, and to avoid repetition, it will not be repeated here.

[0264] The device provided in the embodiments of the present application is described below.

[0265] FIG11 is a schematic block diagram of a tactile signal interpolation device 400 provided in an embodiment of the present application, which is applied to a decapsulation device or a signal processing device.

[0266] As shown in FIG11 , the tactile signal interpolation device 400 may include:

[0267] a determining unit 410 configured to determine an interpolation type of a first haptic event component in a first haptic event by decoding a bit stream of a haptic signal;

[0268] The interpolation unit 420 is used to perform signal value interpolation between the signal value of the first tactile event component and the signal value of the second tactile event component based on the interpolation type of the first tactile event component to obtain the tactile signal, where the second tactile event component is a tactile event component before or after the first tactile event component.

[0269] In some embodiments, the determining unit 410 is specifically configured to:

[0270] Obtaining a second identifier by decoding the bit stream, where the second identifier is used to indicate whether the first identifier exists;

[0271] An interpolation type of the first haptic event component is determined based on the interpolation type indicated by the first identifier.

[0272] In some embodiments, the determining unit 410 is specifically configured to:

[0273] Obtaining a second identifier by decoding the bit stream;

[0274] In a case where the second identifier indicates the presence of the first identifier, the first identifier is obtained by decoding the bit stream.

[0275] In some embodiments, the determining unit 410 is specifically configured to:

[0276] Obtaining a third identifier by decoding the bit stream, where the third identifier is used to indicate whether the first tactile event is a continuous event;

[0277] In a case where the third identifier indicates that the first tactile event is the continuous event, the first identifier is acquired by decoding the bit stream.

[0278] In some embodiments, the determining unit 410 is specifically configured to:

[0279] Obtaining a data structure of the first tactile event by decoding the bit stream, where the data structure of the first tactile event includes the first identifier;

[0280] The interpolation type indicated by the first identifier is determined as the interpolation type of part or all of the haptic event components in the first haptic event, where the part or all of the haptic event components include the first haptic event component.

[0281] In some embodiments, the determining unit 410 is specifically configured to:

[0282] Obtaining a data structure of the first haptic event component by decoding the bit stream, where the data structure of the first haptic event component includes the first identifier;

[0283] The interpolation type indicated by the first identifier is determined as the interpolation type of the first haptic event component.

[0284] In some embodiments, the determining unit 410 is further configured to:

[0285] The interpolation type indicated by the first identifier is determined as the interpolation type of the tactile event component after the first tactile event component, until the data structure in the first tactile event includes a new tactile event component with the first identifier or the last tactile event component in the first tactile event.

[0286] In some embodiments, the interpolation unit 420 is specifically configured to:

[0287] Based on the signal value of the first haptic event component and using the interpolation type of the first haptic event component, interpolation is performed between the signal value of the first haptic event component and the signal value of the second haptic event component to obtain the haptic signal.

[0288] In some embodiments, the interpolation type of the first haptic event component includes at least one of the following: nearest neighbor interpolation, linear interpolation, cubic spline interpolation, Akma interpolation, Bessel interpolation, B-spline interpolation, Lagrange interpolation, Newton interpolation, and custom algorithm interpolation.

[0289] In some embodiments, before determining the interpolation type of the first haptic event component in the first haptic event, the determining unit 410 is further configured to:

[0290] Receive a tactile media file sent by an encapsulation device, and decapsulate the tactile media file to obtain the bitstream; or receive a tactile media segment sent by an encapsulation device, and decapsulate the tactile media segment to obtain the bitstream; or receive a tactile media track sent by a file encapsulation device, and decapsulate the tactile media track to obtain the bitstream.

[0291] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions can refer to the method embodiment. Specifically, the tactile signal interpolation device 400 may correspond to the corresponding subject in the method 200 of the embodiment of the present application, and the various units in the tactile signal interpolation device 400 are respectively for implementing the corresponding processes in the method 200. For the sake of brevity, they are not further described here.

[0292] FIG12 is a schematic block diagram of a tactile signal interpolation device 500 provided in an embodiment of the present application, which is applied to a packaging device or a signal processing device.

[0293] As shown in FIG12 , the tactile signal interpolation device 500 may include:

[0294] An acquisition unit 510 is configured to acquire a tactile signal;

[0295] Determination unit 520 is used to determine a first interpolation type based on the tactile signal, wherein the first interpolation type is used to perform signal value interpolation between the signal value of the first tactile event component and the signal value of the second tactile event component in the first tactile event, where the second tactile event component is a tactile event component before or after the first tactile event component.

[0296] In some embodiments, the determining unit 520 is further configured to:

[0297] A first identifier is encoded to obtain a bit stream of the tactile signal, where the first identifier is used to indicate the first interpolation type.

[0298] In some embodiments, the determining unit 520 is further configured to:

[0299] A second identifier or a third identifier is encoded, where the second identifier is used to indicate whether the first identifier exists, and the third identifier is used to indicate whether the first tactile event is a continuous event.

[0300] In some embodiments, the determining unit 520 is specifically configured to:

[0301] encoding the data structure of the first haptic event and the data structure of the first haptic event component to obtain the bitstream;

[0302] The data structure of the first tactile event includes the first identifier, or the data structure of the first tactile event component includes the first identifier.

[0303] In some embodiments, the determining unit 520 is further configured to:

[0304] encapsulating the bit stream to obtain a tactile media file of the tactile signal;

[0305] The haptic media file, a haptic media file segment in the haptic media file, or a haptic media track in the haptic media file is sent to a decapsulation device.

[0306] In some embodiments, the interpolation type of the first haptic event component includes at least one of the following: nearest neighbor interpolation, linear interpolation, cubic spline interpolation, Akma interpolation, Bessel interpolation, B-spline interpolation, Lagrange interpolation, Newton interpolation, and custom algorithm interpolation.

[0307] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions can refer to the method embodiment. Specifically, the tactile signal interpolation device 500 may correspond to the corresponding subject in the method 300 of the embodiment of the present application, and the various units in the tactile signal interpolation device 500 are respectively for implementing the corresponding processes in the method 300. For the sake of brevity, they are not further described here.

[0308] It should also be understood that the various units in the tactile signal interpolation device 400 or 500 involved in the embodiments of the present application are divided based on logical functions. In actual applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. Even these functions can be assisted by one or more other units. For example, part or all of the tactile signal interpolation device 400 or 500 are merged into one or several other units. For another example, a certain unit (or units) in the tactile signal interpolation device 400 or 500 can also be split into multiple functionally smaller units to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. For another example, the tactile signal interpolation device 400 or 500 can also include other units. In actual applications, these functions can also be assisted by other units and can be achieved by the collaboration of multiple units.

[0309] It should also be understood that the term "module" or "unit" involved in the embodiments of the present application refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0310] According to another embodiment of the present application, the tactile signal interpolation device 400 or 500 involved in the embodiment of the present application, and the method of the embodiment of the present application can be constructed by running a computer program (including program code) capable of executing each step involved in the corresponding method on a general computing device of a general-purpose computer including processing elements and storage elements such as a central processing unit (CPU), a random access memory medium (RAM), and a read-only memory medium (ROM). The computer program can be recorded on, for example, a computer-readable storage medium and loaded into an electronic device through a computer-readable storage medium, and the computer program is used to implement the corresponding method of the embodiment of the present application. In other words, the units involved above can be implemented in the form of hardware, or in the form of instructions in the form of software, or in the form of a combination of hardware and software. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software in the decoding processor. Alternatively, the software may be located in a mature storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The software in the memory may be executed by a processor to complete the steps in the method embodiments mentioned above.

[0311] FIG13 is a schematic structural diagram of an electronic device 600 provided in an embodiment of the present application.

[0312] As shown in Figure 13, the electronic device 600 includes at least a processor 610 and a computer-readable storage medium 620. The processor 610 and the computer-readable storage medium 620 may be connected via a bus or other means. The computer-readable storage medium 620 is used to store a computer program 621, which includes computer instructions. The processor 610 is used to execute the computer instructions stored in the computer-readable storage medium 620. The processor 610 is the computing core and control core of the electronic device 600 and is suitable for implementing one or more computer instructions, specifically loading and executing one or more computer instructions to implement the corresponding method flow or corresponding function.

[0313] As an example, the processor 610 may also be referred to as a central processing unit (CPU). The processor 610 may include, but is not limited to, a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, and the like.

[0314] As an example, the computer-readable storage medium 620 may be a high-speed RAM memory, or a non-volatile memory (Non-Volatile Memory), such as at least one disk storage; optionally, it may also be at least one computer-readable storage medium located away from the aforementioned processor 610. Specifically, the computer-readable storage medium 620 includes, but is not limited to: volatile memory and / or non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0315] As shown in FIG. 13 , the electronic device 600 may further include a transceiver 630 .

[0316] The processor 610 may control the transceiver 630 to communicate with other devices. Specifically, the processor 610 may send information or data to other devices or receive information or data sent by other devices. The transceiver 630 may include a transmitter and a receiver. The transceiver 630 may further include one or more antennas.

[0317] It should be understood that the various components in the electronic device 600 are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0318] It is worth noting that the electronic device 600 can be any electronic device with data processing capabilities; the computer-readable storage medium 620 stores a first computer instruction; the processor 610 loads and executes the first computer instruction stored in the computer-readable storage medium 620 to implement the corresponding steps in method 200; in other words, the first computer instruction in the computer-readable storage medium 620 is loaded by the processor 610 and the corresponding steps are executed. To avoid repetition, this description is not repeated here. The computer-readable storage medium 620 stores a second computer instruction; the processor 610 loads and executes the second computer instruction stored in the computer-readable storage medium 620 to implement the corresponding steps in method 300; in other words, the second computer instruction in the computer-readable storage medium 620 is loaded by the processor 610 and the corresponding steps are executed. To avoid repetition, this description is not repeated here.

[0319] According to another aspect of the present application, an embodiment of the present application provides a chip. The chip can be an integrated circuit chip with signal processing capabilities, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The chip can also be called a system-on-chip, a system-on-chip, a chip system, or a system-on-chip chip, etc. The chip can be applied to various electronic devices that can be installed with the chip, so that the device equipped with the chip can execute the corresponding steps in the various methods or logic block diagrams disclosed in the embodiments of the present application. For example, the chip can be suitable for implementing one or more computer instructions, specifically suitable for loading and executing one or more computer instructions to implement the corresponding method flow or corresponding function.

[0320] According to another aspect of the present application, an embodiment of the present application provides a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device of a computer for storing programs and data. It is understandable that the computer-readable storage medium herein may include both built-in storage media in the computer and, of course, extended storage media supported by the computer. The computer-readable storage medium provides a storage space that stores an operating system of an electronic device. The storage space stores computer instructions suitable for being loaded and executed by a processor. When the computer instructions are read and executed by the processor of the computer device, the computer device executes the corresponding steps in the various methods or logic block diagrams disclosed in the embodiments of the present application.

[0321] According to another aspect of the present application, an embodiment of the present application provides a computer program product or computer program. The computer program product or computer program includes computer instructions, which are stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the corresponding steps in the various methods or logic block diagrams disclosed in the embodiments of the present application. In other words, when software is used to implement the solution provided by the present application, it can be implemented in whole or in part in the form of a computer program product or computer program. The computer program product or computer program includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, the process of the embodiment of the present application is run in whole or in part or the functions of the embodiment of the present application are implemented.

[0322] It is worth noting that the computer involved in this application can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions involved in this application can be stored in a computer-readable storage medium, or can be transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0323] Those skilled in the art will appreciate that the units and process steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. In other words, professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of protection of this application.

[0324] Finally, it should be noted that the above content is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the present application, which should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner unless there is any contradiction. For another example, the various different embodiments of the present application can also be arbitrarily combined, as long as they do not violate the basic idea of ​​the present application, they should also be regarded as the content disclosed in the present application.

Claims

1. A tactile signal interpolation method, characterized in that: Applications in signal processing equipment, including: Determining an interpolation type of a first haptic event component in a first haptic event by decoding a bit stream of a haptic signal; Based on the interpolation type of the first tactile event component, signal value interpolation is performed between the signal value of the first tactile event component and the signal value of a second tactile event component to obtain the tactile signal, and the second tactile event component is a tactile event component before or after the first tactile event component.

2. The method according to claim 1, characterized in that The step of determining the interpolation type of the first tactile event component in the first tactile event by decoding the bit stream of the tactile signal comprises: Obtaining a first identifier by decoding the bit stream, where the first identifier is used to indicate an interpolation type; Based on the interpolation type indicated by the first identifier, an interpolation type of the first haptic event component is determined.

3. The method according to claim 2, characterized in that The step of obtaining a first identifier by decoding the bit stream includes: Obtaining a second identifier by decoding the bit stream, where the second identifier is used to indicate whether the first identifier exists; In a case where the second identifier indicates that the first identifier exists, the first identifier is acquired by decoding the bit stream.

4. The method according to claim 2 or 3, characterized in that: The step of obtaining a first identifier by decoding the bit stream includes: Obtaining a third identifier by decoding the bit stream, where the third identifier is used to indicate whether the first tactile event is a continuous event; In a case where the third identifier indicates that the first tactile event is the continuous event, the first identifier is acquired by decoding the bit stream.

5. The method according to any one of claims 2 to 4, characterized in that The step of obtaining a first identifier by decoding the bit stream includes: Obtaining a data structure of the first tactile event by decoding the bit stream, where the data structure of the first tactile event includes the first identifier; The step of determining the interpolation type of the first haptic event component based on the interpolation type indicated by the first identifier includes: The interpolation type indicated by the first identifier is determined as the interpolation type of some or all tactile event components in the first tactile event, and the some or all tactile event components include the first tactile event component.

6. The method according to any one of claims 2 to 5, characterized in that The step of obtaining a first identifier by decoding the bit stream includes: Obtaining a data structure of the first haptic event component by decoding the bit stream, where the data structure of the first haptic event component includes the first identifier; The step of determining the interpolation type of the first haptic event component based on the interpolation type indicated by the first identifier includes: The interpolation type indicated by the first identifier is determined as the interpolation type of the first haptic event component.

7. The method according to any one of claims 2 to 6, characterized in that The method further comprises: The interpolation type indicated by the first identifier is determined as the interpolation type of the tactile event component after the first tactile event component, until the data structure in the first tactile event includes a new tactile event component with the first identifier or the last tactile event component in the first tactile event.

8. The method according to any one of claims 1 to 7, characterized in that The performing signal value interpolation between the signal value of the first tactile event component and the signal value of the second tactile event component based on the interpolation type of the first tactile event component to obtain the tactile signal includes: Based on the signal value of the first haptic event component and using the interpolation type of the first haptic event component, interpolation is performed between the signal value of the first haptic event component and the signal value of the second haptic event component to obtain the haptic signal.

9. The method according to any one of claims 1 to 8, characterized in that The interpolation type of the first tactile event component includes at least one of the following: nearest neighbor interpolation, linear interpolation, cubic spline interpolation, Akma interpolation, Bessel interpolation, B-spline interpolation, Lagrange interpolation, Newton interpolation, and custom algorithm interpolation.

10. The method according to any one of claims 1 to 9, characterized in that Before determining the interpolation type of the first tactile event component in the first tactile event, the method further includes: Receive a tactile media file sent by a packaging device, and decapsulate the tactile media file to obtain the bit stream; or receive a tactile media segment sent by a packaging device, and decapsulate the tactile media segment to obtain the bit stream; or receive a tactile media track sent by a file packaging device, and decapsulate the tactile media track to obtain the bit stream.

11. A tactile signal interpolation method, characterized in that: Applied to a signal processing device, the method comprises: Acquiring tactile signals; Based on the tactile signal, a first interpolation type is determined, wherein the first interpolation type is used to perform signal value interpolation between a signal value of a first tactile event component and a signal value of a second tactile event component in a first tactile event, wherein the second tactile event component is a tactile event component before or after the first tactile event component.

12. The method according to claim 11, characterized in that The method further comprises: A first identifier is encoded to obtain a bit stream of the tactile signal, wherein the first identifier is used to indicate the first interpolation type.

13. The method according to claim 12, characterized in that The method further comprises: Encode a second identifier or a third identifier, wherein the second identifier is used to indicate whether the first identifier exists, and the third identifier is used to indicate whether the first tactile event is a continuous event.

14. The method according to claim 12 or 13, characterized in that The encoding of the first identifier to obtain a bit stream of the tactile signal includes: encoding the data structure of the first tactile event and the data structure of the first tactile event component to obtain the bit stream; The data structure of the first tactile event includes the first identifier, or the data structure of the first tactile event component includes the first identifier.

15. The method according to any one of claims 11 to 14, characterized in that The method further comprises: encapsulating the bit stream to obtain a tactile media file of the tactile signal; The haptic media file, a haptic media file segment in the haptic media file, or a haptic media track in the haptic media file is sent to a decapsulation device.

16. The method according to any one of claims 11 to 15, characterized in that The interpolation type of the first tactile event component includes at least one of the following: nearest neighbor interpolation, linear interpolation, cubic spline interpolation, Akma interpolation, Bessel interpolation, B-spline interpolation, Lagrange interpolation, Newton interpolation, and custom algorithm interpolation.

17. A tactile signal interpolation device, characterized in that: include: a determining unit, configured to determine an interpolation type of a first tactile event component in a first tactile event by decoding a bit stream of a tactile signal; An interpolation unit is used to interpolate signal values ​​between the signal value of the first tactile event component and the signal value of the second tactile event component based on the interpolation type of the first tactile event component, and obtain the tactile signal, wherein the second tactile event component is a tactile event component before or after the first tactile event component.

18. A tactile signal interpolation device, characterized in that: include: An acquisition unit, used for acquiring a tactile signal; A determination unit is used to determine a first interpolation type based on the tactile signal, wherein the first interpolation type is used to perform signal value interpolation between a signal value of a first tactile event component and a signal value of a second tactile event component in a first tactile event, wherein the second tactile event component is a tactile event component before or after the first tactile event component.

19. An electronic device, characterized in that: include: a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein when the computer program is executed by the processor, the tactile signal interpolation method according to any one of claims 1 to 10 is implemented, or the tactile signal interpolation method according to any one of claims 11 to 16 is implemented.

20. A readable storage medium, characterized in that: The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by the processor, the tactile signal interpolation method described in any one of claims 1 to 10 is implemented, or the tactile signal interpolation method described in any one of claims 11 to 16 is implemented.

21. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the tactile signal interpolation method according to any one of claims 1 to 10, or implement the tactile signal interpolation method according to any one of claims 11 to 16.

Images