Tactile media file encapsulation method, tactile media file de-encapsulation method and corresponding equipment

By encapsulating the haptic media bitstream into multiple tracks according to preset data classification standards, each track containing different types of data and first metadata, the problem of only single track packaging in the prior art is solved, and the haptic media is improved Data flexibility and application effect.

CN119002685APending Publication Date: 2024-11-22TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310578365.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-19
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The prior art supports only encapsulating haptic media content into a single track, limiting the flexibility of haptic media content applications.

Method used

By obtaining the haptic media bitstream of the target haptic media signal and encapsulating it into multiple tracks according to preset data classification criteria, each track containing different types of data and corresponding first metadata, for indicating the data type.

Benefits of technology

The track to be decoded can be selected according to the first metadata of each track during the decoding phase, improving the flexibility of the application of tactile media data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119002685A_ABST
    Figure CN119002685A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a tactile media file encapsulation method, a tactile media file de-encapsulation method and corresponding equipment. The tactile media file encapsulation method comprises the following steps: acquiring a tactile media bit stream corresponding to a target tactile media signal; packaging the tactile media bit stream into a corresponding tactile media file; wherein the tactile media file comprises at least two tracks, each track comprises data of one or more data types in the tactile media bit stream and first metadata of the data, the data types are obtained through division based on a preset data classification standard, and the first metadata is used for indicating the data type of the data. According to the scheme, the tactile media bit stream is encapsulated into a plurality of tracks in the encapsulation stage, so that the track to be decoded can be selected according to the data type indicated in the first metadata of each track in the decoding stage, and the flexibility of tactile media data application is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of tactile media encoding and decoding, and in particular, to a tactile media file encapsulation and decapsulation method and corresponding equipment. Background Art

[0002] The presentation of immersive media content is often accompanied by a variety of wearable devices or interactive devices. Therefore, in terms of the presentation of immersive media, in addition to the traditional visual and auditory presentation, there is also a new presentation method, tactile presentation. Tactile presentation allows users to receive information through their bodies through a tactile presentation mechanism that combines hardware and software, providing an embedded physical sensation and delivering key information about the system the user is using. For example, a mobile phone will vibrate to remind its owner that a message has been received. This vibration is a type of tactile presentation. Tactile presentation can enhance auditory and visual presentation and improve user experience.

[0003] In the current industry, tactile information presentation has been widely used and is usually divided into the following categories: vibrotactile, which directly presents tactile sensations in the form of vibrations; kinematic tactile, which simulates the weight or pressure of an object; and electrotactile, which simulates feedback information of a specific texture through electrical stimulation. It is not difficult to see from the current tactile presentation technology in the industry that, from the vibration tactile of commonly used devices in daily life to the diversified tactile presentation in niche areas, tactile presentation itself is already a presentation method that users are accustomed to. With the popularization of wearable devices and interactive devices, the tactile presentation that users can perceive when consuming media content will no longer be limited to basic vibrotactile, but will include all-round physical sensations such as vibration, pressure, speed, acceleration, temperature, humidity, and smell, which are closer to the real world.

[0004] When transmitting tactile media content, similar to audio and video media, the sender needs to encode and encapsulate the tactile media content, and then the receiver decapsulates and decodes it to obtain the tactile media content. However, the existing technology only supports encapsulating tactile media content into a single track, which limits the flexibility of tactile media content application. Summary of the invention

[0005] The purpose of this application is to solve at least one of the above technical defects. The technical solutions provided by the embodiments of this application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a tactile media file packaging method, comprising:

[0007] Acquire a tactile media bit stream corresponding to a target tactile media signal;

[0008] encapsulating the haptic media bitstream into a corresponding haptic media file;

[0009] The tactile media file includes at least two tracks, and each track includes data of one or more data types in the tactile media bitstream and first metadata of the data, the data types are classified based on a preset data classification standard, and the first metadata is used to indicate the data type of the data.

[0010] In an optional embodiment of the present application, the first metadata includes quantity information of data types of data in the track and identification information of the data types.

[0011] In an optional embodiment of the present application, the preset data classification standard includes any of the following:

[0012] The type of perception to which the data corresponds;

[0013] The channel type corresponding to the data;

[0014] The type of hierarchy the data corresponds to;

[0015] The priority type corresponding to the data.

[0016] In an optional embodiment of the present application, if the preset data classification standard is the perceptual type corresponding to the data, the first metadata also includes attribute information of the perceptual type of the data in the track.

[0017] In an optional embodiment of the present application, if the preset data classification standard is the channel type corresponding to the data, the first metadata also includes channel group flag information of the data in the track, and the channel group flag information is used to indicate whether the data belongs to the same channel group.

[0018] In an optional embodiment of the present application, the first metadata also includes identification information of the perceptual type of the data in the track and attribute information of the perceptual type.

[0019] In an optional embodiment of the present application, the first metadata is stored in a data box corresponding to a track-level sample entry of the track, and the first metadata includes indication information of a preset data classification standard.

[0020] In an optional embodiment of the present application, the first metadata is stored in a data box corresponding to a data type level sample entry of the track, and the data box corresponding to the data type level sample entry contains indication information of a preset data classification standard.

[0021] In an optional embodiment of the present application, the track includes one or more specific sample groups, and each specific sample group includes one specific sample or multiple continuous specific samples in the track.

[0022] In an optional embodiment of the present application, the specific sample does not include tactile media data.

[0023] In an optional embodiment of the present application, the tactile media file further includes a metadata track, the metadata track includes second metadata of the tactile media bitstream, and the second metadata is used to indicate encoding information of the tactile media bitstream.

[0024] In an optional embodiment of the present application, the track further includes second metadata of the tactile media bitstream, and the second metadata is used to indicate encoding information of the tactile media bitstream.

[0025] In a second aspect, an embodiment of the present application provides a tactile media file decapsulation method, comprising:

[0026] Acquire a tactile media file, wherein the tactile media file comprises at least two tracks, and each track comprises data of one or more data types in a tactile media bitstream and first metadata of the data, the data types are obtained based on a preset data classification standard, and the first metadata is used to indicate the data type of the data;

[0027] One or more target tracks are determined based on the first metadata, and the target tracks are decapsulated to obtain corresponding haptic media bitstreams.

[0028] In a third aspect, an embodiment of the present application provides a tactile media file packaging device, including:

[0029] A tactile media bit stream acquisition module, used to acquire a tactile media bit stream corresponding to a target tactile media signal;

[0030] A tactile media bitstream encapsulation module, used for encapsulating the tactile media bitstream into a corresponding tactile media file;

[0031] The tactile media file includes at least two tracks, and each track includes data of one or more data types in the tactile media bitstream and first metadata of the data, the data types are classified based on a preset data classification standard, and the first metadata is used to indicate the data type of the data.

[0032] In an optional embodiment of the present application, the first metadata includes quantity information of data types of data in the track and identification information of the data types.

[0033] In an optional embodiment of the present application, the preset data classification standard includes any of the following:

[0034] The type of perception to which the data corresponds;

[0035] The channel type corresponding to the data;

[0036] The type of hierarchy the data corresponds to;

[0037] The priority type corresponding to the data.

[0038] In an optional embodiment of the present application, if the preset data classification standard is the perceptual type corresponding to the data, the first metadata also includes attribute information of the perceptual type of the data in the track.

[0039] In an optional embodiment of the present application, if the preset data classification standard is the channel type corresponding to the data, the first metadata also includes channel group flag information of the data in the track, and the channel group flag information is used to indicate whether the data belongs to the same channel group.

[0040] In an optional embodiment of the present application, the first metadata also includes identification information of the perceptual type of the data in the track and attribute information of the perceptual type.

[0041] In an optional embodiment of the present application, the first metadata is stored in a data box corresponding to a track-level sample entry of the track, and the first metadata includes indication information of a preset data classification standard.

[0042] In an optional embodiment of the present application, the first metadata is stored in a data box corresponding to a data type level sample entry of the track, and the data box corresponding to the data type level sample entry contains indication information of a preset data classification standard.

[0043] In an optional embodiment of the present application, the track includes one or more specific sample groups, and each specific sample group includes one specific sample or multiple continuous specific samples in the track.

[0044] In an optional embodiment of the present application, the specific sample does not include tactile media data.

[0045] In an optional embodiment of the present application, the tactile media file further includes a metadata track, the metadata track includes second metadata of the tactile media bitstream, and the second metadata is used to indicate encoding information of the tactile media bitstream.

[0046] In an optional embodiment of the present application, the track further includes second metadata of the tactile media bitstream, and the second metadata is used to indicate encoding information of the tactile media bitstream.

[0047] In a fourth aspect, an embodiment of the present application provides a tactile media file decapsulation device, comprising:

[0048] a tactile media file acquisition module, configured to acquire a tactile media file, wherein the tactile media file comprises at least two tracks, and each track comprises data of one or more data types in a tactile media bitstream and first metadata of the data, wherein the data types are classified based on a preset data classification standard, and the first metadata is used to indicate the data type of the data;

[0049] The tactile media file decapsulation module is used to determine one or more target tracks based on the first metadata, and decapsulate the target tracks to obtain corresponding tactile media bitstreams.

[0050] In a fifth aspect, an embodiment of the present application provides a packaging device, including a memory and a processor;

[0051] A computer program is stored in the memory;

[0052] A processor is used to execute a computer program to implement the method provided in the embodiment of the first aspect or any optional embodiment of the first aspect.

[0053] In a sixth aspect, an embodiment of the present application provides a decapsulation device, including a memory and a processor;

[0054] A computer program is stored in the memory;

[0055] A processor is used to execute a computer program to implement the method provided in the embodiment of the second aspect or any optional embodiment of the second aspect.

[0056] In the seventh aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method provided in the embodiment of the first aspect, any optional embodiment of the first aspect, the embodiment of the second aspect, or any optional embodiment of the second aspect is implemented.

[0057] In an eighth aspect, an embodiment of the present application provides a computer program product or a computer program, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that when the computer device executes, the method provided in the embodiment of the first aspect, any optional embodiment of the first aspect, the embodiment of the second aspect, or any optional embodiment of the second aspect is implemented.

[0058] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0059] After obtaining the tactile media bitstream corresponding to the target tactile media signal, the tactile media bitstream is encapsulated into multiple tracks according to the data type corresponding to the preset data classification standard, so that each track contains data of one or more data types derived from the tactile media bitstream and first metadata for indicating the data type of the data contained in the track. The solution encapsulates the tactile media bitstream into multiple tracks in the encapsulation stage, so that in the decoding stage, the track to be decoded can be selected according to the data type indicated in the first metadata of each track, thereby improving the flexibility of tactile media data application. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0061] Figure 1 A system structure diagram for implementing a tactile media file encapsulation method and a tactile media decapsulation method in an embodiment of the present application;

[0062] Figure 2 A schematic diagram of a process of packaging a tactile media file provided in an embodiment of the present application;

[0063] Figure 3 A schematic diagram of a tactile media file obtained in an example of an embodiment of the present application;

[0064] Figure 4 A schematic diagram of another tactile media file obtained in an example of an embodiment of the present application;

[0065] Figure 5 A schematic diagram of a silent sample group in an example of an embodiment of the present application;

[0066] Figure 6 A schematic diagram of a second metadata encapsulation method in an example of an embodiment of the present application;

[0067] Figure 7 A schematic diagram of a process for decapsulating a tactile media file provided in an embodiment of the present application;

[0068] Figure 8 A structural block diagram of a tactile media file packaging device is provided for an embodiment of the present application;

[0069] Fig. 9 A structural block diagram of a tactile media file decapsulation device is provided for an embodiment of the present application;

[0070] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0072] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" may be implemented as "A", or as "B", or as "A and B".

[0073] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0074] First, several terms involved in this application are introduced and explained:

[0075] Haptics: Haptics is the sensory experience that the human body obtains through contact, such as vibration, pressure, temperature, etc.

[0076] Haptics Signal: A haptics signal used to represent a specific modal tactile experience and is rendered on a specific device.

[0077] JSON (Java Script Object Notation): Object notation is a lightweight data exchange format. It uses a text format that is completely independent of programming languages ​​to store and represent data. The simplicity and clear hierarchical structure make JSON an ideal data exchange language, which is easy for people to read and write, and easy for machines to parse and generate, and effectively improves network transmission efficiency.

[0078] Bitstream, or code stream: Bitstream refers to the binary data stream that has been compressed and encoded.

[0079] Track: A track is a collection of media data in the process of media file encapsulation, consisting of multiple time-sequence samples. A media file can consist of one or more tracks. For example, a media file can contain a video media track, an audio media track, and a subtitle media track. In particular, metadata information can also be included in the file as a media type in the form of a metadata media track.

[0080] Sample: Sample is the encapsulation unit in the process of media file encapsulation. A track consists of many samples, each of which corresponds to specific timestamp information. For example, a video media track can consist of many samples, and a sample is usually a video frame. In the embodiment of the present application, a sample in a track can be one or more tactile signals.

[0081] Sample number: Sample Number. The first sample in the track has a number of 1.

[0082] Sample Entry: Sample Entry is used to indicate metadata information related to all samples in the track. For example, the sample entry of a video track usually contains metadata information related to decoder initialization.

[0083] Sample Group: Sample Group is used to group some samples in a track according to specific rules.

[0084] Media Segment: A playable segment that conforms to a certain media format. When playing, it may need to be coordinated with zero or more segments preceding it and the initialization segment.

[0085] Figure 1 1 is a system structure diagram for implementing the tactile media file encapsulation method and the tactile media decapsulation method in the embodiment of the present application, such as Figure 1 As shown, the system may be an immersive system, which may include a server 101 and multiple terminal devices 102, wherein the terminal device 102 may be a mobile phone, a tablet (Pad) or other gaming device, and these terminal devices may present corresponding tactile media signals through corresponding sensors.

[0086] The service provider can collect or produce relevant tactile media signals through the server, and then encode and encapsulate the collected tactile media signals through the server to obtain the corresponding tactile media files, and send the obtained tactile media files to the playback end, and obtain the corresponding tactile media information through the terminal device after decapsulating and decoding the received tactile media files. The user can feel the tactile media information through the terminal device to achieve an immersive experience. It can be understood that when the encapsulation operation is performed through the server, the tactile media file encapsulation method provided in the embodiment of the present application is adopted, and when the decapsulation operation is performed through the player, the tactile media file decapsulation method provided in the embodiment of the present application is adopted.

[0087] Figure 2 A schematic diagram of a tactile media file packaging method provided in an embodiment of the present application, the execution subject of the method may be Figure 1Servers in Figure 2 As shown, the method may include:

[0088] Step S201 : acquiring a haptic media bitstream corresponding to a target haptic media signal.

[0089] Among them, the target tactile media signal is the tactile media to be encapsulated. The tactile media can be vibrotactile, kinematic tactile or electrotactile. Specifically, vibrotactile can simulate vibrations of specific frequencies and intensities through the vibration of the motor of the terminal device. For example, in a shooting game, vibration is used to simulate the specific effects of the use of props. Kinematic tactile simulates the weight or pressure of an object through a kinematic tactile system. For example, in a driving video game, when moving at a higher speed or operating a heavier vehicle, the steering wheel may resist turning. This type of feedback directly affects the user's muscles. In the example of a driving game, the user must apply more force to get the desired response from the steering wheel. Electrotactile presentation uses electrical pulses to provide tactile stimulation to the nerve endings of the user's skin. Electrotactile presentation can create a highly realistic experience for users wearing suits or gloves equipped with electrotactile technology. Many sensations can be simulated with electrical pulses, such as: temperature changes, pressure changes, and wetness.

[0090] Specifically, the server obtains the target tactile media signal by collecting or producing it according to the expected tactile media effect. For example, if it is necessary to produce a specific effect of using a prop in a shooting game by vibration simulation, the corresponding vibration tactile signal is collected or produced. Then, the server converts the target tactile media signal into a corresponding exchange format, namely, a tactile media exchange format. The tactile media exchange format can be in JSON format. Then, the server compresses (i.e., encodes) the target tactile media signal in the exchange format to obtain a corresponding tactile media bit stream (i.e., tactile media code stream).

[0091] Step S202 : encapsulating the haptic media bitstream into a corresponding haptic media file.

[0092] The tactile media file includes at least two tracks, and each track includes data of one or more data types in the tactile media bitstream and first metadata of the data, the data types are classified based on a preset data classification standard, and the first metadata is used to indicate the data type of the data. It should be noted that multiple tracks from the same tactile media bitstream can also become multiple component tracks.

[0093] The preset data classification standard may include one of a perception type corresponding to the data, a channel type corresponding to the data, a level type corresponding to the data, and a priority type corresponding to the data. In other words, the data may be classified according to the perception type, channel type, level type, and priority type. For example, if the preset data classification standard is the perception type corresponding to the data, then the data types classified according to the standard may include vibration perception, temperature perception, etc.

[0094] Specifically, after obtaining the tactile media bitstream through encoding, the server can also organize the tactile media bitstream to obtain a tactile media transmission stream that is easy to transmit, and then the server encapsulates the tactile media transmission stream to encapsulate multiple tracks, and adds corresponding first metadata in each track. Specifically, after obtaining the tactile media bitstream (or the tactile media transmission stream corresponding to the tactile media bitstream), various data types corresponding to the data in the tactile media bitstream are determined according to the preset data classification standard. It should be noted that each time the encapsulation only selects one data classification standard (i.e., the preset classification standard) to determine the various data types corresponding to the data in the tactile media bitstream. Then, the data in the tactile media bitstream is encapsulated according to the data type to obtain multiple tracks, and each track contains data of one or more data types derived from the tactile media bitstream. Then, the corresponding first metadata is added to each track, and the first metadata is at least used to indicate the data type of the data in the track. It can be understood that when decapsulating, the data type in each track can be determined according to the first metadata in the track, and then it is determined whether the data in the track is the required data.

[0095] For example, after obtaining the tactile media bitstream, a preset data classification standard can be selected from the perception type corresponding to the data, the channel type corresponding to the data, the level type corresponding to the data, and the priority type corresponding to the data according to actual needs. For example, "the perception type corresponding to the data" is selected as the preset data classification standard. Then, based on the preset data classification standard, the data type of the data in the tactile media bitstream is determined to include: perception type 1, perception type 2, and perception type 3. Then, the data is packaged according to the data type. Figure 3 As shown, the data corresponding to the perceptual type 1, perceptual type 2 and perceptual type 3 can be respectively encapsulated in track 1, track 2 and track 3, and the corresponding first metadata is added to track 1 to indicate that the data type of the data contained in the track is perceptual type 1, the corresponding first metadata is added to track 2 to indicate that the data type of the data contained in the track is perceptual type 2, and the corresponding first metadata is added to track 3 to indicate that the data type of the data contained in the track is perceptual type 3. Or, as Figure 4As shown, two of the data types can be encapsulated into one track, and the other data type can be encapsulated into one track. For example, the data corresponding to perceptual type 1 and perceptual type 2 can be encapsulated in track 1, and the data corresponding to perceptual type 3 can be encapsulated in track 2. In addition, the corresponding first metadata is added to track 1 to indicate that the data types of the data contained in the track are perceptual type 1 and perceptual type 2, and in track 2, the corresponding first metadata is added to indicate that the data type of the data contained in the track is perceptual type 3.

[0096] The solution provided by the present application, after acquiring the tactile media bitstream corresponding to the target tactile media signal, encapsulates the tactile media bitstream into multiple tracks according to the data type corresponding to the preset data classification standard, so that each track contains data of one or more data types derived from the tactile media bitstream and first metadata for indicating the data type of the data contained in the track. The solution encapsulates the tactile media bitstream into multiple tracks in the encapsulation stage, so that in the decoding stage, the track to be decoded can be selected according to the data type indicated in the first metadata of each track, thereby improving the flexibility of tactile media data application.

[0097] In an optional embodiment of the present application, the first metadata includes quantity information of data types of data in the track and identification information of the data types.

[0098] Specifically, in order to indicate the data type of the data contained in the track, the first metadata in each track may include the quantity information of the data type and the identification information of the data type. Specifically, the quantity information of the data type may be used to indicate the quantity of the data types contained in the track, and the identification information of the data type may be used to indicate the specific type of the data type contained in the tactile media.

[0099] For example, for a certain track, the quantity information of data types in its first metadata indicates that the track contains two data types, and the identification information of the data types indicates that the data types of the data in the track are respectively perceptual type 1 and perceptual type 2. Then, in the decapsulation stage, the number of data types contained in the track is determined by the quantity information of data types in the first metadata of the track, and then it is determined whether to decapsulate the track according to the number of data types. Furthermore, the specific types of data types contained in the track are determined by the identification information of the data types in the first metadata of the track, and then it is determined whether to decapsulate the track according to the specific types of data types.

[0100] In an optional embodiment of the present application, the preset data classification standard includes any of the following:

[0101] The type of perception to which the data corresponds;

[0102] The channel type corresponding to the data;

[0103] The type of hierarchy the data corresponds to;

[0104] The priority type corresponding to the data.

[0105] It should be noted that the data type corresponding to the perception type corresponding to the data may include vibration perception, temperature perception, etc. The channel in the channel type corresponding to the data refers to the channel under a specific perception type. For example, vibration perception may have multiple different channels, and different channels may correspond to devices worn on different parts of the body in the terminal device used by the user. For example, a helmet is a channel under vibration perception, and a glove is another channel under vibration perception. Therefore, the data type corresponding to the channel type corresponding to the data may have different channel types under a specific perception type. The data type corresponding to the hierarchical type corresponding to the data may have different hierarchical types. These hierarchical types are divided into high and low hierarchical types. The characteristic is that they need to be decapsulated from low to high hierarchical levels in the decapsulation stage. For example, for two tracks encapsulated by hierarchical types, if they need to be decapsulated during decapsulation, the track with the lower hierarchical type is decapsulated first, and then the track with the higher hierarchical type is decapsulated. The priority type corresponding to the data may have different priority types. These priority types are divided into high and low priorities. Similar to the hierarchical type, the characteristic is that they need to be decapsulated from high priority to low priority in the decapsulation stage.

[0106] It should be noted that the preset data classification standards are not limited to the above four categories. Other data classification standards can also be adopted according to needs. Its role is to set a unified standard to divide the data types in the tactile media bit stream, and then different data types can be packaged into different tracks during the packaging stage.

[0107] In an optional embodiment of the present application, if the preset data classification standard is the perceptual type corresponding to the data, the first metadata also includes attribute information of the perceptual type of the data in the track.

[0108] Specifically, if the preset data classification standard is the perceptual type corresponding to the data, then in the process of encapsulating the tactile media bitstream, each data is encapsulated into different tracks according to the perceptual type of the data. As can be seen from the foregoing description, if the preset data classification standard is the perceptual type corresponding to the data, then the first metadata of each encapsulated track may include the quantity information of the perceptual type and the identification information of the perceptual type. Furthermore, the first metadata may also include the attribute information of the perceptual type. Specifically, the attribute information of the perceptual type may be a numerical value, and the numerical value corresponds to a characteristic attribute. For example, the corresponding relationship may be as shown in Table 1:

[0109] Table 1

[0110] The value corresponding to the attribute information property 0 Undefined 1 pressure 2 Acceleration 3 speed 4 Location 5 temperature 6 Vibrotactile 7 humidity 8 wind 9 Kinesthetic sense 10 Tactile Texture 11 rigidity 12 Friction

[0111] It should be noted that an attribute of a perceptual type can correspond to one or more perceptual types, that is, an attribute of a perceptual type can correspond to one or more perceptual type identifiers, for example, the perceptual types: temperature type 1, temperature type 2 and temperature type 3 have the attribute of "temperature" as the perceptual type.

[0112] In an optional embodiment of the present application, if the preset data classification standard is the channel type corresponding to the data, the first metadata also includes channel group flag information of the data in the track, and the channel group flag information is used to indicate whether the data belongs to the same channel group.

[0113] Specifically, if the preset data classification standard is the channel type corresponding to the data, then in the process of encapsulating the tactile media bitstream, each data is encapsulated into different tracks according to the channel type of the data. For a track containing data of multiple channel types, the first metadata of the track may also include channel group flag information, and the channel group flag information is used to indicate whether the multiple channel types in the track belong to the same channel group. The channel group can be pre-divided, which can provide richer data information for the subsequent decapsulation process and further improve the flexibility of decapsulation. Specifically, the channel group flag information can be a specific value. For example, if the value corresponding to the channel group flag information is 1, it means that the multiple channel types in the track belong to the same channel group. If the value corresponding to the channel group flag information is 0, it means that the multiple channel types in the track do not belong to the same channel group.

[0114] In an optional embodiment of the present application, the first metadata also includes identification information of the perceptual type of the data in the track and attribute information of the perceptual type.

[0115] Specifically, as can be seen from the foregoing description, a specific perceptual type may include multiple different channel types. Therefore, after the data in the tactile media bitstream is encapsulated into different tracks according to the channel type, the perceptual type information to which the channel type is contained may be added to the first metadata corresponding to each track. Specifically, the identification information of the perceptual type and the attribute information of the perceptual type may be added.

[0116] In an optional embodiment of the present application, the first metadata is stored in a data box corresponding to a track-level sample entry of the track, and the first metadata includes indication information of a preset data classification standard.

[0117] The data structure of the track stores the first metadata of the track through a data box corresponding to the sample entry.

[0118] Specifically, a track-level sample entry may be defined, and then each track may specify corresponding first metadata in a data box of the track-level sample entry. The data box of the track-level sample entry may specify information of a preset data classification standard (or component type), information of the number of data types (or component number), identification information of data types, and other data type related information.

[0119] The information of the preset data classification standard may be a specific value, and each specific value corresponds to a specific preset data classification standard. The corresponding relationship may be shown in Table 2 below:

[0120] Table 2

[0121]

[0122] Then, in the decapsulation stage, the corresponding preset data classification standard can be determined by combining the information of the preset data classification standard in the first metadata in the sample entry of each track with Table 2.

[0123] In an optional embodiment of the present application, the first metadata is stored in a data box corresponding to a data type level sample entry of the track, and the data box corresponding to the data type level sample entry contains indication information of a preset data classification standard.

[0124] Specifically, data type level sample entries may be defined, such as perception type sample entry, channel type sample entry, level type sample entry, and priority type sample entry. Then, each track may specify the corresponding first metadata in the data type level sample entry corresponding to the track. Compared with the track level sample entry, the data type level sample entry itself can indicate the preset data classification standard adopted. Therefore, the data box of the data type level sample entry may specify the quantity information of the data type, the identification information of the data type, and other data type related information.

[0125] For example, if a track contains data of perceptual type 1 and perceptual type 2, if the track adopts track-level sample entry, then the data box of the track-level sample entry can specify information of the preset data classification standard (or component type): the perceptual type corresponding to the data, the number of data types (or component number): 2, identification information of the data type: perceptual type 1 and perceptual type 2, and other data type related information: perceptual type attributes. If the track adopts data type-level sample entry, i.e., perceptual type sample entry, then the data box of the data type-level sample entry can specify information of the number of data types: 2, identification information of data types: perceptual type 1 and perceptual type 2, and other data type related information: perceptual type attributes.

[0126] In an optional embodiment of the present application, the track includes one or more specific sample groups, and each specific sample group includes one specific sample or multiple continuous specific samples in the track.

[0127] Specifically, the data in the track is packaged in units of samples, and the track may include one or more specific sample groups. Then, the corresponding track may be marked by specific sample group information, and the specific sample group information includes a sequence number of a specific sample.

[0128] In an optional embodiment of the present application, the specific sample does not include tactile media data.

[0129] Among them, the specific sample does not contain tactile media data, which can also be called a silent sample, and the corresponding specific sample group can also be called a silent sample group. If the samples therein do not contain tactile media data, then the silent sample group can be directly skipped in the decoding stage without being processed.

[0130] In the prior art, although a silent unit is defined, that is, a silent unit is used in a tactile media bitstream to indicate that there is no tactile signal for a certain period of time. However, when the tactile media bitstream is packaged into multiple tracks, different tracks may have different silent times. Figure 5 As shown, when multiple tracks are packaged, the corresponding silent time in different tracks may be different, and may be different from that in single track packaging. Therefore, the embodiment of the present application defines a silent sample group to identify samples that do not contain tactile signal data in the track. The samples corresponding to the silent sample group of the sample group type are silent samples, that is, the samples do not contain tactile signal data and can be directly skipped during decoding.

[0131] In an optional embodiment of the present application, the tactile media file further includes a metadata track, the metadata track includes second metadata of the tactile media bitstream, and the second metadata is used to indicate encoding information of the tactile media bitstream.

[0132] In an excerpted embodiment of the present application, the track further includes second metadata of the tactile media bitstream, and the second metadata is used to indicate encoding information of the tactile media bitstream.

[0133] Specifically, Figure 6 As shown, the second metadata can be separately encapsulated as a metadata track, so the metadata track needs to be decapsulated and decoded in the decoding stage, or each track can contain a copy of the second metadata, so each track contains the first metadata and the second metadata.

[0134] Figure 7A flowchart of a tactile media file decapsulation method provided in an embodiment of the present application, the execution subject of the method may be Figure 1 Terminal devices in Figure 7 As shown, the method may include:

[0135] Step S701, obtaining a tactile media file.

[0136] The tactile media file includes at least two tracks, and each track includes data of one or more data types in the tactile media bitstream and first metadata of the data, the data types are classified based on a preset data classification standard, and the first metadata is used to indicate the data type of the data.

[0137] The preset data classification standard may include one of a perception type corresponding to the data, a channel type corresponding to the data, a level type corresponding to the data, and a priority type corresponding to the data. In other words, the data may be classified according to the perception type, channel type, level type, and priority type. For example, if the preset data classification standard is the perception type corresponding to the data, then the data types classified according to the standard may include vibration perception, temperature perception, etc.

[0138] Specifically, the tactile media file is obtained from the server, and the server obtains the tactile media file in the following manner:

[0139] After obtaining the tactile media bitstream through encoding, the server can also organize the tactile media bitstream to obtain a tactile media transmission stream that is easy to transmit, and then the server encapsulates the tactile media transmission stream to encapsulate multiple tracks, and adds corresponding first metadata in each track. Specifically, after obtaining the tactile media bitstream (or the tactile media transmission stream corresponding to the tactile media bitstream), various data types corresponding to the data in the tactile media bitstream are determined according to the preset data classification standard. It should be noted that each time the encapsulation only selects one data classification standard (i.e., the preset classification standard) to determine the various data types corresponding to the data in the tactile media bitstream. Then, the data in the tactile media bitstream is encapsulated according to the data type to obtain multiple tracks, and each track contains data of one or more data types derived from the tactile media bitstream. Then, the corresponding first metadata is added to each track, and the first metadata is at least used to indicate the data type of the data in the track. It can be understood that when decapsulating, the data type in each track can be determined according to the first metadata in the track, and then it is determined whether the data in the track is the required data.

[0140] For example, after obtaining the tactile media bitstream, a preset data classification standard can be selected from the perception type corresponding to the data, the channel type corresponding to the data, the level type corresponding to the data, and the priority type corresponding to the data according to actual needs. For example, "the perception type corresponding to the data" is selected as the preset data classification standard. Then, based on the preset data classification standard, the data type of the data in the tactile media bitstream is determined to include: perception type 1, perception type 2, and perception type 3. Then, the data is packaged according to the data type. Figure 3 As shown, the data corresponding to the perceptual type 1, perceptual type 2 and perceptual type 3 can be respectively encapsulated in track 1, track 2 and track 3, and the corresponding first metadata is added to track 1 to indicate that the data type of the data contained in the track is perceptual type 1, the corresponding first metadata is added to track 2 to indicate that the data type of the data contained in the track is perceptual type 2, and the corresponding first metadata is added to track 3 to indicate that the data type of the data contained in the track is perceptual type 3. Or, as Figure 4 As shown, two of the data types can be encapsulated into one track, and the other data type can be encapsulated into one track. For example, the data corresponding to perceptual type 1 and perceptual type 2 can be encapsulated in track 1, and the data corresponding to perceptual type 3 can be encapsulated in track 2. In addition, the corresponding first metadata is added to track 1 to indicate that the data types of the data contained in the track are perceptual type 1 and perceptual type 2, and in track 2, the corresponding first metadata is added to indicate that the data type of the data contained in the track is perceptual type 3.

[0141] Step S702: determine one or more target tracks based on the first metadata, and decapsulate the target tracks to obtain corresponding tactile media bitstreams.

[0142] Specifically, the terminal device determines the target track from each track according to the first metadata in each track, combined with the device capability and consumption preference of the terminal device, then decapsulates the target track to obtain the corresponding tactile media bitstream, further decodes the obtained tactile media bitstream to obtain the tactile media signal in the exchange format, and renders and presents it.

[0143] The solution provided by the present application, after acquiring the tactile media bitstream corresponding to the target tactile media signal, encapsulates the tactile media bitstream into multiple tracks according to the data type corresponding to the preset data classification standard, so that each track contains data of one or more data types derived from the tactile media bitstream and first metadata for indicating the data type of the data contained in the track. The solution encapsulates the tactile media bitstream into multiple tracks in the encapsulation stage, so that in the decoding stage, the track to be decoded can be selected according to the data type indicated in the first metadata of each track, thereby improving the flexibility of tactile media data application.

[0144] The following is an example to illustrate the encapsulation and decapsulation method of the tactile media file in the embodiment of the present application. The method may include:

[0145] (1) The server produces or collects tactile media signals according to the expected tactile media effects, and generates a tactile media signal exchange format.

[0146] (2) The server compresses the haptic media signal into a haptic media bit stream.

[0147] (3) The server organizes the tactile media bit stream into a transmission stream that is easy to transmit.

[0148] (4) The server encapsulates the transport stream into a tactile media file, which contains two tracks, Track1 and Track2, corresponding to the vibration perception type and temperature perception type respectively. The corresponding track should have the following first metadata:

[0149] Track1: The sample entry type is a track-level sample entry. The information specified in the sample entry data box is as follows:

[0150] {Information of preset data classification standard = 0; Number of data types = 1; Identification information of perceptual type = 1; Attribute information of perceptual type = 5}

[0151] At the same time, the sample entry of track1 includes the second metadata.

[0152] At the same time, the silent sample group information is used to identify the silent samples in track1, assuming that they are samples 100 to 200.

[0153] Track2: The sample entry type is a data type level sample entry. The information specified in the data box of the sample entry is as follows:

[0154] {Information of preset data classification standard = 0; Number of data types = 1; Identification information of perceptual type = 2; Attribute information of perceptual type = 6}

[0155] At the same time, the second metadata is repeatedly included in the sample entry of track1.

[0156] At the same time, the silent sample group information is used to identify the silent samples in track2, assuming that they are sample100 to sample400.

[0157] (5) The server may slice the tactile media file into multiple tactile media file segments, each of which includes a complete set of the first metadata and the second metadata.

[0158] (6) The terminal device obtains the tactile media file or a tactile media file segment.

[0159] (7) The terminal device selects the corresponding track (i.e., the target track) for decapsulation and decoding based on the metadata in the tactile media file and the user's device capabilities and consumption preferences, and then obtains the exchange format of the tactile media signal and renders it.

[0160] For example, the device of user 1 does not support temperature-related tactile presentation, so only the tactile media data in track 2 is decoded for presentation, and according to the information of the silent sample group, sample 100 to sample 400 are directly skipped and not decoded.

[0161] For example, if the device of user 2 supports the perception types corresponding to track 1 and track 2, the tactile media data in track 1 and track 2 are decoded for presentation. And according to the information of the silent sample group, sample 100 to sample 200 in track 1 are skipped, and sample 100 to sample 400 in track 2 are not decoded.

[0162] In particular, if in streaming mode, for user 1, it is also possible to choose to transmit only the media resources corresponding to track 2, thereby further saving bandwidth resources.

[0163] Figure 8 A structural block diagram of a tactile media file packaging device is provided for an embodiment of the present application, such as Figure 8 As shown, the device 800 may include: a tactile media bitstream acquisition module 801 and a tactile media bitstream encapsulation module 802, wherein:

[0164] The tactile media bitstream acquisition module 801 is used to acquire the tactile media bitstream corresponding to the target tactile media signal;

[0165] The tactile media bitstream encapsulation module 802 is used to encapsulate the tactile media bitstream into a corresponding tactile media file;

[0166] The tactile media file includes at least two tracks, and each track includes data of one or more data types in the tactile media bitstream and first metadata of the data, the data types are classified based on a preset data classification standard, and the first metadata is used to indicate the data type of the data.

[0167] The solution provided by the present application, after acquiring the tactile media bitstream corresponding to the target tactile media signal, encapsulates the tactile media bitstream into multiple tracks according to the data type corresponding to the preset data classification standard, so that each track contains data of one or more data types derived from the tactile media bitstream and first metadata for indicating the data type of the data contained in the track. The solution encapsulates the tactile media bitstream into multiple tracks in the encapsulation stage, so that in the decoding stage, the track to be decoded can be selected according to the data type indicated in the first metadata of each track, thereby improving the flexibility of tactile media data application.

[0168] In an optional embodiment of the present application, the first metadata includes quantity information of data types of data in the track and identification information of the data types.

[0169] In an optional embodiment of the present application, the preset data classification standard includes any of the following:

[0170] The type of perception to which the data corresponds;

[0171] The channel type corresponding to the data;

[0172] The type of hierarchy the data corresponds to;

[0173] The priority type corresponding to the data.

[0174] In an optional embodiment of the present application, if the preset data classification standard is the perceptual type corresponding to the data, the first metadata also includes attribute information of the perceptual type of the data in the track.

[0175] In an optional embodiment of the present application, if the preset data classification standard is the channel type corresponding to the data, the first metadata also includes channel group flag information of the data in the track, and the channel group flag information is used to indicate whether the data belongs to the same channel group.

[0176] In an optional embodiment of the present application, the first metadata also includes identification information of the perceptual type of the data in the track and attribute information of the perceptual type.

[0177] In an optional embodiment of the present application, the first metadata is stored in a data box corresponding to a track-level sample entry of the track, and the first metadata includes indication information of a preset data classification standard.

[0178] In an optional embodiment of the present application, the first metadata is stored in a data box corresponding to a data type level sample entry of the track, and the data box corresponding to the data type level sample entry contains indication information of a preset data classification standard.

[0179] In an optional embodiment of the present application, the track includes one or more specific sample groups, and each specific sample group includes one specific sample or multiple continuous specific samples in the track.

[0180] In an optional embodiment of the present application, the specific sample does not include tactile media data.

[0181] In an optional embodiment of the present application, the tactile media file further includes a metadata track, the metadata track includes second metadata of the tactile media bitstream, and the second metadata is used to indicate encoding information of the tactile media bitstream.

[0182] In an optional embodiment of the present application, the track further includes second metadata of the tactile media bitstream, and the second metadata is used to indicate encoding information of the tactile media bitstream.

[0183] Fig. 9 The present application provides a structural block diagram of a tactile media file decapsulation device, such as Fig. 9 As shown, the device 900 may include: a tactile media file acquisition module 901 and a tactile media file decapsulation module 902, wherein:

[0184] The tactile media file acquisition module 901 is used to acquire a tactile media file, wherein the tactile media file includes at least two tracks, and each track includes data of one or more data types in a tactile media bit stream and first metadata of the data, the data types are obtained based on a preset data classification standard, and the first metadata is used to indicate the data type of the data;

[0185] The haptic media file decapsulation module 902 is used to determine one or more target tracks based on the first metadata, and decapsulate the target tracks to obtain corresponding haptic media bitstreams.

[0186] The solution provided by the present application, after acquiring the tactile media bitstream corresponding to the target tactile media signal, encapsulates the tactile media bitstream into multiple tracks according to the data type corresponding to the preset data classification standard, so that each track contains data of one or more data types derived from the tactile media bitstream and first metadata for indicating the data type of the data contained in the track. The solution encapsulates the tactile media bitstream into multiple tracks in the encapsulation stage, so that in the decoding stage, the track to be decoded can be selected according to the data type indicated in the first metadata of each track, thereby improving the flexibility of tactile media data application.

[0187] The device of the embodiments of the present application can execute the method provided by the embodiments of the present application, and the implementation principles are similar. The actions performed by each module in the device of each embodiment of the present application correspond to the steps in the method of each embodiment of the present application. For the detailed functional description of each module of the device, please refer to the description in the corresponding method shown in the previous text, which will not be repeated here.

[0188] Reference below Fig.10 , which shows an electronic device suitable for implementing the embodiments of the present application (for example, executing Figure 2 or Figure 7 The electronic device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), wearable devices, etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.10 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0189] The electronic device includes: a memory and a processor, the memory is used to store a program for executing the method described in each of the above method embodiments; the processor is configured to execute the program stored in the memory. The processor here can be referred to as the processing device 1001 described below, and the memory can include at least one of the read-only memory (ROM) 1002, the random access memory (RAM) 1003, and the storage device 1008 described below, as shown below:

[0190] like Fig.10 As shown, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the electronic device 1000 are also stored. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0191] Typically, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device 1000 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.10 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0192] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1009, or installed from the storage device 1008, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present application are executed.

[0193] It should be noted that the computer-readable storage medium mentioned above in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0194] In some embodiments, the terminal devices and servers may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0195] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0196] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0197] Acquire a tactile media bitstream corresponding to a target tactile media signal; encapsulate the tactile media bitstream into a corresponding tactile media file; wherein the tactile media file comprises at least two tracks, and each track comprises data of one or more data types in the tactile media bitstream and first metadata of the data, wherein the data types are obtained based on a preset data classification standard, and the first metadata is used to indicate the data type of the data.

[0198] Alternatively, a tactile media file is obtained, wherein the tactile media file includes at least two tracks, and each track includes data of one or more data types in a tactile media bitstream and first metadata of the data, the data types are divided based on a preset data classification standard, and the first metadata is used to indicate the data type of the data; one or more target tracks are determined based on the first metadata, and the target tracks are decapsulated to obtain a corresponding tactile media bitstream.

[0199] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0200] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0201] The modules or units involved in the embodiments of the present application may be implemented by software or hardware. The name of a module or unit does not limit the unit itself in some cases. For example, the first constraint acquisition module may also be described as a "module for acquiring the first constraint".

[0202] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0203] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0204] The embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that when the computer device executes the computer instructions, the following conditions are achieved:

[0205] Acquire a tactile media bitstream corresponding to a target tactile media signal; encapsulate the tactile media bitstream into a corresponding tactile media file; wherein the tactile media file comprises at least two tracks, and each track comprises data of one or more data types in the tactile media bitstream and first metadata of the data, wherein the data types are obtained based on a preset data classification standard, and the first metadata is used to indicate the data type of the data.

[0206] Alternatively, a tactile media file is obtained, wherein the tactile media file includes at least two tracks, and each track includes data of one or more data types in a tactile media bitstream and first metadata of the data, the data types are divided based on a preset data classification standard, and the first metadata is used to indicate the data type of the data; one or more target tracks are determined based on the first metadata, and the target tracks are decapsulated to obtain a corresponding tactile media bitstream.

[0207] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0208] The above descriptions are only some embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A tactile media file packaging method, characterized in that: include: Acquire a tactile media bit stream corresponding to a target tactile media signal; encapsulating the tactile media bitstream into a corresponding tactile media file; The tactile media file includes at least two tracks, and each track includes data of one or more data types in the tactile media bitstream and first metadata of the data, wherein the data types are divided based on a preset data classification standard, and the first metadata is used to indicate the data type of the data.

2. The method according to claim 1, characterized in that: The first metadata includes quantity information of data types of the data in the track and identification information of the data types.

3. The method according to claim 2, characterized in that The preset data classification standard includes any of the following: the type of perception to which the data corresponds; The channel type corresponding to the data; The type of hierarchy corresponding to the data; The priority type corresponding to the data.

4. The method according to claim 3, characterized in that If the preset data classification standard is the perceptual type corresponding to the data, the first metadata further includes attribute information of the perceptual type of the data in the track.

5. The method according to claim 3, characterized in that: If the preset data classification standard is the channel type corresponding to the data, the first metadata also includes channel group flag information of the data in the track, and the channel group flag information is used to indicate whether the data belongs to the same channel group.

6. The method according to claim 5, characterized in that The first metadata also includes identification information of the perceptual type of the data in the track and attribute information of the perceptual type.

7. The method according to any one of claims 1 to 6, characterized in that The first metadata is stored in a data box corresponding to a track-level sample entry of the track, and the first metadata includes indication information of the preset data classification standard.

8. The method according to any one of claims 1 to 6, characterized in that The first metadata is stored in a data box corresponding to a data type level sample entry of the track, and the data box corresponding to the data type level sample entry contains indication information of the preset data classification standard.

9. The method according to claim 1, characterized in that: The track includes one or more specific sample groups, and each specific sample group includes one specific sample or multiple continuous specific samples in the track.

10. The method according to claim 9, characterized in that The specific sample does not include haptic media data.

11. The method according to claim 1, characterized in that: The tactile media file further includes a metadata track, wherein the metadata track includes second metadata of the tactile media bitstream, and the second metadata is used to indicate encoding information of the tactile media bitstream.

12. The method according to claim 1, characterized in that The track also includes second metadata of the tactile media bitstream, where the second metadata is used to indicate encoding information of the tactile media bitstream.

13. A method for decapsulating a tactile media file, characterized in that: include: Acquire a tactile media file, wherein the tactile media file comprises at least two tracks, and each track comprises data of one or more data types in the tactile media bitstream and first metadata of the data, the data types are obtained based on a preset data classification standard, and the first metadata is used to indicate the data type of the data; One or more target tracks are determined based on the first metadata, and the target tracks are decapsulated to obtain corresponding haptic media bitstreams.

14. A tactile media file packaging device, characterized in that: include: A tactile media bit stream acquisition module, used to acquire a tactile media bit stream corresponding to a target tactile media signal; a tactile media bitstream encapsulation module, used for encapsulating the tactile media bitstream into a corresponding tactile media file; The tactile media file includes at least two tracks, and each track includes data of one or more data types in the tactile media bitstream and first metadata of the data, wherein the data types are divided based on a preset data classification standard, and the first metadata is used to indicate the data type of the data.

15. A tactile media file decapsulation device, characterized in that: include: a tactile media file acquisition module, configured to acquire a tactile media file, wherein the tactile media file comprises at least two tracks, and each track comprises data of one or more data types in the tactile media bitstream and first metadata of the data, wherein the data types are obtained based on a preset data classification standard, and the first metadata is used to indicate the data type of the data; The tactile media file decapsulation module is configured to determine one or more target tracks based on the first metadata, and decapsulate the target tracks to obtain corresponding tactile media bitstreams.

16. A packaging device, characterized in that: including memory and processor; The memory stores a computer program; The processor is configured to execute the computer program to implement the method according to any one of claims 1 to 12.

17. A decapsulation device, characterized in that: including memory and processor; The memory stores a computer program; The processor is configured to execute the computer program to implement the method according to claim 13.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

19. A computer program product, characterized in that The method comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 13.