Tactile media file encapsulation method and apparatus, tactile media file decapsulation method and apparatus, electronic device, storage medium, and program product
By sorting the haptic media signals into multiple tracks according to preset data and adding the first metadata to each track, the flexibility problem of encapsulating a single track in the prior art is solved, and more flexible haptic media data is achieved Applications and better immersive experiences.
Patent Information
- Application Number
- PCT/CN2024/091107
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-19
- Filing Date
- 2024-05-06
- Publication Date
- 2024-11-28
AI Technical Summary
The prior art only supports encapsulating haptic media content into a single track, limiting the application flexibility of haptic media content.
Provide a haptic media file encapsulation method, which divides the target haptic media signal into multiple tracks according to preset data classification, each track contains different types of haptic media data and first metadata for indicating the data type, thereby solving the problem. When encapsulating, select the appropriate track according to the metadata for decoding.
Improves application flexibility of haptic media data, allowing the selection of appropriate data types for presentation based on device capabilities and user preferences, enhancing the immersive experience.
Smart Images

Figure CN2024091107_28112024_PF_FP_ABST
Abstract
Description
Tactile media file encapsulation, decapsulation method and device, electronic device, storage medium, and program product
[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 19, 2023, with application number 202310578365.1 and application name “Tactile media file encapsulation, decapsulation method and corresponding device”. Technical Field
[0002] The present application relates to the field of tactile media encoding and decoding technology. Specifically, the present application relates to a tactile media file encapsulation and decapsulation method and device, electronic equipment, storage medium, and program product.
[0003] Background of the Invention
[0004] Immersive media content is often presented through various wearable or interactive devices. Therefore, in addition to traditional visual and auditory presentations, immersive media also features a new form of tactile presentation. Tactile presentation, through a combination of hardware and software, allows users to receive information through their bodies, providing an embedded physical sensation and conveying key information about the system they are using. For example, a phone vibrates to alert its owner of an incoming message. This vibration is a type of tactile presentation. Tactile presentation can enhance auditory and visual presentations, improving the user experience.
[0005] When transmitting tactile media content, similar to audio and video media, the sending end needs to encode and encapsulate the tactile media content, and then decapsulate and decode it at the receiving end to obtain the tactile media content. However, existing technologies only support encapsulating tactile media content into a single track, which limits the flexibility of tactile media content applications.
[0006] Summary of the Invention
[0007] The purpose of this application is to solve at least one of the above technical deficiencies. The technical solutions provided by the embodiments of this application are as follows:
[0008] In one aspect, an embodiment of the present application provides a tactile media file packaging method, comprising:
[0009] Obtaining a tactile media bitstream corresponding to the target tactile media signal; and
[0010] encapsulating the haptic media bitstream into a haptic media file;
[0011] The tactile media file includes at least two tracks, and each track includes tactile media data of at least one data type in the tactile media bitstream and first metadata of the tactile media data, wherein the data type is obtained based on a preset data classification, and the first metadata is used to indicate the data type of the tactile media data.
[0012] On the other hand, an embodiment of the present application further provides a tactile media file decapsulation method, comprising:
[0013] Obtaining a tactile media file, wherein the tactile media file includes at least two tracks, and each track includes data of at least one data type in a tactile media bitstream and first metadata of the tactile media data, wherein the data type is obtained based on a preset data classification, and the first metadata is used to indicate the data type of the tactile media data; and
[0014] At least one target track is determined based on the first metadata, and the target track is decapsulated to obtain a tactile media bitstream corresponding to each target track.
[0015] On the other hand, an embodiment of the present application further provides a tactile media file packaging device, comprising:
[0016] a tactile media bitstream acquisition module, configured to acquire a tactile media bitstream corresponding to a target tactile media signal; and
[0017] a tactile media bitstream encapsulation module, configured to encapsulate the tactile media bitstream into a tactile media file;
[0018] The tactile media file includes at least two tracks, and each track includes tactile media data of at least one data type in the tactile media bitstream and first metadata of the tactile media data, wherein the data type is obtained based on a preset data classification, and the first metadata is used to indicate the data type of the tactile media data.
[0019] On the other hand, an embodiment of the present application further provides a tactile media file decapsulation device, comprising:
[0020] a tactile media file acquisition module, configured to acquire a tactile media file, wherein the tactile media file includes at least two tracks, and each track includes data of at least one data type in a tactile media bitstream and first metadata of the tactile media data, wherein the data type is obtained based on a preset data classification, and the first metadata is used to indicate the data type of the tactile media data; and
[0021] The tactile media file decapsulation module is configured to determine at least one target track based on the first metadata, and decapsulate the target track to obtain a tactile media bitstream corresponding to each target track.
[0022] On the other hand, an embodiment of the present application further provides an electronic device, including a memory and a processor;
[0023] The memory stores a computer program;
[0024] The processor is configured to execute the computer program to implement the above-mentioned tactile media file encapsulation method or tactile media file decapsulation method.
[0025] On the other hand, an embodiment of the present application further provides a packaging device, including a memory and a processor;
[0026] The memory stores a computer program;
[0027] The processor is configured to execute a computer program to implement the tactile media file packaging method.
[0028] On the other hand, an embodiment of the present application provides a decapsulation device, including a memory and a processor;
[0029] The memory stores a computer program;
[0030] The processor is configured to execute a computer program to implement the above-mentioned tactile media file decapsulation method.
[0031] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned tactile media file encapsulation method or tactile media file decapsulation method is implemented.
[0032] In an eighth aspect, embodiments of the present application provide a computer program product or computer program, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, so that the computer device implements the aforementioned tactile media file encapsulation method or tactile media file decapsulation method when executing the computer instructions.
[0033] BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application.
[0035] FIG1 is a system structure diagram of a method for encapsulating and decapsulating tactile media files according to an embodiment of the present application;
[0036] FIG2 is a flow chart of a tactile media file packaging method provided in an embodiment of the present application;
[0037] FIG3 is a schematic diagram of a tactile media file obtained in an example of an embodiment of the present application;
[0038] FIG4 is a schematic diagram of another tactile media file obtained in an example of an embodiment of the present application;
[0039] FIG5 is a schematic diagram of a silent sample group in an example according to an embodiment of the present application;
[0040] FIG6 is a schematic diagram of a second metadata encapsulation method in an example of an embodiment of the present application;
[0041] FIG7 is a flow chart of a method for decapsulating a tactile media file according to an embodiment of the present application;
[0042] FIG8 is a structural block diagram of a tactile media file packaging device provided in an embodiment of the present application;
[0043] FIG9 is a structural block diagram of a tactile media file decapsulation device provided in an embodiment of the present application;
[0044] FIG10 is a schematic structural diagram of an electronic device provided in an embodiment of the present application.
[0045] Implementation Method
[0046] The following describes the embodiments of the present application in conjunction with the accompanying drawings. It should be understood that the embodiments described below in conjunction with the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.
[0047] Those skilled in the art will understand that, unless otherwise stated, the singular forms "a", "an", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements, and / or components, but do not exclude implementation as other features, information, data, steps, operations, elements, components, and / or combinations thereof supported by the present technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the element can be directly connected or coupled to the other element, or it can refer to the element and the other element establishing a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used here can include wireless connection or wireless coupling. The term "and / or" used here indicates at least one of the items defined by the term, for example, "A and / or B" can be implemented as "A", or as "B", or as "A and B".
[0048] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.
[0049] First, several terms involved in this application are introduced and explained:
[0050] Haptics: Haptics is the sensory experience obtained by the human body through contact, such as vibration, pressure, temperature, etc.
[0051] Haptics media signal: A haptics media signal is used to represent a tactile experience of a specific modality and is rendered and presented on a specific device.
[0052] JSON (Java Script Object Notation) is a lightweight data exchange format. It uses a text format that is completely independent of programming languages to store and represent data. Its simplicity and clear hierarchical structure make JSON an ideal data exchange language. It is easy for humans to read and write, as well as easy for machines to parse and generate, and it significantly improves network transmission efficiency.
[0053] Bitstream, or code stream: Bitstream refers to the binary data stream after compression encoding.
[0054] Track: A track is a collection of media data during the media file encapsulation process, consisting of multiple time-sequential samples. A media file can consist of at least one track. For example, a media file can contain a video media track, an audio media track, and a subtitle media track. Metadata information can also be included in the file as a media type in the form of a metadata media track.
[0055] Sample: A sample is a unit of media file encapsulation. A track consists of many samples, each of which corresponds to a specific timestamp. For example, a video media track can consist of many samples, with a sample typically representing a video frame. In this embodiment of the present application, a sample in a track can be at least one haptic signal.
[0056] Sample number: Sample Number. The first sample in the track has a number of 1.
[0057] Sample Entry: This is used to indicate metadata related to all samples in a track. For example, the sample entry of a video track contains metadata related to decoder initialization.
[0058] Sample Group: Sample Group is used to group some samples in a track according to specific rules.
[0059] A media segment is a playable segment that conforms to a specific media format. Playback may require the cooperation of zero or more preceding segments and an initialization segment.
[0060] In specific applications, the presentation of tactile information can be divided into the following categories:
[0061] Vibrotactile, which directly presents touch in the form of vibrations;
[0062] Kinematic tactile sensation, simulating the movement of an object such as weight or pressure;
[0063] Electrotactile, which simulates feedback information of specific textures through electrical stimulation.
[0064] With the popularization of wearable devices and interactive devices, the tactile perception that users can experience when consuming media content will no longer be limited to the above three types of vibration tactile sensations, but will include comprehensive tactile sensations such as vibration, pressure, speed, acceleration, temperature, humidity, and smell, providing a full range of physical sensations that are closer to the real world.
[0065] Figure 1 is a system structure diagram of a tactile media file encapsulation method and a tactile media decapsulation method implemented in an embodiment of the present application. As shown in Figure 1, the system can be an immersive system, which can include a server 101 and multiple terminal devices 102, wherein the terminal device 102 can be a mobile phone, a tablet (Pad) or other gaming device, and these terminal devices can present corresponding tactile media signals through corresponding sensors.
[0066] The service provider can collect or generate relevant tactile media signals through a server, then encode and encapsulate the collected tactile media signals through the server to obtain corresponding tactile media files, and then send the obtained tactile media files to the playback terminal. The terminal device decapsulates and decodes the received tactile media files to obtain corresponding tactile media information, and the user can feel the tactile media information through the terminal device, achieving an immersive experience. It is understood that the tactile media file encapsulation method provided in the embodiment of the present application is used when the encapsulation operation is performed by the server, and the tactile media file decapsulation method provided in the embodiment of the present application is used when the decapsulation operation is performed by the player.
[0067] FIG2 is a flow chart of a tactile media file packaging method provided in an embodiment of the present application. The execution subject of the method may be the server in FIG1 . As shown in FIG2 , the method may include:
[0068] Step S201 : Acquire a haptic media bitstream corresponding to a target haptic media signal.
[0069] The target haptic media signal is the haptic media signal to be encapsulated. The presentation of haptic media signals can be vibrotactile, kinematic, or electrotactile. Specifically, vibrotactile simulates vibrations of a specific frequency and intensity through the vibration of the terminal device's motor. For example, in a shooting game, vibrations are used to simulate the specific effects of using a power-up. Kinematic haptics simulate the weight or pressure of an object through a kinematic haptic system. For example, in a driving video game, when moving at high speeds or operating a heavy vehicle, the steering wheel may resist turning. This type of feedback directly affects the user's muscles. In the driving game example, the user must apply more force to obtain the desired response from the steering wheel. Electrotactile presentation uses electrical pulses to provide tactile stimulation to the nerve endings in the user's skin. Electrotactile presentation can create a highly realistic experience for users wearing suits or gloves equipped with electrotactile technology. Many sensations can be simulated using electrical pulses, such as temperature changes, pressure changes, and the sensation of moisture.
[0070] Specifically, the server obtains the target tactile media signal by collecting or producing it according to the expected tactile media effect. For example, if it is necessary to produce a specific effect of using a prop in a shooting game by vibrating, the server collects or produces the corresponding vibration tactile signal. Then, the server converts the exchange format of the target tactile media signal into a specified exchange format, namely, the tactile media exchange format. The tactile media exchange format can be in JSON format. Then, the server compresses (i.e., encodes) the target tactile media signal in the exchange format to obtain the corresponding tactile media bit stream (i.e., tactile media code stream).
[0071] Step S202 : Encapsulate the haptic media bitstream into a haptic media file.
[0072] The tactile media file includes at least two tracks, each of which contains tactile media data of at least one data type in the tactile media bitstream and first metadata for the tactile media data. The data type is determined based on a preset data classification, and the first metadata is used to indicate the data type of the tactile media data. It should be noted that multiple tracks from the same tactile media bitstream can also be referred to as component tracks.
[0073] The preset data classification may include one of a perception class, a channel class, a hierarchy class, and a priority class. In other words, data may be classified by perception type, channel type, hierarchy type, and priority type. For example, if the preset data classification is a perception class, then the data types obtained by dividing the data into this class may include vibration perception, temperature perception, etc.
[0074] Specifically, after obtaining the tactile media bit stream through encoding, the server can also organize the tactile media bit stream to obtain a tactile media transmission stream that is easy to transmit. The server then encapsulates the tactile media transmission stream into multiple tracks and adds corresponding first metadata in each track.
[0075] Specifically, after obtaining the tactile media bit stream (or the tactile media transmission stream corresponding to the tactile media bit stream), the various data types corresponding to the data in the tactile media bit stream are determined according to the preset data classification. It should be noted that each time the encapsulation is performed, only one data classification (i.e., the preset classification) is selected to determine the various data types corresponding to the data in the tactile media bit stream.
[0076] Then, the data in the tactile media bit stream is encapsulated according to the data type to obtain a plurality of tracks, each track containing data of one or more data types from the tactile media bit stream.
[0077] Then, corresponding first metadata is added to each track, where the first metadata is at least used to indicate the data type of the tactile media data in the track.
[0078] It is understandable that, during decapsulation, the data type in each track may be determined based on the first metadata in the track, and further, whether the data in the track is required data may be determined.
[0079] For example, after obtaining the tactile media bitstream, a preset data classification can be selected from the perception class, channel class, level class, and priority class based on actual needs. For example, "perception class" can be selected as the preset data classification. Based on this preset data classification, the data types of the data in the tactile media bitstream are determined to include: perception type 1, perception type 2, and perception type 3. The data is then packaged according to the data type.
[0080] As shown in Figure 3, the data corresponding to perception type 1, perception type 2 and perception type 3 can be encapsulated in track 1, track 2 and track 3 respectively, and the corresponding first metadata is added in track 1 to indicate that the data type of the tactile media data contained in the track is perception type 1, the corresponding first metadata is added in track 2 to indicate that the data type of the tactile media data contained in the track is perception type 2, and the corresponding first metadata is added in track 3 to indicate that the data type of the tactile media data contained in the track is perception type 3.
[0081] Alternatively, as shown in FIG4 , two data types may be encapsulated into one track, and another data type may be encapsulated into another track. For example, data corresponding to perceptual type 1 and perceptual type 2 may be encapsulated into track 1, and data corresponding to perceptual type 3 may be encapsulated into track 2. Corresponding first metadata may be added to track 1 to indicate that the data types of the tactile media data contained in the track are perceptual type 1 and perceptual type 2, and corresponding first metadata may be added to track 2 to indicate that the data type of the tactile media data contained in the track is perceptual type 3.
[0082] The solution provided in this application, after acquiring a tactile media bitstream corresponding to a target tactile media signal, encapsulates the tactile media bitstream into multiple tracks according to data types corresponding to preset data classifications. Each track contains data of one or more data types derived from the tactile media bitstream, as well as first metadata indicating the data type of the tactile media data contained in the track. This solution encapsulates the tactile media bitstream into multiple tracks during the encapsulation phase, allowing the selection of the track to be decoded during the decoding phase based on the data type indicated in each track's first metadata, thereby increasing the flexibility of tactile media data applications.
[0083] In an optional embodiment of the present application, the first metadata includes quantity information of data types of the tactile media data in the track and identification information of the data types.
[0084] Specifically, the first metadata in each track may include data type quantity information and data type identification information to indicate the data type of the tactile media data contained in the track. Specifically, the data type quantity information may indicate the number of data types contained in the track, and the data type identification information may indicate the specific type of data type contained in the tactile media.
[0085] For example, for a certain track, the number of data types in its first metadata indicates that the track contains two data types, and the data type identification information indicates that the data types of the data in the track are perceptual type 1 and perceptual type 2, respectively. Therefore, during the decapsulation phase, the number of data types contained in the track is determined based on the number of data types in the first metadata of the track. Then, whether to decapsulate the track is determined based on the number of data types. Furthermore, the specific types of data types contained in the track are determined based on the data type identification information in the first metadata of the track. Then, whether to decapsulate the track is determined based on the specific types of data types.
[0086] In an optional embodiment of the present application, the preset data classification includes any one of the following: perception class, channel class, level class, and priority class.
[0087] It should be noted that the data types corresponding to the perception class may include vibration perception, temperature perception, etc.
[0088] The channel in the channel class refers to the channel under a specific perception type. For example, vibration perception can have multiple different channels, and different channels can correspond to devices worn on different parts of the body in the terminal device used by the user. For example, a helmet is a channel under vibration perception, and a glove is another channel under vibration perception. Therefore, the data type corresponding to the channel class can have different channel types under a specific perception type.
[0089] The data types corresponding to hierarchical classes can have different hierarchical types. These hierarchical types are divided into high and low levels. Their characteristic is that they need to be decapsulated from the lowest level to the highest level during the decapsulation phase. For example, if two tracks encapsulated by hierarchical types need to be decapsulated during decapsulation, the track with the lower hierarchical type will be decapsulated first, and then the track with the higher hierarchical type will be decapsulated.
[0090] Priority classes can have different priority types, which are divided into high and low priorities. Similar to the hierarchical type, their characteristic is that they need to be decapsulated in order from high priority to low priority during the decapsulation phase.
[0091] It should be noted that the preset data classification is not limited to the above four categories. Other data classifications can also be used according to needs. Its function is to set a unified category and divide the data in the tactile media bit stream into different data types, so that different data types can be packaged into different tracks during the packaging stage.
[0092] In an optional embodiment of the present application, if the preset data is classified as a perceptual type, the first metadata further includes attribute information of the perceptual type of the tactile media data in the track.
[0093] Specifically, if the preset data is classified as perceptual, then when encapsulating the tactile media bitstream, each data item is encapsulated into different tracks according to the perceptual type of the tactile media data. As described above, if the preset data is classified as perceptual, the first metadata of each encapsulated track can include information about the number of perceptual types and information about the perceptual type identification.
[0094] Furthermore, the first metadata may also include attribute information of the perceptual type. Specifically, the attribute information of the perceptual type may be a numerical value, and the numerical value corresponds to attribute information of a characteristic. For example, the corresponding relationship may be as shown in Table 1:
[0095] Table 1
[0096] It should be noted that an attribute of a perceptual type can correspond to at least one perceptual type, that is, an attribute of a perceptual type can correspond to an identifier of at least one perceptual type. For example, the attributes of the perceptual types of temperature type 1, temperature type 2, and temperature type 3 are all "temperature".
[0097] In an optional embodiment of the present application, if the preset data is classified as a channel type, the first metadata further includes channel group flag information of the tactile media data in the track, and the channel group flag information is used to indicate whether the tactile media data belongs to the same channel group.
[0098] Specifically, if the preset data is classified as a channel type, when encapsulating the tactile media bitstream, each data item is encapsulated into different tracks according to the channel type of the tactile media data. For a track containing data of multiple channel types, the first metadata of the track may also include channel group flag information, which indicates whether the multiple channel types in the track belong to the same channel group. Channel groups can be pre-divided to provide richer data information for the subsequent decapsulation process, further improving the flexibility of decapsulation.
[0099] Specifically, the channel group flag information can be a specific value. For example, if the value corresponding to the channel group flag information is 1, it means that the multiple channel types in the track belong to the same channel group. If the value corresponding to the channel group flag information is 0, it means that the multiple channel types in the track do not belong to the same channel group.
[0100] In an optional embodiment of the present application, the channel class further indicates a perception type, and the first metadata further includes identification information of the perception type and attribute information of the perception type of the tactile media data in the track.
[0101] Specifically, as can be seen from the above description, a specific perceptual type may include multiple different channel types. Therefore, after the data in the tactile media bitstream is encapsulated into different tracks according to the channel type, the perceptual type information of the channel type included can be added to the first metadata corresponding to each track. Specifically, the identification information of the perceptual type and the attribute information of the perceptual type can be added.
[0102] In an optional embodiment of the present application, the first metadata is stored in a data box corresponding to a track-level sample entry of the track, and the first metadata includes indication information of a preset data classification.
[0103] In the data structure of the track, the first metadata of the track is stored in a data box corresponding to the sample entry.
[0104] Specifically, a track-level sample entry may be defined to indicate metadata information related to all samples in a certain track, such as a sample entry for a video track.
[0105] Then, each track may specify corresponding first metadata in a data box of the track-level sample entry. The data box of the track-level sample entry may specify indication information of a preset data classification (or component type), information about the number of data types (or component number), identification information of the data type, and other data type related information.
[0106] The indication information of the preset data classification may be a specific value, and each specific value corresponds to a specific preset data classification. The corresponding relationship may be shown in Table 2 below:
[0107] Table 2
[0108] Then, in the decapsulation stage, the corresponding preset data category can be determined by combining the indication information of the preset data category in the first metadata in the sample entry of each track with Table 2.
[0109] In an optional embodiment of the present application, the first metadata is stored in a data box corresponding to a data type level sample entry of the track, and the data box corresponding to the data type level sample entry contains indication information of a preset data classification.
[0110] Specifically, a data type-level sample entry can be defined to indicate metadata information related to all samples of a certain data type, such as a perceptual type sample entry, a channel type sample entry, a layer type sample entry, and a priority type sample entry. Each track can then specify corresponding first metadata in the data type-level sample entry corresponding to that track. Compared to the track-level sample entry, the data type-level sample entry itself can indicate the preset data classification used. Therefore, the data box of the data type-level sample entry can specify the quantity information of the data type, the identification information of the data type, and other data type-related information.
[0111] For example, if a track contains data of both perceptual type 1 and perceptual type 2, and if the track uses track-level sample entry, the data box of the track-level sample entry can specify:
[0112] Indicative information of preset data classification (or component type): perception class;
[0113] The number of data types (or components): 2.
[0114] Identification information of the data type: Perception Type 1 and Perception Type 2; and
[0115] Additional data type related information: properties of perceived types.
[0116] If the track uses a data type level sample entry, that is, a perceptual type sample entry, then the data box of the data type level sample entry can specify:
[0117] Number of data types: 2;
[0118] Identification information of the data type: Perception Type 1 and Perception Type 2; and
[0119] Additional data type related information: properties of perceived types.
[0120] In an optional embodiment of the present application, the track includes at least one target sample group, and each target sample group includes one target sample or multiple consecutive target samples in the track.
[0121] Specifically, the data in the track is packaged in units of samples, and the track may include at least one target sample group. Then, the corresponding track can be marked with target sample group information, and the target sample group information includes a sequence number of a target sample.
[0122] In an optional embodiment of the present application, the target sample does not include tactile media data.
[0123] Among them, the target sample does not contain tactile media data and can be called a silent sample. The corresponding target sample group can also be called a silent sample group. The samples contained therein do not contain tactile media data. Therefore, the silent sample group can be directly skipped in the decoding stage without being processed.
[0124] In the prior art, although silent units are defined and used in the tactile media bitstream to indicate that there is no tactile signal presented for a certain period of time, when the tactile media bitstream is encapsulated into multiple tracks, different tracks may have different silent times. As shown by the starting time indicated by the dotted line in Figure 5, when multiple tracks are encapsulated, the starting time of the silent sample group in track 1 and track 2 is not the same, and the starting time of the silent sample group is also different when track 1 is encapsulated with a single track. Therefore, the embodiment of the present application defines a silent sample group to identify samples in the track that do not contain tactile signal data. Corresponding to the sample group type being a silent sample group, the sample is a silent sample, that is, the sample does not contain tactile signal data and can be directly skipped during decoding.
[0125] In an optional embodiment of the present application, the tactile media file further includes a metadata track containing second metadata of the tactile media bitstream. The second metadata is used to indicate encoding information of the tactile media bitstream. The encoding information refers to encoding parameters and configuration information used when compressing the tactile media signal.
[0126] In an abridged embodiment of the present application, the track further includes second metadata of the tactile media bitstream, where the second metadata is used to indicate encoding information of the tactile media bitstream.
[0127] Specifically, as shown in FIG6 , the second metadata may be separately encapsulated into a metadata track, and then the metadata track needs to be decapsulated and decoded first in the decoding stage.
[0128] In other embodiments, each track may also include a copy of the second metadata, so each track includes the first metadata and the second metadata.
[0129] FIG7 is a flow chart of a method for decapsulating a tactile media file according to an embodiment of the present application. The method may be performed by the terminal device in FIG1 . As shown in FIG7 , the method may include:
[0130] Step S701: Acquire a tactile media file.
[0131] The tactile media file includes at least two tracks, and each track includes tactile media data of at least one data type in the tactile media bitstream and first metadata of the tactile media data. The data type is obtained based on a preset data classification, and the first metadata is used to indicate the data type of the tactile media data.
[0132] The preset data classification may include one of a perception class, a channel class, a hierarchy class, and a priority class. In other words, data may be classified by perception type, channel type, hierarchy type, and priority type. For example, if the preset data classification is a perception class, then the data types obtained by dividing the data into this class may include vibration perception, temperature perception, etc.
[0133] Specifically, the tactile media file is obtained from the server, and the server obtains the tactile media file in the following manner:
[0134] After obtaining the tactile media bitstream through encoding, the server may further organize the tactile media bitstream to obtain a tactile media transmission stream that is convenient for transmission. The server then encapsulates the tactile media transmission stream to obtain multiple tracks, and adds corresponding first metadata to each track.
[0135] Specifically, after obtaining the tactile media bit stream (or the tactile media transmission stream corresponding to the tactile media bit stream), the various data types corresponding to the data in the tactile media bit stream are determined according to the preset data classification. It should be noted that each time the encapsulation is performed, only one data classification (i.e., the preset classification) is selected to determine the various data types corresponding to the data in the tactile media bit stream.
[0136] Then, the data in the tactile media bit stream is encapsulated according to the data type to obtain a plurality of tracks, each track containing data of one or more data types from the tactile media bit stream.
[0137] Then, corresponding first metadata is added to each track, where the first metadata is at least used to indicate the data type of the tactile media data in the track.
[0138] It is understandable that, during decapsulation, the data type in each track may be determined based on the first metadata in the track, and further, whether the data in the track is required data may be determined.
[0139] The processing on the server side may refer to the description in the embodiment shown in FIG2 above, and will not be repeated here.
[0140] Step S702 : determining at least one target track based on the first metadata, and decapsulating the target track to obtain a haptic media bitstream corresponding to each target track.
[0141] Specifically, the terminal device determines the target track from each track based on the first metadata in each track, combined with the device capabilities of the terminal device and / or the user's usage preferences. Device capabilities refer to the ability of the terminal device to support data types under a preset data classification, and user preferences refer to the user's preference for tactile presentation of a certain data type.
[0142] Then, the target track is decapsulated to obtain a corresponding tactile media bit stream, and the obtained tactile media bit stream is further decoded to obtain a tactile media signal in a target exchange format, which is then rendered and presented.
[0143] The solution provided in this application, after acquiring a tactile media bitstream corresponding to a target tactile media signal, encapsulates the tactile media bitstream into multiple tracks according to data types corresponding to preset data classifications. Each track contains data of one or more data types derived from the tactile media bitstream, as well as first metadata indicating the data type of the tactile media data contained in the track. This solution encapsulates the tactile media bitstream into multiple tracks during the encapsulation phase, allowing the selection of the track to be decoded during the decoding phase based on the data type indicated in each track's first metadata, thereby increasing the flexibility of tactile media data applications.
[0144] The following describes the encapsulation and decapsulation method of a tactile media file in an embodiment of the present application through an example. The method may include:
[0145] (1) The server produces or collects tactile media signals according to the expected tactile media effects, and converts the exchange format of the target tactile media signals into the specified exchange format.
[0146] (2) The server compresses the haptic media signal in the exchange format into a haptic media bit stream.
[0147] (3) The server organizes the tactile media bit stream into a transmission stream that is easy to transmit.
[0148] (4) The server encapsulates the transport stream into a tactile media file. The file contains two tracks, track 1 and track 2, corresponding to the vibration perception type and temperature perception type, respectively. The corresponding tracks should have the following first metadata:
[0149] Track 1: The sample entry type is a track-level sample entry. The information specified in the sample entry data box is as follows:
[0150] {Indication of preset data classification = 0; Number of data types = 1; Identification information of perceptual type = 1; Attribute information of perceptual type = 5}
[0151] At the same time, the sample entry of track 1 includes second metadata.
[0152] At the same time, the silent sample group information is used to identify the silent samples in track 1, assuming they are samples 100 to 200.
[0153] Track 2: The sample entry type is a data type level sample entry. The information specified in the data box of the sample entry is as follows:
[0154] {Indication of preset data classification = 0; Number of data types = 1; Identification information of perceptual type = 2; Attribute information of perceptual type = 6}
[0155] At the same time, the second metadata is repeatedly included in the sample entry of track 2.
[0156] At the same time, the silent sample group information is used to identify the silent samples in track 2, assuming they are samples 100 to 400.
[0157] (5) The server may slice the tactile media file to obtain multiple tactile media file segments, each of which includes a complete set of the first metadata and the second metadata.
[0158] (6) The terminal device obtains the tactile media file or the tactile media file segment.
[0159] (7) The terminal device selects the corresponding track (i.e., the target track) for decapsulation and decoding based on the metadata in the tactile media file or the tactile media file fragment, combined with the device capabilities of the terminal device and the user's usage preferences, and obtains the exchange format of the tactile media signal and renders it.
[0160] For example, user 1's device does not support temperature-related tactile presentation, so it only decodes the tactile media data in track 2 for presentation. Furthermore, based on the information of the silent sample group, samples 100 to 400 are directly skipped and not decoded.
[0161] For example, if user 2's device supports the corresponding perceptual types of tracks 1 and 2, the tactile media data in tracks 1 and 2 will be decoded and presented. Furthermore, based on the information in the silent sample group, samples 100 to 200 in track 1 and samples 100 to 400 in track 2 will be skipped and not decoded.
[0162] In particular, if in streaming mode, user 1 can also choose to transmit only the media resources corresponding to track 2, further saving bandwidth resources.
[0163] FIG8 is a block diagram of a tactile media file encapsulation device according to an embodiment of the present application. As shown in FIG8 , the device 800 may include a tactile media bitstream acquisition module 801 and a tactile media bitstream encapsulation module 802 , wherein:
[0164] The tactile media bitstream acquisition module 801 is used to acquire the tactile media bitstream corresponding to the target tactile media signal;
[0165] The tactile media bitstream encapsulation module 802 is used to encapsulate the tactile media bitstream into a tactile media file;
[0166] The tactile media file includes at least two tracks, and each track includes tactile media data of at least one data type in the tactile media bitstream and first metadata of the tactile media data, wherein the data type is obtained based on a preset data classification, and the first metadata is used to indicate the data type of the tactile media data.
[0167] The solution provided in this application, after acquiring a tactile media bitstream corresponding to a target tactile media signal, encapsulates the tactile media bitstream into multiple tracks according to data types corresponding to preset data classifications. Each track contains data of one or more data types derived from the tactile media bitstream, as well as first metadata indicating the data type of the tactile media data contained in the track. This solution encapsulates the tactile media bitstream into multiple tracks during the encapsulation phase, allowing the selection of the track to be decoded during the decoding phase based on the data type indicated in each track's first metadata, thereby increasing the flexibility of tactile media data applications.
[0168] In an optional embodiment of the present application, the first metadata includes quantity information of data types of the tactile media data in the track and identification information of the data types.
[0169] In an optional embodiment of the present application, the preset data classification includes any one of the following: perception class, channel class, level class, and priority class.
[0170] In an optional embodiment of the present application, if the preset data is classified as a perceptual type, the first metadata further includes attribute information of the perceptual type of the tactile media data in the track.
[0171] In an optional embodiment of the present application, if the preset data is classified as a channel type, the first metadata further includes channel group flag information of the tactile media data in the track, and the channel group flag information is used to indicate whether the data belongs to the same channel group.
[0172] In an optional embodiment of the present application, the channel class further indicates a perception type, and the first metadata further includes identification information of the perception type and attribute information of the perception type of the tactile media data in the track.
[0173] In an optional embodiment of the present application, the first metadata is stored in a data box corresponding to a track-level sample entry of the track, and the first metadata includes indication information of a preset data classification.
[0174] In an optional embodiment of the present application, the first metadata is stored in a data box corresponding to a data type level sample entry of the track, and the data box corresponding to the data type level sample entry contains indication information of a preset data classification.
[0175] In an optional embodiment of the present application, the track includes at least one target sample group, and each target sample group includes one target sample or multiple consecutive target samples in the track.
[0176] In an optional embodiment of the present application, the target sample does not include tactile media data.
[0177] In an optional embodiment of the present application, the tactile media file further includes a metadata track, the metadata track includes second metadata of the tactile media bitstream, and the second metadata is used to indicate encoding information of the tactile media bitstream.
[0178] In an optional embodiment of the present application, the track further includes second metadata of the tactile media bitstream, where the second metadata is used to indicate encoding information of the tactile media bitstream.
[0179] FIG9 is a structural block diagram of a tactile media file decapsulation device according to an embodiment of the present application. As shown in FIG9 , the device 900 may include: a tactile media file acquisition module 901 and a tactile media file decapsulation module 902, wherein:
[0180] The tactile media file acquisition module 901 is configured to acquire a tactile media file, wherein the tactile media file includes at least two tracks, and each track includes data of at least one data type in a tactile media bitstream and first metadata of the tactile media data, wherein the data type is obtained based on a preset data classification, and the first metadata is used to indicate the data type of the tactile media data;
[0181] The tactile media file decapsulation module 902 is configured to determine at least one target track based on the first metadata, and decapsulate the target track to obtain a tactile media bitstream corresponding to each target track.
[0182] The solution provided in this application, after acquiring a tactile media bitstream corresponding to a target tactile media signal, encapsulates the tactile media bitstream into multiple tracks according to data types corresponding to preset data classifications. Each track contains data of one or more data types derived from the tactile media bitstream, as well as first metadata indicating the data type of the tactile media data contained in the track. This solution encapsulates the tactile media bitstream into multiple tracks during the encapsulation phase, allowing the selection of the track to be decoded during the decoding phase based on the data type indicated in each track's first metadata, thereby increasing the flexibility of tactile media data applications.
[0183] The device of the embodiment of the present application can execute the method provided by the embodiment of the present application, and its implementation principle is similar. The actions performed by each module in the device of each embodiment of the present application correspond to the steps in the method of each embodiment of the present application. For the detailed functional description of each module of the device, please refer to the description in the corresponding method shown in the previous text, and will not be repeated here.
[0184] Reference is made below to FIG10 , which shows a schematic diagram of the structure of an electronic device (e.g., a terminal device or server that executes the method shown in FIG2 or FIG7 ) 1000 suitable for implementing an embodiment of the present application. The electronic devices in the embodiments of the present application may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), wearable devices, and the like, as well as fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in FIG10 is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present application.
[0185] The electronic device includes: a memory and a processor, wherein the memory is used to store a program for executing the methods described in each of the above method embodiments; and the processor is configured to execute the program stored in the memory. The processor here may be referred to as the processing device 1001 described below, and the memory may include at least one of the read-only memory (ROM) 1002, the random access memory (RAM) 1003, and the storage device 1008 described below, as shown below:
[0186] As shown in Figure 10, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 into a random access memory (RAM) 1003. Various programs and data required for the operation of the electronic device 1000 are also stored in the RAM 1003. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.
[0187] Typically, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device 1000 to communicate with other devices wirelessly or by wire to exchange data. Although FIG. 10 illustrates an electronic device having various devices, it should be understood that not all of the devices shown are required to be implemented or present. More or fewer devices may alternatively be implemented or present.
[0188] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1009, or installed from the storage device 1008, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present application are performed.
[0189] It should be noted that the computer-readable storage medium mentioned above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with at least one wire, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.
[0190] In some embodiments, the terminal devices and servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0191] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
[0192] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:
[0193] Obtain a tactile media bitstream corresponding to a target tactile media signal; and encapsulate the tactile media bitstream into a tactile media file; wherein the tactile media file comprises at least two tracks, and each track comprises tactile media data of at least one data type in the tactile media bitstream and first metadata of the tactile media data, wherein the data type is obtained based on a preset data classification, and the first metadata is used to indicate the data type of the tactile media data.
[0194] Alternatively, a tactile media file is obtained, wherein the tactile media file includes at least two tracks, and each track includes data of at least one data type in a tactile media bitstream and first metadata of the tactile media data, the data type is obtained based on a preset data classification, and the first metadata is used to indicate the data type of the tactile media data; and, based on the first metadata, at least one target track is determined, and the target track is decapsulated to obtain a tactile media bitstream corresponding to each target track.
[0195] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0196] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code includes at least one executable instruction for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0197] The modules or units described in the embodiments of this application may be implemented in software or hardware. The name of a module or unit does not, in some cases, limit the unit itself. For example, the first constraint acquisition module may also be described as a "module for acquiring a first constraint."
[0198] The functions described above herein may be performed, at least in part, by at least one hardware logic component. For example, and without limitation, exemplary types of hardware logic components that may be used include: a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), a system on a chip (SOC), a complex programmable logic device (CPLD), and the like.
[0199] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on at least one line, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0200] The present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, so that when the computer device executes the computer instructions, the following conditions are achieved:
[0201] Obtain a tactile media bitstream corresponding to a target tactile media signal; and encapsulate the tactile media bitstream into a tactile media file; wherein the tactile media file comprises at least two tracks, and each track comprises tactile media data of at least one data type in the tactile media bitstream and first metadata of the tactile media data, wherein the data type is obtained based on a preset data classification, and the first metadata is used to indicate the data type of the tactile media data.
[0202] Alternatively, a tactile media file is obtained, wherein the tactile media file includes at least two tracks, and each track includes data of at least one data type in a tactile media bitstream and first metadata of the tactile media data, the data type is obtained based on a preset data classification, and the first metadata is used to indicate the data type of the tactile media data; and, based on the first metadata, at least one target track is determined, and the target track is decapsulated to obtain a tactile media bitstream corresponding to each target track.
[0203] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
[0204] The above descriptions are only partial embodiments of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and modifications without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.
Claims
1. A tactile media file packaging method, executed by an electronic device, comprising: Acquire a tactile media bit stream corresponding to a target tactile media signal; and, encapsulating the haptic media bitstream into a haptic media file; The tactile media file includes at least two tracks, and each track includes tactile media data of at least one data type in the tactile media bitstream and first metadata of the tactile media data, wherein the data type is obtained based on a preset data classification, and the first metadata is used to indicate the data type of the tactile media data.
2. The method according to claim 1, wherein: The first metadata includes quantity information of data types of the tactile media data in the track and identification information of the data types.
3. The method according to claim 2, wherein: The preset data classification includes any one of the following: perception class, channel class, level class, and priority class.
4. The method according to claim 3, wherein: If the preset data is classified as the perception type, the first metadata further includes attribute information of the perception type of the tactile media data in the track.
5. The method according to claim 3, wherein: If the preset data is classified as the channel type, the first metadata further includes channel group flag information of the tactile media data in the track, and the channel group flag information is used to indicate whether the tactile media data belongs to the same channel group.
6. The method according to claim 5, wherein: The channel class further indicates a perception type, and the first metadata further includes identification information of the perception type and attribute information of the perception type of the tactile media data in the track.
7. The method according to any one of claims 1 to 6, wherein: The first metadata is stored in a data box corresponding to a track-level sample entry of the track, and the first metadata includes indication information of the preset data classification.
8. The method according to any one of claims 1 to 6, wherein: The first metadata is stored in a data box corresponding to a data type level sample entry of the track, and the data box corresponding to the data type level sample entry contains indication information of the preset data classification.
9. The method according to claim 1, wherein: The track includes at least one target sample group, and each target sample group includes one target sample or a plurality of continuous target samples in the track.
10. The method according to claim 9, wherein: The target sample does not include tactile media data.
11. The method according to claim 1, wherein: The tactile media file further includes a metadata track, wherein the metadata track includes second metadata of the tactile media bitstream, and the second metadata is used to indicate encoding information of the tactile media bitstream.
12. The method according to claim 1, wherein: The track also includes second metadata of the tactile media bitstream, where the second metadata is used to indicate encoding information of the tactile media bitstream.
13. A tactile media file decapsulation method, executed by an electronic device, comprising: Acquire a tactile media file, wherein the tactile media file comprises at least two tracks, and each track comprises data of at least one data type in a tactile media bitstream and first metadata of the tactile media data, wherein the data type is obtained based on a preset data classification, and the first metadata is used to indicate the data type of the tactile media data; and, At least one target track is determined based on the first metadata, and the target track is decapsulated to obtain a tactile media bitstream corresponding to each target track.
14. A tactile media file packaging device, comprising: A tactile media bit stream acquisition module, used to acquire a tactile media bit stream corresponding to a target tactile media signal; and, A tactile media bitstream encapsulation module, used for encapsulating the tactile media bitstream into a tactile media file; The tactile media file includes at least two tracks, and each track includes tactile media data of at least one data type in the tactile media bitstream and first metadata of the tactile media data, wherein the data type is obtained based on a preset data classification, and the first metadata is used to indicate the data type of the tactile media data.
15. A tactile media file decapsulation device, comprising: a tactile media file acquisition module, used for acquiring a tactile media file, wherein the tactile media file comprises at least two tracks, and each track comprises data of at least one data type in a tactile media bitstream and first metadata of the tactile media data, wherein the data type is obtained based on a preset data classification, and the first metadata is used for indicating the data type of the tactile media data; and, The tactile media file decapsulation module is configured to determine at least one target track based on the first metadata, and decapsulate the target track to obtain a tactile media bitstream corresponding to each target track.
16. An electronic device comprising a memory and a processor; The memory stores a computer program; The processor is configured to execute the computer program to implement the method according to any one of claims 1 to 13.
17. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 1 to 13 when executed by a processor.
18. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.
Citation Information
Patent Citations
Method and device for processing track data in multimedia file, medium and equipment
CN112804256A
Screen recording method and device, equipment and storage medium
CN113973226A
Immersion media processing method and device, equipment and storage medium
CN114697631A
Data processing method and device of point cloud media, equipment and storage medium
CN115061984A
Media file encapsulation and de-encapsulation method and device, equipment and storage medium
CN115883871A
